Squashed 'opendbc/' changes from c0eba09..5eeb33c5

5eeb33c5 Master pedal (#1)
63540dd2 add interceptor
a1aa3b7 Cleanup of Nissan DBC (commaai#218)
c1f7648 reverting changes to Chrysler: speed message seems different from car to car
2de8e9e Chrysler: Speed msg is 5 bytes
f4ac315 Chrysler: add counter to 514
0fb62cf Toyota Blind Spot Monitor (TSS2-only?) (commaai#219)
2ae0327 GitHub actions (commaai#217)
fbbba94 GM: use common gear VALs
dacd8b4 Chrysler commonize gear VALs
d338b9f Add support for 2019 Civic Sedan Diesel. Split GAS_PEDAL_2 out to each car. (commaai#215)
d68f8ff Clean up .gitignore
e82757b Add Lexus NX300H (commaai#214)
745a2bb Removed non high voltage items. Added units.
fafd091 Added High Voltage Management to powertrain file.
15b6b31 Update subaru_global_2017.dbc
a12af19 Add steering error signal in subaru global
d411c17 correct max value
881c065 add electrical gear and fix driver torque
1bbac4b Add reference to CANdevStudio in README file
8f70ea0 Adds 2016 Honda CR-V Executive
5081966 One more fix
fa5dc68 Fix honda dbc files after steer torque addition
e4dfb2f update honda steering signals (commaai#208)
53fc448 added generator test (commaai#207)
cb27d6e Honda Nidec: add new ACC_HUD signals to all other cars other than the CIVIC
6e6779f build on mac AND linux, also gotta not use these python files
79dbfc1 build on mac
4f82d01 gitignore
5cb8345 Honda FCM: diagnostic signals
d309cdc Added linter to opendbc (commaai#203)
d452706 add requirements.txt
ec3b459 deterministic dependency order
a265d35 Azure pipelines ci (commaai#202)
bce9a2e packer depends on libdbc
5d5fdd6 no more python version of libdbc, everything through cython
541705b move CANDefine to parser code
da25c52 add test for can define
0ba7926 unify can packer and parser
25d8800 consistent naming
a5c640a fix linter
be210fe remove obsolete make file
ffd9dca opendbc needs cereal
b559f63 remove more make
d092949 seems to work now
41e8083 don't make
3254d1f think scons works
eb78f6a scons sort of working
0ef1e35 fix gitignore
e155e01 Can migration (commaai#199)
3eded83 Honda: correct steering torque sensor sign to be consistent with standard convention (left+)
32f70e2 Fix outback endianness consistency (commaai#196)
a7da471 Update subaru_outback_2015_eyesight.dbc (commaai#195)

git-subtree-dir: opendbc
git-subtree-split: 5eeb33c5de14b331da901da0222de9c63ff93ab5

Author: JMPZ11

.github/workflows/tests.yml

@@ -0,0 +1,24 @@
+name: Tests
+
+on: [push, pull_request]
+
+jobs:
+  test:
+    runs-on: ubuntu-16.04
+
+    steps:
+      - uses: actions/checkout@v2
+      - run: |
+          set -e
+          docker build -t opendbc .
+        name: "Build"
+      - run: |
+          docker run opendbc bash -c "python -m unittest discover opendbc"
+        name: "Unit tests"
+      - run: |
+          docker run opendbc bash -c "cd opendbc/can/tests/linter_python; PYTHONPATH=/ ./flake8_opendbc.sh"
+          docker run opendbc bash -c "cd opendbc/can/tests/linter_python; PYTHONPATH=/ ./pylint_opendbc.sh"
+        name: "Python linter"
+      - run: |
+          docker run opendbc bash -c "cd opendbc/can/tests/; PYTHONPATH=/ ./test_generator.sh"
+        name: "Generator test"

.gitignore

@@ -1,2 +1,13 @@
 *.pyc
+*.os
+*.tmp
+*.dylib
 .*.swp
+.DS_Store
+
+can/*.so
+can/build/
+can/obj/
+can/packer_pyx.cpp
+can/parser_pyx.cpp
+can/packer_impl.cpp

Dockerfile

@@ -0,0 +1,25 @@
+from ubuntu:16.04
+
+RUN apt-get update && apt-get install -y libzmq3-dev clang wget git autoconf libtool curl make build-essential libssl-dev zlib1g-dev libbz2-dev libreadline-dev libsqlite3-dev llvm libncurses5-dev libncursesw5-dev xz-utils tk-dev libffi-dev liblzma-dev python-openssl
+
+RUN curl -L https://github.com/pyenv/pyenv-installer/raw/master/bin/pyenv-installer | bash
+ENV PATH="/root/.pyenv/bin:/root/.pyenv/shims:${PATH}"
+RUN pyenv install 3.7.3
+RUN pyenv global 3.7.3
+RUN pyenv rehash
+
+COPY requirements.txt /tmp/
+RUN pip install -r /tmp/requirements.txt
+
+ENV PYTHONPATH=/project
+
+# TODO: Add tag to cereal
+RUN git clone https://github.com/commaai/cereal.git /project/cereal
+RUN /project/cereal/install_capnp.sh
+
+WORKDIR /project
+
+COPY SConstruct .
+COPY . /project/opendbc
+
+RUN scons -c && scons -j$(nproc)

README.md

@@ -16,6 +16,9 @@ Wondering what's the DBC file format? [Here](http://www.socialledge.com/sjsu/ind
 
 Use [panda](https://github.com/commaai/panda) to connect your car to a computer.
 
+### How to use reverse engineered DBC
+To create custom CAN simulations or send reverse engineered signals back to the car you can use [CANdevStudio](https://github.com/GENIVI/CANdevStudio) project.
+
 ### DBC file preprocessor
 
 DBC files for different models of the same brand have a lot of overlap. Therefore, we wrote a preprocessor to create DBC files from a brand DBC file and a model specific DBC file. The source DBC files can be found in the generator folder. After changing one of the files run the generator.py script to regenerate the output files. These output files will be placed in the root of the opendbc repository and are suffixed by _generated.
@@ -42,7 +45,7 @@ For example:
     SG_ VEHICLE_SPEED : 7|15@0+ (0.01,0) [0|250] "kph" PCM
     ```
 
-- Signal's size: always use the smallest amount of bits possible. For example, let's say I'm reverse engineering the gas pedal position and I've determined that it's in a 3 bytes message. For 0% pedal position I read a message value of `0x00 0x00 0x00`, while for 100% of pedal position I read `0x64 0x00 0x00`: clearly, the gas pedal position is within the first byte of the message and I might be tempted to define the signal `GAS_POS` as:
+- Signal size: always use the smallest amount of bits possible. For example, let's say I'm reverse engineering the gas pedal position and I've determined that it's in a 3 bytes message. For 0% pedal position I read a message value of `0x00 0x00 0x00`, while for 100% of pedal position I read `0x64 0x00 0x00`: clearly, the gas pedal position is within the first byte of the message and I might be tempted to define the signal `GAS_POS` as:
     ```
     SG_ GAS_POS : 7|8@0+ (1,0) [0|100] "%" PCM
     ```

SConstruct

@@ -0,0 +1,57 @@
+import os
+import subprocess
+
+zmq = 'zmq'
+arch = subprocess.check_output(["uname", "-m"], encoding='utf8').rstrip()
+
+cereal_dir = Dir('.')
+
+cpppath = [
+    '#',
+    '#cereal',
+    "#cereal/messaging",
+    "#opendbc/can",
+    '/usr/lib/include',
+]
+
+AddOption('--test',
+          action='store_true',
+          help='build test files')
+
+AddOption('--asan',
+          action='store_true',
+          help='turn on ASAN')
+
+ccflags_asan = ["-fsanitize=address", "-fno-omit-frame-pointer"] if GetOption('asan') else []
+ldflags_asan = ["-fsanitize=address"] if GetOption('asan') else []
+
+env = Environment(
+  ENV=os.environ,
+  CC='clang',
+  CXX='clang++',
+  CCFLAGS=[
+    "-g",
+    "-fPIC",
+    "-O2",
+    "-Werror=implicit-function-declaration",
+    "-Werror=incompatible-pointer-types",
+    "-Werror=int-conversion",
+    "-Werror=return-type",
+    "-Werror=format-extra-args",
+  ] + ccflags_asan,
+  LDFLAGS=ldflags_asan,
+  LINKFLAGS=ldflags_asan,
+
+  CFLAGS="-std=gnu11",
+  CXXFLAGS="-std=c++14",
+  CPPPATH=cpppath,
+)
+
+Export('env', 'zmq', 'arch')
+
+cereal = [File('#cereal/libcereal.a')]
+messaging = [File('#cereal/libmessaging.a')]
+Export('cereal', 'messaging')
+
+SConscript(['cereal/SConscript'])
+SConscript(['opendbc/can/SConscript'])

acura_ilx_2016_can_generated.dbc

@@ -86,6 +86,13 @@ BO_ 420 VSA_STATUS: 8 VSA
  SG_ COUNTER : 61|2@0+ (1,0) [0|3] "" EON
  SG_ CHECKSUM : 59|4@0+ (1,0) [0|15] "" EON
 
+BO_ 427 STEER_MOTOR_TORQUE: 3 EPS
+ SG_ CONFIG_VALID : 7|1@0+ (1,0) [0|1] "" EON
+ SG_ MOTOR_TORQUE : 1|10@0+ (1,0) [0|256] "" EON
+ SG_ OUTPUT_DISABLED : 22|1@0+ (1,0) [0|1] "" EON
+ SG_ COUNTER : 21|2@0+ (1,0) [0|3] "" EON
+ SG_ CHECKSUM : 19|4@0+ (1,0) [0|15] "" EON
+
 BO_ 432 STANDSTILL: 7 VSA
  SG_ CONTROLLED_STANDSTILL : 0|1@0+ (1,0) [0|1] "" EON
  SG_ WHEELS_MOVING : 12|1@0+ (1,0) [0|1] "" EON
@@ -164,7 +171,7 @@ BO_ 780 ACC_HUD: 8 ADAS
  SG_ BOH : 38|1@0+ (1,0) [0|1] "" BDY
  SG_ ACC_PROBLEM : 37|1@0+ (1,0) [0|1] "" BDY
  SG_ FCM_OFF : 36|1@0+ (1,0) [0|1] "" BDY
- SG_ BOH_2 : 35|1@0+ (1,0) [0|1] "" BDY
+ SG_ FCM_OFF_2 : 35|1@0+ (1,0) [0|1] "" BDY
  SG_ FCM_PROBLEM : 34|1@0+ (1,0) [0|1] "" BDY
  SG_ RADAR_OBSTRUCTED : 33|1@0+ (1,0) [0|1] "" BDY
  SG_ ENABLE_MINI_CAR : 32|1@0+ (1,0) [0|1] "" BDY
@@ -174,10 +181,12 @@ BO_ 780 ACC_HUD: 8 ADAS
  SG_ BOH_4 : 42|1@0+ (1,0) [0|3] "" BDY
  SG_ BOH_5 : 41|1@0+ (1,0) [0|3] "" BDY
  SG_ CRUISE_CONTROL_LABEL : 40|1@0+ (1,0) [0|3] "" BDY
- SG_ HUD_DISTANCE_3 : 52|1@0+ (1,0) [0|1] "" BDY
- SG_ IMPERIAL_UNIT : 54|1@0+ (1,0) [0|1] "" BDY
  SG_ SET_ME_X01_2 : 55|1@0+ (1,0) [0|1] "" BDY
+ SG_ IMPERIAL_UNIT : 54|1@0+ (1,0) [0|1] "" BDY
+ SG_ HUD_DISTANCE_3 : 52|1@0+ (1,0) [0|1] "" BDY
+ SG_ CHIME : 51|3@0+ (1,0) [0|1] "" BDY
  SG_ SET_ME_X01 : 48|1@0+ (1,0) [0|1] "" BDY
+ SG_ ICONS : 63|2@0+ (1,0) [0|1] "" BDY
  SG_ COUNTER : 61|2@0+ (1,0) [0|3] "" BDY
  SG_ CHECKSUM : 59|4@0+ (1,0) [0|3] "" BDY
 
@@ -271,10 +280,11 @@ BO_ 342 STEERING_SENSORS: 6 EPS
  SG_ CHECKSUM : 43|4@0+ (1,0) [0|3] "" EON
 
 BO_ 399 STEER_STATUS: 7 EPS
- SG_ STEER_TORQUE_SENSOR : 7|16@0- (1,0) [-31000|31000] "tbd" EON
- SG_ STEER_TORQUE_MOTOR : 23|16@0- (1,0) [-31000|31000] "tbd" EON
+ SG_ STEER_TORQUE_SENSOR : 7|16@0- (-1,0) [-31000|31000] "tbd" EON
+ SG_ STEER_ANGLE_RATE : 23|16@0- (-0.1,0) [-31000|31000] "deg/s" EON
  SG_ STEER_STATUS : 39|4@0+ (1,0) [0|15] "" EON
  SG_ STEER_CONTROL_ACTIVE : 35|1@0+ (1,0) [0|1] "" EON
+ SG_ STEER_CONFIG_INDEX : 43|4@0+ (1,0) [0|15] "" EON
  SG_ COUNTER : 53|2@0+ (1,0) [0|3] "" EON
  SG_ CHECKSUM : 51|4@0+ (1,0) [0|3] "" EON
 

acura_rdx_2018_can_generated.dbc

@@ -86,6 +86,13 @@ BO_ 420 VSA_STATUS: 8 VSA
  SG_ COUNTER : 61|2@0+ (1,0) [0|3] "" EON
  SG_ CHECKSUM : 59|4@0+ (1,0) [0|15] "" EON
 
+BO_ 427 STEER_MOTOR_TORQUE: 3 EPS
+ SG_ CONFIG_VALID : 7|1@0+ (1,0) [0|1] "" EON
+ SG_ MOTOR_TORQUE : 1|10@0+ (1,0) [0|256] "" EON
+ SG_ OUTPUT_DISABLED : 22|1@0+ (1,0) [0|1] "" EON
+ SG_ COUNTER : 21|2@0+ (1,0) [0|3] "" EON
+ SG_ CHECKSUM : 19|4@0+ (1,0) [0|15] "" EON
+
 BO_ 432 STANDSTILL: 7 VSA
  SG_ CONTROLLED_STANDSTILL : 0|1@0+ (1,0) [0|1] "" EON
  SG_ WHEELS_MOVING : 12|1@0+ (1,0) [0|1] "" EON
@@ -164,7 +171,7 @@ BO_ 780 ACC_HUD: 8 ADAS
  SG_ BOH : 38|1@0+ (1,0) [0|1] "" BDY
  SG_ ACC_PROBLEM : 37|1@0+ (1,0) [0|1] "" BDY
  SG_ FCM_OFF : 36|1@0+ (1,0) [0|1] "" BDY
- SG_ BOH_2 : 35|1@0+ (1,0) [0|1] "" BDY
+ SG_ FCM_OFF_2 : 35|1@0+ (1,0) [0|1] "" BDY
  SG_ FCM_PROBLEM : 34|1@0+ (1,0) [0|1] "" BDY
  SG_ RADAR_OBSTRUCTED : 33|1@0+ (1,0) [0|1] "" BDY
  SG_ ENABLE_MINI_CAR : 32|1@0+ (1,0) [0|1] "" BDY
@@ -174,10 +181,12 @@ BO_ 780 ACC_HUD: 8 ADAS
  SG_ BOH_4 : 42|1@0+ (1,0) [0|3] "" BDY
  SG_ BOH_5 : 41|1@0+ (1,0) [0|3] "" BDY
  SG_ CRUISE_CONTROL_LABEL : 40|1@0+ (1,0) [0|3] "" BDY
- SG_ HUD_DISTANCE_3 : 52|1@0+ (1,0) [0|1] "" BDY
- SG_ IMPERIAL_UNIT : 54|1@0+ (1,0) [0|1] "" BDY
  SG_ SET_ME_X01_2 : 55|1@0+ (1,0) [0|1] "" BDY
+ SG_ IMPERIAL_UNIT : 54|1@0+ (1,0) [0|1] "" BDY
+ SG_ HUD_DISTANCE_3 : 52|1@0+ (1,0) [0|1] "" BDY
+ SG_ CHIME : 51|3@0+ (1,0) [0|1] "" BDY
  SG_ SET_ME_X01 : 48|1@0+ (1,0) [0|1] "" BDY
+ SG_ ICONS : 63|2@0+ (1,0) [0|1] "" BDY
  SG_ COUNTER : 61|2@0+ (1,0) [0|3] "" BDY
  SG_ CHECKSUM : 59|4@0+ (1,0) [0|3] "" BDY
 
@@ -259,8 +268,8 @@ BO_ 392 GEARBOX: 6 XXX
  SG_ GEAR : 36|5@0+ (1,0) [0|31] "" EON
 
 BO_ 399 STEER_STATUS: 6 EPS
- SG_ STEER_TORQUE_SENSOR : 7|12@0- (1,0) [-2047.5|2047.5] "tbd" EON
- SG_ STEER_TORQUE_MOTOR : 23|16@0- (1,0) [-31000|31000] "tbd" EON
+ SG_ STEER_TORQUE_SENSOR : 7|12@0- (-1,0) [-2047.5|2047.5] "tbd" EON
+ SG_ STEER_ANGLE_RATE : 23|16@0- (-0.1,0) [-31000|31000] "deg/s" EON
  SG_ STEER_STATUS : 35|4@0+ (1,0) [0|15] "" EON
  SG_ STEER_CONTROL_ACTIVE : 36|1@0+ (1,0) [0|1] "" EON
  SG_ COUNTER : 45|2@0+ (1,0) [0|3] "" EON

can/SConscript

@@ -0,0 +1,27 @@
+Import('env', 'cereal')
+
+import os
+from opendbc.can.process_dbc import process
+
+dbcs = []
+for x in sorted(os.listdir('../')):
+  if x.endswith(".dbc"):
+    def compile_dbc(target, source, env):
+      process(source[0].path, target[0].path)
+    in_fn = [os.path.join('../', x), 'dbc_template.cc']
+    out_fn = os.path.join('dbc_out', x.replace(".dbc", ".cc"))
+    dbc = env.Command(out_fn, in_fn, compile_dbc)
+    dbcs.append(dbc)
+
+
+libdbc = env.SharedLibrary('libdbc', ["dbc.cc", "parser.cc", "packer.cc", "common.cc"]+dbcs, LIBS=["capnp", "kj"])
+
+# packer
+env.Command(['packer_pyx.so'],
+  [libdbc, 'packer_pyx.pyx', 'packer_pyx_setup.py'],
+  "cd opendbc/can && python3 packer_pyx_setup.py build_ext --inplace")
+
+# parser
+env.Command(['parser_pyx.so'],
+  [libdbc, cereal, 'parser_pyx_setup.py', 'parser_pyx.pyx', 'common.pxd'],
+  "cd opendbc/can && python3 parser_pyx_setup.py build_ext --inplace")

can/__init__.py

@@ -0,0 +1,2 @@
+from opendbc.can.parser_pyx import CANDefine # pylint: disable=no-name-in-module, import-error
+assert CANDefine