Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add MAVSDK tests #13772

Merged
merged 43 commits into from
Dec 26, 2019
Merged

Add MAVSDK tests #13772

merged 43 commits into from
Dec 26, 2019

Conversation

LorenzMeier
Copy link
Member

@LorenzMeier LorenzMeier commented Dec 23, 2019

Work in progress to add integration tests using MAVSDK.

FYI @Stifael, @dagar

Currently needs MAVSDK installed system-wide (e.g. /usr/local/).

Todo:

  • Integrate into PX4 build
  • Add overall timeout checks.
  • Add scripting around a test matrix, PX4 build and shutdown
  • Add to CI
  • Make sure tests can run in CI, local, as well as in the real world.

Dependencies:

psutil:

sudo -H pip3 install psutil

Install MAVSDK:
https://github.com/mavlink/MAVSDK

git submodule update
mkdir -p build/default        
cd build/default
cmake ../..
make -j4
make install

To build:

DONT_RUN=1 make px4_sitl gazebo mavsdk_tests

To run:

test/mavsdk_tests/mavsdk_test_runner.py --speed-factor 20 --iterations 5 --fail-early

I'm not done yet with this convenience target - it builds, but it is missing the iris model file generation step.

make tests_integration

Closes mavlink/MAVSDK#885.

@julianoes julianoes mentioned this pull request Dec 23, 2019
@dagar dagar self-requested a review December 23, 2019 15:20
@dagar
Copy link
Member

dagar commented Dec 23, 2019

Lockstep simulation depending on ekf2_timestamps isn't ideal, but understandable.

dagar
dagar previously approved these changes Dec 23, 2019
@LorenzMeier LorenzMeier force-pushed the pr-add-mavsdk-tests branch 2 times, most recently from 6b230df to aba0948 Compare December 24, 2019 10:04
@LorenzMeier
Copy link
Member Author

Test failure:

https://github.com/PX4/Firmware/commit/c3e4781dd385377a5d546fb8f121799461a347e0/checks?check_suite_id=372803159#step:7:397

[10:52:36|Debug] MAVLink: critical: Data link lost (system_impl.cpp:308)
378
[10:52:36|Debug] MAVLink: info: Data link regained (system_impl.cpp:308)
379
[10:52:39|Debug] MAVLink: info: Landing detected (system_impl.cpp:308)
380
[10:52:39|Debug] MAVLink: info: DISARMED by Auto disarm initiated (system_impl.cpp:308)
381
[10:52:39|Debug] MAVLink: critical: Data link lost (system_impl.cpp:308)
382
[10:52:39|Debug] MAVLink: info: Data link regained (system_impl.cpp:308)
383
[10:52:41|Debug] MAVLink: critical: Data link lost (system_impl.cpp:308)
384
[10:52:41|Debug] MAVLink: info: Data link regained (system_impl.cpp:308)
385
[10:52:44|Debug] MAVLink: critical: Data link lost (system_impl.cpp:308)
386
[10:52:44|Debug] MAVLink: info: Data link regained (system_impl.cpp:308)
387
[10:52:52|Debug] MAVLink: critical: Data link lost (system_impl.cpp:308)
388
[10:52:52|Debug] MAVLink: info: Data link regained (system_impl.cpp:308)
389
[10:53:06|Debug] MAVLink: critical: Data link lost (system_impl.cpp:308)
390
[10:53:06|Debug] MAVLink: info: Data link regained (system_impl.cpp:308)
391
[10:53:15|Debug] MAVLink: critical: Data link lost (system_impl.cpp:308)
392
[10:53:15|Debug] MAVLink: info: Data link regained (system_impl.cpp:308)
393
[10:53:19|Debug] MAVLink: critical: Data link lost (system_impl.cpp:308)
394
[10:53:19|Debug] MAVLink: info: Data link regained (system_impl.cpp:308)
395
[10:53:24|Debug] MAVLink: critical: Data link lost (system_impl.cpp:308)
396
[10:53:24|Debug] MAVLink: info: Data link regained (system_impl.cpp:308)
397
Catch will terminate because it needed to throw an exception.

@julianoes
Copy link
Contributor

@LorenzMeier:

etc/init.d-posix/rcS: 194: etc/init.d-posix/rcS: bc: not found

That's why there are the timeouts. It's because the timeout params are not adjusted to lockstep.

@LorenzMeier LorenzMeier force-pushed the pr-add-mavsdk-tests branch 3 times, most recently from 6790f30 to 483f017 Compare December 24, 2019 13:46
@LorenzMeier
Copy link
Member Author

LorenzMeier commented Dec 24, 2019

Thanks! I missed that and I've fixed that here: 1f5df35

Let's see what a grind run with 100 iterations does now. I still believe there is a sensor init race somewhere.

julianoes and others added 15 commits December 25, 2019 09:32
Unfortunately this commit contains two things:
1. Some cleanup and renaiming.
2. An additional wait until lockstep has been initialized.
   By waiting until HIL_SENSOR messages arrive including timestamps we
   stop the startup script and prevent other modules from running until
   time is set up. This should resolve some busy waiting by various
   modules and prevent races on initialization (e.g. the landing state
   being subscribed by mavlink before being published by the land
   detector).
In general, if anything goes wrong in the startup script, we
should fail entirely because things might not work as expected.

In particular, this prevents that we have to press Ctrl+C twice if the
simulator start call is hung waiting for the simulator to appear and
start communicating. We now press Ctrl+C once and exit straightaway
whereas before we would press it once to get the warning:
"Startup script returned with return value: 2",
and then finally exit on the second press.
This enables unfamiliar users to run the tests quickly without having to memorize all commandsline options.
This should ensure that all processes do still run at full load.
This is cleaner and needs testing.
@LorenzMeier LorenzMeier force-pushed the pr-add-mavsdk-tests branch 2 times, most recently from 05ef7ac to a17da30 Compare December 25, 2019 23:08
@LorenzMeier
Copy link
Member Author

@dagar This is good to go now if it passes.

This allows to run the script in grind mode to find CI failures that are triggered through e.g. race conditions.
Bash or zsh from the last 10 years or so do math using the $((a + b)) syntax. This saves us from having bc as dependency.
This is reasonable as these boards are very old and rovers are a new area of activity.
These are rarely used airframes that should not be missed.
These are rarely used airframes that should not be missed.
This ensures that the user is pushed back to the airframe configuration stage.
This still captures all commits, but does not cover the merged state of the PR. All PRs need to be up-to-date before merging/rebasing, which is better practice anyway.
@LorenzMeier LorenzMeier merged commit aaf0330 into master Dec 26, 2019
@LorenzMeier LorenzMeier deleted the pr-add-mavsdk-tests branch December 26, 2019 09:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Landed State is jumping between modes
3 participants