Clean up docker images on pi agent #2736
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
philipktlin
reviewed
Mar 25, 2020
| displayName: sudo git clean | ||
| # clean up root-owned files and docker images ourselves (anything | ||
| # created during `dotnet vstest ...`) | ||
| sudo git clean -ffdx || true |
There was a problem hiding this comment.
QQ: how new e2e test framework uninstall iotedged?
There was a problem hiding this comment.
sudo apt-get purge --yes libiothsm-std iotedge
philipktlin
approved these changes
Mar 25, 2020
kodiakhq bot
pushed a commit
that referenced
this pull request
Mar 31, 2020
With the introduction of #2736, our self-hosted Pi agent has to download docker images from scratch. The current timeout (8 min) isn't enough for the first test to download all images and run the test. This change increases the timeouts (setup and test) to 10 minutes, which should be enough.
kodiakhq bot
pushed a commit
that referenced
this pull request
Apr 2, 2020
Docker images take a _long_ time to download on Raspberry Pi. Since we recently started deleting all images before running the tests (#2736), the first test to run (currently it's `QuickstartCerts`) has to download images from scratch. Also, the `TempFilterFunc` test has to download a rather large image that doesn't share anything in common with our other images. Both these tests tend to timeout before they can download everything, even after the recent timeout increase to 10 min (#2761). This change expands the Linux arm32v7-specific build setup to: 1. git clean as sudo - we need to clean up the source code tree before running the tests, especially the previous run's test logs and certificates (the latter are owned by root, so the `checkout` task in `e2e-setup.yaml` can't delete them). 2. Parse `context.json` to get the list of required images for this test run. 3. `docker pull` the required images _before any old images are removed_, to take advantage of layer caching which speeds up downloads significantly. 4. Remove old containers, images, docker networks, and volumes. With these changes in place, tests on the Pi are completing in 30 seconds-2.5 minutes. As a result this change also reduces the test timeout to 6 minutes so that failed tests don't slow the run down as much as the previous 10 minute timeout. A few tests that were failing before are still failing and need to be investigated separately: - `TransparentGateway` for MQTT + CA/self-signed certs - `TempFilterFunc` and `ValidateMetrics` (see #2753) - `PriorityQueueModuleToModuleMessages` (see #2762)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Our new end-to-end tests don't clean up docker images/containers on the Raspberry Pi agent before running the tests, so over time the agent device storage fills up. This change cleans up images and containers before each test run (we were already cleaning up log files).
Note I also removed the path specifier (
.) from the end of thegit cleancommand, because per the docs the-dflag is redundant if you use a path specifier. One or the other had to go; I picked the path specifier because the-dflag is more explicit.