dist/tools/flatc: cloning of the tool sporadically fails on the CI

[bergzand]

Description

cloning of the code for dist/tools/flatc sometimes fails with a "fatal: destination path already exists and is not an empty directory" error on Murdock. This causes sporadic build failures on unrelated PRs. My gut feeling tells me that #14289 is probably to blame and that some tuning is required for the flatc repo clone.

Steps to reproduce the issue

Hard to reproduce, but it shows up once every so often on the CI. Could be a race condition on the CI.
Shows up here and here

Expected results

Murdock shows green

Actual results

Murdock fails on the flatbuffers test:

-- running on worker mobi6.inet.haw-hamburg.de thread 2, build number 4309.
make: Entering directory '/tmp/dwq.0.9930617251010042/7d022ffac242b917f14549ad9e832af4/tests/pkg_flatbuffers'
rm -rf /tmp/dwq.0.9930617251010042/7d022ffac242b917f14549ad9e832af4/build/pkg-build/flatbuffers
Building application "tests_pkg_flatbuffers" for "cc1352p-launchpad" with MCU "cc26x2_cc13x2".
[INFO] flatc binary not found - building it from source now
make -C /tmp/dwq.0.9930617251010042/7d022ffac242b917f14549ad9e832af4/dist/tools/flatc
make[1]: Entering directory '/tmp/dwq.0.9930617251010042/7d022ffac242b917f14549ad9e832af4/tests/pkg_flatbuffers'
make[1]: warning: jobserver unavailable: using -j1.  Add '+' to parent make rule.

[INFO] cloning flatbuffers
[INFO] cloning flatbuffers
git-cache: cloning from cache. tag=592c204500d79f46bdf5f225c3de82f69e87b50f/v1.11.0
git-cache: cloning from cache. tag=commit9e7e8cbe9f675123dd41b7c62868acad39188cae-581521
fatal: destination path '/tmp/dwq.0.9930617251010042/7d022ffac242b917f14549ad9e832af4/build/pkg/flatbuffers' already exists and is not an empty directory.
/tmp/dwq.0.9930617251010042/7d022ffac242b917f14549ad9e832af4/pkg/pkg.mk:120: recipe for target '/tmp/dwq.0.9930617251010042/7d022ffac242b917f14549ad9e832af4/build/pkg/flatbuffers/.git' failed
make[1]: *** [/tmp/dwq.0.9930617251010042/7d022ffac242b917f14549ad9e832af4/build/pkg/flatbuffers/.git] Error 128
make[1]: Leaving directory '/tmp/dwq.0.9930617251010042/7d022ffac242b917f14549ad9e832af4/dist/tools/flatc'
/tmp/dwq.0.9930617251010042/7d022ffac242b917f14549ad9e832af4/makefiles/tools/targets.inc.mk:44: recipe for target '/tmp/dwq.0.9930617251010042/7d022ffac242b917f14549ad9e832af4/dist/tools/flatc/flatc' failed
make: *** [/tmp/dwq.0.9930617251010042/7d022ffac242b917f14549ad9e832af4/dist/tools/flatc/flatc] Error 2
make: *** Waiting for unfinished jobs....
[INFO] updating flatbuffers /tmp/dwq.0.9930617251010042/7d022ffac242b917f14549ad9e832af4/build/pkg/flatbuffers/.pkg-state.git-downloaded
echo v1.11.0 > /tmp/dwq.0.9930617251010042/7d022ffac242b917f14549ad9e832af4/build/pkg/flatbuffers/.pkg-state.git-downloaded
[INFO] patch flatbuffers
make: Leaving directory '/tmp/dwq.0.9930617251010042/7d022ffac242b917f14549ad9e832af4/tests/pkg_flatbuffers'
-- build directory size: 5.6M

Versions

See Murdock :)

[aabadie]

I can reproduce the problem locally, but it doesn't happen all the time:

$ rm -rf tests/pkg_flatbuffers/bin/
$ make -C dist/tools/flatc/ distclean
$ make -j -C tests/pkg_flatbuffers --no-print-directory

The error message:

fatal: destination path '/work/riot/RIOT/build/pkg/flatbuffers' already exists and is not an empty directory.
make[1]: *** [/work/riot/RIOT/pkg/pkg.mk:123: /work/riot/RIOT/build/pkg/flatbuffers/.git] Error 128
make: [/work/riot/RIOT/tests/pkg_flatbuffers/../../Makefile.include:670: pkg-prepare] Error 2 (ignored)

is displayed all the time though.

My gut feeling tells me that #14289 is probably to blame

This is certainly the origin of the problem. In fact, both flatc and tests/pkg_flatbuffers uses the source of flatbuffers cloned in $(RIOTBASE)/build/pkg and both are trying to clone the same repository. If one builds without -j, everything is fine so there's a concurrency issue when using -j because both builds are started at the same time and try to clone the same repository concurrently.

A workaround is to clone flatbuffers for flatc in $(RIOTTOOLS)/flatc/bin like it was done before.

[bergzand]

Thanks @aabadie!