Clang sanitizer binaries segfault on CI #1506
Comments
|
Interestingly the MuSig2 PR fails with a segfault in the tests and not in I tried reproducing this locally using our docker file but failed: the tests pass in my local docker image. I started the container with and then executed inside the container. |
|
@jonasnick Can you try setting the additional sanitizer env variables that we set in the CI config? secp256k1/.github/workflows/ci.yml Lines 444 to 446 in 427e86b Those should probably be added to our output for reproduction... I'm just in the process of getting a diff of the Debian package versions. |
I double and triple checked that I used the right input, but believe it or not, the diff is empty... For the record, this is the package list:
I'm starting to believe that this is related. It mentions some |
real-or-random
added a commit
to real-or-random/secp256k1
that referenced
this issue
Mar 19, 2024
real-or-random
added a commit
to real-or-random/secp256k1
that referenced
this issue
Mar 19, 2024
real-or-random
added a commit
to real-or-random/secp256k1
that referenced
this issue
Mar 19, 2024
Good point. Still 0 crashes in 10 runs of |
|
I can reproduce this now after looking into the bug reports. Need to set both on the host for the crashes to happen. |
real-or-random
added a commit
that referenced
this issue
Mar 20, 2024
a5e8ab2 ci: Add sanitizer env variables to debug output (Tim Ruffing) 84a93de ci: Add workaround for ASLR bug in sanitizers (Tim Ruffing) Pull request description: Fixes #1506. This also adds the sanitizer env variables to our debug output as suggested in the same issue. ACKs for top commit: sipa: utACK a5e8ab2 jonasnick: ACK a5e8ab2 Tree-SHA512: 5162d14eeec01e088c600ed77e21c5ffd4dec23327b7e81b5ecac59b7c535cac97cd7b7b744c767766036dfc6d9152a9933eb326cf4065d56c46e2ee858da662
|
Note that the upstream issue should now be fixed: actions/runner-images#9491 (see also google/oss-fuzz#11703). |
|
Hm, okay, and this is their fix (it's the same as our workaround): actions/runner-images@7aba0ab I'd tend to keep the workaround in our code base. I think it's good to have it for documentation purposes. We could maybe expand a bit on the comment, and explain that this will be resolved in future clang releases. |
Let's keep this open until llvm/llvm-project@58f7251 is in a LLVM release (check branch info displayed by GitHub or the table at google/sanitizers#1614 (comment)). Then we can change the comment to say that the workaround can be removed once we move to that clang version. |
Should be backported soon: llvm/llvm-project#86201. |
Backport has landed in the 18.x branch (release pending): llvm/llvm-project@c2a5703 |
|
@thurstond Thanks for the explicit notification! |
|
LLVM 18.1.3 is out, which contains the fix: https://github.com/llvm/llvm-project/releases/tag/llvmorg-18.1.3. |
real-or-random
added a commit
to real-or-random/secp256k1
that referenced
this issue
Apr 4, 2024
real-or-random
added a commit
to real-or-random/secp256k1
that referenced
this issue
Apr 4, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
This is not related to a specific PR, the only reason why master is green is that it doesn't have recent pushes.
The first bad build is https://github.com/bitcoin-core/secp256k1/actions/runs/8317860245?pr=1058, which includes a rebuild of the Docker image... The last good build of the Docker image is https://github.com/bitcoin-core/secp256k1/actions/runs/8208332997/job/22451543872?pr=1479. But I'm not sure which difference caused the issue. There was no change on our side, so something must have changed in Debian. But what? Here are some things to look into:
We should probably diff the package lists (including package versions) from the CI outputs of the Docker image builds. This tells us what Debian packages got updated between the last good and the first bad build...
The text was updated successfully, but these errors were encountered: