Undefined reference

[crystalin]

When building https://github.com/madara-alliance/madara with SSCACHE it fails:

cargo build --release --bin orchestrator => OK
cargo clean
RUSTC_WRAPPER=/home/crystalin/.cargo/bin/sccache cargo build --release --bin orchestrator =>

Error:

  = note: /usr/bin/ld: /home/crystalin/projects/madara/target/release/build/c-kzg-be3bd7e248783731/out/libckzg.a(cf28328113fcf234-c_kzg_4844.o): in function `pairings_verify':
          c_kzg_4844.c:(.text.pairings_verify+0xbd): undefined reference to `blst_p1_cneg'
          /usr/bin/ld: c_kzg_4844.c:(.text.pairings_verify+0xd0): undefined reference to `blst_p1_to_affine'
          /usr/bin/ld: c_kzg_4844.c:(.text.pairings_verify+0xe3): undefined reference to `blst_p1_to_affine'
          /usr/bin/ld: c_kzg_4844.c:(.text.pairings_verify+0xf6): undefined reference to `blst_p2_to_affine'
          /usr/bin/ld: c_kzg_4844.c:(.text.pairings_verify+0x109): undefined reference to `blst_p2_to_affine'
          /usr/bin/ld: c_kzg_4844.c:(.text.pairings_verify+0x117): undefined reference to `blst_miller_loop'
          /usr/bin/ld: c_kzg_4844.c:(.text.pairings_verify+0x12d): undefined reference to `blst_miller_loop'
          /usr/bin/ld: c_kzg_4844.c:(.text.pairings_verify+0x13b): undefined reference to `blst_fp12_mul'
          /usr/bin/ld: c_kzg_4844.c:(.text.pairings_verify+0x146): undefined reference to `blst_final_exp'
          /usr/bin/ld: c_kzg_4844.c:(.text.pairings_verify+0x14e): undefined reference to `blst_fp12_is_one'
          /usr/bin/ld: /home/crystalin/projects/madara/target/release/build/c-kzg-be3bd7e248783731/out/libckzg.a(cf28328113fcf234-c_kzg_4844.o): in function `fr_batch_inv.constprop.0':
          c_kzg_4844.c:(.text.fr_batch_inv.constprop.0+0x66): undefined reference to `blst_fr_mul'
          /usr/bin/ld: c_kzg_4844.c:(.text.fr_batch_inv.constprop.0+0x7c): undefined reference to `blst_uint64_from_fr'
          /usr/bin/ld: c_kzg_4844.c:(.text.fr_batch_inv.constprop.0+0xa9): undefined reference to `blst_fr_eucl_inverse'
          /usr/bin/ld: c_kzg_4844.c:(.text.fr_batch_inv.constprop.0+0xc2): undefined reference to `blst_fr_mul'
          /usr/bin/ld: c_kzg_4844.c:(.text.fr_batch_inv.constprop.0+0xd0): undefined reference to `blst_fr_mul'
          /usr/bin/ld: /home/crystalin/projects/madara/target/release/build/c-kzg-be3bd7e248783731/out/libckzg.a(cf28328113fcf234-c_kzg_4844.o): in function `g1_lincomb_fast.constprop.0':
          c_kzg_4844.c:(.text.g1_lincomb_fast.constprop.0+0x2d): undefined reference to `blst_p1s_mult_pippenger_scratch_sizeof'
          /usr/bin/ld: c_kzg_4844.c:(.text.g1_lincomb_fast.constprop.0+0xe9): undefined reference to `blst_p1s_to_affine'
          /usr/bin/ld: c_kzg_4844.c:(.text.g1_lincomb_fast.constprop.0+0x108): undefined reference to `blst_scalar_from_fr'
          /usr/bin/ld: c_kzg_4844.c:(.text.g1_lincomb_fast.constprop.0+0x150): undefined reference to `blst_p1s_mult_pippenger'
          /usr/bin/ld: /home/crystalin/projects/madara/target/release/build/c-kzg-be3bd7e248783731/out/libckzg.a(cf28328113fcf234-c_kzg_4844.o): in function `evaluate_polynomial_in_evaluation_form.isra.0':
          c_kzg_4844.c:(.text.evaluate_polynomial_in_evaluation_form.isra.0+0x9e): undefined reference to `blst_fr_sub'
          /usr/bin/ld: c_kzg_4844.c:(.text.evaluate_polynomial_in_evaluation_form.isra.0+0xbe): undefined reference to `blst_uint64_from_fr'
          /usr/bin/ld: c_kzg_4844.c:(.text.evaluate_polynomial_in_evaluation_form.isra.0+0xcb): undefined reference to `blst_uint64_from_fr'
          /usr/bin/ld: c_kzg_4844.c:(.text.evaluate_polynomial_in_evaluation_form.isra.0+0x196): undefined reference to `blst_fr_mul'
          /usr/bin/ld: c_kzg_4844.c:(.text.evaluate_polynomial_in_evaluation_form.isra.0+0x1ae): undefined reference to `blst_fr_mul'
          /usr/bin/ld: c_kzg_4844.c:(.text.evaluate_polynomial_in_evaluation_form.isra.0+0x1bc): undefined reference to `blst_fr_add'
          /usr/bin/ld: c_kzg_4844.c:(.text.evaluate_polynomial_in_evaluation_form.isra.0+0x1eb): undefined reference to `blst_fr_from_uint64'
          /usr/bin/ld: c_kzg_4844.c:(.text.evaluate_polynomial_in_evaluation_form.isra.0+0x1f6): undefined reference to `blst_fr_eucl_inverse'
          /usr/bin/ld: c_kzg_4844.c:(.text.evaluate_polynomial_in_evaluation_form.isra.0+0x204): undefined reference to `blst_fr_mul'
          /usr/bin/ld: c_kzg_4844.c:(.text.evaluate_polynomial_in_evaluation_form.isra.0+0x251): undefined reference to `blst_fr_sqr'
          /usr/bin/ld: c_kzg_4844.c:(.text.evaluate_polynomial_in_evaluation_form.isra.0+0x267): undefined reference to `blst_fr_mul'
          /usr/bin/ld: c_kzg_4844.c:(.text.evaluate_polynomial_in_evaluation_form.isra.0+0x27b): undefined reference to `blst_fr_sub'
          /usr/bin/ld: c_kzg_4844.c:(.text.evaluate_polynomial_in_evaluation_form.isra.0+0x289): undefined reference to `blst_fr_mul'
          /usr/bin/ld: /home/crystalin/projects/madara/target/release/build/c-kzg-be3bd7e248783731/out/libckzg.a(cf28328113fcf234-c_kzg_4844.o): in function `verify_kzg_proof_impl':
          c_kzg_4844.c:(.text.verify_kzg_proof_impl+0x42): undefined reference to `blst_p2_generator'
          /usr/bin/ld: c_kzg_4844.c:(.text.verify_kzg_proof_impl+0x58): undefined reference to `blst_scalar_from_fr'
          /usr/bin/ld: c_kzg_4844.c:(.text.verify_kzg_proof_impl+0x6b): undefined reference to `blst_p2_mult'
          /usr/bin/ld: c_kzg_4844.c:(.text.verify_kzg_proof_impl+0x9e): undefined reference to `blst_p2_cneg'
          /usr/bin/ld: c_kzg_4844.c:(.text.verify_kzg_proof_impl+0xac): undefined reference to `blst_p2_add_or_double'
          /usr/bin/ld: c_kzg_4844.c:(.text.verify_kzg_proof_impl+0xb1): undefined reference to `blst_p1_generator'
          /usr/bin/ld: c_kzg_4844.c:(.text.verify_kzg_proof_impl+0xc7): undefined reference to `blst_scalar_from_fr'
          /usr/bin/ld: c_kzg_4844.c:(.text.verify_kzg_proof_impl+0xdc): undefined reference to `blst_p1_mult'
          /usr/bin/ld: c_kzg_4844.c:(.text.verify_kzg_proof_impl+0x16d): undefined reference to `blst_p1_cneg'
          /usr/bin/ld: c_kzg_4844.c:(.text.verify_kzg_proof_impl+0x17b): undefined reference to `blst_p1_add_or_double'
          /usr/bin/ld: c_kzg_4844.c:(.text.verify_kzg_proof_impl+0x180): undefined reference to `blst_p2_generator'
          /usr/bin/ld: /home/crystalin/projects/madara/target/release/build/c-kzg-be3bd7e248783731/out/libckzg.a(cf28328113fcf234-c_kzg_4844.o): in function `compute_kzg_proof_impl':
          c_kzg_4844.c:(.text.compute_kzg_proof_impl+0x16c): undefined reference to `blst_fr_sub'
          /usr/bin/ld: c_kzg_4844.c:(.text.compute_kzg_proof_impl+0x17a): undefined reference to `blst_fr_sub'
          /usr/bin/ld: c_kzg_4844.c:(.text.compute_kzg_proof_impl+0x1a8): undefined reference to `blst_uint64_from_fr'
          /usr/bin/ld: c_kzg_4844.c:(.text.compute_kzg_proof_impl+0x1b5): undefined reference to `blst_uint64_from_fr'
          /usr/bin/ld: c_kzg_4844.c:(.text.compute_kzg_proof_impl+0x26e): undefined reference to `blst_fr_mul'
          /usr/bin/ld: c_kzg_4844.c:(.text.compute_kzg_proof_impl+0x2dd): undefined reference to `blst_fr_sub'
          /usr/bin/ld: c_kzg_4844.c:(.text.compute_kzg_proof_impl+0x2f1): undefined reference to `blst_fr_mul'
          /usr/bin/ld: c_kzg_4844.c:(.text.compute_kzg_proof_impl+0x35d): undefined reference to `blst_fr_sub'
          /usr/bin/ld: c_kzg_4844.c:(.text.compute_kzg_proof_impl+0x36d): undefined reference to `blst_fr_mul'
          /usr/bin/ld: c_kzg_4844.c:(.text.compute_kzg_proof_impl+0x37b): undefined reference to `blst_fr_mul'
          /usr/bin/ld: c_kzg_4844.c:(.text.compute_kzg_proof_impl+0x389): undefined reference to `blst_fr_add'
          /usr/bin/ld: c_kzg_4844.c:(.text.compute_kzg_proof_impl+0x3cd): undefined reference to `blst_p1_compress'
          /usr/bin/ld: /home/crystalin/projects/madara/target/release/build/c-kzg-be3bd7e248783731/out/libckzg.a(cf28328113fcf234-c_kzg_4844.o): in function `blob_to_kzg_commitment':
          c_kzg_4844.c:(.text.blob_to_kzg_commitment+0x6f): undefined reference to `blst_scalar_from_bendian'
          /usr/bin/ld: c_kzg_4844.c:(.text.blob_to_kzg_commitment+0x77): undefined reference to `blst_scalar_fr_check'
          /usr/bin/ld: c_kzg_4844.c:(.text.blob_to_kzg_commitment+0xbf): undefined reference to `blst_fr_from_scalar'
          /usr/bin/ld: c_kzg_4844.c:(.text.blob_to_kzg_commitment+0xe7): undefined reference to `blst_p1_compress'
          /usr/bin/ld: /home/crystalin/projects/madara/target/release/build/c-kzg-be3bd7e248783731/out/libckzg.a(cf28328113fcf234-c_kzg_4844.o): in function `verify_kzg_proof':
          c_kzg_4844.c:(.text.verify_kzg_proof+0x47): undefined reference to `blst_p1_uncompress'
          /usr/bin/ld: c_kzg_4844.c:(.text.verify_kzg_proof+0x8f): undefined reference to `blst_p1_from_affine'
          /usr/bin/ld: c_kzg_4844.c:(.text.verify_kzg_proof+0x97): undefined reference to `blst_p1_is_inf'
          /usr/bin/ld: c_kzg_4844.c:(.text.verify_kzg_proof+0xa3): undefined reference to `blst_p1_in_g1'
          /usr/bin/ld: c_kzg_4844.c:(.text.verify_kzg_proof+0xb2): undefined reference to `blst_scalar_from_bendian'
          /usr/bin/ld: c_kzg_4844.c:(.text.verify_kzg_proof+0xba): undefined reference to `blst_scalar_fr_check'
          /usr/bin/ld: c_kzg_4844.c:(.text.verify_kzg_proof+0xce): undefined reference to `blst_fr_from_scalar'
          /usr/bin/ld: c_kzg_4844.c:(.text.verify_kzg_proof+0xd9): undefined reference to `blst_scalar_from_bendian'
          /usr/bin/ld: c_kzg_4844.c:(.text.verify_kzg_proof+0xe1): undefined reference to `blst_scalar_fr_check'
          /usr/bin/ld: c_kzg_4844.c:(.text.verify_kzg_proof+0x103): undefined reference to `blst_fr_from_scalar'
          /usr/bin/ld: c_kzg_4844.c:(.text.verify_kzg_proof+0x10e): undefined reference to `blst_p1_uncompress'
          /usr/bin/ld: c_kzg_4844.c:(.text.verify_kzg_proof+0x121): undefined reference to `blst_p1_from_affine'
          /usr/bin/ld: c_kzg_4844.c:(.text.verify_kzg_proof+0x129): undefined reference to `blst_p1_is_inf'
          /usr/bin/ld: c_kzg_4844.c:(.text.verify_kzg_proof+0x154): undefined reference to `blst_p1_in_g1'
          /usr/bin/ld: /home/crystalin/projects/madara/target/release/build/c-kzg-be3bd7e248783731/out/libckzg.a(cf28328113fcf234-c_kzg_4844.o): in function `compute_kzg_proof':
          c_kzg_4844.c:(.text.compute_kzg_proof+0x77): undefined reference to `blst_scalar_from_bendian'
          /usr/bin/ld: c_kzg_4844.c:(.text.compute_kzg_proof+0x7f): undefined reference to `blst_scalar_fr_check'
          /usr/bin/ld: c_kzg_4844.c:(.text.compute_kzg_proof+0xc7): undefined reference to `blst_fr_from_scalar'
          /usr/bin/ld: c_kzg_4844.c:(.text.compute_kzg_proof+0xd7): undefined reference to `blst_scalar_from_bendian'
          /usr/bin/ld: c_kzg_4844.c:(.text.compute_kzg_proof+0xdf): undefined reference to `blst_scalar_fr_check'
          /usr/bin/ld: c_kzg_4844.c:(.text.compute_kzg_proof+0xf8): undefined reference to `blst_fr_from_scalar'
          /usr/bin/ld: c_kzg_4844.c:(.text.compute_kzg_proof+0x127): undefined reference to `blst_scalar_from_fr'
          /usr/bin/ld: c_kzg_4844.c:(.text.compute_kzg_proof+0x134): undefined reference to `blst_bendian_from_scalar'
          /usr/bin/ld: /home/crystalin/projects/madara/target/release/build/c-kzg-be3bd7e248783731/out/libckzg.a(cf28328113fcf234-c_kzg_4844.o): in function `compute_blob_kzg_proof':
          c_kzg_4844.c:(.text.compute_blob_kzg_proof+0x5d): undefined reference to `blst_p1_uncompress'
          /usr/bin/ld: c_kzg_4844.c:(.text.compute_blob_kzg_proof+0xb7): undefined reference to `blst_p1_from_affine'
          /usr/bin/ld: c_kzg_4844.c:(.text.compute_blob_kzg_proof+0xbf): undefined reference to `blst_p1_is_inf'
          /usr/bin/ld: c_kzg_4844.c:(.text.compute_blob_kzg_proof+0xcd): undefined reference to `blst_p1_in_g1'
          /usr/bin/ld: c_kzg_4844.c:(.text.compute_blob_kzg_proof+0xf7): undefined reference to `blst_scalar_from_bendian'
          /usr/bin/ld: c_kzg_4844.c:(.text.compute_blob_kzg_proof+0xff): undefined reference to `blst_scalar_fr_check'
          /usr/bin/ld: c_kzg_4844.c:(.text.compute_blob_kzg_proof+0x11a): undefined reference to `blst_fr_from_scalar'
          /usr/bin/ld: c_kzg_4844.c:(.text.compute_blob_kzg_proof+0x176): undefined reference to `blst_p1_compress'
          /usr/bin/ld: c_kzg_4844.c:(.text.compute_blob_kzg_proof+0x186): undefined reference to `blst_sha256'
          /usr/bin/ld: c_kzg_4844.c:(.text.compute_blob_kzg_proof+0x196): undefined reference to `blst_scalar_from_bendian'
          /usr/bin/ld: c_kzg_4844.c:(.text.compute_blob_kzg_proof+0x1a1): undefined reference to `blst_fr_from_scalar'
          /usr/bin/ld: /home/crystalin/projects/madara/target/release/build/c-kzg-be3bd7e248783731/out/libckzg.a(cf28328113fcf234-c_kzg_4844.o): in function `load_trusted_setup.part.0.constprop.0':
          c_kzg_4844.c:(.text.load_trusted_setup.part.0.constprop.0+0xaf): undefined reference to `blst_p1_from_affine'
          /usr/bin/ld: c_kzg_4844.c:(.text.load_trusted_setup.part.0.constprop.0+0xc3): undefined reference to `blst_p1_uncompress'
          /usr/bin/ld: c_kzg_4844.c:(.text.load_trusted_setup.part.0.constprop.0+0x127): undefined reference to `blst_p2_from_affine'
          /usr/bin/ld: c_kzg_4844.c:(.text.load_trusted_setup.part.0.constprop.0+0x13b): undefined reference to `blst_p2_uncompress'
          /usr/bin/ld: c_kzg_4844.c:(.text.load_trusted_setup.part.0.constprop.0+0x17f): undefined reference to `blst_fr_from_uint64'
          /usr/bin/ld: c_kzg_4844.c:(.text.load_trusted_setup.part.0.constprop.0+0x1f4): undefined reference to `blst_fr_mul'
          /usr/bin/ld: c_kzg_4844.c:(.text.load_trusted_setup.part.0.constprop.0+0x1ff): undefined reference to `blst_uint64_from_fr'
          /usr/bin/ld: c_kzg_4844.c:(.text.load_trusted_setup.part.0.constprop.0+0x234): undefined reference to `blst_uint64_from_fr'
          collect2: error: ld returned 1 exit status

  = note: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
  = note: use the `-l` flag to specify native libraries to link
  = note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#rustc-link-lib)

error: could not compile `orchestrator-core` (bin "orchestrator") due to 1 previous error
warning: build failed, waiting for other jobs to finish...

Context:
This is built on WSL (Ubuntu 24.04)
Cargo installed with cargo install sccache --locked (v0.10.0)

stats (after multiple tries):

/home/crystalin/.cargo/bin/sccache --show-stats
Compile requests                    2347
Compile requests executed           1939
Cache hits                          1750
Cache hits (C/C++)                   643
Cache hits (Rust)                   1107
Cache misses                         181
Cache misses (Rust)                  181
Cache hits rate                    90.63 %
Cache hits rate (C/C++)           100.00 %
Cache hits rate (Rust)             85.95 %
Cache timeouts                         0
Cache read errors                      0
Forced recaches                        0
Cache write errors                     0
Cache errors                           0
Compilations                         181
Compilation failures                   8
Non-cacheable compilations             0
Non-cacheable calls                  387
Non-compilation calls                 21
Unsupported compiler calls             0
Average cache write                0.006 s
Average compiler                   0.384 s
Average cache read hit             0.003 s
Failed distributed compilations        0

Non-cacheable reasons:
crate-type                           247
unknown source language               92
-                                     34
missing input                         13
-E                                     1

Cache location                  Local disk: "/home/crystalin/.cache/sccache"
Use direct/preprocessor mode?   yes
Version (client)                0.10.0
Cache size                             2 GiB
Max cache size                        10 GiB

I was able to reproduce it during CI on a GHA server

[sylvestre]

is it a recent regression?

[crystalin]

I tested with 0.9 and 0.82 and got the same result so I would say it is not a regression