Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Crater run for --check-cfg/-Zcheck-cfg #120701

Closed
Urgau opened this issue Feb 6, 2024 · 15 comments
Closed

Crater run for --check-cfg/-Zcheck-cfg #120701

Urgau opened this issue Feb 6, 2024 · 15 comments
Labels
F-check-cfg --check-cfg

Comments

@Urgau
Copy link
Member

Urgau commented Feb 6, 2024

Crater run for --check-cfg (and Cargo -Zcheck-cfg).

Command to be used: @craterbot run name=check-cfg mode=check start=nightly-2024-02-05 end=nightly-2024-02-05+cargoflags=-Zcheck-cfg+rustflags=-Dunexpected_cfgs

@rustbot label +F-check-cfg -needs-triage
@rustbot rustbot added needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. F-check-cfg --check-cfg labels Feb 6, 2024
@Noratrieb
Copy link
Member

Is your goal here getting a feel for how many people will need to add their special cfgs to their list?
Sounds good
@craterbot run name=check-cfg mode=check start=nightly-2024-02-05 end=nightly-2024-02-05+cargoflags=-Zcheck-cfg+rustflags=-Dunexpected_cfgs

@craterbot
Copy link
Collaborator

🚨 Error: failed to parse the command

🆘 If you have any trouble with Crater please ping @rust-lang/infra!
ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

@Noratrieb Noratrieb removed the needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. label Feb 6, 2024
@Urgau
Copy link
Member Author

Urgau commented Feb 6, 2024

Is your goal here getting a feel for how many people will need to add their special cfgs to their list?

@Nilstrieb Yes, and also to check and see if we have not missed a well known cfg.

As for the craterbot command, it should be mode=check-only and not mode=check1, sorry.

Footnotes

  1. https://github.com/rust-lang/crater/blob/master/docs/bot-usage.md#available-experiment-modes ↩

@Noratrieb
Copy link
Member

@craterbot run name=check-cfg mode=check-only start=nightly-2024-02-05 end=nightly-2024-02-05+cargoflags=-Zcheck-cfg+rustflags=-Dunexpected_cfgs

@craterbot
Copy link
Collaborator

👌 Experiment check-cfg created and queued.
🔍 You can check out the queue and this experiment's details.

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

@craterbot craterbot added the S-waiting-on-crater Status: Waiting on a crater run to be completed. label Feb 6, 2024
@craterbot
Copy link
Collaborator

🚧 Experiment check-cfg is now running

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

@craterbot
Copy link
Collaborator

🎉 Experiment check-cfg is completed!
📊 8636 regressed and 1 fixed (414692 total)
📰 Open the full report.

⚠️ If you notice any spurious failure please add them to the blacklist!
ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

@craterbot craterbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-crater Status: Waiting on a crater run to be completed. labels Feb 8, 2024
@Urgau
Copy link
Member Author

Urgau commented Feb 8, 2024

@Nilstrieb can we quickly retry the regressed crates, to remove any OOM, spurious, ...

@craterbot run name=check-cfg-2 mode=check-only start=nightly-2024-02-05 end=nightly-2024-02-05+cargoflags=-Zcheck-cfg+rustflags=-Dunexpected_cfgs crates=https://crater-reports.s3.amazonaws.com/check-cfg/retry-regressed-list.txt p=1

@Noratrieb
Copy link
Member

@craterbot run name=check-cfg-2 mode=check-only start=nightly-2024-02-05 end=nightly-2024-02-05+cargoflags=-Zcheck-cfg+rustflags=-Dunexpected_cfgs crates=https://crater-reports.s3.amazonaws.com/check-cfg/retry-regressed-list.txt p=1

@craterbot
Copy link
Collaborator

👌 Experiment check-cfg-2 created and queued.
🔍 You can check out the queue and this experiment's details.

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

@craterbot craterbot added S-waiting-on-crater Status: Waiting on a crater run to be completed. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Feb 8, 2024
@craterbot
Copy link
Collaborator

🚧 Experiment check-cfg-2 is now running

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

@craterbot
Copy link
Collaborator

🎉 Experiment check-cfg-2 is completed!
📊 8407 regressed and 0 fixed (8636 total)
📰 Open the full report.

⚠️ If you notice any spurious failure please add them to the blacklist!
ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

@craterbot craterbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-crater Status: Waiting on a crater run to be completed. labels Feb 8, 2024
@Urgau
Copy link
Member Author

Urgau commented Feb 10, 2024 •
edited
Loading

Crater report analysis

Context: The crater run enable-by-default cargo -Zcheck-cfg and temporarily put the unexpected_cfgs lint to deny-by-default, and here are the results.

Affected projects: 6708 / 414692 (1.62%)
Total (raw) errors: 200499

50 most seen unexpected cfg errors

Summary: no false-positives1, see the others section for more details.

well, ignoring docsrs (already fixed), rustfmt (deprecated/no longer in use) as well as feature="cargo-clippy" (and feature="clippy")" which are deprecated and should be in the [features] table.

name value comment
docsrs None 24472 errors in 1994 projects
rustfmt None 11084 errors in 818 projects
feature "cargo-clippy" 6910 errors in 29? projects
feature "dim3" 6714 errors in 14 projects
tarpaulin_include None 4938 errors in 212 projects
feature "serde" 4006 errors in 88 projects
doc_cfg None 3807 errors in 130 projects
feature "dim2" 3796 errors in 13 projects
target_os "solana" 3520 errors in 23 projects
tarpaulin None 3052 errors in 97 projects
num None 2842 errors in 2 projects
loom None 2700 errors in 78 projects
enable_x86_avx2 None 2556 errors in 8 projects
fuzzing None 2410 errors in 103 projects
nightly None 2402 errors in 157 projects
feature "tracing" 2035 errors in 10 projects
feature "std" 1948 errors in 138 projects
feature "library" 1910 errors in 72 projects
ossl110 None 1820 errors in 4 projects
feature "inline" 1786 errors in 2 projects
feature "inline_always" 1781 errors in 2 projects
cpp_lib_version "5.14.0" 1746 errors in 3 projects
feature "dox" 1701 errors in 5 projects
cpp_lib_version "5.13.0" 1667 errors in 3 projects
PyPy None 1649 errors in 5 projects
riscv None 1634 errors in 5 projects
cpp_lib_version "5.12.2" 1492 errors in 3 projects
feature "clippy" 1444 errors in 92 projects
docs_rs None 1364 errors in 31 projects
track_caller None 1314 errors in 4 projects
feature "inline-more" 1244 errors in 6 projects
Py_LIMITED_API None 1200 errors in 6 projects
feature "last_seen_epoch" 1008 errors in 252 projects
feature "last_seen_iso_8601" 1008 errors in 252 projects
cpp_lib_version "5.11.3" 988 errors in 3 projects
riscv32 None 983 errors in 2 projects
feature "sqlx" 982 errors in 4 projects
target_arch "spirv" 952 errors in 11 projects
feature "arbitrary_precision" 924 errors in 4 projects
ossl300 None 845 errors in 4 projects
RUSTC_WITH_SPECIALIZATION None 796 errors in 36 projects
has_i128 None 788 errors in 14 projects
feature "unstable" 786 errors in 32 projects
feature "nightly" 752 errors in 92 projects
feature "conversion" 736 errors in 2 projects
no_global_oom_handling None 730 errors in 18 projects
test_utilities None 700 errors in 23 projects
target_pointer_width "128" 690 errors in 43 projects
x11_platform None 690 errors in 7 projects
trybuild None 674 errors in 2 projects

50 most seen unexpected feature cfg

Summary: no false-positives, all of them are missing from the [features] table.

name value comment
feature "cargo-clippy" 8038 errors in 531 projects
feature "dim3" 6714 errors in 14 projects
feature "serde" 4006 errors in 88 projects
feature "dim2" 3796 errors in 13 projects
feature "tracing" 2035 errors in 10 projects
feature "std" 1948 errors in 138 projects
feature "library" 1910 errors in 72 projects
feature "inline" 1786 errors in 2 projects
feature "inline_always" 1781 errors in 2 projects
feature "dox" 1701 errors in 5 projects
feature "clippy" 1444 errors in 92 projects
feature "inline-more" 1244 errors in 6 projects
feature "last_seen_epoch" 1008 errors in 252 projects
feature "last_seen_iso_8601" 1008 errors in 252 projects
feature "sqlx" 982 errors in 4 projects
feature "arbitrary_precision" 924 errors in 4 projects
feature "unstable" 786 errors in 32 projects
feature "nightly" 752 errors in 92 projects
feature "conversion" 736 errors in 2 projects
feature "simd" 654 errors in 9 projects
feature "dist_mode" 584 errors in 4 projects
feature "cons_mode" 460 errors in 3 projects
feature "json" 426 errors in 73 projects
feature "grpc" 392 errors in 4 projects
feature "dashu-int-TODO" 390 errors in 1 projects
feature "1.10.0" 372 errors in 2 projects
feature "1.12.0" 370 errors in 2 projects
feature "nalgebra" 368 errors in 2 projects
feature "test" 352 errors in 21 projects
feature "ffmpeg_5_0" 332 errors in 1 projects
feature "enable-tracing" 328 errors in 1 projects
feature "ffmpeg_4_3" 321 errors in 1 projects
feature "raw_value" 312 errors in 2 projects
feature "debug" 294 errors in 11 projects
feature "cgmath" 240 errors in 2 projects
feature "metrics" 220 errors in 6 projects
feature "mdns" 208 errors in 2 projects
feature "branch_mode" 200 errors in 3 projects
feature "branch_wasm_mode" 192 errors in 3 projects
feature "ffmpeg_4_2" 192 errors in 1 projects
feature "ffmpeg_4_4" 192 errors in 1 projects
feature "dev" 184 errors in 15 projects
feature "wee_alloc" 176 errors in 47 projects
feature "serde-feature" 172 errors in 5 projects
feature "ffmpeg_4_1" 172 errors in 1 projects
feature "critical-section" 170 errors in 43 projects
feature "serialize" 170 errors in 9 projects
feature "failpoints" 164 errors in 3 projects
feature "persistence" 164 errors in 15 projects
feature "jack" 154 errors in 12 projects

50 most seen unexpected target_* cfg

Summary: Many typos (target_os="unix", target_arch="i686", ...), some reference to removed targets (target_os="cloudabi", target_arch="asmjs", ...), as well as some custom cfg (target_os="solana", ...).

name value comment
target_os "solana" 3520 errors in 23 projects
target_arch "spirv" 952 errors in 11 projects
target_pointer_width "128" 690 errors in 43 projects
target_arch "nvptx" 630 errors in 10 projects
target_pointer_width "8" 427 errors in 38 projects
target_os "bitrig" 384 errors in 58 projects
target_os "dos" 368 errors in 7 projects
target_os "unix" 166 errors in 22 projects
target_arch "i686" 140 errors in 14 projects
target_arch "mips64el" 114 errors in 4 projects
target_os "nacl" 86 errors in 14 projects
target_env "wasi" 84 errors in 8 projects
target_arch "asmjs" 82 errors in 14 projects
target_os "darwin" 48 errors in 13 projects
target_feature "1_30" 48 errors in 1 projects
target_feature "session" 44 errors in 1 projects
target_arch "wasm" 44 errors in 5 projects
target_os "wasm" 44 errors in 4 projects
target_os "dragonflybsd" 40 errors in 7 projects
target_arch "riscv" 34 errors in 7 projects
target_arch "armv7" 34 errors in 9 projects
target_arch "mipsel" 34 errors in 9 projects
target_arch "s390" 30 errors in 1 projects
target_arch "aarch" 28 errors in 5 projects
target_os "androideabi" 28 errors in 1 projects
target_os "cloudabi" 26 errors in 5 projects
target_env "kernel" 26 errors in 2 projects
target_arch "alpha" 24 errors in 2 projects
target_arch "parisc" 24 errors in 2 projects
target_arch "arm64" 24 errors in 2 projects
target_feature "iter_advance_by" 24 errors in 3 projects
target_feature "crypto" 20 errors in 3 projects
target_os "cbe" 20 errors in 1 projects
target_arch "x64_64" 20 errors in 3 projects
target_os "switch" 18 errors in 3 projects
target_os "window" 18 errors in 5 projects
target_arch "parisc2.0" 16 errors in 1 projects
target_arch_char_unsigned None 16 errors in 1 projects
target_arch "i386" 16 errors in 3 projects
target_device "esp8266" 16 errors in 1 projects
target_is "macos" 16 errors in 2 projects
target_feature "exact_size_is_empty" 16 errors in 3 projects
target_arch "x64" 16 errors in 2 projects
target_os "solarish" 16 errors in 1 projects
target_arch "amd64" 16 errors in 1 projects
target_arch "mos" 16 errors in 1 projects
target_feature "play-single-file" 16 errors in 1 projects
target_feature "f64" 16 errors in 1 projects
target_os "wasm32" 14 errors in 3 projects
target_arch "xtensa" 14 errors in 3 projects

Summary

There are four big outliers: docsrs (already fixed)2, rustfmt (deprecated/no longer in use) as well as feature="cargo-clippy" (and feature="clippy") both of which are de-facto deprecated and no longer used. There is nothing do to for those.

Therefore (excluding the outliers) there are 188089 unexpected cfgs errors, of which 69897 of the are unexpected feature cfgs and 11125 errors are of target_*.

After manually checking the 3 categories above I wasn't able to find any false positive1; in total 9649 actions would need to be taken across 5959 projects (1.4% of all the projects tested), by either: fixing the typo, removing the staled condition, marking as expected the custom cfg, adding the feature to the features table...

All and all, while the impact is certainly not minimal, I think it is heavily mitigated by the fact that it's warn-by-default, only affects maintainers (never the downstream users) and by it's ability to warn users on unexpected cfgs, a deficiency of the Rust toolchain, reported many times, that will final be fixed.

Footnotes

  1. by "false positive" I mean: missing well known names/values ↩ ↩2

  2. docsrs shouldn't have pop-up in this crater run but I was trick by the off-by-one date of rustup and rustc -V :-| ↩

@Urgau
Copy link
Member Author

Urgau commented Feb 10, 2024

Btw this report was analysed by this custom python script.

@rustbot label -S-waiting-on-review

@Urgau Urgau closed this as completed Feb 10, 2024
@rustbot rustbot removed the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Feb 10, 2024
@Urgau Urgau mentioned this issue Feb 10, 2024
Closed
3 tasks
@Urgau Urgau mentioned this issue May 6, 2024
Merged
@epage epage mentioned this issue May 6, 2024
Closed
@workingjubilee
Copy link
Member

I was not aware that we had ruled out target_pointer_width = "128" as a possibility.

@Urgau Urgau mentioned this issue May 7, 2024
Closed
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
F-check-cfg --check-cfg
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@Urgau @craterbot @workingjubilee @rustbot @Noratrieb