util/show-utils: fix script not working as intended #4560

miles170 · 2023-03-20T10:06:36Z

On FreeBSD, realpath -mP arguments are not available.
Fix error: unexpected argument "--features 'feat_os_unix'" found

The current script doesn't work well since it produces no result but errors.
Check Style/lint -> Initialize workflow variables step on any CICD task.

And cargo clippy --all-targets --features "feat_os_unix" -pXX seems not to work on Ubuntu or FreeBSD.
But cargo clippy --all-targets -pXX works well.

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 22.10
Release:        22.10
Codename:       kinetic
$ cargo clippy --all-targets --features "feat_os_unix" -puu_arch
error: none of the selected packages contains these features: feat_os_unix
$ cargo clippy --all-targets --features "feat_os_unix" -puu_yes
error: none of the selected packages contains these features: feat_os_unix
$ cargo clippy --all-targets -puu_yes
    Finished dev [unoptimized + debuginfo] target(s) in 0.06s
$ cargo clippy --all-targets --all-features -puu_yes
    Finished dev [unoptimized + debuginfo] target(s) in 0.06s

github-actions · 2023-03-20T11:04:47Z

GNU testsuite comparison:

Congrats! The gnu test tests/tail-2/inotify-dir-recreate is no longer failing!

miles170 · 2023-03-20T15:17:45Z

Should cargo clippy --all-targets --features "feat_os_unix" -pXX be replaced by cargo clippy --all-targets -pXX?

tertsdiepraam · 2023-03-20T16:21:29Z

I don't really understand. Why would it be better to remove the features? If we remove it could won't be able to catch problems in unix-only utils, right?

miles170 · 2023-03-21T01:03:18Z

I don't really understand. Why would it be better to remove the features? If we remove it could won't be able to catch problems in unix-only utils, right?

Doesn't show-utils.sh already provide platform-specific utilities?

$ ./util/show-utils.sh --features feat_os_unix
arch base32 base64 basename basenc cat chgrp chmod chown chroot cksum comm cp csplit cut date dd df dir dircolors dirname du echo env expand expr factor false fmt fold groups hashsum head hostname id install join kill link ln logname ls mkdir mkfifo mknod mktemp more mv nice nl nohup nproc numfmt od paste pathchk pinky pr printenv printf ptx pwd readlink realpath relpath rm rmdir seq shred shuf sleep sort split stat stdbuf stty sum sync tac tail tee test timeout touch tr true truncate tsort tty uname unexpand uniq unlink uptime users vdir wc who whoami yes                                                                                                                          
$ ./util/show-utils.sh --features feat_os_windows
arch base32 base64 basename basenc cat cksum comm cp csplit cut date dd df dir dircolors dirname du echo env expand expr factor false fmt fold hashsum head hostname join link ln ls mkdir mktemp more mv nl nproc numfmt od paste pr printenv printf ptx pwd readlink realpath relpath rm rmdir seq shred shuf sleep sort split sum sync tac tail tee test touch tr true truncate tsort uname unexpand uniq unlink vdir wc whoami yes
$ ./util/show-utils.sh --features feat_os_macos
arch base32 base64 basename basenc cat chgrp chmod chown chroot cksum comm cp csplit cut date dd df dir dircolors dirname du echo env expand expr factor false fmt fold groups hashsum head hostid hostname id install join kill link ln logname ls mkdir mkfifo mknod mktemp more mv nice nl nohup nproc numfmt od paste pathchk pinky pr printenv printf ptx pwd readlink realpath relpath rm rmdir seq shred shuf sleep sort split stat stdbuf stty sum sync tac tail tee test timeout touch tr true truncate tsort tty uname unexpand uniq unlink uptime users vdir wc who whoami yes

tertsdiepraam

You seem to have included some irrelevant changes in other files. In the future it's probably better to include those in another PR, but since they're in separate commits, I suppose it's fine for now.

You're right about the feature list, it should indeed not be necessary :)

tertsdiepraam · 2023-03-24T20:09:28Z

.github/workflows/CICD.yml

@@ -206,7 +206,7 @@ jobs:
        # target-specific options
        # * CARGO_FEATURES_OPTION
        CARGO_FEATURES_OPTION='--all-features' ;
-        if [ -n "${{ matrix.job.features }}" ]; then CARGO_FEATURES_OPTION='--features "${{ matrix.job.features }}"' ; fi
+        if [ -n "${{ matrix.job.features }}" ]; then CARGO_FEATURES_OPTION='--features ${{ matrix.job.features }}' ; fi


Why remove the quotes?

If I didn't remove the quotes, the CI is failing to get utilities, as you can see here. https://github.com/miles170/coreutils/actions/runs/4516836207/jobs/7955560726 (Initialize workflow variables)

steps.vars.outputs.CARGO_FEATURES_OPTION=--features "feat_os_unix" Updating crates.io index error: none of the selected packages contains these features: "feat_os_unix", did you mean: feat_os_unix? UTILITY_LIST= steps.vars.outputs.CARGO_UTILITY_LIST_OPTIONS=

I'm assuming the error is because cargo is treating "feat_os_unix" as a feature instead of feat_os_unix.

tertsdiepraam · 2023-03-24T20:09:56Z

.github/workflows/CICD.yml

@@ -319,7 +319,7 @@ jobs:
        # target-specific options
        # * CARGO_FEATURES_OPTION
        CARGO_FEATURES_OPTION='--all-features' ;
-        if [ -n "${{ matrix.job.features }}" ]; then CARGO_FEATURES_OPTION='--features "${{ matrix.job.features }}"' ; fi
+        if [ -n "${{ matrix.job.features }}" ]; then CARGO_FEATURES_OPTION='--features ${{ matrix.job.features }}' ; fi


Same question here

If I didn't remove the quotes, the CI is failing to get utilities, as you can see here. https://github.com/miles170/coreutils/actions/runs/4516836207/jobs/7955561283 (Initialize workflow variables)

steps.vars.outputs.CARGO_FEATURES_OPTION=--features "feat_os_unix" Updating crates.io index error: none of the selected packages contains these features: "feat_os_unix", did you mean: feat_os_unix? UTILITY_LIST= steps.vars.outputs.CARGO_UTILITY_LIST_OPTIONS=

I'm assuming the error is because cargo is treating "feat_os_unix" as a feature instead of feat_os_unix.

tertsdiepraam · 2023-03-24T20:10:30Z

util/show-utils.sh

@@ -22,6 +23,6 @@ cd "${project_main_dir}" &&
        echo "WARN: missing \`jq\` (install with \`sudo apt install jq\`); falling back to default (only fully cross-platform) utility list" 1>&2
        echo "$default_utils"
    else
-        cargo metadata "$*" --format-version 1 | jq -r "[.resolve.nodes[] | { id: .id, deps: [.deps[] | { name:.name, pkg:.pkg }] }] | .[] | select(.id|startswith(\"coreutils\")) | [.deps[] | select((.name|startswith(\"uu_\")) or (.pkg|startswith(\"uu_\")))] | [.[].pkg | match(\"^\\\w+\";\"g\")] | [.[].string | sub(\"^uu_\"; \"\")] | sort | join(\" \")"
-        # cargo metadata "$*" --format-version 1 | jq -r "[.resolve.nodes[] | { id: .id, deps: [.deps[] | { name:.name, pkg:.pkg }] }] | .[] | select(.id|startswith(\"coreutils\")) | [.deps[] | select((.name|startswith(\"uu_\")) or (.pkg|startswith(\"uu_\")))] | [.[].pkg | match(\"^\\\w+\";\"g\")] | [.[].string] | sort | join(\" \")"
+        cargo metadata "$@" --format-version 1 | jq -r "[.resolve.nodes[] | { id: .id, deps: [.deps[] | { name:.name, pkg:.pkg }] }] | .[] | select(.id|startswith(\"coreutils\")) | [.deps[] | select((.name|startswith(\"uu_\")) or (.pkg|startswith(\"uu_\")))] | [.[].pkg | match(\"^\\\w+\";\"g\")] | [.[].string | sub(\"^uu_\"; \"\")] | sort | join(\" \")"


Why this change?

$* expands to a single argument, which results in an error. ($@ expands to multiple arguments)

$ sh ./util/show-utils.sh --features feat_os_unix error: unexpected argument '--features feat_os_unix' found note: argument '--features' exists Usage: cargo metadata --features <FEATURES> For more information, try '--help'.

miles170 · 2023-03-25T04:07:04Z

You seem to have included some irrelevant changes in other files. In the future it's probably better to include those in another PR, but since they're in separate commits, I suppose it's fine for now.

You're right about the feature list, it should indeed not be necessary :)

The reason why I corrected those two errors is that the original lint task was not functioning properly, which resulted in those errors not being detected. After fixing the lint task, those two errors were successfully identified. If not corrected, it would lead to the failure of the lint task.

tertsdiepraam · 2023-03-25T09:24:15Z

That makes sense! Thanks!

tertsdiepraam

Alright, I'm convinced :) So we've been running a faulty CI this whole time? @sylvestre you know bash better than me, does it look good to you?

sylvestre · 2023-03-27T18:25:28Z

Looks good
thanks

miles170 marked this pull request as draft March 20, 2023 11:09

miles170 force-pushed the fix-show-utils branch from 63fa231 to ac181b9 Compare March 20, 2023 12:47

miles170 marked this pull request as ready for review March 20, 2023 12:47

miles170 force-pushed the fix-show-utils branch from ac181b9 to bac4ecb Compare March 20, 2023 15:26

miles170 added 2 commits March 21, 2023 09:20

touch: fix err_expect clippy error

d2a0a73

progress: fix redundant_field_names clippy error

c9a95c3

miles170 force-pushed the fix-show-utils branch from bac4ecb to 8ecc651 Compare March 21, 2023 01:21

util/show-utils: fix script not working as intended

04f1b31

miles170 force-pushed the fix-show-utils branch from 8ecc651 to 04f1b31 Compare March 21, 2023 01:28

miles170 changed the title ~~util/show-utils: fix script not working on FreeBSD~~ util/show-utils: fix script not working as intended Mar 21, 2023

sylvestre requested a review from tertsdiepraam March 24, 2023 19:17

tertsdiepraam reviewed Mar 24, 2023

View reviewed changes

tertsdiepraam approved these changes Mar 25, 2023

View reviewed changes

sylvestre merged commit 97f8e9d into uutils:main Mar 27, 2023

miles170 deleted the fix-show-utils branch March 28, 2023 01:01

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

util/show-utils: fix script not working as intended #4560

util/show-utils: fix script not working as intended #4560

miles170 commented Mar 20, 2023 •

edited

Loading

github-actions bot commented Mar 20, 2023

miles170 commented Mar 20, 2023

tertsdiepraam commented Mar 20, 2023 •

edited

Loading

miles170 commented Mar 21, 2023

tertsdiepraam left a comment •

edited

Loading

tertsdiepraam Mar 24, 2023

miles170 Mar 25, 2023 •

edited

Loading

tertsdiepraam Mar 24, 2023

miles170 Mar 25, 2023 •

edited

Loading

tertsdiepraam Mar 24, 2023

miles170 Mar 25, 2023

miles170 commented Mar 25, 2023

tertsdiepraam commented Mar 25, 2023

tertsdiepraam left a comment

sylvestre commented Mar 27, 2023

util/show-utils: fix script not working as intended #4560

util/show-utils: fix script not working as intended #4560

Conversation

miles170 commented Mar 20, 2023 • edited Loading

github-actions bot commented Mar 20, 2023

miles170 commented Mar 20, 2023

tertsdiepraam commented Mar 20, 2023 • edited Loading

miles170 commented Mar 21, 2023

tertsdiepraam left a comment • edited Loading

Choose a reason for hiding this comment

tertsdiepraam Mar 24, 2023

Choose a reason for hiding this comment

miles170 Mar 25, 2023 • edited Loading

Choose a reason for hiding this comment

tertsdiepraam Mar 24, 2023

Choose a reason for hiding this comment

miles170 Mar 25, 2023 • edited Loading

Choose a reason for hiding this comment

tertsdiepraam Mar 24, 2023

Choose a reason for hiding this comment

miles170 Mar 25, 2023

Choose a reason for hiding this comment

miles170 commented Mar 25, 2023

tertsdiepraam commented Mar 25, 2023

tertsdiepraam left a comment

Choose a reason for hiding this comment

sylvestre commented Mar 27, 2023

miles170 commented Mar 20, 2023 •

edited

Loading

tertsdiepraam commented Mar 20, 2023 •

edited

Loading

tertsdiepraam left a comment •

edited

Loading

miles170 Mar 25, 2023 •

edited

Loading

miles170 Mar 25, 2023 •

edited

Loading