Merge branch 'main' into mktemp-uncomment-fixed-test

.vscode/cspell.dictionaries/workspace.wordlist.txt

@@ -20,6 +20,7 @@ exacl
 filetime
 formatteriteminfo
 fsext
+fundu
 getopts
 getrandom
 globset

CONTRIBUTING.md

@@ -1,20 +1,12 @@
-# Contributing to coreutils
-
-Contributions are very welcome, and should target Rust's main branch until the
-standard libraries are stabilized. You may *claim* an item on the to-do list by
-following these steps:
+<!-- spell-checker:ignore reimplementing toybox RUNTEST -->
 
-1. Open an issue named "Implement [the utility of your choice]", e.g. "Implement
-   ls".
-1. State that you are working on this utility.
-1. Develop the utility.
-1. Add integration tests.
-1. Add the reference to your utility into Cargo.toml and Makefile.
-1. Remove utility from the to-do list in the README.
-1. Submit a pull request and close the issue.
+# Contributing to coreutils
 
-The steps above imply that, before starting to work on a utility, you should
-search the issues to make sure no one else is working on it.
+Contributions are very welcome via Pull Requests. If you don't know where to
+start, take a look at the
+[`good-first-issues`](https://github.com/uutils/coreutils/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22).
+If you have any questions, feel free to ask them in the issues or on
+[Discord](https://discord.gg/wQVJbvJ).
 
 ## Best practices
 
@@ -38,36 +30,240 @@ search the issues to make sure no one else is working on it.
 
 ## Platforms
 
-We take pride in supporting many operating systems and architectures.
+We take pride in supporting many operating systems and architectures. Any code
+you contribute must at least compile without warnings for all platforms in the
+CI. However, you can use `#[cfg(...)]` attributes to create platform dependent features.
 
-**Tip:**
-For Windows, Microsoft provides some images (VMWare, Hyper-V, VirtualBox and Parallels)
-for development:
+**Tip:** For Windows, Microsoft provides some images (VMWare, Hyper-V,
+VirtualBox and Parallels) for development:
 <https://developer.microsoft.com/windows/downloads/virtual-machines/>
 
+## Tools
+
+We have an extensive CI that will check your code before it can be merged. This
+section explains how to run those checks locally to avoid waiting for the CI.
+
+### pre-commit hooks
+
+A configuration for `pre-commit` is provided in the repository. It allows
+automatically checking every git commit you make to ensure it compiles, and
+passes `clippy` and `rustfmt` without warnings.
+
+To use the provided hook:
+
+1. [Install `pre-commit`](https://pre-commit.com/#install)
+1. Run `pre-commit install` while in the repository directory
+
+Your git commits will then automatically be checked. If a check fails, an error
+message will explain why, and your commit will be canceled. You can then make
+the suggested changes, and run `git commit ...` again.
+
+### clippy
+
+```shell
+cargo clippy --all-targets --all-features
+```
+
+The `msrv` key in the clippy configuration file `clippy.toml` is used to disable
+lints pertaining to newer features by specifying the minimum supported Rust
+version (MSRV).
+
+### rustfmt
+
+```shell
+cargo fmt --all
+```
+
+### cargo-deny
+
+This project uses [cargo-deny](https://github.com/EmbarkStudios/cargo-deny/) to
+detect duplicate dependencies, checks licenses, etc. To run it locally, first
+install it and then run with:
+
+```
+cargo deny --all-features check all
+```
+
+### Markdown linter
+
+We use [markdownlint](https://github.com/DavidAnson/markdownlint) to lint the
+Markdown files in the repository.
+
+### Spell checker
+
+We use `cspell` as spell checker for all files in the project. If you are using
+VS Code, you can install the
+[code spell checker](https://marketplace.visualstudio.com/items?itemName=streetsidesoftware.code-spell-checker)
+extension to enable spell checking within your editor. Otherwise, you can
+install [cspell](https://cspell.org/) separately.
+
+If you want to make the spell checker ignore a word, you can add
+
+```rust
+// spell-checker:ignore word_to_ignore
+```
+
+at the top of the file.
+
+## Testing
+
+Testing can be done using either Cargo or `make`.
+
+### Testing with Cargo
+
+Just like with building, we follow the standard procedure for testing using
+Cargo:
+
+```shell
+cargo test
+```
+
+By default, `cargo test` only runs the common programs. To run also platform
+specific tests, run:
+
+```shell
+cargo test --features unix
+```
+
+If you would prefer to test a select few utilities:
+
+```shell
+cargo test --features "chmod mv tail" --no-default-features
+```
+
+If you also want to test the core utilities:
+
+```shell
+cargo test  -p uucore -p coreutils
+```
+
+To debug:
+
+```shell
+gdb --args target/debug/coreutils ls
+(gdb) b ls.rs:79
+(gdb) run
+```
+
+### Testing with GNU Make
+
+To simply test all available utilities:
+
+```shell
+make test
+```
+
+To test all but a few of the available utilities:
+
+```shell
+make SKIP_UTILS='UTILITY_1 UTILITY_2' test
+```
+
+To test only a few of the available utilities:
+
+```shell
+make UTILS='UTILITY_1 UTILITY_2' test
+```
+
+To include tests for unimplemented behavior:
+
+```shell
+make UTILS='UTILITY_1 UTILITY_2' SPEC=y test
+```
+
+### Run Busybox Tests
+
+This testing functionality is only available on *nix operating systems and
+requires `make`.
+
+To run busybox tests for all utilities for which busybox has tests
+
+```shell
+make busytest
+```
+
+To run busybox tests for a few of the available utilities
+
+```shell
+make UTILS='UTILITY_1 UTILITY_2' busytest
+```
+
+To pass an argument like "-v" to the busybox test runtime
+
+```shell
+make UTILS='UTILITY_1 UTILITY_2' RUNTEST_ARGS='-v' busytest
+```
+
+### Comparing with GNU
+
+To run uutils against the GNU test suite locally, run the following commands:
+
+```shell
+bash util/build-gnu.sh
+bash util/run-gnu-test.sh
+# To run a single test:
+bash util/run-gnu-test.sh tests/touch/not-owner.sh # for example
+# To run several tests:
+bash util/run-gnu-test.sh tests/touch/not-owner.sh tests/rm/no-give-up.sh # for example
+# If this is a perl (.pl) test, to run in debug:
+DEBUG=1 bash util/run-gnu-test.sh tests/misc/sm3sum.pl
+```
+
+Note that it relies on individual utilities (not the multicall binary).
+
+### Improving the GNU compatibility
+
+The Python script `./util/remaining-gnu-error.py` shows the list of failing
+tests in the CI.
+
+To improve the GNU compatibility, the following process is recommended:
+
+1. Identify a test (the smaller, the better) on a program that you understand or
+   is easy to understand. You can use the `./util/remaining-gnu-error.py` script
+   to help with this decision.
+1. Build both the GNU and Rust coreutils using: `bash util/build-gnu.sh`
+1. Run the test with `bash util/run-gnu-test.sh <your test>`
+1. Start to modify `<your test>` to understand what is wrong. Examples:
+   1. Add `set -v` to have the bash verbose mode
+   1. Add `echo $?` where needed
+   1. When the variable `fail` is used in the test, `echo $fail` to see when the
+      test started to fail
+   1. Bump the content of the output (ex: `cat err`)
+   1. ...
+1. Or, if the test is simple, extract the relevant information to create a new
+   test case running both GNU & Rust implementation
+1. Start to modify the Rust implementation to match the expected behavior
+1. Add a test to make sure that we don't regress (our test suite is super quick)
+
 ## Commit messages
 
 To help the project maintainers review pull requests from contributors across
 numerous utilities, the team has settled on conventions for commit messages.
 
-From <http://git-scm.com/book/ch5-2.html>:
+From <https://git-scm.com/book/ch5-2.html>:
 
 ```
-Short (50 chars or less) summary of changes
+Capitalized, short (50 chars or less) summary
 
 More detailed explanatory text, if necessary.  Wrap it to about 72
 characters or so.  In some contexts, the first line is treated as the
 subject of an email and the rest of the text as the body.  The blank
 line separating the summary from the body is critical (unless you omit
-the body entirely); tools like rebase can get confused if you run the
+the body entirely); tools like rebase will confuse you if you run the
 two together.
 
+Write your commit message in the imperative: "Fix bug" and not "Fixed bug"
+or "Fixes bug."  This convention matches up with commit messages generated
+by commands like git merge and git revert.
+
 Further paragraphs come after blank lines.
 
   - Bullet points are okay, too
 
-  - Typically a hyphen or asterisk is used for the bullet, preceded by a
+  - Typically a hyphen or asterisk is used for the bullet, followed by a
     single space, with blank lines in between, but conventions vary here
+
+  - Use a hanging indent
 ```
 
 Furthermore, here are a few examples for a summary line:
@@ -103,15 +299,49 @@ uutils: add new utility
 gitignore: add temporary files
 ```
 
-## cargo-deny
+## Code coverage
 
-This project uses [cargo-deny](https://github.com/EmbarkStudios/cargo-deny/) to
-detect duplicate dependencies, checks licenses, etc. To run it locally, first
-install it and then run with:
+<!-- spell-checker:ignore (flags) Ccodegen Coverflow Cpanic Zinstrument Zpanic -->
 
+Code coverage report can be generated using [grcov](https://github.com/mozilla/grcov).
+
+### Using Nightly Rust
+
+To generate [gcov-based](https://github.com/mozilla/grcov#example-how-to-generate-gcda-files-for-a-rust-project) coverage report
+
+```shell
+export CARGO_INCREMENTAL=0
+export RUSTFLAGS="-Zprofile -Ccodegen-units=1 -Copt-level=0 -Clink-dead-code -Coverflow-checks=off -Zpanic_abort_tests -Cpanic=abort"
+export RUSTDOCFLAGS="-Cpanic=abort"
+cargo build <options...> # e.g., --features feat_os_unix
+cargo test <options...> # e.g., --features feat_os_unix test_pathchk
+grcov . -s . --binary-path ./target/debug/ -t html --branch --ignore-not-existing --ignore build.rs --excl-br-line "^\s*((debug_)?assert(_eq|_ne)?\#\[derive\()" -o ./target/debug/coverage/
+# open target/debug/coverage/index.html in browser
 ```
-cargo deny --all-features check all
-```
+
+if changes are not reflected in the report then run `cargo clean` and run the above commands.
+
+### Using Stable Rust
+
+If you are using stable version of Rust that doesn't enable code coverage instrumentation by default
+then add `-Z-Zinstrument-coverage` flag to `RUSTFLAGS` env variable specified above.
+
+
+## Other implementations
+
+The Coreutils have different implementations, with different levels of completions:
+
+* [GNU's](https://git.savannah.gnu.org/gitweb/?p=coreutils.git)
+* [OpenBSD](https://github.com/openbsd/src/tree/master/bin)
+* [Busybox](https://github.com/mirror/busybox/tree/master/coreutils)
+* [Toybox (Android)](https://github.com/landley/toybox/tree/master/toys/posix)
+* [V lang](https://github.com/vlang/coreutils)
+* [SerenityOS](https://github.com/SerenityOS/serenity/tree/master/Userland/Utilities)
+* [Initial Unix](https://github.com/dspinellis/unix-history-repo)
+
+However, when reimplementing the tools/options in Rust, don't read their source codes 
+when they are using reciprocal licenses (ex: GNU GPL, GNU LGPL, etc).
+
 
 ## Licensing