Cargo test quicker by not building untested examples

[dwijnand]

(TIL cargo test --tests runs test-enabled examples.)

Pushes forward from #5464
Fixes #6675
r? @ehuss

[dwijnand]

Test-wise, I tried to adapt tests to the new expectations however, some tests I felt had to be dropped entirely. With the test changes made I felt no further tests were required. But let me know what you think.

[ehuss]

I'm not sure I understand the motivation for this. Examples are intentionally built by default in cargo test to ensure they are not broken. That seems like a good compromise, in that examples almost never have unit tests (and thus building them as a test would be a waste). Not building them at all would risk them becoming stale.

I think it would be reasonable to skip non-tested examples if cargo test includes an argument (cargo test foo). But this is related to the general problem that Cargo does not know which target contains a specific test, and would only be a minor improvement of that problem.

[dwijnand]

I'm not sure I understand the motivation for this. Examples are intentionally built by default in cargo test to ensure they are not broken.

I guess our opinions differ here. I think examples are a great feature built into cargo, but I would've assumed you'd need to run something like cargo check --examples or cargo check --all-targets in CI to ensure they still compile. So I agree with the premise of #6675 that adding a space to a unit test shouldn't force all 12 examples to rebuild when running cargo test.

Two more questions:

1. if adding a space to a cfg test unit test, why doesn't the incremental compiler kick in and realise the examples don't need to recompile?
2. I had a variant of this based on the non-emptiness of test_args, i.e. exclude building non-testable examples if any arguments (e.g TESTNAME) was specified. Would you be happier to land a change along those lines?

[ehuss]

1. if adding a space to a cfg test unit test, why doesn't the incremental compiler kick in and realise the examples don't need to recompile?

I don't know much about how incremental works.

2. I had a variant of this based on the non-emptiness of `test_args`, i.e. exclude building non-testable examples if any arguments (e.g TESTNAME) was specified.  Would you be happier to land a change along those lines?

I think that would be the safer route if it is not too complex.

[dwijnand]

OK, I'll propose that instead.