No way to run coverage for tests in an example

[CAD97]

Using cargo-tarpaulin 0.12.3, --run-types Examples seems to be running the binaries produced by cargo build --examples (and in my case, failing to collect coverage for them some unknown reason). Instead, I want to run the test executable for the examples, produced by cargo test --examples.

Or more specifically, I want all tests run by cargo test --all-targets to run, and the examples' tests do not seem to be included in --run-types Tests.

[xd009642]

I'll have a look some point this week hopefully and see if I can figure it out, hopefully it's just a case of calling cargo wrong in src/cargo.rs

[CAD97]

Yep!

tarpaulin/src/cargo.rs

Lines 143 to 170 in 504acb1

	let mut test_cmd = Command::new("cargo");
	if *ty == RunType::Doctests {
	if let Some(toolchain) = env::var("RUSTUP_TOOLCHAIN")
	.ok()
	.filter(|t| t.starts_with("nightly"))
	{
	test_cmd.args(&[format!("+{}", toolchain).as_str(), "test"]);
	} else {
	test_cmd.args(&["+nightly", "test"]);
	}
	} else {
	if let Ok(toolchain) = env::var("RUSTUP_TOOLCHAIN") {
	test_cmd.arg(format!("+{}", toolchain));
	}
	if *ty != RunType::Examples {
	test_cmd.args(&["test", "--no-run"]);
	} else {
	test_cmd.arg("build");
	}
	}
	test_cmd.args(&["--message-format", "json", "--manifest-path", manifest_path]);
	match ty {
	RunType::Tests => test_cmd.arg("--tests"),
	RunType::Doctests => test_cmd.arg("--doc"),
	RunType::Benchmarks => test_cmd.arg("--benches"),
	RunType::Examples => test_cmd.arg("--examples"),
	};
	init_args(&mut test_cmd, config);

This is doing cargo build --examples for --run-type Examples. If the intent is to run the tests in the examples, the executed command should be cargo test --no-run --examples.

To be clear, I'm not certain if the behavior of --run-type Examples is wrong, persay. Assuming you can get coverage from an arbitrary entry point (and not just the rusttest harness), it can be useful to run examples with coverage for those examples which are oneshot runs.

---

Side note: I think, based on that code, that tarpaulin isn't running inline unit tests. IIUC, cargo test --tests just does the external tests (those in the tests folder and/or specified as tests in Cargo.toml, and --lib or --bins need to be used to get the unit tests out of the lib or bins, respectively. (Again, IIUC, the default of cargo test is approximately cargo test [--lib] [--bin default] --tests --doc (with --lib and --bin as appropriate for present lib/bin targets), and using an explicit picker removes those defaults.)

[xd009642]

I've done a fix for this and tested it on sorbus, it's on the develop branch if you want to use it before the next release. But if not I hope to solve some of the doc test coverage issues this week and then do a new release this weekend or next week 👍