Testing tool for comparing output of macro against expected source code

[dtolnay]

Similar to what https://github.com/dtolnay/trybuild does for testing compilation failures.

See here and here for a basic test setup leveraging trybuild.

I would expect a very similar setup for testing macro expansion -- with a suite of test cases as *.rs files in some directory, adjacent to expected expanded output in *.expanded.rs files with corresponding names. For each test case the library would set up dependencies and stuff correctly then invoke cargo expand to expand macros and format the expanded code with rustfmt. Then the test case would fail if the expanded and formatted code differs from the expected output in the corresponding *.expanded.rs file.

[dtolnay]

This is better than invoking proc macros directly as fn(TokenStream) -> TokenStream from tests for a few reasons:

• It supports testing macro_rules macros as well.
• It does not require factoring testable code out of your proc-macro = true entry point crate solely for testing purposes. We can test the proc macro crate directly.
• The tool can provide a simple command which automatically updates all the *.expanded.rs files to account for changes in the macro's implementation.

[eupn]

@dtolnay I want to give it a try. It's ok?

[dtolnay]

Sounds good! Thanks for taking a look.

[eupn]

@dtolnay macrotest repository is here: https://github.com/eupn/macrotest. I have quick-and-dirty version done and simple macro_rules! macro expansion test project here: https://github.com/eupn/macrotest/tree/master/test-project/tests, check it out.

[eupn]

Oh, I see that you've updated the issue's description. I will update the macrotest's API to resemble the trybuild closely.

[eupn]

Updated the API to be similar to trybuild. https://github.com/eupn/macrotest/blob/master/test-project/tests/tests.rs

@dtolnay what should be considered as "expected failure" case of cargo expand invocation? Any non-success return code? Or should we add a comparison with .stderr file?

[dtolnay]

There are no expected failures. If any cargo expand exits unsuccessful then it should fail the test.

[eupn]

Published on crates.io: macrotest

[eupn]

Added procedural macro example. Even though it has some issues (#6, #7), it works.

[dtolnay]

Nice, looks like a great start. I filed some issues based on trying out the crate. I think we can close out this issue once those are fixed. Thanks!

[eupn]

Thanks for the thorough review! Will fix as many as possible.

[eupn]

@dtolnay issues are fixed

[dtolnay]

Awesome!

[eupn]

@dtolnay btw, which of your crates with macros/proc-macros (or crates that you know) would make use of this tool? I'd be happy to file a PR to some of those crates to employ macrotest in them.

[dtolnay]

I would like to try it in Serde because the logic inside Deserialize derives can be pretty complicated to reason about. serde-rs/serde#1117