feat: Put the test code inline into the test module

[Marwes]

This fixes two major issues with skeptic's current implementation at the
cost of losing compile-fail tests. For many crates I expect this to be a
worthwhile tradeoff. It is possible to restore compile-tests but they
would be forced to use the same shelling out approach as before this PR
so they would suffer from the same problem. As such I prefer to keep
them out instead.

No longer shells out to rustc and attempts to emulate cargo's dependency
resolution.

Fixes #18

Tests are now in a normal test module letting cargo cache and reuse
compilation artifacts as normal.

Fixes #8

BREAKING CHANGE

• Handlebars templates are now used instead of format!
  Since there is no intermediate step we need a way to dynamically
  template. Handlebars is well known and well supported.

• Compile tests that expect panics are not supported
  Since code is generated inline, these tests would cause test
  compilation to fail

[budziq]

Hi @Marwes thanks for your work and sorry for takng so long to respond (I'm away from keyboard since february but I hope to find some time next week).

Your approach looks interesting but and I would be all for it but having no_run tests is an essential usecase for rust-cookbook that I also maintain (the original reason for skeptic in the first place (From cursory glance you are changing these to ignore which will still build them but not run them). Hmm I might leave with the tradeoff if it would spare me chasing cargo internals (but I need some time to look test how it behaves)

Additional nits:

• Travis seams unhappy with your PR (appveyor is broken due to cargo internals change on beta+ nightly that I did not have time to address). Would you be willing to address that?
• The PR is quite noisy with unrelated changes (a lot of formatting and change from error_chain to failure). Would you be willing to at least extract the changes unrelated to library workings to separate commit?
• the should_panic part should be addressable with should_panic test attribute. Would you be willing to look into it?
• I guess that with your approach the whole cargo internals parsing and rustc interaction code might be removed.

Unfortunately I have very limmited bandwidth right now due to personal reasons so I cannot make any guarantees :(

[Marwes]

@budziq I forgot to post an update here but I figured this would indeed be to invasive of a change so I published this independently as https://crates.io/crates/little-skeptic . I am happy enough with that approach for the moment as I mainly use skeptic to test examples of a script language written in Rust (and not Rust code itself) so I don't need the no_run tests.

I might contribute back the parts that work but no promises.

[budziq]

Thanks for the headsup! I hope to find some time in the upcoming future to evaluate your approach.

[budziq]

I had some time to play with your PR and sadly it would not be acceptable as this approach has one additional drawback that I have not thought about. Namely it prohibits usage of extern crate and #[macro_use] in the tested snippets which is a major feature for rust-cookbook and few other major users 😞

[Marwes]

I had some time to play with your PR and sadly it would not be acceptable as this approach has one additional drawback that I have not thought about. Namely it prohibits usage of extern crate and #[macro_use] in the tested snippets which is a major feature for rust-cookbook and few other major users 😞

Yeah, that is also a compromise I had to make to get things working :(. For me it is an acceptable trade off though (even if it wasn't necessary to fix #18) since it speeds up the test time significantly.

[slonopotamus]

@Marwes are there plans to make extern crate work in little-skeptic? Without this, each project using little-skeptic is forced to create a .skt.md file if they want to write tests for their own crate (and normally people write examples how to use their crate, not something different).

[Marwes]

That is not something that is planned, mostly because I don't have a good idea on how to easily get it working (I think it may need to use https://github.com/dtolnay/syn to extract extern crate items and hoist and de-dup them to the top level of the test perhaps, but it feels like a hairball of an issue).

[brson]

I like the idea of putting the test code directly into the test cases, and directly into the tests/skeptic.rs file. I agree though that this patch sacrifices too much (it looks to me like examples lose their main function, which is important in stdx at least). I'm hopeful that a solution can be found yet.

Edit: actually the more I look at it the less I like inline the tests into test functions. Having the documentation be complete crates makes the examples intuitive and behave like one would expect any crate root to behave. Putting it in the function you lose crate attributes, main that doesn't return (), process_exit, and more.