Break out doctesting / refactor

[mortenpi]

This refactors a few parts of the build pipeline to make it possible to run doctests without having to compile the whole document (close #198, close #535, close #756).

• Moves doctesting to an earlier stage in the build.
• Implements doctest = :only to makedocs, which only runs doctests and skips the later build steps.
• When fixing doctests you probably do not care about the output either, so doctest = :fix also skips compilation and rendering of the docs now.
• Creates an API for storing module-level documentation metadata (analogue to an at-meta block), via the exported DocMeta module (e.g. DocMeta.setdocmeta!(module, :DocTestSetup, :(using Foo)). Only supports :DocTestSetup at the moment.
• Adds an exported doctest(::Vector{Module}) function to easily run doctests in a module.

The changes are a bit more extensive than they necessarily should be, but I am also trying to clean up and refactor the internals in the process. There is a bit more info about where I want to go with these changes in the wiki.

This is still very much a WIP and I think I might have to rethink some decisions here. Also needs testing, tests etc. Any drive-by comments will be much appreciated though. This is now ready for review.

• Update the manual page on doctesting

[mortenpi]

So.. this should work now.. theoretically.

One hurdle I came across as I was trying to test it with the Julia manual, is that it actually breaks nearly all the doctests in docstrings. This is because they usually rely on DocTestSetups in the Markdown files that no longer have any effect on the doctesting of the docstrings.

Now, this is not necessarily a bad thing. The whole idea of this is to be able to run doctest without having to go through all the hoops of figuring out which docstring ends where etc. It is also weird that the validity of a doctest depends on some metadata far away in a manual .md file. But having to put at-meta blocks into each docstring (which is now technically supported) is not optimal either.

So I've prototyped a way to add doctest metadata to modules instead. Essentially, every module gets a special gensymed variable that contains the metadata (it's how docstrings themselves are handled). It does require Documenter -- there is the Documenter.DocTests.doctestsetup! function which declares an Expr object as a DocTestSetup of a module.

The question is: when should you actually add the metadata? Ideally, it would be close to the docstrings.

• You could make your package depend on Documenter and just call Documenter.DocTests.doctestsetup!(@__MODULE__, :( ... )) in the module. But we don't want packages to depend on Documenter.

• You can add the metadata in the make.jl file by just making all the doctestsetup! calls after you import your package. In a sense, this is similar to how it's done now, where the metadata lives in the .md files under docs/. This might be a good solution for Base and the standard libraries.

• But I am also thinking about making it possible to define the metadata using a global constant in your module, e.g. const _DOCUMENTER_DOCTESTSETUP = :( ... ). Documenter can then just walk through all the modules passed with modules keyword (these are the ones that get tested anyway) and check for this variable. The downside is that it introduces a visible variable into the module namespace.

[mortenpi]

Ok, this is close to being finished now I think. I've updated the OP with a summary of what this adds and changes (Documenter will also export a doctest function and DocMeta module with this).

Bikeshedding invitation: I can't decide if it should be DocMeta / DocMeta.getdocmeta / DocMeta.setdocmeta! or DocsMeta / DocsMeta.getdocsmeta / DocsMeta.setdocsmeta! though (or maybe something else altogether).