Add builtin standards for Pandoc filter globals

[alerque]

This is the bare minimum to get started. The complete API is pretty well document and we could probably detail all the modules and fields/functions, but one has to start somewhere and I don't have infinite time. Contributions welcome!

[alerque]

@tarleb If you get a chance I'd be happy for feedback on this. I started in an it and ran into several things I didn't expect even after plenty of my own use to pull from.

For starters, does it make any sense to split up filters, readers, and writers as far as what globals they are allowed to read and/or write? I realize there will be a lot of overlap (which is fine) but is there any meaningful distinction at all?
For example will filters only set a global for Math while a writer won't be allowed that but might have MathInline and MathDisplay? Would a filter ever seta Writer or Doc global function for any valid reason?

Also, is there any difference between filters that just set global functions vs, ones that return a table with filter functions? Obviously in many cases they are currently interchangeable, but my take is the latter is a little more robust. Is the former being deprecated in favor of the latter?

[tarleb]

Cool! I think the distinction between reader and writer could be dropped, pandoc just has a single custom category. In fact, having both a reader and a writer in the same file is perfectly fine.

I'm not really familiar with luacheck and will have to do some reading before I can give more qualified comments.

[alerque]

Thanks, that makes sense. How is the custom category scoped? Is it meaningfully different from the filter category? Are those two categories meaningful groupings or would it be possible to have a filter and a reader (or writer) in the same file? Are reader/writer strictly a superset of filter? Would it make sense to throw an error in a filter if one tried to define something unique to reader/writers?

[tarleb]

Some differences: the PANDOC_READER_OPTIONS, PANDOC_WRITER_OPTIONS, and FORMAT globals are set in filters, but not for custom readers/writers.

It might make sense to also distinguish pandoc-lua scripts, i.e., Lua scripts that use pandoc as the interpreter. It's similar to custom readers/writers, but the PANDOC_SCRIPT_FILE global isn't set for those scripts. That information would have to be taken from the arg global, that's only available in those scripts.

[alerque]

Per that comment I've refactored this into pandoc (anything goes), pandoc_filter, pandoc_custom, and pandoc_script presets.

Note that all of these are expected to be used as additions to lua54, eg lua54+pandoc_filter. It would be possible to change that to be based on lua54 out of the box and usage would be as a standalone std. Any thoughts on that? Are there cases where current Pandoc versions ship with something other than 5.4 equivalent VMs? E.g. Are there any platforms out there with Pandoc compiled against LuaJIT for example? Ar is that a viable thing that could happen in the future? Or might the ecosystem grow and Lua 5.6 eventually need to be linted separately for newer projects vs. 5.4 for older ones?

[alerque]

I ask about all those preset arrangements because I'm hopping to get this in a release soon, and any changes to the preset names will be a breaking change if we ever wanted to do so. Adding Pandoc support won't be a breaking change and expanding support for detailed lints (e.g. noting all the specific fields in the pandoc.<module> namespace) will be allowed without a breaking release, but rug-pulling a builtin name would be. Hence the desire to get that bit right ;-)

[tarleb]

The last release with Lua 5.3 was pandoc 2.19. The version shipped with Debian bookworm is 2.17, and some Ubuntu LTS versions also have pandoc versions that use Lua 5.3. Based on that, I'd say that it makes sense to specify the Lua version.

I think the last pandoc version that could, theoretically, have been built against luajit was 2.3.2, so it's really only 5.3 and 5.4 at this point.

[alerque]

Thanks, I'll leave it as an add-on set and expect the user to specify lua54+pandoc_filter or similar as their std. That will give us future flexibility too if/when Pandoc bumps the LUA API version used.