Extending PML via Custom Scripts

[tajmone]

I know that the subject of which scripting language should be used to extend PML has come up many times, but I just don't remember where (possibly, in off-topic comments to threads discussing other aspects of PML); so I'm just creating a new Discussion thread for this.

Here's a good article on why Sublime Text developers chose Python:

• https://www.sublimetext.com/blog/articles/2008/04

The article is good because it also exposes the general considerations to keep in mind when choosing a scripting language to extend binary applications depending on different goals and use cases, and discusses alternative choices to Python too.

[pml-lang]

Thanks for the link.

All scripting languages supported by GraalVM could potentially be used in PML. At the moment these are:

• Javascript
• LLVM
• Python
• R
• Ruby

I totally agree that Python is the best choice (as suggested in the article).

We should also support Javascript (not because I think it's a good language, but because of its popularity). Note: The article mentions "no selection of libraries your users will be able to build on" as an inconvenience of Javascript. That was true in 2008 (when the article was written), but it's no longer an issue since the rise of Node.js in 2009. Note: Node.js is supported in GraalVM.

[tajmone]

I totally agree that Python is the best choice (as suggested in the article).

I wouldn't sat it's "the best choice", just an "obvious" one, and would argue that Ruby is just as good a choice as Python. Both languages are capable of doing the same stuff, they just have different philosophies. It really boils down to personal taste, some people feel naturally attracted to the rigidity of Python's "one and only way to do things" while others prefer the freedom of being able to do anything in many different ways, and most of the technical pros and cons arguments are mere rationalizations — it's like the "square vs non-conformist" debate.

As for ST having chosen Python, there's also a dark side to the story, i.e. that for ST3 they decided to settle on a specific Python version (2.7 IIRC), which the Python org had granted would always receive security patches, except it didn't and at ST they ended up having to manually create patches for the Python library (Linux, Mac and Win) for the whole time. They wrote an article (or post) about it, the maintenance burden, the disappointment of failed promises (so much for the rigid rules and super-precise philosophy of Python, as it turned out in this case, you can't always take for gold all promises).

Python has probably a richer ecosystem to pick from, but let's not forget that Ruby is where all the AsciiDoc top treasures are today — with Asciidoctor being a better implementation that AsciiDoc Python IMO (which also went dormant for a long time) and with many more extra tools and extensions. This is no little consideration when it comes to PML, and it might depend on how you're planning to interface with other lightweight markup syntaxes, and whether AsciiDoc has a higher priority than Markdown, ResT, etc. (Python is probably richer in terms of Markdown and ResT libraries, but possibly JavaScript even more so).

We should also support Javascript (not because I think it's a good language, but because of its popularity).

Agree on that (especially on it not being a good language at all). The NPM ecosystem definitely is a treasure mine of libraries and CLI tools when it comes to markup and format serializations — but there are always the security concerns with Node JS. Also, when it comes to NPM, abundance is not synonym of quality, and many of these packages are makeshift improvisation, and often destined to become stale and unmaintained in the course of time (Node evolves too fast for my personal taste).

LLVM is tuff to setup on Windows, I fear that adopting this would result in Win users feeling cut-off — most Linux and macOS users are going to love LLVM, and the risk is that a lot of focus would go there, leaving the Win userbase marginalized. When it comes to choices that could come at the expense of a specific OS, I'd rather have macOS (and any OS by Apple) be left out, since it's not really an OS but a hardware-bound OS — i.e. you can't just buy it and install it on any hardware, it's a hyper-proprietary and machine/brand-specific OS. Also, it looks like Apple is going to (again) change architecture for their Macs, which is always bad news for binary applications. When Apple decided to kill 32-bit support on their latest OS updates, they basically killed all the legacy software tools of the past, for no good reason — the x86_64 architecture is designed to support multiple modes, which is why it's such a bloated processor focused on backward compatibility. You never know what to expect from Apple, they seem to take frequent radical choices for their OSs, leaving their users with little choices, and macOS software developers with lots of yaks shaving to do to keep on track.

R? That's a cool language, but also one for a very technical audience (more for engineers than writers).

Pity that GraalVM doesn't support Lua, that would have been a good choice too, since its simple, fast and well known.

[tajmone]

Many Scripts or Just One?

What's your take on scripts support in PML?

Would you rather support multiple languages, or stick with one choice?

The former might open extensions to a wider userbase, at the cost of fragmenting and diluting third party assets.

The latter will prevent fragmentation, but would also reduce contributions to those who know (or like) the chosen script.

I tend to prefer choosing one script and sticking to it, so that extensions can interact with each other and to have a consistent ecosystem.

Choosing a single script might also dispense of the need of GraalVM, for you could focus on language specific bindings/wrapper, which will probably give more granular control on how the language integrates with PML, whereas with GraalVM you'll end depending on its upstream choices and development cycle.

[pml-lang]

Would you rather support multiple languages, or stick with one choice?

That's a good question with no easy answer.

I would suggest to first support Python, then Javascipt and Ruby.

When we add scripting to PML, it should be done with great flexibility in mind, so that languages can later easily be added/removed.

Ideally, it should also be possible to have a mixture of different scripting languages used in a single PML document, so that user-defined nodes can easily be shared (e.g. Javascript file my_PML_node_1.js and Python file your_PML_node_2.py can both be used simultaneously in the same PML document). This should be possible to achieve with GraalVM.

Pity that GraalVM doesn't support Lua.

Yes. Maybe it'll be supported in the future.