Support JavaScript based customization of the Editor

[lmihalkovic]

This is actually quite straightforward. My thoughts for the moment are limited to

• allow user to manually store JS editor scripts in ~/Arduino15/scripts/xxxxxx (some structure that makes sense, rather than just a dumping ground)
• not offer any editor for the moment (good news is that from what I saw, REditxxxSomethingOrOther seems to have built-in JS awareness if in the future someone wants to write that)
• basic access to the real Editor/RSTA mechanics from JS scripts (dunno what that API surface will be but the idea is to interpose something in between that does not let user directly trash the Editor)
• have at least a "reformat-code" script (any pointer to existing JS based C++ code formatter welcome) as an example (maybe something simpler as a first useful demo of the feature)

A second step should cover

• registering the scripts against an editor type (e.g. a library.properties validation script should never be launched at a mysketch.ino
• a couple of useful scripts to validate lib.props and keywd.txt (in this case, a script could even make the suggestion of what contents might make sense as a simple unambitious 'Propose a completion keywrds list for this lib' accessible by right clicking on a lib node in the outline view.

[ffissore]

One needs a goal before doing something. I don't think you want to edit JS for the sake of editing it. So what's your goal?

[lmihalkovic]

Thank you for your patience with my ludicrously incomplete first attempt at describing the feature... I tried to clarify things.

The general idea is to allow external contributors to enhance the editor by exposing editor instances to a minimalistic scripting env. No support for writting these scripts (no JS editor, save as, rename, ...) just hooks to run them and a very small controlled api (to avoid future breaking changes when editor code evolves) to support them.

[lmihalkovic]

Implementation note:

• limiting the support code to using the standard scripting api would allow users to write scripts in any of the supported engines without further work on the UI
• if code needs to use private engine api (should not, but who knows) nashorn is target runtime rather than rhino

again just a thought

ps: unlesssomeone says it is idiotic or does it before, I will very likely implement it

[matthijskooijman]

So IIUC, you're proposing to integrate a javascript interpreter (I don't think there is one now) to basically allow creating "add-ons" for the editor in javascript? This sounds like overkill to me - writing those addons in java and providing them as .jar files might be a more obvious solution (in fact, you can do this now to some degree by providing "tool" jars, but it seems nobody is really using this and there is hardly any documentation for it).

[lmihalkovic]

@matthijskooijman

So IIUC, you're proposing to integrate a javascript interpreter (I don't think there is one now)

my intention is to use the one that's already there (hence the nashorn vs rhino reference)

This sounds like overkill to me - writing those addons in java and providing them as .jar files might be a more obvious solution (in fact, you can do this now to some degree by providing "tool" jars, but it seems nobody is really using this and there is hardly any documentation for it)

It seems as programmers we come from distant backgrounds. Getting either of us to fully appreciate the other's view point might not be straightforward (...I guess that where programmers typically pull the beer card). But I think you are partially answering the question of the relevance of the proposal in the rest of your comment: to some degree and nobody really.... IMHO, the current feature as it stands was bound to miss the mark. It is both too much and too little: to much work for an implementer, and too little payback for that effort. What I want to do is to lower the entry point for making some customizations, open up the pool of users that access the feature, and introduce some really neat (but low implementation cost) coolness factors that will ensure people don't think they need a new one, all the while trying not to shoot ourselves in the foot for the future

public interface Tool extends Runnable {
  void init(Editor editor);   // <--- straight shot in the foot
}

I'd rather skip a detailed listing of the reasons why in IMHO the situation today was predictable and focus and how to make things better. I think the current IDE is great, and there are plenty of low hanging fruits that can make it even better.

[matthijskooijman]

There is a Javascript interpreter in the IDE now? What is it used for?

I can see your point about lowering the barrier, using javascript would certainly accomplish that. One problem with using a different language is that you will have to provide a completely new API for everything, unless some kind of automatic wrapping method exists and works well (which is more work, but also means that API might be more thought-through).

Apart from the additional implementation effort on the java side, creating a good API for things like this is typically hard. Keeping things backward compatible is tricky, and it will probably fail if it is not documented properly.

Having said all that - it seems like a nice thing to have, I'm just a bit doubtful about the effort of getting there :-)

[lmihalkovic]

There is a Javascript interpreter in the IDE now? What is it used for?

Sorry for the lack of clarity, nashorn is the standard JS interpreter embedded in JSE8. It is a veeeery well written piece of code that has good performance compared to the venerable Mozilla Rhino. Java has had a scripting engine API for many years now, and even though Rhino supports it, must of the code out there usually targets the Rhino internals. The API is very simple, but enough for most use cases. The benefit of using it is that there are dozens of supported language runtimes for it (the API has support for pre-compiling the scripts if the engine can do that, which goes a long way for performance of the scripting code).

As you point out, API design is make-or-brake zone.. but I am realistically hopeful. As for implementation, depending on how ambitious the scope of the feature is, it is anywhere from a 1/2 day to a couple days of work. At the moment I am looking into code completion and embedded help, but I will write something as simple as my initial scope soon if no-one has done it by then.

[ffissore]

@lmihalkovic I still miss a goal, a use case, a problem which is solved by a scriptable editor. Can you stop talking about the implementation and talk about what's the problem being solved here? The fact that the editor is not scriptable is not a problem per se. Can you tell us a use case?

[ffissore]

Don't get me wrong, I don't want to limit your enthusiasm (which is awesome btw): but without motivations this proposal won't go anywhere

[lmihalkovic]

See #4084 for a more global view of where this fits and some of the additional use cases.

[ffissore]

#4084 only lists proposals, you're still now providing a goal, a use case and/or a problem which is solved by a scriptable editor

[lmihalkovic]

I swear it does contain an honest answer to your request. I am aware that it is in-between the lines, but it is there. By all means close this entry if you deem that it must be so. no problem.

[ffissore]

"user defined scripts for file validation, data visualization" it's the only thing I get: first one is unfeasable unless you have a c++ parser coded in javascript, second one is already done (see serial plotter)
Anything else?

[lmihalkovic]

btw... in the context of what I am describing, it is entirely possible to offer complete C++ parsing in JS without the step you seem to think is necessary

[ffissore]

Again, show me code, or it didn't happen http://memegenerator.net/instance/30625023

[ffissore]

Closing as wontfix. When a problem/need arises with a clear use case that a js plugin can solve, please open a new issue describing such use case

[lmihalkovic]

No pb. I think considering the direction IDE is going in, I think you are right to close this. Sorry I did not do it myself before.