WIP Experiment: Lets see if we can finally get JSX working properly

[lildude]

This is an experiment I'm starting to see what it would take to finally get the JSX syntax highlighting issues sorted out.

For the most part we're held back by the fact the upstream grammar introduced changes that include oniguruma-only regexes and GitHub.com (and Lightshow) need PCRE-friendly grammars.

This PR pulls in the work I'm doing in a PR against my own fork at https://github.com/lildude/language-babel/pull/1 in which I document the changes I've made whilst at the same time trying to automate the changes and opens up the dialogue channel where. If things are simple enough, we might be able to convince the upstream author to accept a PR to switch to PCRE-friendly regexes, but if not, the automation should allow me to add the script to our fork so we can update automatically with each update. It'd be a PITA, but we'll see.

@pchaigno @Alhadis I'd really appreciate your comments, thoughts and findings. I'm not a grammar expert so am flying by the seat of my 👖 for the most part 😄 . I'll slowly work on my fork as and when I spot issues that cause us problems. Feel free to comment in this PR or https://github.com/lildude/language-babel/pull/1

/ref #3044 #3775

[pchaigno]

I'll have a look.

I'm not a grammar expert so am flying by the seat of my 👖 for the most part 😄

I learned a new expression 😄 Sounds Bristish (?)

[Alhadis]

Well. This ought to be interesting. 😀

I'll have a look sometime tomorrow when I'm not so groggy.

[lildude]

I learned a new expression 😄 Sounds Bristish (?)

I think it has a 🇺🇸 origin

[pchaigno]

"Flying by the seat of his trousers," for the Bristish version then 😜

[Alhadis]

@lildude Do you know of any PCRE-based TMCG implementation I could use to test these changes? Literally every program which supports TextMate grammars uses the Oniguruma engine, and testing with Lightshow is very cumbersome (as well as limited in the feedback it offers).

[lildude]

@lildude Do you know of any PCRE-based TMCG implementation I could use to test these changes?

Nope. 😞

[Alhadis]

@lildude Lately I've been tossing around the idea of simply rewriting the JSX/JS grammar from scratch myself. Literally all that kept me from doing so earlier was the fact that I'd never worked in JSX or Flow before. That changed last December/January, and I noticed it wasn't hard at all (it's a wank, but it's not hard).

What do you think? Bear in mind that this would involve writing the JS grammar to ensure correct precedence of expression-groups (and to allow interpolation in JSX tags). So it'd then make sense to use it for the JS grammar as well... 😅

[lildude]

What do you think? Bear in mind that this would involve writing the JS grammar to ensure correct precedence of expression-groups (and to allow interpolation in JSX tags). So it'd then make sense to use it for the JS grammar as well... 😅

If you're up for it and happy to keep things up-to-date as the language evolves, feel free. I won't hold you back 😄

[Alhadis]

Right. I'm going to enjoy this.

[lildude]

Right. I'm going to enjoy this.

Want to start a PR for this?

[Alhadis]

There's not much point in doing so until I've finished :wink

[lildude]

There's not much point in doing so until I've finished :wink

Oooops. Didn't realise it was still WIP. 😊 ✨

[Alhadis]

It'll take some time, because I'm focussing on structuring this in a way that makes updating it possible without a mess (like Atom's current JS grammar). Given the pace at which the JavaScript language is evolving, being wise about structure is important.

There's already an improvement in highlighting multiple import statements, however:

• Before
• After

And yes, before the JSX part of this can happen, the regular JS grammar has to come first. Don't worry, it'll happen. 😉

[lildude]

Closing this in favour of the work @Alhadis is doing to "fix" the grammar. We'll circle back to this once the grammar is ready for prime time.

[Alhadis]

I won't let you guys down. :wink

(Bit busy with life and finishing other things at the moment, but rest assured it's top priority).

[pchaigno]

@Alhadis Did you get a chance to make some progress on this? I'm considering forking https://github.com/babel/babel-sublime to change the scope and use that in the meantime.

[Alhadis]

No, sorry. I haven't been using Atom lately so it's been hard for me to work on any grammar-related projects. 😢

Go right ahead. I'll get to this sooner or later, but I currently can't make any guarantees when.

[Alhadis]

Guys, I might use this as an opportunity to start work on Synapse. The complexity of this grammar will render it utterly opaque to any reader except myself. Future maintenance will become impossible without additional tooling.

Synapse will be a compiler for a domain-specific markup languages specifically dedicated to syntax highlighting. It'll be based off CSON, but with cleaner syntax and support for variables and macros to eliminate the excessive repetition and nesting that's often necessary for TextMate grammars.