Add UL4 syntax highlighting

[doerwalter]

• I'm the package's author and/or maintainer.
• I have have read [the docs][1].
• I have tagged a release with a [semver][2] version number.
• My package repo has a description and a README describing what it's for and how to use it.
• My package doesn't add context menu entries. *
• My package doesn't add key bindings. **
• Any commands are available via the command palette.
• Preferences and keybindings (if any) are listed in the menu and the command palette, and open in split view.
• If my package is a syntax it doesn't also add a color scheme. ***
• If my package is a syntax it is named after the language it supports (without suffixes like "syntax" or "highlighting").
• I use [.gitattributes][3] to exclude files from the package: images, test files, sublime-project/workspace.

[braver]

@deathaxe could you maybe have a look at this? You have more experience with extending syntaxes for things like template languages. @doerwalter takes a different approach than Liquid or Twig. I'm not sure of the benefits or downsides of either approach.

[packagecontrol-bot]

Automated testing result: WARNING

Repo link: UL4
Results help

Packages added:
  - UL4

Processing package "UL4"
  - WARNING: '.sublime-syntax' support has been added in build 3092 and there is no '.tmLanguage' fallback file
    - File: JavaScript-UL4.sublime-syntax
  - WARNING: '.sublime-syntax' support has been added in build 3092 and there is no '.tmLanguage' fallback file
    - File: UL4.sublime-syntax
  - WARNING: '.sublime-syntax' support has been added in build 3092 and there is no '.tmLanguage' fallback file
    - File: CSS-UL4.sublime-syntax
  - WARNING: '.sublime-syntax' support has been added in build 3092 and there is no '.tmLanguage' fallback file
    - File: HTML-UL4.sublime-syntax
  - WARNING: '.sublime-syntax' support has been added in build 3092 and there is no '.tmLanguage' fallback file
    - File: XML-UL4.sublime-syntax
  - WARNING: The package does not contain a top-level LICENSE file. A license helps users to contribute to the package.

[braver]

@doerwalter you can ignore those reports from the bot. A fallback for < 3092 is not required. A license is highly recommended however, MIT is a nice and simple one for no-strings-attached open source work.

[doerwalter]

OK, I've created a new version 1.0.2 that includes a LICENSE file (and also fixes a small bug).

What do I have to do to get my package rechecked?

[deathaxe]

Creating templating syntaxes via push ... with_prototype often resulted in 25.000 contexts sanity limits being hit, so I'd try to avoid it in general. UL4.sublime-syntax however is simple enough to not cause it. Syntax cache of HTML-UL4 is about 7MB. That's also the size of PHP, so it's ok.

It would also be the only valid implementation in case the syntax really targets ST3.

sublime-syntax version: 2 format however requires ST4.

So either remove version: 2 or specify "sublime_text": ">=4107" in repository.

The approach taken for Liquid, PHP, ... to extend HTML/CSS/JS/XML would probably end in less memory usage, would be more save, allow more control about scopes (e.g.: remove string scope in emedded strings), but is more complicated to implement and requires some boilerplate code. It causes some more dependencies with regards to supported ST builds due to base syntaxes having changed over time. Would however be more or less a "copy and paste" task to apply it to UL4.

Remark: I'd prefer file names to follow common naming scheme of extended syntaxes (<Base Syntax> (UL4).sublime-syntax) used by majority of packages, but that's just of stylistic meaning.

[doerwalter]

I've implemented the embedded version of UL4 with the method that https://www.sublimetext.com/docs/syntax.html#including-other-files seems to suggest for templating languages with the Jinja example.

I'm not 100% sure, what "sublime_text": ">=4107" does. Will this enforce a version of SublimeText 4 that has reduced memory usage for syntax highlighting files?

I've now renamed the syntax definitions for the embedded grammars like this:

• CSS-UL4.sublime-syntax -> CSS (UL4).sublime-syntax
• HTML-UL4.sublime-syntax -> HTML (UL4).sublime-syntax
• JavaScript-UL4.sublime-syntax -> JavaScript (UL4).sublime-syntax
• XML-UL4.sublime-syntax -> XML (UL4).sublime-syntax

but I haven't created a new version yet.

[deathaxe]

Yeah, the docs still target ST3, mainly with that regard. It's not wrong. It is just known to have some probably unwanted side effects in certain edge cases.

Syntax files specify version: 2, which slightly changes behavior of syntax rules in backwared-incompatible ways, especially with regards to treatment of meta_content_scope and requires Sublime Text 4. You probably won't realize it due to absence of syntax tests or in best case current syntax is not effected.

The tag is ignored by ST3, but I wouldn't start mixing things up to avoid confusion.

Hence the suggestion to either remove it or restrict installation to ST4 via "sublime_text": ">=4107" entry in repository. Package Control will offer the package only for ST4107+ then.

To be very perfect add "sublime_text": ">=3092" as this is the first build, which supports sublime-syntax format at all, but I doubt someone still using such old build.

[doerwalter]

OK, I can include the "sublime_text": ">=4107", but I'm not sure into which file that goes. This doesn't look like YAML syntax so probably not into UL4.sublime-syntax.

[doerwalter]

OK, I've committed your suggestion. Thanks!

[packagecontrol-bot]

Automated testing result: SUCCESS

Repo link: UL4

Packages added:
  - UL4

Processing package "UL4"
  - All checks passed

[braver]

Thanks guys. Learn something new every day! :)

[doerwalter]

The latest version of Sublime Text (Build 4189) has started to complain about the embedded UL4 syntaxes like this:

Error loading syntax file "Packages/User/LivingLogic.SublimeText.UL4/HTML (UL4).sublime-syntax": Apparent recursion within a with_prototype action: 25000 context sanity limit hit

I guess I have to change the way the embedding is implemented. https://www.sublimetext.com/docs/syntax.html#including-other-files still seems to suggest that using with_prototype is the canonical approach for embedded syntaxes.

Where can I find documentation about the correct way? Do I have to dig through the PHP syntax to find out how it's done?

[deathaxe]

see: sublimehq/sublime_text#6555 (comment)

Examples to start with are packages like Jinja2, Liquid, Svelte, Vue.

Basically extend each involved syntax and add custom UL4 rules via prototype: context and maybe some special string related ones like string-content (they differ a bit accross syntaxes).

Then extend HTML to replace Js/Css inclusions with your customized ones. That's mostly just copy past boilerplate code. Additionally add your custom syntax to HTML as you did for the other syntaxes before.

This looks a bit more complicated and verbose due to needed boilerplate code, but gives way more fine gained control about how and where custom syntax is to be added.