CUE

[myitcv]

Is there an existing issue for this?

• I have searched the existing issues

What is the missing lexer?

CUE (https://cuelang.org/) is an open source configuration, policy, validation, data and schema language. It has a rich set of APIs and tooling, for defining, generating, and validating all kinds of data: configuration, APIs, database schemas, code, etc.

Links to existing syntax definitions

We are working on various syntax definitions based on the language spec:

https://cuelang.org/docs/references/

The most complete representations to date are:

• https://github.com/eonpatapon/tree-sitter-cue
• https://github.com/monogon-dev/intellij-cue/blob/main/src/grammar/cue.bnf

TextMate grammars will follow.

[myitcv]

I would like to suggest that for now we iterate on the initial CUE lexer via pull requests into https://github.com/myitcvforks/chroma. That will allow for testing, iteration etc without creating too much noise here.

As you can see from the tip of master in that repo, I have made a start.

@alecthomas how does that sound?

If that works, please feel free to assign this issue to me in case that helps with your workflow/project management. I'm a full time contributor to the CUE project and will help to coordinate here in whatever way helps make things easy for you and the chroma project.

I also wonder whether you could help advise how best to work with a lexer that is not yet complete. Is there a way of writing an XML-based lexer such that it relaxes in case it encounters a syntactic element it doesn't know about, rather than emitting errors?

[alecthomas]

That's sounds fine to me.

I also wonder whether you could help advise how best to work with a lexer that is not yet complete. Is there a way of writing an XML-based lexer such that it relaxes in case it encounters a syntactic element it doesn't know about, rather than emitting errors?

I think just having a fallback pattern like this should do it:

      <rule pattern=".">
        <token type="Text"/>
      </rule>

[myitcv]

That's sounds fine to me.

Great. I actually bit the bullet and tried to sketch out a pretty decent first cut, which is now pushed up in #858.

I also wonder whether you could help advise how best to work with a lexer that is not yet complete. Is there a way of writing an XML-based lexer such that it relaxes in case it encounters a syntactic element it doesn't know about, rather than emitting errors?

I think just having a fallback pattern like this should do it:

      <rule pattern=".">
        <token type="Text"/>
      </rule>

Thanks, appreciate the pointer. I also learned a great deal from looking at the other lexers.