Snippet grammar fixes and minor formal improvements throughout the LSP spec

[falko17]

While reading through the specification, I've noticed a few style issues as well as some shortcomings in the EBNF grammar for snippets. This PR should fix them. See the list below for details on what was changed.

Detailed changes

• Most of the changes are various grammar, spelling, punctuation, formatting, or consistency improvements¹.
• Some incorrect TypeScript syntax in the examples has been fixed.
• The EBNF grammar for snippets has been improved in the following ways:
  • Incorrect grouping of alternatives in some parts has been fixed.
  • Ambiguities present in the grammar have been fixed, the grammar should now be LL(1)-parseable.
  • The grammar has been made consistent with the description above it. Specifically, characters which have to be escaped per the textual description now also need to be escaped per the grammar rules².
  • A description for the ':+', ':?', and ':-' modifiers has been added, as they seem to not be explained anywhere else in the spec.
    • I have used the VSCode source code as a basis for these.
  • The link has been updated to refer to the right EBNF specification, as the grammar uses the XML variant rather than the one described in the originally linked Wikipedia article.

Note

These changes have only been applied to the upcoming 3.18 LSP specification. If they should be applied to the current version 3.17, I could port the changes over as well.

Footnotes

1. Not all the changes necessarily fix errors, some just reword certain passages to sound more natural, which may be somewhat subjective. ↩

2. Technically, the text clarifying escaping rules could be removed, as the grammar already describes them now. Keeping the text for clarity's sake might still make sense, though. ↩

[falko17]

@microsoft-github-policy-service agree

[dbaeumer]

IMO we need to make choicetext as follows:

([^$},|\] | '\$' | '\}' |'\,' | '\|' | '\\')*

Any reason why you didn't

[falko17]

The reason why $ and } don't need to be escaped within a choice construct is that it cannot occur in the choice in as part of any other construct, so it is uniquely parseable. Only a ,—indicating the next choicetext—and a |}—indicating the end of the choice construct—can occur here as tokens with semantics attached. Anything else (including $ and }) can be ignored by the parser, and thus doesn't need to be escaped.

For example, ${1|close},$variable|} represents a choice construct with choices close} and $variable. These two choices don't have further meaning and are interpreted purely as strings. Contrast this with e.g. ${1|one,\,,two}, which is a choice construct with choices one, ,, and two. If the comma here were not escaped, it would be unclear whether the choice consists of two empty elements or a comma, hence the escape is necessary.

(Formally, $FIRST(\texttt{choicetext}) \cap FOLLOW(\texttt{choicetext}) = \emptyset$ must be true here¹. As $FOLLOW(\texttt{choicetext}) = \{'\texttt{|}',\ '\texttt{,}'\}$, it is only these two characters that need to be escaped, so that they are not part of $FIRST(\texttt{choicetext})$.)

Footnotes

1. FIRST/FIRST conflicts can occur too, but are only relevant insofar that the backslash needs to be escaped. ↩

[dbaeumer]

@falko17 thanks for the great explanation!

[dbaeumer]

@falko17 thanks you so much for the great PR. Highly appreciated. I have several comments about indentation. If you are too busy to fix them I am happy to do so. I have one additional comment about the changes you did to the snippet syntax I would like to get your comment on.

[dbaeumer]

See all my inline comments and my final one.

[falko17]

Thank you for the detailed review, @dbaeumer! I've addressed your comments, please tell me if there is anything else that should be done. Also, sorry about all the indentation problems, my editor displayed tabs as 2 spaces, which is why I didn't catch these. Should be all fixed now.