Additional Options for Quoted Strings

[MattEdwardsWaggleBee]

In my use case, I wanted to add backtick to quote strings. Looking at the Scanner it seems straightforward to add more types. If this is something you think could be useful to others I can start a PR.

Not sure what your prefer approach would be, but my initial thoughts are:

1. Add more StringLiteralQuotes enums and follow the pattern for single and double quotes.
2. A refactor so users can set any character they want to use for quoting.
3. Make scanner's private ReadQuotedString method public and people can make their own parsers

As a workaround I hacked option 3 by creating a BacktickLiteral, that uses UnsafeAccessor to access the private method.

public class BacktickLiteral : Parser<TextSpan>
{
    public override bool Parse( ParseContext context, ref ParseResult<TextSpan> result )
    {
        context.EnterParser( this );

        var start = context.Scanner.Cursor.Offset;

        var success = UnsafeScannerHelper.ReadQuotedString( context.Scanner, '`' );

        var end = context.Scanner.Cursor.Offset;

        if ( success )
        {
            // Remove quotes
            var decoded = Character.DecodeString( new TextSpan( context.Scanner.Buffer, start + 1, end - start - 2 ) );

            result.Set( start, end, decoded );
        }

        context.ExitParser( this );
        return success;
    }

    public class UnsafeScannerHelper
    {
        public static bool ReadQuotedString( Scanner scanner, char quoteChar )
        {
            return ReadQuotedString( scanner, quoteChar, out _ );

            // Parlot has support for quoting strings with other characters
            [UnsafeAccessor( UnsafeAccessorKind.Method, Name = nameof( Scanner.ReadQuotedString ) )]
            extern static bool ReadQuotedString( Scanner scanner, char quoteChar, out ReadOnlySpan<char> result );
        }
    }
}

[sebastienros]

Feel free to create a PR with your suggestions, this seems sensible. Do you have known languages in mind that use it while keeping the existing \ escape sequence?

[MattEdwardsWaggleBee]

Interesting point about the \ escape sequence. I commonly see it in scripting for template literals, markdown, or raw strings; many of those usages would ignore escaping. Depending on what developers are doing, they might want a more robust parser. In my usage, I wanted to keep the string as-is since I was going to forward it to string.Format(). I'll look into a PR in the next couple of weeks and let you see if it feels right. At a minimum making ReadQuotedString public might be enough for most people.

[MattEdwardsWaggleBee]

Support added in #202