Reject invalid string literal whitespace on unescape

[jonmeow]

This is based on discussion on #732: that we should probably parse the invalid whitespace, then reject it as part of string validation, rather than having different parses. I worry the question of "how is this parsed" may lead to subtly unexpected results if we aren't consistent, so I'm switching the logic from the lexer to the unescape library (and also adjusting the list of rejected whitespace).

[zygoloid]

Sorry, I missed this in #732:

I think we do want to disallow newlines here; both the toolchain implementation and #199 do that (#199 allows "characters other [...] vertical whitespace"). This is important in making """ string literals work: we want

var x: String = """
""";

to unambiguously be a block string literal, not three simple string literals "", "\n", "".

To match #199, we should disallow \v\f\r here too.

[jonmeow]

Done, added a small change to the design phrasing.

Note, if we don't have implicit string concatenation (which I think was the plan?) this is unambiguous.

[zygoloid]

Yes, it's unambiguous at the parsing level either way, and at the lexing level, a max munch rule would do the right thing here (we'd lex the """\n""" token because it's longer). I think probably the best argument for the change is to improve the behavior when a " is accidentally missed from the end of a string literal.

[zygoloid]

Thanks!

[zygoloid]

Yes, it's unambiguous at the parsing level either way, and at the lexing level, a max munch rule would do the right thing here (we'd lex the """\n""" token because it's longer). I think probably the best argument for the change is to improve the behavior when a " is accidentally missed from the end of a string literal.