Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unicode in whitespace #135

Closed
moorereason opened this issue Jan 8, 2022 · 2 comments
Closed

Unicode in whitespace #135

moorereason opened this issue Jan 8, 2022 · 2 comments
Assignees
Labels
bug Something isn't working implemented in v3 Fixes + features which were implemented in v3 release. TOML spec An issue relating to the library's TOML spec conformance.

Comments

@moorereason
Copy link

The non-conforming TOML snippet

I'm using escape sequences to represent the Unicode code points of non-ASCII chars:

0=0
\u2000\u2000
1=1\u2000\u2000
2=2

As a hexdump:

$ echo "0=0\n\u2000\u2000\n1=1\u2000\u2000\n2=2" | hexdump -C
00000000  30 3d 30 0a e2 80 80 e2  80 80 0a 31 3d 31 e2 80  |0=0........1=1..|
00000010  80 e2 80 80 0a 32 3d 32  0a                       |.....2=2.|
00000019

What you expected

I expected an error because \u2000 is not valid TOML whitespace.

What you got

$ echo "0=0\n\u2000\u2000\n1=1\u2000\u2000\n2=2" | tt_decoder
{"0":{"type":"integer","value":"0"},"1":{"type":"integer","value":"1"},"2":{"type":"integer","value":"2"}}

Environment

toml++ version and/or commit hash:
v3 cdf85a9

Any other useful information:
The fuzzer triggered on \u2000 which is a general punctuation code point from what I can tell (I'm not a Unicode guru). I didn't try any other code points.

Found while doing differential fuzzing against go-toml. The fuzzer is merciless. 😄

@moorereason moorereason added bug Something isn't working TOML spec An issue relating to the library's TOML spec conformance. labels Jan 8, 2022
@marzer
Copy link
Owner

marzer commented Jan 8, 2022

Haha, nice, thanks.

The fuzzer is merciless.

This is great stuff, shows where all the testing gaps are (both in my tests and the spec ones)

marzer added a commit that referenced this issue Jan 9, 2022

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
also:
- added `parse_result::at_path()`
@marzer
Copy link
Owner

marzer commented Jan 9, 2022

Fixed in v3, thanks!

@marzer marzer closed this as completed Jan 9, 2022
@marzer marzer added the implemented in v3 Fixes + features which were implemented in v3 release. label Jan 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working implemented in v3 Fixes + features which were implemented in v3 release. TOML spec An issue relating to the library's TOML spec conformance.
Projects
None yet
Development

No branches or pull requests

2 participants