Tighten the parsing logic of chunked encoding #78
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
According to RFC 9112, a chunk size consists of a series of hexadecimal characters, which may be optionally followed by whitespace and extensions beginning with a semicolon.
Picohttpparser's parsing logic has been more permissive. It recognizes a sequence of hexadecimal characters as the size, disregarding any subsequent characters. For example, although "123x\r\n" does not conform to RFC 9112, picohttpparser has interpreted this line as indicating a chunk size of 291 (0x123) bytes.
Recently, concerns have surfaced about this lenient behavior. Specifically, there is a risk that if a client, acting as a proxy, forwards a malformed chunk size which is interpreted differently by that client, it could enable splitting attacks.
To mitigate the concern, this PR tightens the parsing logic. Now, the first non-hexadecimal character must be a whitespace, a semicolon, or the end of line. While it is the responsibility of each endpoint to send HTTP messages in accordance with the specifications, these changes ensure a more robust handling of potential deviations.
We express our gratitude to Ben Kallus and Keran Mu for independently bringing this issue to our attention.