\xa0 and \x85 are stripped from the ends of header values

[kenballus]

Given that these bytes are allowed in header values (due to obs-text), and are not considered whitespace by the standard, they shouldn't be stripped during header-field OWS stripping.

An example request that demonstrates the bug:

GET / HTTP/1.1\r\n
Test: \xa0\x85abc\x85\xa0\r\n
Host: a\r\n
\r\n

Waitress will see the Test header as having a value of abc, but the vast majority of other HTTP implementations will see it as having a value of \xa0\x85abc\x85\xa0.

[kenballus]

I have confirmed that this bug is present in the current main branch of Waitress:

$ git log -n 1
commit 4e0d8c4a951e9a274889609f90fbe31d5253fa82 (HEAD -> main, origin/main, origin/HEAD)
Author: Delta Regeer <bertjw@regeer.org>
Date:   Sun Feb 4 16:30:32 2024 -0700

    Prep 3.0.0

To verify this for yourself, run the following script:

import base64
from waitress import serve

RESERVED_HEADERS = ("CONTENT_LENGTH", "CONTENT_TYPE")


def app(environ, start_response) -> list[bytes]:
    response_body: bytes = (
        b'{"headers":['
        + b",".join(
            b'["'
            + base64.b64encode(k.encode("latin1")[len("HTTP_") if k not in RESERVED_HEADERS else 0 :])
            + b'","'
            + base64.b64encode(environ[k].encode("latin1"))
            + b'"]'
            for k in environ
            if k.startswith("HTTP_") or k in RESERVED_HEADERS
        )
        + b']}'
    )
    start_response(
        "200 OK", [("Content-type", "application/json"), ("Content-Length", f"{len(response_body)}")]
    )
    return [response_body]


if __name__ == "__main__":
    serve(app, listen="127.0.0.1:8088")

Then send it a request with \xa0 and \x85 on either side of a header value:

printf 'GET / HTTP/1.1\r\nTest: \xa0\x85a\xa0\x85\r\n\r\n' | timeout 1 nc localhost 8088

And observe its output:

HTTP/1.1 200 OK
Content-Length: 33
Content-Type: application/json
Date: Mon, 05 Feb 2024 00:46:59 GMT
Server: waitress

{"headers":[["VEVTVA==","YQ=="]]}

Base64-decoding YQ==, we see that the reported value is a, when it should be \xa0\x85a\xa0\x85.