Synapse responds with a 404 with HTML content for unknown URL prefixes

[clokep]

See element-hq/element-web#19738 (comment) for some background, but if you request an endpoint for Synapse that doesn't have a prefix registered (see here) then Synapse responds with a 404 including HTML content (the default Twisted 404 handler).

This wasn't an issue when adding new /unstable or /r0 endpoints since the prefix was registered, but the /hierarchy endpoint is the first /v1 endpoint we had. This will also bite us once we need to rev an endpoint to /v2 (or /v4) since neither of those are registered right now.

[t3chguy]

then Synapse responds with a 404 including HTML content (the default Twisted 404 handler).

and without an ACAO header so browser-based clients can't even read that its a 404 as the browser just gives them a generic failure due to CORS.

[clokep]

This was reverted due to poor interaction with a legacy feature.

[clokep]

Cross-ref to a similar Dendrite issue: matrix-org/dendrite#2739.