remove Heroku specific Req headers from being sent to Origin

[bulk88]

-saves bytes, and avoids triggering IDS/WAF alarms since browser finger
printing will prove these headers are unnatural and on SSL must be a MITM
attack

[coveralls]

Coverage remained the same at 100.0% when pulling 7271e29 on bulk88:no_heroku_headers_to_origin into 3bab870 on Rob--W:master.

[Rob--W]

Can you provide sources or proof that the presence of these request headers are causing issues?

I'm not against cleaning up some headers but as it may break client expectations, I need to see clear advantages to accepting this PR.

[Rob--W]

I have no objection against the removal of the listed headers, except maybe for x-forwarded-* because it is a standard non-Heroku specific header, and people may be relying on its existence.

[bulk88]

Specifically I am dealing with Cloudflare but you dont need to know that for this PR.

No cyber security company will publish their rules for preventing CSRF/PCI/DDOS attacks. A CORS proxy is fundamentally is a security breaking tool.

I've never seen a web server that sends "403 forbidden" if the Origin header is set in a request GET, for what was supposed to be a same-origin XHR or <form> POST or <img> element, but sooner or later in the future it will be best practices to, for a web server to check that Referrer or Origin must be missing, string "null", or same-origin URLs or return 403/Forbidden and never let the XSS request reach the DB/backend. I in this PR would like to strip the heroku-specific headers atleast. I think Origin/Referrer should stay the same as a last measure debug/defense to the orgin server's admin logs and should reveal the domain that is doing an anti-CORS CORS fetch. Heroku specific or all unknown headers could be triggered to block the request by generic "layer 7" security software, but values of "known always-present browser header fields" will be passed through to the backend app specific code unmolested regardless of the header value's exact byte string contents, example checking if Origin/Referrer has same TLD+1 as API server's TLD+1.

Here is an example of extra headers current on the public instance.

await fetch("https://cors-anywhere.herokuapp.com/http://scooterlabs.com/echo?argname=argval");

results in

Simple webservice echo test: make a request to this endpoint to return the HTTP request parameters and headers. Results available in plain text, JSON, or XML formats. See http://www.cantoni.org/2012/01/08/simple-webservice-echo-test for more details, or https://github.com/bcantoni/echotest for source code.

Array
(
    [method] => GET
    [headers] => Array
        (
            [Host] => scooterlabs.com
            [Connection] => close
            [User-Agent] => Mozilla/5.0 (Windows NT 6.1; rv:80.0) Gecko/20100101 Firefox/80.0
            [Accept] => */*
            [Accept-Language] => en-US,en;q=0.5
            [Accept-Encoding] => gzip, deflate, br
            [Referer] => http://example.com/
            [Origin] => http://example.com
            [X-Request-Id] => 93366f3c-a5b8-46fa-9249-bec762fce9da
            [X-Forwarded-For] => 74.64.17.23
            [X-Forwarded-Proto] => https
            [X-Forwarded-Port] => 443
            [Via] => 1.1 vegur
            [Connect-Time] => 0
            [Total-Route-Time] => 0
        )

    [request] => Array
        (
            [argname] => argval
        )

    [client_ip] => 74.64.17.23
    [time_utc] => 2020-09-26T17:34:10+0000
    [info] => Echo service from Scooterlabs (http://www.scooterlabs.com)
)

It is funny that that HTTP debug tool "[client_ip] => 74.64.17.23" published my Charter/Spectrum IP address from X-Forwarded-For header even tho cors-anywhere.herokuapp.com is an Amazon IP address on outgoing requests.

For comparison, my Firefox sent

GET /http://scooterlabs.com/echo?argname=argval HTTP/1.1
Host: cors-anywhere.herokuapp.com
User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:80.0) Gecko/20100101 Firefox/80.0
Accept: */*
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Referer: http://example.com/
Origin: http://example.com
Connection: keep-alive

So all webservers see those extra headers. I would agree on the public instance X-Forwarded-* should be left inplace in case an admin is trying to block abuse from cors-anywhere.herokuapp.com but X-Request-Id, Via, Connect-Time, Total-Route-Time need to be removed.

https://blog.cloudflare.com/moving-from-recaptcha-to-hcaptcha/

The second most popular is our IP Firewall. This is similar to the Firewall and Bot Rules, but much less granular at the IP, ASN, or country level. The majority of CAPTCHAs for this category are done for rules written at the ASN or country level. Presumably our customers find some level of distrust with a particular ASN (for example, why would there be supposed user traffic coming from a cloud infrastructure provider?) or are being attacked from specific countries.

Since I bet 100% of cors-anywhere instances are running on "cloud infrastructure provider" ASNs/IP addrs, and not on residential or mobile ISP ASNs or not-ISP corporate ASNs, cors-anywhere is very likely to be caught by security software as high risk user/IP address. Minimizing the amount of data that exists to the origin webserver to trigger a block rule is the best idea.

On a private CORS proxy I wrote, I do set Origin/Referrer inside the proxy, to same domain as destination URL

var url = new URL(desturl);
h['Origin'] = url.origin;
h['Referer'] = url.origin;
h['Host'] = url.host;

so it is impossible for the destination webserver to know a CORS proxy connected to it, but that is too evil to put on your public instance as a default.

Cloudflare will always will catch ANY cors-anywhere instance because of TLS Client Hello fingerprinting,. They even catch same machine/same IP wget/curl with byte-wise identical HTTP headers to Chrome. Node/Curl/Wget always requests the entire multi KB long X509 TLS certificate in their Client Hello, that triggers Cloudflare that "its a bot, real browsers save X509 certificates forever in cache and NEVER ask for them again until cert expires", so cors-anywhere (Node.js process SSL stack) gets shown the CAPTCHA screen instantly. Anyways, heroku specific headers shouldn't be going to final webserver.

[Rob--W]

So all webservers see those extra headers. I would agree on the public instance X-Forwarded-* should be left inplace in case an admin is trying to block abuse from cors-anywhere.herokuapp.com but X-Request-Id, Via, Connect-Time, Total-Route-Time need to be removed.

Agreed. If you update the PR to only remove Heroku-specific headers instead of the more generic xfwd headers, then I'm going to merge it.

A CORS proxy is fundamentally is a security breaking tool.

It's not. This project only offers access to resources that are available without authentication. A non-web app (e.g. a Python script) could request the same data without a proxy. The only case where a CORS proxy could introduce a new security risk is when it is placed in a network where other nodes in the network have a misguided belief that every request is confidential (e.g. an open wiki within an intranet). But this is not an issue with this specific project, as it applies to any open proxy.

Cloudflare will always will catch ANY cors-anywhere instance because of TLS Client Hello fingerprinting,. They even catch same machine/same IP wget/curl with byte-wise identical HTTP headers to Chrome. Node/Curl/Wget always requests the entire multi KB long X509 TLS certificate in their Client Hello, that triggers Cloudflare that "its a bot, real browsers save X509 certificates forever in cache and NEVER ask for them again until cert expires", so cors-anywhere (Node.js process SSL stack) gets shown the CAPTCHA screen instantly.

Interesting. Note that even if this project somehow offered perfect cloaking, then it would not be able to pass the challenge because this proxy intentionally strips cookies in both directions.

[bulk88]

Agreed. If you update the PR to only remove Heroku-specific headers instead of the more generic xfwd headers, then I'm going to merge it.

Repushed, I left a comment that the headers exist for private instances and its upto private instance admin to uncomment them if they want.

Interesting. Note that even if this project somehow offered perfect cloaking, then it would not be able to pass the challenge because this proxy intentionally strips cookies in both directions.

Correct. Cors-anywhere disables 2 ways cookies. Without the __cfruid/__cfduid/cf_clearance cookies a CF captcha cant be solved. And obviously the client side UI JS browser side must detect the 403 forbidden captcha, rewrite URLs in the HTML to point to the heroku proxy domain, draw the HTML to the user, user answers the captcha, client UI continues in non-interactive mode doing AJAX calls to the cors proxy, but with all future requests passing in the 3 clouldflare cookies.

In my private instance I turned on 2 way cookies along with cookie domain rewriting, but it is incredibly insecure unless the proxy is permanently locked to fetching from a single non-CORS origin content domain, since all content domain servers would see the cookies of all other content domain servers ever visited by a particular user/browser instance and cors proxy instance combo,

This is fantasy design by now, prefixing the content domain to HTTP only cookies names when passing response to client (content server->client), and stripping the content domain prefix off of the cookie name on client->content server and dropping wrong content server domain cookies from req to content server. But design breaks down, if cors-anywhere is used a navigation/browsing (no Origin Header) proxy (yet another hack), not-HTTP-only cookies must have their names intact in case JS wants to manipulate them through document.cookies and cant have proprietary prefixes on the cookie names.

[Rob--W]

Thanks.

[Rob--W]

Deployed.