gateway: harden path prefix

[ghost]

See #1971 (comment) ff.

Only allow paths, so we stay local to this gateway,
and don't get tricked into linking to a different host,
or even redirecting to one.

I can't come up with a concrete attack right off the bat,
but who knows how things are going to change.
Not restricting it would be silly.

Edit:

• make this a configurable set of allowed prefixes instead

[jbenet]

Maybe path.Clean? Not sure

[ghost]

Clean("http://foobar.com/") = "http:/foobar.com" :(

I think starts-with-slash should be good enough, and is also simpler to specify (as in api spec) than the specifics of a golang stdlib function.

[jbenet]

sounds good

[jbenet]

what about paths like:

//foo.com/bar/baz

valid http path that might point elsewhere

[jbenet]

doing this in a header is particularly dangerous because this could come from a client, so malicious request could cause links to be served back to a user that point to some evil place. maybe this should be a config? not sure what the right + secure way to do this is.

[ghost]

//foo.com/bar/baz

shit I didn't think about that :/

maybe this should be a config

A whitelist of allowed prefixes would work. Setting only one fixed prefix defeats the purpose, since the basic idea behind all this is to have the daemon exposed at multiple arbitrary URIs: ipfs.io/, ipfs.io/refs, etc.

[jbenet]

ah yeah.

[rht]

Here is a gateway yanked from the current dev0.4.0 (as of b4ab684).

[whyrusleeping]

@lgierth update here?

[ghost]

No update but let's keep it open so that I don't forget. It's important (see labels) but I keep postponing it...

[whyrusleeping]

@lgierth if you come to seattle i'm not going to let you leave until you merge this one :P

[ghost]

Updated to use a configurable allowlist of path prefixes:

gateway: enforce allowlist for path prefixes

The gateway accepts an X-Ipfs-Path-Prefix header,
and assumes that it is mounted in a reverse proxy
like nginx, at this path. Links in directory listings,
as well as trailing-slash redirects need to be rewritten
with that prefix in mind.

We don't want a potential attacker to be able to
pass in arbitrary path prefixes, which would end up
in redirects and directory listings, which is why
every prefix has to be explicitly allowed in the config.

Previously, we'd accept any X-Ipfs-Path-Prefix header.

Example:

We mount blog.ipfs.io (a dnslink page) at ipfs.io/blog.

nginx_ipfs.conf:

location /blog/ {
    rewrite "^/blog(/.*)$" $1 break;
    proxy_set_header Host blog.ipfs.io;
    proxy_set_header X-Ipfs-Gateway-Prefix /blog;
    proxy_pass http://127.0.0.1:8080;
}

.ipfs/config:

"Gateway": {
    "PathPrefixes": ["/blog"],
    // ...
},

dnslink:

> dig TXT _dnslink.blog.ipfs.io
dnslink=/ipfs/QmWcBjXPAEdhXDATV4ghUpkAonNBbiyFx1VmmHcQe9HEGd

[ghost]

Okay it does work as expected, I tested it the wrong manual way the other day.

1. set Gateway.PathPrefixes = ['/blog'] in the config
2. curl -v localhost:8080/static/ -H 'Host: blog.ipfs.io' -H 'X-Ipfs-Gateway-Prefix: /blog'
3. Expect links of the form /blog/static/... in the resulting directory listing
4. curl -v localhost:8080/static/ -H 'Host: blog.ipfs.io' -H 'X-Ipfs-Gateway-Prefix: //evil.com/blog'
5. Expect untouched links of the form /static/...

• fix unrelated tests

[ghost]

• fix unrelated tests

Fixed the tests -- filed #2528 about the CircleCI failure

[ghost]

Yay circleci is green

[whyrusleeping]

Looks good to me, nice to finally merge this :)