docs(ADDRESSING): default DNSLink to ipfs://

[lidel]

This PR proposes that ipfs://{dnslink-fqdn} becomes the default URL representation of DNSLink websites in web browsers

Motivation

Interesting quirk of where we are right now:
DNSLink resides under /ipns/{fqdn}, but it is not related to IPNS in any way.

I was thinking about the future, when we have protocol handlers in web browsers and what will be shown in address bar.

Then I realized 1:1 mapping will show all DNSLink websites under ipns://{fqdn} 😱

Proposed change

To avoid the future of ipns:// we need to support DNSlink websites under ipfs://{fqdn}, and make ipns://{fqdn} a redirect to ipfs://{fqdn}.

It is easy to implement:

• IF valid {cid}, load it
• ELSE IF DNSLink record exists for {fqdn}, resolve it and load the content
• ELSE return "invalid address" error

This will solve a set of problems:

• people being confused why "IPFS websites" are on ipns:// instead of ipfs://
• future-proofing: in the long run, we will keep supporting /ipns/{fqdn}, but will probably end up switching DNSLink to its own namespace at /dnslink/{fqdn} (Feature request: add link type /dns/<domain name> notes#348 (comment)). Detaching default from ipns:// now, saves us headache later
• removing the problem of cross-protocol boundary on DNSLink websites (Protocol Handler: Cross-protocol Content mozilla/libdweb#52)
  • tl;dr this is a big security issue, as cross-protocol rules are undefined and vendors default to blocking such requests ("better safe than sorry")
  • DNSLink website could be backed by any IPNS content path in DNS TXT record which enables cross-protocol on the IPFS side, without mixing protocol handlers in web browser

What about ipns:// ?

It would remain and work for real IPNS paths. We would still support DNSLink on it, but as a redirect to ipfs:// URL. I imagine loading websites via "raw IPNS" (ipns://{libp2p-key}) will be pretty rare event and similar to loading website via raw IP (bad UX, but works without DNS).

cc #147

[npfoss]

Why is ipfs:// any better than ipns://? Neither one should have anything to do with DNS, ideally, right? To me, the ipfs prefix means static content. If anything, the one with "Name" in its name should be associated with human readable names, but I agree with all of the problems you stated about doing that.

I don't really know enough to confidently offer solutions, but maybe it could be dweb:// followed by a multiaddr which supports /ipfs/ or /ipns/ or /dns/ (or whatever /dns/ is going to be, /dnslink/ maybe--I don't remember).

[autonome]

• +1 on overall simplification of the address surface, and also not requiring a custom scheme for naming. We are now more aligned with http, dat in this regard.

• The implementation flow as you've written it above looks like a switch without a default case handler. Maybe for a different issue, but it would be good to define explicit handling of unresolvable strings not just here, but throughout the addressing spec.

• Let's get broader input from people working on IPNS like @aschmahmann and @hugomrdias before landing this change.

[Stebalien]

I'm concerned this will cause more problems than it will solve:

• We're unlikely to support /ipfs/foo.com (immutability is a requirement).
• Supporting ipfs://foo.com but not /ipfs/foo.com is really confusing.

I'm also concerned this doesn't fix the issue. What does this mean for, e.g., javascript-backed requests from ipfs://? Those are cross-origin? If we can't make HTTP(s) requests, we're going to have a hard time getting adoption.

---

If push comes to shove, we could go with something like:

dweb://thing.{ipfs,ipns,eth,dns,etc.}

[aschmahmann]

@lidel I'm a little confused (sorry for the long response), can you help me understand what are the main issues this helps with?

TLDR:

1. Doesn't DNSLink need to point at arbitrary paths like IPNS, DNS, ENS, ..?
2. Can't we use dweb:/?
3. It seems like we need to tackle cross-protocol contexts anyhow.

DNSLink can point at an arbitrary path. As far as I can tell the currently supported path types are /dnslink/X, /ipfs/bafyX, /ipns/bafyX, but it can theoretically support arbitrary paths (/ens, /stack, etc.)

Is the idea that since currently DNSLink websites (the content ultimately resolved by /dnslink/{fqdn}) are more likely to have HTML/JS that loads content from ipfs:// than from ipns://, and that we'll be helping with the cross-protocol boundary issue in this case? If so then wouldn't we be better off with both IPNS and IPFS having DNSLink (since then ipns://{fqdn} could point to an IPFS object that has links to ipns://{fqdn | bafyX} that resolve?

The point that is starting to crystallize for me here is that given that DNSLink (and IPNS) both take input => path and that path can involve arbitrary protocols we're really just asking for cross-protocol communication.

Figuring out the details here is hard, and I imagine asking for dnslink:// in addition to ipfs:// and ipns:// is pretty annoying. However, it was my understanding that the solution to this is to use dweb:/ as @Stebalien mentioned above. It's worth noting that dweb:/ seems to totally sidestep the cross-protocol issues in a potentially heavy handed way unless we can utilize our own type of secure context to define things like dweb:/ipns at root allows for resolving internal mutable links where as dweb:/ipfs at root only allows for resolving internal immutable links.

[lidel]

Thanks for joining the discussion!
Made a quick pass at addressing key concerns:

Main reason is to improve UX/DX of IPFS addressing in web browsers

I want to be clear about the motivation for this change.
Proposed change gives us clear, self-describing URLs: ipfs://tr.wikipedia-on-ipfs.org

It simplifies things by moving the main use case in browser contexts (DNSLink) to the main protocol handler, removing cognitive overhead when dealing with "IPFS websites".

In the browser, the idea is that people don't care about our protocol handler extravaganza and just want to use intuitive ipfs://{dnslink}. If we make it possible, then serious complexity around cross-protocol requests disappears as a side effect.

DNSLink can still point at arbitrary content paths such as /ipns/{libp2p-key} or /ipfs/{cid}, but concepts and differences between "ipfs", "ipns", "dnslink"... all this complexity is hidden from end user, which I believe is an overall UX/DX improvement for DNSLink-based websites (and follows what dat:// already does).

dweb:/{ipfs,ipns,dnslink,etc} does not provide Origin isolation

@npfoss @Stebalien @aschmahmann : URIs like this are already listed in existing spec, but won't be useful as canonical addresses in browsers any time soon due to the lack of proper Origin isolation. Making this work requires browser vendors to change some security-sensitive code paths, which is not feasible at this stage.

Cross-protocol requests

Note that cross-protocol in IPFS ecosystem means different things than cross-protocol in browser context. Want to make sure we discuss the latter.

From the browser perspective, cross-protocol thing happening behind DNSLink is internal to IPFS. Browser asks IPFS for bytes to render, and gets them. The only thing browser cares about now is how to handle requests made by a rendered page to Origins under different protocol (Origin = protocol+hostname+port).

What this PR does is to simplify protocol handler for the most popular use case (DNSLink): it removes the question of how to handle requests for ipfs:// on ipns:// and vice versa, which may be important if we end up with cross-protocol requests being blocked by default by vendors, at least initially. This makes DNSLink websites over ipfs:// useful in more strict contexts.

Cross-protocol requests, especially to http(s):// are tricky subject.
See libdweb discussion on this at mozilla/libdweb#52

What to do with ipns://

It would remain. We would still support DNSLink on it. I imagine loading websites via "raw IPNS" (ipns://{libp2p-key}) will be pretty rare event and similar to loading website via raw IP (bad UX, but works without DNS).

Other concerns

@Stebalien:
I'm concerned this will cause more problems than it will solve:

• We're unlikely to support /ipfs/foo.com (immutability is a requirement).

Correct, we won't. Proposed change is limited to interop URLs (ipfs:// and ipns://) in contexts such as web browsers.

• Supporting ipfs://foo.com but not /ipfs/foo.com is really confusing.

Yes, but when/why would browser user make that conversion?
How often anyone would do that realistically?
I'd argue "ipfs website" having ipns:// instead of ipfs:// is far more confusing, as it is touching mainstream users who do not care about URLs & paths.

@aschmahmann:

1. Doesn't DNSLink need to point at arbitrary paths like IPNS, DNS, ENS, ..?

Yes. That is the point. In web browser DNSLink hides it all behind human-readable DNS name.
When user loads ipfs://example.com they just want the website. It should not matter DNSLink at example.com pointed at other DNSLink at foo.com, which pointed at /ipns/{libp2p-key}, which then pointed at /ipfs/{cid}.
See Main reason above.

2. Can't we use dweb:/?

Sadly no: it does not provide valid Origin separation. See About dweb: above.

I imagine asking for dnslink:// in addition to ipfs:// and ipns:// is pretty annoying.

Yes, we already struggle with ipfs:// and ipns://.

---

Let me know if I missed the target anywhere.

[Stebalien]

dweb:/{ipfs,ipns,dnslink,etc} does not provide Origin isolation

My suggestion was dweb://foo.{ipfs,ipns,eth,...}. Basically, just pick some generic protocol for all dweb/p2p protocols, create a generic extension that handles them, and then demultiplex them later in the addon.

[lidel]

@Stebalien I see, yes, that would keep hash and protocol name in "Origin" segment, producing URLs like:

dweb://{cid}.ipfs/
dweb://{libp2p-key}.ipns/
dweb://{dnslink-fqdn}.ipns/
dweb://{enslink-fqdn}.ens/
dweb://{hash}.dat/
dweb://{hash}.ssb/

While it "solves" cross-protocol and Origin problems, it does it at a cost (ux/performance):

• DNSLink websites hide the fact of using IPFS even further ('ipns', 'dweb', but no 'ipfs')
  • Allowing DNSLink in .ipfs namespace dweb://{dnslink-fqdn}.ipfs/ could help, but dweb://example.com.ipfs feels awkward when compared to ipfs://example.com or dat://example.com
• multiplexing
  • introduces a separate extension to get the basic functionality working
    • it is not realistic to ask people to install two extensions
    • what will happen is ipfs-companion, dat and ssb extension ship with own dweb:// handlers and the UX mess of resolving conflicts will be pushed to the user:
      • we could detect dweb://{hash}.dat in ipfs-companion and open a page asking user to instal "dweb demultiplexer extension", however the user still needs to make that extension the default handler by picking it from a list of 3 or more - really bad UX
  • even if we do this, demultiplexing step is unclear given (non-)existing APIs for keeping address in location bar. It would require all dweb extensions to agree on protocol and access controls and cross-extension message passing, which would decrease performance, given how message passing works in webextensions

Right now, having a shared dweb:// is a much more complex proposition that introduces known unknowns: does not feel like a good fit given where we are with webextension APIs.
It is something we may as well do in the future instead of or in addition to dweb:{path} URI, when protocol handler APIs are present and well understood.

TL;DR I am not saying collapsing everything under dweb:// is not a valid solution,
but it feels orthogonal to the change proposed in this PR.

[hugomrdias]

i agree with @Stebalien proposal with some remarks

IMO i think DNSLink should be supported everywhere it's just a link, an alias.
We just resolve using DNS and continue normal operation just like we do in ipns right now. So:
/ipfs/domain.tld
/ipns/domain.tld

for protocol handlers using "dns links":

dweb://domain.tld.ipfs
ipfs+dweb://domain.tld
ipfs+https://domain.tld
dweb://ipfs:domain.tld

or just dweb://domain.tld and the browser/extension discovers the protocol during the dns resolution. ie. browser does the dns query, passes the response to the extensions registered for dweb:// and each extension checks the response to see if it should respond or ignore. The browser can even do most of the work and with the response from dns change the protocol to ipfs+dweb: if it detects dnslink or dat+dweb for whatever they use, They already hide the protocol from the end user so its the same for them and the UX doesnt change.

for hashes:

dweb://{cid}.ipfs/
dweb://{libp2p-key}.ipns/

for this we actually should own ipfs and ipns TLDs

[lidel]

Meeting Notes

We had a meeting on 2019-07-31 and discussed pros and cons of this PR.

• pros
  • less confusing UX of DNSLink websites in address badr
• cons
  • breaking 1:1 mapping with paths, current mental model is:
    • /ipfs/ == immutable
    • /ipns/ == "mutable naming for IPFS"
  • redirect the other way?
    • yes, we can do that, less controversial:
    • (A) ipfs://{dnslink} → ipns://{dnslink} (sounds like safer choice atm+++)
    • (B) ipfs://{dnslink} → dweb://{dnslink} (maybe in futuree, require discussion with others about shared ownership of dweb namespace/handler)

Conclusion

• We allow DNSLink in ipfs://{dnslink} but only in web browsers and only as a convenience redirect to ipns://{dnslink}
  • ipfs-companion will implement Redirect ipfs://example.net to ipns:// ipfs-companion#534
• dweb://{id}.{ipfs|ipns|dat|ssb|etc} proposed by @Stebalien is tracked in Shared stewardship of the "dweb" protocol handler arewedistributedyet/arewedistributedyet#28 (comment)