Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revisit how Companion does preload at public gateways #546

Open
2 tasks
lidel opened this issue Jul 28, 2018 · 1 comment
Open
2 tasks

Revisit how Companion does preload at public gateways #546

lidel opened this issue Jul 28, 2018 · 1 comment
Labels
help wanted Seeking public contribution on this issue kind/discussion Topical discussion; usually not changes to codebase kind/maintenance Work required to avoid breaking changes or harm to project's status quo status/deferred Conscious decision to pause or backlog

Comments

@lidel
Copy link
Member

lidel commented Jul 28, 2018

Background

Right now Companion preloads uploaded content via HTTP HEAD requests to a public gateway.
Public gateway can be customized by the user, and there is an option on Preferences screen to disable preload.

The js-ipfs and infra teams are working on a solution that will fix discoverability and restore "realtime feel" in browser contexts: ipfs/js-ipfs#1459. In short there will be a set of "official" preload gateways that will be used for every write operation made by js-ipfs when running in web browser.

UPDATE: recursive preloads are now possible with:

https://node0.preload.ipfs.io/api/v0/refs?arg={CID}&r=true
https://node1.preload.ipfs.io/api/v0/refs?arg={CID}&r=true

TODO

Update preloadAtPublicGateway and in addition to existing HTTP HEAD request to user-specified public gateway, do additional asynchronous recursive preload via HTTP GET request to one of preload nodes. In case of "quick-upload" screen, the CID should be of the wrapping directory.

Open Questions

Given discussion in ipfs/js-ipfs#1459, questions arise:

  • (1): should we switch from HTTP HEAD per CID to /api/v0/refs?r=true&arg=<cid> ?
    • 👍 when adding N files you only need 1 preload request (for root) to /api/v0/refs instead of N+1 HTTP HEADs (one per file + wrapping dir).
    • 👎 refs requires /api/ to be exposed to the internet, some public gateways hide everything but /ipfs/.
      • This means we need to use specific gateway instead of the one picked by the user, or implement a fallback to HTTP GET if refs returns statuc code different than 200.
  • (2): should we enable preload for all window.ipfs operations? (mirroring behaviour of js-ipfs)
    • initial answer is yes, the rationale for doing this is we want to ensure fantastic UX and DX and key to that is for dapp data to not disappear from the swarm immediately when user closes laptop.
    • Given js-ipfs will be doing preload by default starting with next release, should we limit it to state when running against non-js-ipfs backend?
    • If implemented, global "preload" checkbox in Companion should also control preload in window.ipfs
  • (3): Should global "preload" checkbox in Companion disable implicit preload when running against embedded js-ipfs as the backend ?
    • initial answer is yes, otherwise user is confused why preload requests are still sent by the browser
@lidel lidel added help wanted Seeking public contribution on this issue kind/discussion Topical discussion; usually not changes to codebase UX kind/maintenance Work required to avoid breaking changes or harm to project's status quo area/window-ipfs Issues related to IPFS API exposed on every page labels Jul 28, 2018
@lidel lidel changed the title Revisit the way we Companion does preload at public gateways Revisit how Companion does preload at public gateways Jul 28, 2018
@alanshaw
Copy link
Member

FYI the HEAD request won't preload a file if it is bigger than max chunk size (unless you traverse the tree that's created and send a request for EVERY node that's created, as well as all it's children). This is why js-ipfs uses the refs API.

@lidel lidel added the status/deferred Conscious decision to pause or backlog label Sep 22, 2018
@lidel lidel removed the area/window-ipfs Issues related to IPFS API exposed on every page label Oct 19, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Seeking public contribution on this issue kind/discussion Topical discussion; usually not changes to codebase kind/maintenance Work required to avoid breaking changes or harm to project's status quo status/deferred Conscious decision to pause or backlog
Projects
No open projects
Status: Needs Grooming
Development

No branches or pull requests

3 participants