Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Beacon metadata bundler #9

Open
6 tasks
Zorlin opened this issue Aug 28, 2024 · 1 comment
Open
6 tasks

Beacon metadata bundler #9

Zorlin opened this issue Aug 28, 2024 · 1 comment
Assignees
Labels
enhancement New feature or request

Comments

@Zorlin
Copy link
Contributor

Zorlin commented Aug 28, 2024

Goal:
To make an endpoint we can use to download the entire metadata set of a site and mirror it or copy it,

Specification:
The beacon metadata bundler will have a simple job, which it will do continuously in a single sequence:

  • Analyse each entire OrbitDB database involved in serving its Riff site
  • Compile the data into a JSON bundle
  • Snapshot that bundle, upload that bundle to iroh to be served as a blob, upload that bundle to IPFS to be served
  • Get the CID of that blob
  • Publish the CID at an API endpoint (and if configured update an IPNS name via Kubo)
  • Bonus points if it keeps the set in memory at all times so that it can publish compressed snapshots very quickly.

The endpoint will be exposed only internally on an API path on port 5451, which will be used by a caching proxy that grabs the CIDs and publishes them for people to access (thus not allowing access directly to the beacon itself).

@Zorlin Zorlin added the enhancement New feature or request label Aug 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

4 participants
@Zorlin @julienmalard @benya7 and others