Skip to content
This repository has been archived by the owner on Feb 8, 2023. It is now read-only.

Serve Wikipedia from NYC Mesh nodes #165

Open
simplymathematics opened this issue Sep 10, 2016 · 8 comments
Open

Serve Wikipedia from NYC Mesh nodes #165

simplymathematics opened this issue Sep 10, 2016 · 8 comments

Comments

@simplymathematics
Copy link

Hey. I work with NYC Mesh GitHub.com/nycmeshnet. We want to serve wikipedia from every node, running ipfs. We also have a high bandwidth ixp back haul. Let's talk.

@jbenet
Copy link
Member

jbenet commented Sep 17, 2016

@meyerscr sure, what do you need? how can we help?

@jbenet jbenet changed the title help us do this!!!! Serve Wikipedia from NYC Mesh nodes Sep 17, 2016
@ghost
Copy link

ghost commented Sep 18, 2016

There's two things you can do to get started:

  1. we don't have wikipedia mirrored on IPFS just yet, check out Wikipedia ipfs-inactive/archives#20
  2. start running IPFS within the nycmesh network with an empty bootstrap list (ipfs bootstrap rm --all) and see whether MDNS discovery works on the nycmesh network (ipfs swarm peers).

@simplymathematics
Copy link
Author

MDNS does work via NYC Mesh because we have a VLAN even if there is not a Layer 2 connection (we use TINC if we have to 'mesh' over layer 3).

I had more technical questions about the actual deployment. For now, we're just going to add our wiki to the mesh.

Is it as simple as "ipfs add /www" since mediawiki generates links that are always internal? What about the config settings--I don't want to have to version control who is an admin. Also, will IPNS be give me an IP address to interface with so that I can send people to the ipfs wiki using a URL?

I'm pretty new to your software and there isn't anyone around with expertise, so I'm quite grateful for your help on this matter. I work with the Technology committee for Wikimedia-NYC. So, if we get this working on a small wiki, I can make a case to putting a mirror of the the entirety of wikimedia commons on our network. Since we have a datacenter, we can even use our copy as the 'official' ipfs copy, running on a building with 100Gb/s fiber lines running to London et al.

@ghost
Copy link

ghost commented Sep 23, 2016

Oh I see, thanks for explaining, and wow that sounds great, especially for nyc mesh! Happy to help, @jbenet and I met a few other folks of NYC Mesh at d.b.a. in april :)

You can't exactly put the wikimedia application on ipfs. Data in ipfs is always static (content-addressed with hashes), and is made dynamic by updating a pointer to a new version of the data (IPNS). What you can do is

  1. quickest way to useful results: make a readonly static version of it, and ipfs add -r static_page/ -- this will turn out at a couple TB I think. You can keep this mirror up-to-date and announce the IPNS path somewhere. People can fetch it through ipfs itself or use the http-to-ipfs gateway.
  2. more complex, but waaay more exciting: add the raw data for all pages to IPFS, then build a javascript app which fetches data from IPFS and uses that. You can even have static, incrementally-updated search indices in IPFS.

@ghost
Copy link

ghost commented Sep 23, 2016

And btw, if you get this working, I'd be very interested to replicate your setup within the Freifunk network in berlin.

@simplymathematics
Copy link
Author

I think I already have the application. Outernet built a browser-side content library, called librarian. It's very late here--I'll send you the link tomorrow.

@simplymathematics
Copy link
Author

https://librarian.outernet.is/en/files/

I think this might help?

@simplymathematics
Copy link
Author

Oh nevermind. Y'all already have a server side content browser.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants