A distributed server

[rom1504]

This is a design document / idea document that might go into a .md file instead of an issue at some point.
We discussed recently with @Karang how would a distributed server look like with prismarinejs stuff.
The important idea is to make a performant distributed server, the design matter just as much if not more as the performance of one instance.
How can we make prismarinejs distributable ?

1. The server should be mostly stateless : this make it easy to scale horizontally
2. It means we need to store the world in something distinct from the server that will just act as a cache
3. Prismarine-world come into play : it will be sync and not async as currently and try to contain all the state of the world : chunks but also entities and inventories
4. Once we have a good prismarine-world, things like prismarine-physics can be built on top and make it clean (it can be used for both mineflayer and flying-squid)
5. The server instances will then all have their own world cache, these instances could be partitionned by region to avoid concurrency issues.
6. The prismarine-world cache can be backed by any storage including anvil on an nfs, or simply using a fast db (memcache, reddis, with value being a chunk maybe)

This kind of design should allow :

• Easier manipulation of the world (all sync)
• Many user of a same world abstraction : mineflayer, flying-squid, but also modules in the middle, proxies, viewers, machine learning models,...
• Scalable servers
• Scalable multi bots (which are basically servers with multiple outgoing connections)

This design might not even need a lot of work (mostly prismarine-world refactoring), and would unlock some nice use cases.

[philipbeadle]

This is exactly what I am doing. I work for https://holochain.org and have been experimenting with Minecraft. I did a bunch of digging and found flying-squid and have worked out how I think it can be done easily. Please get in contact
philip.beadle@holo.host

[rom1504]

Great, maybe we could join our discord so we can talk about it ("advanced coders" room could fit for this kind of topic)

The current status is we are using prismarine-world in prismarine-physics and flying-squid. I am trying to use it in more of our projects so we have all the state in there and so it will help for this project.

Would be great to talk about this, maybe you could help for some parts ?

[jessiepathfinder]

A SQL Storage Provider would do the trick

[Pandapip1]

As someone that has a little of bit of experience with them, I wonder if zero-knowledge proofs might make this easier

[rom1504]

can't see how

[Pandapip1]

can't see how

It allows the distributed computation to be verified without e.g. leaking your coordinates.

[ajh123]

Could bungeecord help?

[BasToTheMax]

How would this work for redstone? How is this being ticked accross multiple servers?

[Pandapip1]

Update to my zk proof idea: SMPC seems more applicable for this task than ZKPs. Either are pretty expensive operations, so many ticks (~5 seconds worth seems like a good balance) would have to be bundled into a single round.

Nodes could communicate any number of ways. I'm leaning towards either IPFS PubSub or IPFS's libp2p.

This seems shockingly doable.

[extremeheat]

WebRTC is standardized P2P protocol and works in the browser. Since there is a contributor working on squid in browser, it could be used for multiplayer between other clients without the need for central server. This would be useful practically beyond being decentralized for the sake of it.

[extremeheat]

For standard single player worlds over WebRTC peers, there would still technically be a “host” hosting the server and being the authoritative party. They would receive all the packets traditionally sent to a central server either directly or relay via another peer, and it would be standard Minecraft encryption, so other peers can’t decode after client handshake. So if the host quits then everyone else also quits.

For fully decentralized, this would only really work for mini games (to cut down on state related and simulation related issues). It will be a bit more challenging as a custom protocol will be necessary. Since there would need to be some voting system to decide on approving/rejecting individual client actions, based on everyone running their own version of flying squid. If clients don’t acknowledge the rejection all the peers would close their connection.

For the latter it would likely be much easier to start from scratch with a basic server than using flying-squid.

[Pandapip1]

https://gun.js.org/ might be worth looking into for storing and syncing state.