Skip to content

Commit

Permalink
feat!: client abstraction
Browse files Browse the repository at this point in the history
  • Loading branch information
marcus-pousette committed Jun 22, 2023
1 parent 4c6319f commit 69c2eef
Show file tree
Hide file tree
Showing 324 changed files with 7,040 additions and 7,823 deletions.
4 changes: 2 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ jobs:
strategy:
matrix:
node-version: [18.x]
folder: [./packages/client,./packages/log ./packages/programs,./packages/libp2p/direct-block ./packages/libp2p/direct-stream ./packages/libp2p/direct-sub ./packages/libp2p/libp2p-test-utils ./packages/libp2p/peerbit-libp2p ./packages/utils ./packages/server-node]
folder: [./packages/client,./packages/log ./packages/programs,./packages/transport/direct-block ./packages/transport/direct-stream ./packages/transport/direct-sub ./packages/transport/libp2p-test-utils ./packages/transport/peerbit-libp2p ./packages/utils ./packages/server-node]
if: github.event_name == 'push'
runs-on: ubuntu-22.04
steps:
Expand Down Expand Up @@ -88,7 +88,7 @@ jobs:
matrix:
os: [ ubuntu-22.04 ]
node-version: [18.x]
folder: [./packages/client,./packages/log ./packages/programs,./packages/libp2p/direct-block ./packages/libp2p/direct-stream ./packages/libp2p/direct-sub ./packages/libp2p/libp2p-test-utils ./packages/libp2p/peerbit-libp2p ./packages/utils ./packages/server-node]
folder: [./packages/client,./packages/log ./packages/programs,./packages/transport/direct-block ./packages/transport/direct-stream ./packages/transport/direct-sub ./packages/transport/libp2p-test-utils ./packages/transport/peerbit-libp2p ./packages/utils ./packages/server-node]
name: Test (Node v${{ matrix.node-version }}, OS ${{ matrix.os }})
runs-on: ${{ matrix.os }}
steps:
Expand Down
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ orbitdb
/orbit-db
orbit-db-stores
lib
peerbit/
/peerbit
.vscode
/keystore/
keystore-test
Expand Down
26 changes: 25 additions & 1 deletion .release-please-manifest.json
Original file line number Diff line number Diff line change
@@ -1 +1,25 @@
{"packages/server-node/node":"1.0.5","packages/server-node/frontend":"0.2.2","packages/libp2p/direct-block":"1.0.4","packages/libp2p/direct-stream":"1.0.4","packages/libp2p/direct-sub":"1.0.4","packages/libp2p/libp2p-test-utils":"1.0.4","packages/client":"1.0.4","packages/programs/clock-service":"1.0.4","packages/utils/crypto":"1.0.4","packages/programs/data/document":"1.0.4","packages/programs/acl/identity-access-controller":"1.0.4","packages/libp2p/peerbit-libp2p":"1.0.4","packages/log":"1.0.4","packages/programs/program-interface":"1.0.4","packages/programs/rpc":"1.0.4","packages/programs/data/string":"1.0.4","packages/utils/test-utils":"1.0.4","packages/programs/acl/trusted-network":"1.0.4","packages/utils/logger":"1.0.0","packages/utils/time":"1.0.0","packages/utils/uint8arrays":"1.0.0","packages/utils/cache":"0.0.7","packages/utils/lazy-level":"0.0.9"}
{
"packages/server-node/node": "1.0.6",
"packages/server-node/frontend": "0.2.2",
"packages/libp2p/direct-block": "1.0.4",
"packages/libp2p/direct-stream": "1.0.4",
"packages/libp2p/direct-sub": "1.0.4",
"packages/libp2p/libp2p-test-utils": "1.0.4",
"packages/client": "1.0.5",
"packages/programs/clock-service": "1.0.5",
"packages/utils/crypto": "1.0.4",
"packages/programs/data/document": "1.0.5",
"packages/programs/acl/identity-access-controller": "1.0.5",
"packages/libp2p/peerbit-libp2p": "1.0.4",
"packages/log": "1.0.4",
"packages/programs/program-interface": "1.0.4",
"packages/programs/rpc": "1.0.4",
"packages/programs/data/string": "1.0.4",
"packages/utils/test-utils": "1.0.4",
"packages/programs/acl/trusted-network": "1.0.5",
"packages/utils/logger": "1.0.0",
"packages/utils/time": "1.0.0",
"packages/utils/uint8arrays": "1.0.0",
"packages/utils/cache": "0.0.7",
"packages/utils/lazy-level": "0.0.9"
}
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@

<br>
<p align="center">
<img width="140" src="./docs/peerbit-logo.png" alt="Peerbit icon Icon">
<img width="140" src="./docs/@peerbit/logo.png" alt="Peerbit icon Icon">
</p>

<h1 align="center" style="font-size: 5vmin;">
Expand Down
2 changes: 1 addition & 1 deletion docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ Every peer has an identity which is simply their public key, this key can *curre
Data can be shared and encrypted on a granular level, you can decide exactly what parts of metadata should be public and not. When you create a commit or a query request, you can specify exactly who is going to be able to decrypt the message. If you want an end to end conversation between two identities, you just include the other peers' public key as a receiver and you would be certain that know one in the middle would be able to read your message.

## Scaling networks
Peerbit is built on top of a [pubsub](./../packages/libp2p/direct-sub/) protocol that automatically optimizes the routing for packages so that the network stays overall healthy. If some path in the network gets congested, packages are routed with alternative routes, potentially over WebRTC, Websocket and TCP connections.
Peerbit is built on top of a [pubsub](./../packages/transport/direct-sub/) protocol that automatically optimizes the routing for packages so that the network stays overall healthy. If some path in the network gets congested, packages are routed with alternative routes, potentially over WebRTC, Websocket and TCP connections.

This is useful when you are building a app that requires streaming large amount of data, with a network consisting of peers with limited bandwidth. An example is a streaming service, where a streamer can write video stream chunks into a database and these chunks can propagage to thousands of peers without having to send to all of them directly.

Expand Down
6 changes: 3 additions & 3 deletions docs/coverpage.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,9 @@
[Get Started](introduction)
[Demo Sandbox](https://codesandbox.io/s/xv36w4695o)
[GitHub](https://github.com/jhildenbiddle/docsify-themeable) -->
<!-- ![logo](./peerbit-logo.png) -->
<!-- ![logo](./@peerbit/logo.png) -->
<p align="center">
<img width="200" src="./peerbit-logo.png" alt="Peerbit icon Icon">
<img width="200" src="./@peerbit/logo.png" alt="Peerbit icon Icon">
</p>

# Peerbit
Expand All @@ -27,7 +27,7 @@

<!-- <br>
<p align="center">
<img width="200" src="./peerbit-logo.png" alt="Peerbit icon Icon">
<img width="200" src="./@peerbit/logo.png" alt="Peerbit icon Icon">
</p>
<h1 align="center" style="font-size: 5vmin;">
Expand Down
14 changes: 5 additions & 9 deletions docs/examples/document-store.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,13 +12,9 @@

/// [imports]
import { field, variant } from "@dao-xyz/borsh";
import { Program } from "@dao-xyz/peerbit-program";
import { Peerbit } from "@dao-xyz/peerbit";
import {
DocumentIndex,
Documents,
SearchRequest,
} from "@dao-xyz/peerbit-document";
import { Program } from "@peerbit/program";
import { Peerbit } from "peerbit";
import { Documents, SearchRequest } from "@peerbit/document";
import { v4 as uuid } from "uuid";
/// [imports]

Expand Down Expand Up @@ -81,8 +77,8 @@ await peer2.dial(peer);

const store2 = await peer2.open<PostsDB>(store.address);

// Wait for peer1 to be reachable for query. This line only necessary when testing locally
await store.waitFor(peer2.libp2p);
// Wait for peer1 to be reachable for query
await store.waitFor(peer2.libp2p.peerId);

const responses: Post[] = await store2.posts.index.search(
new SearchRequest({
Expand Down
12 changes: 6 additions & 6 deletions docs/examples/text-store.ts
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
import { Peerbit } from "@dao-xyz/peerbit";
import { Program } from "@dao-xyz/peerbit-program";
import { PublicSignKey } from "@dao-xyz/peerbit-crypto";
import { Range, DString, StringOperation } from "@dao-xyz/peerbit-string";
import { Peerbit } from "peerbit";
import { Program } from "@peerbit/program";
import { PublicSignKey } from "@peerbit/crypto";
import { Range, DString, StringOperation } from "@peerbit/string";
import { field, variant } from "@dao-xyz/borsh";
import { Entry } from "@dao-xyz/peerbit-log";
import { Entry } from "@peerbit/log";

@variant("collaborative_text") // You have to give the program a unique name
class CollaborativeText extends Program {
Expand Down Expand Up @@ -37,7 +37,7 @@ class CollaborativeText extends Program {

const peer = await Peerbit.create();
const document = await peer.open(new CollaborativeText());
console.log(document.address); /// this address can be opened by another peer
console.log(document.address.toString()); /// this address can be opened by another peer

// ...
await document.string.add("hello", new Range({ offset: 0n, length: 5n }));
Expand Down
4 changes: 3 additions & 1 deletion docs/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ A cloneable boilerplate project can be found [here](https://github.com/dao-xyz/p

### Install
```sh
npm install @dao-xyz/peerbit
npm install peerbit
```

### Imports
Expand All @@ -23,6 +23,8 @@ npm install @dao-xyz/peerbit
### Seaching for the document from another peer
[another-client](./examples/document-store.ts ':include :type=code :fragment=another-client')

Read more about the Document store [here](./modules/program/document-store.md).




2 changes: 1 addition & 1 deletion docs/modules/client/client.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ The peer client

## Installation
```sh
npm install @dao-xyz/peerbit
npm install peerbit
```

## Config
Expand Down
2 changes: 1 addition & 1 deletion docs/modules/client/connectivity-direct.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { Peerbit } from "@dao-xyz/peerbit";
import { Peerbit } from "peerbit";
import type { Multiaddr } from "@multiformats/multiaddr";

const peerA = await Peerbit.create();
Expand Down
2 changes: 1 addition & 1 deletion docs/modules/client/connectivity-relay.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { Peerbit } from "@dao-xyz/peerbit";
import { Peerbit } from "peerbit";

const peerA = await Peerbit.create();
const peerB = await Peerbit.create();
Expand Down
2 changes: 1 addition & 1 deletion docs/modules/client/example.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/// [imports]
import { Peerbit } from "@dao-xyz/peerbit";
import { Peerbit } from "peerbit";
/// [imports]

/// [memory]
Expand Down
6 changes: 3 additions & 3 deletions docs/modules/deployment/deployment.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
# Deployment

## Serverless
Since Peerbit at the current stage is only javascript modules, you can deploy your project to any package manager, like NPM or Github Packages, in order to import them into your app directly, like a react project or electron app.
Since Peerbit at the current stage is only javascript modules, you can deploy your project to any package manager, like NPM or Github Packages, in order to import them into your app directly, for example a React project or Electron app.

## Server
Sometimes it make sense to deploy a Peerbit on a server that can be accessed through a domain. There are mainly two reasons why you want to do this:
- Hole punching. Two browser can not connect to each other directly, and requires an intermediate that enables a direct connection to be made (if possible)
- A replicator that is always online. While a clients can store data themselves, sometimes you need to be sure that there is always one node online.
- Hole punching. Two browser can not connect to each other directly without the aid on an intermediate peer that allows the browser clients to find other
- A replicator that is always online. While a client in the browser can store data themselves, sometimes you need to be sure that there is always one node online.

To deploy a server node, there is a handy CLI. [See this](./server-node.md)
2 changes: 1 addition & 1 deletion docs/modules/encoding/borsh.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import {
serialize,
deserialize,
} from "@dao-xyz/borsh";
import { randomBytes } from "@dao-xyz/peerbit-crypto";
import { randomBytes } from "@peerbit/crypto";
abstract class AbstractPost {}

@variant(0) // V0
Expand Down
2 changes: 1 addition & 1 deletion docs/modules/encoding/json.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import {
serialize,
deserialize,
} from "@dao-xyz/borsh";
import { randomBytes } from "@dao-xyz/peerbit-crypto";
import { randomBytes } from "@peerbit/crypto";

interface Message {
title: string;
Expand Down
8 changes: 4 additions & 4 deletions docs/modules/encryption/encrypted-document-store.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import { field, variant } from "@dao-xyz/borsh";
import { Observer, Program } from "@dao-xyz/peerbit-program";
import { Peerbit } from "@dao-xyz/peerbit";
import { Documents } from "@dao-xyz/peerbit-document";
import { Observer, Program } from "@peerbit/program";
import { Peerbit } from "peerbit";
import { Documents } from "@peerbit/document";
import { v4 as uuid } from "uuid";

class Message {
Expand Down Expand Up @@ -83,7 +83,7 @@ await store.messages.put(message, {
const store2 = await client2.open<DocumentStore>(store.address, {
role: new Observer(),
});
await store2.waitFor(client.libp2p);
await store2.waitFor(client.libp2p.peerId);

const messageRetrieved = await store2.messages.index.get(message.id);

Expand Down
8 changes: 4 additions & 4 deletions docs/modules/encryption/encrypted-log.ts
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
import { field, variant } from "@dao-xyz/borsh";
import { Program } from "@dao-xyz/peerbit-program";
import { Log } from "@dao-xyz/peerbit-log";
import { Peerbit } from "@dao-xyz/peerbit";
import { Program } from "@peerbit/program";
import { Log } from "@peerbit/log";
import { Peerbit } from "peerbit";
import { waitForResolved } from "../../../packages/utils/time/src";

// This class extends Program which allows it to be replicated amongst peers
@variant("xxxsstore")
@variant("simple_store")
class SimpleStore extends Program {
@field({ type: Log })
log: Log<string>; // Documents<?> provide document store functionality around your Posts
Expand Down
14 changes: 5 additions & 9 deletions docs/modules/program/composition.ts
Original file line number Diff line number Diff line change
@@ -1,11 +1,7 @@
import { field, variant } from "@dao-xyz/borsh";
import { Observer, Program } from "@dao-xyz/peerbit-program";
import { Peerbit } from "@dao-xyz/peerbit";
import {
DocumentIndex,
Documents,
SearchRequest,
} from "@dao-xyz/peerbit-document";
import { Observer, Program } from "@peerbit/program";
import { Peerbit } from "peerbit";
import { DocumentIndex, Documents, SearchRequest } from "@peerbit/document";
import { v4 as uuid } from "uuid";

@variant(0) // version 0
Expand Down Expand Up @@ -111,7 +107,7 @@ const forum2 = await client2.open<Forum>(forum.address, {
});

// Wait for client 1 to be available (only needed for testing locally)
await forum2.waitFor(client.libp2p);
await forum2.waitFor(client.libp2p.peerId);

// find channels from the forum from client2 perspective
const channels = await forum2.channels.index.search(new SearchRequest());
Expand All @@ -122,7 +118,7 @@ expect(channels[0].name).toEqual("general");
const channel2 = await client2.open<Channel>(channels[0]);

// Wait for client 1 to be available (only needed for testing locally)
await channel2.waitFor(client.libp2p);
await channel2.waitFor(client.libp2p.peerId);

// find messages
const messages = await channel2.db.posts.index.search(new SearchRequest());
Expand Down
2 changes: 1 addition & 1 deletion docs/modules/program/document-store.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ The document store is program/database you can use within your apps. The purpose
The document store is a separate package and you can install it with

```
npm install @dao-xyz/peerbit-document
npm install @peerbit/document
```


Expand Down
14 changes: 7 additions & 7 deletions docs/modules/program/document-store.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
/// [imports]
import { field, variant, option } from "@dao-xyz/borsh";
import { Observer, Program } from "@dao-xyz/peerbit-program";
import { Peerbit } from "@dao-xyz/peerbit";
import { Observer, Program } from "@peerbit/program";
import { Peerbit } from "peerbit";
import { v4 as uuid } from "uuid";

import {
Expand All @@ -11,7 +11,7 @@ import {
MissingField,
SearchRequest,
Sort,
} from "@dao-xyz/peerbit-document";
} from "@peerbit/document";
/// [imports]

/// [definition]
Expand Down Expand Up @@ -131,7 +131,7 @@ class Channel extends Program {
/// [definition]

/// [insert]
import { waitForResolved } from "@dao-xyz/peerbit-time";
import { waitForResolved } from "@peerbit/time";

// Start two clients that ought to talk to each other
const peer = await Peerbit.create();
Expand All @@ -149,8 +149,8 @@ const channelFromClient2 = await peer2.open<Channel>(
}
);

// Wait for peer1 to be reachable for query. This line only necessary when testing locally
await channelFromClient2.waitFor(peer.libp2p);
// Wait for peer1 to be reachable for query
await channelFromClient2.waitFor(peer.libp2p.peerId);

// Lets write some things
const message1 = new Post("hello world");
Expand Down Expand Up @@ -279,7 +279,7 @@ import {
SortDirection,
Or,
And,
} from "@dao-xyz/peerbit-document";
} from "@peerbit/document";

new SearchRequest({
query: [
Expand Down
8 changes: 4 additions & 4 deletions docs/modules/program/example.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
/// [program]
/// [definition]
import { Log } from "@dao-xyz/peerbit-log";
import { Program } from "@dao-xyz/peerbit-program";
import { Log } from "@peerbit/log";
import { Program } from "@peerbit/program";
import { field, variant } from "@dao-xyz/borsh";

// The line below will make sure that every time the database manifest
Expand All @@ -27,8 +27,8 @@ class MyDatabase extends Program {

/// [definition]

import { Observer, Replicator } from "@dao-xyz/peerbit-program";
import { Peerbit } from "@dao-xyz/peerbit";
import { Observer, Replicator } from "@peerbit/program";
import { Peerbit } from "peerbit";

const client = await Peerbit.create();

Expand Down
2 changes: 1 addition & 1 deletion docs/sharding/sharding.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ Imagine the commit above is made, so that the merged graph gets the label "DOG",


### 1.
The first thing we need to do is to hash the labels of the peers (IPFS IDs) and the DOG label with a hash function (more details on this function later)
The first thing we need to do is to hash the labels of the peers (PeerIds) and the DOG label with a hash function (more details on this function later)

<p align="center">
<img width="400" src="./p2.png" alt="p2">
Expand Down
Loading

0 comments on commit 69c2eef

Please sign in to comment.