Converge

Converge is a set of libraries for working with convergent replicated datatypes (CRDTs) in Clojure and ClojureScript.

Modules:

https://clojure.org/reference/deps_and_cli#_dependencies Both modules can be imported by specifying a :git/url, :sha, and :deps/root in your deps.edn :deps like the following example:

converge/converge {:git/url   "https://github.com/evidentsystems/converge"
                   :sha       "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
                   :deps/root "./converge"}

converge/transit  {:git/url   "https://github.com/evidentsystems/converge"
                   :sha       "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
                   :deps/root "./transit"}

;; etc.

Main module: converge (path converge/)

The main module, converge, provides a convergent reference type for Clojure and ClojureScript, with support for various pluggable backends implementing different convergence algorithms. Its local modification and access API resembles the behavior of an Atom (i.e. swap!, reset!, and deref). However, it also has API functions for accessing (converge.api/peek-patches, converge.api/pop-patches!) patches generated by local modifications, and for applying patches from the local actor to/from remote actors or merging with a remote convergent ref (converge.api/merge!).

The main guarantee provided by this reference type is that local swap!/reset! operations and patching/merging between various distributed actors will converge: that is, two convergent references with the same underlying log of operations will have the same value, regardless of the order in which these references received these operations.

This module also provides two functions for synchronizing convergent references: converge.api/clock and converge.api/patch-from-clock.

Finally, this main module provides the converge.serialize namespace, which contains generic functions for transforming converge types in preparation for serialization/deserialization (used by transit, nippy, etc. modules).

Module: converge-nippy (path nippy/)

The converge-nippy module wires up the serialization handlers to work with nippy serialization in Clojure.

Module: converge-storage (path storage/)

The converge-storage module defines a small API for storing, synchronizing, and fetching convergent refs to/from various storage systems:

• The local filesystem (also useful with git and other VCS) via:
  • converge.storage.filesystem/sync-directory: initializes new convergent reference storage directory or patches an existing one (by adding a new file to the directory) as needed based on the state of the given in-memory convergent ref
  • converge.storage.filesystem/from-directory: creates an in-memory convergent reference from contents of a previously sync'd directory
• Other storage systems are under consideration

Module: converge-transit (path transit/)

The converge-transit module wires up the serialization handlers to work with transit-clj and transit-cljs.

More modules to come!

We have high-level plans to provide other functionality commonly used with CRDT, such as implementation of the synchronization protocol atop various communication channels:

• WebRTC
• Websocket
• HTTP

We also plan to support other serialization mechanisms like Fressian.

Finally, we plan to provide a means of tracking/diffing/merging structured data (EDN) alongside the semi-structured data (files, lines, directories) in a Git repository.

Status

⚠️ Converge is beta quality software. Its API and implementation may change. Use at your own risk!

Evident Systems is using Converge in oNote to support real-time and async/repository-based collaboration.

License

All converge libaries in this repository are subject to the following:

Copyright 2020 Evident Systems LLC

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.