Skip to content

Latest commit

 

History

History
 
 

node

README.md

title
Node.js

This file is generated by src/clients/docs_generate.zig.

tigerbeetle-node

The TigerBeetle client for Node.js.

Prerequisites

Linux >= 5.6 is the only production environment we support. But for ease of development we also support macOS.

  • NodeJS >= 14

Setup

Run:

$ npm install tigerbeetle-node

Now, create test.js and copy this into it:

const { createClient } = require("tigerbeetle-node");
console.log("Import ok!");

Finally, build and run:

$ node run test.js

Now that all prerequisites and dependencies are correctly set up, let's dig into using TigerBeetle.

If you run into issues, check out the distribution-specific install steps that are run in CI to test support:

Example projects

This document is primarily a reference guide to the client. Below are various sample projects demonstrating features of TigerBeetle.

  • Basic: Create two accounts and transfer an amount between them.
  • Two-Phase Transfer: Create two accounts and start a pending transfer between them, then post the transfer.

Sidenote: BigInt

TigerBeetle uses 64-bit integers for many fields while JavaScript's builtin Number maximum value is 2^53-1. The n suffix in JavaScript means the value is a BigInt. This is useful for literal numbers. If you already have a Number variable though, you can call the BigInt constructor to get a BigInt from it. For example, 1n is the same as BigInt(1).

Creating a Client

A client is created with a cluster ID and replica addresses for all replicas in the cluster. The cluster ID and replica addresses are both chosen by the system that starts the TigerBeetle cluster.

Clients are thread-safe. But for better performance, a single instance should be shared between multiple concurrent tasks.

Multiple clients are useful when connecting to more than one TigerBeetle cluster.

In this example the cluster ID is 0 and there are three replicas running on ports 3001, 3002, and 3003.

const client = createClient({
  cluster_id: 0,
  replica_addresses: ['3001', '3002', '3003']
});

The following are valid addresses:

  • 3000 (interpreted as 127.0.0.1:3000)
  • 127.0.0.1:3000 (interpreted as 127.0.0.1:3000)
  • 127.0.0.1 (interpreted as 127.0.0.1:3001, 3001 is the default port)

Creating Accounts

See details for account fields in the Accounts reference.

let account = {
  id: 137n,
  user_data: 0n,
  reserved: Buffer.alloc(48, 0),
  ledger: 1,
  code: 718,
  flags: 0,
  debits_pending: 0n,
  debits_posted: 0n,
  credits_pending: 0n,
  credits_posted: 0n,
  timestamp: 0n,
};

let accountErrors = await client.createAccounts([account]);

Account Flags

The account flags value is a bitfield. See details for these flags in the Accounts reference.

To toggle behavior for an account, combine enum values stored in the AccountFlags object (in TypeScript it is an actual enum) with bitwise-or:

  • AccountFlags.linked
  • AccountFlags.debits_must_not_exceed_credits
  • AccountFlags.credits_must_not_exceed_credits

For example, to link two accounts where the first account additionally has the debits_must_not_exceed_credits constraint:

let account0 = { /* ... account values ... */ };
let account1 = { /* ... account values ... */ };
account0.flags = AccountFlags.linked | AccountFlags.debits_must_not_exceed_credits;
accountErrors = await client.createAccounts([account0, account1]);

Response and Errors

The response is an empty array if all accounts were created successfully. If the response is non-empty, each object in the response array contains error information for an account that failed. The error object contains an error code and the index of the account in the request batch.

See all error conditions in the create_accounts reference.

let account2 = { /* ... account values ... */ };
let account3 = { /* ... account values ... */ };
let account4 = { /* ... account values ... */ };
accountErrors = await client.createAccounts([account2, account3, account4]);
for (const error of accountErrors) {
  switch (error.code) {
    case CreateAccountError.exists:
      console.error(`Batch account at ${error.index} already exists.`);
	  break;
    default:
      console.error(`Batch account at ${error.index} failed to create: ${CreateAccountError[error.code]}.`);
  }
}

To handle errors you can either 1) exactly match error codes returned from client.createAccounts with enum values in the CreateAccountError object, or you can 2) look up the error code in the CreateAccountError object for a human-readable string.

Account Lookup

Account lookup is batched, like account creation. Pass in all IDs to fetch. The account for each matched ID is returned.

If no account matches an ID, no object is returned for that account. So the order of accounts in the response is not necessarily the same as the order of IDs in the request. You can refer to the ID field in the response to distinguish accounts.

const accounts = await client.lookupAccounts([137n, 138n]);
console.log(accounts);
/*    
 * [{
 *   id: 137n,
 *   user_data: 0n,
 *   reserved: Buffer,
 *   ledger: 1,
 *   code: 718,
 *   flags: 0,
 *   debits_pending: 0n,
 *   debits_posted: 0n,
 *   credits_pending: 0n,
 *   credits_posted: 0n,
 *   timestamp: 1623062009212508993n,
 * }]
 */

Create Transfers

This creates a journal entry between two accounts.

See details for transfer fields in the Transfers reference.

let transfer = {
  id: 1n,
  pending_id: 0n,
  debit_account_id: 1n,
  credit_account_id: 2n,
  user_data: 0n,
  reserved: 0n,
  timeout: 0n,
  ledger: 1,
  code: 720,
  flags: 0,
  amount: 10n,
  timestamp: 0n,
};
let transferErrors = await client.createTransfers([transfer]);

Response and Errors

The response is an empty array if all transfers were created successfully. If the response is non-empty, each object in the response array contains error information for an transfer that failed. The error object contains an error code and the index of the transfer in the request batch.

See all error conditions in the create_transfers reference.

for (const error of transferErrors) {
  switch (error.code) {
    case CreateTransferError.exists:
      console.error(`Batch transfer at ${error.index} already exists.`);
	  break;
    default:
      console.error(`Batch transfer at ${error.index} failed to create: ${CreateTransferError[error.code]}.`);
  }
}

To handle errors you can either 1) exactly match error codes returned from client.createTransfers with enum values in the CreateTransferError object, or you can 2) look up the error code in the CreateTransferError object for a human-readable string.

Batching

TigerBeetle performance is maximized when you batch API requests. The client does not do this automatically for you. So, for example, you can insert 1 million transfers one at a time like so:

for (let i = 0; i < transfers.len; i++) {
  const transferErrors = await client.createTransfers(transfers[i]);
  // error handling omitted
}

But the insert rate will be a fraction of potential. Instead, always batch what you can.

The maximum batch size is set in the TigerBeetle server. The default is 8191.

const BATCH_SIZE = 8191;
for (let i = 0; i < transfers.length; i += BATCH_SIZE) {
  const transferErrors = await client.createTransfers(transfers.slice(i, Math.min(transfers.length, BATCH_SIZE)));
  // error handling omitted
}

Queues and Workers

If you are making requests to TigerBeetle from workers pulling jobs from a queue, you can batch requests to TigerBeetle by having the worker act on multiple jobs from the queue at once rather than one at a time. i.e. pulling multiple jobs from the queue rather than just one.

Transfer Flags

The transfer flags value is a bitfield. See details for these flags in the Transfers reference.

To toggle behavior for a transfer, combine enum values stored in the TransferFlags object (in TypeScript it is an actual enum) with bitwise-or:

  • TransferFlags.linked
  • TransferFlags.pending
  • TransferFlags.post_pending_transfer
  • TransferFlags.void_pending_transfer

For example, to link transfer0 and transfer1:

transfer0 = { /* ... transfer values ... */ };
transfer1 = { /* ... transfer values ... */ };
transfer0.flags = TransferFlags.linked;
// Create the transfer
transferErrors = await client.createTransfers([transfer0, transfer1]);

Two-Phase Transfers

Two-phase transfers are supported natively by toggling the appropriate flag. TigerBeetle will then adjust the credits_pending and debits_pending fields of the appropriate accounts. A corresponding post pending transfer then needs to be sent to post or void the transfer.

Post a Pending Transfer

With flags set to post_pending_transfer, TigerBeetle will post the transfer. TigerBeetle will atomically roll back the changes to debits_pending and credits_pending of the appropriate accounts and apply them to the debits_posted and credits_posted balances.

transfer = {
  id: 2n,
  pending_id: 1n,
  flags: TransferFlags.post_pending_transfer,
  timestamp: 0n,
};
transferErrors = await client.createTransfers([transfer]);

Void a Pending Transfer

In contrast, with flags set to void_pending_transfer, TigerBeetle will void the transfer. TigerBeetle will roll back the changes to debits_pending and credits_pending of the appropriate accounts and not apply them to the debits_posted and credits_posted balances.

transfer = {
  id: 2n,
  pending_id: 1n,
  flags: TransferFlags.void_pending_transfer,
  timestamp: 0n,
};
transferErrors = await client.createTransfers([transfer]);

Transfer Lookup

NOTE: While transfer lookup exists, it is not a flexible query API. We are developing query APIs and there will be new methods for querying transfers in the future.

Transfer lookup is batched, like transfer creation. Pass in all ids to fetch, and matched transfers are returned.

If no transfer matches an id, no object is returned for that transfer. So the order of transfers in the response is not necessarily the same as the order of ids in the request. You can refer to the id field in the response to distinguish transfers.

const transfers = await client.lookupTransfers([1n, 2n]);
console.log(transfers);
/*
 * [{
 *   id: 1n,
 *   pending_id: 0n,
 *   debit_account_id: 1n,
 *   credit_account_id: 2n,
 *   user_data: 0n,
 *   reserved: 0n,
 *   timeout: 0n,
 *   ledger: 1,
 *   code: 720,
 *   flags: 0,
 *   amount: 10n,
 *   timestamp: 1623062009212508993n,
 * }]
 */

Linked Events

When the linked flag is specified for an account when creating accounts or a transfer when creating transfers, it links that event with the next event in the batch, to create a chain of events, of arbitrary length, which all succeed or fail together. The tail of a chain is denoted by the first event without this flag. The last event in a batch may therefore never have the linked flag set as this would leave a chain open-ended. Multiple chains or individual events may coexist within a batch to succeed or fail independently.

Events within a chain are executed within order, or are rolled back on error, so that the effect of each event in the chain is visible to the next, and so that the chain is either visible or invisible as a unit to subsequent events after the chain. The event that was the first to break the chain will have a unique error result. Other events in the chain will have their error result set to linked_event_failed.

const batch = [];
let linkedFlag = 0;
linkedFlag |= CreateTransferFlags.linked;

// An individual transfer (successful):
batch.push({ id: 1n /* , ... */ });

// A chain of 4 transfers (the last transfer in the chain closes the chain with linked=false):
batch.push({ id: 2n, /* ..., */ flags: linkedFlag }); // Commit/rollback.
batch.push({ id: 3n, /* ..., */ flags: linkedFlag }); // Commit/rollback.
batch.push({ id: 2n, /* ..., */ flags: linkedFlag }); // Fail with exists
batch.push({ id: 4n, /* ..., */ flags: 0 });          // Fail without committing.

// An individual transfer (successful):
// This should not see any effect from the failed chain above.
batch.push({ id: 2n, /* ..., */ flags: 0 });

// A chain of 2 transfers (the first transfer fails the chain):
batch.push({ id: 2n, /* ..., */ flags: linkedFlag });
batch.push({ id: 3n, /* ..., */ flags: 0 });

// A chain of 2 transfers (successful):
batch.push({ id: 3n, /* ..., */ flags: linkedFlag });
batch.push({ id: 4n, /* ..., */ flags: 0 });

const errors = await client.createTransfers(batch);

/**
 * console.log(errors);
 * [
 *  { index: 1, error: 1 },  // linked_event_failed
 *  { index: 2, error: 1 },  // linked_event_failed
 *  { index: 3, error: 25 }, // exists
 *  { index: 4, error: 1 },  // linked_event_failed
 * 
 *  { index: 6, error: 17 }, // exists_with_different_flags
 *  { index: 7, error: 1 },  // linked_event_failed
 * ]
 */

Development Setup

On Linux and macOS

In a POSIX shell run:

$ git clone https://github.com/${GITHUB_REPOSITY:-tigerbeetledb/tigerbeetle}
$ cd tigerbeetle
$ git checkout $GIT_SHA
$ ./scripts/install_zig.sh
$ cd src/clients/node
$ npm install --include dev
$ npm pack
$ [ "$TEST" = "true" ] && mvn test || echo "Skipping client unit tests"

On Windows

Not yet supported.