secretvaults

Wrapper classes for simplifying usage of Nillion's Secret Vault and the nilQL encryption and decryption library.

Description and Purpose

This wrapper provides functions to simplify usage of Nillion's SecretVault and nilQL.

NilQLWrapper: Lightweight wrapper for encryption and decryption using nilQL

• Encrypts data into shares for distributed storage across nodes
• Handles structured data with %allot markers for selective encryption
• Recombines shares and decrypts data marked %share using unify
• Manages secret keys for encryption/decryption operations
• Recombines and decrypts shares to recover original data
• Maintains compatibility with SecretVault timestamps
• No node configuration required when used standalone

SecretVaultWrapper: Wrapper for Secret Vault API operations

Authentication

• Handles JWT creation and management per node
• Manages node authentication automatically

Schema Operations

Create: Deploy schema across nodes (/api/v1/schemas)

• Creates schemas with optional custom ID
• Validates schema structure
• Distributes to all nodes

Read: List available schemas (/api/v1/schemas)

• Retrieves schema configurations
• Shows schema metadata and structure

Delete: Remove schema definition (/api/v1/schemas)

• Deletes schema across all nodes

Data Operations

Create: Upload data to the specified schema collection (/api/v1/data/create)

• Writes data to multiple nodes
• Encrypts specified fields with %allot markers before distribution
• Distributes encrypted shares marked %share across nodes

Read: Retrieve data from the specified schema collection (/api/v1/data/read)

• Retrieves data from all nodes
• Recombines encrypted shares marked %share from nodes to decrypt specified fields automatically
• Returns decrypted record

Update: Update data in the specified schema collection (/api/v1/data/update)

• Encrypts specified fields with %allot markers before distribution
• Distributes encrypted shares marked %share across nodes, updating existing records matching the provided filter

Delete: Delete data from the specified schema collection (/api/v1/data/delete)

• Deletes existing records on all nodes that match the provided filter

Flush: Remove all documents in a schema collection (/api/v1/data/flush)

• Removes all data across nodes from a schema collection

Query Operations

Create: Deploy query across nodes (/api/v1/queries)

• Creates query with optional custom ID
• Includes Mongo Aggregation Pipeline and custom Runtime Variables
• Distributes to all nodes

Execute: Runs the query across nodes (/api/v1/queries/execute)

• Executes the query on all nodes with the provided variables
• Recombines encrypted shares or aggregation results marked %share from nodes to decrypt specified fields automatically
• Returns decrypted record

Read: List available queries (/api/v1/queries)

• Retrieves available queries
• Shows query definition and structure

Delete: Remove query across nodes (/api/v1/queries)

• Deletes query across all nodes

Installation and Usage

You can install the package using pip:

pip install secretvaults

The library can be imported in the usual ways:

import secretvaults
from secretvaults import *

Standalone NilQLWrapper Example

An example workflow that demonstrates use of the wrapper is presented below:

Run examples:

python3 examples/nilql_encryption.py

SecretVaultWrapper Example

Copy the .env.example to create a .env file that uses the example org:

cp .env.example .env

Run example to encrypt and upload data to all nodes, then read data from nodes:

python3 examples/store_encryption/data_create_read.py

Development

All installation and development dependencies are fully specified in pyproject.toml. The project.optional-dependencies object is used to specify optional requirements for various development tasks. This makes it possible to specify additional options (such as docs, lint, and so on) when performing installation using pip:

python -m pip install ".[docs,lint]"

Documentation

The documentation can be generated automatically from the source files using Sphinx:

python -m pip install ".[docs]"
cd docs
sphinx-apidoc -f -E --templatedir=_templates -o _source .. && make html

Testing and Conventions

All unit tests are executed and their coverage is measured when using pytest (see the pyproject.toml file for configuration details):

python -m pip install ".[test]"
python -m pytest test

Style conventions are enforced using Pylint:

python -m pip install ".[lint]"
python -m pylint src/secretvaults

Contributions

To contribute to the source code, open an issue or submit a pull request on the GitHub page for this library.

Versioning

The version number format for this library and the changes to the library associated with version number increments conform to Semantic Versioning 2.0.0.

Publishing

This library can be published as a package on PyPI via the GitHub Actions workflow found in .github/workflows/build-publish-sign-release.yml that follows the recommendations found in the Python Packaging User Guide.

Ensure that any links in this README document to the Read the Docs documentation of this package (or its dependencies) have appropriate version numbers. Also ensure that the Read the Docs project for this library has an automation rule that activates and sets as the default all tagged versions.