Skip to content

Find out more on www.waive.studio Waive is an international collaboration between Thunderboom Records and four European heritage institutions: The Netherlands Institute for Sound and Vision, the BBC Archives, the Catalan Media Corporation and the Baltic Audiovisual Archival Council.

Notifications You must be signed in to change notification settings

ThunderboomRecords/WAIVE

Repository files navigation

WAIVE-Plugins

AI x Archive music tools

WAIVE-Midi screenshot

WAIVE-Sampler screenshot

A plugin suite that combines music, sound and MIDI generation with European cultural archives. Aims to be an offline, modular version of WAIVE-Studio that can be integrated into your DAW. Built with DISTRHO Plugin Framework and ONNX Runtime.

  • WAIVE-Midi: a rhythmic pattern generator
  • WAIVE-Sampler: a sample player, sample library and sample generator in one

Developed by Arran Lyon for Thunderboom Records. Contributions and pull-requests welcome, especially regarding stability improvements and Windows release.

Download and Install

Currently, there is no installer provided, so you must install the plugins manually:

If you are reinstalled the plugin from an older, develpment version, you may need to delete the old database file as described in the Troubleshooting section.

  1. Download and extract the latest archive from the Releases page for your platform (currently only available for macOS Apple Silicon and Linux). You can find the download links under the Assets heading.
  2. Choose which plugin format you prefer and place the corresponding bundle (e.g. WAIVE_Sampler.vst3) in your plugins path of your DAW:
    • Common (system-wide) plugin paths:

      VST2 VST3 CLAP AudioUnit
      macOS Library/Audio/Plug-Ins/VST Library/Audio/Plug-Ins/VST3 Library/Audio/Plug-Ins/CLAP Library/Audio/Plug-Ins/Components
      Linux /usr/lib/vst /usr/lib/vst3 /usr/lib/clap --
      Windows C:\Program Files\Common Files\VST2 C:\Program Files\Common Files\VST3 C:\Program Files\Common Files\CLAP --
    • To access these folders in macOS, open Finder, press Shift+Command+G, and enter the path you wish to access.

  3. In your DAW, rescan plugins if it does not do so automatically.

Ableton Live Notes

  • If you installed the plug-ins in the above directories, then make sure that "Use VST3 System Plug-in Folders" is On (same for VST2) in Settings > Plug-ins before rescanning.
  • To use WAIVE-Midi, add this plugin to an empty MIDI track. Then, on a separate MIDI track add your instrument (e.g. WAIVE-Sampler) then set MIDI From to the name of the first track and make sure to set it to use Post-FX.
  • The VST3 and AudioUnit version of WAIVE-Midi currently are not loading in Live, so use the VST2 version.

Build Instructions

To build WAIVE-Plugins from source.

Pre-requisites

Requires statically built onnxruntime for your platform. You can download pre-built libraries from csukuangfj/onnxruntime-libs, or build them yourself (such as with ort-builder). Tested with 1.17.1.

Requires cmake and ninja:

  • on Mac, with homebrew: $ brew install cmake ninja
  • on Linux: use your distributions package manager
  • Windows: coming soon

Requires vcpkg:

  • on Mac:
    $ git clone https://github.com/microsoft/vcpkg "$HOME/vcpkg"
    $ export VCPKG_ROOT="$HOME/vcpkg"
    (you may wish to add the last line to your .bashrc or .zshrc to make it permanent)
  • on Windows/Linux: vcpkg installation instructions

Linux/macOS

$ git clone --recursive https://github.com/ThunderboomRecords/WAIVE.git
$ cd WAIVE/

Copy the lib/ and include/ folders from the static built onnxruntime you downloaded in the prerequisite step into a new folder WAIVE/external/onnxruntime/, then from project root:

$ mkdir build
$ cmake --preset=default
$ cmake --build ./build -j8 --config Release

The plugins are found in build/bin folder. Move your prefered format binary to your plugins folder (see instructions above).

Windows

Coming soon.

Troubleshooting

WAIVE-Sampler not loading sources list

  • Make sure you are connected to the internet
  • Click "View Folder" button to open up the location the database is saved in you file browser. Delete the file WAIVE.db, close and remove the plugin from the track, and re-add it and re-open.
    • On macOS, this is located at /Users/[your username]/Library/Application Support/WAIVE

About

WAIVE-Plugins are built on top of several bespoke music and audio analysis models in order to create a unique and custom musical toolbox. These models will continue to be fine-tuned and updated on new data.

WAIVE-Midi utilises a flexible approach of modelling rhythmic data first outlined by Jon Gillick and collaborators. This works by encoding a percussion loop into it's Score and Groove components, then a model (a Variational AutoEncoder, VAE) is trained to combine them into a final output sequence. By adjusting the groove information, the same pattern (score) can be played in a different style. For the plugin, we also trained 2 smaller models (VAEs) that can encode the training data and can generate new Grooves and Scores in a variety of styles.

We use a combination of MIDI data and raw audio to collect rhythm information to train the models. In the case of raw audio, first the percussion instruments are isolated using demucs, then a method (Non-Negative Matrix Factorisation, demoed here) to further separate the individual percussion instruments is used to extract the score and groove information.

Rhythmic datasets include:

WAIVE-Sampler uses common techniques from machine listening to make measurements (called "features") of the audio, giving each sound a unique fingerprint and description of the audio content which can then be interpreted by other algorithms. At the same time, another method attempts to identify when percussive moments (called "onsets") occur using Complex Spectral Difference. The features are used to automatically identify appropriate moments in the Source audio that could be used for different drum hits, e.g. onsets that contain a broadband of frequency content can be shaped to make a snare sound.

For the Sample Map, the (log) Mel Spectrogram of the sample is measured, which computes a detailed representation of the frequency content of the audio. A simple, pretrained fully connected neural-net reads the measured features and outputs a 2D coordinate on where the audio sample should be placed on the map, e.g. such that kick drum sounds a grouped together, away from the hi-hat samples.

For the Tag cloud, we use an off-the-shelf word embedding model (GloVe) to produce a 2D representation (using TSNE dimensionality reduction) of all the tags in the database, such that semantically similar tags a clustered together to enable easier exploration of the available sources.

References

Licenses

About

Find out more on www.waive.studio Waive is an international collaboration between Thunderboom Records and four European heritage institutions: The Netherlands Institute for Sound and Vision, the BBC Archives, the Catalan Media Corporation and the Baltic Audiovisual Archival Council.

Resources

Stars

Watchers

Forks

Packages

No packages published

Languages