Skip to content

TarsosDSP, a Real-Time Audio Processing Framework in Java

License

Notifications You must be signed in to change notification settings

srubin/TarsosDSP

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Note: you don’t need to use this fork to run TarsosDSP on Android. As of version 2.0, TarsosDSP runs on Android natively. See http://0110.be/posts/TarsosDSP_on_Android_-_Audio_Processing_in_Java_on_Android

TarsosDSP

TarsosDSP is a Java library for audio processing. Its aim is to provide an easy-to-use interface to practical music processing algorithms implemented, as simply as possible, in pure Java and without any other external dependencies. TarsosDSP features an implementation of a percussion onset detector and a number of pitch detection algorithms: YIN, the Mcleod Pitch method and a “Dynamic Wavelet Algorithm Pitch Tracking” algorithm. Also included is a Goertzel DTMF decoding algorithm, a time stretch algorithm (WSOLA), resampling, filters, simple synthesis, some audio effects, and a pitch shifting algorithm.

To show the capabilities of the library, TarsosDSP example applications are available. Head over to the TarosDSP release directory for freshly baked binaries and code smell free (that is the goal anyway), oven-fresh sources.

Some information about TarsosDSP can be found in the paper TarsosDSP, a Real-Time Audio Processing Framework in Java, by Joren Six, Olmo Cornelis, and Marc Leman, in Proceedings of the 53rd AES Conference (AES 53rd), 2014. If you use TarsosDSP in academic research, please cite this paper.

@inproceedings{six2014tarsosdsp,
  author      = {Joren Six and Olmo Cornelis and Marc Leman},
  title       = {{TarsosDSP, a Real-Time Audio Processing Framework in Java}},
  booktitle   = {{Proceedings of the 53rd AES Conference (AES 53rd)}}, 
  year        =  2014
}

Quickly Getting Started with TarsosDSP

Head over to the TarsosDSP release repository and download the latest TarsosDSP library. To get up to speed quickly, check the TarsosDSP Example applications for inspiration and consult the API documentation.

TarsosDSP Example Applications

TarsosDSP contains some ready made example applications. Most have a Java Swing user interface. They show which functionality is present in the library and how to use it.

  • SoundDetector show how you loudness calculations can be done. When input sound is over a defined limit an event is fired.
  • PitchDetector this demo application shows real-time pitch detection. When pitch is detected the hertz value is printed together with a probability.
  • PercussionDetector show the percussion (onset) dectection. Clapping your hands causes an event. This demo application also shows the influence of the two parameters on the algorithm.
  • UtterAsterisk a game with the goal to sing as close to a melody a possible. Technically it shows real-time pitch detection with YIN or MPM.
  • Spectrogram in Java shows a spectrogram and detected pitch, either live or from an audio file. It is interesting to see which frequencies are picked as fundamentals.
  • Goertzel DTMF decoding an implementation of the Goertzel Algorithm. A fancy user interface shows what goes on under the hood.
  • Audio Time Stretching – Implementation in Pure Java Using WSOLA an implementation of a time stretching algorithm. WSOLA makes it possible to change the play back speed of audio without changing the pitch. The play back speed can be changed at any moment, even when there is audio playing.
  • Audio Feature Extraction a command line application to do simple feature extraction.
  • Audio Synthesis a command line application to do simple audio synthesis.
  • Pitch Shifting an example application that does pitch shifting, either in real-time on a microphone input, or on recorded audio. Also included is a command line application to do pitch shifting.

Developing TarsosDSP

If you want to build from source, or want to improve TarsosDSP follow the instructions below. Contributions to TarsosDSP are more than welcome, if you have a an algorithm to add or find a bug, do not hesitate to send me a message.

TarsosDSP uses Apache Ant as a build system. The instructions below detail how you can build from source. When everything runs correctly you should be able to run all example applications and have the latest version of the TarsosDSP library for inclusion in your projects. Also the Javadoc documentation for the API should be available.

TarsosDSP with Ant

To you need Apache Ant and git installed on your system. The following commands fetch the source and build the library and example jars:

git clone https://JorenSix@github.com/JorenSix/TarsosDSP.git
cd TarsosDSP
cd build
ant tarsos_dsp_library #Builds the core TarsosDSP library
ant build_examples #Builds all the TarsosDSP examples
ant javadoc #Creates the documentation in TarsosDSP/doc

Source Code Organization & Developing

The library is separated into three source folders: 1) the main core functionality in src, TarsosDSP example applications in examples and unit tests in test.

  • src contains the source files of the core DSP libraries.
  • test contains unit tests for some of the DSP functionality.
  • examples contains a couple of example applications with a Java Swing user interface.
  • build contains ANT build files. Either to build Java documentation or runnable JAR-files for the example applications.

To make development with Eclipse easy, make sure both test, src and examples are marked as source folders.

Credits

TarsosDSP was developed at University College Ghent, School of Arts between 2009 and 2013, from late 2013 the project is supported by University Ghent, IPEM.

The TarsosDSP borrows algorithms from various other libraries or research paper. Below a complete list of credits can be found.

Changelog

Version 1.0
2012-04-24

First release which includes several pitch trackers and a time stretching algorithm, amongst other things. Downloads and javadoc API can be found at the TarsosDSP release directory

Version 1.1
2012-06-4

Changed how the audio dispatcher stops. Added StopAudioProcessor.
Added FastYin implementation by Matthias Mauch
Added AMDF pitch estimator by Eder Souza

Version 1.2
2012-08-21

Modified the interface of PitchDetector to return a more elaborate result structure with pitch, probability and a boolean “is pitched”.
Added an implementation of an envelope follower or envelope detector.

Version 1.3
2012-09-19

TarsosDSP can do audio synthesis now. The first simple unit generators are included in the library.
It has a new audio feature extraction feature, implemented in the FeatureExtractor example.
Added ASCII-art to the source code (this is the main TarsosDSP 1.3 feature).

Version 1.4
2012-10-31

Included a resample feature, implemented by libresample4j. Together with the WSOLA implementation, it can be used for pitch shifting (similar to Phase Vocoding). A pitch shifting example (both with a CLI and a UI) is added in the 1.4 version of the TarsosDSP library as well.

Version 1.5
2013-04-30

Converted TarsosDSP to maven. This is known as the Malaryta-release. The “Malaryta” release is provided to you by RikkiMongoose (idea, documents, git things) and Ultar (converting to maven, refactoring). Malaryta is the capital of Malaryta Raion, Brest Region in the Republic of Belarus. Both of developers spent their childhood in Brest, and think that title Malaryta is as strange as Ubuntu or Whistler. The 1.5 release also includes various FFT window functions from the cool Minim project by Damien Di Fede.

Version 1.6
2013-06-12

This release features practical onset and beat detection algorithms. A complex domain onset detection and a spectral flux onset detection algorithm are added. This release also includes a way to guess a beat from onsets. Parts of the BeatRoot system, by Simon Dixon, are included to this end. Also included in this release is an implementation of the Constant-Q transform.

Version 1.7
2013-10-08

This release adds the ability to extract the MFCC from an audio signal. Also an example of the Constant-Q transform is added, together with a reusable visualization class library. The build system is reverted back to pure ANT.