Arza

Monorepo for the Arza project

Table of content

• Table of content
• Installation
• Hardware
• Web client
  • Options
    • About filenamePattern
  • Note on static assets
• Generating the video
  • Install ImageMagick and verify your dependencies
  • Basic Fourier transform using ImageMagick
  • Generate the PNG sequence
  • Build the video from the PNG sequence
• Development
• License%

Installation

$ git clone https://github.com/chevalvert/arza && cd arza

Hardware

This project relies on a custom piece of hardware, which sniffs nearby WiFi probe requests, and send a fake keyPress on the client computer. It uses a NodeMCU in promiscuous mode, and a Trinket to simulate a keyboard.

See respectively hardware/nodemcu and hardware/trinket for instructions on how to flash the firmwares.

Web client

To run the web client in production, simply open client/build/index.html in any modern web browser (Google Chrome is recommended).

Options

Setting the web client options can be done it two ways:

• in development via client/src/pages/pages.js. This is the preferred way.
• in production, by editing directly the JSON object in client/build/index.html. This is not ideal but useful for in-situ fine tuning.

In both cases, the option object is described below:

{

  app: {
    logger: false, // Display logging informations, useful for debugging without devtools
    fullscreen: true
  },

  hardware: {
    key: 'p' // Key triggered by the Arza custom hardware
  },

  video: {
    source: 'arza-crop-pp.mp4',  // Source of the video
    passivePlaybackRate: 0.25,   // Playback rate of the video when no trigger
    activePlaybackRate: 10       // Playback rate of the video during a trigger
    triggerDuration: [500, 1000] // Range of the random duration for each trigger in ms
  },

  SfxHandler: {
    soundsLength: 4,       // Length of SFX sound files
    speakersLength: 6,     // Length of SFX spatialized versions
    maxConcurrentPlays: 6, // Do not play more than X sounds
    filenamePattern: 'sfx/{{speakerIndex:1}}_AZ_Sfx{{soundIndex:01}}.wav'
  },

  DroneHandler: {
    volume: 0.75,     // From 0 to 1
    soundsLength: 6,  // Length of Drone sound files
    filenamePattern: 'drones/Drones_AZ_{{soundIndex:01}}.wav'
  }
}

About filenamePattern

SFxHandler and DroneHandler options both have a filenamePattern key, defining how the filename is computed when fetched.

The basic syntax allows sequence definitions with custom n-based index and zero padding scheme:

{{index:1}}.wav   → 1.wav, 2.wav, 3.wav, …
{{index:0}}.wav   → 0.wav, 1.wav, 2.wav, …
{{index:01}}.wav  → 01.wav, 02.wav, 03.wav, …
{{index:000}}.wav → 000.wav, 001.wav, 002.wav, …

Note on static assets

Due to size limitations, static assets such as sounds and video are not put under version control. They will need to be placed manually in the production client/build/ directory, alongside client/build/index.html.

Generating the video

The video used by the client is built from a PNG sequence generated with ImageMagick with fftw.

Install ImageMagick and verify your dependencies

Refer to ImageMagick and fftw for the installation process.

$ convert --version
> Version: ImageMagick 6.9.10-23 Q16 x86_64 20190101 https://imagemagick.org
> …
> Delegates (built-in): … fftw png tiff …

Basic Fourier transform using ImageMagick

This is how the generation works:

# Resize input tif images
$ for i in {1..2}; do convert -monitor $i.tif -resize '1280>' $i.png; done

# FFT
$ for i in {1..2}; do convert -monitor $i.png -fft $i-fft.png; done

# This is just to see the spectrum, no need for computation
$ for i in {1..2}; do convert -monitor $i-fft-0.png -contrast-stretch 0 -evaluate log 10000 $i-fft-spectrum.png; done

# IFT
$ convert -monitor 1-fft-0.png 2-fft-1.png -ift arza.png

^{See Fourier Transform Processing With ImageMagick for an in-depth tutorial}

Generate the PNG sequence

# Convert sources tif to exploitable png
convert -monitor 'A.tif' -resize '1920>' 'A.png'
convert -monitor 'B.tif' -resize '1920>' 'B.png'

# Apply Fast Fourier Transform to the image A
convert -monitor 'A.png' -fft 'A-fft.png'

# Apply hue rotation (from 0 to 100%) to the B image before applying FFT to it and merging
# the two domains into the final frame
mkdir 'frames'

for v in {0..1000..1}; do
  echo -e "\n$v"
  # Float division may not work in all shells: it will work in zsh
  convert -monitor 'B.png' -modulate 100,100,$(($v / 10.0)) -fft 'B-fft.png'
  convert -monitor "A-fft-0.png" "B-fft-1.png" -ift "frames/$v.png"
done

Build the video from the PNG sequence

$ ffmpeg -framerate 24 -i 'frames/%d.png' -vf scale=1920:-1 -c:v libx264 -pix_fmt yuv420p 'arza.mp4'
$ ffmpeg -i 'arza.mp4' -filter:v 'crop=1920:1080:0:0' 'arza-cropped.mp4'
$ ffmpeg -i 'arza-cropped.mp4' -filter_complex '[0:v]reverse,fifo[r];[0:v][r] concat=n=2:v=1 [v]' -map '[v]' 'arza-cropped-ping-pong.mp4'

Development

$ npm install    # install all npm dependencies
$ npm run start  # start the web client dev server
$ npm run build  # build the web client in client/build

License

MIT.