Skip to content
Mark Jessop edited this page May 20, 2021 · 133 revisions

auto_rx logo

Automatic Radiosonde Receiver Utilities

This fork of rs1279's RS codebase provides a set of utilities ('auto_rx') to allow automatic reception and uploading of Radiosonde positions to multiple services, including:

There is also a web interface available on http://localhost:5000 , allowing display of station status and basic tracking of the sonde position.

Currently we support the following radiosonde types:

  • Vaisala RS92
  • Vaisala RS41
  • Graw DFM06/DFM09/DFM17/PS-15
  • Meteomodem M10 & M20
  • Intermet iMet-4 (and 'narrowband' iMet-1 sondes)
  • Intermet iMet-54
  • Lockheed Martin LMS6 (400 MHz and 1680 MHz variants, including the new 'LMS-X' unreleased type)
  • Meisei iMS-100
  • Meteo-Radiy MRZ (400 MHz variants)

Support for other radiosondes may be added as required - please send us sondes to test with!

This software is under regular development. Please update regularly to get bug-fixes and improvements!

Improvements from the upstream RS codebase will be merged into this codebase when/where appropriate. A big thanks to rs1729 for continuing to develop and improve these decoders, and working with us to make auto_rx decode all the radiosondes!

Please consider joining the Google Group to receive updates on new software features: https://groups.google.com/forum/#!forum/radiosonde_auto_rx

Presentations

Contacts

You can often find us in the #highaltitude IRC Channel on Freenode.

Licensing Information

All software within this repository is licensed under the GNU General Public License v3. Refer this repositories LICENSE file for the full license text.

Radiosonde telemetry data captured via this software and uploaded into the Sondehub Database system is licensed under Creative Commons BY-SA v2.0. Telemetry data uploaded into the APRS-IS network is generally considered to be released into the public domain.

By uploading data into these systems (by enabling the relevant uploaders within the station.cfg file) you as the user agree for your data to be made available under these licenses. Note that uploading to Sondehub is enabled by default.

Latest Updates

Refer to the Releases for information on the latest updates.

Please note that packets from versions older than v1.3.0 are now being blocked from entering HabHub, due to changes to the uploaded packet contents. Please update!

1. General Theory of Operation

This software performs the following steps:

  1. Use rtl_power to scan across a user-defined frequency range, and detect peaks in the spectrum.
  2. For each detected peak frequency, run the rs_detect utility, which determines if a radiosonde signal is present, and what type it is. This detection is performed using correlation, and will detect a sonde many dB below the threshold where a decode is possible.
  3. If a radiosonde signal is found, start demodulating it, and upload data to various internet services.
  4. If no peaks are found, or if no packets are heard from the radiosonde in a given amount of time (3 minutes by default), go back to step 1.

The latest versions can make use of multiple RTLSDRs to allow for tracking of many radiosondes simultaneously. The number of simultaneous radiosondes you can track is limited only by the number of RTLSDRs you have setup!

By running auto_rx continuously, not just at known radiosonde launch times, you may see other radiosonde launches (military or otherwise) that would otherwise go un-noticed. Here in Australia, we discovered the Bureau of Meteorology's Ozone sounding schedule this way (which was later confirmed by the Bureau - thanks guys!).

2. Hardware Requirements

  • Ideally, a standalone single-board computer (Raspberry Pi 3 or newer, ODroid, etc...) to run the software. Other Linux machines will also work. It will not work under WSL.
  • A RTLSDR receiver.
    • A receiver with a TCXO is mandatory, and you must make sure the PPM offset value is set correctly (see here). The demodulators are quite sensitive to frequency offsets, and the warm-up drift of the non-TCXO RTLSDRs is enough to throw it off. I've had great success with the RTL-SDR.com 'v3' dongles. The NooElec SMArt dongles (with the TCXO option) are also good.
  • An antenna suitable for receiving on 400-406 MHz (a basic 1/4 wave monopole is usually good enough). If you have an 70cm amateur-band antenna, that will probably work fine too.
  • For optimal receive performance, a preamplifier and a band-pass filter, like this one. You can still achieve pretty good results without one.
  • For those in the US trying to receive 1680 MHz LMS6 sondes, a higher gain antenna (short helical) and a preamp is required.

3. Setup & Configuration - Docker

Probably the easiest way to get auto_rx going is using Docker - A guide on setting up using Docker is available here: https://github.com/projecthorus/radiosonde_auto_rx/wiki/Docker

If you want to run auto_rx 'natively', then continue on with the guide below.

4. Setup & Configuration - 'Native'

This section is intended to assist with setup under a fresh Raspbian installation (Raspbian Buster or newer). The instructions should be usable on other Debian-based systems, noting that we require at least Python 3.6.

For a good guide on setting up a 'headless' (no display) Raspberry Pi 2/3/4, look here.

4.1. Software Dependencies

Update your system, then install the required dependencies using:

$ sudo apt-get update
$ sudo apt-get upgrade
$ sudo apt-get install python3 python3-numpy python3-setuptools python3-crcmod python3-requests python3-dateutil python3-pip python3-flask sox git build-essential libtool cmake usbutils libusb-1.0-0-dev rng-tools libsamplerate-dev libatlas3-base libgfortran5

4.2. RTL-SDR Utilities

As we may we wish to use the bias-tee option in the newer v3 RTLSDRs, we need a fairly recent version of the rtl-sdr software. Recent ubuntu distributions provide this support in the rtl-sdr package (package version 0.5.3-12 or newer), and as of 2020, Raspbian supports it too!

This means that you can probably just install the rtl-sdr utilities by running:

$ sudo apt-get install rtl-sdr

If you install via the system package manager, then most likely the required udev rules (which enable the pi user to access the RTLSDR) will not NOT have been installed (... sigh). To install these, run:

$ sudo wget -O /etc/udev/rules.d/20-rtlsdr.rules https://raw.githubusercontent.com/osmocom/rtl-sdr/master/rtl-sdr.rules

Then, reboot the RPi.

4.2.1 Installing rtl-sdr from source (if required)

If your package manager's version of rtl-sdr does not provide bias-tee support (no -T option), then you will need to compile from source.

IMPORTANT NOTE: Make sure to remove any installations of rtl-sdr already on your system before compiling/installing from source. (i.e. sudo apt-get remove rtl-sdr librtlsdr0 librtlsdr-dev

$ git clone https://github.com/osmocom/rtl-sdr.git
$ cd rtl-sdr
$ mkdir build
$ cd build
$ cmake -DINSTALL_UDEV_RULES=ON -DDETACH_KERNEL_DRIVER=ON ../
$ sudo make install
$ sudo ldconfig

4.2.2 RTL-DVB Kernel Module Blacklisting

Because the kernel rtl_dvb driver (yes, these SDRs can be used to watch TV after all) will inhibit our intended use, we need to blacklist a few kernel modules.

If you installed rtl-sdr from source as above, then a blacklist file should have already been installed, otherwise create a new module blacklist file using:

$ sudo nano /etc/modprobe.d/rtlsdr-blacklist.conf

and add the lines:

blacklist dvb_usb_rtl28xxu
blacklist rtl2832
blacklist rtl2830
blacklist dvb_usb_rtl2832u
blacklist dvb_usb_v2
blacklist dvb_core

Save the file (Ctrl+x, y), and reboot the Raspberry Pi.

4.2.3 - Check your RTLSDRs are working!

At this point it is worth checking that you can communicate with your RTLSDR, which can be achieved by running:

$ rtl_test

Found 1 device(s):
  0:  Realtek, RTL2838UHIDIR, SN: 00000002

Using device 0: Generic RTL2832U OEM
Found Rafael Micro R820T tuner
Supported gain values (29): 0.0 0.9 1.4 2.7 3.7 7.7 8.7 12.5 14.4 15.7 16.6 19.7 20.7 22.9 25.4 28.0 29.7 32.8 33.8 36.4 37.2 38.6 40.2 42.1 43.4 43.9 44.5 48.0 49.6
[R82XX] PLL not locked!
Sampling at 2048000 S/s.

Info: This tool will continuously read from the device, and report if
samples get lost. If you observe no further output, everything is fine.

Reading samples in async mode...

Hit Ctrl+C to kill rtl_test. If you see lots of warnings such as lost at least <X> bytes, this indicates USB bandwidth issues, or some other issue with the RTLSDR. One or two lines are usually OK...

4.3. Cloning & Configuring the Radiosonde Auto RX Repository

From your home directory, run:

$ git clone https://github.com/projecthorus/radiosonde_auto_rx.git
$ cd radiosonde_auto_rx/auto_rx
$ ./build.sh
$ cp station.cfg.example station.cfg

Edit station.cfg (i.e. nano station.cfg) and modify the settings as required. For detailed information on configuration settings, please read through the Configuration Settings page.

Note that after major updates, you will usually need to re-copy the station.cfg.example file, as new options will have been added.

4.4. Python Dependencies

We now need to grab a few additional packages using Pip. (This needs to be run from within the auto_rx directory we just cd'd into before)

$ sudo pip3 install -r requirements.txt

A few errors may be shown while pip does it's thing. The packages we need should still be installed however.

4.4. Initial Testing

To check everything is operational, start up the automatic receiver script using:

(from within radiosonde_auto_rx/auto_rx)
$ python3 auto_rx.py

Depending on how you have configured the software, you should see output similar to the following:

2020-12-09 09:02:57,021 INFO:Reading configuration file...
2020-12-09 09:03:00,347 INFO:Config - Tested SDR #0 OK
2020-12-09 09:03:00,353 INFO:Started Flask server on http://0.0.0.0:5000
 * Serving Flask app "autorx.web" (lazy loading)
2020-12-09 09:03:00,356 INFO:Telemetry Logger - Started Telemetry Logger Thread.
 * Environment: production
   WARNING: Do not use the development server in a production environment.
   Use a production WSGI server instead.
2020-12-09 09:03:00,358 INFO:OziMux - Started OziMux / Payload Summary Exporter
 * Debug mode: off
2020-12-09 09:03:00,830 INFO:Version - Local Version: 1.3.2  Current Master Version: 1.3.2
2020-12-09 09:03:00,831 INFO:Task Manager - SDR #0 has been allocated to Scanner.
2020-12-09 09:03:04,102 INFO:Scanner #0 - Starting Scanner Thread
2020-12-09 09:03:04,112 INFO:Scanner #0 - Running frequency scan.

This indicates that the software is successfully scanning for radiosondes. The rest of the functionality can only easily be tested with a signal from a radiosonde. These are launched at around 11:15Z and 23:15Z at most major airports around the world.

When a sonde is found, the following output will be seen:

2018-06-01 21:56:12,077 INFO:Scanner #0 - Detected peaks on 1 frequencies (MHz): [402.5]
2018-06-01 21:56:17,521 INFO:Detected new RS41 sonde on 402.500 MHz!
2018-06-01 21:56:17,521 INFO:Halting Scanner to decode detected radiosonde.
2018-06-01 21:56:17,521 INFO:Scanner #0 - Waiting for current scan to finish...
2018-06-01 21:56:25,686 INFO:Scanner #0 - Scanner Thread Closed.
2018-06-01 21:56:25,687 INFO:SDR #0 has been allocated to Decoder (RS41, 402.500 MHz).
2018-06-01 21:56:27,556 INFO:Decoder #0 RS41 402.500 - Starting decoder subprocess.
2018-06-01 21:56:30,868 INFO:Telemetry Logger - Opening new log file: ./log/20180601-113422_N3740572_RS41_402500_sonde.log
2018-06-01 21:56:37,172 INFO:Habitat - Listener information uploaded.
2018-06-01 21:56:41,747 INFO:APRS-IS - Uploaded to APRS-IS: ;N3740572 *111111z3509.05S/13854.80EO116/046/A=053079 RS41 Radiosonde 402.500 MHz 5.2m/s http://bit.ly/2Bj4Sfk
...

Depending on what exporter options have been enabled (Habitat, APRS, OziPlotter), you will see indications when telemetry has been uploaded. If no exporters are enabled, you will only see telemetry information with the 'verbose' logging option enabled (-v).

A web interface which displays the log information as above, along with a sonde position map and frequency scan output, is available on port 5000, i.e. http://localhost:5000/ (Or replace localhost with your RPi's IP address if accessing from another computer).

To shut-down the automatic reception script hit Ctrl-C, and the threads will eventually close.

When a Radiosonde is successfully detected, a log-file will be created in the auto_rx/log/ directory, containing the telemetry for that flight.

4.5. Automatic / Continuous Operation

4.5.1. Option #1 - Operation as a systemd Service (RECOMMENDED)

auto_rx can be operated in a 'continuous' mode, running as a systemd service.

To set this up, the auto_rx.service file (located in radiosonde_auto_rx/auto_rx) must be edited to include your username, and the path to this directory.

$ sudo cp auto_rx.service /etc/systemd/system/
$ sudo nano /etc/systemd/system/auto_rx.service

If you are not running auto_rx as the 'pi' user, you will need to edit the auto_rx.service file and modify the ExecStart, WorkingDirectory and User fields. Otherwise, leave all settings at their defaults:

[Unit]
Description=auto_rx
After=syslog.target

[Service]
ExecStart=/usr/bin/python3 /home/pi/radiosonde_auto_rx/auto_rx/auto_rx.py -t 0
Restart=always
RestartSec=120
WorkingDirectory=/home/pi/radiosonde_auto_rx/auto_rx/
User=pi
SyslogIdentifier=auto_rx

[Install]
WantedBy=multi-user.target

If you are behind a proxy server, then you can add the following lines into the [Service] section to use it:

Environment=HTTP_PROXY=http://your.proxy.here:8080
Environment=HTTPS_PROXY=http://your.proxy.here:8080

Once/if edited, install and start the service using:

$ sudo systemctl enable auto_rx.service
$ sudo systemctl start auto_rx.service

The log output can be viewed by running:

$ sudo journalctl -u auto_rx.service -f -n

To stop the service, simply run:

$ sudo systemctl stop auto_rx.service

4.5.2. Option #2 - Operation via Crontab (Backup option if systemd not available - not recommended)

As radiosonde launch times are quite predictable, auto_rx.py can be started around launch-time using a crontab entry.

Note that when a process runs within cron, it doesn't have access to the user's $PATH environment variable, and hence utilities like rtl_power and rtl_fm will not be found. This can be fixed by declaring the PATH environment variable within the crontab. An example crontab is below:

PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/games:/usr/games`
15 11 * * * /home/<your_username>/radiosonde_auto_rx/auto_rx/auto_rx.sh
15 23 * * * /home/<your_username>/radiosonde_auto_rx/auto_rx/auto_rx.sh

Note: The auto_rx.sh script performs the following actions:

  • Changes directory into your auto_rx dir, then
  • Starts auto_rx.py with a 3 hour timeout, which means auto_rx will exit after 3 hours regardless of if any telemetry is being received.

5 - Updates

Check the releases page occasionally for information on new updates to the software. Refer to this page for information on how to perform updates to auto_rx.

Sometimes we will block telemetry uploads to HabHub from older versions, to avoid causing issues when we change things like uploaded telemetry formats.