A toolkit for the detection of Traveling Ionospheric Disturbances (TIDs) in the Earth's atmosphere as a result of tsunami waves, earthquakes, large explosions and other phenomena. A collaboration between individuals and teams the NASA Jet Propulsion Laboratory (JPL), Sapienza University of Rome and the University of California - Los Angeles (UCLA).
Top-Down View of TIDs Detected by Satellite over Hawaii
Tsunamis can trigger internal gravity waves (IGWs) that are able to propagate to the ionosphere, causing a perturbation in the natural Total Electron Content (TEC). These perturbations are often referred as Traveling Ionospheric Disturbances (TIDs) and are easily detectable through the Global Navigation Satellite System (GNSS) signal. The perturbation in the below image is generated by a TID.
GOPM Ground Station G07 Satellite slant Total Electron Content (sTEC) data
The large quantity of GNSS data currently available allows us to explore the possibility of using deep learning methods for TID detection. This toolkit demonstrates the effectiveness of training a convolutional neural network (CNN) to detect signs of IGWs. The approach utilized in this toolkit employs Gramian Angular Difference Fields (GADFs) to encode the time-series as images for model training.
Animation of GADFs over TIDs
In a real-time system, slant Total Electron Content (sTEC) may be calculated, windowed, converted to an image using GADFs and fed into a model for predicting whether a TID is present and occurring. Currently, simple experiments show conversion from float data to a prediction takes approximately 1.1. seconds.
First, setup a Python virtual environment in which to install project dependencies and then install the library. Then, pull the appropriate branch and then install:
pip install .
Make sure to check out the notebooks and data directories
which contain Jupyter notebooks with latest work and the source data
used in the experiments.
This project requires that Python version is versioned between 3.5 and
3.8. Requirements for the software, running tests, and notebooks are
listed in requirements.txt, requirements_ci.txt, and requirements_notebooks.txt,
and may be installed into a virtual environment in the following way:
pip install -r requirements.txt
pip install -r requirements_ci.txt # for unit tests Some of the visualizations within the Jupyter notebooks require the geos library, which can be installed using homebrew on macOS:
brew install geos
On a linux machine, GEOS can be installed without root access if working in Anaconda environments:
conda install -c anaconda geos
Then, install the notebook dependencies:
pip install -r requirements_notebooks.txt
The data includes data from both a 2012 Hawaii tsunami event and
a 2015 Chilean earthquake.
Within the the data directory there is from each event, organized
by subdirectories which correspond to the year and day of year.
First, data must be downloaded and unpacked into the data directory:
curl -O https://tsunami-detection.s3-us-west-1.amazonaws.com/data.tar.gz && tar -xvzf data.tar.gz && rm data.tar.gzData used for experiments is located in data/experiments, while raw historical
data is available in the chile and hawaii directories. The data is also used in a variety of tests.
In every folder, you find a file for each satellite in view from a GPS station. The value of the slant total electron content (sTEC) encountered by the GPS signal during its path in the ionosphere from the satellite (e.g G10) to the GPS receiver (e.g ahup) is available for select days of the year.
The data files have 7 columns:
- Sod: it represents the second of the day, it is my time array
- dStec/dt: the variations in time of the slant total electron content (the parameter of interest) epoch by epoch (it is like a velocity)
- Lon: longitude epoch by epoch of the IPP, the point to which we refer the sTEC estimations
- Lat: latitude epoch by epoch of the IPP, the point to which we refer the sTEC estimations
- Hipp: height epoch by epoch of the IPP, the point to which we refer the sTEC estimations
- Azi: the azimuth of the satellite epoch by epoch
- Ele: the elevation of the satellite epoch by epoch (usually we don’t consider data with elevation under 20 degrees since they are too noisy)
| Metric | Validation Score |
|---|---|
| recall | 84.6% |
| precision | 100% |
| F1 Score | 91.7% |
Please use the issue tracker to report any erroneous behavior or desired feature requests.
If you would like to contribute to development, please fork the repository and make
any changes to a branch which corresponds to an open issue. Hot fixes
and bug fixes can be represented by branches with the prefix fix/ versus
feature/ for new capabilities or code improvements. Pull requests will
then be made from these branches into the repository's dev branch
prior to being pulled into master. Pull requests which are works in
progress or ready for merging should be indicated by their respective
prefixes ([WIP] and [MRG]). Pull requests with the [MRG] prefix will be
reviewed prior to being pulled into the master branch.
When contributing, please ensure to run unit tests and add additional tests as
necessary if adding new functionality. To run the unit tests, use pytest:
python -m pytest --cov=tidd --capture=no --log-cli-level=CRITICAL
This should report the result of your unit tests as well as information about code coverage.
Semantic versioning is used for this project. If contributing, please conform to semantic versioning guidelines when submitting a pull request.
Core contributors are responsible for maintaining changelog.md in
coordination with new releases.
To release a new version of the software, simply tag the realse after building the distribution and wheels:
python setup.py sdist bdist_wheel
git tag x.x.x -m 'some commit message here'
git push origin --tags <branch_name>
git add dist/
git push origin <branch_name>This project is licensed under the Apache 2.0 license.
If citing this work, use the following:
@inproceedings{Constantinou2023,
doi = {10.1109/igarss52108.2023.10282501},
url = {https://doi.org/10.1109/igarss52108.2023.10282501},
year = {2023},
month = jul,
publisher = {{IEEE}},
author = {Valentino Constantinou and Michela Ravanelli and Hamlin Liu and Jacob Bortnik},
title = {A Deep Learning Approach for Detection of Internal Gravity Waves in Earth's Ionosphere},
booktitle = {{IGARSS} 2023 - 2023 {IEEE} International Geoscience and Remote Sensing Symposium}
}
- Real-Time Detection of Tsunami Ionospheric Disturbances with a Stand-Alone GNSS Receiver: A Preliminary Feasibility Demonstration. Savastano G., et. al.. Nature Scientific Reports, 2017. PDF.
- Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholding. Hundman K., et. al. Knowledge Discovery and Data Mining (KDD), 2018. PDF.
- Imaging Time-Series to Improve Classification and Imputation. Whang Z., Oats T. 2015. PDF.