Skip to content

Commit

Permalink
v0.6.0
Browse files Browse the repository at this point in the history
  • Loading branch information
ereoh committed Oct 21, 2024
1 parent 5e0e37e commit 850c7b1
Show file tree
Hide file tree
Showing 577 changed files with 19,201 additions and 1,446 deletions.
15 changes: 4 additions & 11 deletions .github/PULL_REQUEST_TEMPLATE/pull_request_template.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,18 +7,11 @@ Describe your changes here specifying if the change is a bug fix, enhancement, n
Describe how you tested and verified your changes here (changes captured in existing tests, built and ran new tests, etc.).

## Before Submitting
- [ ] Check mypy locally
- `pip3 install mypy==1.2.0`
- `mypy --ignore-missing-imports torchsig`
- Address any error messages
- [ ] Lint check locally
- `pip3 install flake8`
- `flake8 --select=E9,F63,F7,F82 torchsig`
- Address any error messages
- [ ] Check for bugs/errors
- [ ] Run example notebooks
- `examples/`
- Ensure all notebooks run successfully.
- [ ] Run formatter if needed
- `pip3 install git+https://github.com/GooeeIOT/pyfmt.git`
- `pyfmt torchsig`
- [ ] Run test suite locally
- `pytest --ignore-glob=*_figures.py --ignore-glob=*_benchmark.py`
- Ensure tests are successful prior to submitting PR

33 changes: 22 additions & 11 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,17 +1,28 @@
# Common Python Files
__pycache__/
*.pyc
*.mdb
*.swp
torchsig.egg-info
.vscode/
*.png
build/
checkpoints/
lightning_logs/
*.pt
*.jpg
*.benchmarks/
*.ipynb_checkpoints/
dist/
examples/*.ipynb_checkpoints/
gr-spectrumdetect/examples/*.pt
**.ipynb_checkpoints**
**.pytest_cache
*.benchmarks/

**.pyc
**.mdb
**.lmdb
**.swp
**.png
**.txt
**.npy
**.jpg
**.pt*
**.pth

# TorchSig
torchsig.egg-info
examples/runs
examples/datasets
examples/yolo
examples/plots
109 changes: 62 additions & 47 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,74 +6,61 @@
</a>

-----
![build](https://github.com/torchDSP/torchsig/actions/workflows/pip_build.yml/badge.svg?branch=37-automate-install-tests)

[TorchSig](https://torchsig.com) is an open-source signal processing machine learning toolkit based on the PyTorch data handling pipeline. The user-friendly toolkit simplifies common digital signal processing operations, augmentations, and transformations when dealing with both real and complex-valued signals. TorchSig streamlines the integration process of these signals processing tools building on PyTorch, enabling faster and easier development and research for machine learning techniques applied to signals data, particularly within (but not limited to) the radio frequency domain. An example dataset, TorchSigNarrowband, based on many unique communication signal modulations is included to accelerate the field of modulation classification. Additionally, an example wideband dataset, TorchSigWideband, is also included that extends TorchSigNarrowband with larger data example sizes containing multiple signals enabling accelerated research in the fields of wideband signal detection and recognition.

[TorchSig](https://torchsig.com) is an open-source signal processing machine learning toolkit based on the PyTorch data handling pipeline. The user-friendly toolkit simplifies common digital signal processing operations, augmentations, and transformations when dealing with both real and complex-valued signals. TorchSig streamlines the integration process of these signals processing tools building on PyTorch, enabling faster and easier development and research for machine learning techniques applied to signals data, particularly within (but not limited to) the radio frequency domain. An example dataset, Sig53, based on many unique communication signal modulations is included to accelerate the field of modulation classification. Additionally, an example wideband dataset, WidebandSig53, is also included that extends Sig53 with larger data example sizes containing multiple signals enabling accelerated research in the fields of wideband signal detection and recognition.
# Getting Started

*TorchSig is currently in beta*
## Prerequisites
- Ubuntu &ge; 20.04
- Hard drive storage with:
- &ge; 500 GB for Narrowband
- &ge; 10 TB for Wideband
- CPU with &ge; 4 cores
- GPU with &ge; 16 GB storage (reccomended)
- Python &ge; 3.9

## Key Features
---
TorchSig provides many useful tools to facilitate and accelerate research on signals processing machine learning technologies:
- The `SignalData` class and its `SignalMetadata` objects enable signals objects and meta data to be seamlessly handled and operated on throughout the TorchSig infrastructure.
- The `Sig53` Dataset is a state-of-the-art static modulations-based RF dataset meant to serve as the next baseline for RFML classification development & evaluation.
- The `ModulationsDataset` class synthetically creates, augments, and transforms the largest communications signals modulations dataset to date in a generic, flexible fashion.
- The `WidebandSig53` Dataset is a state-of-the-art static wideband RF signals dataset meant to serve as the baseline for RFML signal detection and recognition development & evaluation.
- The `WidebandModulationsDataset` class synthetically creates, augments, and transforms the largest wideband communications signals dataset in a generic, flexible fashion.
- Numerous signals processing transforms enable existing ML techniques to be employed on the signals data, streamline domain-specific signals augmentations in signals processing machine learning experiments, and signals-specific data transformations to speed up the field of expert feature signals processing machine learning integration.
- TorchSig also includes a model API similar to open source code in other ML domains, where several state-of-the-art convolutional and transformer-based neural architectures have been adapted to the signals domain and pretrained on the `Sig53` and `WidebandSig53` datasets. These models can be easily used for follow-on research in the form of additional hyperparameter tuning, out-of-the-box comparative analysis/evaluations, and/or fine-tuning to custom datasets.


## Documentation
---
Documentation can be found [online](https://torchsig.readthedocs.io/en/latest/) or built locally by following the instructions below.
```
cd docs
pip install -r docs-requirements.txt
make html
firefox build/html/index.html
```
We highly reccomend Ubuntu or using a Docker container.

## Installation
---
Clone the `torchsig` repository and simply install using the following commands:
Clone the `torchsig` repository and install using the following commands:
```
git clone https://github.com/TorchDSP/torchsig.git
cd torchsig
pip install .
```

## Generating the Datasets
If you'd like to generate the named datasets without messing with your current Python environment, you can build the development container and use it to generate data at the location of your choosing.
# Usage

## Generating the Datasets with Command Line
To create the narrowband dataset:
```
docker build -t torchsig -f Dockerfile .
docker run -u $(id -u ${USER}):$(id -g ${USER}) -v `pwd`:/workspace/code/torchsig torchsig python3 torchsig/scripts/generate_sig53.py --root=/workspace/code/torchsig/data --all=True
python3 ./scripts/generate_narrowband.py --root ./examples/datasets --all --num-workers=4
```
To create the wideband dataset:
```
python3 ./scripts/generate_wideband.py --root ./examples/datasets --all --num-workers=4
```

For the wideband dataset, you can do:
## Generating the Datasets in Docker
Docker can be used to generate the datasets without modifying your current Python environment. Build a Docker container:

```
docker build -t torchsig -f Dockerfile .
docker run -u $(id -u ${USER}):$(id -g ${USER}) -v `pwd`:/workspace/code/torchsig torchsig python3 torchsig/scripts/generate_wideband_sig53.py --root=/workspace/code/torchsig/data --all=True
```

If you do not need to use Docker, you can also just generate using the regular command-line interface

To create the narrowband dataset with the Docker container:
```
python3 torchsig/scripts/generate_sig53.py --root=torchsig/data --all=True
docker run -u $(id -u ${USER}):$(id -g ${USER}) -v `pwd`:/workspace/code/torchsig torchsig python3 torchsig/scripts/generate_narrowband.py --root=/workspace/code/torchsig/data --all
```

or for the wideband dataset:

To create the wideband dataset with the Docker container:
```
python3 torchsig/scripts/generate_wideband_sig53.py --root=torchsig/data --all=True
docker run -u $(id -u ${USER}):$(id -g ${USER}) -v `pwd`:/workspace/code/torchsig torchsig python3 torchsig/scripts/generate_wideband.py --root=/workspace/code/torchsig/data --all
```

Then, be sure to point scripts looking for ```root``` to ```torchsig/data```.

## Using the Dockerfile
If you have Docker installed along with compatible GPUs and drivers, you can try:
## Jupyter Notebook Examples with Docker and GPUs
The example jupyter notebooks can be run within Docker with GPU support, try the command:

```
docker build -t torchsig -f Dockerfile .
Expand All @@ -83,13 +70,41 @@ docker exec torchsig_workspace jupyter notebook --allow-root --ip=0.0.0.0 --no-b

Then use the URL in the output in your browser to run the examples and notebooks.

## License
---

# Key Features
TorchSig provides many useful tools to facilitate and accelerate research on signals processing machine learning technologies:
- The `SignalData` class and its `SignalMetadata` objects enable signals objects and meta data to be seamlessly handled and operated on throughout the TorchSig infrastructure.
- The `TorchSigNarrowband` Dataset is a state-of-the-art static modulations-based RF dataset meant to serve as the next baseline for RFML classification development & evaluation.
- The `ModulationsDataset` class synthetically creates, augments, and transforms the largest communications signals modulations dataset to date in a generic, flexible fashion.
- The `TorchSigWideband` Dataset is a state-of-the-art static wideband RF signals dataset meant to serve as the baseline for RFML signal detection and recognition development & evaluation.
- The `WidebandModulationsDataset` class synthetically creates, augments, and transforms the largest wideband communications signals dataset in a generic, flexible fashion.
- Numerous signals processing transforms enable existing ML techniques to be employed on the signals data, streamline domain-specific signals augmentations in signals processing machine learning experiments, and signals-specific data transformations to speed up the field of expert feature signals processing machine learning integration.
- TorchSig also includes a model API similar to open source code in other ML domains, where several state-of-the-art convolutional and transformer-based neural architectures have been adapted to the signals domain and pretrained on the `TorchSigNarrowband` and `TorchSigWideband` datasets. These models can be easily used for follow-on research in the form of additional hyperparameter tuning, out-of-the-box comparative analysis/evaluations, and/or fine-tuning to custom datasets.


# Documentation
Documentation can be found [online](https://torchsig.readthedocs.io/en/latest/) or built locally by following the instructions below.
```
cd docs
pip install -r docs-requirements.txt
make html
firefox build/html/index.html
```


# License
TorchSig is released under the MIT License. The MIT license is a popular open-source software license enabling free use, redistribution, and modifications, even for commercial purposes, provided the license is included in all copies or substantial portions of the software. TorchSig has no connection to MIT, other than through the use of this license.

# Publications
| Title | Year | Cite (APA) |
| ----- | ---- | ---------- |
| [TorchSig: A GNU Radio Block and New Spectrogram Tools for Augmenting ML Training](https://events.gnuradio.org/event/24/contributions/628/attachments/190/473/TorchSig_GRCon2024_paper.pdf) | 2024 | Vallance, P., Oh, E., Mullins, J., Gulati, M., Hoffman, J., & Carrick, M. (2024, September). TorchSig: A GNU Radio Block and New Spectrogram Tools for Augmenting ML Training. In Proceedings of the GNU Radio Conference (Vol. 9, No. 1). |
| [Large Scale Radio Frequency Wideband Signal Detection & Recognition](https://doi.org/10.48550/arXiv.2211.10335)| 2022 | Boegner, L., Vanhoy, G., Vallance, P., Gulati, M., Feitzinger, D., Comar, B., & Miller, R. D. (2022). Large Scale Radio Frequency Wideband Signal Detection & Recognition. arXiv preprint arXiv:2211.10335. |
| [Large Scale Radio Frequency Signal Classification](https://doi.org/10.48550/arXiv.2207.09918) | 2022 | Boegner, L., Gulati, M., Vanhoy, G., Vallance, P., Comar, B., Kokalj-Filipovic, S., ... & Miller, R. D. (2022). Large Scale Radio Frequency Signal Classification. arXiv preprint arXiv:2207.09918. |


# Citing TorchSig

## Citing TorchSig
---
Please cite TorchSig if you use it for your research or business.

```bibtext
Expand All @@ -103,4 +118,4 @@ Please cite TorchSig if you use it for your research or business.
note={arXiv:2207.09918}
url={https://arxiv.org/abs/2207.09918}
}
```
```
2 changes: 1 addition & 1 deletion docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ name: torch_sig_container_${PROJECT_NAME}
services:
torchsig_service:
build: .
image: torchsig_github:v0.5.0
image: torchsig_github
container_name: torchsig_${PROJECT_NAME}
stdin_open: true
tty: true
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
torchsig.datasets.conf.NarrowbandCleanEbNoTrainConfig
=====================================================

.. currentmodule:: torchsig.datasets.conf

.. autoclass:: NarrowbandCleanEbNoTrainConfig
:members:
:show-inheritance:
:inherited-members:
:special-members: __call__, __add__, __mul__



.. rubric:: Methods

.. autosummary::
:nosignatures:






.. rubric:: Attributes

.. autosummary::

~NarrowbandCleanEbNoTrainConfig.eb_no
~NarrowbandCleanEbNoTrainConfig.include_snr
~NarrowbandCleanEbNoTrainConfig.level
~NarrowbandCleanEbNoTrainConfig.name
~NarrowbandCleanEbNoTrainConfig.num_iq_samples
~NarrowbandCleanEbNoTrainConfig.num_samples
~NarrowbandCleanEbNoTrainConfig.seed
~NarrowbandCleanEbNoTrainConfig.use_class_idx


Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
torchsig.datasets.conf.NarrowbandCleanEbNoTrainQAConfig
=======================================================

.. currentmodule:: torchsig.datasets.conf

.. autoclass:: NarrowbandCleanEbNoTrainQAConfig
:members:
:show-inheritance:
:inherited-members:
:special-members: __call__, __add__, __mul__



.. rubric:: Methods

.. autosummary::
:nosignatures:






.. rubric:: Attributes

.. autosummary::

~NarrowbandCleanEbNoTrainQAConfig.eb_no
~NarrowbandCleanEbNoTrainQAConfig.include_snr
~NarrowbandCleanEbNoTrainQAConfig.level
~NarrowbandCleanEbNoTrainQAConfig.name
~NarrowbandCleanEbNoTrainQAConfig.num_iq_samples
~NarrowbandCleanEbNoTrainQAConfig.num_samples
~NarrowbandCleanEbNoTrainQAConfig.seed
~NarrowbandCleanEbNoTrainQAConfig.use_class_idx


Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
torchsig.datasets.conf.NarrowbandCleanEbNoValConfig
===================================================

.. currentmodule:: torchsig.datasets.conf

.. autoclass:: NarrowbandCleanEbNoValConfig
:members:
:show-inheritance:
:inherited-members:
:special-members: __call__, __add__, __mul__



.. rubric:: Methods

.. autosummary::
:nosignatures:






.. rubric:: Attributes

.. autosummary::

~NarrowbandCleanEbNoValConfig.eb_no
~NarrowbandCleanEbNoValConfig.include_snr
~NarrowbandCleanEbNoValConfig.level
~NarrowbandCleanEbNoValConfig.name
~NarrowbandCleanEbNoValConfig.num_iq_samples
~NarrowbandCleanEbNoValConfig.num_samples
~NarrowbandCleanEbNoValConfig.seed
~NarrowbandCleanEbNoValConfig.use_class_idx


Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
torchsig.datasets.conf.NarrowbandCleanEbNoValQAConfig
=====================================================

.. currentmodule:: torchsig.datasets.conf

.. autoclass:: NarrowbandCleanEbNoValQAConfig
:members:
:show-inheritance:
:inherited-members:
:special-members: __call__, __add__, __mul__



.. rubric:: Methods

.. autosummary::
:nosignatures:






.. rubric:: Attributes

.. autosummary::

~NarrowbandCleanEbNoValQAConfig.eb_no
~NarrowbandCleanEbNoValQAConfig.include_snr
~NarrowbandCleanEbNoValQAConfig.level
~NarrowbandCleanEbNoValQAConfig.name
~NarrowbandCleanEbNoValQAConfig.num_iq_samples
~NarrowbandCleanEbNoValQAConfig.num_samples
~NarrowbandCleanEbNoValQAConfig.seed
~NarrowbandCleanEbNoValQAConfig.use_class_idx


Loading

0 comments on commit 850c7b1

Please sign in to comment.