Welcome to Geospatial Analysis Pipelines (GAPs)!

Geospatial Analysis Pipelines (GAPs) is a framework designed to assist researchers and software developers add execution tools to their geospatial python models. Born from the open-source reV model, GAPs is a robust and easy-to-use engine that provides a rich set of features such as command-line interface (CLI) generation and documentation, basic High-Performance Computing (HPC) scaling capabilities, configuration file generation, job status monitoring, and more.

Who should use GAPs

GAPs is intended to be used by researchers and/or software developers who have implemented a working python model but have not yet added any external model execution tools. Within minimal effort, developers can use GAPs to add a variety of utility for end-users, including a complete set of CLI commands and documentation pulled from the model run function docstrings. In addition, GAPs provides basic HPC execution capabilities, particularly catered towards embarrassingly parallel geospatial models (e.g. single-location models such as the System Advisor Model). GAPs can automatically distribute the execution of such models over a large geospatial extent (e.g. CONUS) across many parallel HPC nodes.

GAPs is NOT a workflow management system (WMS), and therefore does not provide any of the in-depth tools/capabilities expected from a proper WMS. However, GAPs-supported models can sometimes be included as part of the workflow in WMS tools like Torc.

To get started, take a look at the examples for analysts or model developers or dive straight into the full documentation.

Installing GAPs

NOTE: The installation instruction below assume that you have python installed on your machine and are using conda as your package/environment manager.

1. Clone the gaps repository.

   • Using ssh: git clone git@github.com:NREL/gaps.git
   • Using https: git clone https://github.com/NREL/gaps.git

2. Create and activate the gaps environment and install the package:

   1. Create a conda env: conda create -n gaps python=3.10
   2. Activate the newly-created conda env: conda activate gaps
   3. Change directories into the repository: cd gaps
   4. Prior to running pip below, make sure the branch is correct (install from main!): git branch -vv
   5. Install gaps and its dependencies by running: pip install -e . (or pip install -e .[dev] if running a dev branch or working on the source code)

Development

This repository uses pylint to lint the code and black to format it (check out the black formatting style). If you wish to contribute to this repository, your code will have to adhere to both of these guidelines and pass all existing tests.

Acknowledgments

Paul Pinchuk and Grant Buster. Geospatial Analysis Pipelines. 2023. https://doi.org/10.11578/dc.20230426.7

The authors of this code would like to thank ExxonMobil Corporation for their contributions to this effort.