README.md

Badges

fair-software.eu recommendations
(1/5) code repository
(2/5) license
(3/5) community registry
(4/5) citation
(5/5) checklist
howfairis
Other best practices
Static analysis
Coverage
Documentation
GitHub Actions
Citation data consistency
Tests & SonarQube analysis

SLURM CLI-API Proxy client is a tool designed to bridge existing applications and scripts that rely on the SLURM CLI. The tool mimics a selection of SLURM CLI commands, translating them into REST API calls, enabling seamless integration of existing tools with external SLURM workload managers.

Due to the extensive range of optional SLURM command arguments, only a limited subset is supported. However, the tool is designed for flexibility—the support to new arguments can be easily enabled by mapping them to the corresponding API request parameters or payload properties. Additionally, the design prioritizes extensibility, allowing support for alternative SLURM API versions (currently working with v0.0.39). The developers documentation detailing these design elements is still a work in progress.

Setup and usage

Requirement

• Linux, MacOS or Windows with WSL
• Python version ≥3.10

Installation

# Check python version (requiring ≥3.10)
python --version

# Create a new virtual environment
python -m venv env
source env/bin/activate

# install 
pip install .

A virtual environment is required to install the the non-pypi dependencies. You can also use conda to manage python environments. If running from a shell terminal, the virtual environment where the package was installed must be active on it.

Testing

Unit and integration tests

#Start a dockerized slurm cluster on localhost (works on Linux only)
source src/tests/slurm_test_scripts/start_dockerized_slurm.sh

#Run tests 
pytest

Running unit tests only (doesn't require the dockerized slurm cluster)

# Run tests, excluding integration ones (which require the dockerized slurm cluster running on localhost)
pytest -m "not integration"

Static type checking

#Static type checking with mypy. 
mypy src --ignore-missing-imports --check-untyped-defs

Usage from a shell terminal

1. Define the URI of the target SLURM API through the PROXY_SLURM_API_URL environment variable:

#Example:
export PROXY_SLURM_API_URL=http://slurm-controller:6820

2. Set the SLURM_JWT environment variable with the API token of the target SLURM API. An script is provided to do this if you have ssh access to the SLURM workload manager:

# Setting the SLURM_JWT variable (can be obtained by running 'scontrol token' on the SLURM workload manager)
export SLURM_JWT=<token>

# Setting the SLURM_JWT variable through the provided script (password for opening an ssh session will be requested)
# source update_token.sh <slurm-wlm-user> <slurm-wlm-host>. E.g.:
source update_token.sh userx slurm-controller

3. Run slurm commands as you would do* with the real ones:

#sbatch help
sbatch --help

#request a job defined on a shell script
sbatch --job-name jobx --chdir /home/userx  src/tests/slurm_test_scripts/slurm_write_job.sh

#capture a job definition through STDIN and request its execution
sbatch --job-name jobx --chdir /home/userx

#show running jobs
squeue 

#show running jobs in json format
squeue --json

🚧 Alpha release notice* 🚧

• Only sbatch and squeue commands have been implemented, with a limited number of arguments
• Design refinements are still in progress
• Documentation and testing coverage are ongoing

If you encounter issues or have feedback, please report them through the project's issue tracker.

Contributing

If you want to contribute to the development of slurm_api_cli_proxy, have a look at the contribution guidelines.

Credits

This package was created with Copier and the NLeSC/python-template.