A Simulation as a Service-Implementation: Worker

This repository contains the source code that actually implements the functionality offered as a Service via the SIMaaS-API developed at the chair of automation- and energy systems at Saarland university (UdS AES).

More specifically, its capability is to execute simulations based on Functional Mock-up Units (FMUs) using FMPy, a Python library developed at Dassault Systèmes.

Requests for performing simulations are distributed to the worker instances by means of a distributed task queue implemented using Celery. Consequently, the available computing power can easily be increased both by allowing one instance to use more cores and by starting more instances in a cluster.

Installation

The code can be executed directly on GNU/Linux host machines or containerized using Podman or Docker. In both cases, some environment variables (ENVVARs) must be set according to the table below.

Configuration

All environment variables without a default value MUST be set before starting the program. Otherwise, a fatal error will be raised and the program will terminate with a non-zero exit code.

Environment Variable	Description	Default Value
SIMWORKER_BROKER_HREF	The scheme and authority-parts of a URL pointing to the RabbitMQ-instance to be used as message broker.	—
SIMWORKER_BACKEND_HREF	The scheme and authority-parts of a URL pointing to the Redis-instance to be used as result backend.	—
SIMWORKER_TMPFS_PATH	The path at which to store temporary files. Should be a tmpfs-mount for better speed but can be a normal directory, too.	—
SIMWORKER_TMPFS_MAXSIZE	The maximum amount of bytes that the temporary files are allowed to consume. Iff the total file size of all temporary files exceeds this limit, the least recently used files are deleted.	—
SIMWORKER_LOG_STRUCTURED	Whether to output logs as JSON-objects ("true") or formatted strings ("false").	"false"
SIMWORKER_LOG_LEVEL	The minimal included log level that shall be printed to stdout.	"INFO"
DYMOLA_RUNTIME_LICENSE	The path to the Dymola license file necessary to simulate FMUs created using Dymola. Obviously this is only necessary iff the FMUs require a license to execute.	—

Running via celery worker

First, install the dependencies (specified in pyproject.toml/poetry.lock) into a new Poetry-environment using poetry install. Then, activate the environment (poetry shell) and run the commands below from within this environment.

Running several instances with a fixed thread count

source .env; celery -A worker worker --loglevel=INFO --concurrency=1 -n worker1@%h
source .env; celery -A worker worker --loglevel=INFO --concurrency=1 -n worker2@%h
source .env; celery -A worker worker --loglevel=INFO --concurrency=1 -n worker3@%h

The optimal number of worker processes/threads depends on the application — read the note on concurrency in the Celery-manual.

Dynamically spawn new threads if necessary

source .env; celery -A worker worker --loglevel=INFO --autoscale=10,3

If you want to use the GUI of FMPy for testing within the Poetry-environment, you’ll need to install the development dependencies as well (Poetry installs them by default) and you probably also need to install python3-tkinter or similar via your operating systems package manager.

Running Containerized

Using Poetry for building container images is complicated/an overkill, so the dependency manifest needs to be exported in a format that pip can use when building the image using poetry export -f requirements.txt --output requirements.txt --without-hashes before attempting to build.

poetry export -f requirements.txt --output requirements.txt --without-hashes

podman build -t simaas-worker:latest . # build and tag image
podman run \
  --name simaas-worker \
  --env SIMWORKER_BROKER_HREF=... \
  --env SIMWORKER_BACKEND_HREF=... \
  --env SIMWORKER_TMPFS_PATH=... \
  --env SIMWORKER_TMPFS_MAXSIZE=... \
  --rm -d \
  simaas-worker:latest # run as container

If a Dymola license is required to execute the FMUs, then you will need to mount the license file into the container and set the ENVVAR to the path under which the license is available from within the container.

Usage

This component is not intended to be used directly — its sole purpose is to do jobs put into the queue by an instance of the SIMaaS-API.

Roadmap

We will work on the following issues in the near future:

• ❏ Fix the test suite that is currently broken (!)

• ❏ Fix type casting for enumerations

• ❏ Provide robust solution for unit mapping when generating RDF representations

We might also spend time on the following (but can’t say when for sure):

• ❏ Support FMUs for model exchange

• ❏ Support version 3.0 of the FMI standard (not yet released)

Development

In case you find problems, have questions or ideas: please open an issue so that we can discuss it!

Code formatting is handled by black and isort, so please install the development dependencies (poetry install --dev) and run them before submitting a pull request.

Acknowledgements

From January 2017 to March 2021, this work was supported by the SINTEG-project “Designetz” funded by the German Federal Ministry of Economic Affairs and Energy (BMWi) under grant 03SIN224.

License

The source code is licensed under the MIT License. This is specified in the format suggested by the REUSE SOFTWARE-initiative — in short: SPDX IDs are included in every textual file and the license text can be found in ./LICENSES/.