Minimal example using MNIST for easy implementation of highly configurable ray tune CLIs based on jsonargparse. Only requires a function that uses LightningCLI to run fit. The example is an adaptation of ray/tune/examples/mnist_pytorch_lightning.py. To try out the code first install the packages listed in requirements.txt.

The normal MNIST pytorch-lightning CLI is implemented in file mnist_lightning_cli.py. For example, to run a fit you could do:

./mnist_lightning_cli.py fit --config mnist_fit_config.yaml

The ray tune CLI is implemented in file mnist_ray_tune_cli.py. To run a hyperparameter search, this CLI first receives ray options, followed by "--" and then LightningCLI fit options. For example:

./mnist_ray_tune_cli.py --config mnist_tune_config.yaml -- --config mnist_fit_config.yaml

This repo is only intended to illustrate the concept. The idea would be to contribute an improved version either to pytorch-lightning or ray or make it an independent pypi package. The improved version would probably be a class similar to LightningCLI to allow customization. It would also automatically save the tune config to a standard location to ease experiment reporting and reproducibility.