MLflow Workspace

Newsletter

👉🏽 Do you enjoy our content and want to read super-detailed articles about AI? If so, subscribe to our Tinz Twins Hub and get our popular data science cheat sheets for FREE.

General

The MLflow environment serves as a tracking server for managing the machine learning lifecycle. The repo contains several infrastructure services that start in a Docker stack.

Infrastructure services

The Docker stack uses four infrastructure services. These services can be started manually so that you get an MLflow working environment in Docker.

• JupyterLab (docker-container: jupyter)
• MLflow Tracking Server (docker-container: mlflow)
• Artifact Store as SFTP Server (docker-container: sftp)
• Backend Storage as postgres db (docker-container: postgres)

Start and stop the MLflow workspace

1. Start MLflow workspace: sh start_docker_stack.sh
2. Visit http://127.0.0.1:5001 for MLflow UI
3. Visit http://127.0.0.1:8888 for JupyterLab UI (enter the password mlflow)
4. Stop MLflow workspace: sh stop_docker_stack.sh

Test your MLflow workspace setup

• Visit the JupyterLab (http://127.0.0.1:8888) and execute the notebook mlflow_example.ipynb. If no error appears, then your MLflow workspace is set up correctly.

Adding new dependencies for JupyterLab

• Add new dependencies to file docker/jupyter/requirements.txt

Apple silicon or arm64 architecture

• Please open the .env file and change the variable SFTP_IMAGE to tinztwins/sftp-server:arm64.

Troubleshooting

• On Linux systems there may be permission problems with the Artifact Store. The current workaround is to set the permissions manually (sudo chmod -R 777).

Useful Tools

• SFTP-Server: Cyberduck

References

• Deploy Your Own MLflow Workspace On-Premise with Docker in Towards Data Science / April 17, 2023
• How to setup an MLflow 2.0 Workspace with Docker? in Dev Genius / Dec 4, 2022