google_colab.md

title
ClearML Agent on Google Colab

Google Colab is a common development environment for data scientists. It supports a convenient IDE as well as compute provided by Google.

Users can transform a Google Colab instance into an available resource in ClearML using ClearML Agent.

This tutorial goes over how to create a ClearML worker node in a Google Colab notebook. Once the worker is up and running, users can send Tasks to be executed on Google Colab's hardware.

Prerequisites

• Be signed up for ClearML (or have a server deployed).
• Have a Google account to access Google Colab.

Steps

1. Open up this Google Colab notebook.

2. Run the first cell, which installs all the necessary packages:

   !pip install git+https://github.com/allegroai/clearml
   !pip install clearml-agent
   

3. Run the second cell, which exports this environment variable:

   ! export MPLBACKEND=TkAg
   

   This environment variable makes Matplotlib work in headless mode, so it won't output graphs to the screen.

4. Create new credentials. Go to your Settings page > WORKSPACE section. Under App Credentials, click + Create new credentials, and copy the information that pops up.

5. Set the credentials. In the third cell, enter your own credentials:

   from clearml import Task
   
   Task.set_credentials(
        api_host="https://api.clear.ml", 
        web_host="https://app.clear.ml", 
        files_host="https://files.clear.ml", 
        key='6ZHX9UQMYL874A1NE8', 
        secret='=2h6#%@Y&m*tC!VLEXq&JI7QhZPKuJfbaYD4!uUk(t7=9ENv'
   )

6. In the fourth cell, launch a clearml-agent that will listen to the default queue:

   !clearml-agent daemon --queue default
   

   For additional options for running clearml-agent, see the clearml-agent reference.

   After executing cell 4, the worker appears in the Orchestration page of your server. Clone experiments and enqueue them to your hearts content! The clearml-agent will fetch experiments and execute them using the Google Colab hardware.