intro-to-ml-pipelines.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RC0pFC2bGuaz"
      },
      "source": [
        "# Intro to Machine Learning Pipelines\n",
        "\n",
        "<a target=\"_blank\" href=\"https://colab.research.google.com/github/unionai-oss/intro-to-ml-pipelines/blob/main/intro-to-ml-pipelines.ipynb\">\n",
        "  <img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/>\n",
        "</a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mzqGuApAEcvU"
      },
      "source": [
        "Google Colab / Jupyter notebook"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "0YG0xUWqt0eG",
        "outputId": "292d102e-9484-4903-9c3b-c5c770e97cdb"
      },
      "outputs": [],
      "source": [
        "print(\"Hello, world!\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "89e1mzmREc5U"
      },
      "source": [
        "# Workshop Setup\n",
        "\n",
        "Sign up for Union while libraries are installing below:\n",
        "\n",
        "- Union sign up: https://signup.union.ai/\n",
        "- Union Platform: https://serverless.union.ai/\n",
        "- Hugging Face Sign up: https://huggingface.co/\n",
        "\n",
        "Install python libraries by running the code cell below:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": true,
        "id": "J8d3u5gl3mvM"
      },
      "outputs": [],
      "source": [
        "%%bash\n",
        "pip install -U scikit-learn==1.4.1.post1 \\\n",
        "    matplotlib==3.8.3 \\\n",
        "    seaborn==0.13.2 \\\n",
        "    union==0.1.85 \\\n",
        "    keyrings.alt==5.0.0 \\\n",
        "    huggingface_hub==0.24.0 \\\n",
        "    joblib==1.3.2 \\\n",
        "    pyarrow==17.0.0 \\\n",
        "    python-dotenv \\\n",
        "    flytekit==1.13.8"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6nroELgiFiJI"
      },
      "source": [
        "Auth Union"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "MHmjN0PBFe8f",
        "outputId": "dba2678f-3010-456e-9d93-21dad664eace"
      },
      "outputs": [],
      "source": [
        "!union create login --serverless --auth device-flow"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "awvHr4VgFr6R"
      },
      "source": [
        "## Build a Simple Workflow\n",
        "- [Task Docs](https://docs.union.ai/byoc/core-concepts/tasks/)\n",
        "- [Workflow Docs](https://docs.union.ai/byoc/core-concepts/workflows/)\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "collapsed": true,
        "id": "9roHK4yoFf_A",
        "outputId": "1782f5be-fa0b-4901-da30-e5028f17bcee"
      },
      "outputs": [],
      "source": [
        "%%writefile simple_wf.py\n",
        "\n",
        "# Import libraries and modules\n",
        "# task\n",
        "from flytekit import task, workflow\n",
        "\n",
        "@task\n",
        "def hello_world(name: str) -> str:\n",
        "    return f\"Hello {name}\"\n",
        "\n",
        "# workflow\n",
        "@workflow\n",
        "def main(name: str) -> str:\n",
        "    return hello_world(name=name)\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "rusS2jq8F8d0",
        "outputId": "3be8f164-cceb-4fea-d580-97073fda0f1e"
      },
      "outputs": [],
      "source": [
        "# Run locally\n",
        "!union run simple_wf.py main --name Flyte"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "mfE-f7Y5F8p8",
        "outputId": "04e5b142-decc-4ac3-cb64-81002d5fa2e3"
      },
      "outputs": [],
      "source": [
        "# Run on Union\n",
        "!union run --remote simple_wf.py main --name Flyte"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "_AqWp5nSEe87"
      },
      "outputs": [],
      "source": [
        "!union run simple_wf.py main --help"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "7VDmcARWETwK"
      },
      "outputs": [],
      "source": [
        "!union run --help"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ycjgn7SzDYoz"
      },
      "source": [
        "# Machine Learning Workflow\n",
        "\n",
        "- Create a Container image\n",
        "- Download Dataset\n",
        "- Process Data & Visualize\n",
        "- Train Machine Learning Model\n",
        "- Evaluate Machine Learning Model\n",
        "- Save Model to Hugging Face Model Hub\n",
        "\n",
        "https://huggingface.co/"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "66QL-uU5vhzH",
        "outputId": "041833e3-3773-4570-aa76-ad6d92c18e98"
      },
      "outputs": [],
      "source": [
        "# Secrets in Union\n",
        "# HF Secret\n",
        "!union create secret hf_token"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "kT5BEYwqFVpX",
        "outputId": "2b1add2c-11f5-4b83-f914-a48690daaab8"
      },
      "outputs": [],
      "source": [
        "!union get secret"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "zktPlgQWFTs8"
      },
      "outputs": [],
      "source": [
        "# !union delete secret hf_token"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "30dDbzdqDXI1",
        "outputId": "8a9955a6-e4b3-412f-c976-44c849401a4c"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Overwriting workflow.py\n"
          ]
        }
      ],
      "source": [
        "%%writefile workflow.py\n",
        "# Import libraries and modules\n",
        "import html\n",
        "import io\n",
        "from textwrap import dedent\n",
        "from typing import List\n",
        "\n",
        "import joblib\n",
        "import matplotlib as mpl\n",
        "import matplotlib.pyplot as plt\n",
        "import pandas as pd\n",
        "import base64\n",
        "import seaborn as sns\n",
        "import os\n",
        "from dotenv import load_dotenv\n",
        "from flytekit import (Deck, ImageSpec, Resources, Secret, current_context,\n",
        "                      task, workflow)\n",
        "from huggingface_hub import HfApi\n",
        "from sklearn.datasets import load_iris\n",
        "from sklearn.metrics import ConfusionMatrixDisplay, classification_report\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.neighbors import KNeighborsClassifier\n",
        "\n",
        "# Define the container image to use for the tasks on Union with ImageSpec\n",
        "image = ImageSpec(\n",
        "    packages=[\n",
        "        \"union==0.1.85\",\n",
        "        \"flytekit==1.13.8\",\n",
        "        \"scikit-learn==1.4.1.post1\",\n",
        "        \"matplotlib==3.8.3\",\n",
        "        \"seaborn==0.13.2\",\n",
        "        \"joblib==1.3.2\",\n",
        "        \"huggingface_hub==0.24.0\",\n",
        "        \"pyarrow==17.0.0\",\n",
        "        \"python-dotenv\"\n",
        "    ],\n",
        ")\n",
        "\n",
        "load_dotenv()\n",
        "\n",
        "# Helper function to convert a matplotlib figure into an HTML string\n",
        "def _convert_fig_into_html(fig: mpl.figure.Figure) -> str:\n",
        "    img_buf = io.BytesIO()\n",
        "    fig.savefig(img_buf, format=\"png\")\n",
        "    img_base64 = base64.b64encode(img_buf.getvalue()).decode()\n",
        "    return f'<img src=\"data:image/png;base64,{img_base64}\" alt=\"Rendered Image\" />'\n",
        "\n",
        "\n",
        "# Task: Download the dataset\n",
        "@task(\n",
        "    cache=True,\n",
        "    cache_version=\"7\",\n",
        "    container_image=image,\n",
        "    requests=Resources(cpu=\"2\", mem=\"2Gi\"),\n",
        ")\n",
        "def download_dataset() -> pd.DataFrame:\n",
        "    iris = load_iris()\n",
        "    iris_df = pd.DataFrame(data=iris.data, columns=iris.feature_names)\n",
        "    iris_df['target'] = iris.target\n",
        "    return iris_df\n",
        "\n",
        "# Task: Process the dataset\n",
        "@task(\n",
        "    enable_deck=True,\n",
        "    container_image=image,\n",
        "    requests=Resources(cpu=\"2\", mem=\"2Gi\"),\n",
        ")\n",
        "def process_dataset(data_df: pd.DataFrame) -> tuple[pd.DataFrame, pd.DataFrame]:\n",
        "\n",
        "    # Perform the train-test split\n",
        "    train_df, test_df = train_test_split(data_df,\n",
        "                                         test_size=0.2,\n",
        "                                         random_state=42,\n",
        "                                         stratify=data_df['target'])\n",
        "\n",
        "    # Seaborn pairplot full dataset\n",
        "    pairplot = sns.pairplot(data_df, hue=\"target\")\n",
        "\n",
        "    metrics_deck = Deck(\"Metrics\")\n",
        "    metrics_deck.append(_convert_fig_into_html(pairplot))\n",
        "\n",
        "    return train_df, test_df\n",
        "\n",
        "\n",
        "# Task: Train a model\n",
        "@task(\n",
        "    container_image=image,\n",
        "    requests=Resources(cpu=\"3\", mem=\"3Gi\"),\n",
        ")\n",
        "def train_model(dataset: pd.DataFrame, n_neighbors: int = 3) -> KNeighborsClassifier:\n",
        "    X_train, y_train = dataset.drop(\"target\", axis=\"columns\"), dataset[\"target\"]\n",
        "    model = knn = KNeighborsClassifier(n_neighbors=n_neighbors)\n",
        "    return model.fit(X_train, y_train)\n",
        "\n",
        "# Evaluate the model using the test dataset\n",
        "@task(\n",
        "    container_image=image,\n",
        "    enable_deck=True,\n",
        "    requests=Resources(cpu=\"2\", mem=\"2Gi\"),\n",
        ")\n",
        "def evaluate_model(model: KNeighborsClassifier, dataset: pd.DataFrame) -> KNeighborsClassifier:\n",
        "    ctx = current_context()\n",
        "\n",
        "    X_test, y_test = dataset.drop(\"target\", axis=\"columns\"), dataset[\"target\"]\n",
        "    y_pred = model.predict(X_test)\n",
        "\n",
        "    # Plot confusion matrix in deck\n",
        "    fig, ax = plt.subplots()\n",
        "    ConfusionMatrixDisplay.from_predictions(y_test, y_pred, ax=ax)\n",
        "\n",
        "    metrics_deck = Deck(\"Metrics\")\n",
        "    metrics_deck.append(_convert_fig_into_html(fig))\n",
        "\n",
        "    # Add classification report\n",
        "    report = html.escape(classification_report(y_test, y_pred))\n",
        "    html_report = dedent(\n",
        "        f\"\"\"\\\n",
        "    <h2>Classification report</h2>\n",
        "    <pre>{report}</pre>\"\"\"\n",
        "    )\n",
        "    metrics_deck.append(html_report)\n",
        "\n",
        "    ctx.decks.insert(0, metrics_deck)\n",
        "\n",
        "    return model\n",
        "\n",
        "# Upload the model to Hugging Face Hub\n",
        "@task(\n",
        "    container_image=image,\n",
        "    requests=Resources(cpu=\"1\", mem=\"1Gi\"),\n",
        "    secret_requests=[Secret(group=None, key=\"hf_token\")],\n",
        ")\n",
        "def upload_model_to_hf(model: KNeighborsClassifier, repo_name: str, model_name: str) -> str:\n",
        "    ctx = current_context()\n",
        "\n",
        "    # set hf_token from local or union secret\n",
        "    hf_token = os.getenv(\"HF_TOKEN\")\n",
        "    if hf_token is None:\n",
        "        # If HF_TOKEN is not found, attempt to get it from the Flyte secrets\n",
        "        hf_token = ctx.secrets.get(key=\"hf_token\")\n",
        "        print(\"Using Hugging Face token from Union secrets.\")\n",
        "    else:\n",
        "        print(\"Using Hugging Face token from env.\")\n",
        "\n",
        "    # Create a new repository (if it doesn't exist)\n",
        "    api = HfApi()\n",
        "    api.create_repo(repo_name, token=hf_token, exist_ok=True)\n",
        "\n",
        "    # save model\n",
        "    joblib.dump(model, model_name)\n",
        "\n",
        "    # Upload the model to the HF repository\n",
        "    api.upload_file(\n",
        "        path_or_fileobj=model_name,\n",
        "        path_in_repo=model_name,\n",
        "        repo_id=repo_name,\n",
        "        commit_message=\"Upload model\",\n",
        "        repo_type=None,\n",
        "        token=hf_token\n",
        "    )\n",
        "    return f\"Model uploaded to Hugging Face Hub: {repo_name}{model_name}\"\n",
        "\n",
        "# Task: Model prediction with custom input data, good for testing\n",
        "@task(\n",
        "    container_image=image,\n",
        "    enable_deck=True,\n",
        "    requests=Resources(cpu=\"2\", mem=\"2Gi\"),\n",
        ")\n",
        "def model_predict(model: KNeighborsClassifier, pred_data: List[List[float]]) -> List[int]:\n",
        "    predictions = model.predict(pred_data)\n",
        "    return predictions.tolist()\n",
        "\n",
        "\n",
        "# Main workflow that orchestrates the tasks\n",
        "@workflow\n",
        "def main(repo_name: str, model_name: str, n_neighbors: int = 3,\n",
        "          pred_data: List[List[float]] = [[1.5, 2.3, 1.3, 2.4]]) -> KNeighborsClassifier:\n",
        "    data_df = download_dataset()\n",
        "    train, test = process_dataset(data_df)\n",
        "    model = train_model(dataset=train, n_neighbors=n_neighbors)\n",
        "    evaluated_model = evaluate_model(model=model, dataset=test)\n",
        "    model_predict(model=model, pred_data=pred_data)\n",
        "    model_name = model_name\n",
        "    upload_result = upload_model_to_hf(model=evaluated_model,\n",
        "                                       repo_name=repo_name,\n",
        "                                       model_name=model_name)\n",
        "    return model\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "EBi1v9mfyZxe"
      },
      "outputs": [],
      "source": [
        "! union run workflow.py main --help"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6xP-FsYtDXNY",
        "outputId": "b108f738-3c07-4fe7-eee0-eff6c76c8ba2"
      },
      "outputs": [],
      "source": [
        "# Run workflow remotely (Union serverless) or your own compute\n",
        "!union run --remote workflow.py main --repo_name YOURUSERNAME/REPOID --model_name model.pkl"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eH3ez5efKJF4"
      },
      "source": [
        "View workflow in Union dashboard"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LberVtzTj7u2"
      },
      "source": [
        "## Download Model from Hugging Face Hub"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "VmiVkrZTkvKJ"
      },
      "outputs": [],
      "source": [
        "from huggingface_hub import hf_hub_download\n",
        "import joblib\n",
        "hf_token = \"\"\n",
        "\n",
        "hf_model = hf_hub_download(repo_id=\"YOURUSERNAME/REPOID\",\n",
        "                filename=\"model.pkl\",\n",
        "                token=hf_token)\n",
        "\n",
        "model = joblib.load(hf_model)\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Dovy0q7hlvHQ"
      },
      "outputs": [],
      "source": [
        "model"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "kvQSUyOyl24G"
      },
      "outputs": [],
      "source": [
        "print(model.predict([[3,5,6,5]]))\n",
        "print(model.predict([[.4,.2,.1,.6]]))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "nQjqX2WADXZc"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Remote API"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# initialize remote context\n",
        "from union.remote import UnionRemote\n",
        "remote = UnionRemote()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# search for the 100 most recent executions of the workflow\n",
        "recent_executions = remote.recent_executions(limit=100)\n",
        "executions = [\n",
        "    e for e in recent_executions if e.spec.launch_plan.name == \"workflow.main\"\n",
        "]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "executions"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "#get latest execution id\n",
        "recent_ex_id = executions[0].id.name\n",
        "print(recent_ex_id)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "execution = remote.fetch_execution(name=recent_ex_id)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "execution"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "execution.outputs"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "model_uri = execution.outputs[\"o0\"].remote_source\n",
        "print(model_uri)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# fetch a task\n",
        "predict_task = remote.fetch_task(name=\"workflow.model_predict\")\n",
        "\n",
        "# Fecth specific version of a task\n",
        "# task = remote.fetch_task(name=\"workflow.model_predict\", version=\"_XyPQzsykVFTceNWitQmAg\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# view task details\n",
        "predict_task"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "from flytekit.types.file import FlyteFile\n",
        "inputs = {\n",
        "    \"pred_data\": [[1.5,2.3,1.3,2.4]],\n",
        "    \"model\": FlyteFile(model_uri)\n",
        "}\n",
        "\n",
        "# # Execute the task\n",
        "execution = remote.execute(predict_task, inputs=inputs)\n",
        "# execution = remote.execute(task, inputs=inputs, wait=True) # wait for execution to finish\n",
        "\n",
        "url = remote.generate_console_url(execution)\n",
        "print(f\"🚀 Union Serverless execution url: {url}\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "synced_execution = remote.sync(execution)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "synced_execution"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "outputs = synced_execution.outputs[\"o0\"]\n",
        "print(outputs)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### All together:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "from union.remote import UnionRemote\n",
        "from flytekit.types.file import FlyteFile\n",
        "\n",
        "remote = UnionRemote()\n",
        "\n",
        "def get_latest_execution_model(limit=100):\n",
        "\n",
        "  recent_executions = remote.recent_executions(limit=limit)\n",
        "  executions = [\n",
        "      e for e in recent_executions if e.spec.launch_plan.name == \"workflow.main\"\n",
        "  ]\n",
        "\n",
        "  recent_ex_id = executions[0].id.name\n",
        "  execution = remote.fetch_execution(name=recent_ex_id)\n",
        "  model_uri = execution.outputs[\"o0\"].remote_source\n",
        "\n",
        "  return model_uri\n",
        "\n",
        "def make_prediction(model_uri, pred_data):\n",
        "\n",
        "  predict_task = remote.fetch_task(name=\"workflow.model_predict\")\n",
        "\n",
        "\n",
        "  inputs = {\n",
        "      \"pred_data\": pred_data,\n",
        "      \"model\": FlyteFile(model_uri)\n",
        "  }\n",
        "\n",
        "  # # Execute the task\n",
        "  execution = remote.execute(predict_task, inputs=inputs, wait=True)\n",
        "\n",
        "  response = execution.outputs[\"o0\"]\n",
        "\n",
        "  return response\n",
        "\n",
        "model_uri = get_latest_execution_model()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "make_prediction(model_uri, [[-3.0,-5.3,-6.3,-5.0]])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Note: some things to consider when using remote API for inference:\n",
        "- Using [Union Artifacts](https://www.union.ai/blog-post/data-aware-event-driven-ai-orchestration-with-artifacts) to store the model and dataset\n",
        "- Making inference a separate workflow\n",
        "- Currently good for batch inference\n",
        "- We're working on new features that will make models more readily available for inference in Union!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HWDQUtyMowD_"
      },
      "source": [
        "Keep Learning:\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "vrg5hIHt8Cic"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "yWjGImpsBP2s"
      },
      "outputs": [],
      "source": []
    }
  ],
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}