cpd4.0/notebooks/python_sdk/instance-management/Machine Learning artifacts export and import.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "\n",
        "# Export/Import assets with `ibm-watson-machine-learning`\n",
        "\n",
        "This notebook demonstrates an example for exporting/importing assets using Watson Machine Learning service. It contains steps and code to work with [ibm-watson-machine-learning](https://pypi.python.org/pypi/ibm-watson-machine-learning) library available in PyPI repository."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Learning goals\n",
        "\n",
        "The learning goals of this notebook are:\n",
        "\n",
        "-  Download an externally trained Keras model.\n",
        "-  Persist an external model in Watson Machine Learning repository.\n",
        "-  Export the model from the space\n",
        "-  Import the model to another space( For Cloud Pak for Data 4.0, the space/project where assets are imported has to be empty ) and deploy\n",
        "\n",
        "\n",
        "\n",
        "## Contents\n",
        "\n",
        "This notebook contains the following parts:\n",
        "\n",
        "1.\t[Setup](#setup)\n",
        "2.\t[Download externally created Keras model](#download)\n",
        "3.\t[Persist externally created Keras model](#persistence)\n",
        "4.\t[Export the model](#export)\n",
        "5.\t[Import the model](#import)\n",
        "6.\t[Deploy and score the imported model](#scoring)\n",
        "7.  [Clean up](#cleanup)\n",
        "8.\t[Summary and next steps](#summary)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "<a id=\"setup\"></a>\n",
        "## 1. Set up the environment\n",
        "\n",
        "Before you use the sample code in this notebook, you must perform the following setup tasks:\n",
        "\n",
        "-  Contact with your Cloud Pack for Data administrator and ask him for your account credentials"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Connection to WML\n",
        "\n",
        "Authenticate the Watson Machine Learning service on IBM Cloud Pack for Data. You need to provide platform `url`, your `username` and `api_key`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "username = 'PASTE YOUR USERNAME HERE'\n",
        "api_key = 'PASTE YOUR API_KEY HERE'\n",
        "url = 'PASTE THE PLATFORM URL HERE'"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {},
      "outputs": [],
      "source": [
        "wml_credentials = {\n",
        "    \"username\": username,\n",
        "    \"apikey\": api_key,\n",
        "    \"url\": url,\n",
        "    \"instance_id\": 'openshift',\n",
        "    \"version\": '4.0'\n",
        "}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Alternatively you can use `username` and `password` to authenticate WML services.\n",
        "\n",
        "```\n",
        "wml_credentials = {\n",
        "    \"username\": ***,\n",
        "    \"password\": ***,\n",
        "    \"url\": ***,\n",
        "    \"instance_id\": 'openshift',\n",
        "    \"version\": '4.0'\n",
        "}\n",
        "\n",
        "```"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "###  `ibm-watson-machine-learning` is packaged in Cloud Pak For Data 4.0\n",
        "**Note:** `ibm-watson-machine-learning` documentation can be found <a href=\"http://ibm-wml-api-pyclient.mybluemix.net/\" target=\"_blank\" rel=\"noopener no referrer\">here</a>."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {},
      "outputs": [],
      "source": [
        "from ibm_watson_machine_learning import APIClient\n",
        "\n",
        "client = APIClient(wml_credentials)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Create two spaces. One for export and one for import\n",
        "\n",
        "**Tip**: You can refer to example for space management apis [here](https://github.com/IBM/watson-machine-learning-samples/blob/master/cpd4.0/notebooks/python_sdk/instance-management/Space%20management.ipynb).\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Space has been created. However some background setup activities might still be on-going. Check for 'status' field in the response. It has to show 'active' before space can be used. If its not 'active', you can monitor the state with a call to spaces.get_details(space_id)\n",
            "\n",
            "export space_id: be32da95-1735-4c8a-96aa-e4d6b8b6a340\n",
            "\n",
            "Space has been created. However some background setup activities might still be on-going. Check for 'status' field in the response. It has to show 'active' before space can be used. If its not 'active', you can monitor the state with a call to spaces.get_details(space_id)\n",
            "\n",
            "import space_id: 68dbf09b-5ddb-4c0c-a925-2ee9327f511d\n"
          ]
        }
      ],
      "source": [
        "import uuid\n",
        "import json\n",
        "\n",
        "space_name = str(uuid.uuid4())\n",
        "\n",
        "export_space_metadata = {\n",
        "              client.spaces.ConfigurationMetaNames.NAME: 'client_space_export_' + space_name,\n",
        "              client.spaces.ConfigurationMetaNames.DESCRIPTION: space_name + ' description'\n",
        "}\n",
        "\n",
        "space = client.spaces.store(meta_props=export_space_metadata)\n",
        "\n",
        "export_space_id = client.spaces.get_id(space)\n",
        "print(\"{}export space_id: {}{}\".format('\\n', export_space_id, '\\n'))\n",
        "\n",
        "\n",
        "import_space_metadata = {\n",
        "              client.spaces.ConfigurationMetaNames.NAME: 'client_space_import_' + space_name,\n",
        "              client.spaces.ConfigurationMetaNames.DESCRIPTION: space_name + ' description'\n",
        "}\n",
        "\n",
        "space = client.spaces.store(meta_props=import_space_metadata)\n",
        "\n",
        "import_space_id = client.spaces.get_id(space)\n",
        "print(\"{}import space_id: {}\".format('\\n', import_space_id))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "pycharm": {
          "name": "#%% md\n"
        }
      },
      "source": [
        "<a id=\"download\"></a>\n",
        "## 2. Download externally created Keras model and data\n",
        "In this section, you will download externally created Keras models and data used for training it."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "pycharm": {
          "is_executing": false,
          "name": "#%%\n"
        }
      },
      "outputs": [],
      "source": [
        "import os\n",
        "import wget\n",
        "import ssl\n",
        "\n",
        "data_dir = 'MNIST_DATA'\n",
        "if not os.path.isdir(data_dir):\n",
        "    os.mkdir(data_dir)\n",
        "    \n",
        "model_path = os.path.join(data_dir, 'mnist_keras.h5.tgz')\n",
        "if not os.path.isfile(model_path):\n",
        "    wget.download(\"https://github.com/IBM/watson-machine-learning-samples/raw/master/cpd4.0/models/keras/mnist_keras.h5.tgz\", out=data_dir)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {},
      "outputs": [],
      "source": [
        "data_dir = 'MNIST_DATA'\n",
        "if not os.path.isdir(data_dir):\n",
        "    os.mkdir(data_dir)\n",
        "    \n",
        "filename = os.path.join(data_dir, 'mnist.npz')\n",
        "if not os.path.isfile(filename):\n",
        "    wget.download('https://s3.amazonaws.com/img-datasets/mnist.npz', out=data_dir)  "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "pycharm": {
          "is_executing": false,
          "name": "#%%\n"
        }
      },
      "outputs": [],
      "source": [
        "import numpy as np\n",
        "\n",
        "dataset = np.load(filename)\n",
        "x_test = dataset['x_test']"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "<a id=\"persistence\"></a>\n",
        "## 3. Persist externally created Keras model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "In this section, you will learn how to store your model in Watson Machine Learning repository by using the Watson Machine Learning Client."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### 3.1: Publish model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Define model name, type and software specification needed to deploy model later."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "pycharm": {
          "is_executing": false,
          "name": "#%%\n"
        }
      },
      "outputs": [],
      "source": [
        "sofware_spec_uid = client.software_specifications.get_id_by_name(\"default_py3.8\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "pycharm": {
          "is_executing": false,
          "name": "#%%\n"
        }
      },
      "outputs": [],
      "source": [
        "client.set.default_space(export_space_id)\n",
        "\n",
        "metadata = {\n",
        "            client.repository.ModelMetaNames.NAME: 'External Keras model',\n",
        "            client.repository.ModelMetaNames.TYPE: 'tensorflow_2.1',\n",
        "            client.repository.ModelMetaNames.SOFTWARE_SPEC_UID: sofware_spec_uid\n",
        "}\n",
        "\n",
        "published_model = client.repository.store_model(\n",
        "    model=model_path,\n",
        "    meta_props=metadata)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### 3.2: Get model details"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "pycharm": {
          "is_executing": false,
          "name": "#%%\n"
        }
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "{\n",
            "  \"entity\": {\n",
            "    \"software_spec\": {\n",
            "      \"id\": \"e4429883-c883-42b6-87a8-f419d64088cd\",\n",
            "      \"name\": \"default_py3.7\"\n",
            "    },\n",
            "    \"type\": \"tensorflow_2.1\"\n",
            "  },\n",
            "  \"metadata\": {\n",
            "    \"created_at\": \"2020-12-08T13:29:30.489Z\",\n",
            "    \"id\": \"7bd7c597-01f5-496f-8131-f9464248cd4b\",\n",
            "    \"modified_at\": \"2020-12-08T13:29:48.846Z\",\n",
            "    \"name\": \"External Keras model\",\n",
            "    \"owner\": \"1000330999\",\n",
            "    \"space_id\": \"be32da95-1735-4c8a-96aa-e4d6b8b6a340\"\n",
            "  },\n",
            "  \"system\": {\n",
            "    \"warnings\": []\n",
            "  }\n",
            "}\n"
          ]
        }
      ],
      "source": [
        "import json\n",
        "\n",
        "published_model_uid = client.repository.get_model_uid(published_model)\n",
        "model_details = client.repository.get_details(published_model_uid)\n",
        "print(json.dumps(model_details, indent=2))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### 3.3 Get all models in the space"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "space_id is automatically picked up from client.set.default_space() api call before"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "pycharm": {
          "is_executing": false,
          "name": "#%%\n"
        }
      },
      "outputs": [],
      "source": [
        "models_details = client.repository.list_models()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "<a id=\"export\"></a>\n",
        "## 4. Export"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Help on method start in module ibm_watson_machine_learning.export_assets:\n",
            "\n",
            "start(meta_props, space_id=None, project_id=None) method of ibm_watson_machine_learning.export_assets.Export instance\n",
            "     Start the export. Either space_id or project_id has to be provided and is mandatory.\n",
            "     ALL_ASSETS is by default False. No need to provide explicitly unless it has to be set to True\n",
            "     Either ALL_ASSETS or ASSET_TYPES or ASSET_IDS has to be given in the meta_props. Only one of these can be\n",
            "     provided\n",
            "    \n",
            "     In the meta_props:\n",
            "    \n",
            "     ALL_ASSETS is a boolean. When set to True, it exports all assets in the given space\n",
            "     ASSET_IDS is an array containing the list of assets ids to be exported\n",
            "     ASSET_TYPES is for providing the asset types to be exported. All assets of that asset type will be exported\n",
            "                 Eg: wml_model, wml_model_definition, wml_pipeline, wml_function, wml_experiment,\n",
            "                 software_specification, hardware_specification, package_extension, script\n",
            "    \n",
            "    **Parameters**\n",
            "    \n",
            "    .. important::\n",
            "         #. **meta_props**:  meta data. To see available meta names use **client.export_assets.ConfigurationMetaNames.get()**\n",
            "    \n",
            "            **type**: dict\n",
            "    \n",
            "         #. **space_id**:  Space identifier**\n",
            "    \n",
            "            **type**: str\n",
            "    \n",
            "         #. **project_id**:  Project identifier**\n",
            "    \n",
            "            **type**: str\n",
            "    \n",
            "    \n",
            "    **Output**\n",
            "    \n",
            "    .. important::\n",
            "         **returns**: Response json\n",
            "    \n",
            "         **return type**: dict\n",
            "    \n",
            "    \n",
            "    **Example**\n",
            "    \n",
            "         >>> metadata = {\n",
            "         >>>    client.export_assets.ConfigurationMetaNames.NAME: \"export_model\",\n",
            "         >>>    client.export_assets.ConfigurationMetaNames.ASSET_IDS: [\"13a53931-a8c0-4c2f-8319-c793155e7517\", \"13a53931-a8c0-4c2f-8319-c793155e7518\"]\n",
            "         >>> }\n",
            "         >>> details = client.export_assets.start(meta_props=metadata, space_id=\"98a53931-a8c0-4c2f-8319-c793155e4598\")\n",
            "    \n",
            "         >>> metadata = {\n",
            "         >>>    client.export_assets.ConfigurationMetaNames.NAME: \"export_model\",\n",
            "         >>>    client.export_assets.ConfigurationMetaNames.ASSET_TYPES: [\"wml_model\"],\n",
            "         >>> }\n",
            "         >>> details = client.export_assets.start(meta_props=metadata, space_id=\"98a53931-a8c0-4c2f-8319-c793155e4598\")\n",
            "    \n",
            "         >>> metadata = {\n",
            "         >>>    client.export_assets.ConfigurationMetaNames.NAME: \"export_model\",\n",
            "         >>>    client.export_assets.ConfigurationMetaNames.ALL_ASSETS: True,\n",
            "         >>> }\n",
            "\n"
          ]
        }
      ],
      "source": [
        "help(client.export_assets.start)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "client.export_assets has these apis. For any help on these apis, type 'help(api_name)' in your notebook \n",
        "Example: help(client.export_assets.start), help(client.export_assets.get_details)\n",
        "\n",
        "1. client.export_assets.start: This starts the export job. export job is asynchronously executed\n",
        "2. client.export_assets.get_details: Given export_id and corresponding space_id/project_id, this gives the export job details. Usually used for monitoring the export job submitted with start api\n",
        "3. client.export_assets.list: Prints summary of all the export jobs\n",
        "4. client.export_assets.get_exported_content: Downloads the exported content. This information will be used by the import process\n",
        "5. client.export_assets.delete: Deletes the given export job\n",
        "6. client.export_assets.cancel: Cancels the given export job if running"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### 4.1: Start the export process"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Start the export process for the model created. Either ASSET_IDS or ASSET_TYPES or ALL_ASSETS can be provided.\n",
        "If you have more than one model ids, you need to provide them as array like client.export_assets.ConfigurationMetaNames.ASSET_IDS: [model_id1, model_id2]\n",
        "Refer to the help api above to see different usages and details"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "export job with id a182d2e2-8d1f-46b9-9b11-28e2d2e7ce9a has started. Monitor status using client.export_assets.get_details api. Check 'help(client.export_assets.get_details)' for details on the api usage\n",
            "{\n",
            "  \"entity\": {\n",
            "    \"assets\": {\n",
            "      \"all_assets\": false,\n",
            "      \"asset_ids\": [\n",
            "        \"7bd7c597-01f5-496f-8131-f9464248cd4b\"\n",
            "      ]\n",
            "    },\n",
            "    \"status\": {\n",
            "      \"state\": \"pending\"\n",
            "    }\n",
            "  },\n",
            "  \"metadata\": {\n",
            "    \"created_at\": \"2020-12-08T13:30:00.306Z\",\n",
            "    \"creator_id\": \"1000330999\",\n",
            "    \"id\": \"a182d2e2-8d1f-46b9-9b11-28e2d2e7ce9a\",\n",
            "    \"name\": \"export_model\",\n",
            "    \"space_id\": \"be32da95-1735-4c8a-96aa-e4d6b8b6a340\",\n",
            "    \"url\": \"/v2/asset_exports/a182d2e2-8d1f-46b9-9b11-28e2d2e7ce9a\"\n",
            "  }\n",
            "}\n"
          ]
        }
      ],
      "source": [
        "metadata = { client.export_assets.ConfigurationMetaNames.NAME: \"export_model\",\n",
        "             client.export_assets.ConfigurationMetaNames.ASSET_IDS: [published_model_uid]                                                                         \n",
        "           }\n",
        "\n",
        "details = client.export_assets.start(meta_props=metadata, space_id=export_space_id)\n",
        "print(json.dumps(details, indent=2))\n",
        "\n",
        "export_job_id = details[u'metadata'][u'id']"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### 4.2: Monitor the export process"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "pending\n",
            "completed\n",
            "{\n",
            "  \"entity\": {\n",
            "    \"assets\": {\n",
            "      \"all_assets\": false,\n",
            "      \"asset_ids\": [\n",
            "        \"7bd7c597-01f5-496f-8131-f9464248cd4b\"\n",
            "      ]\n",
            "    },\n",
            "    \"status\": {\n",
            "      \"state\": \"completed\"\n",
            "    }\n",
            "  },\n",
            "  \"metadata\": {\n",
            "    \"created_at\": \"2020-12-08T13:30:00.306Z\",\n",
            "    \"creator_id\": \"1000330999\",\n",
            "    \"id\": \"a182d2e2-8d1f-46b9-9b11-28e2d2e7ce9a\",\n",
            "    \"name\": \"export_model\",\n",
            "    \"space_id\": \"be32da95-1735-4c8a-96aa-e4d6b8b6a340\",\n",
            "    \"updated_at\": \"2020-12-08T13:30:07.102Z\",\n",
            "    \"url\": \"/v2/asset_exports/a182d2e2-8d1f-46b9-9b11-28e2d2e7ce9a\"\n",
            "  }\n",
            "}\n"
          ]
        }
      ],
      "source": [
        "import time\n",
        "\n",
        "start_time = time.time()\n",
        "diff_time = start_time - start_time\n",
        "while True and diff_time < 10 * 60:\n",
        "    time.sleep(3)\n",
        "    response = client.export_assets.get_details(export_job_id, space_id=export_space_id)\n",
        "    state = response[u'entity'][u'status'][u'state']\n",
        "    print(state)\n",
        "    if state == 'completed' or state == 'error' or state == 'failed':\n",
        "        break\n",
        "    diff_time = time.time() - start_time\n",
        "\n",
        "print(json.dumps(response, indent=2))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### 4.3: Get the exported content"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Successfully saved export content to file: 'EXPORT_DATA/exported_content_b2d90b7d-f95b-4121-84cc-58cab5d43de4.zip'\n",
            "EXPORT_DATA/exported_content_b2d90b7d-f95b-4121-84cc-58cab5d43de4.zip\n"
          ]
        }
      ],
      "source": [
        "export_dir = 'EXPORT_DATA'\n",
        "\n",
        "if not os.path.isdir(export_dir):\n",
        "    os.mkdir(export_dir)\n",
        "    \n",
        "export_file_name = 'exported_content_' + str(uuid.uuid4()) + '.zip'\n",
        "    \n",
        "export_file_path = os.path.join(export_dir, export_file_name)\n",
        "\n",
        "details = client.export_assets.get_exported_content(export_job_id, \n",
        "                                                    space_id = export_space_id, \n",
        "                                                    file_path = export_file_path)\n",
        "\n",
        "print(details)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "<a id=\"import\"></a>\n",
        "## 5. Import"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "client.import_assets has these apis. For any help on these apis, type 'help(api_name)' in your notebook \n",
        "Example: help(client.import_assets.start), help(client.import_assets.get_details)\n",
        "\n",
        "1. client.import_assets.start: This starts the import job. import job is asynchronously executed\n",
        "2. client.import_assets.get_details: Given import_id and corresponding space_id/project_id, this gives the import job details. Usually used for monitoring the import job submitted with start api\n",
        "3. client.import_assets.list: Prints summary of all the import jobs\n",
        "4. client.import_assets.delete: Deletes the given import job\n",
        "5. client.import_assets.cancel: Cancels the given import job if running"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### 5.1: Start the import process"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "import job with id 6d7c3f3f-6a9b-4942-b05f-d4381222fa73 has started. Monitor status using client.import_assets.get_details api. Check 'help(client.import_assets.get_details)' for details on the api usage\n",
            "{\n",
            "  \"entity\": {\n",
            "    \"status\": {\n",
            "      \"state\": \"pending\"\n",
            "    }\n",
            "  },\n",
            "  \"metadata\": {\n",
            "    \"created_at\": \"2020-12-08T13:30:41.646Z\",\n",
            "    \"creator_id\": \"1000330999\",\n",
            "    \"id\": \"6d7c3f3f-6a9b-4942-b05f-d4381222fa73\",\n",
            "    \"space_id\": \"68dbf09b-5ddb-4c0c-a925-2ee9327f511d\",\n",
            "    \"url\": \"/v2/asset_imports/6d7c3f3f-6a9b-4942-b05f-d4381222fa73\"\n",
            "  }\n",
            "}\n"
          ]
        }
      ],
      "source": [
        "details = client.import_assets.start(file_path=export_file_path,\n",
        "                                     space_id=import_space_id)\n",
        "print(json.dumps(details, indent=2))\n",
        "\n",
        "import_job_id = details[u'metadata'][u'id']"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### 5.2: Monitor the import process"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "completed\n",
            "{\n",
            "  \"entity\": {\n",
            "    \"status\": {\n",
            "      \"state\": \"completed\"\n",
            "    }\n",
            "  },\n",
            "  \"metadata\": {\n",
            "    \"created_at\": \"2020-12-08T13:30:41.646Z\",\n",
            "    \"creator_id\": \"1000330999\",\n",
            "    \"id\": \"6d7c3f3f-6a9b-4942-b05f-d4381222fa73\",\n",
            "    \"space_id\": \"68dbf09b-5ddb-4c0c-a925-2ee9327f511d\",\n",
            "    \"updated_at\": \"2020-12-08T13:30:44.922Z\",\n",
            "    \"url\": \"/v2/asset_imports/6d7c3f3f-6a9b-4942-b05f-d4381222fa73\"\n",
            "  }\n",
            "}\n",
            "\n",
            "List of models: \n",
            "\n",
            "------------------------------------  --------------------  ------------------------  --------------\n",
            "ID                                    NAME                  CREATED                   TYPE\n",
            "0e22514a-927b-4909-a562-e12ca0c46c29  External Keras model  2020-12-08T13:30:43.002Z  tensorflow_2.1\n",
            "------------------------------------  --------------------  ------------------------  --------------\n",
            "\n",
            "model id for deployment: 0e22514a-927b-4909-a562-e12ca0c46c29\n"
          ]
        }
      ],
      "source": [
        "import time\n",
        "\n",
        "start_time = time.time()\n",
        "diff_time = start_time - start_time\n",
        "while True and diff_time < 10 * 60:\n",
        "    time.sleep(3)\n",
        "    response = client.import_assets.get_details(import_job_id,\n",
        "                                                space_id=import_space_id)\n",
        "    state = response[u'entity'][u'status'][u'state']\n",
        "    print(state)\n",
        "    if state == 'completed' or state == 'error' or state == 'failed':\n",
        "         break\n",
        "    diff_time = time.time() - start_time\n",
        "\n",
        "print(json.dumps(response, indent=2))\n",
        "\n",
        "client.set.default_space(import_space_id)\n",
        "\n",
        "print(\"{}List of models: {}\".format('\\n', '\\n'))\n",
        "client.repository.list_models()\n",
        "details = client.repository.get_model_details()\n",
        "\n",
        "for obj in details[u'resources']:\n",
        "    if obj[u'metadata'][u'name'] == \"External Keras model\":\n",
        "        model_id_for_deployment = obj[u'metadata'][u'id']\n",
        "\n",
        "print(\"{}model id for deployment: {}\".format('\\n', model_id_for_deployment))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "List the import and export jobs"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Export jobs: \n",
            "\n",
            "------------------------------------  ------------  ------------------------  ---------\n",
            "ID                                    NAME          CREATED                   STATUS\n",
            "a182d2e2-8d1f-46b9-9b11-28e2d2e7ce9a  export_model  2020-12-08T13:30:00.306Z  completed\n",
            "------------------------------------  ------------  ------------------------  ---------\n",
            "\n",
            "Import jobs:\n",
            "------------------------------------  ------------------------  ---------\n",
            "ID                                    CREATED                   STATUS\n",
            "6d7c3f3f-6a9b-4942-b05f-d4381222fa73  2020-12-08T13:30:41.646Z  completed\n",
            "------------------------------------  ------------------------  ---------\n"
          ]
        }
      ],
      "source": [
        "print(\"Export jobs: \\n\")\n",
        "client.export_assets.list(space_id=export_space_id)\n",
        "print(\"\\nImport jobs:\")\n",
        "client.import_assets.list(space_id=import_space_id)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "<a id=\"scoring\"></a>\n",
        "## 6. Deploy and score the imported model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### 6.1: Create model deployment"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "#### Create online deployment for published model"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "metadata": {
        "pycharm": {
          "is_executing": false,
          "name": "#%%\n"
        }
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "\n",
            "#######################################################################################\n",
            "\n",
            "Synchronous deployment creation for uid: '0e22514a-927b-4909-a562-e12ca0c46c29' started\n",
            "\n",
            "#######################################################################################\n",
            "\n",
            "\n",
            "initializing.\n",
            "ready\n",
            "\n",
            "\n",
            "------------------------------------------------------------------------------------------------\n",
            "Successfully finished deployment creation, deployment_uid='eb2c9c76-c5eb-455d-994c-02331e1af342'\n",
            "------------------------------------------------------------------------------------------------\n",
            "\n",
            "\n"
          ]
        }
      ],
      "source": [
        "metadata = {\n",
        "    client.deployments.ConfigurationMetaNames.NAME: \"Deployment of external Keras model\",\n",
        "    client.deployments.ConfigurationMetaNames.ONLINE: {}\n",
        "}\n",
        "\n",
        "created_deployment = client.deployments.create(model_id_for_deployment, meta_props=metadata)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "metadata": {},
      "outputs": [],
      "source": [
        "deployment_uid = client.deployments.get_uid(created_deployment)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Now you can print an online scoring endpoint. "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "metadata": {
        "pycharm": {
          "is_executing": false,
          "name": "#%%\n"
        }
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "https://wmlgmc-cpd-wmlgmc.apps.wmlautoai.cp.fyre.ibm.com/ml/v4/deployments/eb2c9c76-c5eb-455d-994c-02331e1af342/predictions\n"
          ]
        }
      ],
      "source": [
        "scoring_endpoint = client.deployments.get_scoring_href(created_deployment)\n",
        "print(scoring_endpoint)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "You can also list existing deployments."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "client.deployments.list()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### 6.2: Get deployment details"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 21,
      "metadata": {
        "pycharm": {
          "is_executing": false,
          "name": "#%%\n"
        }
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "{\n",
            "  \"entity\": {\n",
            "    \"asset\": {\n",
            "      \"id\": \"0e22514a-927b-4909-a562-e12ca0c46c29\"\n",
            "    },\n",
            "    \"custom\": {},\n",
            "    \"deployed_asset_type\": \"model\",\n",
            "    \"hardware_spec\": {\n",
            "      \"id\": \"Not_Applicable\",\n",
            "      \"name\": \"XXS\",\n",
            "      \"num_nodes\": 1\n",
            "    },\n",
            "    \"name\": \"Deployment of external Keras model\",\n",
            "    \"online\": {},\n",
            "    \"space_id\": \"68dbf09b-5ddb-4c0c-a925-2ee9327f511d\",\n",
            "    \"status\": {\n",
            "      \"online_url\": {\n",
            "        \"url\": \"https://wmlgmc-cpd-wmlgmc.apps.wmlautoai.cp.fyre.ibm.com/ml/v4/deployments/eb2c9c76-c5eb-455d-994c-02331e1af342/predictions\"\n",
            "      },\n",
            "      \"state\": \"ready\"\n",
            "    }\n",
            "  },\n",
            "  \"metadata\": {\n",
            "    \"created_at\": \"2020-12-08T13:30:59.496Z\",\n",
            "    \"id\": \"eb2c9c76-c5eb-455d-994c-02331e1af342\",\n",
            "    \"modified_at\": \"2020-12-08T13:30:59.496Z\",\n",
            "    \"name\": \"Deployment of external Keras model\",\n",
            "    \"owner\": \"1000330999\",\n",
            "    \"space_id\": \"68dbf09b-5ddb-4c0c-a925-2ee9327f511d\"\n",
            "  }\n",
            "}\n"
          ]
        }
      ],
      "source": [
        "details = client.deployments.get_details(deployment_uid)\n",
        "print(json.dumps(details, indent=2))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### 6.3: Score"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "You can use below method to do test scoring request against deployed model."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Let's first visualize two samples from dataset, we'll use for scoring. You must have matplotlib package installed"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 22,
      "metadata": {},
      "outputs": [],
      "source": [
        "%matplotlib inline\n",
        "import matplotlib.pyplot as plt"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 23,
      "metadata": {},
      "outputs": [
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAScAAABxCAYAAACX+mUfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAHF0lEQVR4nO3dzUsWXxTA8bF+kZRlSWEGJZFgSZkLIYzA3hYWFZX0AhEtXNWiRYsW0qYXejVaVKv6A4SKSqQXSAiSiAyiAgUXlUFEkUm0kKzot/jB4Z75NY+PjzPjuc/z/azO4Q4zdzEc7ty5c6foz58/AQBYM2miOwAAf0NxAmASxQmASRQnACZRnACYRHECYNI/o7SzzsCOoonuQJ7h3rbjr/c2IycAJlGcAJhEcQJgEsUJgEkUJwAmUZwAmERxAmASxQmASRQnACZRnACYNNrnKwAS1tbWpvLh4WGJX716pdpu3LgReZ4DBw6ovKGhQeJ9+/aNp4sTgpETAJMoTgBMKhrlBwd8uW0HuxLEa0Lv7d27d0t8/fr1RK5RVVUl8cOHD1XbwoULE7lmjtiVAIA/KE4ATKI4ATCJOSd/MOcUr1TvbXeOKQiyn2dasmSJypuamiR+8+aNauvo6Ig8z8mTJ1Xe2tqa1fVTwpwTAH9QnACYxApxICHPnz+X+NatW5HHLVu2TOXu49mcOXNUW0lJicQjIyOqbeXKlSp/+fKlxIODg1n02BZGTgBMojgBMIniBMCk1Oecwl9VX716VeL58+ertuLiYon37t2r2ubNmyexu0wfsOLjx48Sh5fsuPNMDx48UG0VFRVZnT+8m0FfX1/ksZs3b87qnJYwcgJgEsUJgEmprxBftGiRyt+9e5fTeWbOnClxTU3NeLqUkwULFkh85MgR1VZfX5/EJVkhHq9UV4gPDAyofMaMGRKXlZXldM4VK1ao/PXr15HHdnV1qXzt2rU5XTMhrBAH4A+KEwCTKE4ATEp9KcG1a9dU7i6xD88d9fb2SvzixQvV9ujRI4mfPn2q2txd/t6/f59136ZMmaJy99MB97Vw+Jru/FMQJDbnBI9VVlbGcp7z589L3N/fn/FY93OW8KctPmDkBMAkihMAk7zdbG5oaEji8COf+1jV09OT9TmnTp2q8urqaonDm359/fpV4itXrqi2gwcPZn3NMWApQbzM3tuuzs5Ole/cuVPiHz9+qLby8nKVt7e3S9zY2JhA72LDUgIA/qA4ATCJ4gTAJG93wpw9e7bE69atizxu/fr1OV/j5s2bErtzXEEQBLW1tRLv2bMn52sAmbi7aQbB/+eZXOGfKBifZxoVIycAJlGcAJjk7VKCJHz+/Fnly5cvj2xzN81rbm5OtmP/YSlBvMze29u2bZM4vBGd+1i3f/9+1Xbp0iWVuz9DMI6lBAD8QXECYBLFCYBJ3i4lSEL4MxR3nmnWrFmqzf20BRiP8I4XT548kTi8dGDu3LkSHz16VLV5NMeUFUZOAEyiOAEwqeAf67q7uyU+c+ZM5HF37txRefj/9kCuduzYofIvX75EHuv+v3Hx4sWJ9ckCRk4ATKI4ATCJ4gTApIKfc7p7967EIyMjqm3Dhg0SNzQ0pNYn5L+Ojg6Jwzu5utasWaPy48ePJ9Ulcxg5ATCJ4gTAJIoTAJMKbs5peHhY5ffv35c4/PeVY8eOSRz+4SYwFoODgyo/deqUxOG5TlddXZ3K8+0TlUwYOQEwieIEwKSCe6xz/zUfBPo17saNG1XbqlWrUukT8t+FCxdU/uzZs8hj3Z0wC2npQBgjJwAmUZwAmERxAmBS3v99pbOzU+Xbt29X+fTp0yW+d++eajP2yQp/X4lXqvd2cXGxyjMtH/jw4YPEFRUVifXJEP6+AsAfFCcAJuXlUgJ3Ne6hQ4dU269fv1S+adMmiY09xqFAuffveL5MKC0tjTzPz58/Jf727VvkOYaGhlR+8eLFrK49efJklZ89e1biadOmZXUORk4ATKI4ATCJ4gTApLyYc/r9+7fKm5qaJH779q1qq6qqUvmJEyeS6xiQg9ra2ljOs2vXLonDSxI+ffokcXt7eyzXy6S8vFzi8M9AozByAmASxQmASXmxQry/v1/l1dXVkce6G8sHQRBs2bIlkT4lgBXi8Ur13g7/OPP27dtpXn5M3GUHkyZFj1+2bt2q8vr6+shjV69eLfFfluywQhyAPyhOAEyiOAEwyds5p4GBAYkbGxsj29ra2lTb4cOHVV5U5M1Ujjcd9cSE3tvnzp2TONMOBWG9vb0Sj2UJQEtLi8orKysjj21ubpZ46dKlWV9jHJhzAuAPihMAk7x9rGttbZX49OnTkcf19PSoPNPrTuN4rIuX2Xu7APFYB8AfFCcAJlGcAJjkza4Ejx8/Vvnly5cnqCcA0sDICYBJFCcAJnnzWNfd3a3y79+/Rx7rbihXUlKSWJ8AJIeREwCTKE4ATKI4ATDJmzmnTOrq6lTe1dUlcVlZWdrdARADRk4ATKI4ATDJ210JChC7EsSLe9sOdiUA4A+KEwCTKE4ATBptKQHzHMhX3NvGMXICYBLFCYBJFCcAJlGcAJhEcQJgEsUJgEn/ArNniicMP7ZlAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 2 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "for i, image in enumerate([x_test[0], x_test[1]]):\n",
        "    plt.subplot(2, 2, i + 1)\n",
        "    plt.axis('off')\n",
        "    plt.imshow(image, cmap=plt.cm.gray_r, interpolation='nearest')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Prepare scoring payload with records to score."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 24,
      "metadata": {},
      "outputs": [],
      "source": [
        "score_0 = x_test[0].flatten().tolist()\n",
        "score_1 = x_test[1].flatten().tolist()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 25,
      "metadata": {
        "pycharm": {
          "is_executing": false,
          "name": "#%%\n"
        }
      },
      "outputs": [],
      "source": [
        "scoring_payload = {\"input_data\": [{\"values\": [score_0, score_1]}]}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Use ``client.deployments.score()`` method to run scoring."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 26,
      "metadata": {
        "pycharm": {
          "is_executing": false,
          "name": "#%%\n"
        }
      },
      "outputs": [],
      "source": [
        "predictions = client.deployments.score(deployment_uid, scoring_payload)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 27,
      "metadata": {
        "pycharm": {
          "is_executing": false,
          "name": "#%%\n"
        }
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "{\n",
            "  \"predictions\": [\n",
            "    {\n",
            "      \"id\": \"dense_2\",\n",
            "      \"fields\": [\n",
            "        \"prediction\",\n",
            "        \"prediction_classes\",\n",
            "        \"probability\"\n",
            "      ],\n",
            "      \"values\": [\n",
            "        [\n",
            "          [\n",
            "            0.0,\n",
            "            0.0,\n",
            "            0.0,\n",
            "            0.0,\n",
            "            0.0,\n",
            "            0.0,\n",
            "            0.0,\n",
            "            1.0,\n",
            "            0.0,\n",
            "            0.0\n",
            "          ],\n",
            "          7,\n",
            "          [\n",
            "            0.0,\n",
            "            0.0,\n",
            "            0.0,\n",
            "            0.0,\n",
            "            0.0,\n",
            "            0.0,\n",
            "            0.0,\n",
            "            1.0,\n",
            "            0.0,\n",
            "            0.0\n",
            "          ]\n",
            "        ],\n",
            "        [\n",
            "          [\n",
            "            0.0,\n",
            "            0.0,\n",
            "            1.0,\n",
            "            0.0,\n",
            "            0.0,\n",
            "            0.0,\n",
            "            0.0,\n",
            "            0.0,\n",
            "            0.0,\n",
            "            0.0\n",
            "          ],\n",
            "          2,\n",
            "          [\n",
            "            0.0,\n",
            "            0.0,\n",
            "            1.0,\n",
            "            0.0,\n",
            "            0.0,\n",
            "            0.0,\n",
            "            0.0,\n",
            "            0.0,\n",
            "            0.0,\n",
            "            0.0\n",
            "          ]\n",
            "        ]\n",
            "      ]\n",
            "    }\n",
            "  ]\n",
            "}\n"
          ]
        }
      ],
      "source": [
        "print(json.dumps(predictions, indent=2))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "<a id=\"cleanup\"></a>\n",
        "## 7. Clean up "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 28,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Export job deleted\n",
            "Import job deleted\n",
            "DELETED\n",
            "DELETED\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "'SUCCESS'"
            ]
          },
          "execution_count": 28,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "client.export_assets.delete(export_job_id, space_id=export_space_id)\n",
        "client.import_assets.delete(import_job_id, space_id=import_space_id)\n",
        "\n",
        "client.spaces.delete(export_space_id)\n",
        "client.spaces.delete(import_space_id)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "If you want to clean up all created assets:\n",
        "- experiments\n",
        "- trainings\n",
        "- pipelines\n",
        "- model definitions\n",
        "- models\n",
        "- functions\n",
        "- deployments\n",
        "\n",
        "please follow up this sample [notebook](https://github.com/IBM/watson-machine-learning-samples/blob/master/cpd4.0/notebooks/python_sdk/instance-management/Machine%20Learning%20artifacts%20management.ipynb)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "<a id=\"summary\"></a>\n",
        "## 8. Summary and next steps"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        " You successfully completed this notebook! You learned how to use export/import assets client apis. Check out our _[Online Documentation](https://console.ng.bluemix.net/docs/services/PredictiveModeling/index.html?pos=2)_ for more samples, tutorials, documentation, how-tos, and blog posts. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Authors\n",
        "\n",
        "*Mithun - *vbmithun@in.ibm.com**, Software Engineer"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Copyright © 2020-2025 IBM. This notebook and its source code are released under the terms of the MIT License."
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.10"
    },
    "pycharm": {
      "stem_cell": {
        "cell_type": "raw",
        "metadata": {
          "collapsed": false
        },
        "source": []
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 4
}