4_improved_descriptor.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "4_improved_descriptor.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/antonioenas/Denoising_HPatches_for_Descriptor_Learning/blob/master/4_improved_descriptor.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "metadata": {
        "id": "ZInL8SHokoU3",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# 4) Improved descriptor - Deep learning coursework, Imperial College London\n",
        "**Antonio Enas, 01070859, AE1115**\n",
        "\n",
        "\n",
        "---\n",
        "\n",
        "\n",
        "In this notebook, the user can generate, train and save the improved descriptor model.\n",
        "\n",
        "The notebook can be used to train both onto clean patches and denoised patches, prior loading of a denoiser model.\n",
        "To use denoised patches, load a Keras .h5 model in `denoiser_model` and set `use_clean_bool = True`.\n",
        "\n",
        "\n",
        "Structure:\n",
        "1. Environment set up\n",
        "2. Define model and loss\n",
        "3. Model training"
      ]
    },
    {
      "metadata": {
        "id": "LP0dAehykbu5",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "use_clean_bool = True\n",
        "denoiser_model = None"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "6_QUBl_OC0Hv",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# 1. Environment set up\n",
        "This section ensures all libraries and dataset are correctly imported.\n",
        "\n",
        "If data is already mounted select:\n",
        "              `import_data = False`"
      ]
    },
    {
      "metadata": {
        "id": "-nm263h1Ae_j",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import_data = False"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "BUgnMDQT1C67",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Optional GPU check"
      ]
    },
    {
      "metadata": {
        "id": "ZZG4BqkENEyd",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Taken from\n",
        "# https://stackoverflow.com/questions/48750199/google-colaboratory-misleading-information-about-its-gpu-only-5-ram-available\n",
        "# memory footprint support libraries/code\n",
        "!ln -sf /opt/bin/nvidia-smi /usr/bin/nvidia-smi\n",
        "!pip install gputil\n",
        "!pip install psutil\n",
        "!pip install humanize\n",
        "import psutil\n",
        "import humanize\n",
        "import os\n",
        "import GPUtil as GPU\n",
        "GPUs = GPU.getGPUs()\n",
        "# Colab only provides one GPU and it is not always guaranteed\n",
        "gpu = GPUs[0]\n",
        "def printm():\n",
        "  process = psutil.Process(os.getpid())\n",
        "  print(\"RAM Free: \" + humanize.naturalsize( psutil.virtual_memory().available ), \" | Proc size: \" + humanize.naturalsize( process.memory_info().rss))\n",
        "  print(\"GPU RAM Free: {0:.0f}MB | Used: {1:.0f}MB | Util {2:3.0f}% | Total {3:.0f}MB\".format(gpu.memoryFree, gpu.memoryUsed, gpu.memoryUtil*100, gpu.memoryTotal))\n",
        "printm()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "_rUCd3Te1KMl",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Cloning the repository and downloading the data"
      ]
    },
    {
      "metadata": {
        "id": "ytiLXn8wmoQP",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "if import_data == False:\n",
        "  # Change directory\n",
        "  %cd /content/keras_triplet_descriptor\n",
        "else:\n",
        "  # Clone repo\n",
        "  !git clone https://github.com/MatchLab-Imperial/keras_triplet_descriptor\n",
        "  %cd /content/keras_triplet_descriptor\n",
        "  # Download data\n",
        "  !wget -O hpatches_data.zip https://imperialcollegelondon.box.com/shared/static/ah40eq7cxpwq4a6l4f62efzdyt8rm3ha.zip\n",
        "  # Extract data\n",
        "  !unzip -q ./hpatches_data.zip\n",
        "  !rm ./hpatches_data.zip\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "Rjyr96hR_4wS",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Importing Necessary Modules"
      ]
    },
    {
      "metadata": {
        "id": "o0KYfe-at9KN",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "!pip install -U seaborn\n",
        "\n",
        "import sys\n",
        "import json\n",
        "import os\n",
        "import glob\n",
        "import time\n",
        "import tensorflow as tf\n",
        "import numpy as np\n",
        "import cv2\n",
        "import random\n",
        "\n",
        "import keras\n",
        "from keras import backend as K\n",
        "from keras.models import Sequential, Model\n",
        "from keras.layers import Dense, Dropout, Activation, Flatten, Input, Lambda, Reshape\n",
        "from keras.layers import Conv2D, MaxPooling2D, BatchNormalization \n",
        "from keras.layers import Input, UpSampling2D, concatenate  \n",
        "\n",
        "from read_data import HPatches, DataGeneratorDesc, hpatches_sequence_folder, DenoiseHPatches, tps\n",
        "from utils import generate_desc_csv, plot_denoise, plot_triplet\n",
        "\n",
        "# Plotting tools\n",
        "import matplotlib.pyplot as plt\n",
        "import seaborn as sns\n",
        "%matplotlib inline\n",
        "plt.style.use('seaborn-white')\n",
        "plt.rcParams[\"patch.force_edgecolor\"] = True\n",
        "\n",
        "# To download files\n",
        "from google.colab import files\n",
        "\n",
        "from sklearn.decomposition import PCA\n",
        "import pandas as pd"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "FwFuggKiFh-Y",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "random.seed(1234)\n",
        "np.random.seed(1234)\n",
        "tf.set_random_seed(1234)\n",
        "\n",
        "hpatches_dir = './hpatches'\n",
        "splits_path = './splits.json'\n",
        "\n",
        "splits_json = json.load(open(splits_path, 'rb'))\n",
        "split = splits_json['a']\n",
        "\n",
        "train_fnames = split['train']\n",
        "test_fnames = split['test']\n",
        "\n",
        "seqs = glob.glob(hpatches_dir+'/*')\n",
        "seqs = [os.path.abspath(p) for p in seqs]   \n",
        "seqs_train = list(filter(lambda x: x.split('/')[-1] in train_fnames, seqs)) \n",
        "seqs_test = list(filter(lambda x: x.split('/')[-1] in split['test'], seqs)) \n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "Fz9DKXVfPvES",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# 2. Define model and loss\n",
        "In this section the baseline architecture is recreated with different losses."
      ]
    },
    {
      "metadata": {
        "id": "sZkAJV1bSyZi",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Architecture"
      ]
    },
    {
      "metadata": {
        "id": "MNnSwg7p1pPY",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "def get_descriptor_model(shape):\n",
        "  \n",
        "  '''Architecture copies HardNet architecture'''\n",
        "  \n",
        "  init_weights = keras.initializers.he_normal()\n",
        "  \n",
        "  descriptor_model = Sequential()\n",
        "  descriptor_model.add(Conv2D(32, 3, padding='same', input_shape=shape, use_bias = True, kernel_initializer=init_weights))\n",
        "  descriptor_model.add(BatchNormalization(axis = -1))\n",
        "  descriptor_model.add(Activation('relu'))\n",
        "\n",
        "  descriptor_model.add(Conv2D(32, 3, padding='same', use_bias = True, kernel_initializer=init_weights))\n",
        "  descriptor_model.add(BatchNormalization(axis = -1))\n",
        "  descriptor_model.add(Activation('relu'))\n",
        "\n",
        "  descriptor_model.add(Conv2D(64, 3, padding='same', strides=2, use_bias = True, kernel_initializer=init_weights))\n",
        "  descriptor_model.add(BatchNormalization(axis = -1))\n",
        "  descriptor_model.add(Activation('relu'))\n",
        "\n",
        "  descriptor_model.add(Conv2D(64, 3, padding='same', use_bias = True, kernel_initializer=init_weights))\n",
        "  descriptor_model.add(BatchNormalization(axis = -1))\n",
        "  descriptor_model.add(Activation('relu'))\n",
        "\n",
        "  descriptor_model.add(Conv2D(128, 3, padding='same', strides=2,  use_bias = True, kernel_initializer=init_weights))\n",
        "  descriptor_model.add(BatchNormalization(axis = -1))\n",
        "  descriptor_model.add(Activation('relu'))\n",
        "\n",
        "  descriptor_model.add(Conv2D(128, 3, padding='same', use_bias = True, kernel_initializer=init_weights))\n",
        "  descriptor_model.add(BatchNormalization(axis = -1))\n",
        "  descriptor_model.add(Activation('relu'))\n",
        "  descriptor_model.add(Dropout(0.3))\n",
        "\n",
        "  descriptor_model.add(Conv2D(128, 8, padding='valid', use_bias = True, kernel_initializer=init_weights))\n",
        "  \n",
        "  # Final descriptor reshape\n",
        "  descriptor_model.add(Reshape((128,)))\n",
        "  \n",
        "  return descriptor_model"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "LzWAFTy7uNm3",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Losses\n",
        "Several losses are here defined for experimentation."
      ]
    },
    {
      "metadata": {
        "id": "o07E4JRAuFDM",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Original baseline triplet loss"
      ]
    },
    {
      "metadata": {
        "id": "-S6MDsUSmWZw",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "def triplet_loss(x):\n",
        "  \n",
        "  output_dim = 128\n",
        "  a, p, n = x\n",
        "  _alpha = 1.0\n",
        "  positive_distance = K.mean(K.square(a - p), axis=-1)\n",
        "  negative_distance = K.mean(K.square(a - n), axis=-1)\n",
        "  \n",
        "  return K.expand_dims(K.maximum(0.0, positive_distance - negative_distance + _alpha), axis = 1)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "U9SKuFV-uwYe",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Lossless triplet loss"
      ]
    },
    {
      "metadata": {
        "id": "GAXjNV6_3ilX",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Lossless triplet loss\n",
        "def lossless_triplet_loss(x):\n",
        "  \n",
        "  output_dim = 128\n",
        "  a, p, n = x\n",
        "  _alpha = 0.2\n",
        "  positive_distance = K.mean(K.square(a - p), axis=-1)\n",
        "  negative_distance = K.mean(K.square(a - n), axis=-1)\n",
        "  \n",
        "  lossless = K.log(positive_distance+10+1e-18)-K.log(positive_distance+2+1e-18)\n",
        "  \n",
        "  return K.expand_dims(K.maximum(0.0, lossless), axis = 1)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "AEX7QXJfu1k0",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Quadratic skewed loss"
      ]
    },
    {
      "metadata": {
        "id": "0ojbFQh9Gc_1",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "def quadratic_triplet_loss(x):\n",
        "  \n",
        "  output_dim = 128\n",
        "  a, p, n = x\n",
        "  _alpha = 0.2\n",
        "  positive_distance = K.mean(K.square(a - p), axis=-1)\n",
        "  negative_distance = K.mean(K.square(a - n), axis=-1)\n",
        "  \n",
        "  lossless = (1+positive_distance+1e-12)**2-negative_distance\n",
        "  \n",
        "  return K.expand_dims(lossless, axis = 1)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "xP3ckNL5vMX5",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Inverse slope triplet loss"
      ]
    },
    {
      "metadata": {
        "id": "OJm9fF3DoqzN",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "def inverse_triplet_loss(x):\n",
        "  \n",
        "  output_dim = 128\n",
        "  a, p, n = x\n",
        "  _alpha = 0.2\n",
        "  positive_distance = K.mean(K.square(a - p), axis=-1)\n",
        "  negative_distance = K.mean(K.square(a - n), axis=-1)\n",
        "  \n",
        "  lossless = 1/(negative_distance+0.9)-1/(1.5+positive_distance)\n",
        "  \n",
        "  return K.expand_dims(K.maximum(0.0, lossless), axis = 1)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "BoMfk8nsvS4R",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Hard constraint N-positive regularised loss (**proposed**)"
      ]
    },
    {
      "metadata": {
        "id": "4UqJwCEYCWrp",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "def N_triplet_loss(x):\n",
        "  \n",
        "  # Margin\n",
        "  _alpha = 1.0\n",
        "  \n",
        "  # Unpack input\n",
        "  a, p, n = x\n",
        "  \n",
        "  # Evaluate distances within batch\n",
        "  positive_distance = K.mean(K.square(a - p), axis=-1)\n",
        "  negative_distance = K.mean(K.square(a - n), axis=-1)\n",
        "  \n",
        "  # Count good pairs\n",
        "  hard_mask = K.greater(negative_distance,negative_distance)\n",
        "  \n",
        "  # Adjusted triplet loss\n",
        "  _adjusted_loss = (positive_distance - negative_distance + 1)/K.sum(hard_mask)\n",
        "  \n",
        "  return K.expand_dims(K.maximum(0.0, lossless), axis = 1)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "ZJi1Z2EkuW0Y",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Compiler and generator"
      ]
    },
    {
      "metadata": {
        "id": "0XgihFjgmBgn",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Build the siamese network and choose optimiser."
      ]
    },
    {
      "metadata": {
        "id": "W1CmPPTmmOEa",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "optimiser_name = 'adam'"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "DVmDZIRTHPDa",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "shape = (32, 32, 1)\n",
        "xa = Input(shape=shape, name='a')\n",
        "xp = Input(shape=shape, name='p')\n",
        "xn = Input(shape=shape, name='n')\n",
        "descriptor_model = get_descriptor_model(shape)\n",
        "ea = descriptor_model(xa)\n",
        "ep = descriptor_model(xp)\n",
        "en = descriptor_model(xn)\n",
        "\n",
        "loss = Lambda(N_triplet_loss)([ea, ep, en])\n",
        "\n",
        "descriptor_model_trip = Model(inputs=[xa, xp, xn], outputs=loss)\n",
        "\n",
        "descriptor_model_trip.compile(loss='mean_absolute_error', optimizer=optimiser_name, metrics = ['accuracy'])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "OGpHsSSPwL72",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Plot a triplet."
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "outputId": "2bba443f-9cdf-4c27-c694-d1dfdd7088fa",
        "id": "ZKrKhGqOMn93",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 183
        }
      },
      "cell_type": "code",
      "source": [
        "# Plot an example of triplet\n",
        "plot_triplet(training_generator)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAcwAAACmCAYAAABXw78OAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi40LCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcv7US4rQAAIABJREFUeJztnXl0FUXaxh92UVQQZZF9uwlIgmGL\n7KCIIKuKwyoi4jpyFBxhPkFFwQUdx4OAB+S4OyP7IiKCAsoisiOIgAiogEQ0siOI0N8fnptJPV3c\nqhsSIvr8zskfb9/u6urq6q50PfW+b54gCAIIIYQQIiZ5c7sCQgghxLmABkwhhBDCAw2YQgghhAca\nMIUQQggPNGAKIYQQHmjAFEIIITzQgBkHu3btQkJCAh599NHcror4kxPta//85z+zZT8hcpK/Sj88\nZwfMESNGICEhAXXr1sWxY8dyuzriT8S0adOQkJAQ+qtevToaNmyIv//971i1alWO1qF48eIYOXIk\nevToYWx/6aWXsGvXLud+4o9NtI8lJydj586dp93v6quvxi233HIWa+bHX7Uf5s/tCmSFX3/9FdOn\nT0fevHlx6NAhfPDBB+jUqVNuV0v8yWjfvj1atmyZYR87dgzbt2/HxIkTsWDBAowYMQIdOnTIkXMX\nLlwYrVu3Nrbt3LkTI0eORJ06dVC2bNnT7ifOHY4fP45hw4bh5Zdfzu2qePNX7ofn5BfmvHnzsG/f\nPnTt2hV58uTBpEmTcrtK4k9IJBJB69atM/46deqEAQMGYMqUKShcuDCefPJJnDhx4qzVZ8OGDWft\nXOLskJqaik8++QTz5s3L7ap481fuh+fkgDlx4kQAwK233oo6depg9erV2LZtm7HP8uXLkZCQgJde\negmrV69Gjx49kJKSgpSUFPTt29c6DfL++++ja9euGfvddddd2Lhxo7UOW7duRZ8+fVC7dm2kpKTg\n9ttvx3fffRfa7913380oMykpCa1bt8YLL7yAo0ePGvslJCSgd+/eWLx4Ma677jo0atQoq80jcphy\n5cqhfv362L9/P7Zu3Qrg9y+F0aNH4/rrr0dycjJSUlJw8803Y/LkyaHjV6xYgTvvvBNNmjRBUlIS\nmjZtiv79++Orr77K2Ic1oVtuuQX9+/cHAPTq1QsJCQnYtWtXaL/u3bsjMTERP/zwQ+i8aWlpSExM\nNKbNfv75ZwwfPhxXX301atasidTUVNxzzz34/PPPs6/BxGm54447UKFCBTz55JOhd8LpmDJlCjp3\n7oxatWohJSUFN9xwA9566y2cOnXK2O/w4cMYNmwYGjdujOTkZHTu3BlLly7F5MmTkZCQgGnTphn7\nv//+++jRowdq166NpKQkXHfddRgxYgQOHjyYsc9fvR+ecwPmjh07sGLFCqSkpKBixYoZU7G2FxMA\nbNmyBffddx/q1auHRx99FG3btsXixYvRr18/Y79XXnkF/fv3R9GiRfHYY4+hf//+2Lp1K7p164b1\n69cb+6anp+Ouu+5CUlIShg4divbt22PJkiUYOHCgsd+YMWPw0EMPIQgC3H///Xj00Udx5ZVXYuzY\nsbjzzjtDHfzYsWN4/PHH0b17dzz88MNn2lQiBznvvPMAAL/99htOnTqFu+++G6NGjUJiYiKGDBmC\nBx98EIUKFcKQIUPwwgsvZBy3Zs0a9O7dG7t27ULfvn3x1FNPoXv37li5ciV69OiB77//3nq+fv36\nZUx59evXDyNHjkTx4sVD+7Vt2xZBEODDDz8M/fbBBx8gCIKMaeQDBw6ga9eumDFjBtq0aYPhw4fj\n9ttvx+bNm9GjRw8sW7bsjNtJxKZgwYJ45JFHkJaWhlGjRjn3f+aZZzB48GCUKFECQ4YMwcCBA3HZ\nZZdh+PDheOSRR4x9//GPf+Dtt99GcnIyhgwZgiZNmmDAgAH49NNPQ+VOmDAB/fv3x8mTJzFo0CAM\nGzYMDRs2xOuvv47bbrst4131l++HwTnGM888E0QikWDSpElBEATBoUOHglq1agWpqanB8ePHM/b7\n7LPPgkgkEiQkJATr1q0zyujVq1cQiUSC7777LgiCIEhPTw+uuOKKoGfPnsGpU6cy9tu2bVuQkJAQ\n9OnTJwiCINi5c2dGmatWrTLK7NOnTxCJRILvv/8+CIIgSEtLC2rUqBG0a9fOqFcQBMHQoUODSCQS\nzJ49O2NbtNyZM2eeaROJM2Tq1KlBJBIJxo0bZ/396NGjQZMmTYLk5OTg6NGjwezZs4NIJBI88sgj\nxn4nTpwIOnToEFSvXj1IS0sLgiAIhg0bFkQikWD9+vXGvps2bQp69+4dfPLJJ0EQ/K+vDRo0KGOf\nF198MYhEIsFnn32WsY33S09PD2rUqBH07NkzVO8uXboENWvWDPbv3x8EQRA89dRTQWJiYuj5SEtL\nC+rUqRO0b9/eq71E/ET7WPRe9uvXL6hRo0awZcsWY78WLVpk3MtNmzYFkUgkGDp0aKi8fv36BZFI\nJNi4cWMQBEHwxRdfBJFIJOjRo4ex38qVK4OEhIQgEokEU6dOzdg+YsSIoFu3bsGhQ4eM/fv37x9E\nIpFg5cqVGdv+yv3wnPrCjC72KVy4MNq0aQMAKFKkCFq1aoV9+/bho48+Ch1Tu3Zt1KpVy9iWlJQE\nANi7dy8A4MMPP8SJEyfQoUMH5MmTJ2O/ypUr45133sH//d//GcfXrFkTderUMbYlJCQYZS5YsAC/\n/fYbbrrpJhQsWNDY96abbgIALFy40NieL18+Y5GJyF2OHz+OgwcPZvz9+OOPWLlyJe6++2788MMP\nuOOOO1C4cOGM/6K7du1qHJ8/f3507NgRJ0+exKJFizK2AcDq1auNfRMTE/Haa6+hadOmZ1TnSy65\nBA0aNMDq1auRnp6esT0tLQ3r1q1Ds2bNcPHFFwP4fQquSpUqqFSpknGdhQsXRt26dbFlyxYcOHDg\njOoj/Hj44YdRsGBBPP744whOk0Bqzpw5AIDrr7/euF8HDx7EddddB+D36X7gd0kKANq1a2eUUbdu\nXdSuXTtU9sCBA/Hf//4XRYoUwalTp3Do0CEcPHgQ5cuXBwDs3r07ruv5s/bDc2qVbHSxT4cOHVCk\nSJGM7TfeeCNmzpyJSZMm4frrrzeOid7wzBQqVAjA79NpADJ0qHLlyoX2TUlJCW2rUKFCaFvhwoUB\nIMPFZfv27QCAatWqhfatVKkSAOCbb74xtl9yySU4//zzQ/uL3GH06NEYPXp0aHvRokUxaNAg3Hbb\nbQD+d6+rVq0a2pfvdbdu3TBz5kw8/fTTmDlzJpo2bYqGDRuiTp06GYPpmdKuXTssXrwYH330Ebp0\n6QIgPA126NAh7N27F3v37kW9evVOW9aePXsyXmwi5yhVqhTuu+8+PPvss5g+fTpuvPHG0D5ff/01\nAKBnz56nLSc6pR8d4Gzvqlq1aoX+YTt8+DDGjBmDefPmIS0tLePdGOXkyZPxXRD+nP3wnBowo4t9\n6tevj2+//TZje6lSpXDppZfis88+w86dO42Bj7/ubEQHuQIFCnjVw2e/qIAfHUgzE9W/fvnlF2P7\nBRdc4HV+cXb429/+ZvyHnjdvXhQtWhSVK1dGvnz5MrYfPXoUBQoUsPY1vtcVKlTA9OnT8corr2Du\n3LkYO3Ysxo4di+LFi6Nfv37o1q3bGde7ZcuWKFSoEObNm2e8qC666CI0b94cAHDkyBEAv3/ZxtLL\ny5Qpc8b1EX7ceuutmD59Op577jlcc801oQEies/+/e9/49JLL7WWcdlllwH4X3+L9r/MXHjhhYYd\nBAHuuusurFq1Co0bN0a/fv1QokQJ5MuXD++9916WvRD+jP3wnBkwt2/fnjHdMGTIkNPuN2XKlIxV\nXL5ERetDhw5lvYJE9EvRtvIt2pk1QP6xKVeuHFJTU537nX/++Thx4gR+/fXX0KAZvf+Z73WpUqUw\nePBgDB48GJs3b8bChQvx9ttvY+jQoTj//PPRsWPHM6p3kSJF0Lx5cyxYsAAHDhzAL7/8gnXr1uHm\nm2/OqF+0PidOnPC6RpHz5M+fH4899hh69uyJ559/Hk888YTxe/SelStXDsnJyTHLit7n48ePh347\nfPiwYa9fvx6rVq1C/fr1MX78eOTN+z+lbsmSJVm6FuDP2Q/PGQ0z+l/OzTffjJEjR4b+nn32WeTL\nlw9Tp04NTSe4iP73Ep2azcz8+fPx7rvvxl3f6PRcZleBKNGplcqVK8ddrvjjEeteR92dqlSpYj02\nMTER99xzD1555RUAyDZ/vPbt2+PEiRP4+OOPQ9NgwO9fGSVLlsS3335raExRfv7552yph4iPevXq\noVOnTpg8eXJodX60n61ZsyZ03JEjR4zBsWTJkgBgXXXNrhrRiD2pqanGYAkAK1euzMJV/I8/Wz88\nJwbM6GKfggULYsCAAYYzefSvY8eOaNmyJX788Ud8/PHHcZXfrFkzFChQADNmzDAc0X/44Qfcf//9\nmDJlStx1vvrqq1GgQAFMnToVv/76q/FbdGo5KtSLc5voMvsJEyYY26P9tlChQmjWrBkA4M477zSW\n6UeJavKxJIToy8z21cA0a9YMF154IRYtWoT58+ejTJkyqFu3rrFPmzZt8Ntvv+HNN980th84cACd\nOnVC3759necR2c/AgQNRpEgRPPbYY4Z2GF3o+M4774TCgT733HO46qqrMnzBo2svoguFoqxatSo0\n4EZn2Hhhz7Rp0zL0+czn+yv3w3NiSnbu3LnYv38/brzxRlxyySWn3a9nz56YO3cuJk+ejD59+niX\nX7JkSdx7770YOXIkbrvtNtxwww04evQo3nrrLQAI+Vf6cNlll+GBBx7Ac889h169eqF9+/YoUKAA\nli1bhvfffx+tWrXKmMcX5zYtW7ZE8+bNMXnyZBw/fhypqak4cuQIZs+eje3bt2Pw4MEoVqwYgN/1\n92ifaNOmDS6++GL89NNPmDRpEvLnzx9aaZuZaBiysWPHYtu2bWjatGnGAjamYMGCuPbaa7FgwQIc\nPnwYt99+u7ECHADuuecezJ8/H+PGjUN6ejrq1auH9PR0TJgwAenp6ejVq1c2tZCIh+LFi+OBBx7I\nmJKNLlxMTEzErbfeijfeeAPdunVDly5dkD9//oxIQR06dMjYNzU1FTVr1sSiRYvw4IMPomHDhti9\nezcmTZqEtm3bYtasWRnnS0lJQenSpTFr1iyULFkSlSpVwooVK7Bs2TI89thjGDBgAKZPn45ixYqh\nTZs2f+l+eE4MmJkj+8Sifv36iEQiWLx4cWi1rIt7770XpUuXxltvvYUnnngCefPmRZ06dfDiiy8i\nMTExS/Xu27cvSpcujTfeeAP/+te/cPLkSVSoUAEPPfQQevfunaUyxR+PPHnyYNSoURg/fjxmzZqF\nOXPmoGDBgqhRowbGjBljuAr17dsXJUqUwMSJE/Hiiy/i8OHDuOiii3DllVdi+PDh1iX/UVq3bo05\nc+bg008/xfbt25GUlIRSpUqddv927dplRHOxxbwtWrQoJk2ahDFjxmDhwoWYMWMGChcujFq1amH4\n8OGoX7/+GbSKOBO6deuGadOm4YsvvjC2P/zww6hWrRomTpyIp59+GqdOnULFihVD75Q8efJg7Nix\neOqpp/DJJ59g4cKFqFmzJkaPHp3hchL9UixUqBDGjRuH4cOH480338R5552HBg0a4D//+Q9KlCiB\nWbNm4dNPP8XYsWPRpk2bv3Q/zBOczulHCCHEn45nnnkGr732Gl5++eUMqUD4cU5omEIIIfw5duwY\nHnzwwVDQlePHj+ODDz5AgQIFMgK4CH/OiSlZIYQQ/kT9L6dNm4YDBw6gZcuWOHbsGKZMmYI9e/ag\nb9++MdeDCDuakhVCiD8hJ06cwKuvvoqZM2diz549OHXqFCpXrozOnTuje/fuocU3wo0GTCGEEMID\naZhCCCGEBzE1TPZl5DBvRYsWNewWLVoYti3UHC+TjjrGRuEYiRy8/Mcff4xZHp8zmhkkMxxLcdCg\nQYbNgYY53xsHZOdlz7aA7Qw7Hi9dutSwozEWo3Db8/HsRMwRMmzxb6tXr27YmRO62o7hDC05xYAB\nA2LWg6+N77ktgtKqVasMO3OACuD3DDSZ4awh3M/mz59v2Bx4nQMTAAhFUeEsOlwnvj/shM77c6Qh\nm8N3NKtOlGgWlSgc/J/bMhoVJgrHGeXn05Z84JZbbjFsTjJsS2yc0zz//POGzYHwOXoYP4+2bBqu\niGMc0IRjS/M5+H4zfDwA7N+/P6468e8XXXSRYXN/4P5ke/bY5YTLdCWc4Hbg6+S237lzZ6iMtWvX\nGvbGjRtjnuN0EY70hSmEEEJ4oAFTCCGE8CDmlGw0VUwUjnPJn7Vz5841bJufD09LcfqZaCzEKPyJ\nnzmtEgAjL6bN5msAwtMUPAXL02s8/cmJn3mKkMvnqTcgPA3Beet4CpGnJblObDO2GKWXX355zDrZ\nMq2cDbhP8NQ/T6nPmDHDsHfs2BEqM3M6OCA8vcX3nM/Jv8+ePduwOX6xbQqcp7T5Orgfcl+vWLGi\nYXNf5z7E08gAQgnKa9SoEdon1jlcdeR+Zgv+zVOw3HdzYx2iLQ1fZlzThrbpUtfzw1OyLlxpBQ8e\nPBjaxlOs8Sam4DJ5upOv+6effgqVwXmGWeLituXr5HO4psdtQdxt09Xx/B5FX5hCCCGEBxowhRBC\nCA80YAohhBAexNQwWUviuWVOisuJljdt2hQqk+e4WdPkcE1cB16izPoH6zbRZM2Z4eX6rOsw7HLw\n3nvvGXZaWpphs8uBbZlzNGl1lIsvvtiwWd9iTY3Zs2ePYXO72nQh1qNYK3DpojkFn9el9XC/ZNcH\nINyP+NpdNusufE7WTXh/IHxPuQxOkcTXzf2MYXeN8ePHh/ZhfZ21Wf6d3WNs7hOZYT3K5lrGejI/\n87nR79jVgWGNi91OXPoi4NYT+flzaXcMu6IBYQ2S6+nSC122y/3NVgd+h3OfYT3Z1U58TpuOyvtk\ndb2GvjCFEEIIDzRgCiGEEB5owBRCCCE8iKlh8rwuazDsT8X+iLZ5YdaGWBNx6Tzsa8YaybJlywyb\n9UYgrHOyZlKpUiXDZn2RNU+u47p162LWCQDKly9v2HxdkUjEsHnen/Ut1iy5HVnDA4A1a9YYNvus\ncpiz5s2bh8rICbjfcN25nqy72XziWHPiEIy29skMty/7HzLcz23H8DlZw+Tf+Tr37dtn2Lxm4IIL\nLgjVgbVW3of7DT+frFlxxgvW2m165O7du2Oe00cPPNu4/DR9/Pi4X7tsV6g8fgfY9GLGFV6Pf+dz\nuEIC2urAfY6PcfllsoYZb0hB2z6My882ir4whRBCCA80YAohhBAeaMAUQgghPIipYbIOw/O8HGfS\nx5eFNZMSJUoYNmt7Ll9Q1j94Dp19voCwnx7rh+x36dIDOSZryZIlDZvbyVYGx9xkvZB1O96fz8l1\nssXUZW116tSphs1aw5AhQ0Jl5ASse3E/5PvBsSptfpjcPi5dlPVGlz8ax1C1aUVbtmwx7FatWoX2\niVVH9hXk3/l+2rTAw4cPGzanheO2dPnpMS5NEwj7DNu01j8arKO5/BlPty0z3JbxapauOvnUIV5c\nvqQ2X2He5vLL5H7u6nMuP00bvI9vO+kLUwghhPBAA6YQQgjhgQZMIYQQwgMNmEIIIYQHMRf9sBDK\niwzq1q1r2N98841hf/nll6EyXQ7j1apVM2wOzMwLQHiBiM/CI17QwQuPeAGIK6ExC8i8WMa26Kdz\n586G/frrr5++wggHU+BFP7wgi6+BF8oA4UATS5YsMWx2lP+j0qFDB8O2BQh3LRZjm4MI8D22BSbI\nDPdzIJwIgBekcYAMhu8HXxMnW+fAIrZjuA/YFoedCbwICAgvbuF24IVIZwPfBMJRXEEHbPAz6rM4\nJTOuBVi2OvCCGA7g4Qro7lpo5APX2/Ve4euId/FUVurki74whRBCCA80YAohhBAeaMAUQgghPIip\nYbJDP+s8rAV26dLFsGfPnh0qkwN6s2bCTqyuxMms07gCdwNh/YmDQbP+xDpp2bJlDZsTlrJWtX37\n9lAdOJh3/fr1DZu1Jb6OK664wrBdiWJt8/xcBz6Hb0Di7CYhIcGwuf253nw/unbtGipzzpw5hl2s\nWDHD5r7MgQgY3p+1HptmzNs44fo111xj2NwHuF9xkA5+dmy4+gX3I1diZU5Mzu3G5dnqwORGIANb\nPWPhox+6tDUO6M42txPXkc/J+iTg1hxtx8Szv8sGwu8R13sl3oTxWSEruiegL0whhBDCCw2YQggh\nhAcaMIUQQggP4prAPnXqlGGz5sVaU9WqVUNl8Jw6+56xzfP6HDya9UdOcGvTQ1gjY22IE0yzlsva\nAWtqjK0d5s+fb9hcb/ZH5QDGfF3cTuxXxnoYYA9MnxlXkuScgvsV3y+X/sD3CwjfA+6rrmDarHu7\nEkrbglA3adLEsFeuXGnYX331lWGz/67L35f9aBs3bhyqA99z9u/lpNSc3Jf14hUrVhg2t4tNb+Lr\n4LZzrVvICWy+u5mJN/GyD6z3snbn0up81hi4zsG/87vNpSfze8emYbrq4Aro7lpPEG+CAFsdXOeI\noi9MIYQQwgMNmEIIIYQHGjCFEEIID2JqmK+++qphc+xY1oVYB9i8eXOozB07dhh2lSpVDJt91XhO\nnLUnThbMeogt5ifPeXPCaNYXW7dubdgc65LrxIl8GzRoEKoDa7XsT8q+oKwtsZ7sirlr09RY12Q9\nIrc0zOXLlxs2J3/ma+P7abtWvkeshbvifPI5uP35d1s8VI7/y/oQ/8515vtVvXp1w+brtvmCss+w\nK2E0+027kv2yFmSrA7c94/LDywlcCcJd+MRYPdPrcvnE8vPrg6ufu3RS/t2mu3K9XP6kOXH/Xbqp\nNEwhhBAiG9GAKYQQQnigAVMIIYTwIKaGyf6IDGsuPB/OvwNhTYT9wji2rCsPIes6PnEGWUflOK6s\nBW3YsMGwOX8mXwP7VC5atChUB/axY02TfRH5ulkHYq2INTaXnxngFxfybLBlyxbDHj9+vGGzXybb\nNp9T9nFMT0837OTkZMPm9s+O+JV8Xaw7c7/h6+DniX11uc62vIP8fLEPcmJiomFv27bNsFlXLVOm\njGFzfFubjzJrcdx3uR3OBi7NkjWvrOSJdJ3D5QvMWqCP7sbPMJfJ/tqsJ/J7I6sxWDPD15EdZWYm\nK8+qr26qL0whhBDCAw2YQgghhAcaMIUQQggPYopUrIOxVsd6BesjtlyUrFdwmawFcCxLJl5/OSCs\nkXA8TM6VyD6Qe/bsMezvvvvOsNk/jnVbIKypNWrUKOYxNWrUMOydO3caNuth3A42XY/bhv0uc0NL\nAsI6CvvauvwVbXotXwv3szVr1hi2Sxvnvm3r6wzr0OyPxjZr5ezbyVo878/tBoR9Wrkf8e8ufZ7h\ne2F7/ritLr/88phlng3ONB9mVveJhcsn0ifPpMs3kzVMrjOXyc+my7aVybg0x+zQOF36sC/6whRC\nCCE80IAphBBCeKABUwghhPAgpobpiq/IeiTPd7OvIRDWCljzYJv95Vi3YTiXns0/irUgly7D893s\nW8ZaLus+SUlJoTJZO2Bdh/0sWVNjvZh97vh3jk1rI0+ePIadW36YfH9YW2UNc9OmTYZtix/Mvn/c\nl1mnZh83bn/uM+xHa6sD920+B2uv27dvN2zWqfmaXHlZgXAsWa4n+2XydXHbc79jzdIWN5bL5OeL\n++7ZgNvFFUM1u30HbbhyBfP999Ew+X3Ix7i0XC7PpXHatnHbuXRUF1nJh5lVTVNfmEIIIYQHGjCF\nEEIIDzRgCiGEEB7EJVKxZunyn7JpYJxPb+3atYbt0mnYZl3Iph0xLh86ngNnv0vWiipUqBDX8UC4\nHVx5B1lbuuqqqwybtV3Or2nzE2RdkzW13NIw+Z6yXsjty9daunTpUJm8jbU31llYi+PclS4/TVs/\nZH29UqVKhs1aDteRrzslJcWwWV+06dZcT5f2yufgdpk9e7Zh+6xj4Lb0yaGZ0/AzyveKyYofn+t5\n4rZjH3S2fTRMxhUT90yxXSNv43UpTLxtmx1xnn1zieoLUwghhPBAA6YQQgjhgQZMIYQQwgMNmEII\nIYQHMVVoFuxZvGWHchZOeREDAJQrV86webGLy7m7SpUqhs3JnnnBACfABYCyZcsaNieQ5sUVfI69\ne/caNi9K4UVAHMgACC+24AUefB28KIUXAfEiFA68bXNq54VAfA6fpLg5gWuxC/c7xrZoxBXw+/PP\nPzdsXmDBC2h44UqhQoUM25Y8ncvkhVt8j/g6eHGTa4EbL6CzlclBAvjZ4OeVy1y+fLlhs+M7B3O3\n1cGVPP1swM8Pv4dcydVdi4RscBn8/uRFPbwQ0BWcHXAHn3HhOt5nwQ2/w13HxPs7t5vvAp7M+CyY\nAvSFKYQQQnihAVMIIYTwQAOmEEII4UFMDZO1ItYWWLtjTcWmobCm0bZtW8O2Oflnhuf1WWdjTdSW\ngLpq1aqGnZCQYNisb3GZ3A7r1683bHYG5jl8IOxI73Le5nl7DhbN18SaDAcot5XJ+gQHQT9b8HlZ\nH3TpKLZ+x+3BWh23D2txrB9xknAu3xaEmnVQfhZY++Yy+B4vW7bMsH2ClruCj1SuXNmwuZ04cEGt\nWrUMm9vRpjdzHbKSjDu74fvteoZZ87Jpg/Fqcfwu4zUg3B/Kly/vrIMLlzYbL1kJfO4KEM9wHX0C\nqbvuBY8rp0NfmEIIIYQHGjCFEEIIDzRgCiGEEB7EnLBmTZJ9z1if4AS4tsDLrAVEIhHDZq1o8eLF\nhs3+Uawvsg8f20BYe+Vz8nWxxsn+cOzHuWTJEsO2BQNnnYb9uFjfYl2ObT6edaJvvvkmVAfWi1lL\nYO3wbMH+iKw/uPwwfTQw7tu1a9c27CAIYp6T2599c211ZF3M5X/I95D9eVkv4mfBpqOyXyXrqvx8\ncQJovgZuB+4zNv82vr98Ha6E7jmBK8kx34vs0DAZLoPfp7weg+uclSDkrmP4Ol06q60deBu/Z/h3\nVxLr7Ai2zpolP4unQ1+YQgghhAcaMIUQQggPNGAKIYQQHsTUMF2JXHnumee7bf6HrIFwGazdsc8N\n6zisJx46dCjm+YCwrsnHsBYUdNuaAAAOUUlEQVTE89t8XWvWrDHsrVu3GjbrQkD4ulq1amXY3A6s\nUbJPHteR/es45icQ1t1Yz/Kd189p4o1FadNAXInI2f+QfUF37txp2NxH+Jy2JLkubYb7Dd9D7qdb\ntmwxbE4ibuv7fI9Zf2c/y4ULFxo2t70rUblPMmiuJ7ftHwGXb6BPImZXvFJXv3bFdfWNhxqrDJdP\nI7cD66w23Zy3+eiemYnX79LWjq4yfNEXphBCCOGBBkwhhBDCAw2YQgghhAcxNUz2sWI9gn3ZXPFP\ngbAeZZvzzkylSpUMm3Ub1oq4fFssWVfcQM5lOGPGDMNmbYn1iw0bNhi2LT5jxYoVDZvzW9atW9ew\nWdPkc5QoUSJmndiH1nYMayC5EdPTdl7uR9zv2K/PphnzNr5W1og5Luvu3bvjsvft2xeqA18X639c\nR44Vy/eQ9SPuty1atAjVgZ83Pif7j65YscKwWUfltnf5A9vg68gOP7szxaWr8ToI2/PF/oYurc2l\nUZ5pbksb8ca75Xcnv2ds2j3rnvHm1HS1G7ezbe1MVuLP2tAXphBCCOGBBkwhhBDCAw2YQgghhAdx\n+WFyvj7WTHh/1jOAsG8f26yx8DmrVKli2Nu3bzds1lQ4Vq2tXi7t7osvvjDstLQ0w77pppsMm2O0\n2ub1WaPkOrA2wJrJypUrY/7OmgrXGQjrEazb5ZaWxBol2+wryLZNM3bFCuXfOSYqt81XX31l2Bxz\n1Qa3J5+Tn5+lS5fG/J19KDdv3mzYtljAjRo1Mmz2a+ZzsG8o90uOw+yzjoHbnnVU17qGs0G8+qHL\nTxMIP+MuXY3LdJ3D1m4unfNM9UOf6+brdMXhdbUTa5Q+Ppa8jc/BOujp0BemEEII4YEGTCGEEMID\nDZhCCCGEBzE1TFfORpdvG/srAmFNknFpTQzPZ7MfGcfOBICyZcsaNseuZL2P68Dn4DikKSkphj1u\n3LhQHVi75bZljYz3X7t2rWGzhsm5SG3+qKzLsd9tbsHtyxol4+OH6dJiWJtj30Dut9z+PrkJ2ZeW\ntVn+nWPPch1r1Khh2D5aIPezTZs2GTbrnj/99JNh87PD7wi+V7Y68HXH+8z/EeD7y/FRAbuWHqsM\nl8bNayNc2iAQ1ubijWfr8hXlOtruN7+jbX6SmTnT2LE+GmZWY8vqC1MIIYTwQAOmEEII4YEGTCGE\nEMKDmJPsrEewhsIaJWtPrC0B7jihnHeQY1fyPD7D+ghrMLYyOWYn+11ymax5/vzzz4bNvm7Jycmh\nOnBbsW7KPq58L3h/1mrZx9I2R8/3x6VXnC34vFxP1kBcfl1A2C/VFUPVdXxqaqphc5+x6a6sdfM+\nXCe+hy6dtGvXrob9+uuvh+rAbcP9iPsy69qsgfLzzBooP2tA+HnimLpZyet4prh8P13PRnbkonT9\nznXkuNq2OvDaB/abta1tyAxr+fw+tfmYM/HqhXwdLo0yK3qky9fzdOgLUwghhPBAA6YQQgjhgQZM\nIYQQwgMNmEIIIYQHMZVOXmzhCvDNjvA2IZ0XzPDCIQ7ezIsGeGEE15EXb3DwaCC82MKV/JfLPH78\nuGEvXLjQsKtXr27YNmF9+fLlhs2LKXghBMPtxsez8O2TDJrvrytgQE7B99S1+IwXx/gEUnYtiuL2\n4n7Iic15cYxtsZkrUDUv9OLrPHLkiGHzQqOmTZsadrly5Zzn4HpyEHlOMu4Krs7Pim3RDy9m4XPa\nAk/kND5BxDPjWngChBencdvxe8G1WMVVRw50YYP7vWvxEv/O5/AJ2MHEG/DdFXzB55yuIA8KXCCE\nEEJkIxowhRBCCA80YAohhBAexNQwWW9knYy1JVuwdYYdo3numJ27+ZyrV682bNb6WDdgnRUIz1/X\nq1fPsDk5M+s8XCYnsZ40aZJh24ItcICG8uXLGzbPuXPQa5cDuU8QAt7m0k1zC1fgAr5W1t2AsPbC\ngR448LnLAZ8dun2CPOzdu9ewXZokB/Tm523RokWG3aJFC8O26TLvvvuuYXM/Y8d21mZdydZZf7S1\nC18H30/WbnMDX0f2WPtzW/Ezy0EFXGQlGTRrjBxogOvo0ijjDfBg23amSatd+rEtuHu8Ad1Ph74w\nhRBCCA80YAohhBAeaMAUQgghPIg5Uf/1118bNs+h16pVy7B5Hph1NyCsX7BuxnPkPMfOWhPrAOzT\nxb6hQDjBs0u/Yq3I5fu5fv16w7b5lbGewQmf2WZ92KX7cIBrtm314n1yyw/TlbCb25/rbUuE7dLF\n2EeO7w/7L7Km6dJ2gLBuzcmbN2zYYNguf0TWm9577z3DZr9aILyGgPsV++lxO3Db871i20cb4mfU\nJ6B3dhOvxsXvJR/djMmKD2Os/W0aJ5+DfTldQeNdGiYf7xOEPqs+kNmJNEwhhBAiB9GAKYQQQnig\nAVMIIYTwIOYkO/tksd7ImgxrgTVr1gyVyT6MrKmsWbMm5u/ffvutYXPcV57H5+MBu8aVGVcyYdZ5\nWENjfYt1ICCsHbHt0khc8Va5DuzzB4TvBds23fNs4NJOXdqN7f5xX2bYr5L3Z39Dbn/Wq2x14BjE\nrDFyDOKNGzcatksnXbt2rbMOfI+5b3K/Yz2f25r7FbcjH2+rF99P9j/NDVyapcvv2XYMkxW9N97j\nXc+Ky5/YpbP6aPf8bLh00HiP5/exjXg16dOhL0whhBDCAw2YQgghhAcaMIUQQggPYmqY1157rWGv\nWLHCsFlXY32RY2MCYV2T49WyrsO+ahy31aVf7dq1K1QH1vf4GM5VyfuzfsFaH+tCtnyYHMuU/Uld\nGolLN2W92eYTxm3Hvoi5kZcQiF/LcbUFENatWSfla3XlbeU+U7FiRcNmH2bAHVuU/YNZm+F+yc+f\n65ps9eR2ceVN5XOwBsbntPm/utYx2PxHzzau/Kh8DfzsAOFn2pXXkYlX47T97oo/6/KjdeXgzIov\nqIt4Y8nyc2B73zLc9j45dAF9YQohhBBeaMAUQgghPNCAKYQQQngQU8Nk/a9MmTKGzT5WbNt0HNZM\nWK9gDYTjZTI8n82+bjYdZ/bs2YbdtWtXw+b8lzxHzrpNsWLFDJvbyZafkeNnsp7FWgBfB+sCrNtx\nTF1brEvWZfi6cysvoUurYR3N5TsGhK+FtTw+hrXyGjVqGHZCQoJhszbPsWdt8LNQoUIFw2ad9PPP\nPzdsvn8+sX9Zq2HtzRXnk9uJ/TD5+bP5YXKsWO7buRFblOE6sGbJfro23Yw1TF5HwL+79MN4tUAb\nWcmpGU95PsfzdXEZLn2RtX22fTTMePOdRtEXphBCCOGBBkwhhBDCAw2YQgghhAcxJ3LT0tIMOykp\nybA5nmblypUN2xaLlLU1V2xLtlmLYw2GdVfbnD3H3GQ9kXNRus7Jx5cvX96wbbFrXbFJGdYvWPtl\nm+NxcrsD4Xl81vlsuufZgDUtbm/uV6wp2/wwWUvj9mbdhPvNl19+adjc1320G76HrM/zdfH94ONd\n/sC2GJu8jduFdVOXls7n5H5m8+NzxYrld8LZgO8fPxsuHa1cuXKhMl1+lq4+x7Yrpqqtrc9UD/bx\ns3TB9XK9V1xxXln7ZdsGt50rn/Dp0BemEEII4YEGTCGEEMIDDZhCCCGEBzE1TNYreC6a54VZF6pb\nt26oTNaX2H+N/brYp4ZjpNapU8ewOfbskiVLQnVgzWvp0qWG7fLrYY2S/Sw5TqyPFsh6BGsFPMfu\n8j3k+Jw2jdTlQ/VH8IcDwvVgXZvjvm7evDlUBueaZJ9FzuXKvy9btsyw2c+SdXHuY0BY7+O+2rFj\nR8Nev369Ybv88FjrZX9g2z5ss4bJzyvbXCd+Z9i0c1cM46z6yGUnLp9JVyxowP08xevD6NJEbRqm\n673hIjv25+viOvH71nX/s6Kjut7Bvus19IUphBBCeKABUwghhPBAA6YQQgjhgQZMIYQQwoOY6iov\nZuGFECxC80ITW+ACPiYSiRi2K3DBO++8E/P3e++917BtAeD5GE58zdfNgQd4ERAvAOD9OSA14A6m\nzot0XIl9WUhn4dwWhJ4XWHEZuZVAmheBcJCA5s2bG/b8+fMN25a4nEV9Dp6dnJxs2HxPuf23bdtm\n2Dt27DBsW9vx88B9kwOFuO45w/vbglC7FvlwHV3O9K5g7LZ3AOMKvp0buBInu4LUn25brN9di4Cy\nEujc5ZDvur/x4lMnbjtXW/K7LCuLo7JrQaO+MIUQQggPNGAKIYQQHmjAFEIIITyIqWGyMy7raqyH\nVK1aNe4KsAbC+h9rSRxkfMKECYbdoEEDw2ZtCgg7mZctW9awWd/iOrg0FtZ6bUEDWNtxJdHlc7ID\nuSsYtG3OnoNAcOB6buuzBQe84OAWzZo1M+xq1aoZ9ogRI0Jlcj/ja2Xdk+vg0oL4fmZFu+O+3Lhx\nY8NmLYefFdbe+fm0beM6uK7D1S+5nW0JpF3BDnyCaWc3fB3x6mY+DvuuY7JbTwRyJgl1LHy0QZee\nyO8yV5AIVxJ0IHw/XYHvT4e+MIUQQggPNGAKIYQQHmjAFEIIITyIqWFyUGueD+dEsKw1sY+XDdb3\n2Gdxz549hp2enm7YnNh31KhRhl2hQoXQOTlwtisYsGtenjVL1v5s8+PcNjynzmXy/i59kef5WZcF\nwhoY14Hv/9mCA+rb/Akzw9p5o0aNQvts2bLFsLk9WcPk9nIlUmZsSZBdfZ21dfarZJv9iVnD5N+B\nsCbJ7eDS0rlM7qfcbjYtl7Vz1uNtgetzGtbBWDfjBNI+CYfj1Shdtus9ZPMddWlzrt/j9SW14dKD\nXcmd+V3G7wN+X9s0cL5/jMvvNoq+MIUQQggPNGAKIYQQHmjAFEIIITzIEwRBkNuVEEIIIf7o6AtT\nCCGE8EADphBCCOGBBkwhhBDCAw2YQgghhAcaMIUQQggPNGAKIYQQHvw/2M2jVVZJz+gAAAAASUVO\nRK5CYII=\n",
            "text/plain": [
              "<Figure size 576x396 with 3 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "metadata": {
        "id": "_X91J_WrYoTQ",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Print the model summary."
      ]
    },
    {
      "metadata": {
        "id": "SUj6CXsIAgJ8",
        "colab_type": "code",
        "outputId": "42a03dd6-d282-47b3-f6c4-a3ceb010a90f",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 386
        }
      },
      "cell_type": "code",
      "source": [
        "descriptor_model_trip.summary()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "__________________________________________________________________________________________________\n",
            "Layer (type)                    Output Shape         Param #     Connected to                     \n",
            "==================================================================================================\n",
            "a (InputLayer)                  (None, 32, 32, 1)    0                                            \n",
            "__________________________________________________________________________________________________\n",
            "p (InputLayer)                  (None, 32, 32, 1)    0                                            \n",
            "__________________________________________________________________________________________________\n",
            "n (InputLayer)                  (None, 32, 32, 1)    0                                            \n",
            "__________________________________________________________________________________________________\n",
            "sequential_4 (Sequential)       (None, 128)          1336928     a[0][0]                          \n",
            "                                                                 p[0][0]                          \n",
            "                                                                 n[0][0]                          \n",
            "__________________________________________________________________________________________________\n",
            "lambda_4 (Lambda)               (None, 1)            0           sequential_4[1][0]               \n",
            "                                                                 sequential_4[2][0]               \n",
            "                                                                 sequential_4[3][0]               \n",
            "==================================================================================================\n",
            "Total params: 1,336,928\n",
            "Trainable params: 1,336,032\n",
            "Non-trainable params: 896\n",
            "__________________________________________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "VoRNHVIzloS8",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Create the training and validation generators (with or without clean patches)."
      ]
    },
    {
      "metadata": {
        "id": "YIR1cH4fDwKj",
        "colab_type": "code",
        "outputId": "6938da32-ac97-4695-aeb1-aee54219957b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 193
        }
      },
      "cell_type": "code",
      "source": [
        "### Descriptor loading and training\n",
        "# Loading images\n",
        "hPatches = HPatches(train_fnames=train_fnames, test_fnames=test_fnames,\n",
        "                    denoise_model=denoiser_model, use_clean=use_clean_bool)\n",
        "\n",
        "# Creating training generator\n",
        "training_generator = DataGeneratorDesc(*hPatches.read_image_file(hpatches_dir, train=1), num_triplets=100000)\n",
        "# Creating validation generator\n",
        "val_generator = DataGeneratorDesc(*hPatches.read_image_file(hpatches_dir, train=0), num_triplets=10000)\n"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Using denoised patches\n",
            "100%|██████████| 116/116 [00:36<00:00,  3.16it/s]\n",
            "Denoising patches...\n",
            "100%|██████████| 15589/15589 [04:15<00:00, 60.92it/s]\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "100%|██████████| 100000/100000 [00:01<00:00, 71001.65it/s]\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Using denoised patches\n",
            "100%|██████████| 116/116 [00:20<00:00,  5.67it/s]\n",
            "Denoising patches...\n",
            "100%|██████████| 9525/9525 [02:36<00:00, 60.99it/s]\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "100%|██████████| 10000/10000 [00:00<00:00, 25435.58it/s]\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "metadata": {
        "id": "y8VSlIH_nBVz",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# 3. One epoch training"
      ]
    },
    {
      "metadata": {
        "id": "QPyc8as42WTQ",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "descriptor_history = descriptor_model_trip.fit_generator(\n",
        "    generator=training_generator,\n",
        "    epochs=1,\n",
        "    verbose=1,\n",
        "    validation_data=val_generator)\n",
        "\n",
        "\n",
        "# Save model\n",
        "descriptor_model.save('nloss_descriptor_1e.h5') \n",
        "# Uploads files to external hosting\n",
        "!curl -F \"file=@nloss_descriptor_1e.h5\" https://file.io"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "ag32cVgUpSLJ",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# 4. Full training"
      ]
    },
    {
      "metadata": {
        "id": "T_zwZXN5plnZ",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Train until 20 epochs are reached."
      ]
    },
    {
      "metadata": {
        "id": "A7wZEk2sBzRK",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "descriptor_history = descriptor_model_trip.fit_generator(generator = training_generator,\n",
        "                                                         epochs = 20,\n",
        "                                                         verbose = 1,\n",
        "                                                         validation_data = val_generator)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "EPsqK22Vp424",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Save fully trained model."
      ]
    },
    {
      "metadata": {
        "id": "f1bn0IGWp95y",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Save model\n",
        "descriptor_model.save('nloss_descriptor_20e.h5') \n",
        "# Uploads files to external hosting\n",
        "!curl -F \"file=@nloss_descriptor_20e.h5\" https://file.io"
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}