generated-sci-papers-baseline-bert-tf-keras.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "a076fb35",
   "metadata": {
    "papermill": {
     "duration": 0.011015,
     "end_time": "2022-07-26T11:46:41.423102",
     "exception": false,
     "start_time": "2022-07-26T11:46:41.412087",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Detecting generated scientific papers\n",
    "\n",
    "Competition [website here...](https://www.kaggle.com/competitions/detecting-generated-scientific-papers/overview)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eb548cdd",
   "metadata": {
    "papermill": {
     "duration": 0.009216,
     "end_time": "2022-07-26T11:46:41.441947",
     "exception": false,
     "start_time": "2022-07-26T11:46:41.432731",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "### Description\n",
    "This competition is a part of the shared task hosted within the third workshop on Scholarly Document Processing ([SDP 2022](https://sdproc.org/2022/)), being held in association with the 29th International Conference on Computational Linguistics ([COLING 2022](https://coling2022.org/)).\n",
    "\n",
    "There are increasing reports that research papers can be written by computers, which presents a series of concerns (e.g., see [1]). In this challenge, we explore the state of the art in detecting automatically generated papers. We frame the detection problem as a binary classification task: given an excerpt of text, label it as either human-written or machine-generated. We provide a corpus of over 5000 excerpts from automatically written papers, based on the work by Cabanac et al. [2], as well as documents collected by Elsevier publishing and editorial teams. As a control set, we provide a 5x larger corpus of openly accessible human-written as well as generated papers from the same scientific domains of documents. We also encourage contributions that aim to extend this dataset with other computer-generated scientific papers, or papers that propose valid metrics to assess automatically generated papers against those written by humans.\n",
    "\n",
    "### Acknowledgements\n",
    "We thank Cyril Labbé, Basile Dubois-Binnaire, Guillaume Cabanac, and Alexander Magazinov for their input in the ideation phase of the task preparation.\n",
    "\n",
    "### Links\n",
    "[1] Holly Else. (2021). [\"'Tortured phrases' give away fabricated research papers.\"](https://www.nature.com/articles/d41586-021-02134-0) Nature.\n",
    "\n",
    "[2] Guillaume Cabanac, Cyril Labbé, and Alexander Magazinov. (2021). [\"Tortured phrases: A dubious writing style emerging in science. Evidence of critical issues affecting established journals.\"](https://arxiv.org/abs/2107.06751)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d2b94ff2",
   "metadata": {
    "papermill": {
     "duration": 0.009479,
     "end_time": "2022-07-26T11:46:41.461046",
     "exception": false,
     "start_time": "2022-07-26T11:46:41.451567",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Importing Libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "32e1fed4",
   "metadata": {
    "_cell_guid": "b1076dfc-b9ad-4769-8c92-a6c4dae69d19",
    "_uuid": "8f2839f25d086af736a60e9eeb907d3b93b6e0e5",
    "execution": {
     "iopub.execute_input": "2022-07-26T11:46:41.482438Z",
     "iopub.status.busy": "2022-07-26T11:46:41.481947Z",
     "iopub.status.idle": "2022-07-26T11:46:46.661573Z",
     "shell.execute_reply": "2022-07-26T11:46:46.658947Z"
    },
    "papermill": {
     "duration": 5.19295,
     "end_time": "2022-07-26T11:46:46.663895",
     "exception": false,
     "start_time": "2022-07-26T11:46:41.470945",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2.6.4\n"
     ]
    }
   ],
   "source": [
    "import numpy as np # linear algebra\n",
    "import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n",
    "\n",
    "import os, re\n",
    "\n",
    "import tensorflow as tf\n",
    "from tensorflow.keras.layers import Dense, Input, Dropout\n",
    "from tensorflow.keras.optimizers import Adam\n",
    "from tensorflow.keras.models import Model\n",
    "#from transformers import TFBertModel\n",
    "#import transformers\n",
    "\n",
    "print(tf.version.VERSION)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "6b8c8999",
   "metadata": {
    "_kg_hide-input": false,
    "_kg_hide-output": true,
    "execution": {
     "iopub.execute_input": "2022-07-26T11:46:46.686146Z",
     "iopub.status.busy": "2022-07-26T11:46:46.685358Z",
     "iopub.status.idle": "2022-07-26T11:46:56.988190Z",
     "shell.execute_reply": "2022-07-26T11:46:56.987049Z"
    },
    "papermill": {
     "duration": 10.316308,
     "end_time": "2022-07-26T11:46:56.990577",
     "exception": false,
     "start_time": "2022-07-26T11:46:46.674269",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: transformers in /opt/conda/lib/python3.7/site-packages (4.18.0)\r\n",
      "Requirement already satisfied: importlib-metadata in /opt/conda/lib/python3.7/site-packages (from transformers) (4.11.4)\r\n",
      "Requirement already satisfied: sacremoses in /opt/conda/lib/python3.7/site-packages (from transformers) (0.0.53)\r\n",
      "Requirement already satisfied: tqdm>=4.27 in /opt/conda/lib/python3.7/site-packages (from transformers) (4.64.0)\r\n",
      "Requirement already satisfied: requests in /opt/conda/lib/python3.7/site-packages (from transformers) (2.27.1)\r\n",
      "Requirement already satisfied: numpy>=1.17 in /opt/conda/lib/python3.7/site-packages (from transformers) (1.21.6)\r\n",
      "Requirement already satisfied: huggingface-hub<1.0,>=0.1.0 in /opt/conda/lib/python3.7/site-packages (from transformers) (0.7.0)\r\n",
      "Requirement already satisfied: tokenizers!=0.11.3,<0.13,>=0.11.1 in /opt/conda/lib/python3.7/site-packages (from transformers) (0.12.1)\r\n",
      "Requirement already satisfied: pyyaml>=5.1 in /opt/conda/lib/python3.7/site-packages (from transformers) (6.0)\r\n",
      "Requirement already satisfied: packaging>=20.0 in /opt/conda/lib/python3.7/site-packages (from transformers) (21.3)\r\n",
      "Requirement already satisfied: filelock in /opt/conda/lib/python3.7/site-packages (from transformers) (3.6.0)\r\n",
      "Requirement already satisfied: regex!=2019.12.17 in /opt/conda/lib/python3.7/site-packages (from transformers) (2021.11.10)\r\n",
      "Requirement already satisfied: typing-extensions>=3.7.4.3 in /opt/conda/lib/python3.7/site-packages (from huggingface-hub<1.0,>=0.1.0->transformers) (4.1.1)\r\n",
      "Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /opt/conda/lib/python3.7/site-packages (from packaging>=20.0->transformers) (3.0.9)\r\n",
      "Requirement already satisfied: zipp>=0.5 in /opt/conda/lib/python3.7/site-packages (from importlib-metadata->transformers) (3.8.0)\r\n",
      "Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.7/site-packages (from requests->transformers) (3.3)\r\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.7/site-packages (from requests->transformers) (2022.6.15)\r\n",
      "Requirement already satisfied: charset-normalizer~=2.0.0 in /opt/conda/lib/python3.7/site-packages (from requests->transformers) (2.0.12)\r\n",
      "Requirement already satisfied: urllib3<1.27,>=1.21.1 in /opt/conda/lib/python3.7/site-packages (from requests->transformers) (1.26.9)\r\n",
      "Requirement already satisfied: joblib in /opt/conda/lib/python3.7/site-packages (from sacremoses->transformers) (1.1.0)\r\n",
      "Requirement already satisfied: six in /opt/conda/lib/python3.7/site-packages (from sacremoses->transformers) (1.16.0)\r\n",
      "Requirement already satisfied: click in /opt/conda/lib/python3.7/site-packages (from sacremoses->transformers) (8.0.4)\r\n",
      "\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\u001b[33m\r\n",
      "\u001b[0m"
     ]
    }
   ],
   "source": [
    "!pip install transformers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "bbee814c",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-26T11:46:57.012946Z",
     "iopub.status.busy": "2022-07-26T11:46:57.011980Z",
     "iopub.status.idle": "2022-07-26T11:46:59.050955Z",
     "shell.execute_reply": "2022-07-26T11:46:59.049983Z"
    },
    "papermill": {
     "duration": 2.05282,
     "end_time": "2022-07-26T11:46:59.053733",
     "exception": false,
     "start_time": "2022-07-26T11:46:57.000913",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from transformers import BertTokenizer, TFBertForSequenceClassification, BertConfig, TFBertModel"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8122c151",
   "metadata": {
    "papermill": {
     "duration": 0.009801,
     "end_time": "2022-07-26T11:46:59.073749",
     "exception": false,
     "start_time": "2022-07-26T11:46:59.063948",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Loading data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "1a693f7c",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-26T11:46:59.095143Z",
     "iopub.status.busy": "2022-07-26T11:46:59.094846Z",
     "iopub.status.idle": "2022-07-26T11:46:59.627296Z",
     "shell.execute_reply": "2022-07-26T11:46:59.626380Z"
    },
    "papermill": {
     "duration": 0.545737,
     "end_time": "2022-07-26T11:46:59.629409",
     "exception": false,
     "start_time": "2022-07-26T11:46:59.083672",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((5350, 3), (21403, 2))"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train = pd.read_csv(\"../input/detecting-generated-scientific-papers/fake_papers_train_part_public.csv\")\n",
    "test = pd.read_csv(\"../input/detecting-generated-scientific-papers/fake_papers_test_public.csv\")\n",
    "\n",
    "train.shape, test.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "aeed4c85",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-26T11:46:59.651647Z",
     "iopub.status.busy": "2022-07-26T11:46:59.650874Z",
     "iopub.status.idle": "2022-07-26T11:46:59.664789Z",
     "shell.execute_reply": "2022-07-26T11:46:59.663761Z"
    },
    "papermill": {
     "duration": 0.027201,
     "end_time": "2022-07-26T11:46:59.667008",
     "exception": false,
     "start_time": "2022-07-26T11:46:59.639807",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>text</th>\n",
       "      <th>fake</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Modern two-dimensional imaging is of such qual...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>Background: The optimal sequence of systemic p...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>5</td>\n",
       "      <td>This chapter opens with a discussion of the ef...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10</td>\n",
       "      <td>The time scale of the ultra-short-term can str...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>23</td>\n",
       "      <td>Electronic nose or machine olfaction are syste...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id                                               text  fake\n",
       "0   1  Modern two-dimensional imaging is of such qual...     0\n",
       "1   2  Background: The optimal sequence of systemic p...     1\n",
       "2   5  This chapter opens with a discussion of the ef...     1\n",
       "3  10  The time scale of the ultra-short-term can str...     1\n",
       "4  23  Electronic nose or machine olfaction are syste...     1"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3d9c8012",
   "metadata": {
    "papermill": {
     "duration": 0.01062,
     "end_time": "2022-07-26T11:46:59.687922",
     "exception": false,
     "start_time": "2022-07-26T11:46:59.677302",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "- Checking Missing Values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "4c011c78",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-26T11:46:59.709107Z",
     "iopub.status.busy": "2022-07-26T11:46:59.708842Z",
     "iopub.status.idle": "2022-07-26T11:46:59.721238Z",
     "shell.execute_reply": "2022-07-26T11:46:59.720247Z"
    },
    "papermill": {
     "duration": 0.025389,
     "end_time": "2022-07-26T11:46:59.723285",
     "exception": false,
     "start_time": "2022-07-26T11:46:59.697896",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(False, False)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.isna().sum().any(), test.isna().sum().any()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2dd0e393",
   "metadata": {
    "papermill": {
     "duration": 0.009911,
     "end_time": "2022-07-26T11:46:59.744046",
     "exception": false,
     "start_time": "2022-07-26T11:46:59.734135",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "- Checking target distribution"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "a6429cc7",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-26T11:46:59.765841Z",
     "iopub.status.busy": "2022-07-26T11:46:59.765548Z",
     "iopub.status.idle": "2022-07-26T11:46:59.777615Z",
     "shell.execute_reply": "2022-07-26T11:46:59.776601Z"
    },
    "papermill": {
     "duration": 0.02556,
     "end_time": "2022-07-26T11:46:59.779706",
     "exception": false,
     "start_time": "2022-07-26T11:46:59.754146",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1    3664\n",
       "0    1686\n",
       "Name: fake, dtype: int64"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train[\"fake\"].value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "75946954",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-26T11:46:59.801419Z",
     "iopub.status.busy": "2022-07-26T11:46:59.801163Z",
     "iopub.status.idle": "2022-07-26T11:46:59.806244Z",
     "shell.execute_reply": "2022-07-26T11:46:59.805417Z"
    },
    "papermill": {
     "duration": 0.018336,
     "end_time": "2022-07-26T11:46:59.808214",
     "exception": false,
     "start_time": "2022-07-26T11:46:59.789878",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "#\"[CLS]\" + train.text + \"[SEP]\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2147383f",
   "metadata": {
    "papermill": {
     "duration": 0.012634,
     "end_time": "2022-07-26T11:46:59.831016",
     "exception": false,
     "start_time": "2022-07-26T11:46:59.818382",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Pre-processing Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "6b35f7f2",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-26T11:46:59.852498Z",
     "iopub.status.busy": "2022-07-26T11:46:59.852221Z",
     "iopub.status.idle": "2022-07-26T11:46:59.861249Z",
     "shell.execute_reply": "2022-07-26T11:46:59.860407Z"
    },
    "papermill": {
     "duration": 0.022033,
     "end_time": "2022-07-26T11:46:59.863284",
     "exception": false,
     "start_time": "2022-07-26T11:46:59.841251",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>text</th>\n",
       "      <th>fake</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Modern two-dimensional imaging is of such qual...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>Background: The optimal sequence of systemic p...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>5</td>\n",
       "      <td>This chapter opens with a discussion of the ef...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10</td>\n",
       "      <td>The time scale of the ultra-short-term can str...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>23</td>\n",
       "      <td>Electronic nose or machine olfaction are syste...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id                                               text  fake\n",
       "0   1  Modern two-dimensional imaging is of such qual...     0\n",
       "1   2  Background: The optimal sequence of systemic p...     1\n",
       "2   5  This chapter opens with a discussion of the ef...     1\n",
       "3  10  The time scale of the ultra-short-term can str...     1\n",
       "4  23  Electronic nose or machine olfaction are syste...     1"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "replic = train.head().copy()\n",
    "replic"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "7a7b5fc7",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-26T11:46:59.885808Z",
     "iopub.status.busy": "2022-07-26T11:46:59.885088Z",
     "iopub.status.idle": "2022-07-26T11:47:07.122080Z",
     "shell.execute_reply": "2022-07-26T11:47:07.121159Z"
    },
    "papermill": {
     "duration": 7.250519,
     "end_time": "2022-07-26T11:47:07.124414",
     "exception": false,
     "start_time": "2022-07-26T11:46:59.873895",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>text</th>\n",
       "      <th>fake</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>modern dimensional imaging such quality that e...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>background optimal sequence systemic palliativ...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>5</td>\n",
       "      <td>this chapter open with discussion effect long ...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10</td>\n",
       "      <td>time scale ultra short term strengthen diverse...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>23</td>\n",
       "      <td>electronic nose machine olfaction system used ...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5345</th>\n",
       "      <td>26727</td>\n",
       "      <td>this paper shmoop statistical method derive er...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5346</th>\n",
       "      <td>26728</td>\n",
       "      <td>this paper author examine effect surgical ster...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5347</th>\n",
       "      <td>26734</td>\n",
       "      <td>possibility using mezzanine card pmcs module o...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5348</th>\n",
       "      <td>26743</td>\n",
       "      <td>soil biodiversity keystone natural world this ...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5349</th>\n",
       "      <td>26747</td>\n",
       "      <td>this chapter barlow colleague present result c...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5350 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         id                                               text  fake\n",
       "0         1  modern dimensional imaging such quality that e...     0\n",
       "1         2  background optimal sequence systemic palliativ...     1\n",
       "2         5  this chapter open with discussion effect long ...     1\n",
       "3        10  time scale ultra short term strengthen diverse...     1\n",
       "4        23  electronic nose machine olfaction system used ...     1\n",
       "...     ...                                                ...   ...\n",
       "5345  26727  this paper shmoop statistical method derive er...     1\n",
       "5346  26728  this paper author examine effect surgical ster...     1\n",
       "5347  26734  possibility using mezzanine card pmcs module o...     0\n",
       "5348  26743  soil biodiversity keystone natural world this ...     1\n",
       "5349  26747  this chapter barlow colleague present result c...     1\n",
       "\n",
       "[5350 rows x 3 columns]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from nltk.stem import WordNetLemmatizer\n",
    "\n",
    "stemmer = WordNetLemmatizer()\n",
    "\n",
    "def preprocess_text(document):\n",
    "        # Remove all the special characters\n",
    "        document = re.sub(r'\\W', ' ', str(document))\n",
    "\n",
    "        # remove all single characters\n",
    "        document = re.sub(r'\\s+[a-zA-Z]\\s+', ' ', document)\n",
    "\n",
    "        # Remove single characters from the start\n",
    "        document = re.sub(r'\\^[a-zA-Z]\\s+', ' ', document)\n",
    "\n",
    "        # Substituting multiple spaces with single space\n",
    "        document = re.sub(r'\\s+', ' ', document, flags=re.I)\n",
    "\n",
    "        # Removing prefixed 'b'\n",
    "        document = re.sub(r'^b\\s+', '', document)\n",
    "\n",
    "        # Converting to Lowercase\n",
    "        document = document.lower()\n",
    "        #return document\n",
    "\n",
    "        # Lemmatization\n",
    "        tokens = document.split()\n",
    "        tokens = [stemmer.lemmatize(word) for word in tokens]\n",
    "        tokens = [word for word in tokens if len(word) > 3]\n",
    "\n",
    "        preprocessed_text = ' '.join(tokens)\n",
    "        return preprocessed_text\n",
    "\n",
    "train.text = train.text.apply(preprocess_text)\n",
    "train"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0939a26e",
   "metadata": {
    "papermill": {
     "duration": 0.010749,
     "end_time": "2022-07-26T11:47:07.146051",
     "exception": false,
     "start_time": "2022-07-26T11:47:07.135302",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "### Configuration"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "a4293d63",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-26T11:47:07.168767Z",
     "iopub.status.busy": "2022-07-26T11:47:07.168479Z",
     "iopub.status.idle": "2022-07-26T11:47:07.172917Z",
     "shell.execute_reply": "2022-07-26T11:47:07.171979Z"
    },
    "papermill": {
     "duration": 0.018418,
     "end_time": "2022-07-26T11:47:07.175163",
     "exception": false,
     "start_time": "2022-07-26T11:47:07.156745",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "AUTO = tf.data.experimental.AUTOTUNE\n",
    "\n",
    "# Configuration\n",
    "BATCH_SIZE = 16\n",
    "MAX_LEN = 256"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1eac9aa3",
   "metadata": {
    "papermill": {
     "duration": 0.010537,
     "end_time": "2022-07-26T11:47:07.197228",
     "exception": false,
     "start_time": "2022-07-26T11:47:07.186691",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "### Bert Encoder"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "367cfea6",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-26T11:47:07.219855Z",
     "iopub.status.busy": "2022-07-26T11:47:07.219600Z",
     "iopub.status.idle": "2022-07-26T11:47:07.226349Z",
     "shell.execute_reply": "2022-07-26T11:47:07.225465Z"
    },
    "papermill": {
     "duration": 0.020513,
     "end_time": "2022-07-26T11:47:07.228335",
     "exception": false,
     "start_time": "2022-07-26T11:47:07.207822",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "def bert_encode(texts, tokenizer, max_len=MAX_LEN):\n",
    "    input_ids = []\n",
    "    token_type_ids = []\n",
    "    attention_mask = []\n",
    "    \n",
    "    for text in texts:\n",
    "        \"\"\"\n",
    "        token = tokenizer(text, max_length=max_len, truncation=True, padding='max_length',\n",
    "                         add_special_tokens=True)\n",
    "        \"\"\"\n",
    "        token = tokenizer.encode_plus(\n",
    "            text,\n",
    "            None,\n",
    "            add_special_tokens=True,\n",
    "            max_length=max_len,\n",
    "            padding=\"max_length\",\n",
    "            return_token_type_ids=True,\n",
    "            truncation=True\n",
    "        )\n",
    "        \n",
    "        input_ids.append(token['input_ids'])\n",
    "        token_type_ids.append(token['token_type_ids'])\n",
    "        attention_mask.append(token['attention_mask'])\n",
    "    \n",
    "    return np.array(input_ids), np.array(token_type_ids), np.array(attention_mask)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "76ec9103",
   "metadata": {
    "papermill": {
     "duration": 0.010417,
     "end_time": "2022-07-26T11:47:07.249567",
     "exception": false,
     "start_time": "2022-07-26T11:47:07.239150",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "### Loading bert tokenizer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "8bd86cd0",
   "metadata": {
    "_kg_hide-output": true,
    "execution": {
     "iopub.execute_input": "2022-07-26T11:47:07.271967Z",
     "iopub.status.busy": "2022-07-26T11:47:07.271722Z",
     "iopub.status.idle": "2022-07-26T11:47:07.343162Z",
     "shell.execute_reply": "2022-07-26T11:47:07.342262Z"
    },
    "papermill": {
     "duration": 0.085012,
     "end_time": "2022-07-26T11:47:07.345367",
     "exception": false,
     "start_time": "2022-07-26T11:47:07.260355",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "bert = \"../input/huggingface-bert-variants/bert-base-cased/bert-base-cased\" #\"bert-base-uncased\" \n",
    "tokenizer = BertTokenizer.from_pretrained(bert)#, do_lower_case=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "84c13504",
   "metadata": {
    "_kg_hide-input": true,
    "_kg_hide-output": true,
    "execution": {
     "iopub.execute_input": "2022-07-26T11:47:07.369338Z",
     "iopub.status.busy": "2022-07-26T11:47:07.368470Z",
     "iopub.status.idle": "2022-07-26T11:47:07.375139Z",
     "shell.execute_reply": "2022-07-26T11:47:07.374143Z"
    },
    "papermill": {
     "duration": 0.021025,
     "end_time": "2022-07-26T11:47:07.377561",
     "exception": false,
     "start_time": "2022-07-26T11:47:07.356536",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"\\nbert_large = '../input/huggingface-bert-variants/bert-large-cased/bert-large-cased'\\nbert_base = '../input/huggingface-bert-variants/bert-base-cased/bert-base-cased'\\nbert_base_uncased = '../input/huggingface-bert-variants/bert-base-uncased'\\ndistilbert_base = '../input/huggingface-bert-variants/distilbert-base-cased/distilbert-base-cased'\\ndistilbert_base_squad = '../input/huggingface-bert-variants/distilbert-base-cased-distilled-squad/distilbert-base-cased-distilled-squad'\\n\\n#tokenizer = transformers.BertTokenizer.from_pretrained(bert_base_uncased)\\n#tokenizer.save_pretrained('.')\\n\""
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\"\"\"\n",
    "bert_large = '../input/huggingface-bert-variants/bert-large-cased/bert-large-cased'\n",
    "bert_base = '../input/huggingface-bert-variants/bert-base-cased/bert-base-cased'\n",
    "bert_base_uncased = '../input/huggingface-bert-variants/bert-base-uncased'\n",
    "distilbert_base = '../input/huggingface-bert-variants/distilbert-base-cased/distilbert-base-cased'\n",
    "distilbert_base_squad = '../input/huggingface-bert-variants/distilbert-base-cased-distilled-squad/distilbert-base-cased-distilled-squad'\n",
    "\n",
    "#tokenizer = transformers.BertTokenizer.from_pretrained(bert_base_uncased)\n",
    "#tokenizer.save_pretrained('.')\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ee34cacd",
   "metadata": {
    "papermill": {
     "duration": 0.010729,
     "end_time": "2022-07-26T11:47:07.398964",
     "exception": false,
     "start_time": "2022-07-26T11:47:07.388235",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "### Splitting Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "e11e9c23",
   "metadata": {
    "_kg_hide-output": true,
    "execution": {
     "iopub.execute_input": "2022-07-26T11:47:07.421582Z",
     "iopub.status.busy": "2022-07-26T11:47:07.421295Z",
     "iopub.status.idle": "2022-07-26T11:47:07.429331Z",
     "shell.execute_reply": "2022-07-26T11:47:07.428460Z"
    },
    "papermill": {
     "duration": 0.021629,
     "end_time": "2022-07-26T11:47:07.431245",
     "exception": false,
     "start_time": "2022-07-26T11:47:07.409616",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0       0\n",
       "1       1\n",
       "2       1\n",
       "3       1\n",
       "4       1\n",
       "       ..\n",
       "5345    1\n",
       "5346    1\n",
       "5347    0\n",
       "5348    1\n",
       "5349    1\n",
       "Name: fake, Length: 5350, dtype: int64"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y = train.pop('fake')\n",
    "y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "167678fc",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-26T11:47:07.454472Z",
     "iopub.status.busy": "2022-07-26T11:47:07.453766Z",
     "iopub.status.idle": "2022-07-26T11:47:07.467345Z",
     "shell.execute_reply": "2022-07-26T11:47:07.466354Z"
    },
    "papermill": {
     "duration": 0.027151,
     "end_time": "2022-07-26T11:47:07.469379",
     "exception": false,
     "start_time": "2022-07-26T11:47:07.442228",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((5029,), (321,), (5029,), (321,))"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "X_train, X_valid, y_train, y_valid = train_test_split(train['text'], y, test_size=0.06, random_state=42, shuffle=True, stratify=y)\n",
    "X_train.shape, X_valid.shape, y_train.shape, y_valid.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f3d4d6c7",
   "metadata": {
    "papermill": {
     "duration": 0.011273,
     "end_time": "2022-07-26T11:47:07.492208",
     "exception": false,
     "start_time": "2022-07-26T11:47:07.480935",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "### bert encoding train and valid data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "6324e526",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-26T11:47:07.515976Z",
     "iopub.status.busy": "2022-07-26T11:47:07.515483Z",
     "iopub.status.idle": "2022-07-26T11:47:26.005115Z",
     "shell.execute_reply": "2022-07-26T11:47:26.004149Z"
    },
    "papermill": {
     "duration": 18.504099,
     "end_time": "2022-07-26T11:47:26.007430",
     "exception": false,
     "start_time": "2022-07-26T11:47:07.503331",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "X_train = bert_encode(X_train.astype(str), tokenizer)\n",
    "X_valid = bert_encode(X_valid.astype(str), tokenizer)\n",
    "\n",
    "y_train = y_train.values\n",
    "y_valid = y_valid.values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "850b68f2",
   "metadata": {
    "_kg_hide-output": true,
    "execution": {
     "iopub.execute_input": "2022-07-26T11:47:26.031894Z",
     "iopub.status.busy": "2022-07-26T11:47:26.031613Z",
     "iopub.status.idle": "2022-07-26T11:47:30.899617Z",
     "shell.execute_reply": "2022-07-26T11:47:30.897693Z"
    },
    "papermill": {
     "duration": 4.882055,
     "end_time": "2022-07-26T11:47:30.901626",
     "exception": false,
     "start_time": "2022-07-26T11:47:26.019571",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2022-07-26 11:47:26.094530: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2022-07-26 11:47:26.095591: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2022-07-26 11:47:26.096251: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2022-07-26 11:47:26.099329: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F FMA\n",
      "To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.\n",
      "2022-07-26 11:47:26.099636: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2022-07-26 11:47:26.100327: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2022-07-26 11:47:26.101008: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2022-07-26 11:47:30.517996: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2022-07-26 11:47:30.519068: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2022-07-26 11:47:30.519735: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2022-07-26 11:47:30.520312: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 15047 MB memory:  -> device: 0, name: Tesla P100-PCIE-16GB, pci bus id: 0000:00:04.0, compute capability: 6.0\n"
     ]
    }
   ],
   "source": [
    "train_dataset = (\n",
    "    tf.data.Dataset\n",
    "    .from_tensor_slices((X_train, y_train))\n",
    "    .repeat()\n",
    "    .shuffle(2048)\n",
    "    .batch(BATCH_SIZE)\n",
    "    .prefetch(AUTO)\n",
    ")\n",
    "\n",
    "valid_dataset = (\n",
    "    tf.data.Dataset\n",
    "    .from_tensor_slices((X_valid, y_valid))\n",
    "    .batch(BATCH_SIZE)\n",
    "    .cache()\n",
    "    .prefetch(AUTO)\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8142e8aa",
   "metadata": {
    "papermill": {
     "duration": 0.011357,
     "end_time": "2022-07-26T11:47:30.924944",
     "exception": false,
     "start_time": "2022-07-26T11:47:30.913587",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "### Setup GPU"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "3a81062f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-26T11:47:30.948509Z",
     "iopub.status.busy": "2022-07-26T11:47:30.948205Z",
     "iopub.status.idle": "2022-07-26T11:47:30.954553Z",
     "shell.execute_reply": "2022-07-26T11:47:30.953527Z"
    },
    "papermill": {
     "duration": 0.021735,
     "end_time": "2022-07-26T11:47:30.957837",
     "exception": false,
     "start_time": "2022-07-26T11:47:30.936102",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1 Physical GPUs, 1 Logical GPU\n"
     ]
    }
   ],
   "source": [
    "gpus = tf.config.list_physical_devices('GPU')\n",
    "if gpus:\n",
    "  # Restrict TensorFlow to only use the first GPU\n",
    "  try:\n",
    "    tf.config.set_visible_devices(gpus[0], 'GPU')\n",
    "    logical_gpus = tf.config.list_logical_devices('GPU')\n",
    "    print(len(gpus), \"Physical GPUs,\", len(logical_gpus), \"Logical GPU\")\n",
    "  except RuntimeError as e:\n",
    "    # Visible devices must be set before GPUs have been initialized\n",
    "    print(e)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3a4e4561",
   "metadata": {
    "papermill": {
     "duration": 0.011296,
     "end_time": "2022-07-26T11:47:30.980186",
     "exception": false,
     "start_time": "2022-07-26T11:47:30.968890",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Modeling"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "4b7a02f9",
   "metadata": {
    "_kg_hide-input": true,
    "_kg_hide-output": true,
    "execution": {
     "iopub.execute_input": "2022-07-26T11:47:31.003813Z",
     "iopub.status.busy": "2022-07-26T11:47:31.003561Z",
     "iopub.status.idle": "2022-07-26T11:47:31.010756Z",
     "shell.execute_reply": "2022-07-26T11:47:31.009873Z"
    },
    "papermill": {
     "duration": 0.021014,
     "end_time": "2022-07-26T11:47:31.012749",
     "exception": false,
     "start_time": "2022-07-26T11:47:30.991735",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "def build_model(bert_model, max_len=MAX_LEN):    \n",
    "    input_ids = Input(shape=(max_len,), dtype=tf.int32, name=\"input_ids\")\n",
    "    token_type_ids = Input(shape=(max_len,), dtype=tf.int32, name=\"token_type_ids\")\n",
    "    attention_mask = Input(shape=(max_len,), dtype=tf.int32, name=\"attention_mask\")\n",
    "\n",
    "    sequence_output = bert_model.bert(input_ids, token_type_ids=token_type_ids, attention_mask=attention_mask)[0]\n",
    "    clf_output = sequence_output[:, 0, :]\n",
    "    clf_output = Dropout(.1)(clf_output)\n",
    "    out = Dense(1, activation='sigmoid')(clf_output)\n",
    "    \n",
    "    model = Model(inputs=[input_ids, token_type_ids, attention_mask], outputs=out)\n",
    "    model.compile(Adam(learning_rate=1e-5), loss='binary_crossentropy', \n",
    "                  metrics=['accuracy', tf.keras.metrics.Precision(),]\n",
    "                 )\n",
    "    \n",
    "    return model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "a2ecc04d",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-26T11:47:31.037180Z",
     "iopub.status.busy": "2022-07-26T11:47:31.036934Z",
     "iopub.status.idle": "2022-07-26T11:47:31.044905Z",
     "shell.execute_reply": "2022-07-26T11:47:31.043910Z"
    },
    "papermill": {
     "duration": 0.022629,
     "end_time": "2022-07-26T11:47:31.047142",
     "exception": false,
     "start_time": "2022-07-26T11:47:31.024513",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "def build_model2(bert_model, max_len=MAX_LEN):    \n",
    "    input_ids = Input(shape=(max_len,), dtype=tf.int32, name=\"input_ids\")\n",
    "    input_masks_ids = Input(shape=(max_len,), dtype=tf.int32, name=\"masked_token\")\n",
    "    segment_ids = Input(shape=(max_len,), dtype=tf.int32, name=\"segment_ids\")\n",
    "\n",
    "    sequence_output = bert_model.bert(input_ids, attention_mask=input_masks_ids, token_type_ids=segment_ids)[0]\n",
    "    clf_output = sequence_output[:, 0, :]\n",
    "    clf_output = Dropout(.1)(clf_output)\n",
    "    out = Dense(1, activation='sigmoid')(clf_output)\n",
    "    \n",
    "    model = Model(inputs=[input_ids, input_masks_ids, segment_ids], outputs=out)\n",
    "    model.compile(Adam(learning_rate=1e-5), loss='binary_crossentropy', \n",
    "                  metrics=['accuracy', tf.keras.metrics.Precision(),]\n",
    "                 )\n",
    "    \n",
    "    return model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "5a9651f0",
   "metadata": {
    "_kg_hide-output": true,
    "execution": {
     "iopub.execute_input": "2022-07-26T11:47:31.070795Z",
     "iopub.status.busy": "2022-07-26T11:47:31.070280Z",
     "iopub.status.idle": "2022-07-26T11:47:42.292865Z",
     "shell.execute_reply": "2022-07-26T11:47:42.291852Z"
    },
    "papermill": {
     "duration": 11.236814,
     "end_time": "2022-07-26T11:47:42.295021",
     "exception": false,
     "start_time": "2022-07-26T11:47:31.058207",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Some layers from the model checkpoint at ../input/huggingface-bert-variants/bert-base-cased/bert-base-cased were not used when initializing TFBertModel: ['nsp___cls', 'mlm___cls']\n",
      "- This IS expected if you are initializing TFBertModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n",
      "- This IS NOT expected if you are initializing TFBertModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n",
      "All the layers of TFBertModel were initialized from the model checkpoint at ../input/huggingface-bert-variants/bert-base-cased/bert-base-cased.\n",
      "If your task is similar to the task the model of the checkpoint was trained on, you can already use TFBertModel for predictions without further training.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"model\"\n",
      "__________________________________________________________________________________________________\n",
      "Layer (type)                    Output Shape         Param #     Connected to                     \n",
      "==================================================================================================\n",
      "input_ids (InputLayer)          [(None, 256)]        0                                            \n",
      "__________________________________________________________________________________________________\n",
      "attention_mask (InputLayer)     [(None, 256)]        0                                            \n",
      "__________________________________________________________________________________________________\n",
      "token_type_ids (InputLayer)     [(None, 256)]        0                                            \n",
      "__________________________________________________________________________________________________\n",
      "bert (TFBertMainLayer)          TFBaseModelOutputWit 108310272   input_ids[0][0]                  \n",
      "                                                                 attention_mask[0][0]             \n",
      "                                                                 token_type_ids[0][0]             \n",
      "__________________________________________________________________________________________________\n",
      "tf.__operators__.getitem (Slici (None, 768)          0           bert[0][0]                       \n",
      "__________________________________________________________________________________________________\n",
      "dropout_37 (Dropout)            (None, 768)          0           tf.__operators__.getitem[0][0]   \n",
      "__________________________________________________________________________________________________\n",
      "dense (Dense)                   (None, 1)            769         dropout_37[0][0]                 \n",
      "==================================================================================================\n",
      "Total params: 108,311,041\n",
      "Trainable params: 108,311,041\n",
      "Non-trainable params: 0\n",
      "__________________________________________________________________________________________________\n",
      "CPU times: user 6.48 s, sys: 523 ms, total: 7 s\n",
      "Wall time: 11.2 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "# Define Bert Config\n",
    "#config = BertConfig.from_pretrained(bert, #output_hidden_states=True, \n",
    "                                    #hidden_dropout_prob=0.2, attention_probs_dropout_prob=0.2\n",
    "#                                   )\n",
    "\n",
    "# Define Bert Model\n",
    "#transformer_model = TFBertForSequenceClassification.from_pretrained(bert, config=config)\n",
    "\n",
    "transformer_model = (TFBertModel.from_pretrained(bert))\n",
    "\n",
    "# Define Keras Model\n",
    "model = build_model(transformer_model, max_len=MAX_LEN)\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "73ecbf25",
   "metadata": {
    "papermill": {
     "duration": 0.011285,
     "end_time": "2022-07-26T11:47:42.319200",
     "exception": false,
     "start_time": "2022-07-26T11:47:42.307915",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "### Training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "8cbc8fbf",
   "metadata": {
    "_kg_hide-output": true,
    "execution": {
     "iopub.execute_input": "2022-07-26T11:47:42.344693Z",
     "iopub.status.busy": "2022-07-26T11:47:42.343473Z",
     "iopub.status.idle": "2022-07-26T12:25:13.367013Z",
     "shell.execute_reply": "2022-07-26T12:25:13.365959Z"
    },
    "papermill": {
     "duration": 2251.180506,
     "end_time": "2022-07-26T12:25:13.511188",
     "exception": false,
     "start_time": "2022-07-26T11:47:42.330682",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/32\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2022-07-26 11:47:52.656992: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:185] None of the MLIR Optimization Passes are enabled (registered 2)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "240/240 [==============================] - 142s 534ms/step - loss: 0.4052 - accuracy: 0.8120 - precision: 0.8344 - val_loss: 0.3010 - val_accuracy: 0.8910 - val_precision: 0.8715\n",
      "Epoch 2/32\n",
      "240/240 [==============================] - 124s 517ms/step - loss: 0.1627 - accuracy: 0.9378 - precision: 0.9479 - val_loss: 0.1745 - val_accuracy: 0.9283 - val_precision: 0.9457\n",
      "Epoch 3/32\n",
      "240/240 [==============================] - 124s 517ms/step - loss: 0.0919 - accuracy: 0.9664 - precision: 0.9745 - val_loss: 0.1550 - val_accuracy: 0.9470 - val_precision: 0.9552\n",
      "Epoch 4/32\n",
      "240/240 [==============================] - 124s 517ms/step - loss: 0.0688 - accuracy: 0.9747 - precision: 0.9809 - val_loss: 0.1678 - val_accuracy: 0.9439 - val_precision: 0.9509\n",
      "Epoch 5/32\n",
      "240/240 [==============================] - 124s 517ms/step - loss: 0.0326 - accuracy: 0.9885 - precision: 0.9920 - val_loss: 0.2044 - val_accuracy: 0.9502 - val_precision: 0.9435\n",
      "Epoch 6/32\n",
      "240/240 [==============================] - 124s 517ms/step - loss: 0.0157 - accuracy: 0.9956 - precision: 0.9973 - val_loss: 0.1851 - val_accuracy: 0.9470 - val_precision: 0.9432\n",
      "Epoch 7/32\n",
      "240/240 [==============================] - 124s 516ms/step - loss: 0.0060 - accuracy: 0.9987 - precision: 0.9989 - val_loss: 0.2023 - val_accuracy: 0.9533 - val_precision: 0.9638\n",
      "Epoch 8/32\n",
      "240/240 [==============================] - 124s 517ms/step - loss: 0.0030 - accuracy: 0.9992 - precision: 0.9992 - val_loss: 0.2606 - val_accuracy: 0.9470 - val_precision: 0.9552\n",
      "Epoch 9/32\n",
      "240/240 [==============================] - 124s 516ms/step - loss: 0.0019 - accuracy: 0.9997 - precision: 1.0000 - val_loss: 0.2580 - val_accuracy: 0.9502 - val_precision: 0.9595\n",
      "Epoch 10/32\n",
      "240/240 [==============================] - 124s 516ms/step - loss: 0.0016 - accuracy: 0.9997 - precision: 1.0000 - val_loss: 0.2236 - val_accuracy: 0.9502 - val_precision: 0.9595\n",
      "Epoch 11/32\n",
      "240/240 [==============================] - 124s 517ms/step - loss: 0.0021 - accuracy: 0.9997 - precision: 1.0000 - val_loss: 0.2723 - val_accuracy: 0.9439 - val_precision: 0.9509\n",
      "Epoch 12/32\n",
      "240/240 [==============================] - 124s 516ms/step - loss: 0.0016 - accuracy: 0.9997 - precision: 0.9996 - val_loss: 0.2496 - val_accuracy: 0.9502 - val_precision: 0.9595\n",
      "Epoch 13/32\n",
      "240/240 [==============================] - 124s 517ms/step - loss: 0.0019 - accuracy: 0.9995 - precision: 0.9996 - val_loss: 0.2475 - val_accuracy: 0.9533 - val_precision: 0.9638\n",
      "Epoch 14/32\n",
      "240/240 [==============================] - 124s 516ms/step - loss: 0.0017 - accuracy: 0.9995 - precision: 1.0000 - val_loss: 0.2472 - val_accuracy: 0.9470 - val_precision: 0.9552\n",
      "Epoch 15/32\n",
      "240/240 [==============================] - 124s 517ms/step - loss: 0.0014 - accuracy: 0.9997 - precision: 0.9996 - val_loss: 0.2424 - val_accuracy: 0.9533 - val_precision: 0.9638\n",
      "Epoch 16/32\n",
      "240/240 [==============================] - 124s 516ms/step - loss: 0.0012 - accuracy: 0.9997 - precision: 1.0000 - val_loss: 0.2468 - val_accuracy: 0.9470 - val_precision: 0.9552\n",
      "Epoch 17/32\n",
      "240/240 [==============================] - 124s 516ms/step - loss: 0.0012 - accuracy: 0.9992 - precision: 0.9996 - val_loss: 0.2483 - val_accuracy: 0.9502 - val_precision: 0.9595\n",
      "Epoch 18/32\n",
      "240/240 [==============================] - 124s 517ms/step - loss: 0.0023 - accuracy: 0.9997 - precision: 0.9996 - val_loss: 0.2461 - val_accuracy: 0.9533 - val_precision: 0.9638\n"
     ]
    }
   ],
   "source": [
    "early_stopping = tf.keras.callbacks.EarlyStopping(\n",
    "    patience=15,\n",
    "    min_delta=0.001,\n",
    "    restore_best_weights=True,\n",
    ")\n",
    "\n",
    "reduce_lr = tf.keras.callbacks.ReduceLROnPlateau(\n",
    "    monitor='val_loss', factor=0.7,\n",
    "    patience=1, #min_lr=1357e-6\n",
    ")\n",
    "\n",
    "history = model.fit(\n",
    "    train_dataset,\n",
    "    validation_data=valid_dataset,\n",
    "    steps_per_epoch=240,\n",
    "    #batch_size=512,\n",
    "    epochs=32,\n",
    "    callbacks=[early_stopping, reduce_lr],\n",
    "    #verbose=0\n",
    "\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "7e4fc7d3",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-26T12:25:14.094955Z",
     "iopub.status.busy": "2022-07-26T12:25:14.094490Z",
     "iopub.status.idle": "2022-07-26T12:25:14.796203Z",
     "shell.execute_reply": "2022-07-26T12:25:14.795305Z"
    },
    "papermill": {
     "duration": 1.021015,
     "end_time": "2022-07-26T12:25:14.798214",
     "exception": false,
     "start_time": "2022-07-26T12:25:13.777199",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best Validation Loss: 0.1550\n",
      "Best Validation Accuracy: 0.9533\n",
      "Best Validation Precision: 0.9638\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAD4CAYAAAAEhuazAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAA0DUlEQVR4nO3deXwU9f348dd7Nxe5SCCRKxwJBAXBM3JUwAsRrYKtWrzqUZV6a7V+a79qa63+avVbrW2pSBWvagG1tVStiCeicgQEERQSwpXIkXCTQK59//6YCWyWBDZkk91s3s/HYx+7O/OZ2fdONu+Z+cxnPh9RVYwxxrQvnnAHYIwxpvVZ8jfGmHbIkr8xxrRDlvyNMaYdsuRvjDHtUEy4AwiUkZGhffr0CXcYxhjTpixatKhMVTODLR9xyb9Pnz7k5+eHOwxjjGlTRGRdU8pbtY8xxrRDlvyNMaYdsuRvjDHtUMTV+Rtj2qfq6mqKi4vZt29fuEOJaAkJCWRlZREbG9us9VjyN8ZEhOLiYlJSUujTpw8iEu5wIpKqsnXrVoqLi8nOzm7WuqzaxxgTEfbt20fnzp0t8R+CiNC5c+eQnB1Z8jfGRAxL/IcXqm0UNcl/Z0U1T71fwFfFO8IdijHGRLygkr+IjBWRlSJSKCL3HqLcRSKiIpLnN+2X7nIrReScUATdEI8Hnnx/FZ8Vbm2pjzDGRLnk5ORwh9BqDnvBV0S8wCTgbKAYWCgiM1V1RUC5FOAOYL7ftIHApcCxQHfgfRHpr6q1ofsKjpSEWDJT4llTtifUqzbGmKgTzJH/EKBQVYtUtQqYBoxvoNxvgd8D/lcixgPTVLVSVdcAhe76WkRORhJFpeUttXpjTDuhqtxzzz0MGjSIwYMHM336dAA2btzIqFGjOOGEExg0aBCffvoptbW1XHPNNfvLPvnkk2GOPjjBNPXsAWzwe18MDPUvICInAT1V9W0RuSdg2XkBy/YI/AARmQhMBOjVq1dwkTcgJzOJWcs3H/HyxpjI8Jv/LGfFd7tCus6B3VP59QXHBlX2n//8J0uWLGHp0qWUlZVxyimnMGrUKF599VXOOecc7rvvPmpra6moqGDJkiWUlJTw9ddfA7Bjx46Qxt1Smn3BV0Q8wBPA3Ue6DlWdoqp5qpqXmRl0p3QHyclIZlt5FTsqqo54HcYYM3fuXC677DK8Xi9dunThtNNOY+HChZxyyik8//zzPPjggyxbtoyUlBRycnIoKiritttu49133yU1NTXc4QclmCP/EqCn3/ssd1qdFGAQ8LHbBKkrMFNExgWxbEjlZCYBUFRWzkm94lrqY4wxLSzYI/TWNmrUKObMmcPbb7/NNddcw1133cVVV13F0qVLmTVrFpMnT2bGjBlMnTo13KEeVjBH/guBXBHJFpE4nAu4M+tmqupOVc1Q1T6q2genmmecqua75S4VkXgRyQZygQUh/xau7Aw3+Vu9vzGmGUaOHMn06dOpra2ltLSUOXPmMGTIENatW0eXLl244YYbuP7661m8eDFlZWX4fD4uuugiHn74YRYvXhzu8INy2CN/Va0RkVuBWYAXmKqqy0XkISBfVWceYtnlIjIDWAHUALe0REufOj07JRLjEWvxY4xplh/84Ad88cUXHH/88YgIjz32GF27duXFF1/k8ccfJzY2luTkZF566SVKSkq49tpr8fl8APzud78Lc/TBEVUNdwz15OXlaXMGcznzDx9zdJcUnr7y5BBGZYxpad988w0DBgwIdxhtQkPbSkQWqWpeI4scJGru8K2Tk5HEmjKr9jHGmEOJvuSfmcyasnJ8vsg6ozHGmEgSdck/OyOJyhofJTv2hjsUY4yJWFGX/HPcFj9W9WOMMY2LvuSf6XTMVFRqLX6MMaYxUZf8M5LjSImPociO/I0xplFRl/xFhJxMa/FjjDGHEnXJH5yqH7vL1xjTkg7V9//atWsZNGhQK0bTdFGZ/LMzkijZsZe9VS12M7ExxrRpwXTs1ubUdfC2dms5A7q1jR72jDF+/nsvbFoW2nV2HQznPtro7HvvvZeePXtyyy23APDggw8SExPDRx99xPbt26murubhhx9m/PiGhjNp3L59+7jpppvIz88nJiaGJ554gjPOOIPly5dz7bXXUlVVhc/n44033qB79+786Ec/ori4mNraWh544AEmTJjQrK/dmKhM/v4dvFnyN8YEY8KECdx55537k/+MGTOYNWsWt99+O6mpqZSVlTFs2DDGjRvXpEHUJ02ahIiwbNkyvv32W8aMGcOqVauYPHkyd9xxB1dccQVVVVXU1tbyzjvv0L17d95++20Adu7c2SLfFaI++VtzT2PapEMcobeUE088kS1btvDdd99RWlpKeno6Xbt25Wc/+xlz5szB4/FQUlLC5s2b6dq1a9DrnTt3LrfddhsAxxxzDL1792bVqlUMHz6cRx55hOLiYn74wx+Sm5vL4MGDufvuu/nFL37B+eefz8iRI1vq60ZnnX9iXAzdOyZYix9jTJNccsklvP7660yfPp0JEybwyiuvUFpayqJFi1iyZAldunRh3759h19REC6//HJmzpxJhw4dOO+88/jwww/p378/ixcvZvDgwdx///089NBDIfmshkTlkT9AdmYSqy35G2OaYMKECdxwww2UlZXxySefMGPGDI466ihiY2P56KOPWLduXZPXOXLkSF555RXOPPNMVq1axfr16zn66KMpKioiJyeH22+/nfXr1/PVV19xzDHH0KlTJ6688krS0tJ49tlnW+BbOqI2+edkJPPvJSWoapPq54wx7dexxx7L7t276dGjB926deOKK67gggsuYPDgweTl5XHMMcc0eZ0333wzN910E4MHDyYmJoYXXniB+Ph4ZsyYwcsvv0xsbCxdu3blf//3f1m4cCH33HMPHo+H2NhYnn766Rb4lo6g+vMXkbHAUziDuTyrqo8GzL8RuAWoBfYAE1V1hYj0Ab4BVrpF56nqjYf6rOb251/n+c/W8Jv/rCD//tFkJMc3e33GmJZl/fkHLxT9+R/2yF9EvMAk4GygGFgoIjNVdYVfsVdVdbJbfhzOgO5j3XmrVfWEYAMKFf8WP5b8jTGmvmCqfYYAhapaBCAi04DxOEMzAqCqu/zKJwFh70y/r9vB25qyPQzJ7hTmaIwx0WjZsmX8+Mc/rjctPj6e+fPnhymi4AWT/HsAG/zeFwNDAwuJyC3AXUAccKbfrGwR+RLYBdyvqp82sOxEYCJAr169gg7+ULqndSAuxmPdPBjThrS1a3SDBw9myZIlrfqZoRp6N2RNPVV1kqr2BX4B3O9O3gj0UtUTcXYMr4rIQXddqeoUVc1T1bzMzMyQxOP1CH06J7Lakr8xbUJCQgJbt24NWXKLRqrK1q1bSUhIaPa6gjnyLwF6+r3Pcqc1ZhrwNICqVgKV7utFIrIa6A80/4puEHIykinYsrs1PsoY00xZWVkUFxdTWloa7lAiWkJCAllZWc1eTzDJfyGQKyLZOEn/UuBy/wIikquqBe7b7wMF7vRMYJuq1opIDpALFDU76iBlZybxwbebqan1EeONyvvZjIkasbGxZGdnhzuMduOwyV9Va0TkVmAWTlPPqaq6XEQeAvJVdSZwq4iMBqqB7cDV7uKjgIdEpBrwATeq6raW+CINyclIorpW2bB97/7WP8YYY4K8yUtV3wHeCZj2K7/XdzSy3BvAG80JsDly/Fr8WPI3xpgDorouJMevrb8xxpgDojr5pyfFkZ4Yay1+jDEmQFQnf3CqftaUWdfOxhjjL+qTf3ZGklX7GGNMgKhP/jmZSWzZXcmeyppwh2KMMREj+pN/htvix47+jTFmv+hP/u5g7kVW72+MMftFffLv3TkREWvuaYwx/qI++cfHeMlK70CRDelojDH7RX3yB6fev6jUqn2MMaZO+0j+mUmsKSu3rmKNMcbVPpJ/RhIVVbVs3lUZ7lCMMSYitI/k73bwZlU/xhjjaCfJv665p130NcYYaCfJv0tKAh1ivdbc0xhjXO0i+Xs8QnZGknXwZowxrqCSv4iMFZGVIlIoIvc2MP9GEVkmIktEZK6IDPSb90t3uZUick4og2+KnMwkq/YxxhjXYZO/iHiBScC5wEDgMv/k7npVVQer6gnAY8AT7rIDccb8PRYYC/zVXV/olW+F938D333Z4OycjCQ2bKugsqa2RT7eGGPakmCO/IcAhapapKpVwDRgvH8BVd3l9zYJqGtQPx6YpqqVqroGKHTXF3reWPj8T7D8zQZn52Qm41PYsK2iRT7eGGPakmCSfw9gg9/7YndaPSJyi4isxjnyv72Jy04UkXwRyS8tLQ029voSUqHXcCiY3eDsujF8bVQvY4wJ4QVfVZ2kqn2BXwD3N3HZKaqap6p5mZmZRx5E7hjYshx2Fh80a39zT0v+xhgTVPIvAXr6vc9ypzVmGnDhES7bPLljnOcGjv5TEmLJTIm3Fj/GGENwyX8hkCsi2SISh3MBd6Z/ARHJ9Xv7faDAfT0TuFRE4kUkG8gFFjQ/7EZkHg0dex2y6seO/I0xBmIOV0BVa0TkVmAW4AWmqupyEXkIyFfVmcCtIjIaqAa2A1e7yy4XkRnACqAGuEVVW665jQjkng1Lp0FNJcTE15vdNzOJWcs3t9jHG2NMW3HY5A+gqu8A7wRM+5Xf6zsOsewjwCNHGmCT5Y6B/Odg3efQ94x6s3IyktlWvoEdFVWkJca1WkjGGBNpou8O3+yR4I1vsOqnrsWP3exljGnvoi/5xyVBnxFQ8N5Bs6zFjzHGOKIv+YNT9bO1ALYV1Zvcs1MiMR6xFj/GmHYvSpP/2c5zwfv1Jsd6PfTqlGhH/saYdi86k3/nvtCpb6NVP2uszt8Y085FZ/IHp+pn7adQVb8vH6dr53J8PhvP1xjTfkVx8j8bavbB2rn1JudkJlNZ46Nkx94wBWbMYajCkldhy7fhjsREsehN/r1PhdjEg6p+ctzmnlb1YyLWginw5k0w+VR4/8GDzl6NCYXoTf6xCZB9GhTMco6kXNn7m3taix8TgUoWw6z7oN/ZcNwEmPsk/HXYQY0XjGmu6E3+4FT97FgPZQX7J2Umx5MSH2M3epnIs3cHvHYNJHeBH06BC/8KV78F3jh45SJn3u5NYQ7SRIvoT/5Qr+pHRKzFj4k8qvDvW2BXCVzyPCR2cqZnj4SbPoMz7odv34G/nAIL/gY+G5HONE90J/+0XpA54KB6f+vd00Sc+c/At2/B6AehZ8BgdzHxcNo9cPMX0ONkeOfn8Oxo2Lg0LKGa6BBUx25tWv8x8MVfoXI3xKcAToufN5d8x96qWjrEtcyQwqaVzJsMK9+BDunBPWITwh3xwUoWwXv3Q/9zYfitjZfr3Bd+/C/4+g14916YcjoMvQnO+OX+37YxwYr+5J87Bj57Coo+gQHnAwc6eFu7tZwB3VLDGZ1pjvXz4d1fODf07d4Ie7c7D19N48vEdAjYIaQdeD3wQsg6ubWid+zd7tTlp3R16vhFDl1eBAZfDP3Ogvd/A/MmwYo34dzH9v++jQlG9Cf/nkMhPtWp+nH/Ofw7eLPk30bVVMF/7oCOPeGncyA+2ZmuClV7DuwIGn3scJ63FTnPFVudqpcfvQRHj22d76AK/74Vdn0H1757oJ4/GB3S4YI/wgmXw3/uhOlXwNHnOTuBtJ6HW9qY4JK/iIwFnsIZzOVZVX00YP5dwPU4A7aUAj9R1XXuvFpgmVt0vaqOC1HswfHGOv36F8x2/tlEDnTtbM09267P/wSl38Bl0w8kfnCOjONTnEdar+DXV7EN/v5DJ4n+4Bnn6LqlzZ/s1POPeQR6nnJk6+g5BH76Ccz7K3z8KEwa6lQDDb0JvNF/bGeO3GEv+IqIF5gEnAsMBC4TkYEBxb4E8lT1OOB14DG/eXtV9QT30bqJv07uGNj9HWxeDkBiXAzdOiZYi5+2autqmPM4DBgXuqP0xE5w1UzoOQzeuB7yp4ZmvY0pXgTvPeAcrQ+/pXnr8sbCqXfALfOd1kHv3e9cDyjOD0moJjoF09pnCFCoqkWqWoUzQPt4/wKq+pGq1t2GOA9noPbI0W+081wwa/+knMwkVlvyb3tU4e27nLbv5z52+PJNkZAKV77uHCy89TOY+8fQrr/O/nr+bjB+0uHr+YOV1gsumwYT/u5UYz072qkS+vqfzk1i6+fDlm9gZzHs2wk+X2g+17RJwZwX9gA2+L0vBoYeovx1wH/93ieISD5OldCjqvpm4AIiMhGYCNCrVxNO1YOV0hW6He9U/Yy8G3CGdPz3khJUFQnVP59peV/NgKKP4bz/g9RuoV9/bAe49BX410/h/V9D5S4484HQJWhVePMW5wL1T5pYzx8MERhwAeScDh8+AguegUXPN14+LuVANVlC6oHX8anuw50elwyeGBCP8/B4nc8SD4g3YLrHndfAdG+cc50mKSN027Q5fD6ornAeVeXucwVUl7vPFQ1Pq1e2AtQHCR0bfsSnBkxLdba7J7wt7UNaKSgiVwJ5wGl+k3uraomI5AAfisgyVV3tv5yqTgGmAOTl5bVMd5u5Y+DTPzhHXR3Syc5IYte+GraWV5GRHH/45U34VWyDWb+EHnmQ95OW+xxvLPzwb07C+/QPsG+Xc5YRin/WeX+FlW/DOb+DrLzmr68x8Slw7qMw6h4o3+I0dd63y9mZVe52H7sOPO/bdaDMzpID86pa6LpYbCKk94G03s5zeh9I731gWlxi8z9D1TkD2rHeOdvZWQw7N7iPYtixASrKmrZO8UBskhNfrPuISwQE9mx2zqj27XR2CIdekbuzDdgpHDUQznrgSL9xkwST/EsA/+YDWe60ekRkNHAfcJqqVtZNV9US97lIRD4GTgRWBy7f4nLHOPXEqz+EQRfVa/Fjyb+NmP2A8491wVPOkWRL8nidz0lIhc//7CTC8ZOadxG1OB9m/wqO/j4Muyl0sR5KUmfncaR8tQd2Ar5a5wjX/1Fvmvva5zt4Wl3Zmn1O0t2xDravdR5r5jhH1fXiPqr+DsF/R5Ha3fn71FTBruIDiX2HX2Kve67ZV3+9sYnOmUfHLKc2ILmLM/RrbGIDz/4J3p0WEx/cGUtttbMj3bfD3bm6O4V9/q931p+3Y4NzZtRKgvklLwRyRSQbJ+lfClzuX0BETgSeAcaq6ha/6elAhapWikgGcCr1Lwa3nh4nO83jCmbDoIvom+m0EFlTtoch2SE+9Taht3YufPl3OPVO6DqodT5TBM7+rXNU9uHDTgK8eKqTAJqqYhu8dq2TuC4MYT1/S/N43Xsh0lruM1ShvMxvh7AGtruv1893bmpTv+sTnlgnnvIyIKCiILmLk9i7DIL+Y51En+Ym+449nRzQGtveG9v8HW8LO2zyV9UaEbkVmIXT1HOqqi4XkYeAfFWdCTwOJAOvufXndU06BwDPiIgP5+Lyo6q6ooW+y6F5vM6F34LZ4PPRPa0DcTEe6+ahLaipdC5cpvWG037Rup8t4lSdxKfCf/8HXv0RTHilfvPSw6nrt2f3RvjJLCcBmQNEIDnTeTRUFVZb7RzJ1+0Qdqxzdqap3Q8cxdc9jmTH3E4FdQ6rqu8A7wRM+5Xf69GNLPc5MLg5AYZU7hhY9hps/BJvj5Pp0zmR1Zb8I9/cJ2FrAVz5Rmjqgo/E0J869ej/vgVe/gFcMSP4JP7FJKcLirGPtv4dxNHAGwudcpyHCZno7tgtUN+zAHGO/qkb0tFu9Ipopauci66DLj7QZDdcTrgcLnkRNi6BFy6APVsOuwgbFjqtho45H4be2OIhGhOs9pX8kzo7p5VuL585mcms31ZBTa21d45Iqk57+9gOMPZ34Y7GMXAcXD4dtq2GqWOdi3SNqdgGr7v1/OP/0nbq+U270L6SPzhVPyWLYU8pORlJVNcqG7bbeL4RackrsG4unP0QJB8V7mgO6Hum07tmeZmzAygrPLiMqjMU4+5NcMkLVs9vIk47TP5nAwqrP9jf3NOqfiJQeZnTTUGv4XDiVeGO5mC9hsE1bzlNCZ8fC5uW1Z//xV9g1bsw5mGnpZkxEab9Jf+uxzttiAveIyfDabFhLX4i0Kz/hco9bpv+CP2ZdjvOuUvXGwcvfN9plgiwYYEz8PqAC5wLxcZEoAj9r2pBHo9z9F/4AekJHtITY63FT6RZ/SF8NR1G/Awyjw53NIeWket209AZXr7Q6UfntWshtQeMs3p+E7naX/IHJ/nv2wEl+dbiJ9JU74W37nIGaHH7YYp4ab2c/vjTs50LvHs2u/X8aeGOzJhGtc/kn3OG0+lUwXvkZCZbtU8kmfO4c4fn+U9G5pCLjUnp4lwDGDgexv0ZepwU7oiMOaT2mfw7pDkX7AreIycziS27K9lTeYih/0zr2LzCGXLz+Msh57TDl480iZ2ckcBOuCzckRhzWO0z+YNT9bNpGQOSnCqfNXb0H14+H7x1p9ONwpiHwx2NMVGvHSf/MQAMKF8AQJHV+4fX4hdgw3w455GI7gzLmGjRfpP/UQMhtQeZGz9BxJp7htXuTTD7QcgeBcdblYkxraH9Jn8RyD0b75pP6JMWQ5EN6Rg+797r3Cz1/SetaaQxraT9Jn9wqn6qdjMmZR1FpVbtExar3oPl/4JRP4eMfuGOxph2o30n/+zTwBvHKL5kTVk5qi0zgqRpRFU5vH03ZBztDNJijGk1QSV/ERkrIitFpFBE7m1g/l0iskJEvhKRD0Skt9+8q0WkwH1cHcrgmy0+GXqfyrHl86ioqmXzrsrDL9NWVe+FjUudDscixce/g53r4YI/QkzrDV9njAki+YuIF5gEnAsMBC4TkYEBxb4E8lT1OOB13KEaRaQT8GtgKDAE+LU7tGPkyB1DWnkRWVIavVU/lbvhpQvhmVHw4gVOH/PhtnEpfPFXOOkq6P29cEdjTLsTzJH/EKBQVYtUtQqYBoz3L6CqH6lq3XD183AGeQc4B5itqttUdTswGxgbmtBDxG3yebpnSXRe9K3cDX+/GIoXwtCboPRbeG40/OMy56aq1rZ9rdN9w7NnOzdFjf5N68dgjAkq+fcA/EesKHanNeY64L9HuGzr69wXTc/mrJil0dfc0z/xX/I8nPso3L4EzrzfGRD96e/BP3/qJOSWtulreON6+NNJsPglOH4CXDfb2QEYY1pdUGP4BktErgTygCbdmy8iE4GJAL169QplSMF8OJI7huELXuDV0m2t+9ktKTDxD3RP1uKTnQHJ866Dz/4I85+Br9+Ak69xWtykdA1tHOs+d8bgLXgP4pJh+M0w7GZndCtjTNgEc+RfAvT0e5/lTqtHREYD9wHjVLWyKcuq6hRVzVPVvMzMzGBjD53cMSRQScfNC1r/s1tCY4nfX2InZ4Ss25fAST+GRc/DUyfA+7+Bvdub9/k+H6x8F547B54/F0oWwRn3w8++drpusMRvTNgFk/wXArkiki0iccClwEz/AiJyIvAMTuL3H9V6FjBGRNLdC71j3GmRpc+pVHviObZ8HpU1teGOpnmCSfz+Urs5PWjesgAGnO8cpT91PHz6BFRVHHrZQLXVsHQ6TD4V/jEBdn0H5z4Od34Np91jQxkaE0EOm/xVtQa4FSdpfwPMUNXlIvKQiIxziz0OJAOvicgSEZnpLrsN+C3ODmQh8JA7LbLEdmBr5jBO9yxhw7YmJrxI0tTE769zX7joWbhxrjN04ge/gT+dAAv+BjVVh162qsIp9+eT4F8TneakP5gCty+GoRMhLrFZX8sYE3oSaTc25eXlaX5+fqt/bvGsP5H1xQN8OnYWI4cNa/XPb7bmJP6GrJ/nVAGt/xzSesMZ98Hgi8HjPVBm73ZY+CzMmwwVZZA1BEbeBbnnRO7Qi8ZEKRFZpKp5wZa3/1BXx+PPA8BT8F6YIzkCoU784Ix3cO07cMXrkJDqHNFPHgHfvgO7NsJ7D8CTg+HDh52BS679L1z3Hhx9riV+Y9qAkLb2actSuvajiCwyN30S7lCapiUSfx238zv6ngUr/gUfPgLTLgPEmTfoIjj1Dug6OHSfaYxpFZb8/SxPGsrY8n9D5R6nSWSka8nE78/jcRL9gHGw9B+wrQhOuho6ZbfM5xljWpwlfz+buowktugNWDMHjjkv3OEcmn/iv3hqyyV+f95YpzsGY0ybZ5WzfqTXcPZoApXfvBvuUA4tMPEfe2G4IzLGtDGW/P30OSqdub7BSOHsyOr90p8lfmNMCFjy95OTmcRHvhOIK/8OVkXevWiW+I0xoWLJ30/PTol8qHnsjOvq3KH6yiWwaVm4w3JY4jfGhJAlfz+xXg8pnbryQNZUp6vhDfNh8kinN8ptReELzBK/MSbELPkHyM5IYtW2WhhxJ9yxFEb8DL55C/5yijPk4O7NrRvQjvWW+I0xIWdNPQPkZCYxt7AMn0/xdEiH0b+GoT+FTx6DRS/AkledLolPvR0SOrZMEHtKYcWbsOx12DAPPDGW+I0xIWXJP0BOZjKVNT5KduylZye3Q7KUrnD+EzD8Fvjo/8Gn/wf5z8GIu2DIDRDbofkfvG8XfPs2LHsNij4GrYWjBsKZDzh96qT3af5nGGOMy5J/gOyMJABWbtp9IPnX6dwXLn7OOer/4Lcw+wGY9zScfi+ccAV4m7g5q/c5g5x8/brTuqhmH6T1crpMGHwxdDk2RN/KGGPqs+Qf4PisNDKS45j62RpGD+zScKFux8OVrztDIb7/IPzndvj8z87wiAPHO/3eNKa2BtbOgWVvwDczoXIXJGU6d84OvgSyTjn08sYYEwKW/AN0iPNy0+n9+O1bK/i8sIzv9ctovHCfEc44tCvfgQ8egteuhu4nwlm/hr5nHCinCsX5TpXO8n9B+RaIT4UBFzh95mSf1vSzBmOMaQbrz78B+6prOf3xj+melsAbN30PCeZI3FcLX013rgns3OAk9GE3Q/EC58LtjnXgjYf+5zhH+LljIDah5b+MMaZdaJH+/EVkrIisFJFCEbm3gfmjRGSxiNSIyMUB82rd0b32j/AV6RJivdx2Vj8Wr9/BxytLg1vI44UTLofbFsHYR2Hz186NYnP/CJ37wYVPwz0FMOFlGDjOEr8xJqwOe+QvIl5gFXA2UIwzHONlqrrCr0wfIBX4OTBTVV/3m7dHVYPuHzkSjvwBqmp8nPXEx6QmxPKfW0fg8TSxHr5yt9M7aNYpkHxUywRpjDGuljjyHwIUqmqRqlYB04B6/Qer6lpV/QrwNSnaCBYX4+HOs/qz/LtdzFq+qekriE+BY75vid8YE5GCSf49gA1+74vdacFKEJF8EZknIhc2VEBEJrpl8ktLg6xmaQUXntiDvplJPDF7FbW+yLo2YowxzdEa3Tv0dk9FLgf+KCJ9Awuo6hRVzVPVvMzMzFYIKThej/Czs/tTsGUP/1n6XbjDMcaYkAkm+ZcAPf3eZ7nTgqKqJe5zEfAxcGIT4gu78wZ1Y0C3VJ58fxXVtVFTq2WMaeeCSf4LgVwRyRaROOBSIKhWOyKSLiLx7usM4FRgxaGXiiwej3D32f1Zt7WCNxYVhzscY4wJicMmf1WtAW4FZgHfADNUdbmIPCQi4wBE5BQRKQYuAZ4RkeXu4gOAfBFZCnwEPOrfSqitOGvAURzfM40/fVBAZU1tuMMxxphms5u8gjS3oIwrn5vPgxcM5JpTs8MdjjHG1NMiN3kZOLVfZ4Zmd+IvH61mb5Ud/Rtj2jZL/kESEe4eczRleyp56Yu14Q7HGGOaxZJ/EwzJ7sSo/plM/mQ1u/dVhzscY4w5Ypb8m+jnY/qzvaKaqXPXhjsUY4w5Ypb8m+i4rDTGDOzCs58WsaOiKtzhGGPMEbHkfwTuGtOfPVU1TJlTFO5QjDHmiFjyPwLHdE3lguO68/xnayndXRnucIwxpsks+R+hO0fnUllTy9Mfrw53KMYY02SW/I9QTmYyF52Uxd/nr2Pjzr3hDscYY5rEkn8z3H5WLqrKnz8sDHcoxhjTJJb8m6Fnp0QuPaUXMxZuYP3WinCHY4wxQbPk30y3ntkPr0d46oOCcIdijDFBs+TfTF1SE7hqeG/+9WUxhVv2hDscY4wJiiX/ELjxtL4kxHp58v1V4Q7FGGOCYsk/BDonx/OTU7N5+6uNrPhuV7jDMcaYwwoq+YvIWBFZKSKFInJvA/NHichiEakRkYsD5l0tIgXu4+pQBR5pbhiVQ2pCDE/MtqN/Y0zkO2zyFxEvMAk4FxgIXCYiAwOKrQeuAV4NWLYT8GtgKDAE+LWIpDc/7MjTsUMsE0fl8P43m/ly/fZwh2OMMYcUzJH/EKBQVYtUtQqYBoz3L6Cqa1X1KyBwhPNzgNmquk1VtwOzgbEhiDsiXXNqNp2S4uzo3xgT8YJJ/j2ADX7vi91pwWjOsm1OcnwMN5/el08LyphXtDXc4RhjTKMi4oKviEwUkXwRyS8tLQ13OM1y5bDeHJUSzxPvrSLSxkc2xpg6wST/EqCn3/ssd1owglpWVaeoap6q5mVmZga56siUEOvltjP7sWDtNuYUlIU7HGOMaVAwyX8hkCsi2SISB1wKzAxy/bOAMSKS7l7oHeNOi2oTTulFj7QO/OG9lXb0b4yJSIdN/qpaA9yKk7S/AWao6nIReUhExgGIyCkiUgxcAjwjIsvdZbcBv8XZgSwEHnKnRbW4GA93js7lq+KdXDV1Aeu2loc7JGOMqUci7cg0Ly9P8/Pzwx1Gs6kqL89bx2PvrqS61sedo/tz/chsYr0RcZnFGBNlRGSRquYFW94yUQsREa4a3of37zqN04/O5PfvfssFf57Lkg07wh2aMcZY8m9pXTsm8MyP85h85clsr6jiB3/9jAdnLmdPZU24QzPGtGOW/FvJ2EFdef+u07hqWG9e/GItZz/xCbNXbA53WMaYdsqSfytKSYjlN+MH8cZN3yM1IZYbXsrnxpcXsWnnvnCHZoxpZyz5h8FJvdJ56/YR/M/Yo/lo5RbOfuITXv5iLT5fZF18N8ZEL0v+YRLr9XDz6f2YdecojuvZkQf+vZyLJ3/Oyk27wx2aMaYdsOQfZn0ykvj7dUP5wyXHs6asnO//6VMen/Ut+6prwx2aMSaKWfKPACLCRSdn8cHdpzPuhO5M+mg1Y/84h88LrXsIY0zLsOQfQTolxfHEj07g79cNRYHLn53Pz19byvbyqnCHZoyJMpb8I9CI3Axm3TmKm0/vy5tfljD6iU/4umRnuMMyxkQRS/4RKiHWy/+MPYb/3DaChFgvl/9tHsuKbQdgjAkNS/4RbkC3VKZNHEZKQixXPDuPpdY9hDEmBCz5twE9OyUy/afD6JgYy5XPzbcxgo0xzWbJv43ISk9k2sThpCfGcdVzC1hsOwBjTDNY8m9DeqR1YNrEYXRKdnYAi9ZF/dAIxpgWYsm/jeme1oHpE4eTmRLPVc8tIH+t7QCMMU0XVPIXkbEislJECkXk3gbmx4vIdHf+fBHp407vIyJ7RWSJ+5gc4vjbpa4dE5g2cRhdUhO4auoCFqyxHYAxpmkOm/xFxAtMAs4FBgKXicjAgGLXAdtVtR/wJPB7v3mrVfUE93FjiOJu97qkOjuArh0TuOb5Bcwr2hrukIwxbUgwR/5DgEJVLVLVKmAaMD6gzHjgRff168BZIiKhC9M05Ch3B9CtYwLXPr+QL1bbDsAYE5xgkn8PYIPf+2J3WoNl3AHfdwKd3XnZIvKliHwiIiMb+gARmSgi+SKSX1pa2qQv0N4dlZLAtInDyUrvwLUvLLD+gIwxQWnpC74bgV6qeiJwF/CqiKQGFlLVKaqap6p5mZmZLRxS9MlMiecfE4fRq1Mi176wkLkFtgMwxhxaMMm/BOjp9z7LndZgGRGJAToCW1W1UlW3AqjqImA10L+5QZuDZSTH848bhpGdkcR1Ly5kzio7gzLGNC6Y5L8QyBWRbBGJAy4FZgaUmQlc7b6+GPhQVVVEMt0LxohIDpALFIUmdBOoc3I8r7o7gOtfyufjlVvCHZIxJkIdNvm7dfi3ArOAb4AZqrpcRB4SkXFuseeAziJSiFO9U9ccdBTwlYgswbkQfKOqWrvEFtQpKY5/3DCMfpnJTHxpER99azsAY8zBRDWyxo3Ny8vT/Pz8cIfR5u2oqOKKZ+dTsHkPT195EmcN6BLukIwxLUhEFqlqXrDl7Q7fKJWWGMcr1w/l6K4p3Pj3Rby/YnO4QzLGRBBL/lEsLTGOv183lIHdUrnplUXMWr4p3CEZYyKEJf8o1zExlpeuG8rA7h356cuLuOb5BXyxeiuRVt1njGldlvzbgY4dYnnl+qH8fEx/lhXv5LK/zePCSZ/xzrKN1PpsJ2BMe2QXfNuZfdW1vLG4mL/NKWLt1gp6d07khpE5XHxyFgmx3nCHZ4w5Qk294GvJv52q9SnvLd/E5E9Ws7R4J52T4rjme3348fDepCXGhTs8Y0wTWfI3TaKqzF+zjWc+Wc1HK0tJjPMy4ZSeXDcim6z0xHCHZ4wJkiV/c8RWbtrNlDlF/HtJCQpccFw3Jo7qy8DuB3XHZIyJMJb8TbN9t2Mvz3+2hlfnr6e8qpZR/TO5cVQOw/t2xnrqNiYyWfI3IbNzbzWvzF/H1LlrKdtTyeAeHfnpaTmMPbYrMV5rKGZMJLHkb0JuX3Utb35ZwpQ5RRSVldMlNZ6RuZmM6JfB9/p15qiUhHCHaEy7Z8nftBifT5n9zWZmLv2OzwvL2F5RDcAxXVM4tV8GI/plMCS7E0nxMWGO1Jj2x5K/aRU+n7Ji4y4+LSjjs8IyFqzdRlWNj1ivcGKvdEb0y2BEbgbH9ehoVUTGtAJL/iYs9lXXkr92O3MLnZ3B19/tRBVSEmIYntOZEbkZnNovg5yMJLtobEwLaGryt/NzExIJsV5G5DpH+wDbyqv4YvVW5haWMbewlPfcXkW7d0xwqohyMzg+K43OyXEkx8fYDsGYVhbUkb+IjAWeArzAs6r6aMD8eOAl4GRgKzBBVde6834JXAfUArer6qxDfZYd+Uen9Vsr9u8IPivcys691fvnxXk9pCfF0ikpns5JcXQKeHROiiPdfe6UFEdaYhxej+0sjPEX8iN/dxjGScDZQDGwUERmquoKv2LXAdtVtZ+IXAr8HpggIgNxhn08FugOvC8i/VW1NvivZKJBr86JXN65F5cP7UWtT1n+3U5Wbd7D9vIqtpZXsa28km3u6w3bK9hWXsXufTUNrksE0hPjSE+MpXNSPCkJMcTHeojzeoiP8R54Heu8j4vxEB8T+Nrjvj4wLdbrwSMgIngEPCJ4RBABr0fc9w3M94DXr6wq1Ph81PqUGp8eeK7V/dNrVamp1YAyvnplPR7wejx4RfB6DjxiPPXfez2yv0yMV+qV97g7SfWBTxXFefapourE2uh7nDvAfe70um0vONug3uv985zpAB7PwdPFb7t5xCnjCdie+1+38R28qlJZ46Oq1kdltfNcVeM8KmtqD7z2m5+aEMPpRx/VKvEFU+0zBChU1SIAEZkGjAf8k/944EH39evAX8Q5jx8PTFPVSmCNO8zjEOCL0IRv2iKvRzguK43jstIOWa6qxsf2iiq27qliW3kV2yqq2LbnwE6i7nnTrn3OP5nfP1Wl+7BeS9s2/x1B4A653jMHdkged2dTt6MG8HjqT/PfcflrqCbkoCkN/KRqfOok89oDv8Pq2qb/9o7vmRZRyb8HsMHvfTEwtLEyqlojIjuBzu70eQHL9gj8ABGZCEwE6NWrV7CxmygXF+OhS2oCXVKP/D6Cmtr6R16V1c4/Zt3Ooe4ftW5H4dt/pOu+9kGt6v4j4Hrzff7TnNe1Pq13dO48ew689zrJy/99vfl+R/N166v3UKXW56Om1vnMmsD57pmDzz2z8LnJzD9R+ifA/e/xe+8JeO93RqO4Zwg426DeNP/pAIrfGYRTzue+qftudWcctX5nHz73e9Zt51pf/b9JrXsWAxxYxu9MZf97xVkOrRdL3brqYnJ2HX4aOOEInBS40/CK83uNi/EQ5z1w9hl4prl/vv/7GKdsQqynVZtJR8QFX1WdAkwBp84/zOGYKBLj9RDj9WAdlRpTXzANsEuAnn7vs9xpDZYRkRigI86F32CWNcYY08qCSf4LgVwRyRaROJwLuDMDyswErnZfXwx8qE7l2UzgUhGJF5FsIBdYEJrQjTHGHKnDVvu4dfi3ArNwmnpOVdXlIvIQkK+qM4HngJfdC7rbcHYQuOVm4FwcrgFusZY+xhgTfnaHrzHGRIGmtvO3TleMMaYdsuRvjDHtkCV/Y4xphyz5G2NMOxRxF3xFpBRY14xVZABlIQqnNbS1eMFibi1tLea2Fi9EV8y9VTUz2JVEXPJvLhHJb8oV73Bra/GCxdxa2lrMbS1eaN8xW7WPMca0Q5b8jTGmHYrG5D8l3AE0UVuLFyzm1tLWYm5r8UI7jjnq6vyNMcYcXjQe+RtjjDkMS/7GGNMOtcnkLyJjRWSliBSKyL0NzI8Xkenu/Pki0icMYfrH01NEPhKRFSKyXETuaKDM6SKyU0SWuI9fhSPWgJjWisgyN56DetsTx5/c7fyViJwUjjj94jnab/stEZFdInJnQJmwb2cRmSoiW0Tka79pnURktogUuM/pjSx7tVumQESubqhMK8X7uIh86/7d/yUiaY0se8jfUCvH/KCIlPj97c9rZNlD5pdWjnm6X7xrRWRJI8s2fTurO0RdW3ngdCu9GsgB4oClwMCAMjcDk93XlwLTwxxzN+Ak93UKsKqBmE8H3gr39g2IaS2QcYj55wH/xRnlbhgwP9wxB/xONuHc+BJR2xkYBZwEfO037THgXvf1vcDvG1iuE1DkPqe7r9PDFO8YIMZ9/fuG4g3mN9TKMT8I/DyI380h80trxhww/w/Ar0K1ndvikf/+AeVVtQqoG1De33jgRff168BZEjjoZitS1Y2quth9vRv4hgbGMm6DxgMvqWMekCYi3cIdlOssYLWqNudu8RahqnNwxr3w5/+bfRG4sIFFzwFmq+o2Vd0OzAbGtlScdRqKV1XfU9Ua9+08nFH6IkYj2zgYweSXFnGomN389SPgH6H6vLaY/BsaUD4wkdYbUB6oG1A+7NwqqBOB+Q3MHi4iS0XkvyJybOtG1iAF3hORRSIysYH5wfwtwuVSGv9HibTtDNBFVTe6rzcBXRooE6nb+yc4Z4ANOdxvqLXd6lZVTW2kai1St/FIYLOqFjQyv8nbuS0m/zZLRJKBN4A7VXVXwOzFOFUUxwN/Bt5s5fAaMkJVTwLOBW4RkVHhDigY4gw3Og54rYHZkbid61HnPL5NtMEWkftwRul7pZEikfQbehroC5wAbMSpRmkrLuPQR/1N3s5tMfk3Z0D5sBGRWJzE/4qq/jNwvqruUtU97ut3gFgRyWjlMANjKnGftwD/wjkl9hfM3yIczgUWq+rmwBmRuJ1dm+uqzNznLQ2UiajtLSLXAOcDV7g7rIME8RtqNaq6WVVrVdUH/K2RWCJqG8P+HPZDYHpjZY5kO7fF5N+cAeXDwq2vew74RlWfaKRM17rrEiIyBOdvE7YdlogkiUhK3WucC3xfBxSbCVzltvoZBuz0q7oIp0aPkiJtO/vx/81eDfy7gTKzgDEiku5WWYxxp7U6ERkL/A8wTlUrGikTzG+o1QRcj/pBI7EEk19a22jgW1UtbmjmEW/n1riK3QJXxc/DaTGzGrjPnfYQzg8RIAHnlL8QWADkhDneETin8V8BS9zHecCNwI1umVuB5TitC+YB3wtzzDluLEvduOq2s3/MAkxy/w7LgLwI+G0k4STzjn7TImo74+yYNgLVOHXK1+Fck/oAKADeBzq5ZfOAZ/2W/Yn7uy4Erg1jvIU4deN1v+e61nXdgXcO9RsKY8wvu7/Tr3ASerfAmN33B+WXcMXsTn+h7vfrV7bZ29m6dzDGmHaoLVb7GGOMaSZL/sYY0w5Z8jfGmHbIkr8xxrRDlvyNMaYdsuRvjDHtkCV/Y4xph/4/l0l4iocNlagAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAD4CAYAAAAD6PrjAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAztElEQVR4nO3deXwU9f348dc7BwlXIJBwhnAoyh2RFFBaQSkWrbciUOuBVmtbrUcvtK1Sq7Xfb9tfq9ZasUXFqqj49SjFegHFFrAG5L5BcgEhJBCOEJLsvn9/zOxmEzbJkmyySfb9fDz2kd2Zz8y+d3Yz75nP5zOfEVXFGGOMAYiJdADGGGNaDksKxhhj/CwpGGOM8bOkYIwxxs+SgjHGGL+4SAdwOlJSUnTAgAGRDsMYY1qV1atXH1TV1FDKtqqkMGDAALKysiIdhjHGtCoikh1qWas+MsYY42dJwRhjjJ8lBWOMMX6WFIwxxvhZUjDGGOMXUlIQkXkickBENtYyX0TkSRHZKSLrReTcgHk3i8gO93FzwPQxIrLBXeZJEZHGfxxjjDGNEeqZwgvA1DrmXwIMdh93AM8AiEg34GFgHDAWeFhEkt1lngFuD1iurvUbY4xpBiFdp6Cqy0VkQB1FrgTmqzMO9yoR6SoivYFJwIeqWgwgIh8CU0VkGZCkqqvc6fOBq4D3Gvg5jAlKVSn3eCmv9HLS96jwVHte7vFyssI333NKmUqPN9Ifo0mICLExziMu8G9sTPXXMUJcTEzA/OrTY8Jwjq+A16t4vEpltb/eqteeWqZXmx++7yo+NoaE+BgS4mJJiKvxPC6WhPgY2tUsExdDQrzzPC5GEBFUnRhr/q7Kq/3eqp7XNv3WCQNJ7tgubJ+vNuG6eK0vkBvwOs+dVtf0vCDTTyEid+CcfZCenh6mcE1LVFJaweZ9R9h98BgnygN2zkF22uXBdvCVnqDTw6EtVm621VuphOO7Cse2iREnsVR4vHgbuT4RuPKcPq0qKTQZVZ0LzAXIzMxsoz/j6OL1KtnFpWzZd4TNe4+wZZ/z2FtSFrR8O98RWJAjtnZxMXROjCPFPXILLNcuLobEgCO3uo/wqi9f9Z4xxMW2zf4YqlVH2l73aNbjCTxS9wY5Eq8+vdKjKOH5t4x1z1ziYoXYmFrOVmKlzrOYmHCctkC1o3v/kfspZ5M1DkSCHfV7vM5vLchvzPcbDTa96jda/ayjOYQrKeQD/QJep7nT8nGqkAKnL3OnpwUpb9qYYycr2bb/CJv3HfXv/LftP0ppuQeA2BhhUEpHMgd0Y2jvJIb27sxZPTvTKTHO+aeJjWm2f4ZoI+LsgONiIx1JyyMixMcK8bExkBDpaJpXuJLCu8BdIrIAp1G5RFX3icj7wK8CGpcvBh5Q1WIROSIi44FPgZuAp8IUi4mQvYdPsDG/hC2+BLD/CNlFpf75SYlxDO2dxPWZ/RjWO4mhvZMY3LMTifG2VzKmpQgpKYjIqzhH/CkikofToygeQFX/DCwGLgV2AqXALHdesYj8EvjMXdUjvkZn4Ls4vZra4zQwWyNzK1Th8fLBpgJeWrWHVbudr1YE+nfrwLDeSVx7bhpDeycxrE8Sfbok2lG/MS2caCtqbcrMzFQbJbVl2Fdyglf/m8uC/+Zw4OhJ0pLbM3NsOuMHdWdIr850TGjxzVXGRA0RWa2qmaGUtf9cEzKvV1mxq4iXVu3hoy0H8Koy6axUfn1efyae1YPYMDXyGWMix5KCqVdJaQUL1+Tx8qpsdh88TnKHeG7/yiBuGJdOv24dIh2eMSaMLCmYWm3IK+Fvq7J5Z10+ZRVezk3vyu+nZ3DJiN7WOGxMG2VJwVRTVuFh0fp9vLQqm3W5h2kfH8vVo/tyw7j+jOjbJdLhGWOamCUFA0B20XFe/jSH17NyOVxawRmpHZlz+TCuGZNGUmJ8pMMzxjQTSwpRruBIGQ/+3wY+3nqA2Bjha8N78s3x/TlvUHfrPmpMFLKkEMXW5BzizpdWc+xkJfdMHsw3xqXTMykx0mEZYyLIkkKUej0rl5+9tZGeXRKYf9v5DOmVFOmQjDEtgCWFKFPp8fLY4i08/589TDizO3+ceW6zjLxojGkdLClEkUPHy/neK2tYsauIWycM5MFLh7TZEUCNMQ1jSSFKbN1/hNvnZ1FQcpLfXDeKaZn96l/IGBN1LClEgfc27OMHb6yjU0Icr317PKPTk+tfyBgTlSwptGFer/KHj7bz5JKdnNOvK8/eOMZ6Fxlj6mRJoY06WlbBfa+t46MtBUwbk8YvrxphQ1MYY+plSaEN2nPwOLfPz2L3wePMuXwYN58/wC5EM8aExJJCG7N8eyF3vbKGmBjhpVvHcv6ZKZEOyRjTilhSaCNUlb988gWPv7eFs3p25rmbMm1Ya2PMaQupk7qITBWRbSKyU0RmB5nfX0Q+FpH1IrJMRNLc6ReKyNqAR5mIXOXOe0FEvgiYd044P1g0KavwcP/r63hs8Ra+NrwXb37nfEsIxpgGqfdMQURigaeBKUAe8JmIvKuqmwOK/RaYr6ovishFwOPAjaq6FDjHXU83nHs4fxCw3I9UdWFYPkmU2ldygm+/tJr1eSXcP+Us7rrwTGLsDmjGmAYKpfpoLLBTVXcDiMgC4EogMCkMA+53ny8F3g6ynuuA91S1tMHRmmqy9hRz59/WcKK8krk3juHi4b0iHZIxppULpfqoL5Ab8DrPnRZoHXCN+/xqoLOIdK9RZgbwao1pj7lVTr8XkYRgby4id4hIlohkFRYWhhBu27dl3xG+9/Iapj27kk4Jsbz1vQmWEIwxYRGuhuYfAn8UkVuA5UA+4PHNFJHewEjg/YBlHgD2A+2AucBPgEdqrlhV57rzyczM1DDF2yptzC/hyY938MHmAjolxPGdiWfw7Yln0KW93QTHGBMeoSSFfCBwoJw0d5qfqu7FPVMQkU7Atap6OKDI9cBbqloRsMw+9+lJEXkeJ7GYID7POcRTS3ayZOsBOifG8f3Jg7l1wgC6drDRTY0x4RVKUvgMGCwiA3GSwQzgG4EFRCQFKFZVL84ZwLwa65jpTg9cpreq7hPnqqqrgI0N+gRtWNaeYp74eAef7DhI1w7x/GDKWdw8YYDdHtMY02TqTQqqWikid+FU/cQC81R1k4g8AmSp6rvAJOBxEVGc6qPv+ZYXkQE4Zxr/qrHql0UkFRBgLXBnoz9NG6CqrNpdzJMf72Dl7iK6d2zH7EuG8M3x/emUYJeVGGOalqi2nmr6zMxMzcrKinQYTUJV+ffOgzz58Q4+23OI1M4JfPuCQXxjXDod2lkyMMY0nIisVtXMUMra3ibCVJVl2wp54uMdrM09TO8uifziiuFM/1I/G8DOGNPsLClEiNerfLilgD8u2cmG/BL6dm3PY1eP4LoxaSTEWTIwxkSGJYUIWLbtAL9+bytb9x+lf/cO/O+1o7j63L7E260xjTERZkmhme0uPMbt87NIS+7A76dncPmoPnafZGNMi2FJoRmpKnP+vpnEuFhe+/Z4enS2u6AZY1oWO0RtRh9uLmD59kLunXKWJQRjTItkSaGZlFV4eGTRZs7q2Ymbzusf6XCMMSYoqz5qJn/+1y7yDp3gldvHWYOyMabFsr1TM8gtLuWZZbu4bFRvzj/Dbo9pjGm5LCk0g18u2kyMCD/9+tBIh2KMMXWypNDElm07wAebC7h78pn07tI+0uEYY0ydLCk0oZOVHn7x980MTOnIbV8eGOlwjDGmXpYUmtC8f+/hi4PHefjyYTZ0hTGmVbCk0ET2l5Tx1JIdTBnWk0ln94h0OMYYExJLCk3kV4u3UOlVHrpsWKRDMcaYkFlSaAKrdhfx7rq9fGfiGfTr1iHS4RhjTMjs4rUwq/R4efidTaQlt+c7k86IdDiti6cSdi+DnBUwaBL0/zLE2HFLi+OphKKdULARCjZBl74w/Bro0C1yManCvrWwdTH0GAJnXwrxrby3X2mxs30LNsL+jTD1cUhMavK3DSkpiMhU4Amc23H+RVV/XWN+f5z7MqcCxcA3VTXPnecBNrhFc1T1Cnf6QGAB0B1YDdyoquWN/kQR9tKqbLYVHOXZG8fYTXJCtX8DrFsA61+H4wecaZ/8DpLSYNT1kDEDUs+ObIzR6sQhZ4fk2zEVbIADW8Fz0pkvMaBeeG82nD0VRs2AwRdDXLvmia8kHza87vx+CrdWTU9IgmFXQsZMSD+vZR9ceD1QvNv5P/Bv541wJL+qTIcUOP+uZkkK9d6OU0Rige3AFCAP+AyYqaqbA8q8ASxS1RdF5CJglqre6M47pqqdgqz3deD/VHWBiPwZWKeqz9QVS0u/HWfh0ZNc9NtljO6fzIuzvoSIRDqkluvIPtjwBqx/zfkHiImHs77mJIABX4GdHzn/6LuWgHqgz7nOvBHXQke7KjzsvB4o/sLZ6QcmgSN5VWU6pECvEdDTffQaASlnQ+EWWPeas3M+XgjtuznfU8ZM6HsuhPv/4OQx2PJ3WPcqfLEcUOg3HjKmw7Crqg4yNr8DFceha7qTrDJmQPcIn72XlThH/75tXLARDmyBilJnvsRCyllV27nXCOg5Ejr3bNTbns7tOENJCucBc1T1a+7rBwBU9fGAMpuAqaqaK86esERVk9x5pyQFt0wh0EtVK2u+R21aelL40RvreHttPv+89wLOSD0lD7ZMngrI/a9z6t99MMQ2YY1ieSls/Yfzz7x7qXOE2TezamcfrPrhaAFsXOgss38DxMQ5R6IZM+CsqRCX0HTxVpbDwe1QkutUTzRWpx7Qd0z4d5INUbAZsv/jHp1uggOba+yYBlffKfUaAZ161h27p9JJ4utehW2LobIMup/pfFejpjs754byepwEsG4BbHnXibVrfyfxjLo++M6+/DhsWeT+3pYBCmlfcuJp6uourxcO7zn1LOtwTlWZ9snuNh7pJtrhkDoE4sM/gnK4k8J1ODv8b7mvbwTGqepdAWVeAT5V1SdE5BrgTSBFVYtEpBJYC1QCv1bVt0UkBVilqme6y/cD3lPVEXXF0pKTwpqcQ1zzpxXcOfEMZl8yJNLh1M1X/7puAWxYCKUHnemxCU59rG8n4PuhNuafx+uF7H9XHbmVH4Mu/ZydRMYMZ+cTqoJNVdVMx/ZDYhfnnztjJvQb27id7fGDNU7fNznVEd6Khq8zmK79q3aSzX3UerTAOTtbt8DZQQEkdq3aKfXy7ZiGNn7HVFbifN/rFjjJB5wzwFHTnWqdUKtBDmxxdurrX4ej+yChC4y42jnyTx8f+nd+ZK/z2de+6pzZxMQ71V0ZM+HMKY2r7jp5zEmqgb+fA5ud3zo4VWzdz3S2bWASSOrTbAcIkUgKfYA/AgOB5cC1wAhVPSwifVU1X0QGAUuAyUAJISYFEbkDuAMgPT19THZ2diifq1l5vMpVT/+HA0fLWPKDSXRMaKHt9zXrX2PbOQ1yI66BirLqVQfHC6uWS+obsNNwf9TdBkFMHW0mhdthvbsDL8mFdp1huK+O9/zG1fF6Pc6R37oFTjVC5QlIHlh11NitjqvHPZVQtMM9hQ/4Jz62v6pMp141PutA5+i5sQq3OjH7jlr7jXOPWq92jhqbQnmpc9S+bgHs+tg9Oxvj7FTPvgS6pDX9junQHud3sG4BFO+CuEQYcpnzfQ2adOrZ6bEDzsHK+gWwb52z7QdPcc8OL2lcwlKF/esbVt2l6hzp+xrYfb+f4i8Adz+a0MXZ+QdW/6QOhXaR7YXY7NVHNcp3AraqalqQeS8Ai3DOJNpM9dErn+bw4FsbeHLmaK7I6FP/Ar5t3hxHCbXWv86A4VfVvjM6WnBq/fLB7U79PkB8B+gxtHr9clJf2P6+81571zhHSGdMdt7r7Eub5h/j5FHY/K6zA/niE+fzpZ/nvOcZk50dUm2NpDHxzul64FlRr5FN32ZRkl91xF64xUnOZ/mOWr/a+EZar9fpwbXuVdj0DpQfdRrtM6Y7ySD1rPB8jtOlCnlZTlwb34Syw9Cxh5PIR17n7FzXLXDak9QDvc9xtsmIa6FTavjjCVrdNdjZTsOvceKrVv2zCU6WuAuLc7BQs/qna3rLqB6sIdxJIQ6noXkykI/T0PwNVd0UUCYFKFZVr4g8BnhU9SERSQZKVfWkW2YlcKWqbnYbp98MaGher6p/qiuWlpgUDh0v58LfLePsnp1ZcMf4+huXj+6Hv13rnM4G1tf2HOH0sAlHHbnXA1/8yzka8tW/Jg8IOJIe1LD1VpTBwW0B/yjukdKJQ9XL9Rzp7JRHXgedezX644TscG7VmdDB7dXnBTaS+v6JU85qvl4ywfiPWhc4SeJ4IXToDiOuc3ZMfU6zkfbgTic5rnsNSnKgXSen4TVjBvSf0LJ64FSehB0fOJ99+/tV1XSd+1T1OOvRjKMKB6vu8mnXqarqx3fg0GMYJLSSdkPCnBTcFV4K/AGnS+o8VX1MRB4BslT1XbeK6XGcc6jlwPfcRHA+8CzgxblQ7g+q+ld3nYNwuqR2Az7H6cZ6sq44WmJS+NnbG3j1v7n84/tfZkiveupJy0rg+a873c+GX+XUlx7Y7ByhgNOImnJ2jdPPkU4DZSgKNldV2RzdF1DnPsOpqmiKIxhVJ8H5TqMHTHBijiRV2Ps55H3m1NuHofdGk/NUuEetC5zGeM9JJ2llzICR10PXfsGXKy12jrrXLYD8LPfs7CLnjGDI1yNebRGS0mLnSD2pLwy8oO5qyeZwaA/s+NBpWO81AroOaFkJtQHCnhRaipaWFDbml3D5H//NLecP4OHLh9dduKLMOUPI/RS+8RqcOdmZ7vVA0a5Tq2qO7q1atmOPIEe5gyE2vqr+dd2rzlFnTJzTcObrndMEPRlMEztxuOqoNWcFIDDgy86Z3rArnOqmmkfZPUe4CWRa856dmVbBkkIz8HqV6/68gpziUj7+wSS6tI+vo7AHXr8Jti6Ca//qVKvUp7S4+oUs+zc4DZUe9/q+2HZO42rRTrcf/+iq+lfrx992+BtpX3XOMOPaO1WMZYedI9mR05xkEOmzM9OinU5SaKHdZFq+tz7PZ03OYX5z3ai6E4IqLLrPSQhT/ye0hABON9CBFzgPH08FHNxRddFL4TanB0nGTKcrqWl7kgfAxB/DBT9yqsPWv+b0vx9xXfCeO8Y0kv2iGuBIWQWPv7eV0eldufbcUzpZVbf0MVjzInzlBzD+zsa9cWw89BzmPLi+cesyrYuIcy1Gv7GRjsS0cZYUGuCJj3ZQdPwkz9/yJWJi6mi8/fRZWP4bGH0jXPTz5gvQGGMaqHU3qUfA9oKjvLBiDzPHpjMyrUvtBTe+Ce/9BM7+Olz2hxbZd9kYY2qypHAaVJWH39lE58Q4fnRxHaN27loC//dt5yKq6/5q9b7GmFbDksJp+MeGfazcXcQPLz6b5I61XPSUvwYWfNPpYz7z1dY/prsxJqpYUjgNz/5rN0N6dWbm2FpGezy4E16+Djp2h2++Ce27Nmt8xhjTWJYUQqSq7C48xvhB3YkN1rh8ZB+8dLXz/JtvQVLv5g3QGGPCwCq7Q1R0vJzj5R76dw8ybMCJw87VyieK4ea/Q8qZzR6fMcaEgyWFEOUUOzcgSe9WIylUnIBXZzgDsN3whjP0rjHGtFKWFEKUU+QkhWpnCp5KWHgr5KxyehmdcWGEojPGmPCwpBCibDcppCW7SUEVFt3rjO54yW+cMYeMMaaVs4bmEOUUl9IrKZHEeHdY3yW/hM9fcsakGXdHZIMzxpgwsaQQopzi46T7qo5W/Rk++R2MuQUu/GlE4zLGmHCypBCinOJS+nfr4Ny74J8/ce4x+/X/Z8NXGGPaFEsKISir8FBw5CQTZD28dadza8Nr/xr5O0QZY0yYWVIIQU5xKe2oYOrOXzh3PJvxit3RzBjTJoWUFERkqohsE5GdIjI7yPz+IvKxiKwXkWUikuZOP0dEVorIJnfe9IBlXhCRL0Rkrfs4J2yfKsxyikq5MvY/JJYVwsWP2vAVxpg2q96kICKxwNPAJcAwYKaIDKtR7LfAfFUdBTwCPO5OLwVuUtXhwFTgDyLSNWC5H6nqOe5jbaM+SRPKLjrO7bH/oDJ1uHNTdGOMaaNCOVMYC+xU1d2qWg4sAK6sUWYYsMR9vtQ3X1W3q+oO9/le4ACQGo7Am1PCniWcFZNP7IS7rWHZGNOmhZIU+gK5Aa/z3GmB1gHXuM+vBjqLSPfAAiIyFmgH7AqY/JhbrfR7EUkI9uYicoeIZIlIVmFhYQjhht+YvJc4KN2RUO+vbIwxrVS4Gpp/CEwUkc+BiUA+4PHNFJHewEvALFX1upMfAIYAXwK6AT8JtmJVnauqmaqamZoagZOMvZ8z9ORa/tXtOuceycYY04aFMsxFPtAv4HWaO83PrRq6BkBEOgHXquph93US8A/gp6q6KmCZfe7TkyLyPE5iaXH0P09xTNuzZ8D1kQ7FGGOaXChnCp8Bg0VkoIi0A2YA7wYWEJEUEfGt6wFgnju9HfAWTiP0whrL9Hb/CnAVsLERn6NpHMqGzW/ziucievVodU0hxhhz2upNCqpaCdwFvA9sAV5X1U0i8oiIXOEWmwRsE5HtQE/gMXf69cAFwC1Bup6+LCIbgA1ACvBomD5T+Kx6BkV4vnIq/bt1jHQ0xhjT5EIaJVVVFwOLa0x7KOD5QmBhkOX+BvytlnW27L6dJw7Bmvnk9LmE/bu6n3ofBWOMaYPsiubaZM2DiuMs6z6d2BihT1e7gtkY0/ZZUgim8iR8+iyccRGrT6bRt2t74mJtUxlj2j7b0wWz/nU4VgDnf5+couPB78tsjDFtkCWFmrxeWPEU9BoJgyaRXVxq7QnGmKhhSaGmnR/CwW1w/vcpKavkcGmFJQVjTNSwpFDTiqcgqS8Mv5rcYue+zFZ9ZIyJFpYUAuWvgT2fwPjvQGw8OW5S6GdnCsaYKGFJIdCKpyAhCc69GYDsIt+Zgl24ZoyJDpYUfNwhLRhzCyQmAZBTfJzuHdvRKSGka/yMMabVs6Tgs+pPIDFO1ZErp7jUqo6MMVHFkgJAaTGseQlGToOkPv7J2UWl1shsjIkqlhTAP6QF59/tn1Th8bL38AnrjmqMiSqWFPxDWkyGnsP9k/MPncCrWFIwxkQVSwrrX4PjB2DC96tNzi62nkfGmOgT3UnB64UVf3SGtBg4sdos3zUKdqZgjIkm0Z0UdnzgDmlxD4hUm5VTdJyEuBh6dE6IUHDGGNP8ojsprHgKktJg+FWnzPJ1R42JkVOXM8aYNiqkpCAiU0Vkm4jsFJHZQeb3F5GPRWS9iCwTkbSAeTeLyA73cXPA9DEissFd55PuvZqbT/5qyP43nPddiI0/ZXZ2USn9rerIGBNl6k0KIhILPA1cAgwDZorIsBrFfgvMV9VRwCPA4+6y3YCHgXHAWOBhEUl2l3kGuB0Y7D6mNvrTnI4VT0FCFzj3plNmqSo5xaWk2zUKxpgoE8qZwlhgp6ruVtVyYAFwZY0yw4Al7vOlAfO/BnyoqsWqegj4EJgqIr2BJFVdpaoKzAeuatxHOQ2H9sDmdyBzFiR0PmV20fFySss91shsjIk6oSSFvkBuwOs8d1qgdcA17vOrgc4i0r2OZfu6z+taJwAicoeIZIlIVmFhYQjhhmDln0BiYdy3g86uGgjPkoIxJrqEq6H5h8BEEfkcmAjkA55wrFhV56pqpqpmpqamNn6FpcXw+alDWgTKKT4OWHdUY0z0CWX4z3ygX8DrNHean6ruxT1TEJFOwLWqelhE8oFJNZZd5i6fVmN6tXU2may/QkVptSEtasopOoEIpCVbUjDGRJdQzhQ+AwaLyEARaQfMAN4NLCAiKSLiW9cDwDz3+fvAxSKS7DYwXwy8r6r7gCMiMt7tdXQT8E4YPk/dKsrg07lw5lehZ8228irZxcfplZRIYnxsk4dkjDEtSb1JQVUrgbtwdvBbgNdVdZOIPCIiV7jFJgHbRGQ70BN4zF22GPglTmL5DHjEnQbwXeAvwE5gF/BeuD5UrXxDWpz//TqL5dqQ2caYKBXS3WNUdTGwuMa0hwKeLwQW1rLsPKrOHAKnZwEjTifYRvF6YeUfodcoGHhBnUWzi0qZeFYY2i+MMaaViZ4rmne8Dwe3w4RTh7QIdKLcw4GjJ63nkTEmKkVPUljxFHTpB8NqXmJRXe4hpzuqVR8ZY6JRdCSFvNWQ/R8YH3xIi0BV1yjYkNnGmOgTHUlhxZPukBY31lvUhsw2xkSzkBqaW73hV8OgiUGHtKgpp+g4nRPiSO5Q9xmFMca0RVGSFK4KuWi2OxBecw/aaowxLUF0VB+dhpziUqs6MsZELUsKATxeJa/4hA2ZbYyJWpYUAhQcKaPc47UzBWNM1LKkEMDfHbWbdUc1xkQnSwoBfENm29XMxphoZUkhQE5xKXExQu8uiZEOxRhjIsKSQoDsolL6JrcnLtY2izEmOtneL4B1RzXGRDtLCgEsKRhjop0lBVfJiQoOl1ZYI7MxJqpZUnDl2kB4xhgTWlIQkakisk1EdorI7CDz00VkqYh8LiLrReRSd/oNIrI24OEVkXPcecvcdfrm9QjrJztNvmsU0u0aBWNMFKt3QDwRiQWeBqYAecBnIvKuqm4OKPYznHs3PyMiw3Bu3TlAVV8GXnbXMxJ4W1XXBix3g3tbzojLdq9RsCEujDHRLJQzhbHATlXdrarlwAKg5u3LFEhyn3cB9gZZz0x32RYpt7iU7h3b0SkhOgaONcaYYEJJCn2B3IDXee60QHOAb4pIHs5Zwt1B1jMdeLXGtOfdqqOfS4THqs4uKrWzBGNM1AtXQ/NM4AVVTQMuBV4SEf+6RWQcUKqqGwOWuUFVRwJfcR9Bb4smIneISJaIZBUWFoYp3FNZd1RjjAktKeQD/QJep7nTAt0GvA6gqiuBRCAlYP4MapwlqGq++/co8ApONdUpVHWuqmaqamZqamoI4Z6+8kovew+foL8lBWNMlAslKXwGDBaRgSLSDmcH/26NMjnAZAARGYqTFArd1zHA9QS0J4hInIikuM/jgcuAjURI/uETeBX6WVIwxkS5eltVVbVSRO4C3gdigXmquklEHgGyVPVd4AfAcyJyH06j8y2qqu4qLgByVXV3wGoTgPfdhBALfAQ8F7ZPdZpy3GsU+ne37qjGmOgWUlcbVV2M04AcOO2hgOebgQm1LLsMGF9j2nFgzGnG2mRyimzIbGOMAbuiGXDOFBLiYkjtlBDpUIwxJqIsKeB2R+3WgZiYiPaKNcaYiLOkgHVHNcYYn6hPCqrqJAVrTzDGGEsKB4+VU1rusWsUjDEGSwrk2EB4xhjjZ0mh2IbMNsYYn6hPCtlFpYhAWnL7SIdijDERF/VJIae4lF5JiSTGx0Y6FGOMiThLCkXWHdUYY3yiPilk2zUKxhjjF9VJ4US5h8KjJ23MI2OMcUV1UvD1PLIhs40xxmFJARsy2xhjfKI6KWT7hsy2MwVjjAGiPCnkFJfSOSGOrh3iIx2KMca0CFGfFNK7d0DEhsw2xhiI9qRQVGo9j4wxJkBISUFEporINhHZKSKzg8xPF5GlIvK5iKwXkUvd6QNE5ISIrHUffw5YZoyIbHDX+aQ08+G6x6vkHTphPY+MMSZAvUlBRGKBp4FLgGHATBEZVqPYz4DXVXU0MAP4U8C8Xap6jvu4M2D6M8DtwGD3MbXhH+P07T9SRrnHS38bCM8YY/xCOVMYC+xU1d2qWg4sAK6sUUaBJPd5F2BvXSsUkd5AkqquUlUF5gNXnU7gjeXreWRXMxtjTJVQkkJfIDfgdZ47LdAc4JsikgcsBu4OmDfQrVb6l4h8JWCdefWsEwARuUNEskQkq7CwMIRwQ5Prv0bBkoIxxviEq6F5JvCCqqYBlwIviUgMsA9Id6uV7gdeEZGkOtZzClWdq6qZqpqZmpoapnCdIbPjYoTeXRLDtk5jjGnt4kIokw/0C3id5k4LdBtum4CqrhSRRCBFVQ8AJ93pq0VkF3CWu3xaPetsUtnFpfRNbk9cbFR3wDLGmGpC2SN+BgwWkYEi0g6nIfndGmVygMkAIjIUSAQKRSTVbahGRAbhNCjvVtV9wBERGe/2OroJeCcsnyhEuTY6qjHGnKLepKCqlcBdwPvAFpxeRptE5BERucIt9gPgdhFZB7wK3OI2IF8ArBeRtcBC4E5VLXaX+S7wF2AnsAt4L3wfq37Zdh8FY4w5RSjVR6jqYpwG5MBpDwU83wxMCLLcm8CbtawzCxhxOsGGS0lpBSUnKqyR2RhjaojKCnXf6Kjpdo2CMcZUE5VJIbvYrlEwxphgojIp+M8UrPrIGGOqic6kUFRK947t6JQQUpOKMcZEjehMCu6Q2cYYY6qLykPl7KJSvjQgOdJhGNPqVVRUkJeXR1lZWaRDMUBiYiJpaWnExzf8xmFRlxTKK73sKzlBeregQy0ZY05DXl4enTt3ZsCAAXazqghTVYqKisjLy2PgwIENXk/UVR/lHz6BVyG9u3VHNaaxysrK6N69uyWEFkBE6N69e6PP2qIuKdiQ2caElyWEliMc30XUJQUbMtsYY2oXdUkhu6iUhLgYenROiHQoxhjT4kRfUnBHR7VTXmNMqCorKyMdQrOJut5HucWlVnVkTBP4xd83sXnvkbCuc1ifJB6+fHidZa666ipyc3MpKyvjnnvu4Y477uCf//wnDz74IB6Ph5SUFD7++GOOHTvG3XffTVZWFiLCww8/zLXXXkunTp04duwYAAsXLmTRokW88MIL3HLLLSQmJvL5558zYcIEZsyYwT333ENZWRnt27fn+eef5+yzz8bj8fCTn/yEf/7zn8TExHD77bczfPhwnnzySd5++20APvzwQ/70pz/x1ltvhXX7NIWoSgqqSk5xKeed0T3SoRhjwmTevHl069aNEydO8KUvfYkrr7yS22+/neXLlzNw4ECKi53R+n/5y1/SpUsXNmzYAMChQ4fqXXdeXh4rVqwgNjaWI0eO8MknnxAXF8dHH33Egw8+yJtvvsncuXPZs2cPa9euJS4ujuLiYpKTk/nud79LYWEhqampPP/889x6661Nuh3CJaqSQuGxk5SWe+hvPY+MCbv6juibypNPPuk/As/NzWXu3LlccMEF/r763bp1A+Cjjz5iwYIF/uWSk+u/gHXatGnExsYCUFJSws0338yOHTsQESoqKvzrvfPOO4mLi6v2fjfeeCN/+9vfmDVrFitXrmT+/Plh+sRNK6qSQlXPI7tGwZi2YNmyZXz00UesXLmSDh06MGnSJM455xy2bt0a8joC2xdr9vHv2LFqX/Hzn/+cCy+8kLfeeos9e/YwadKkOtc7a9YsLr/8chITE5k2bZo/abR0UdXQnF3kJIV+dqZgTJtQUlJCcnIyHTp0YOvWraxatYqysjKWL1/OF198AeCvPpoyZQpPP/20f1lf9VHPnj3ZsmULXq+3zjr/kpIS+vZ1RkJ44YUX/NOnTJnCs88+62+M9r1fnz596NOnD48++iizZs0K34duYiElBRGZKiLbRGSniMwOMj9dRJaKyOcisl5ELnWnTxGR1SKywf17UcAyy9x1rnUfPcL3sYLLKS5FBNKS2zf1WxljmsHUqVOprKxk6NChzJ49m/Hjx5OamsrcuXO55ppryMjIYPr06QD87Gc/49ChQ4wYMYKMjAyWLl0KwK9//Wsuu+wyzj//fHr37l3re/34xz/mgQceYPTo0dV6I33rW98iPT2dUaNGkZGRwSuvvOKfd8MNN9CvXz+GDh3aRFsg/MS5lXIdBURige3AFCAP+AyY6d6C01dmLvC5qj4jIsOAxao6QERGAwWquldERgDvq2pfd5llwA/d23KGJDMzU7OyQi5+ivtfW8vK3UWsfGByg9dhjKmyZcuWVrXDa2533XUXo0eP5rbbbmu29wz2nYjIalXNDGX5UCq5xgI7VXW3u/IFwJXA5oAyCiS5z7sAewFU9fOAMpuA9iKSoKonQwku3HzXKBhjTFMbM2YMHTt25He/+12kQzktoSSFvkBuwOs8YFyNMnOAD0TkbqAj8NUg67kWWFMjITwvIh7gTeBRDXLaIiJ3AHcApKenhxBu7XKKS7nw7NRGrcMYY0KxevXqSIfQIOFqaJ4JvKCqacClwEsi4l+3iAwH/gf4dsAyN6jqSOAr7uPGYCtW1bmqmqmqmampDd+hl5ZXUnj0pJ0pGGNMHUJJCvlAv4DXae60QLcBrwOo6kogEUgBEJE04C3gJlXd5VtAVfPdv0eBV3CqqZpMbvEJwIbMNsaYuoSSFD4DBovIQBFpB8wA3q1RJgeYDCAiQ3GSQqGIdAX+AcxW1f/4CotInIj4kkY8cBmwsZGfpU42ZLYxxtSv3qSgqpXAXcD7wBbgdVXdJCKPiMgVbrEfALeLyDrgVeAWt33gLuBM4KEaXU8TgPdFZD2wFufM47kwf7ZqcnwXrllSMMaYWoV0iZ2qLgYW15j2UMDzzcCEIMs9Cjxay2rHhB5m4+UUl9I5MY6uHRp+71JjjGnrouaK5uwiGzLbmGjXqVOnSIfQ4rWOwTjCILe4lCG9O0c6DGParvdmw/4N4V1nr5Fwya/Du84WoLKyssWOhRQVZwoer5J7qJT0btbzyJi2ZPbs2dXGM5ozZw6PPvookydP5txzz2XkyJG88847Ia3r2LFjtS43f/58/zAWN97o9J4vKCjg6quvJiMjg4yMDFasWMGePXsYMWKEf7nf/va3zJkzB4BJkyZx7733kpmZyRNPPMHf//53xo0bx+jRo/nqV79KQUGBP45Zs2YxcuRIRo0axZtvvsm8efO49957/et97rnnuO+++xq62eqmqq3mMWbMGG2I3OLj2v8ni/TlVdkNWt4YE9zmzZsj+v5r1qzRCy64wP966NChmpOToyUlJaqqWlhYqGeccYZ6vV5VVe3YsWOt66qoqAi63MaNG3Xw4MFaWFioqqpFRUWqqnr99dfr73//e1VVrays1MOHD+sXX3yhw4cP96/zN7/5jT788MOqqjpx4kT9zne+459XXFzsj+u5557T+++/X1VVf/zjH+s999xTrdzRo0d10KBBWl5erqqq5513nq5fvz7o5wj2nQBZGuJ+tmWev4SZv+eR3XHNmDZl9OjRHDhwgL1791JYWEhycjK9evXivvvuY/ny5cTExJCfn09BQQG9evWqc12qyoMPPnjKckuWLGHatGmkpKQAVfdLWLJkif8eCbGxsXTp0qXeG/f4BucD5wY+06dPZ9++fZSXl/vv/1DbfR8uuugiFi1axNChQ6moqGDkyJGnubVCEx1JwR0y265RMKbtmTZtGgsXLmT//v1Mnz6dl19+mcLCQlavXk18fDwDBgw45T4JwTR0uUBxcXF4vV7/67ruz3D33Xdz//33c8UVV7Bs2TJ/NVNtvvWtb/GrX/2KIUOGNOlQ3FHRppBdXEpcjNC7S2KkQzHGhNn06dNZsGABCxcuZNq0aZSUlNCjRw/i4+NZunQp2dnZIa2ntuUuuugi3njjDYqKioCq+yVMnjyZZ555BgCPx0NJSQk9e/bkwIEDFBUVcfLkSRYtWlTn+/nuz/Diiy/6p9d234dx48aRm5vLK6+8wsyZM0PdPKctKpJCTnEpacntiYuNio9rTFQZPnw4R48epW/fvvTu3ZsbbriBrKwsRo4cyfz58xkyZEhI66ltueHDh/PTn/6UiRMnkpGRwf333w/AE088wdKlSxk5ciRjxoxh8+bNxMfH89BDDzF27FimTJlS53vPmTOHadOmMWbMGH/VFNR+3weA66+/ngkTJoR0K9GGqvd+Ci1JQ++n8PTSnRwtq2T2JaH9OIwxobH7KTSvyy67jPvuu4/Jk2u/J0xz3E+h1fvehWdGOgRjjGmww4cPM3bsWDIyMupMCOEQFUnBGGN8NmzY4L/WwCchIYFPP/00QhHVr2vXrmzfvr1Z3suSgjGmUVS1VQ0fM3LkSNauXRvpMJpEOJoDrOXVGNNgiYmJFBUVhWVnZBpHVSkqKiIxsXG9LO1MwRjTYGlpaeTl5VFYWBjpUAxOkk5LS2vUOiwpGGMaLD4+3n8lrmkbrPrIGGOMnyUFY4wxfpYUjDHG+LWqK5pFpBAIbSCTU6UAB8MYTnOwmJtea4sXLObm0tpirive/qqaGspKWlVSaAwRyQr1Mu+WwmJueq0tXrCYm0trizlc8Vr1kTHGGD9LCsYYY/yiKSnMjXQADWAxN73WFi9YzM2ltcUclnijpk3BGGNM/aLpTMEYY0w9LCkYY4zxa3NJQUSmisg2EdkpIrODzE8Qkdfc+Z+KyIAIhBkYTz8RWSoim0Vkk4jcE6TMJBEpEZG17uOhSMQaEM8eEdngxnLKrfDE8aS7jdeLyLmRiDMgnrMDtt1aETkiIvfWKBPxbSwi80TkgIhsDJjWTUQ+FJEd7t+g92EUkZvdMjtE5OYIx/wbEdnqfvdviUjXWpat83fUzDHPEZH8gO//0lqWrXP/0ozxvhYQ6x4RWVvLsqe/jVW1zTyAWGAXMAhoB6wDhtUo813gz+7zGcBrEY65N3Cu+7wzsD1IzJOARZHevgHx7AFS6ph/KfAeIMB44NNIx1zjN7If52KeFrWNgQuAc4GNAdP+F5jtPp8N/E+Q5boBu92/ye7z5AjGfDEQ5z7/n2Axh/I7auaY5wA/DOG3U+f+pbnirTH/d8BD4drGbe1MYSywU1V3q2o5sAC4skaZK4EX3ecLgckSwTuEqOo+VV3jPj8KbAH6RiqeMLkSmK+OVUBXEekd6aBck4FdqtrQK+ObjKouB4prTA78vb4IXBVk0a8BH6pqsaoeAj4EpjZVnIGCxayqH6hqpftyFdC4sZzDrJbtHIpQ9i9hV1e87r7reuDVcL1fW0sKfYHcgNd5nLqD9Zdxf7glQPdmia4eblXWaCDYfQHPE5F1IvKeiAxv3shOocAHIrJaRO4IMj+U7yFSZlD7P1BL2sY+PVV1n/t8P9AzSJmWvL1vxTlrDKa+31Fzu8ut8ppXSzVdS9zOXwEKVHVHLfNPexu3taTQaolIJ+BN4F5VPVJj9hqc6o4M4Cng7WYOr6Yvq+q5wCXA90TkggjHExIRaQdcAbwRZHZL28anUKc+oNX0IReRnwKVwMu1FGlJv6NngDOAc4B9OFUyrcFM6j5LOO1t3NaSQj7QL+B1mjstaBkRiQO6AEXNEl0tRCQeJyG8rKr/V3O+qh5R1WPu88VAvIikNHOYgfHku38PAG/hnFYHCuV7iIRLgDWqWlBzRkvbxgEKfFVv7t8DQcq0uO0tIrcAlwE3uMnsFCH8jpqNqhaoqkdVvcBztcTSorazu/+6BnittjIN2cZtLSl8BgwWkYHuUeEM4N0aZd4FfL0zrgOW1PajbQ5uneBfgS2q+v9qKdPL1+4hImNxvreIJDIR6SginX3PcRoVN9Yo9i5wk9sLaTxQElAFEkm1HlW1pG1cQ+Dv9WbgnSBl3gcuFpFkt9rjYndaRIjIVODHwBWqWlpLmVB+R82mRpvX1bXEEsr+pTl9FdiqqnnBZjZ4Gzd1y3lzP3B6vmzH6SXwU3faIzg/UIBEnOqDncB/gUERjvfLOFUC64G17uNS4E7gTrfMXcAmnN4Oq4DzIxjvIDeOdW5Mvm0cGK8AT7vfwQYgswX8Ljri7OS7BExrUdsYJ2HtAypw6qtvw2nv+hjYAXwEdHPLZgJ/CVj2Vvc3vROYFeGYd+LUvft+z77efn2AxXX9jiIY80vub3U9zo6+d82Y3den7F8iEa87/QXf7zegbKO3sQ1zYYwxxq+tVR8ZY4xpBEsKxhhj/CwpGGOM8bOkYIwxxs+SgjHGGD9LCsYYY/wsKRhjjPH7/46p91Tz2KP6AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAD4CAYAAAAD6PrjAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAA1AElEQVR4nO3deXxU5fX48c/JZINAgEBYww6CyCoBQWRRBHFXqiJVa2tbqxZrtVqx/VYttVVba1eqP1fUWkXpIloVgYKKChIg7FtYE8ISkrBmz5zfH/dmmGxkIJNMkjnv1yuvzNz73Dtnbib3zLPc54qqYowxxgBEhDoAY4wxDYclBWOMMT6WFIwxxvhYUjDGGONjScEYY4xPZKgDOBPt2rXTHj16hDoMY4xpVFatWnVYVRMDKduokkKPHj1ISUkJdRjGGNOoiMieQMta85ExxhgfSwrGGGN8LCkYY4zxsaRgjDHGx5KCMcYYn4CSgoi8IiKHRGRDNetFRP4sImkisk5Ezvdbd7uIbHd/bvdbPlxE1rvb/FlEpPZvxxhjTG0EWlOYA0w5zfrLgb7uz53AcwAikgA8BlwAjAQeE5E27jbPAd/32+50+zfGGFMPArpOQVU/E5EepylyLfC6OvNwLxeR1iLSCZgALFTVHAARWQhMEZGlQLyqLneXvw5cB3x0lu/DmDNWUurlZGEpJ4pKOFlYwolC57fzuNS3rLC4NNShliMiREdGEO2JIDoygijfbyGm3POI6st5PEQEofFYgdJSpbjUS2GJl+JSL0WlXopLlKLSUopK1H3uLi/1UlT22LdMKSrxEiFCVKQQ7Ymo+n1UfC+V3p8QFRmB16u+1ygqObX/quIpX859XuKl2KtERkiF4yinP7aeCKIjhWiPx/c+Ij0RRASpDaR5dCSeYO3sNIJ18VoXIN3veYa77HTLM6pYXomI3IlT+6Bbt25BCtc0RKrK0fxiDp8opKDY+Sct+4f2nWzKftx/7rJ1znp113l9J6lTJ/pS57GbAI4XlFBY4g04tobUuGm3QAlPix4YT5/2Ler8dRr8Fc2q+gLwAkBycrL9OzRSRSVeDh0v4OCxAg4cLeTAsbLHBeUen8mJuirRnlPfGKM9EcTFRNIiJpK4GA+dW8cSFxN5alm0s7yF/zK3bMuYKOJiPMTFRBITGUFD6vLyepVib/lvweW+qZeU/0ZeVOlburNNsG6w5fH7Rh3j9605KrLyt+eK3/DLykR5BK9S7fsov6ysFnLqvfiX90RItbWkqr7NV1UuMkIo9apfLcNbqYZROZ6qa0XB0q5FdND2dTrBSgr7gK5+z5PcZftwmpD8ly91lydVUd40UkUlXlJ255CRm8+BY+6J3u+Ef/hEUaVtoiMj6BgfS8f4WAYntWbygBg6xMeS2DKG2CjPqRNH2Ym+wskkyiN+653nDenkXVciIoSYCA8xkZ5QhxJUHgFPhIfYqIbxviI9QqQHmkU3jHjqS7CSwnxghoi8jdOpfFRV94vIAuA3fp3Lk4FHVDVHRI6JyChgBfAt4C9BisXUE69X+Xp3Du+lZvLh+v0czS/2rUuIi6ZDfCwd42MYnNTKfRxLh1axvkTQunlUWJzEjWlMAkoKIvIWzjf+diKSgTOiKApAVZ8HPgSuANKAPOA77rocEfkVsNLd1ayyTmfgHpxRTc1wOpitk7kRUFU27T/G/NRM5q/NZP/RAppHe5g8oANXDe5Mv44taR8f0+S+xRoTLiRY7Yr1ITk5WW2W1NBIz8njvdR9vJeayfZDJ4iMEMafk8g1QzszaUAHmkc3+O4pY8KWiKxS1eRAytp/sqnW4ROF/Hfdft5L3cfqvUcAGNkjgSeuG8iVgzrRJq5+Or6MMfXHkoIp50RhCZ9sPMB7qZksSztMqVfp37ElD0/pz9VDOpHUpnmoQzTG1CFLCoaiEi+fbcviP6n7WLT5IAXFXrq0bsYPxvXi2qFd6NexZahDNMbUE0sKYe5/Ww7y03nrOXyikIS4aG4c3pVrh3ZmePc2NjLImDBkSSFMFZV4+e3HW3hp2S4GdIrndzcM5qK+7Yjy2MS5xoQzSwphaE/2Se59aw3rMo5y++juPHLFuQ3mgiFjTGhZUggz76/N5JF/rSdC4PlbhzNlYMdQh2SMaUAsKYSJ/KJSZn2wkbe+Tmd49zb86eahNpLIGFOJJYUwsO3gcWb8YzXbD53gngm9uX/SOdZ3YIypkiWFJkxVmbsyncff30iLmEhev2MkY/smhjosY0wDZkmhiTpeUMzP/r2B99dmclGfdjw7bQjtW8aGOixjTANnSaEJWpdxhHvfWkNGbj4PXdaPu8f3JqIe7thkjGn8LCk0IarKK1/s5qmPNpPYIoa5d44iuUdCqMMyxjQilhSaiNyTRTz47loWbznEpAEd+N0Ng2nd3CasM8acGUsKTcDXu3L40VtryDlZxONXD+D2C3vYFBXGmLNiSaERK/Uqs5ek8cdF2+iW0Jx/3XMhA7u0CnVYxphGzJJCI+T1Kp9tz2L2kjRW7s7luqGdeeL6QbSIsT+nMaZ27CzSiJwsLOGfqzOY8+VudmadJLFlDL+9YTA3Dk+y5iJjTFBYUmgE9mbn8fpXu5mbks7xghKGJLXij9OGcsWgTkRH2pXJxpjgsaTQQKkqX+3I5tUvd7No80E8IlwxqBPfHtOD87u1CXV4xpgmypJCA5NfVMp7qfuY8+Vuthw4TkJcNDMu7sMtF3SnYyu7ItkYU7cCSgoiMgX4E+ABXlLVpyqs7w68AiQCOcCtqpohIhcDf/Ar2h+4WVX/IyJzgPHAUXfdt1U1tRbvpVHLPJLPG8v38NbXezmSV+y78c3VQzrbvQ6MMfWmxqQgIh5gNjAJyABWish8Vd3kV+wZ4HVVfU1ELgGeBG5T1SXAUHc/CUAa8Infdg+p6rygvJNGSFVJ2ZPLnC928/HGA6gql53Xke+M6cmIHnY7TGNM/QukpjASSFPVnQAi8jZwLeCfFAYAD7iPlwD/qWI/NwAfqWreWUfbRBSWlPLB2v28+uUuNuw7RqtmUXxvbE9uG9Xd7nFgjAmpQJJCFyDd73kGcEGFMmuBqThNTNcDLUWkrapm+5W5GXi2wna/FpFHgcXATFUtrPjiInIncCdAt27dAgi3YSssKeXav37BlgPHOadDC35z/SCuG9aZ5tHWvWOMCb1gjWd8EBgvImtw+gn2AaVlK0WkEzAIWOC3zSM4fQwjgATg4ap2rKovqGqyqiYnJjb+ewG88dUethw4zh+mDWHBj8fxzQu6WUIwxjQYgZyN9gFd/Z4nuct8VDUTp6aAiLQAvqGqR/yK3AT8W1WL/bbZ7z4sFJFXcRJLk5Zzsog/Ld7OhH6JXD8sKdThGGNMJYHUFFYCfUWkp4hE4zQDzfcvICLtRKRsX4/gjETyNx14q8I2ndzfAlwHbDjj6BuZPy3aRl5RKT+/4txQh2KMMVWqMSmoagkwA6fpZzPwjqpuFJFZInKNW2wCsFVEtgEdgF+XbS8iPXBqGp9W2PWbIrIeWA+0A56o3Vtp2NIOHefvK/byzZHd6NuhZajDMcaYKgXUmK2qHwIfVlj2qN/jeUCVQ0tVdTdOZ3XF5ZecSaCN3W8+3ELzaA8/vrRvqEMxxphq2cQ59eDz7Vn8b8sh7r2kD21bxIQ6HGOMqZYlhTpW6lWe+GAz3RKac/uFPUIdjjHGnJYlhTr2Tko6Ww8e55HL+xMTadNVGGMaNksKdeh4QTG//2QrI3q0YcrAjqEOxxhjamRJoQ49t3QHh08U8X9XDrB5jIwxjYIlhTqSnpPHS8t2MXVYF4Z0bR3qcIwxJiCWFOrIbxdsJULgoSn9Qh1K4+L1QkYKeEtrLmtCy1sK+1bD8YOhjsShCgc3QuHxUEcSXCcPw/p5zv9GPbBJd+rAqj25vL82kx9N7EunVs1CHU7j8ulT8OnT0GEQTHkSeo4NdUTG38ls2LEYtn8CaYsgPxeimsOYH8OF90J0iGb5PbgJPp4Juz6FuPYw8VEYegtENMLvvV4v7E+F7Qud47xvFaDQtjd0HlbnLy+qWucvEizJycmakpIS6jBOS1W5/m9fknkknyUPTiAuxvJuwHYvg9euhp7jIHsnHN0L514Dk38FbXqEOrrw5PXCgbWnTlAZKYBC83bQdxL0mgBbP4RN70GrrjDpl3DeVKivPrS8HFjya0h5BWLi4cIZsO0TyPgaOg2Fy5+GbqPqJ5bayD8CO/7nHOe0hXAyCxBISoa+k51j3XHIWSc5EVmlqskBlbWkEFzz12byo7fW8NsbBnNTcteaNzCOvBx4bgxENYMffAYRHvjyr7DsWaeZ4sIZcNEDENMi1JE2fQVHYceSUyeoEwcBgS7nnzpBdRpW/gS1exl8NBMOroduo2HKU9B5aN3FWFoMK1+GpU86zUUjvgsTHoHmCU4z0vp5sPBROJ4JA78Bl/4SWjeg/0dVOLQJti1wjnP6CtBSiG0NfS51jnOfiRDXLigvZ0khRAqKS5n4+09p3TyK92dcRESEjTgKiCq8Nd1pjvjeovInk6P7YPEvYd1caNERLn0cBk9rnM0CDZUqHNrs1AS2L4T05eAtgdhWp05QvSdCixqmrveWwpo3YPGvIC8bht3qNOO0aB/ceNMWwcc/g8NbnZrKZU9ChwGVyxWdhC/+5PwgMOY+5ydUTVyFJ5zmrbLjfMydbLrjYDfZToYuw8ET/NYFSwohMntJGr9bsJW3vj+K0b3bhjqcxmPFC/DRQ84/9+h7qi6TvhI+fthpX+0yHKY8DV1H1H1s3lI4lgkE4f+kWRuIaSCTIRbllT9BHXXvo9VhEJxTdoJKPrsTVMFR+PS3sOJ5iGwG4x+CC+6CyFpO8XI4DT75OWz7GNr0hMt+A/0ur7mp6sheWPgYbPwXxCc5TVwDv1H3TVyqkJ3mHuNPYM+XUFoE0S2h98VubeBSiO9Ut3FgSSEkDh0v4OLfLWVMn3a88K2Ajr0BOLAeXpwIvcbDN985/T+q1+vUGBY9DicOODWGSx+H+M7BjalcZ+piyM8Jzn6j4mDs/TB6htNMFgreUljzd/jfr5x26+gWzrftsmahYB7Lsz2JV+RLMv8PImPPPsns+RI+ehgOrIOuo5yBDF3OP7N91KQoD3Z/fqrpLXe3szyxv3N8+052XjsyOrivWwNLCiHwyL/WMW9VBp/cP56e7eJCHU7jUHQS/t94p0347i8Cbz8tPOH0NXz5V6fv4aIHnD6Hsz3Rer2Qucb5J97+iTPM0r8ztetI8NTyn1gVti+Aze+HpkMWYNfn8PEjTrt/11Ew/qfQY2zdn6ACbe6pqC6ao7ylkPomLJ7lDPUceouzz5Ydzn6f2TtOdcTvXgalhc6IrJ7jTjW/tel+9vsPAksK9Wzz/mNc+efP+c6YnvziqgA+7Mbx3gznW+u33nNqCmcqdzd88gvYPB9adYPJs2DAdYGdaPNynFpA2kLnd95hyo326HOpM3ol2H0Xuz5zTpBlJ+a6+LZaUc5O5zht+cA5TpN+CeddX78Jydcx/BsnqSffARf/zOkYrsruL5zmwgN11HFdcAw++x0sf86pcYx7EEbdE1jtozjfOfmXJYLcXc7ytn3d2sAk6HYhRMUGL95asqRQj1SVW19ewcbMY3z64MW0ah4V6pAahw3/hHl3wNifON/UamPX584Y9YMboPsY5wTSaXD5MmVjv9MWnRr7rV5o3tZJAH0mQe9LIK4e+oIqNuEM+aZzDILdtlxwDD5/xjnxRUSFvukKnKa5pb85NYT04p85CcLj/t/k7nFGDW36j9P+P3lW3daosnfAJ//nDKtt0xMmPwH9r6z8ejk7Ybv72dn9OZQUOP0lPcc5SaDPpZDQs25iDAJLCvVo8eaDfPe1FB67egDfGdNwPxQNSu5ueH4sJPaD73x06oRQG95SWP26c6LNy4Hzb4MLfwT71zqJIG3RqbHfZUMr+0xyLgYK1UimgmPw+e9h+d8gIrL2zWBlyppd/veE856H3gKX/KJeOjQDdnCj05S161Onvf3SX8K+FPjizyARcNH99Xsx3I7/OfFkbYGe453aVF62kwjSFjodxgAJvd3+l0uh+0UNqjZwOpYU6klxqZfL/vgZKCy4fxxRngBOLmmLYMHPncd9LnV+ul9Y+5EZjUVpMbwyBQ5vh7s+D35ba/4Rp1lgxfPOsEqAZgnOmO8+k4I69jtocnY6346D0d8Qiuaps6XqfENf8PNTTTCDbnQGD7RKqv94Sktg1avOxXD5uc6yyFin36UsEST0qv+4gsCSQj157cvdPDZ/Iy99K5lLB9TQUXUiCxY8AuvfhbZ9nA992RC1qDi3GuomiaZ89e6ix2HZH+DGOU67dl05vN2p6ieNdE6KEY3gXhblOoIvcE/owwPbtiH0G5ytkkLY8C/n/6I+hhnXJC8H1r3jTCvR46LQNrcFiSWFenA0r5jxzyzhvM7x/P27F1Q/NbaqU5X/5BfOaJuxP4GxDzg1g6KTzokgbaHTaXVkj7NNWYdVn0udNvJGUkWt0Y4l8Mb1TtPONX8JdTQNU7nRMVkwZLrb31DNUNGyDtMVz7v9Bg/A6B82iROZCR5LCvXgiQ828fIXu/jvvWMZ0Dm+6kKHt8P7P4Y9y5zRCFf/0WlHr4qq0+lVliDKhrZFNnMmheszqVFXXzmRBc+PcS7jv3Np6K4qbSwq9Te4bexlJ/uq+g0mPgot7WZOprKgJwURmQL8CfAAL6nqUxXWdwdeARKBHOBWVc1w15UC692ie1X1Gnd5T+BtoC2wCrhNVYtOF0dDSQq7Dp9k8h8+ZeqwJJ6+YXDlAiWFThPJ5793/oknzYJh3zqzDs2iPNjzxamLYHJ2OssTerkJYlLjqdp6vfCPm5z27juXQIfzQh1R45Gzy+1vmH/qaty4RFjwM2e0VbfRTjNTPcyeaRqvoCYFEfEA24BJQAawEpiuqpv8yrwLfKCqr4nIJcB3VPU2d90JVa00i5mIvAP8S1XfFpHngbWq+tzpYmkoSeEHb6Tw+fbDLH1oAu1bVmja2fMlvH8fHN7mXEp/2ZO1uzCmTPaOU+Pqd30OJflOJ9jAG5wmg7a9a/8adeXLvzpXtl7xDIz8fqijaZx2L3OG3R5wv1+d6XUZJqwFOymMBh5X1cvc548AqOqTfmU2AlNUNV2cxvWjqhrvrquUFNwyWUBHVS2p+BrVaQhJ4asd2Ux/cTkPTj6HGZf0PbUiP9f5Rrf6decf9qpnnW/zdaG4wKlFbPkAUv/hjOgZfBOMfRDa9amb1zxbmWvgpUlwzmUw7e92AqsNb6kzUKHwOAy7ren0NZk6dyZJIZDZrroA6X7PM4ALKpRZC0zFaWK6HmgpIm1VNRuIFZEUoAR4SlX/g9NkdERVS/z22aWaN3MncCdAt27dAnlPdcbrVZ747yY6t4rle2Pdtn1V50Ksjx9xxjVfeK8zhW90HU51ERXrDrGcCOMfhi//4lwtum6uU3MY9xAknlN3rx+owuPOBWot2jsdy5YQaifCA0NuDnUUpokL1lU7DwLjRWQNMB7YB5TdT7G7m6G+CfxRRM6onUNVX1DVZFVNTkysYereOvavNfvYmHmMhy/vT2yUx7n68s0b4Z/fhVZdnA7UyU/UbUKoqGVHuOzX8ON1ztWqWz6A2SOdk/GhLfUXR1X++6BzodrUF6ufzsAY06AEUlPYB/jfnSLJXeajqpk4NQVEpAXwDVU94q7b5/7eKSJLgWHAP4HWIhLp1hYq7bMhmvPlLs7rHM/VA9s7V14ufRIQZ1qFkXeGdix8i/bOHcrG3OfUHL5+0Rn7PeBaZ+Kz+u7cXfs2rHsbxs+EHmPq97WNMWctkJrCSqCviPQUkWjgZmC+fwERaSciZft6BGckEiLSRkRiysoAY4BN6nRkLAFucLe5HXivtm+mLqkqu7JOcm3iQSJeugQW/sK5HP6HK2DU3Q3n4qi4ds4IlR+vdzqg0xbDcxfC3FtPdVLWtewd8N+fOMNwxz1UP69pjAmKGpOC+01+BrAA2Ay8o6obRWSWiFzjFpsAbBWRbUAH4Nfu8nOBFBFZi5MEnvIbtfQw8ICIpOH0MbwcpPdUJ3JOFjG9dD7f2/p9OHEIbnodpr/VsG7x5y+urTNu/cfrYNxPYeen8PxF8PYtkJlad69bUuQ0XUVEwjderJO7SBlj6o5dvBagNXty6PrKECI6DCDhjnecWxU2JvlHnKtel//NuWnJOZc7zUrBnhdnwc/hq7/CtDfh3KuCu29jzFk5k9FHdqPbAGVl7qKdHKO439WNLyEANGsNE2Y6zUoX/x/s/QpevNjpKE9f6cwRX1rsjKY6W9sXOglhxPcsIRjTSFndPkAlGasAaNWrAUzYVRuxrdzbGf4AVr7oXFj28qXly4jHaf4p+/H4PY4oWxdV/rknyhnt1P48ZwSWMaZRsqQQoNhD6yjBQ2xSFdNaNEax8c7kfCPvdEYp5ec4U02Xlji/fT+l7u/iCs9LnJqF//PuFzojoBrD1BvGmCpZUghQ22Ob2BvZnV5N7YQX0xKG3x7qKIwxDYT1KQRCle6F2zgY1z/UkRhjTJ2ypBCA4pw9tOY4J9sOCnUoxhhTpywpBOBI2grngU1PbIxp4iwpBKBgzyqK1EN89yGhDsUYY+qUJYUARB5cy1btSlJim1CHYowxdcqSQk1UaX1kIxu1Nx3ibf56Y0zTZkmhJrm7aFZ6nH3N++OJsPsBGGOaNksKNXEnjzvSZmBo4zDGmHpgSaEmmWsoIhJP+3NDHYkxxtQ5Swo1KMlYzSZvN7q0a4ST4BljzBmypHA6Xi+yfy3rvb3omtA81NEYY0yds6RwOrm78BQfZ532omtCE5vzyBhjqmBJ4XQy1wCwwdvTagrGmLBgSeF0MtdQLNEcju1BfGxUqKMxxpg6Z1Nnn07mGnZH9aZzfHyoIzHGmHphNYXqeL2wfy3rrenIGBNGrKZQnew0KDrB8pJudLOkYIwJEwHVFERkiohsFZE0EZlZxfruIrJYRNaJyFIRSXKXDxWRr0Rko7tumt82c0Rkl4ikuj9Dg/augsHtZE4ttZqCMSZ81JgURMQDzAYuBwYA00VkQIVizwCvq+pgYBbwpLs8D/iWqp4HTAH+KCKt/bZ7SFWHuj+ptXonwZa5hlJPM3ZoZ6spGGPCRiA1hZFAmqruVNUi4G3g2gplBgD/cx8vKVuvqttUdbv7OBM4BCQGI/A6tz+VnPj+lOKxpGCMCRuBJIUuQLrf8wx3mb+1wFT38fVASxFp619AREYC0cAOv8W/dpuV/iAiMVW9uIjcKSIpIpKSlZUVQLhB4C2F/WvZG3MOngihUyubMtsYEx6CNfroQWC8iKwBxgP7gNKylSLSCXgD+I6qet3FjwD9gRFAAvBwVTtW1RdUNVlVkxMT66mScXgbFOexid50ad2MSI8N0jLGhIdARh/tA7r6PU9yl/m4TUNTAUSkBfANVT3iPo8H/gv8XFWX+22z331YKCKv4iSWhsHtZF5RaCOPjDHhJZCvwCuBviLSU0SigZuB+f4FRKSdiJTt6xHgFXd5NPBvnE7oeRW26eT+FuA6YEMt3kdwZaZCdAtWHG1jcx4ZY8JKjUlBVUuAGcACYDPwjqpuFJFZInKNW2wCsFVEtgEdgF+7y28CxgHfrmLo6Zsish5YD7QDngjSe6q9zDWUdhhEVl6pDUc1xoSVgC5eU9UPgQ8rLHvU7/E8YF4V2/0d+Hs1+7zkjCKtL6UlcGA9R869BcCaj4wxYcV6UCs6vBVK8tnXvD9gScEYE14sKVTkdjJvld6AJQVjTHixpFBR5hqIbsnGgna0jI2kVTObMtsYEz4sKVSUuQY6D2VPbgHdEprjDI4yxpjwYEnBX2kxHNgAnYeyNyePrm2s6cgYE14sKfg7tBlKC/F2HEp6bj7d2lpSMMaEF0sK/txO5pxW51FU4rVrFIwxYceSgr/MNRDTil3e9oCNPDLGhB9LCv72p7r9CfmAJQVjTPixpFCmpLBcJ7MIdGlt8x4ZY8KLJYUyhzaBtxg6DyM9J49O8bFER9rhMcaEFzvrlXE7mek8zBmOak1HxpgwZEmhTGYqNGsDrbuTnptn/QnGmLBkSaFM5hroNJSCEi8HjxVaUjDGhCVLCgDFBU6fQudhZOTmAdiFa8aYsGRJAeDgRvCW+PoTAOtTMMaEJUsKAPv9Opmz3aRg8x4ZY8KQJQVw+hOat4VWSezNyadZlId2LaJDHZUxxtQ7SwrgjDzqPAxEfCOPbMpsY0w4sqRQnO/Mjtp5GADpdo2CMSaMBZQURGSKiGwVkTQRmVnF+u4islhE1onIUhFJ8lt3u4hsd39u91s+XETWu/v8s4Tqq/mBDaCl0HkYqsreHLtGwRgTvmpMCiLiAWYDlwMDgOkiMqBCsWeA11V1MDALeNLdNgF4DLgAGAk8JiJt3G2eA74P9HV/ptT63ZyNsiuZOw0l+2QReUWldEuwOY+MMeEpkJrCSCBNVXeqahHwNnBthTIDgP+5j5f4rb8MWKiqOaqaCywEpohIJyBeVZerqgKvA9fV7q2cpcw1ENce4jvbcFRjTNgLJCl0AdL9nme4y/ytBaa6j68HWopI29Ns28V9fLp91o/9qac6md2kYM1HxphwFayO5geB8SKyBhgP7ANKg7FjEblTRFJEJCUrKysYuzyl6CRkbSnXyQyQZNcoGGPCVCBJYR/Q1e95krvMR1UzVXWqqg4Dfu4uO3Kabfe5j6vdp9++X1DVZFVNTkxMDCDcM3BgPagXOg8FYG9OHu1bxtAs2hPc1zHGmEYikKSwEugrIj1FJBq4GZjvX0BE2olI2b4eAV5xHy8AJotIG7eDeTKwQFX3A8dEZJQ76uhbwHtBeD9nxq+TGbCRR8aYsFdjUlDVEmAGzgl+M/COqm4UkVkico1bbAKwVUS2AR2AX7vb5gC/wkksK4FZ7jKAe4CXgDRgB/BRsN5UwDJToWUniO8EQHpOviUFY0xYiwykkKp+CHxYYdmjfo/nAfOq2fYVTtUc/JenAAPPJNigy1zj608oKvGSeTTfRh4ZY8Ja+F7RXHgcDm/zNR3tO5KPqg1HNcaEt/BNCvvXAVpp5JE1HxljwlkYJ4VU57ffyCOwpGCMCW/hmxQy10B8F2jRHnBqCtGREbRvGRPiwIwxJnTCOym4TUfg1BS6tmlGRIRNmW2MCV/hmRQKjkJ2mq/pCOwaBWOMgXBNCvvXOb8r1hQsKRhjwlx4JgXflcxOUjiaV8zxghKrKRhjwl74JoVW3SCuLYBNmW2MMa7wTQoV+hPAhqMaY0z4JYX8XMjdVak/AaymYIwx4ZcU9q91fleoKbSNi6ZFTEBTQRljTJMVfkmhwnTZ4Fy4lmS1BGOMCdOk0KYHNE/wLUrPtWsUjDEGwjIppJbrTygp9bIvN59uCc1CF5MxxjQQ4ZUU8nLgyJ5yTUf7jxZQ4lWrKRhjDOGWFMr6E/xqCuk28sgYY3zCMyl0GuJbZNcoGGPMKeGVFPanQkJvaNbat2hvTh6REUKnVtanYIwx4ZUUMlPLXZ8AkJ6bT5c2zfDYlNnGGBNGSeHkYTiaXq4/AWzKbGOM8RdQUhCRKSKyVUTSRGRmFeu7icgSEVkjIutE5Ap3+S0ikur34xWRoe66pe4+y9a1D+o7qygz1fldISmk25TZxhjjU+O8DiLiAWYDk4AMYKWIzFfVTX7F/g94R1WfE5EBwIdAD1V9E3jT3c8g4D+qmuq33S2qmhKct1KDzDWAQMfBvkXHC4rJOVlkNQVjjHEFUlMYCaSp6k5VLQLeBq6tUEaBePdxKyCziv1Md7cNjcw10LYPxMb7FqXn5AM28sgYY8oEkhS6AOl+zzPcZf4eB24VkQycWsK9VexnGvBWhWWvuk1HvxCRKnt6ReROEUkRkZSsrKwAwq1GhXsygw1HNcaYioLV0TwdmKOqScAVwBsi4tu3iFwA5KnqBr9tblHVQcBY9+e2qnasqi+oarKqJicmJp5ddMcPwvHMSkkhI9e9cK2NJQVjjIHAksI+oKvf8yR3mb/vAu8AqOpXQCzQzm/9zVSoJajqPvf3ceAfOM1UdWN/qvO7ippCfGwkrZpH1dlLG2NMYxJIUlgJ9BWRniISjXOCn1+hzF5gIoCInIuTFLLc5xHATfj1J4hIpIi0cx9HAVcBG6grvk7mQeWDzsmjW1urJRhjTJkaRx+paomIzAAWAB7gFVXdKCKzgBRVnQ/8BHhRRO7H6XT+tqqqu4txQLqq7vTbbQywwE0IHmAR8GLQ3lVFmWsgsR/EtCi3eG9OHv07tqyzlzXGmMYmoFuNqeqHOB3I/sse9Xu8CRhTzbZLgVEVlp0Ehp9hrGdv9AwoPFZukderZOTkM2lAh3oLwxhjGrrwuP9kz7GVFh08XkBRqddGHhljjJ/wmeaigrJrFGzkkTHGnBK2ScGuUTDGmMrCOilECHRubVNmG2NMmbBNCuk5eXRq1YzoyLA9BMYYU0nYnhFtymxjjKnMkoIxxhifsEwK+UWlZB0vpGuC9ScYY4y/sEwKvonwrKZgjDHlhGVSsOGoxhhTNUsKxhhjfMI2KcRFe0iIiw51KMYY06CEZVJIz8mja0JzqrnZmzHGhK0wTQr51slsjDFVCI9ZUv2oKntz8riob7uaCxtjTqu4uJiMjAwKCgpCHYoBYmNjSUpKIirq7O8mGXZJ4fCJIvKLS62T2ZggyMjIoGXLlvTo0cOaY0NMVcnOziYjI4OePXue9X7CrvnIRh4ZEzwFBQW0bdvWEkIDICK0bdu21rW2sEsK6Tl24ZoxwWQJoeEIxt8i7JJCWU0hqY1NcWGMMRWFXVJIz8mjY3wssVGeUIdijGmAUlJS+NGPflTt+szMTG644YZ6jKh+hV1H896cPJsIz5gwUlpaiscT+JfA5ORkkpOTq13fuXNn5s2bF4zQGqSAkoKITAH+BHiAl1T1qQrruwGvAa3dMjNV9UMR6QFsBra6RZer6l3uNsOBOUAz4EPgPlXVWr6fGqXn5DGqd9u6fhljws4v39/IpsxjQd3ngM7xPHb1edWu3717N1OmTGH48OGsXr2a8847j9dff50BAwYwbdo0Fi5cyE9/+lMSEhJ47LHHKCwspHfv3rz66qu0aNGClStXct9993Hy5EliYmJYvHgxq1at4plnnuGDDz7g008/5b777gOc9vrPPvuM7OxsrrrqKjZs2EBBQQF33303KSkpREZG8uyzz3LxxRczZ84c5s+fT15eHjt27OD666/nt7/9bVCPTV2pMSmIiAeYDUwCMoCVIjJfVTf5Ffs/4B1VfU5EBuCc5Hu463ao6tAqdv0c8H1ghVt+CvDRWb6PgBSWlLL/WIGNPDKmCdm6dSsvv/wyY8aM4Y477uBvf/sbAG3btmX16tUcPnyYqVOnsmjRIuLi4nj66ad59tlnmTlzJtOmTWPu3LmMGDGCY8eO0axZ+VaEZ555htmzZzNmzBhOnDhBbGxsufWzZ89GRFi/fj1btmxh8uTJbNu2DYDU1FTWrFlDTEwM/fr1495776Vr1671c1BqIZCawkggTVV3AojI28C1gH9SUCDefdwKyDzdDkWkExCvqsvd568D11HHSWFfbj6qNhzVmLpwum/0dalr166MGTMGgFtvvZU///nPAEybNg2A5cuXs2nTJl+ZoqIiRo8ezdatW+nUqRMjRowAID4+vtK+x4wZwwMPPMAtt9zC1KlTSUpKKrd+2bJl3HvvvQD079+f7t27+5LCxIkTadWqFQADBgxgz549jSIpBNLR3AVI93ue4S7z9zhwq4hk4Hzrv9dvXU8RWSMin4rIWL99ZtSwTwBE5E4RSRGRlKysrADCrZ5do2BM01NxGGbZ87i4OMC5qGvSpEmkpqaSmprKpk2bePnllwPa98yZM3nppZfIz89nzJgxbNmyJeC4YmJifI89Hg8lJSUBbxtKwRp9NB2Yo6pJwBXAGyISAewHuqnqMOAB4B8iUjkdn4aqvqCqyaqanJiYWKsg03PzAUsKxjQle/fu5auvvgLgH//4BxdddFG59aNGjeKLL74gLS0NgJMnT7Jt2zb69evH/v37WblyJQDHjx+vdOLesWMHgwYN4uGHH2bEiBGVksLYsWN58803Adi2bRt79+6lX79+dfI+60sgSWEf4F/nSXKX+fsu8A6Aqn4FxALtVLVQVbPd5auAHcA57vb+9bCq9hl06Tl5xERGkNgypubCxphGoV+/fsyePZtzzz2X3Nxc7r777nLrExMTmTNnDtOnT2fw4MGMHj2aLVu2EB0dzdy5c7n33nsZMmQIkyZNqnQ18B//+EcGDhzI4MGDiYqK4vLLLy+3/p577sHr9TJo0CCmTZvGnDlzytUQGiOpacCPiEQC24CJOCfulcA3VXWjX5mPgLmqOkdEzgUW4zQHtQNyVLVURHoBnwODVDVHRL4GfsSpjua/qOqHp4slOTlZU1JSzvKtwl1vrCIt6wSLHhh/1vswxpyyefNmzj333JC9/u7du30jgYyjqr+JiKxS1erH2fqpsaNZVUtEZAawAGe46SuqulFEZgEpqjof+Anwoojcj9Pp/G1VVREZB8wSkWLAC9ylqjnuru/h1JDUj6jjTmZw+hSs6cgYY6oX0HUK7jf4Dysse9Tv8SZgTBXb/RP4ZzX7TAEGnkmwtaGqpOfkMbJnQn29pDGmjvXo0cNqCUEWNtNcHMkr5nhhiU2EZ4wxpxE2SSE914ajGmNMTcImKez1TZlt8x4ZY0x1wi8ptLGagjHGVCdskkJ6Th7tWkQTFxN2E8MaY0zAwiYpOFNmWy3BmHDWokWLen/N+fPn89RTT1W7vqb7N9S3sPnavDcnj/O7tQl1GMY0XR/NhAPrg7vPjoPg8upPqKFwpvdnuOaaa7jmmmuqXV/T/RvqW1jUFEpKvWQesSmzjWlqZs6cyezZs33PH3/8cZ544gkmTpzI+eefz6BBg3jvvfcC2tfSpUsZN24cV155Jf369eOuu+7C6/UCTg3jJz/5CUOGDOGrr77i73//OyNHjmTo0KH84Ac/oLS0FICPP/6Y888/nyFDhjBx4kQA5syZw4wZMwB49913GThwIEOGDGHcuHG+173qqqsAyMnJ4brrrmPw4MGMGjWKdevW+d7XHXfcwYQJE+jVq5dvJtg6oaqN5mf48OF6NvZmn9TuD3+gc7/ee1bbG2OqtmnTppC+/urVq3XcuHG+5+eee67u3btXjx49qqqqWVlZ2rt3b/V6vaqqGhcXV+2+lixZojExMbpjxw4tKSnRSy+9VN99911VVQV07ty5quq856uuukqLiopUVfXuu+/W1157TQ8dOqRJSUm6c+dOVVXNzs5WVdVXX31Vf/jDH6qq6sCBAzUjI0NVVXNzc32ve+WVV6qq6owZM/Txxx9XVdXFixfrkCFDVFX1scce09GjR2tBQYFmZWVpQkKC7/UrqupvgjP7REDn2bBoPjo1HNVqCsY0JcOGDePQoUNkZmaSlZVFmzZt6NixI/fffz+fffYZERER7Nu3j4MHD9KxY8ca9zdy5Eh69eoFwPTp01m2bBk33HADHo+Hb3zjGwC+u7OV3YchPz+f9u3bs3z5csaNG0fPnj0BSEioPHvCmDFj+Pa3v81NN93E1KlTK61ftmwZ//ynMwnEJZdcQnZ2NseOOXezu/LKK4mJiSEmJob27dtz8ODBSvd3CIawSgrd2lpSMKapufHGG5k3bx4HDhxg2rRpvPnmm2RlZbFq1SqioqLo0aNHpdlPq1PdvRliY2N9/Qiqyu23386TTz5Zruz7779f4/6ff/55VqxYwX//+1+GDx/OqlWrAooL6u/+DGHRp7A3J48oj9AxPrbmwsaYRmXatGm8/fbbzJs3jxtvvJGjR4/Svn17oqKiWLJkCXv27Al4X19//TW7du3C6/Uyd+7cSvdmAOeOavPmzePQoUOA0w+wZ88eRo0axWeffcauXbt8yyvasWMHF1xwAbNmzSIxMZH09PRy6/3vz7B06VLatWtX5R3h6lLY1BSS2jTHEyE1FzbGNCrnnXcex48fp0uXLnTq1IlbbrmFq6++mkGDBpGcnEz//v0D3teIESOYMWMGaWlpXHzxxVx//fWVygwYMIAnnniCyZMn4/V6iYqKYvbs2YwaNYoXXniBqVOn4vV6ad++PQsXLiy37UMPPcT27dtRVSZOnMiQIUP49NNPfevLOpQHDx5M8+bNee21187+wJylGu+n0JCc7f0U/rY0jeMFJTw8JfAPhzGmZqG+n0IwLV26lGeeeYYPPvgg1KHUSp3fT6EpuGdCn1CHYIwxjUJYJAVjjCmzfv16brvttnLLYmJiWLFiBRMmTAhNUA2IJQVjTK2oaqVROw3ZoEGDSE1NDXUYdSIY3QFhMfrIGFM3YmNjyc7ODsrJyNSOqpKdnU1sbO1GWVpNwRhz1pKSksjIyCArKyvUoRicJF3bC9osKRhjzlpUVJTvCl7TNFjzkTHGGB9LCsYYY3wsKRhjjPFpVFc0i0gWEPhEJuW1Aw4HMZz6YDHXvcYWL1jM9aWxxXy6eLuramIgO2lUSaE2RCQl0Mu8GwqLue41tnjBYq4vjS3mYMVrzUfGGGN8LCkYY4zxCaek8EKoAzgLFnPda2zxgsVcXxpbzEGJN2z6FIwxxtQsnGoKxhhjamBJwRhjjE+TSwoiMkVEtopImojMrGJ9jIjMddevEJEeIQjTP56uIrJERDaJyEYRua+KMhNE5KiIpLo/j4YiVr94dovIejeWSrfCE8ef3WO8TkTOD0WcfvH08zt2qSJyTER+XKFMyI+xiLwiIodEZIPfsgQRWSgi293fbarZ9na3zHYRuT3EMf9ORLa4f/t/i0jrarY97eeonmN+XET2+f39r6hm29OeX+ox3rl+se4WkdRqtj3zY6yqTeYH8AA7gF5ANLAWGFChzD3A8+7jm4G5IY65E3C++7glsK2KmCcAH4T6+PrFsxtod5r1VwAfAQKMAlaEOuYKn5EDOBfzNKhjDIwDzgc2+C37LTDTfTwTeLqK7RKAne7vNu7jNiGMeTIQ6T5+uqqYA/kc1XPMjwMPBvDZOe35pb7irbD+98CjwTrGTa2mMBJIU9WdqloEvA1cW6HMtUDZ3bDnARMlhHcIUdX9qrrafXwc2Ax0CVU8QXIt8Lo6lgOtRaRTqINyTQR2qOrZXhlfZ1T1MyCnwmL/z+trwHVVbHoZsFBVc1Q1F1gITKmrOP1VFbOqfqKqJe7T5UDt5nIOsmqOcyACOb8E3enidc9dNwFvBev1mlpS6AKk+z3PoPIJ1lfG/eAeBdrWS3Q1cJuyhgErqlg9WkTWishHInJe/UZWiQKfiMgqEbmzivWB/B1C5Waq/wdqSMe4TAdV3e8+PgB0qKJMQz7ed+DUGqtS0+eovs1wm7xeqaaZriEe57HAQVXdXs36Mz7GTS0pNFoi0gL4J/BjVT1WYfVqnOaOIcBfgP/Uc3gVXaSq5wOXAz8UkXEhjicgIhINXAO8W8XqhnaMK1GnPaDRjCEXkZ8DJcCb1RRpSJ+j54DewFBgP06TTGMwndPXEs74GDe1pLAP6Or3PMldVmUZEYkEWgHZ9RJdNUQkCichvKmq/6q4XlWPqeoJ9/GHQJSItKvnMP3j2ef+PgT8G6da7S+Qv0MoXA6sVtWDFVc0tGPs52BZ05v7+1AVZRrc8RaRbwNXAbe4yaySAD5H9UZVD6pqqap6gReriaVBHWf3/DUVmFtdmbM5xk0tKawE+opIT/db4c3A/Apl5gNlozNuAP5X3Ye2Prhtgi8Dm1X12WrKdCzr9xCRkTh/t5AkMhGJE5GWZY9xOhU3VCg2H/iWOwppFHDUrwkklKr9VtWQjnEF/p/X24H3qiizAJgsIm3cZo/J7rKQEJEpwE+Ba1Q1r5oygXyO6k2FPq/rq4klkPNLfboU2KKqGVWtPOtjXNc95/X9gzPyZRvOKIGfu8tm4XxAAWJxmg/SgK+BXiGO9yKcJoF1QKr7cwVwF3CXW2YGsBFntMNy4MIQxtvLjWOtG1PZMfaPV4DZ7t9gPZDcAD4XcTgn+VZ+yxrUMcZJWPuBYpz26u/i9HctBrYDi4AEt2wy8JLftne4n+k04DshjjkNp+297PNcNtqvM/Dh6T5HIYz5Dfezug7nRN+pYszu80rnl1DE6y6fU/b59Stb62Ns01wYY4zxaWrNR8YYY2rBkoIxxhgfSwrGGGN8LCkYY4zxsaRgjDHGx5KCMcYYH0sKxhhjfP4/DxuD4euObHsAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "history_df = pd.DataFrame(history.history)\n",
    "# Start the plot at epoch 1\n",
    "history_df.loc[0:, ['loss', 'val_loss']].plot()\n",
    "history_df.loc[0:, ['accuracy', 'val_accuracy']].plot()\n",
    "history_df.loc[0:, ['precision', 'val_precision']].plot()\n",
    "\n",
    "print((\"Best Validation Loss: {:0.4f}\" +\\\n",
    "      \"\\nBest Validation Accuracy: {:0.4f}\" +\\\n",
    "      \"\\nBest Validation Precision: {:0.4f}\")\\\n",
    "      .format(history_df['val_loss'].min(), \n",
    "              history_df['val_accuracy'].max(),\n",
    "             history_df['val_precision'].max()))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5c4b0a1c",
   "metadata": {
    "papermill": {
     "duration": 0.314017,
     "end_time": "2022-07-26T12:25:15.379830",
     "exception": false,
     "start_time": "2022-07-26T12:25:15.065813",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "### Saving Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "18e24e50",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-26T12:25:15.921970Z",
     "iopub.status.busy": "2022-07-26T12:25:15.921620Z",
     "iopub.status.idle": "2022-07-26T12:25:46.437033Z",
     "shell.execute_reply": "2022-07-26T12:25:46.436019Z"
    },
    "papermill": {
     "duration": 30.792055,
     "end_time": "2022-07-26T12:25:46.439755",
     "exception": false,
     "start_time": "2022-07-26T12:25:15.647700",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2022-07-26 12:25:22.950024: W tensorflow/python/util/util.cc:348] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.\n"
     ]
    }
   ],
   "source": [
    "model.save('dgsp_keras_model.v1')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d4daa0ff",
   "metadata": {
    "papermill": {
     "duration": 0.283538,
     "end_time": "2022-07-26T12:25:46.999833",
     "exception": false,
     "start_time": "2022-07-26T12:25:46.716295",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Test Prediction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "a9d82374",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-26T12:25:49.220941Z",
     "iopub.status.busy": "2022-07-26T12:25:49.220377Z",
     "iopub.status.idle": "2022-07-26T12:27:20.430211Z",
     "shell.execute_reply": "2022-07-26T12:27:20.429201Z"
    },
    "papermill": {
     "duration": 93.236518,
     "end_time": "2022-07-26T12:27:20.741435",
     "exception": false,
     "start_time": "2022-07-26T12:25:47.504917",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 1min 31s, sys: 0 ns, total: 1min 31s\n",
      "Wall time: 1min 31s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "# Processing\n",
    "test.text = test.text.apply(preprocess_text)\n",
    "\n",
    "# Encoding\n",
    "test_text = bert_encode(test.text.astype(str), tokenizer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "d6963546",
   "metadata": {
    "_kg_hide-output": true,
    "execution": {
     "iopub.execute_input": "2022-07-26T12:27:21.271655Z",
     "iopub.status.busy": "2022-07-26T12:27:21.270947Z",
     "iopub.status.idle": "2022-07-26T12:30:53.391988Z",
     "shell.execute_reply": "2022-07-26T12:30:53.391110Z"
    },
    "papermill": {
     "duration": 212.388432,
     "end_time": "2022-07-26T12:30:53.394025",
     "exception": false,
     "start_time": "2022-07-26T12:27:21.005593",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "669/669 [==============================] - 212s 313ms/step\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([[0.99972063],\n",
       "       [0.9999404 ],\n",
       "       [0.9988065 ],\n",
       "       ...,\n",
       "       [0.99998546],\n",
       "       [0.99995434],\n",
       "       [0.6572295 ]], dtype=float32)"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "preds = model.predict(test_text, verbose=1)\n",
    "preds"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "41bfa40e",
   "metadata": {
    "papermill": {
     "duration": 0.30686,
     "end_time": "2022-07-26T12:30:54.008099",
     "exception": false,
     "start_time": "2022-07-26T12:30:53.701239",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "987580d1",
   "metadata": {
    "papermill": {
     "duration": 0.30268,
     "end_time": "2022-07-26T12:30:54.661377",
     "exception": false,
     "start_time": "2022-07-26T12:30:54.358697",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Submission"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "34db8212",
   "metadata": {
    "papermill": {
     "duration": 0.499737,
     "end_time": "2022-07-26T12:30:55.466736",
     "exception": false,
     "start_time": "2022-07-26T12:30:54.966999",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "- Saving with Threshold >= 0.5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "dfa7c807",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-26T12:30:56.153426Z",
     "iopub.status.busy": "2022-07-26T12:30:56.153081Z",
     "iopub.status.idle": "2022-07-26T12:30:56.215975Z",
     "shell.execute_reply": "2022-07-26T12:30:56.214977Z"
    },
    "papermill": {
     "duration": 0.419558,
     "end_time": "2022-07-26T12:30:56.218047",
     "exception": false,
     "start_time": "2022-07-26T12:30:55.798489",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1    14792\n",
       "0     6611\n",
       "Name: fake, dtype: int64"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sub = pd.read_csv('../input/detecting-generated-scientific-papers/sample_submission.csv')\n",
    "sub['fake'] = (preds >= 0.5).astype(int)\n",
    "\n",
    "sub.to_csv(\"submission.csv\", index=False)\n",
    "sub['fake'].value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cb6598e4",
   "metadata": {
    "papermill": {
     "duration": 0.304685,
     "end_time": "2022-07-26T12:30:56.826581",
     "exception": false,
     "start_time": "2022-07-26T12:30:56.521896",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "- Saving With Threshold  0.45 to 0.15"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "bdd49ffd",
   "metadata": {
    "_kg_hide-output": true,
    "execution": {
     "iopub.execute_input": "2022-07-26T12:30:57.438048Z",
     "iopub.status.busy": "2022-07-26T12:30:57.437699Z",
     "iopub.status.idle": "2022-07-26T12:30:57.749099Z",
     "shell.execute_reply": "2022-07-26T12:30:57.747960Z"
    },
    "papermill": {
     "duration": 0.619389,
     "end_time": "2022-07-26T12:30:57.751129",
     "exception": false,
     "start_time": "2022-07-26T12:30:57.131740",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "=== Threshold 0.6 === \n",
      "1    14653\n",
      "0     6750\n",
      "Name: fake, dtype: int64 \n",
      "\n",
      "=== Threshold 0.575 === \n",
      "1    14682\n",
      "0     6721\n",
      "Name: fake, dtype: int64 \n",
      "\n",
      "=== Threshold 0.55 === \n",
      "1    14720\n",
      "0     6683\n",
      "Name: fake, dtype: int64 \n",
      "\n",
      "=== Threshold 0.525 === \n",
      "1    14761\n",
      "0     6642\n",
      "Name: fake, dtype: int64 \n",
      "\n",
      "=== Threshold 0.475 === \n",
      "1    14836\n",
      "0     6567\n",
      "Name: fake, dtype: int64 \n",
      "\n",
      "=== Threshold 0.45 === \n",
      "1    14866\n",
      "0     6537\n",
      "Name: fake, dtype: int64 \n",
      "\n",
      "=== Threshold 0.425 === \n",
      "1    14891\n",
      "0     6512\n",
      "Name: fake, dtype: int64 \n",
      "\n",
      "=== Threshold 0.4 === \n",
      "1    14931\n",
      "0     6472\n",
      "Name: fake, dtype: int64 \n",
      "\n",
      "=== Threshold 0.375 === \n",
      "1    14961\n",
      "0     6442\n",
      "Name: fake, dtype: int64 \n",
      "\n",
      "=== Threshold 0.35 === \n",
      "1    14994\n",
      "0     6409\n",
      "Name: fake, dtype: int64 \n",
      "\n"
     ]
    }
   ],
   "source": [
    "Thresholds = [0.6, 0.575, 0.55, 0.525, 0.475, 0.45, 0.425, 0.4, 0.375, 0.35]\n",
    "\n",
    "for tresh in Thresholds:\n",
    "    sub['fake'] = (preds >= tresh).astype(int)\n",
    "    sub.to_csv(f\"submission_trsh_{tresh}.csv\", index=False)\n",
    "    print(f\"=== Threshold {tresh} === \\n{sub['fake'].value_counts()} \\n\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.12"
  },
  "papermill": {
   "default_parameters": {},
   "duration": 2668.694063,
   "end_time": "2022-07-26T12:31:02.052289",
   "environment_variables": {},
   "exception": null,
   "input_path": "__notebook__.ipynb",
   "output_path": "__notebook__.ipynb",
   "parameters": {},
   "start_time": "2022-07-26T11:46:33.358226",
   "version": "2.3.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}