Flu_CatBoost-tuned-OptunaCVI.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1>Flu Shot Learning: Predict H1N1 and Seasonal Flu Vaccines</h1>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "import pandas_profiling as pf\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "train = pd.read_csv('./training_set_features.csv', index_col='respondent_id')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "test = pd.read_csv('./test_set_features.csv', index_col='respondent_id')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "labels = pd.read_csv('./training_set_labels.csv', index_col='respondent_id')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Imputation strategy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "num_cols = train.select_dtypes('number').columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "cat_cols = ['race', 'sex', \n",
    "       'marital_status', 'rent_or_own',  'hhs_geo_region',\n",
    "       'census_msa', 'employment_industry', 'employment_occupation']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "ord_cols = ['age_group', 'education',  'income_poverty',\n",
    "        'employment_status']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "assert len(num_cols)+len(cat_cols)+len(ord_cols) == train.shape[1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.preprocessing import MinMaxScaler, StandardScaler, OneHotEncoder\n",
    "from sklearn.impute import SimpleImputer\n",
    "from sklearn.compose import ColumnTransformer\n",
    "from category_encoders import OrdinalEncoder as oe\n",
    "from catboost import CatBoostClassifier\n",
    "from catboost import Pool, cv\n",
    "from sklearn.metrics import roc_curve, roc_auc_score\n",
    "import optuna"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Impute train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Categorical columns None\n",
    "for col in (cat_cols+ord_cols):\n",
    "    train[col] = train[col].fillna(value='None')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "for col in num_cols:\n",
    "    train[col] = train[col].fillna(value=-1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Impute test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Categorical columns None\n",
    "for col in (cat_cols+ord_cols):\n",
    "    test[col] = test[col].fillna(value='None')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "for col in num_cols:\n",
    "    test[col] = test[col].fillna(value=-1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train test split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split( train, labels, test_size=0.3, random_state=68)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get the index number of categorical features\n",
    "categorical_features_indices = np.where(X_train.dtypes != float)[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Optimize with Optuna and cross validation\n",
    "\n",
    "Using just the selected features idetified in the previous notebook, led to poor perfomances; it seems to be better to use all the columns. CatBoost can easily deal with useless features not using them. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_dataset = Pool(data=X_train,\n",
    "                     label=y_train.h1n1_vaccine,\n",
    "                     cat_features = categorical_features_indices)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "def objective(trial):\n",
    "    param = {\n",
    "        'iterations':trial.suggest_categorical('iterations', [100,200,300,500,1000,1200,1500]),\n",
    "        'learning_rate':trial.suggest_float(\"learning_rate\", 0.001, 0.3),\n",
    "        'random_strength':trial.suggest_int(\"random_strength\", 1,10),\n",
    "        'bagging_temperature':trial.suggest_int(\"bagging_temperature\", 0,10),\n",
    "        'max_bin':trial.suggest_categorical('max_bin', [4,5,6,8,10,20,30]),\n",
    "        'grow_policy':trial.suggest_categorical('grow_policy', ['SymmetricTree', 'Depthwise', 'Lossguide']),\n",
    "        'min_data_in_leaf':trial.suggest_int(\"min_data_in_leaf\", 1,10),\n",
    "        'od_type' : \"Iter\",\n",
    "        'od_wait' : 100,\n",
    "        \"depth\": trial.suggest_int(\"max_depth\", 2,10),\n",
    "        \"l2_leaf_reg\": trial.suggest_loguniform(\"l2_leaf_reg\", 1e-8, 100),\n",
    "         'one_hot_max_size':trial.suggest_categorical('one_hot_max_size', [5,10,12,100,500,1024]),\n",
    "        'custom_metric' : ['AUC'],\n",
    "        \"loss_function\": \"Logloss\",\n",
    "        'auto_class_weights':trial.suggest_categorical('auto_class_weights', ['Balanced', 'SqrtBalanced']),\n",
    "        }\n",
    "\n",
    "    scores = cv(train_dataset,\n",
    "            param,\n",
    "            fold_count=5, \n",
    "            early_stopping_rounds=10,         \n",
    "            plot=False, verbose=False)\n",
    "\n",
    "    return scores['test-AUC-mean'].max()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Even though CatBoost is quite fast and I used early stopping, it's going to run 100 fits so it takes a lot of time. You can avoid re-running the optimisation and copy the parameters from the print report replacing `**trial.params` when calling CatBoost"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:10:29,502]\u001b[0m A new study created in memory with name: no-name-6d494258-5754-40be-85bc-9ad0ddf11f82\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 01:10:56,712]\u001b[0m Trial 0 finished with value: 0.8637121084738049 and parameters: {'iterations': 300, 'learning_rate': 0.013964954297408176, 'random_strength': 1, 'bagging_temperature': 8, 'max_bin': 10, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 4, 'max_depth': 5, 'l2_leaf_reg': 21.328495943450676, 'one_hot_max_size': 100, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 0 with value: 0.8637121084738049.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 01:11:09,451]\u001b[0m Trial 1 finished with value: 0.8638461530457093 and parameters: {'iterations': 1200, 'learning_rate': 0.11477165079768124, 'random_strength': 9, 'bagging_temperature': 6, 'max_bin': 5, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 5, 'max_depth': 7, 'l2_leaf_reg': 0.5714362138520529, 'one_hot_max_size': 500, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 1 with value: 0.8638461530457093.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:11:31,776]\u001b[0m Trial 2 finished with value: 0.8639277452137699 and parameters: {'iterations': 1500, 'learning_rate': 0.06018181691194878, 'random_strength': 6, 'bagging_temperature': 4, 'max_bin': 10, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 6, 'max_depth': 4, 'l2_leaf_reg': 7.937278880272323e-06, 'one_hot_max_size': 12, 'auto_class_weights': 'Balanced'}. Best is trial 2 with value: 0.8639277452137699.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:11:41,329]\u001b[0m Trial 3 finished with value: 0.8608185752295553 and parameters: {'iterations': 100, 'learning_rate': 0.04917655399574397, 'random_strength': 1, 'bagging_temperature': 5, 'max_bin': 30, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 3, 'max_depth': 6, 'l2_leaf_reg': 1.253388039132331e-06, 'one_hot_max_size': 500, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 2 with value: 0.8639277452137699.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 01:11:50,036]\u001b[0m Trial 4 finished with value: 0.862663502918525 and parameters: {'iterations': 1500, 'learning_rate': 0.20567786125414012, 'random_strength': 8, 'bagging_temperature': 7, 'max_bin': 20, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 7, 'max_depth': 2, 'l2_leaf_reg': 0.0011659140576640084, 'one_hot_max_size': 12, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 2 with value: 0.8639277452137699.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:11:55,142]\u001b[0m Trial 5 finished with value: 0.8599896908158534 and parameters: {'iterations': 100, 'learning_rate': 0.21103536986772822, 'random_strength': 4, 'bagging_temperature': 4, 'max_bin': 6, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 6, 'max_depth': 4, 'l2_leaf_reg': 1.5248267732768012e-08, 'one_hot_max_size': 100, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 2 with value: 0.8639277452137699.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:12:21,311]\u001b[0m Trial 6 finished with value: 0.8630183917790957 and parameters: {'iterations': 1000, 'learning_rate': 0.039222305770230614, 'random_strength': 5, 'bagging_temperature': 3, 'max_bin': 20, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 10, 'max_depth': 4, 'l2_leaf_reg': 3.544948380552023e-06, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 2 with value: 0.8639277452137699.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:12:52,729]\u001b[0m Trial 7 finished with value: 0.8620990008631028 and parameters: {'iterations': 1000, 'learning_rate': 0.045869610193466345, 'random_strength': 9, 'bagging_temperature': 10, 'max_bin': 30, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 8, 'max_depth': 5, 'l2_leaf_reg': 6.629274905463984e-06, 'one_hot_max_size': 12, 'auto_class_weights': 'Balanced'}. Best is trial 2 with value: 0.8639277452137699.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:12:57,383]\u001b[0m Trial 8 finished with value: 0.8514819320769826 and parameters: {'iterations': 1500, 'learning_rate': 0.2775338323862574, 'random_strength': 6, 'bagging_temperature': 4, 'max_bin': 4, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 9, 'max_depth': 9, 'l2_leaf_reg': 0.0010943835695463693, 'one_hot_max_size': 500, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 2 with value: 0.8639277452137699.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:13:01,061]\u001b[0m Trial 9 finished with value: 0.8591475043535439 and parameters: {'iterations': 1000, 'learning_rate': 0.16527427195333977, 'random_strength': 1, 'bagging_temperature': 9, 'max_bin': 8, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 2, 'max_depth': 5, 'l2_leaf_reg': 5.547808519611033e-05, 'one_hot_max_size': 100, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 2 with value: 0.8639277452137699.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:13:13,900]\u001b[0m Trial 10 finished with value: 0.86312465124197 and parameters: {'iterations': 500, 'learning_rate': 0.10255531532633722, 'random_strength': 3, 'bagging_temperature': 0, 'max_bin': 10, 'grow_policy': 'Depthwise', 'min_data_in_leaf': 1, 'max_depth': 2, 'l2_leaf_reg': 1.2457134741928232e-08, 'one_hot_max_size': 1024, 'auto_class_weights': 'Balanced'}. Best is trial 2 with value: 0.8639277452137699.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:13:31,547]\u001b[0m Trial 11 finished with value: 0.8616743981860878 and parameters: {'iterations': 1200, 'learning_rate': 0.11052261047321708, 'random_strength': 10, 'bagging_temperature': 1, 'max_bin': 5, 'grow_policy': 'Depthwise', 'min_data_in_leaf': 5, 'max_depth': 8, 'l2_leaf_reg': 1.5237074558155712, 'one_hot_max_size': 5, 'auto_class_weights': 'Balanced'}. Best is trial 2 with value: 0.8639277452137699.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:13:49,317]\u001b[0m Trial 12 finished with value: 0.8627903938137103 and parameters: {'iterations': 200, 'learning_rate': 0.09690271753319116, 'random_strength': 7, 'bagging_temperature': 6, 'max_bin': 5, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 6, 'max_depth': 7, 'l2_leaf_reg': 0.12547937691306765, 'one_hot_max_size': 12, 'auto_class_weights': 'Balanced'}. Best is trial 2 with value: 0.8639277452137699.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:13:59,621]\u001b[0m Trial 13 finished with value: 0.85896048235637 and parameters: {'iterations': 1200, 'learning_rate': 0.1457224504434956, 'random_strength': 10, 'bagging_temperature': 2, 'max_bin': 5, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 4, 'max_depth': 10, 'l2_leaf_reg': 0.038036172848702254, 'one_hot_max_size': 500, 'auto_class_weights': 'Balanced'}. Best is trial 2 with value: 0.8639277452137699.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:14:37,845]\u001b[0m Trial 14 finished with value: 0.8677815245165366 and parameters: {'iterations': 1500, 'learning_rate': 0.07546864165714305, 'random_strength': 8, 'bagging_temperature': 6, 'max_bin': 10, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 7, 'max_depth': 7, 'l2_leaf_reg': 72.18032173984822, 'one_hot_max_size': 10, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 14 with value: 0.8677815245165366.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:16:03,842]\u001b[0m Trial 15 finished with value: 0.8634974959527698 and parameters: {'iterations': 1500, 'learning_rate': 0.011350538270844507, 'random_strength': 7, 'bagging_temperature': 5, 'max_bin': 10, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 8, 'max_depth': 3, 'l2_leaf_reg': 1.7957419364299216e-07, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 14 with value: 0.8677815245165366.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 01:16:41,131]\u001b[0m Trial 16 finished with value: 0.8669783181926874 and parameters: {'iterations': 1500, 'learning_rate': 0.07114368764050794, 'random_strength': 5, 'bagging_temperature': 7, 'max_bin': 10, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 7, 'max_depth': 8, 'l2_leaf_reg': 59.30204606639954, 'one_hot_max_size': 10, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 14 with value: 0.8677815245165366.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:17:13,681]\u001b[0m Trial 17 finished with value: 0.8658613573184872 and parameters: {'iterations': 1500, 'learning_rate': 0.07916430077690408, 'random_strength': 3, 'bagging_temperature': 8, 'max_bin': 10, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 10, 'max_depth': 10, 'l2_leaf_reg': 43.272305097041944, 'one_hot_max_size': 10, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 14 with value: 0.8677815245165366.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:17:26,100]\u001b[0m Trial 18 finished with value: 0.865327007791503 and parameters: {'iterations': 1500, 'learning_rate': 0.1444241922111979, 'random_strength': 5, 'bagging_temperature': 10, 'max_bin': 10, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 8, 'max_depth': 8, 'l2_leaf_reg': 7.251031771279269, 'one_hot_max_size': 10, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 14 with value: 0.8677815245165366.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:17:53,798]\u001b[0m Trial 19 finished with value: 0.8568209709960097 and parameters: {'iterations': 200, 'learning_rate': 0.014674758092745856, 'random_strength': 8, 'bagging_temperature': 7, 'max_bin': 8, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 7, 'max_depth': 8, 'l2_leaf_reg': 84.11021785041464, 'one_hot_max_size': 10, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 14 with value: 0.8677815245165366.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 01:18:07,950]\u001b[0m Trial 20 finished with value: 0.8552130418981457 and parameters: {'iterations': 300, 'learning_rate': 0.07383261224712181, 'random_strength': 4, 'bagging_temperature': 7, 'max_bin': 6, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 9, 'max_depth': 9, 'l2_leaf_reg': 0.010192014938373961, 'one_hot_max_size': 5, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 14 with value: 0.8677815245165366.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:18:45,197]\u001b[0m Trial 21 finished with value: 0.8660425278086379 and parameters: {'iterations': 1500, 'learning_rate': 0.07876046202849701, 'random_strength': 3, 'bagging_temperature': 8, 'max_bin': 10, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 10, 'max_depth': 10, 'l2_leaf_reg': 66.09221125080096, 'one_hot_max_size': 10, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 14 with value: 0.8677815245165366.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:18:56,901]\u001b[0m Trial 22 finished with value: 0.8612015207041264 and parameters: {'iterations': 1500, 'learning_rate': 0.12711633459157837, 'random_strength': 3, 'bagging_temperature': 9, 'max_bin': 10, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 7, 'max_depth': 9, 'l2_leaf_reg': 3.402512143664688, 'one_hot_max_size': 10, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 14 with value: 0.8677815245165366.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:19:18,772]\u001b[0m Trial 23 finished with value: 0.8659444313076863 and parameters: {'iterations': 500, 'learning_rate': 0.07656572504789284, 'random_strength': 4, 'bagging_temperature': 6, 'max_bin': 4, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 9, 'max_depth': 7, 'l2_leaf_reg': 49.973740114421126, 'one_hot_max_size': 10, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 14 with value: 0.8677815245165366.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:19:50,888]\u001b[0m Trial 24 finished with value: 0.8586495002556218 and parameters: {'iterations': 1500, 'learning_rate': 0.03245113001722131, 'random_strength': 2, 'bagging_temperature': 8, 'max_bin': 10, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 10, 'max_depth': 10, 'l2_leaf_reg': 0.24307125645822508, 'one_hot_max_size': 10, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 14 with value: 0.8677815245165366.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:19:58,340]\u001b[0m Trial 25 finished with value: 0.865411807281529 and parameters: {'iterations': 1500, 'learning_rate': 0.17650748421269213, 'random_strength': 5, 'bagging_temperature': 9, 'max_bin': 10, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 7, 'max_depth': 6, 'l2_leaf_reg': 9.654947043651513, 'one_hot_max_size': 1024, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 14 with value: 0.8677815245165366.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:20:13,053]\u001b[0m Trial 26 finished with value: 0.8631789938825676 and parameters: {'iterations': 1500, 'learning_rate': 0.08906730961663088, 'random_strength': 2, 'bagging_temperature': 7, 'max_bin': 10, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 8, 'max_depth': 8, 'l2_leaf_reg': 1.540660461935242, 'one_hot_max_size': 10, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 14 with value: 0.8677815245165366.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:20:33,407]\u001b[0m Trial 27 finished with value: 0.8645055104418322 and parameters: {'iterations': 1500, 'learning_rate': 0.1307998015265274, 'random_strength': 7, 'bagging_temperature': 6, 'max_bin': 10, 'grow_policy': 'Depthwise', 'min_data_in_leaf': 9, 'max_depth': 9, 'l2_leaf_reg': 61.61099781108686, 'one_hot_max_size': 10, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 14 with value: 0.8677815245165366.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:20:54,039]\u001b[0m Trial 28 finished with value: 0.8604788740772078 and parameters: {'iterations': 200, 'learning_rate': 0.02883948957336234, 'random_strength': 6, 'bagging_temperature': 8, 'max_bin': 8, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 5, 'max_depth': 7, 'l2_leaf_reg': 99.11999661303703, 'one_hot_max_size': 10, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 14 with value: 0.8677815245165366.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 01:21:18,084]\u001b[0m Trial 29 finished with value: 0.8678425816294796 and parameters: {'iterations': 300, 'learning_rate': 0.06497113989042527, 'random_strength': 2, 'bagging_temperature': 8, 'max_bin': 10, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 4, 'max_depth': 6, 'l2_leaf_reg': 14.608689376691892, 'one_hot_max_size': 10, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:21:36,338]\u001b[0m Trial 30 finished with value: 0.866345342440848 and parameters: {'iterations': 300, 'learning_rate': 0.05721513400326056, 'random_strength': 8, 'bagging_temperature': 5, 'max_bin': 6, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 4, 'max_depth': 6, 'l2_leaf_reg': 11.867862081465056, 'one_hot_max_size': 1024, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 01:21:54,677]\u001b[0m Trial 31 finished with value: 0.8666377100088573 and parameters: {'iterations': 300, 'learning_rate': 0.06044247755774006, 'random_strength': 8, 'bagging_temperature': 5, 'max_bin': 6, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 4, 'max_depth': 6, 'l2_leaf_reg': 15.041539218385608, 'one_hot_max_size': 1024, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 01:22:14,735]\u001b[0m Trial 32 finished with value: 0.8396449797115549 and parameters: {'iterations': 300, 'learning_rate': 0.0010484157152135937, 'random_strength': 9, 'bagging_temperature': 6, 'max_bin': 6, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 3, 'max_depth': 6, 'l2_leaf_reg': 0.7309516629009586, 'one_hot_max_size': 1024, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 01:22:34,097]\u001b[0m Trial 33 finished with value: 0.8666151562693842 and parameters: {'iterations': 300, 'learning_rate': 0.06467637781292747, 'random_strength': 9, 'bagging_temperature': 6, 'max_bin': 6, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 3, 'max_depth': 5, 'l2_leaf_reg': 12.597336631370117, 'one_hot_max_size': 1024, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 01:22:44,169]\u001b[0m Trial 34 finished with value: 0.8648758538435553 and parameters: {'iterations': 300, 'learning_rate': 0.11969371427319658, 'random_strength': 7, 'bagging_temperature': 3, 'max_bin': 30, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 4, 'max_depth': 7, 'l2_leaf_reg': 3.576919784231605, 'one_hot_max_size': 1024, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:23:12,340]\u001b[0m Trial 35 finished with value: 0.859091682907307 and parameters: {'iterations': 300, 'learning_rate': 0.01922536188539828, 'random_strength': 8, 'bagging_temperature': 7, 'max_bin': 20, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 5, 'max_depth': 6, 'l2_leaf_reg': 0.01110740126745361, 'one_hot_max_size': 5, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 01:23:31,336]\u001b[0m Trial 36 finished with value: 0.863158373108166 and parameters: {'iterations': 300, 'learning_rate': 0.055530577898201136, 'random_strength': 9, 'bagging_temperature': 4, 'max_bin': 10, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 6, 'max_depth': 7, 'l2_leaf_reg': 0.2999542912425356, 'one_hot_max_size': 100, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:23:40,244]\u001b[0m Trial 37 finished with value: 0.860102292586526 and parameters: {'iterations': 100, 'learning_rate': 0.09819662809204062, 'random_strength': 10, 'bagging_temperature': 5, 'max_bin': 6, 'grow_policy': 'Depthwise', 'min_data_in_leaf': 2, 'max_depth': 5, 'l2_leaf_reg': 16.96570606441989, 'one_hot_max_size': 10, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 01:24:11,155]\u001b[0m Trial 38 finished with value: 0.8613987943400353 and parameters: {'iterations': 300, 'learning_rate': 0.04351625872521103, 'random_strength': 6, 'bagging_temperature': 7, 'max_bin': 4, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 3, 'max_depth': 7, 'l2_leaf_reg': 0.050190944316327105, 'one_hot_max_size': 10, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:24:15,230]\u001b[0m Trial 39 finished with value: 0.8626228201741094 and parameters: {'iterations': 100, 'learning_rate': 0.2874658409563674, 'random_strength': 8, 'bagging_temperature': 5, 'max_bin': 10, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 5, 'max_depth': 4, 'l2_leaf_reg': 2.0117436612164323, 'one_hot_max_size': 500, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:24:47,028]\u001b[0m Trial 40 finished with value: 0.8664808857607789 and parameters: {'iterations': 500, 'learning_rate': 0.057808600250280534, 'random_strength': 2, 'bagging_temperature': 3, 'max_bin': 20, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 6, 'max_depth': 8, 'l2_leaf_reg': 32.05231185945637, 'one_hot_max_size': 12, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:25:05,048]\u001b[0m Trial 41 finished with value: 0.8669707847344054 and parameters: {'iterations': 300, 'learning_rate': 0.06575788966724784, 'random_strength': 9, 'bagging_temperature': 6, 'max_bin': 6, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 3, 'max_depth': 5, 'l2_leaf_reg': 12.612382300234481, 'one_hot_max_size': 1024, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 01:25:15,519]\u001b[0m Trial 42 finished with value: 0.8559329123945962 and parameters: {'iterations': 300, 'learning_rate': 0.08646516140561726, 'random_strength': 9, 'bagging_temperature': 6, 'max_bin': 6, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 2, 'max_depth': 6, 'l2_leaf_reg': 6.224141167211693e-05, 'one_hot_max_size': 1024, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:25:32,205]\u001b[0m Trial 43 finished with value: 0.8657036872924841 and parameters: {'iterations': 300, 'learning_rate': 0.06574071488565231, 'random_strength': 7, 'bagging_temperature': 7, 'max_bin': 6, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 3, 'max_depth': 5, 'l2_leaf_reg': 4.991483373522689, 'one_hot_max_size': 1024, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:26:13,627]\u001b[0m Trial 44 finished with value: 0.8662470254221912 and parameters: {'iterations': 1000, 'learning_rate': 0.040159802373041564, 'random_strength': 10, 'bagging_temperature': 4, 'max_bin': 6, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 4, 'max_depth': 4, 'l2_leaf_reg': 27.169675005679508, 'one_hot_max_size': 1024, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:26:24,899]\u001b[0m Trial 45 finished with value: 0.8656305113146493 and parameters: {'iterations': 300, 'learning_rate': 0.10749441242004769, 'random_strength': 1, 'bagging_temperature': 9, 'max_bin': 30, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 1, 'max_depth': 5, 'l2_leaf_reg': 0.8618436335502659, 'one_hot_max_size': 100, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:28:15,851]\u001b[0m Trial 46 finished with value: 0.8531606281496948 and parameters: {'iterations': 1200, 'learning_rate': 0.0016337710519915005, 'random_strength': 8, 'bagging_temperature': 6, 'max_bin': 6, 'grow_policy': 'Depthwise', 'min_data_in_leaf': 4, 'max_depth': 6, 'l2_leaf_reg': 16.87497059013893, 'one_hot_max_size': 1024, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 01:28:33,565]\u001b[0m Trial 47 finished with value: 0.8575190730336326 and parameters: {'iterations': 300, 'learning_rate': 0.026849174892958155, 'random_strength': 9, 'bagging_temperature': 4, 'max_bin': 10, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 5, 'max_depth': 4, 'l2_leaf_reg': 4.7861759394234475, 'one_hot_max_size': 1024, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 01:29:03,448]\u001b[0m Trial 48 finished with value: 0.8661831297129355 and parameters: {'iterations': 300, 'learning_rate': 0.0483092253795715, 'random_strength': 10, 'bagging_temperature': 5, 'max_bin': 5, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 2, 'max_depth': 7, 'l2_leaf_reg': 92.0737694176115, 'one_hot_max_size': 10, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 01:29:08,867]\u001b[0m Trial 49 finished with value: 0.8617732182712139 and parameters: {'iterations': 1000, 'learning_rate': 0.2457867258250544, 'random_strength': 6, 'bagging_temperature': 7, 'max_bin': 10, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 6, 'max_depth': 3, 'l2_leaf_reg': 0.0002072843918850602, 'one_hot_max_size': 12, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:29:20,439]\u001b[0m Trial 50 finished with value: 0.8629960013224526 and parameters: {'iterations': 1200, 'learning_rate': 0.09511859430997442, 'random_strength': 8, 'bagging_temperature': 8, 'max_bin': 8, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 7, 'max_depth': 5, 'l2_leaf_reg': 0.3421045550335927, 'one_hot_max_size': 500, 'auto_class_weights': 'Balanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:29:39,472]\u001b[0m Trial 51 finished with value: 0.8668158112176615 and parameters: {'iterations': 300, 'learning_rate': 0.07062031273292821, 'random_strength': 9, 'bagging_temperature': 6, 'max_bin': 6, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 3, 'max_depth': 6, 'l2_leaf_reg': 21.87454777612621, 'one_hot_max_size': 1024, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:29:55,660]\u001b[0m Trial 52 finished with value: 0.8664607421859725 and parameters: {'iterations': 300, 'learning_rate': 0.06944652884369705, 'random_strength': 9, 'bagging_temperature': 6, 'max_bin': 6, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 3, 'max_depth': 6, 'l2_leaf_reg': 28.59467519636285, 'one_hot_max_size': 1024, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:30:07,682]\u001b[0m Trial 53 finished with value: 0.8641865640283596 and parameters: {'iterations': 300, 'learning_rate': 0.08592864607113056, 'random_strength': 8, 'bagging_temperature': 5, 'max_bin': 6, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 4, 'max_depth': 7, 'l2_leaf_reg': 1.9580074436866481, 'one_hot_max_size': 1024, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:30:35,096]\u001b[0m Trial 54 finished with value: 0.8656570707542347 and parameters: {'iterations': 300, 'learning_rate': 0.052775160268300654, 'random_strength': 9, 'bagging_temperature': 8, 'max_bin': 6, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 2, 'max_depth': 6, 'l2_leaf_reg': 88.33138628560816, 'one_hot_max_size': 5, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 01:30:45,751]\u001b[0m Trial 55 finished with value: 0.8635528771721864 and parameters: {'iterations': 1500, 'learning_rate': 0.1128658933245675, 'random_strength': 7, 'bagging_temperature': 7, 'max_bin': 6, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 3, 'max_depth': 8, 'l2_leaf_reg': 6.90888861613466, 'one_hot_max_size': 1024, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:31:12,799]\u001b[0m Trial 56 finished with value: 0.8669801065862369 and parameters: {'iterations': 500, 'learning_rate': 0.07040880033205815, 'random_strength': 10, 'bagging_temperature': 6, 'max_bin': 4, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 1, 'max_depth': 7, 'l2_leaf_reg': 27.638497307480904, 'one_hot_max_size': 10, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:31:41,710]\u001b[0m Trial 57 finished with value: 0.8663983142530369 and parameters: {'iterations': 500, 'learning_rate': 0.0749542552743055, 'random_strength': 10, 'bagging_temperature': 6, 'max_bin': 4, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 1, 'max_depth': 8, 'l2_leaf_reg': 95.62573391802724, 'one_hot_max_size': 10, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:32:20,866]\u001b[0m Trial 58 finished with value: 0.867449205018185 and parameters: {'iterations': 500, 'learning_rate': 0.03933729305804938, 'random_strength': 5, 'bagging_temperature': 7, 'max_bin': 4, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 7, 'max_depth': 7, 'l2_leaf_reg': 37.047196469857376, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:33:04,616]\u001b[0m Trial 59 finished with value: 0.8673183022875552 and parameters: {'iterations': 500, 'learning_rate': 0.039286990898770816, 'random_strength': 5, 'bagging_temperature': 8, 'max_bin': 4, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 7, 'max_depth': 7, 'l2_leaf_reg': 38.201855811730915, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:33:52,639]\u001b[0m Trial 60 finished with value: 0.8668460823761241 and parameters: {'iterations': 500, 'learning_rate': 0.03548616072871777, 'random_strength': 5, 'bagging_temperature': 9, 'max_bin': 4, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 7, 'max_depth': 7, 'l2_leaf_reg': 36.48309499228472, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:34:48,020]\u001b[0m Trial 61 finished with value: 0.8611874556520476 and parameters: {'iterations': 500, 'learning_rate': 0.016323850939847694, 'random_strength': 4, 'bagging_temperature': 8, 'max_bin': 4, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 7, 'max_depth': 7, 'l2_leaf_reg': 6.986172387501033e-08, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 01:35:28,094]\u001b[0m Trial 62 finished with value: 0.8665936191821061 and parameters: {'iterations': 500, 'learning_rate': 0.04539205557019281, 'random_strength': 5, 'bagging_temperature': 7, 'max_bin': 4, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 8, 'max_depth': 8, 'l2_leaf_reg': 97.23599341407251, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:36:21,392]\u001b[0m Trial 63 finished with value: 0.8665261657467749 and parameters: {'iterations': 500, 'learning_rate': 0.02530064962087815, 'random_strength': 4, 'bagging_temperature': 10, 'max_bin': 4, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 8, 'max_depth': 7, 'l2_leaf_reg': 7.5965752847674475, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 01:36:47,020]\u001b[0m Trial 64 finished with value: 0.8666619586728765 and parameters: {'iterations': 500, 'learning_rate': 0.0833181566870092, 'random_strength': 5, 'bagging_temperature': 8, 'max_bin': 4, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 7, 'max_depth': 8, 'l2_leaf_reg': 53.03243237650822, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:37:32,972]\u001b[0m Trial 65 finished with value: 0.86674779076699 and parameters: {'iterations': 500, 'learning_rate': 0.03746715273128324, 'random_strength': 5, 'bagging_temperature': 7, 'max_bin': 4, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 6, 'max_depth': 7, 'l2_leaf_reg': 2.489186839417331, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:37:52,813]\u001b[0m Trial 66 finished with value: 0.8658020352529542 and parameters: {'iterations': 500, 'learning_rate': 0.09382508464078353, 'random_strength': 4, 'bagging_temperature': 8, 'max_bin': 4, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 7, 'max_depth': 8, 'l2_leaf_reg': 43.552771859523325, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:40:40,790]\u001b[0m Trial 67 finished with value: 0.86468636497787 and parameters: {'iterations': 1500, 'learning_rate': 0.006502367236263101, 'random_strength': 3, 'bagging_temperature': 9, 'max_bin': 10, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 8, 'max_depth': 9, 'l2_leaf_reg': 0.7955229093286997, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 01:41:02,367]\u001b[0m Trial 68 finished with value: 0.8622102816453536 and parameters: {'iterations': 200, 'learning_rate': 0.05218725775094241, 'random_strength': 6, 'bagging_temperature': 7, 'max_bin': 4, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 8, 'max_depth': 9, 'l2_leaf_reg': 8.363933861372319, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 01:41:25,705]\u001b[0m Trial 69 finished with value: 0.8638070728747298 and parameters: {'iterations': 500, 'learning_rate': 0.06280465408667814, 'random_strength': 4, 'bagging_temperature': 6, 'max_bin': 10, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 6, 'max_depth': 7, 'l2_leaf_reg': 0.107087351365461, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:41:43,145]\u001b[0m Trial 70 finished with value: 0.865365069068625 and parameters: {'iterations': 1500, 'learning_rate': 0.1060736968948093, 'random_strength': 6, 'bagging_temperature': 7, 'max_bin': 4, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 6, 'max_depth': 8, 'l2_leaf_reg': 16.451293586736217, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:42:30,052]\u001b[0m Trial 71 finished with value: 0.8674237822222797 and parameters: {'iterations': 500, 'learning_rate': 0.03529417580748597, 'random_strength': 5, 'bagging_temperature': 9, 'max_bin': 4, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 7, 'max_depth': 7, 'l2_leaf_reg': 40.587072692328164, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:43:24,471]\u001b[0m Trial 72 finished with value: 0.8650571226619057 and parameters: {'iterations': 500, 'learning_rate': 0.023709857025305134, 'random_strength': 5, 'bagging_temperature': 9, 'max_bin': 4, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 7, 'max_depth': 7, 'l2_leaf_reg': 98.30702337599311, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 01:44:03,373]\u001b[0m Trial 73 finished with value: 0.8676217390224592 and parameters: {'iterations': 500, 'learning_rate': 0.04491337085598132, 'random_strength': 5, 'bagging_temperature': 10, 'max_bin': 4, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 7, 'max_depth': 7, 'l2_leaf_reg': 46.74564109440635, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:44:46,852]\u001b[0m Trial 74 finished with value: 0.8667797625609929 and parameters: {'iterations': 500, 'learning_rate': 0.03558468864586682, 'random_strength': 5, 'bagging_temperature': 10, 'max_bin': 4, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 7, 'max_depth': 7, 'l2_leaf_reg': 40.60942754942078, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:45:41,617]\u001b[0m Trial 75 finished with value: 0.8593637412211121 and parameters: {'iterations': 500, 'learning_rate': 0.010391092449489485, 'random_strength': 6, 'bagging_temperature': 10, 'max_bin': 4, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 7, 'max_depth': 7, 'l2_leaf_reg': 4.047290427067444, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 01:46:21,234]\u001b[0m Trial 76 finished with value: 0.8671362681777449 and parameters: {'iterations': 500, 'learning_rate': 0.04086994270605372, 'random_strength': 4, 'bagging_temperature': 8, 'max_bin': 4, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 8, 'max_depth': 6, 'l2_leaf_reg': 59.493710690805, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:46:55,124]\u001b[0m Trial 77 finished with value: 0.8667472183625471 and parameters: {'iterations': 500, 'learning_rate': 0.04427886189316924, 'random_strength': 4, 'bagging_temperature': 9, 'max_bin': 4, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 9, 'max_depth': 6, 'l2_leaf_reg': 23.5280186283757, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:47:53,169]\u001b[0m Trial 78 finished with value: 0.8669730181592294 and parameters: {'iterations': 500, 'learning_rate': 0.02932628325544484, 'random_strength': 5, 'bagging_temperature': 8, 'max_bin': 4, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 8, 'max_depth': 7, 'l2_leaf_reg': 56.78883345071473, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 01:48:44,801]\u001b[0m Trial 79 finished with value: 0.8650717377014763 and parameters: {'iterations': 500, 'learning_rate': 0.019586865455170827, 'random_strength': 3, 'bagging_temperature': 10, 'max_bin': 4, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 9, 'max_depth': 6, 'l2_leaf_reg': 7.58242311844949, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 01:49:10,066]\u001b[0m Trial 80 finished with value: 0.8659094227754416 and parameters: {'iterations': 500, 'learning_rate': 0.0498020849674778, 'random_strength': 2, 'bagging_temperature': 9, 'max_bin': 4, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 8, 'max_depth': 6, 'l2_leaf_reg': 1.4094201395240415, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:49:41,440]\u001b[0m Trial 81 finished with value: 0.8675618102454917 and parameters: {'iterations': 500, 'learning_rate': 0.05888777250291839, 'random_strength': 5, 'bagging_temperature': 8, 'max_bin': 10, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 7, 'max_depth': 7, 'l2_leaf_reg': 97.22765815034077, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:50:13,916]\u001b[0m Trial 82 finished with value: 0.8675683248623012 and parameters: {'iterations': 500, 'learning_rate': 0.05779094756662237, 'random_strength': 5, 'bagging_temperature': 8, 'max_bin': 20, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 6, 'max_depth': 7, 'l2_leaf_reg': 96.88458080208805, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:50:44,369]\u001b[0m Trial 83 finished with value: 0.8666454182905474 and parameters: {'iterations': 500, 'learning_rate': 0.05910590568694325, 'random_strength': 5, 'bagging_temperature': 8, 'max_bin': 20, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 6, 'max_depth': 7, 'l2_leaf_reg': 64.89190932348808, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:51:26,747]\u001b[0m Trial 84 finished with value: 0.867261097195615 and parameters: {'iterations': 500, 'learning_rate': 0.03642422569895594, 'random_strength': 4, 'bagging_temperature': 8, 'max_bin': 20, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 7, 'max_depth': 6, 'l2_leaf_reg': 91.24137613598603, 'one_hot_max_size': 100, 'auto_class_weights': 'Balanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 01:51:35,360]\u001b[0m Trial 85 finished with value: 0.8527332808427357 and parameters: {'iterations': 100, 'learning_rate': 0.03182801605328976, 'random_strength': 6, 'bagging_temperature': 9, 'max_bin': 20, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 7, 'max_depth': 7, 'l2_leaf_reg': 1.7287857312718e-06, 'one_hot_max_size': 100, 'auto_class_weights': 'Balanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 01:52:00,940]\u001b[0m Trial 86 finished with value: 0.8662952075744158 and parameters: {'iterations': 500, 'learning_rate': 0.05151359542693153, 'random_strength': 5, 'bagging_temperature': 9, 'max_bin': 20, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 6, 'max_depth': 7, 'l2_leaf_reg': 13.96803463688055, 'one_hot_max_size': 100, 'auto_class_weights': 'Balanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:52:44,421]\u001b[0m Trial 87 finished with value: 0.8536333243214228 and parameters: {'iterations': 500, 'learning_rate': 0.005587251493407559, 'random_strength': 6, 'bagging_temperature': 8, 'max_bin': 10, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 7, 'max_depth': 6, 'l2_leaf_reg': 95.20029440763818, 'one_hot_max_size': 100, 'auto_class_weights': 'Balanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 01:53:00,321]\u001b[0m Trial 88 finished with value: 0.8596507638523256 and parameters: {'iterations': 500, 'learning_rate': 0.07815821494370423, 'random_strength': 5, 'bagging_temperature': 8, 'max_bin': 20, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 5, 'max_depth': 8, 'l2_leaf_reg': 0.0051269121064137675, 'one_hot_max_size': 500, 'auto_class_weights': 'Balanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:53:09,966]\u001b[0m Trial 89 finished with value: 0.8633970617957111 and parameters: {'iterations': 500, 'learning_rate': 0.1590889669146761, 'random_strength': 4, 'bagging_temperature': 8, 'max_bin': 20, 'grow_policy': 'Depthwise', 'min_data_in_leaf': 7, 'max_depth': 7, 'l2_leaf_reg': 25.78282884239321, 'one_hot_max_size': 100, 'auto_class_weights': 'Balanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:53:19,069]\u001b[0m Trial 90 finished with value: 0.8646346311815012 and parameters: {'iterations': 1000, 'learning_rate': 0.19380306466196903, 'random_strength': 5, 'bagging_temperature': 9, 'max_bin': 20, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 6, 'max_depth': 6, 'l2_leaf_reg': 39.748623101737444, 'one_hot_max_size': 12, 'auto_class_weights': 'Balanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:54:05,047]\u001b[0m Trial 91 finished with value: 0.8672613105882643 and parameters: {'iterations': 500, 'learning_rate': 0.04159795215874046, 'random_strength': 4, 'bagging_temperature': 8, 'max_bin': 5, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 8, 'max_depth': 6, 'l2_leaf_reg': 50.824687485542164, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 29 with value: 0.8678425816294796.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:54:32,663]\u001b[0m Trial 92 finished with value: 0.8679538793376752 and parameters: {'iterations': 500, 'learning_rate': 0.05745075659543725, 'random_strength': 4, 'bagging_temperature': 8, 'max_bin': 5, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 7, 'max_depth': 6, 'l2_leaf_reg': 11.323094517862078, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 92 with value: 0.8679538793376752.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:55:03,101]\u001b[0m Trial 93 finished with value: 0.8666428357874487 and parameters: {'iterations': 500, 'learning_rate': 0.05578533217452494, 'random_strength': 4, 'bagging_temperature': 7, 'max_bin': 5, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 8, 'max_depth': 7, 'l2_leaf_reg': 10.120301361598623, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 92 with value: 0.8679538793376752.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:55:43,932]\u001b[0m Trial 94 finished with value: 0.8670971048751233 and parameters: {'iterations': 500, 'learning_rate': 0.04405810361555019, 'random_strength': 5, 'bagging_temperature': 8, 'max_bin': 5, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 7, 'max_depth': 6, 'l2_leaf_reg': 5.5094781229035705, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 92 with value: 0.8679538793376752.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:56:02,921]\u001b[0m Trial 95 finished with value: 0.8658559183114605 and parameters: {'iterations': 200, 'learning_rate': 0.06280579178193019, 'random_strength': 3, 'bagging_temperature': 10, 'max_bin': 5, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 6, 'max_depth': 5, 'l2_leaf_reg': 18.3323690397114, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 92 with value: 0.8679538793376752.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 01:57:09,999]\u001b[0m Trial 96 finished with value: 0.8669981506343409 and parameters: {'iterations': 1200, 'learning_rate': 0.023526954910059936, 'random_strength': 5, 'bagging_temperature': 9, 'max_bin': 5, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 7, 'max_depth': 7, 'l2_leaf_reg': 3.4499080041144694, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 92 with value: 0.8679538793376752.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:57:51,360]\u001b[0m Trial 97 finished with value: 0.8675828937703939 and parameters: {'iterations': 500, 'learning_rate': 0.04764629733652711, 'random_strength': 6, 'bagging_temperature': 8, 'max_bin': 5, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 8, 'max_depth': 8, 'l2_leaf_reg': 34.66747574081746, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 92 with value: 0.8679538793376752.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:58:14,173]\u001b[0m Trial 98 finished with value: 0.8654133560338433 and parameters: {'iterations': 500, 'learning_rate': 0.07884167458539679, 'random_strength': 6, 'bagging_temperature': 9, 'max_bin': 30, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 6, 'max_depth': 8, 'l2_leaf_reg': 11.606932663595112, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 92 with value: 0.8679538793376752.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:58:35,183]\u001b[0m Trial 99 finished with value: 0.8673006416018987 and parameters: {'iterations': 500, 'learning_rate': 0.06801893522755295, 'random_strength': 1, 'bagging_temperature': 7, 'max_bin': 5, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 7, 'max_depth': 8, 'l2_leaf_reg': 29.171787980364385, 'one_hot_max_size': 5, 'auto_class_weights': 'Balanced'}. Best is trial 92 with value: 0.8679538793376752.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    }
   ],
   "source": [
    "sampler = optuna.samplers.TPESampler(seed=68)  # Make the sampler behave in a deterministic way.\n",
    "study = optuna.create_study(direction=\"maximize\", sampler=sampler)\n",
    "study.optimize(objective, n_trials=100)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of finished trials: 100\n",
      "Best trial:\n",
      "  Value: 0.8679538793376752\n",
      "  Params: \n",
      "    iterations=500,\n",
      "    learning_rate=0.05745075659543725,\n",
      "    random_strength=4,\n",
      "    bagging_temperature=8,\n",
      "    max_bin=5,\n",
      "    grow_policy=Lossguide,\n",
      "    min_data_in_leaf=7,\n",
      "    max_depth=6,\n",
      "    l2_leaf_reg=11.323094517862078,\n",
      "    one_hot_max_size=10,\n",
      "    auto_class_weights=Balanced,\n"
     ]
    }
   ],
   "source": [
    "print(\"Number of finished trials: {}\".format(len(study.trials)))\n",
    "print(\"Best trial:\")\n",
    "trial = study.best_trial\n",
    "print(\"  Value: {}\".format(trial.value))\n",
    "print(\"  Params: \")\n",
    "for key, value in trial.params.items():\n",
    "    print(\"    {}={},\".format(key, value))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Check the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "final_model = CatBoostClassifier(verbose=False,  cat_features=categorical_features_indices, \n",
    "                          **trial.params)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<catboost.core.CatBoostClassifier at 0x7f87e2007bd0>"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "final_model.fit(X_train, y_train.h1n1_vaccine)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictions_h1 = final_model.predict_proba(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictions_h1 = predictions_h1[:,1].reshape(-1,1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import roc_curve, roc_auc_score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_roc(y_true, y_score, label_name, ax):\n",
    "    fpr, tpr, thresholds = roc_curve(y_true, y_score)\n",
    "    ax.plot(fpr, tpr)\n",
    "    ax.plot([0, 1], [0, 1], color='grey', linestyle='--')\n",
    "    ax.set_ylabel('TPR')\n",
    "    ax.set_xlabel('FPR')\n",
    "    ax.set_title(\n",
    "        f\"{label_name}: AUC = {roc_auc_score(y_true, y_score):.4f}\"\n",
    "    )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmEAAAHwCAYAAADuJ7gwAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAABYhUlEQVR4nO3de3zU1YH//9fJ5E4SAgkJEAiEa7gjdwUBAZWLCioXb4i3Zdtq2939Pbrt7n53u7vt7rZ76dZua62KVdsqBERAFBRFUUBBQO7XcA2ESwgkIffMzPn9MYEGTEICmXwmM+/n48HDzMxnJu9kMHlzPudzjrHWIiIiIiLNK8zpACIiIiKhSCVMRERExAEqYSIiIiIOUAkTERERcYBKmIiIiIgDVMJEREREHKASJiIiIuIAlTCRFsYYc8wYM8npHE4xxqwyxsxzOoeIyM1SCRMJIsaYSGPMkuqiZo0x4xv5/J8YY3YZY9zGmH/2S8ibZK2dYq19vbk+nzHmierv5exa7l9fy/FXlWRjzAhjzPvGmAJjzAVjzGZjzJN+yPnXxpgzxphCY8yrxpioeo6dYIzZZowpMsYcMcbMr/HYi8aY4hp/Kowxl2o8/pwxZkv1/a9d87o39fdPJNSohIkEn/XAY8CZG3huNvC3wHtNmqhlmwdcqP5voxhjbgXWAuuAHkAS8G1gSlMGNMbcDfwImAh0BboB/1LHsRHAO8DvgNbAHOAXxphBANbab1lr4y7/Ad4CFtd4iVzgp8CrdcS5mb9/IiFFJUykZRpsjNlZPeqxyBgTDWCtrbTW/tJaux7wXPskY8xrxpjfGGPeM8ZcMsZsMsZ0v/y4tfZ1a+0q4NK1z62NMSaqeoSnf4372hljyowxKcaYNsaYlcaYPGPMxeqPO9U4tq0x5vfGmNzqx5fVeGy6MWZ79WjNYWPM5Or7PzXGPFP98RPGmPXGmP+ufv5RY8yUGq/R2hizwBhz2hhzyhjzU2OMq6HfZGNMF2AcMB+42xiT2tDnVvsv4HVr7c+tteetz1Zr7ezrPrNx5gELrLV7rLUXgZ8AT9RxbFsgAfhDdZ6vgH1A32sPNMa0Ah4Erow8WmuXWmuXAfnXHn+9v38icjWVMJGWaTYwGcgABlL3L9zaPIxvlKQNvpGvf7vRENbaCmBp9WvWzLbOWnsO38+Y3wNdgHSgDPh1jWP/AMQC/YAU4H/BdwoPeAP4AZAIjAWO1RFjJHAASAb+E1hgjDHVj70OuPGNQt0C3AVcLnDp1QUyvZ4v8XFgi7X2bXxF5dF6jr2KMSYWuBVY0ojnjKnOVNefMXU8tR+wo8btHUCqMSbp2gOttWfxjW49aYxxVY/WdcE3gnWtB4E84LOGfg0i0nAqYSIt06+stbnW2gvAu8DgRjx3qbV2s7XWDfypkc+tzZtcXcIeqb4Pa22+tfZta22ptfYSvsI3DsAY0wHfablvWWsvWmurrLXrql/jaeBVa+0aa63XWnvKWru/js9/3Fr7srXWg690dcBXQFKrX/+vrLUl1aXwf4GHqrOdsNYmWmtP1PO1PX75a6n+b2NOSbbB9zP2dEOfYK1dX52prj+1FSWAOKCwxu3LH8fXcfxbwD8BFcDnwD9Ya3NqOW4e8Ia11jb0axCRhlMJE2mZas63KcX3S7g5nlubtUCMMWZk9em7wfjmHGGMiTXG/M4Yc9wYU4RvRCWx+pRgZ+BC9emza3UGDjfw81/5eqy1pdUfxuEb3YkATl8eScI3DyqlIS9qjBmNb6RxYfVdbwIDjDGDq2+7q1//WhFAFXAR8OIrhf5WjO8U42WXP/7GaWVjTCawCF/BjMQ3iva3xphp1xzXGV9hfsMfgUVEJUxEbpK11gtk4RsNewRYWT3qBfD/Ab2BkdbaBHynFQEMkAO0NcYk1vKyOUD3Wu5vjBx8Iz3JNUaSEqy1/Rr4/HnVObcbY84Am6rvf7z6vyeA9BqnPi+fgkzBNzpXCnyB75Regxhjbr/mysRr/9xex1P3AINq3B4EnLXWfmPeFtAfOGCt/aB6lPEAvgsxrr1Y4HFgo7X2SEPzi0jjqISJBJnqyfLR1TcjjTHRNYvCdZ4bUf3cMCC8+rkNmcj+Jr6r7B7lz6fvwHc6rAwoMMa0BX58+QFr7WlgFfBC9QT+CGPM5ZK2AN+cpYnGmDBjTFr1CE6DVb/+h8D/GGMSql+nuzFm3PWeW/09mI1vQv7gGn++CzxqjAnHV8rKgR9Vf59aAT8DtgDHq1/qb4EnjDE/uDw/yxgzyBizkFpYaz+veWViLX8+ryPyG8DTxpi+xpg2wP8DXqvj2K+Bnsa3TIUxvgsz7uHqOWXgK2HfeA1jTHj198cFuKq/9vAaj9/w3z+RUKMSJhJ8DuArPmnAB9Ufd2ngc1+uPv5h4B+qP557vSdZazcBJUBHfMXqsl8CMcB54Etg9TVPnYvv1N1+4BzwV9Wvtxl4Et8crkJ8Szw09Guo6fIpt734Tg8uofr0YPXE/OI6JubPwPe1v2GtPXP5D75y6AImV1+UMA0YD5wEjlR//bMvz6Gy1m4EJlT/OWKMuQC8BLx/A19Lnay1q/FdlPAJvgJ4nBqF1/gWuP376mMPA08BvwKK8H1v367+2i4ffyvQiauXprjs/+H73vwI31IUZdX3XXYzf/9EQorRfEsRERGR5qeRMBEREREHqISJyHWZb25lc/nPi05nExFpqXQ6UkRERMQBGgkTERERcUD49Q8JLMnJybZr165OxxARERG5rq1bt5631rar7bEWV8K6du3Kli1bnI4hIiIicl3GmON1PabTkSIiIiIOUAkTERERcYBKmIiIiIgDVMJEREREHKASJiIiIuIAlTARERERB6iEiYiIiDhAJUxERETEASphIiIiIg5QCRMRERFxgEqYiIiIiANUwkREREQcoBImIiIi4gCVMBEREREH+K2EGWNeNcacM8bsruNxY4z5lTEm2xiz0xgzxF9ZRERERAKNP0fCXgMm1/P4FKBn9Z/5wG/9mEVEREQkoIT764WttZ8ZY7rWc8h04A1rrQW+NMYkGmM6WGtP+yuTiIiIBDeP11Jc4W7QsZGuMGIiXX5OVDe/lbAGSANyatw+WX2fSpiIiEiAqXB7yLtUceX2Jwfy2HWygHBX80wv93ot7+08TfvW0RhT93EHzxY3+DUfG5XOT2cMaIJ0N8bJElbbt9DWeqAx8/GdsiQ9Pd2fmURERFqkM4XlXCyt5OsTBbi9Xj49kEerqG/+mv9k/zliI124wuppMrU4XVhe6/3JcVE3lLexfCfOoMLtpV/HhDqP65YcR7jLcEt6m1pfozhnH1hLfJd+9Gkf77e8DeFkCTsJdK5xuxOQW9uB1tqXgJcAhg0bVmtRExERCSaXS4fXwu5ThWw9fpGyKg8AF0sqeWX9UaLCw4hwhVFe5cHtrf3XY7fkVlfdTo6LpKzKw+09kxuZB6IiwhjYKbH6Duif1pq+9RSiQFJRUcGKFSs4dWgvffv2Zeborpj6htSagZMlbAXwnDFmITASKNR8MBERCSUXSyrZcvwiBvjTpuOcLaogOiKMbScKGvT8lIQo7urbHoCyKg99OiSQ3CqS/mmtaRUVTpvYCMeLRiDIy8sjKyuL/Px8Jk2axG233RYQ3xe/lTBjzFvAeCDZGHMS+DEQAWCtfRF4H5gKZAOlwJP+yiIiIuKU04VlvPDJYY7ll3D4XDGtosKvzGmqbf7S8K5tGNMjmaLyKiZkpgBQWulhbM929GofR2JMJABhhmabj9WSlZWVsWDBAsLDw5k7dy4ZGRlOR7rCn1dHPnydxy3wrL8+v4iISFOq8njJPleMrXHWb9/pIo6cL2bf6UtX5l+dLSxn56kCEqIjcIWZb8ylGtMjmfho37HdkuOIiXTx1OgMjIHM9vEqVk3EWosxhpiYGKZNm0aXLl1ISAisU6dOno4UEREJGN5a5lQdyy8h71IFf/fOLo7kldT7/HbxUcRHheO1lugIF/3TWpMcF4nba0lvG8u3xnUnOsK55RBCSXFxMW+//Ta33XYbPXv2ZMAA566ArI9KmIiIhAxrLUVlbpbvOMWpgjJW7TpDUlwkXzdgDlZ0RBgPj0hnZEZSzVekR0ocHRNjiI3Ur9RAcOLECRYvXkx5eTmVlZVOx6mX/saIiEhQKq/yrWtVUunm6de2EBPpIvvcN+dg5V2q4PaeyRRXuBnfK+Wqxyo9HnqlxpMSH83gzomOLuwp9bPWsmnTJtasWUNiYiKPPfYYqampTseql0qYiIi0CPnFFfzw7V1ER9Q9Z6rC7WXN3rO0i4+6amHRy6YOaE9FlZfbeyYzdWAHUuKj/RlZmlF2djYffPABvXv3ZsaMGURHB/57qxImIiKOsdaSd6mCZdtPUen2fuNxt9fywZ6znLxQyqUaW9F0a9fqG8f6Xg86to6mfeto7ujdjtjIcPp1TCAm0sW0AR0CYlkCaVput5vw8HB69OjBnDlz6N27d4t5n1XCRETEr0or3WzPKaCkwrfQ6LYTF/F6LZcq3Ly56USDXiM20kVaYgwPDe/Ms3f0IKyRq71LcNq3bx+rVq3i8ccfJzk5mczMTKcjNYpKmIiINFqVx8uZa5ZeKCyr4tUNR4kK//O8qcVbcupcyT0qPIzwMMOdfVMZ1S2J+wZ1JC76m7+WDFoPS67m9Xr5+OOP2bhxI2lpaURERDgd6YaohImISK3cHi+//OgQu3MLiauxB+Hnh85TWFZV73Pbxfv2E2wdE0F+SSV/c2cvhndte2V9rB4pcVquQW5ISUkJS5Ys4dixYwwbNoy7776b8PCWWWdaZmoREblph85eYvn23Ksmunu8sHrPGVLio1h3MO/K/clxUSTE+H5ltG0VSYTLMKlPKsO6tr3qNVtFurirX/tGbw4t0lAbN27k5MmTzJgxg0GDBjkd56aohImIhKCP953l6de31Pn4kbwwBnVqTWFZFcufG0PrmJZ5ukeCg7WWsrIyYmNjueOOOxg0aBApKSnXf2KAUwkTEQlyeZcqKK/y8MmBc+w6WUi4y/DW5hwAvj2+O39zZ6+rjtccLAkklZWVrFy5klOnTjF//nyioqKCooCBSpiISND5xZqDvPHFMSJdYZyrZa2slPgo2raKZEh6G344uWVdTSahJT8/n6ysLM6dO8cdd9xBZGSk05GalEqYiEgL4Km+wvDg2Uu8uyOXmAgXR/NLOHmxjKKyKvafuUSEyxDhCqO00rcUxJxhnTEGCkqrGNMzmegIF5nt4+mf1trJL0WkQfbv38+yZcsICwvj0UcfpUePHk5HanIqYSIiAepcUTnf+uNWtl1nX8M+HRLomhTLkPQ2JFVvGH17z2QmZAb2li0idbHWsnHjRtq2bcvs2bNJTEx0OpJfqISJiASIE/mlfHk0n725RazZe5ZTBWVXHstsH8/UAR1we7wM6JTI+N7tMECYMVq4VIJGSUkJxhhiY2OZM2cOUVFRLXb5iYYI3q9MRCRAnS4sI7+4EoCTF8tY9vUpVu85843jIl1hPH5rF/5hWp8Wsw2LyI06deoUWVlZdOjQgYceeohWrWrfmiqYqISJiDSDM4XljPqPjwkPM3WuIN86JoJ/uqcvw7u2JSUhSouZSkiw1rJ161ZWr15NfHw848aNczpSs1EJExHxk3OXylm0OYeFX+VcObXYMzWeEV3b0DM1npTqVeVTEqIZmNZapxUl5FRVVfHee++xY8cOevTowQMPPEBMTIzTsZqNSpiISBN45fMjZJ8rBuDI+RJyLpRy+pq9Fb89vruWhBCpobKykqNHjzJu3DjGjRsXcqfdVcJERBpo6/GL/PKjgyRE/3n1+PXZV++jmJoQhccL54srmJCZwsBOrXnytgxax2rFeZHLjh8/TqdOnWjVqhXf+c53iIqKcjqSI1TCRETqUOn28os1B3lx3WGMAVtjKlePlDgAkuN8+ygO79qWv5/ah85tYx1KKxL4vF4v69at47PPPuPOO+/ktttuC9kCBiphIhJiqjxeisvdV923J7eIf1y+m4SYCGqeDNmeU3Dl48SYCGYO7cT43imM7pHcPGFFgkhpaSlLly7l8OHDDB48mOHDhzsdyXEqYSIStArLqljw+RFeWX+U9gnRGAOH80rqfc7YXu2u+jgmIox/v38ASXGh+691kZt1+vRpFi1aRHFxMffccw9DhgwJuflftVEJE5EWqbC0ipyLpQB8dewCAF+fKGDFjlyS43z7y52vXosLIMIVRo/UOHqlxtMqKpz+HROuPOa1vlXnb+2e1IxfgUhoCQ8P58knnyQtLc3pKAFDJUxEAkql28v2nALKqzxsOHyejdn5JF4zqb3C7WXz0Qt1vkZ621h6t4+nwu0lvW0s9w3qSLd2cf6OLiI1VFVVsW/fPgYOHEiHDh34zne+Q1hYmNOxAopKmIg4rrC0ii+P5rPoqxzW7j9X6zG3pCde+dha3zY+fTskMGVABwAGdmpNVHgYraLCiXDpB72IkwoKCsjKyuL06dOkpKTQvn17FbBaqISJiGM2Hj7P9xduJ+9SxVX3TxvQgbm3diEmwkW/jgmEq1SJtBjZ2dksXboUr9fLQw89RPv27Z2OFLBUwkSk2S3cfIK/f2cXNXfv+c747kwd0IH0pNir1uESkZZjw4YNfPTRR6SmpjJ79mzatm3rdKSAphImIn53PL+E7y3czo6cAlxhBk+N9vX7J4czvlc7XSklEgRat27NwIEDueeee4iI0D+mrkclTET8wu3x8qdNJ/jxij1X3T+qW1v6tE9gTM9kxvdOcSidiDSV06dPc/78eQYMGED//v3p37+/05FaDJUwEWlyH+09yzNvbLnqvv+ZNYgHhqRpxEskiGzfvp333nuP+Ph4+vTpQ3i4akVj6LslIjftUnkVx86XYrH8y7t72Xr8IgAdWkez7NnRpCZEO5xQRJqS2+1m1apVbNu2jYyMDB588EEVsBug75iIXJfXa3FXz+PadDSfzUcvYIzhVx8f+sYcr8vefGYkt2l7H5Gg4/F4eO211zh16hSjR49mwoQJWn7iBqmEiYS4wrIqNmafZ+epQnbkFFBQWkWEy0D1aUOv17LrVGGdz4+LCmfO8M7ERLgYkNYaY2B0j2SiI1zN9SWISDNyuVz06tWLMWPGkJmZ6XScFk0lTCSEZZ8rZtIv1n3j/tt7JhNWY+7Wbd2TaBcfRa/UeKo8XkZ1S2JEV9+l52FhmuMlEuystaxfv5709HS6dOnC2LFjnY4UFFTCREJQhdvDv7y7lzc3nQDAFWZ497kxdE2OJTZSPxZE5M/Ky8t55513OHjwICNHjqRLly5ORwoa+mkrEgLOF1dw4MwlDPDm5hOs3Hn6ymMPDEnjF7MHO5ZNRALX2bNnWbRoEYWFhUyZMoXhw4c7HSmoqISJBBlrLQWlVVS4vWzPucjvPjvC1ycKvnHcwyM68+N7+2nulojU6uzZs7zyyivExMTwxBNP0LlzZ6cjBR2VMJEgYa3lWH4p33vr61on0j85uiuT+/n2cOuS1Ir2rbVshIjULSUlhdtuu43hw4cTFxfndJygpBIm0sJZa3l/1xmefXPbVff/6/R+eL2Wkd2S6N4ujshwXUIuIvUrKipi5cqVTJ06lcTERO644w6nIwU1lTCRFiq3oIwX1x3mjS+OX7kvzMD/zB7E3f3aa4K9iDTK0aNHWbJkCW63m/z8fBITE52OFPT0U1qkBfrtp4f5+er9V273SInj14/cQmb7BAdTiUhLZK1lw4YNrF27lqSkJObMmUNyshZabg4qYSIBaufJAn69NpvE2Iir7n9/1xmKK9yAbz2vN54aof0YReSGffnll3z88cf069eP++67j8jISKcjhQyVMJEA88n+c/zFG1uubBMUHRFGm9g//1CMjfRdzbjs2dH0SNFkWRG5MdZajDEMGTKEqKgobrnlFv2DrpmphIkEkF99fIhfrDkIQGpCFH89qRcPjUh3OJWIBJtdu3axefNmHn/8caKiohgyZIjTkUKSSphIALhYUsndv/yMc5cqAPjxvX15cnSGw6lEJNh4PB4+/PBDNm/eTHp6OpWVlURERFz/ieIXKmEiDjtdWMat/7H2yu3/e/gW7h3U0cFEIhKMLl26xOLFi8nJyWHUqFFMmjQJl0uLNTtJJUzEITkXSrn9Pz+5cntgp9Ysf3a05mSIiF+sWLGCM2fO8OCDD9K/f3+n4wgqYSLNbuvxi/zru3vYcfLPq9r/5dhu/N3UPg6mEpFgZK3F4/EQHh7O1KlTcbvdtGvXzulYUk0lTKSZ/fOKPew6VUiXpFjG9mzHT2boX6Qi0vQqKipYsWIF1lpmzZpFmzZtnI4k11AJE2lGlwuYMbDuB9oORET8Iy8vj6ysLPLz85k4caLTcaQOKmEizcBay+83HOO1jccA+J9Zg5wNJCJBa8+ePSxfvpzIyEgef/xxunbt6nQkqYNKmIgfWWu5VOFm7H9+QkFpFQBvf/tWhnZp63AyEQlG5eXlvP/++7Rv356ZM2eSkKCtzAKZSpiIH7g9XtbuP8f8P2y96v43/2KkCpiINLnS0lJiYmKIjo7miSeeoG3btlp+ogVQCRNpYifySxn7X59cdd8P7u7Nt8d1JyxMy0+ISNM6ceIEixcvZvjw4YwdO1ZXP7YgKmEiTcDjtew7XcQ9/7f+yn2uMMN73xtDZnudDhCRpmetZdOmTaxZs4bExER69+7tdCRpJJUwkZtQ6fbyo6U7Wbrt1FX3P//QYO7u157oCJ0OEJGmV1lZyYoVK9izZw+9e/dmxowZREdHOx1LGkklTOQGvbX5BH+3dNeV2w8O6cS0ge25pXMb2rSKdDCZiAS7c+fOsX//fiZOnMjo0dppo6VSCRNppJIKN/P/sIUN2fkA3NotiV/MGUSH1jEOJxORYJeXl0e7du3o1KkT3//+94mPj3c6ktwElTCRRvjfNQd5/uNDV26/NHcod/Vr72AiEQkFXq+Xjz76iC+++IK5c+fSrVs3FbAgoBImch1er2XW775g6/GLV+57cnRX/ubOXsRHRziYTERCQXFxMW+//TbHjh1j2LBhpKenOx1JmohKmEgt1uw9y+/WHaawrIpD54qv3N+nQwI/uLsXEzJTHUwnIqEiJyeHxYsXU1ZWxowZMxg0SLttBBOVMJFqP1+9nxXbc6n0eMm7VHHl/nG92hEdEcbzD92iqx1FpFnl5eURHh7O008/Tfv2mvoQbFTCJOSVV3n4j/f38foXxwGYMbgjpZUe7r8ljSkDOjicTkRCTWVlJWfOnCE9PZ0hQ4bQv39/IiN1xXUwUgmTkPXv7+/jpc+OXHXf8w8NZvrgNIcSiUioy8/PJysri4KCAv7qr/6KmJgYFbAgphImIaWkws3a/efYnlPAgvVHARiQ1pqxvZJ5aHg6ndvGOpxQRELV/v37WbZsGWFhYcyePZuYGC17E+xUwiRkbMg+z6OvbLrqvv+dM4j7b+nkUCIREd/2Q2vXrmX9+vV07NiRWbNmkZiY6HQsaQYqYRL09p8p4l9W7OWLI77FVTPbx/Orh28hMTaClHht8yEizjLGUFFRwZAhQ5gyZQrh4frVHCr0TkvQyj53iUm/+OzK7dYxETw2Kp0f3J3pYCoREZ+TJ0/icrno0KEDU6ZM0dZDIUglTILS6t2n+dYftwHQr2MCD49I57FRXRxOJSLiO/24ZcsWVq9eTXp6OvPmzVMBC1F+LWHGmMnA84ALeMVa+7NrHm8N/BFIr87y39ba3/szkwS/j/edvVLA5gzrzM9nDnQ4kYiIT1VVFe+99x47duygR48ePPDAA05HEgf5rYQZY1zAb4A7gZPAV8aYFdbavTUOexbYa6291xjTDjhgjPmTtbbSX7kkeF0sqeTuX37GueqFVn84OZNvj+/ucCoREZ/i4mL++Mc/cvbsWcaNG8e4ceM0Ahbi/DkSNgLIttYeATDGLASmAzVLmAXije9vYRxwAXD7MZMEsbmvbuLcpQoiw8N4fs5gLbQqIgElNjaWtm3bMnHiRHr27Ol0HAkA/ixhaUBOjdsngZHXHPNrYAWQC8QDc6y1Xj9mkiB0urCMW/9j7ZXbB386xcE0IiJ/5vV62bhxI4MHDyYuLo7Zs2c7HUkCSJgfX7u2MVZ7ze27ge1AR2Aw8GtjTMI3XsiY+caYLcaYLXl5eU2dU1qwtzafuKqArfvBeOfCiIjUUFpayptvvsnHH3/Mrl27nI4jAcifI2Engc41bnfCN+JV05PAz6y1Fsg2xhwFMoHNNQ+y1r4EvAQwbNiwa4uchKDFW3L47brDHMkrAWBSn1RemTfM4VQiIj65ublkZWVRXFzMvffey5AhQ5yOJAHInyXsK6CnMSYDOAU8BDxyzTEngInA58aYVKA3cASRehw9X8IPluwEIC0xhv+cOZDRPZIdTiUi4pOdnc3ChQuJi4vjqaeeomPHjk5HkgDltxJmrXUbY54DPsC3RMWr1to9xphvVT/+IvAT4DVjzC58py9/aK09769M0rJZa/nrRdtZtt03oDqlf3t++9hQh1OJiFwtLS2NAQMGcOeddxIbq/1opW7Gdyaw5Rg2bJjdsmWL0zHEAR/vO8vTr/ve+6fHZPCP9/R1OJGIiM/FixdZv349U6dOxeVyOR1HAogxZqu1ttb5MloxXwJezoVSFqw/ymsbjwGwcP4oRnVLcjaUiEi1Q4cOsXTpUgCGDx9O+/btHU4kLYVKmASkCreHSreXnScLmffqZtxe34jt0C5tVMBEJCBYa1m3bh3r1q0jNTWV2bNn07ZtW6djSQuiEiYB5y/e2MKavWevuq9Xahzvfe92Ilz+XFVFRKThVq1axVdffcWgQYOYNm0aERERTkeSFkYlTALKrz4+dKWAzR/bjbiocEb3SKZ3+3gVMBEJKMOGDSMlJYWhQ4dq+yG5ISphEjBO5JfyizUHAVj1/dvp0+Eb6/aKiDjq66+/Jjc3l6lTp5KSkkJKSorTkaQFUwmTgHCqoIyx//UJAE+NzlABE5GA4na7WbVqFdu2bSMjIwO3263Tj3LTVMLEceVVHkb/7M9bD/3TvVp6QkQCR0FBAYsXLyY3N5cxY8Zwxx13EBam6RFy81TCxHHP/mkbAKkJUXz5dxMdTiMi8mder5c33niD0tJS5syZQ2ZmptORJIiohImjLpZU8vH+cwBs/NFETW4VkYBweSHzsLAwpk2bRmJiIklJWh5HmpbGU8Ux54rKueUnawAY3DkRV5gKmIg4r7y8nIULF7Jp0yYAunfvrgImfqGRMHHEthMXeeCFjQD0T0sg6y9vdTiRiAicOXOGrKwsCgsL6dGjh9NxJMiphEmzO3mx9EoBG9MjmT8+M9LhRCIisGPHDlauXElMTAxPPPEEnTt3djqSBDmVMGl2z1Rvwv3IyHT+/f4BDqcREYG8vDyWLVtG165defDBB4mLi3M6koQAlTBpNm9uOsHfv7Pryu3/N62Pg2lERKCqqoqIiAjatWvH3Llz6dq1q5afkGajv2nid+culfPQS19cKWC390xm/Q/vIDZS/wYQEeccOXKE559/nsOHDwPQrVs3FTBpVvotKH419fnP2Xu66Mrtn87oz2OjujiYSERCnbWWDRs2sHbtWpKTk2ndurXTkSREqYSJ35wvrrhSwH46oz9TB3SgbatIh1OJSCgrLy9n+fLl7N+/n379+nHfffcRGamfS+IMlTDxi5+v3s9vP/UN8X9/Yk+NfolIQNi9ezcHDx7k7rvvZuTIkVogWhylEiZNrqi86koBG9urHc9N0Fo7IuKsS5cuER8fz9ChQ0lPTyclJcXpSCKamC9Nq7TSzcB//hCAu/ul8sZTI4hw6a+ZiDjD4/Hw/vvv88ILL1BQUIAxRgVMAoZGwqTJbDqSz5yXvrxy+8XHhjqYRkRCXVFREYsXL+bkyZOMGjWK+Ph4pyOJXEUlTG5aYWkVg/71wyu3O7aO5rO/vUNzLUTEMceOHWPJkiVUVlYyc+ZM+vXr53QkkW9QCZObsie3kGm/Wn/l9jvfuY1b0ts4mEhExLcFUXR0NPPmzaNdu3ZOxxGplUqY3LDCsqqrCtjR/5iq0S8RcUxFRQWlpaW0adOGqVOn4vV6iYqKcjqWSJ00Y1pu2L3/5ytgQ7u04djPpqmAiYhj8vLyePnll1m4cCFer5eIiAgVMAl4GgmTG/LJ/nOcuFAKwJJv3epwGhEJZbt372bFihVERkYyc+ZMbT0kLYZKmDTaF4fzefK1rwB45fFhGgETEUd4PB7WrFnDpk2b6Ny5MzNnziQhIcHpWCINphImjfbwy75lKKb0b8+kvqkOpxGRUGWtJScnhxEjRnDXXXfhcrmcjiTSKCph0ihr958FoFu7VvxW64CJiANycnJITk4mJiaGJ554goiICKcjidwQnTiXBql0e8naksNTr20B4BezBzsbSERCjrWWL774gt///vesXbsWQAVMWjSNhEmDTPrFuisT8acP7sjgzonOBhKRkFJZWcmKFSvYs2cPmZmZTJw40elIIjdNJUyu69j5kisF7LMf3EF6UqzDiUQklFy4cIG33nqL/Px8Jk6cyOjRo3VBkAQFlTCpV2FZFeP/+1MAfjF7kAqYiDS7iIgIXC4Xc+fOJSMjw+k4Ik1Gc8KkXk9VL0XRqU0M9wzs6HAaEQkVXq+XLVu24PV6iY+P5y//8i9VwCToaCRM6vTK50fYevwiAKu+fzuR4ersIuJ/xcXFvP322xw7doy4uDgyMzN1+lGCkkqYfIO1ll99nM3/fnQQgJXfHUN8tK5AEhH/y8nJYfHixZSVlTFjxgwyMzOdjiTiNyph8g3/9cEBXvj0MAD/dE9f+qe1djiRiISC7du38+6779K6dWuefvpp2rdv73QkEb9SCZOrlFS4rxSwzX8/kZSEaIcTiUioaNeuHZmZmdx7771ER+tnjwQ/TfKRq7z8+REA0tvGqoCJiN/l5+fzxRdfAJCWlsasWbNUwCRkaCRMrrJiey4Ay58d7XASEQl2+/btY/ny5bhcLgYOHEirVq2cjiTSrFTC5IqdJws4cr6E1jERtGkV6XQcEQlSXq+XtWvXsmHDBjp27Mjs2bNVwCQkqYQJAL/99DA/X70fgMdGpTucRkSClbWWRYsWcfDgQYYOHcrkyZMJD9evIglN+psv/PHL41cK2PMPDWb64DSHE4lIsDLG0L9/fzIzM7nlllucjiPiKJWwEJdzoZT/t2w3AAvnj2JUtySHE4lIsLHWsmXLFiIjIxk0aBADBgxwOpJIQFAJC1GvbTjKP7+798rtHilxKmAi0uSqqqpYuXIlO3fupE+fPgwcOFCr34tUUwkLQcUV7isFrF/HBGYO7cSTo7Unm4g0rQsXLpCVlcXZs2cZP348Y8eOVQETqUElLAT9Y/Xpx0dGpvPv9+u0gIg0veLiYl5++WWMMTz66KP06NHD6UgiAUclLMQcOnuJd74+BcA/TuvrcBoRCVZxcXHccccd9OrVi8TERKfjiAQkrZgfYr79p20AzLu1CzGRLofTiEgwKS0t5a233uLkyZMAjBgxQgVMpB4qYSGkqLyK7HPFAPzL9P4OpxGRYHLq1CleeuklDh8+zMWLF52OI9Ii6HRkiNh6/CIP/nYjAE/c1tXZMCISNKy1bNu2jVWrVhEXF8dTTz1Fx44dnY4l0iKohIWAf1y2mz98eRyAyf3a84/3aC6YiDSN/fv3s3LlSrp3784DDzxAbGys05FEWgyVsCD3+aG8KwXs3+7vz6MjuzicSESCgdfrJSwsjN69ezN9+nQGDhxIWJhmuIg0hv6PCWL/uGw3cxdsBuDZO7qrgIlIkzh06BC//e1vKSoqIiwsjMGDB6uAidwAjYQFscsjYG88NYKxvdo5nEZEWjqv18u6dev47LPPSE1NxePxOB1JpEVTCQtSlwvYyIy2KmAictPKyspYunQp2dnZDBo0iGnTphEREeF0LJEWTSUsCK0/dP7Kqvg/vrefw2lEJBisXbuWo0ePMm3aNIYOHarth0SagEpYkDlTWM5jCzYBMKlPKn07JjicSERassrKSiIjI5k4cSK33HKLlp8QaUKaSRlklm33bUn015N68cq8YQ6nEZGWyu12s2LFCl577TXcbjfR0dEqYCJNTCNhQSS/uIKfrdoPwF+MzXA4jYi0VAUFBWRlZXH69GnGjBmjKx9F/EQlLEgUV7gZ+tOPAOjWrhWxkXprRaTxsrOzWbp0KV6vl4ceeojevXs7HUkkaOk3dRDweC39f/wBAB1aR7P2/xvvbCARaZG8Xi8fffQR8fHxzJ49m6SkJKcjiQQ1lbAWzlrLgvVHrtze+KMJDqYRkZaorKwMl8tFZGQkDz/8MDExMURGRjodSyToqYS1cH//zi7e2pwDwOZ/mKjLxkWkUc6cOUNWVhbp6enMmDGD1q1bOx1JJGSohLVgbo/3SgF78y9GkhIf7XAiEWlJduzYwcqVK4mJiWHo0KFOxxEJOSphLdifNp0AYGCn1tzWPdnhNCLSUrjdblavXs3WrVvp2rUrM2fOpFWrVk7HEgk5KmEt1IWSSn68Yg8Av3lkiMNpRKQlKSkpYe/evYwePZoJEyZoCQoRh6iEtVDD/823HMWIjLZ0bhvrcBoRaQlOnz5N+/btad26Nc899xyxsfrZIeIk/fOnBbLW4vFaABb+xSiH04hIoLPW8vnnn/Pyyy+zdetWABUwkQCgkbAWqOc/rAJg/thuhIXpakgRqVt5eTnLli3jwIED9O/fn4EDBzodSUSq+bWEGWMmA88DLuAVa+3PajlmPPBLIAI4b60d589MLd3x/BLc1aNg35vY0+E0IhLIzp49S1ZWFgUFBUyePJkRI0ZoGRuRAOK3EmaMcQG/Ae4ETgJfGWNWWGv31jgmEXgBmGytPWGMSfFXnmBgreWZ17cA8OJjQ4iL0kCmiNSttLQUt9vNvHnzSE9PdzqOiFzDn3PCRgDZ1toj1tpKYCEw/ZpjHgGWWmtPAFhrz/kxT4u3ZOtJDp0rBmBcL/VVEfkmj8fDoUOHAMjIyOC73/2uCphIgPJnCUsDcmrcPll9X029gDbGmE+NMVuNMY/7MU+L9/fv7AJg9V/dTkyky+E0IhJoioqKeO2113jzzTfJy8sDIDxcI+Yigcqf/3fWNvHA1vL5hwITgRjgC2PMl9bag1e9kDHzgflAyP6L7kR+KVUeiyvMkNk+wek4IhJgjh49yttvv01lZSUzZ86kXbt2TkcSkevwZwk7CXSucbsTkFvLMeettSVAiTHmM2AQcFUJs9a+BLwEMGzYsGuLXEg4cPYSAL+cM9jZICIScL744gvWrFlDUlIS8+bNUwETaSH8eTryK6CnMSbDGBMJPASsuOaY5cDtxphwY0wsMBLY58dMLday7acA6JUa73ASEQk0LpeLPn368Mwzz6iAibQgfhsJs9a6jTHPAR/gW6LiVWvtHmPMt6off9Fau88YsxrYCXjxLWOx21+ZWrL3dp4GoGdKnMNJRCQQnDt3jsLCQnr27Mnw4cMZPny4lp8QaWH8OmPTWvs+8P419714ze3/Av7Lnzlaup+u9K3q0SY2Qouzigi7d+9mxYoVxMfH0717d+39KNJC6bKZAFde5eGV9UcB+OhvtI6tSCjzeDysWbOGTZs20blzZ2bNmqUCJtKCqYQFuHv/bz0Az4zJICkuyuE0IuKUqqoq/vjHP3LixAlGjhzJnXfeiculpWpEWjKVsAB3eXHWv5/ax+EkIuKkiIgIOnTowLBhwxgwYIDTcUSkCaiEBbDHXtkEwPTBHTUXTCQEWWvZtGkTGRkZpKamMnnyZKcjiUgT0mSCALX7VCHrs88D8C/39XM4jYg0t4qKCpYsWcIHH3zA119/7XQcEfEDjYQFqHuq54K98OgQEmMjHU4jIs0pLy+PrKws8vPzmTRpErfddpvTkUTED1TCAtBvPsm+8vHUAR0cTCIize306dO89tprhIeHM3fuXDIyMpyOJCJ+ohIWgNYd8G28u+FHExxOIiLNLSUlhUGDBjFmzBgSErRPrEgw05ywAGOtZfOxC6TER5GWGON0HBFpBsXFxSxdupTS0lJcLhdTp05VARMJASphAealz44A0K+jfgCLhIITJ07wu9/9jn379nH69Gmn44hIM9LpyACz61QhAM8/fIvDSUTEn6y1bN68mQ8//JDExEQee+wxUlNTnY4lIs1IJSzArKzeqDshOsLhJCLiTxs3buSjjz6id+/ezJgxg+joaKcjiUgzUwkLIDtyCgCYOqC9s0FExO8GDx6My+Vi5MiRGKPFmEVCkeaEBZAj531bFD0yoovDSUTEH/bt28dbb72Fx+OhVatWjBo1SgVMJIRpJCyAVLq9AGS0a+VwEhFpSl6vl7Vr17JhwwbS0tIoLy+nVSv9fy4S6lTCAsiWYxcBiHRpgFIkWJSUlLBkyRKOHTvG0KFDmTx5MuHh+tErIiphAeXyWYnkOG1TJBIsFi9ezKlTp5g+fTqDBw92Oo6IBBCVsABxJK+YrC0n6dQmRnNERFo4ay3WWsLCwpg8eTIA7dvrghsRuZpKWICY8ZsNAIzunuxwEhG5GVVVVaxcuZLIyEimTZum8iUiddLkowCQc6GUonI3AD+fOdDhNCJyoy5cuMCCBQvYuXMncXFxWGudjiQiAUwjYQHgxyv2APBv9/d3OImI3KgDBw7wzjvvEBYWxqOPPkqPHj2cjiQiAU4lLAB4q/+1/OhIrQ8m0hKVlpaydOlSkpKSmD17NomJiU5HEpEWQCXMYfnFFXx6II+BnVo7HUVEGqmiooLIyEhiY2N5/PHHSU1N1fITItJgmhPmsOXbcwHon6YSJtKSnDp1ihdeeIGtW7cCkJaWpgImIo2inxgOW7L1JAB/PamXw0lEpCGstWzdupXVq1cTFxdHx44dnY4kIi2USpiDjuQVs/d0EQDt4qMcTiMi11NVVcV7773Hjh076NGjB/fffz+xsbFOxxKRFqrRJcwY4wIestb+yQ95QsqsF78A4PmHBjsbREQaJCcnh507dzJu3DjGjh1LWJhmdIjIjavzJ4gxJsEY83fGmF8bY+4yPt8FjgCzmy9icPrVx4fIL6kEYPrgNIfTiEh9CgsLAejWrRvPPvss48ePVwETkZtW30+RPwC9gV3AM8CHwExgurV2ejNkC2pvbT4BQNZf3upwEhGpi9fr5ZNPPuFXv/oVJ0/65m8mJSU5nEpEgkV9pyO7WWsHABhjXgHOA+nW2kvNkiyIHckr5nRhOZGuMEZktHU6jojU4vLaX4cPH2bQoEGkpqY6HUlEgkx9Jazq8gfWWo8x5qgKWNNYvecMAD+doRXyRQJRbm4uWVlZFBcXM23aNIYOHYoxxulYIhJk6ithg4wxRcDlnzwxNW5ba22C39MFIbfHy3+uPgDAPYM6OJxGRGpz5MgRrLU8+eSTpKVpzqaI+EedJcxa62rOIKHi3Z2+xVl7p8YTG6kVQkQChdvtJj8/n9TUVEaPHs3QoUOJiYlxOpaIBLH6ro6MNsb8VfXVkfONMWoMTeCvF+0A4FcP3+JwEhG5rKCggFdffZU33niDiooKjDEqYCLid/UVq9fxzQv7HJgK9AO+3xyhgtXXJy4C0K9jAr3bxzucRkQAsrOzWbp0KV6vl/vvv5+oKC2cLCLNo74S1rfG1ZELgM3NEyl4zf+Db4+5f79/gMNJRMRay2effcann35Kamoqs2fPpm1bXa0sIs2noVdHunVl0M3Lu1QBwMBO2qxbJBDk5eUxcOBA7rnnHiIiIpyOIyIhpr4SNrj6akjwXRGpqyNvwo6cAgBmDO6oS91FHHTmzBkiIyNp27Yt999/P2FhYfp/UkQcUd+K+TustQnVf+KtteE1PlYBa6S5CzYBMP0WXe4u4pTt27ezYMECVq1aBYDL5VIBExHH1DcSZpstRZCz1lJU7qZdfBR39E5xOo5IyHG73axatYpt27bRtWtXZsyY4XQkEZF6S1iKMeZv6nrQWvsLP+QJSit2+NYGu71HssNJREJPcXExb731Frm5uYwePZoJEyZo820RCQj1lTAXEMefV8yXG/TJ/nMA/M1dvRxOIhJ6oqKiiIiIYPbs2fTp08fpOCIiV9RXwk5ba/+12ZIEscjwMMIMdGoT63QUkZBgreWrr75i0KBBREVFMW/ePM39EpGAU18J00+sJuL2WDomavVtkeZQXl7OO++8w8GDB7HWMnLkSBUwEQlI9ZWwic2WIsi5vZbwMP0SEPG3s2fPsmjRIgoLC5k8eTIjRoxwOpKISJ3q28D7QnMGCWbZ54oJUwkT8atDhw6RlZVFTEwM8+bNIz093elIIiL10qbcfrb7VCF7TxeRkdzK6SgiQS01NZWePXsydepU4uLinI4jInJduk7bz9ZnnwfgmdszHE4iEnyKior46KOPsNaSkJDA7NmzVcBEpMXQSJgfWWv52ar9AMwc2snhNCLB5ejRoyxZsgS3283AgQNJSdFCyCLSsqiE+dHx/FIAOrWJISrc5XAakeBgrWXjxo18/PHHJCUlMWfOHJKTtRCyiLQ8KmF+9MKn2QD88739HE4iEjzef/99tmzZQt++fbnvvvuIiopyOpKIyA1RCfOjY+d9I2ET++g0iUhTGThwIG3btmXUqFFa/0tEWjSVMD86fqGEjq2j9YtC5Cbt2rWL/Px8xo8fT+fOnencubPTkUREbpqujvQTay1niypolxDtdBSRFsvj8bBq1SqWLl3K0aNH8Xg8TkcSEWkyGgnzk0qPF4CeKbpcXuRGXLp0icWLF5OTk8OoUaOYNGkSLpcucBGR4KES5ic/Xr4HgH4dExxOItLyuN1uFixYQGlpKQ8++CD9+/d3OpKISJNTCfODiyWVLPwqB4B7B3V0OI1Iy2GtxRhDeHg4d911F8nJyVr/S0SCluaENTGv13LLT9YA8L0JPUiO0+XzIg1RUVHBkiVL2LlzJwB9+/ZVARORoKaRsCa2+dif9z3/3sSeDiYRaTny8vLIysoiPz9fG2+LSMhQCWtinx3MA2Dh/FGEuzTQKHI9e/bsYcWKFURERPD444/TtWtXpyOJiDQLlbAm9sKnhwEYkt7G4SQige/06dMsWbKETp06MWvWLBISdCGLiIQOlbAmdPKib4X85LhIIsM1CiZSF4/Hg8vlokOHDsyaNYvevXtr+QkRCTlqCk0oa8tJAP727kyHk4gErhMnTvDrX/+a3NxcwDcBXwVMREKRRsKa0Mf7zgIwdWAHh5OIBB5rLZs2bWLNmjUkJiYSHq4fPyIS2vRTsImcL65gT24R6W1jiYvSt1WkpsrKSlasWMGePXvo3bs3M2bMIDpaW3qJSGhTW2gir64/CsD0wVqcVeRaW7ZsYe/evUycOJHRo0drU3sREVTCmszl3ynP3tHD2SAiAaS8vJzo6GhGjRpFly5dSEtLczqSiEjA0MT8JvKbTw4T4TJER2iCsYjX62XNmjW88MILFBcXExYWpgImInINjYQ1oSqPdTqCiOOKi4t5++23OXbsGMOGDdPcLxGROqiENQFrfeVr7qguDicRcVZOTg6LFy+mrKyM6dOnM3jwYKcjiYgELL+ejjTGTDbGHDDGZBtjflTPccONMR5jzEx/5vEXj9dXwlLitVm3hLaNGzcSHh7O008/rQImInIdfhsJM8a4gN8AdwInga+MMSustXtrOe7nwAf+yuJvO04WAuBy6YovCT2VlZVUVlYSFxfHfffdB0BMTIzDqUREAp8/R8JGANnW2iPW2kpgITC9luO+C7wNnPNjFr/63zUHARiYluhsEJFmlp+fz4IFC8jKysJaS0xMjAqYiEgD+XNOWBqQU+P2SWBkzQOMMWnA/cAEYLgfs/hVSaUbgNE9khxOItJ89u/fz7JlywgLC+Ouu+7S2l8iIo3kzxJW20/kay8f/CXwQ2utp74f4MaY+cB8gPT09KbK12Q8XktyXJR+CUlI8Hq9fPLJJ6xfv56OHTsya9YsEhMTnY4lItLi+LOEnQQ617jdCci95phhwMLq8pIMTDXGuK21y2oeZK19CXgJYNiwYQG3DsTOk4VM7tfe6RgizaKqqop9+/YxZMgQpkyZoj0gRURukD9/en4F9DTGZACngIeAR2oeYK3NuPyxMeY1YOW1BawliI4Io8LtcTqGiF+dPn2a5ORkoqKieOaZZ7T+l4jITfLbxHxrrRt4Dt9Vj/uALGvtHmPMt4wx3/LX53VKz9R4pyOI+IW1li1btvDKK6/w6aefAqiAiYg0Ab+eR7DWvg+8f819L9Zx7BP+zOJPbo8lPEzzwST4VFVV8d5777Fjxw569OjBmDFjnI4kIhI0NJnjJllrcXtVwiT4XLx4kUWLFnH27FnGjRvHuHHjdPGJiEgTUgm7SdtOXASgpFJzwiS4eL1eKioqeOSRR+jZs6fTcUREgo5fty0KBZ8eyAPgjt4pDicRuXler5fdu3djrSUpKYnnnntOBUxExE80EnaT3t91GoChXdo4nETk5pSWlrJ06VIOHz5MdHQ0PXr0wOVyOR1LRCRoqYTdpGP5pbRtFUlMpH5ZScuVm5tLVlYWxcXF3HPPPXTv3t3pSCIiQU8l7Ca4PV48XkufDlqeQlquHTt28O677xIXF8eTTz5JWlqa05FEREKCSthN2Hu6CICeKSph0nLFxcWRkZHB/fffT2xsrNNxRERChkrYTThw5hIAt3XXxt3SshQUFHDs2DEGDx5M9+7d6datm5afEBFpZiphN2HFDt9WmIM6JzobRKQRsrOzWbp0KQC9e/cmJiZGBUxExAEqYTfh80PnAUhN0BYuEvistXz22Wd8+umnpKamMnv2bGJiYpyOJSISslTCblBhWRUA43u3cziJyPVZa1m0aBEHDhxg4MCB3HPPPURERDgdS0QkpKmE3aCi6hJ2Z99Uh5OIXJ8xhoyMDLp3786wYcN0+lFEJACohN2gs0XlAMRHazRBAtfXX39Nq1at6NWrFyNHjnQ6joiI1KASdoMKSn0jYfHR+hZK4HG73axatYpt27bRp08fevXq5XQkERG5hhrEDTp4zrc8Rec2mtgsgaWwsJCsrCxyc3MZPXo0EyZMcDqSiIjUQiXsBh3NKwGgUxstbimBo6ioiN/97nd4vV7mzJlDZmam05FERKQOKmE3qMrjBSA6QntGSuCIj49nxIgRDBgwgKQkLSIsIhLIwpwO0FLtP3OJlPgop2OIUF5ezttvv01eXh7GGMaPH68CJiLSAqiE3QBrLfvPXKKdSpg47MyZM7z00kvs3buX06dPOx1HREQaQacjb8CHe88CEB6mtZbEOTt27GDlypXExMTwxBNP0LlzZ6cjiYhII6iE3YAV2317Rv7DtL4OJ5FQtXv3bpYtW0aXLl2YOXMmcXFxTkcSEZFGUgm7Aa2iXCREhzMio63TUSTEWGsxxpCZmcnkyZMZPnw4YWGaVSAi0hLpp/cNuFBSpasipdkdPXqUV199lbKyMsLDwxk5cqQKmIhIC6aRsBtQUFrJpXK30zEkRFhr2bBhA2vXriUpKYny8nJiYrRIsIhIS6cSdgO2HL/IgLTWTseQEFBeXs7y5cvZv38//fr147777iMyMtLpWCIi0gRUwhpp96lCAIrKqxxOIqFg9erVHDx4kLvvvpuRI0dijK7IFREJFiphjbTp6AUAvjWuu8NJJJh5PB5cLheTJk1iyJAhpKenOx1JRESamEpYI528WArAnX1THU4iwcjj8fDhhx+Sl5fHY489RlxcnJafEBEJUiphjbTtRAEASa00L0eaVlFREUuWLCEnJ4eRI0dirXU6koiI+JFKWCOVVrgJM2hujjSpY8eOsWTJEiorK3nwwQfp37+/05FERMTPVMIaqdLj5d5BHZ2OIUHE4/GwYsUKoqOjmTdvHu3atXM6koiINAOVsEbweC3H80sZ3lUr5cvNq6ioIDw8HJfLxSOPPEJ8fDxRUdoUXkQkVGi57UbIL64AQFN15Gbl5eXx8ssvs2bNGgCSk5NVwEREQoxGwhrhs0PnARjapY3DSaQl27NnD8uXLycyMpLMzEyn44iIiENUwhqhrMoDwB2ZmrMjjefxePjoo4/48ssv6dy5MzNnziQhIcHpWCIi4hCVsEZYd+AcAK1jIhxOIi3RxYsX2bp1KyNGjOCuu+7C5dIm8CIioUwlrBHOF1cCEBupb5s0XH5+PklJSSQnJ/Pss8/SurX2HRUREU3Mb5Tdpwrp1CbG6RjSQlhr+fLLL3nhhRfYtWsXgAqYiIhcoSGdRggzhm7ttIWMXF9lZSUrVqxgz549ZGZm0rNnT6cjiYhIgFEJayBrLZUeL4M6aSRD6nf+/HkWLVpEfn4+EydOZPTo0dphQUREvkElrIGyzxUDUOXRImFSv7y8PEpLS5k7dy4ZGRlOxxERkQClEtZAR8+XAHBr9ySHk0gg8nq9nDx5kvT0dPr06UO3bt20+KqIiNRLE/Mb6OTFMgCSWkU6nEQCTXFxMX/4wx94/fXXuXjxIoAKmIiIXJdGwhpo4+F8ADq0jnY4iQSSnJwcFi9eTFlZGffddx9t2mg3BRERaRiVsAbaeNi3ZVFbjYRJtc2bN/PBBx/QunVrnn76adq3b+90JBERaUFUwhrIFWYY2qWNrnKTK0pLS+nRowf3338/0dEaIRURkcZRCWugS+Vubdwt5OfnU1paSufOnRk3bhyAirmIiNwQTcxvgHNF5QBUur0OJxEn7d+/n5dffpl3330Xay3GGBUwERG5YRoJa4DtOQUA9OkQ72wQcYTX62Xt2rVs2LCBjh07MmvWLJUvERG5aSphDXA8vxSAfh21Wn6oqaysZOHChRw9epQhQ4YwZcoUwsP1v42IiNw8/TZpgKP5voVauyTFOpxEmltERARxcXHcd9993HLLLU7HERGRIKIS1gBR4b6pc/HREQ4nkeZgrWXbtm1069aNNm3a8MADDzgdSUREgpAm5jdAlcer9cFCRFVVFcuXL2flypV89dVXTscREZEgppGwBtibW0R4mCZiB7sLFy6QlZXF2bNnGT9+PGPHjnU6koiIBDGVsAaIi47gUvklp2OIH+Xm5vLGG29gjOHRRx+lR48eTkcSEZEgpxLWAMfzSxjYSVdGBrPk5GR69+7N+PHjtf+jiIg0C80Ja4CySg8FpVVOx5AmVlpaynvvvUdlZSWRkZHcf//9KmAiItJsNBLWAK4wQ9+OCU7HkCaUm5tLVlYWxcXF9OnTh27dujkdSUREQoxKWAN4vPbKMhXSsl1efmLVqlXExcXx1FNP0bFjR6djiYhICFIJawCP1xLu0tWRwWD9+vWsXbuW7t2788ADDxAbqwV4RUTEGSphDZBfUkl4mEbCgsGAAQOw1jJmzBjC9J6KiIiD9FvoOgpKKwEoqXA7nERu1KFDh1i+fDnWWhITExk7dqwKmIiIOE4jYdex82QhAAM7JzobRBrNWsu6detYt24dqamplJWV6fSjiIgEDJWw66h0ewEY3CnR2SDSKGVlZSxdupTs7GwGDRrEtGnTiIjQ3p8iIhI4VMKu43BeMQAR4ZqY31JYa3nzzTfJzc1l2rRpDB06FGP0/omISGBRCbsOV/Weke0Toh1OIg1hrcUYw6RJk3C5XHTq1MnpSCIiIrVSCbuOc5cqAIiJdDmcROrjdruvrP11xx130KVLF6cjiYiI1EuXiF3H+WJfCYt06VsVqAoKCnj11VfZtm0b1lqn44iIiDSIRsKuIyrcRXxUuOYUBajDhw/z9ttv4/V6mTNnDpmZmU5HEhERaRCVsOvweL20itK3KRAVFxezcOFC2rZty+zZs0lKSnI6koiISIP5tV0YYyYDzwMu4BVr7c+uefxR4IfVN4uBb1trd/gzU2N5vH+enC+Bwe12Ex4eTlxcHA899BCdO3cmMjLS6VgiIiKN4reJTsYYF/AbYArQF3jYGNP3msOOAuOstQOBnwAv+SvPjfJ4vSphAeTMmTO88MIL7N27F4Du3burgImISIvkz5GwEUC2tfYIgDFmITAd2Hv5AGvtxhrHfwkE3HoCxy+Uog4WGHbs2MHKlSuJiYkhPj7e6TgiIiI3xZ8lLA3IqXH7JDCynuOfBlb5Mc8NsRbyiyudjhHS3G43H3zwAVu2bKFr1648+OCDxMXFOR1LRETkpvizhNU2flTr+gHGmDvwlbAxdTw+H5gPkJ6e3lT5GuRIXjGD0xOb9XPK1bKzs9myZQu33XYbEydO1ObbIiISFPxZwk4CnWvc7gTkXnuQMWYg8AowxVqbX9sLWWtfonq+2LBhw5ptIaiSCjdF5W5KKz3N9SmlhtLSUmJjY8nMzGT+/Pl06NDB6UgiIiJNxp9DCl8BPY0xGcaYSOAhYEXNA4wx6cBSYK619qAfs9yQy5t3390v1eEkocVay/r163n++ec5e/YsgAqYiIgEHb+NhFlr3caY54AP8C1R8aq1do8x5lvVj78I/BOQBLxQvRiq21o7zF+ZGstbvfp6dIS2LGou5eXlLFu2jAMHDtC/f3/atGnjdCQRERG/8Os6Ydba94H3r7nvxRofPwM8488MN8NbfeJTq+U3j7Nnz5KVlUVBQQGTJ09mxIgR+t6LiEjQ0lLw9bi8D6GWqGgeO3fupLKyknnz5jX7BRgiIiLNTSWsHpdHwsI0GuM3Ho+HwsJC2rZty4QJE7j11lu1/ISIiIQEXetfD69GwvyqqKiI1157jddff52qqipcLpcKmIiIhAyNhNXjcgnTvKSmd+zYMZYsWUJlZSXTp08nIiLC6UgiIiLNSiWsHlanI5uctZaNGzfy8ccfk5SUxLx582jXrp3TsURERJqdSlg9Li/SqtORTcday5EjR+jTpw/33XcfUVFRTkcSERFxhEpYPbLPFQPgUgu7aXl5eURHRxMfH8+cOXOIiIjQaV4REQlpmphfj8N5vhI2qFOis0FauN27d/Pyyy+zapVvf/bIyEgVMBERCXkaCavH2aJyADomxjicpGXyeDysWbOGTZs20blzZ6ZMmeJ0JBERkYChElaPwrIqACLDNWDYWMXFxSxevJgTJ04wYsQI7rrrLlwubf8kIiJymUpYPS6UVJKR3MrpGC1SeHg4FRUVPPDAAwwYMMDpOCIiIgFHJaweZ4rKqXR7nY7RYlhr2blzJ/369SM6Opr58+cTFqZRRBERkdqohNUjKtxFany00zFahIqKCt5991327NlDVVUVw4YNUwETERGph0pYPS6UVNBNpyOvKy8vj6ysLPLz85k0aRJDhw51OpKIiEjAUwmrg7WWs0UVV7YuktodOnSIJUuWEB4ezty5c8nIyHA6koiISIugElaHc5cqAGgdoz0N65OYmEinTp2YPn06CQkJTscRERFpMTRppw6XJ+QP69rW4SSBp7i4mA0bNmCtpV27dsydO1cFTEREpJE0ElaHKo+vhEW4tLJ7TTk5OSxevJiysjIyMzNJSkpyOpKIiEiLpBJWhyqPby6Y9o30sdayefNmPvzwQ1q3bs0zzzyjAiYiInITVMLqkFtQBoDHq4n5AO+//z5btmyhV69e3H///URHa+kOERGRm6ESVofLV0V2TdISFQA9e/YkISGBMWPGaPNtERGRJqASVoe86qsjQ3nfyP3791NUVMSIESPo1asXvXr1cjqSiIhI0AjdhnEdReW+zbvjo0Ovp3q9Xj766CMWLVrErl278Hq1dZOIiEhTC72G0UCX12ht2yrS2SDNrKSkhLfffpujR48ydOhQJk+erO2HRERE/EAlrA4HzlwCINIVOgWkqqqKl19+mZKSEqZPn87gwYOdjiQiIhK0VMLqcPh8CRBaS1RERERw++2307FjRzp06OB0HBERkaAWOsM8jXSxpJJbuyUF/ZWAVVVVLFu2jOzsbACGDh2qAiYiItIMVMLqcO5SOe4gn5B+4cIFFixYwI4dO8jLy3M6joiISEjR6cg6RLjC6BLEa4QdOHCAd955h7CwMB599FF69OjhdCQREZGQohJWj2BdniInJ4eFCxfSoUMHZs+eTWJiotORREREQk5wtowm4PFawoNsUr61FmMMnTp14t5772XgwIGEh+uvgIiIiBM0J6wObq8lLIhK2KlTp3jxxRfJz8/HGMOQIUNUwERERByk38K1cHu8VLq9uILgykhrLVu3bmX16tXExcVRVVXldCQRERFBJaxWF0t9RcVrHQ5yk6qqqnjvvffYsWMH3bt354EHHiA2NtbpWCIiIoJKWK1OXCgFoHu7ln115IYNG9ixYwdjx45l3Lhx2n5IREQkgKiE1SLvUgUAbWJb5r6RVVVVREREMHr0aLp06UJGRobTkUREROQaGhqphaf6PGR6Uss6def1evnkk0946aWXKC8vJyIiQgVMREQkQGkkrBaXV8pvSftGlpaW8s4775Cdnc2gQYNwuVxORxIREZF6qITV4vJIWEtZJyw3N5esrCyKi4uZNm0aQ4cODfo9L0VERFo6lbBanLpYBrSMkTBrLWvWrMFay5NPPklaWprTkURERKQBVMJqcXmR1vjoCIeT1M3tduN2u4mOjuaBBx7A5XJp+QkREZEWRCWsHjERgTmvqqCggKysLFq1asUjjzxCfHy805FERESkkVTCauGtnhMWiGcjs7OzWbp0KV6vl7Fjx2rul4iISAulElaLyyvlB1LBsdby2Wef8emnn5KSksKcOXNo27at07FERETkBqmE1cISeCNhZWVlbN26lYEDB3LPPfcQERG489VERETk+lTCahFII2F5eXkkJSURGxvL/PnzadWqVUDkEhERkZujFfNrYa0NiFGw7du389JLL/H5558DEBcXpwImIiISJDQSVguvtYQ5WHbcbjerV69m69atdO3alWHDhjmWRURERPxDJawWXotjJaywsJDFixdz6tQpRo8ezYQJEwgL04CliIhIsFEJq4XXWnBoIKy0tJSCggJmz55Nnz59nAkhIiIifqcSVgtrm/vzWQ4fPkyPHj3o0KED3//+93X1o4iISJDTea5a7M0twu3xNsvnKi8vZ9GiRfzpT3/i+PHjACpgIiIiIUAjYbVo2yqyWTbvPnv2LFlZWRQUFDB58mTS09P9/jlFREQkMKiE1aLK4yUjuZVfP8fu3btZvnw50dHRzJs3TwVMREQkxKiE1WJ3biHxUf49JWitJS0tjZkzZxIXF+fXzyUiIiKBRyWsFq1jIrhYUtXkr1tUVERubi6ZmZkMGDCA/v37a/FVERGREKUSVgu3x9K3Y0KTvubRo0dZsmQJ1loyMjKIiopSARMREQlhKmG1cHstEa6mKUjWWjZu3MjHH39MUlISs2fPJioqqkleW0RERFoulbBanCsqp2+Hmx8Js9ayePFi9u3bR9++fbnvvvtUwERERARQCatVUbmb4gr3Tb+OMYbk5GTuuusuRo0apdOPIiIicoVK2DXKqzwA9EqNv+HX2LVrF61btyY9PZ0JEyY0VTQREREJIiph17hQUglAZHjjNxPweDysWbOGTZs20bdvX639JSIiInVSCbvG5W0jOyXGNOp5ly5dYvHixeTk5DBy5EjuvPPOpg8nIiIiQUMl7Bpeb3UNa8T0rYsXL7JgwQIqKyt58MEH6d+/v3/CiYiISNBQCatDWCMm0ScmJtK3b1+GDRtGSkqKH1OJiIhIsGj8xKcg57W+kbDr7d9dUVHBu+++S2FhIcYYpk6dqgImIiIiDaYSdo0rZyPrKWF5eXm88sorfP311xw/frx5gomIiEhQ0enIa9grI2G1t7C9e/eyfPlywsPDmTt3LhkZGc0ZT0RERIKEStg1Lo+E1Wbnzp288847dOrUiVmzZpGQ0LT7S4qIiEjoUAn7hrpHwnr37s348eMZM2YMLperuYOJiIhIEPHrnDBjzGRjzAFjTLYx5ke1PG6MMb+qfnynMWaIP/M0xOWRsMsl7MSJE/zpT3+iqqqKqKgoxo0bpwImIiIiN81vJcwY4wJ+A0wB+gIPG2P6XnPYFKBn9Z/5wG/9laehLl8dCZYvv/yS119/nQsXLlBcXOxoLhEREQku/jwdOQLIttYeATDGLASmA3trHDMdeMP6ZsN/aYxJNMZ0sNae9mOuelkL4Xg4+tVatpw8TO/evZkxYwbR0dFORRIREZEg5M8Slgbk1Lh9EhjZgGPSAMdKmNdaRkccJ//kRSZMmMCYMWMwjVi4VURERKQh/FnCamsu11572JBjMMbMx3e60u+bYsdEuChrl8mwoe24ffRgv34uERERCV3+LGEngc41bncCcm/gGKy1LwEvAQwbNqyeRSRuXrd2cWR9T5tvi4iIiH/58+rIr4CexpgMY0wk8BCw4ppjVgCPV18lOQoodHI+mIiIiEhz8dtImLXWbYx5DvgAcAGvWmv3GGO+Vf34i8D7wFQgGygFnvRXHhEREZFA4tfFWq217+MrWjXve7HGxxZ41p8ZRERERAKRNvAWERERcYBKmIiIiIgDVMJEREREHKASJiIiIuIAlTARERERB6iEiYiIiDhAJUxERETEASphIiIiIg5QCRMRERFxgEqYiIiIiANUwkREREQcoBImIiIi4gCVMBEREREHqISJiIiIOEAlTERERMQBxlrrdIZGMcbkAceb4VMlA+eb4fNIw+k9CTx6TwKT3pfAo/ckMDXH+9LFWtuutgdaXAlrLsaYLdbaYU7nkD/TexJ49J4EJr0vgUfvSWBy+n3R6UgRERERB6iEiYiIiDhAJaxuLzkdQL5B70ng0XsSmPS+BB69J4HJ0fdFc8JEREREHKCRMBEREREHhHQJM8ZMNsYcMMZkG2N+VMvjxhjzq+rHdxpjhjiRM9Q04H15tPr92GmM2WiMGeREzlByvfekxnHDjTEeY8zM5swXqhryvhhjxhtjthtj9hhj1jV3xlDTgJ9frY0x7xpjdlS/J086kTOUGGNeNcacM8bsruNxx37Xh2wJM8a4gN8AU4C+wMPGmL7XHDYF6Fn9Zz7w22YNGYIa+L4cBcZZawcCP0FzLfyqge/J5eN+DnzQvAlDU0PeF2NMIvACcJ+1th8wq7lzhpIG/r/yLLDXWjsIGA/8jzEmslmDhp7XgMn1PO7Y7/qQLWHACCDbWnvEWlsJLASmX3PMdOAN6/MlkGiM6dDcQUPMdd8Xa+1Ga+3F6ptfAp2aOWOoacj/KwDfBd4GzjVnuBDWkPflEWCptfYEgLVW741/NeQ9sUC8McYAccAFwN28MUOLtfYzfN/nujj2uz6US1gakFPj9snq+xp7jDStxn7PnwZW+TWRXPc9McakAfcDLzZjrlDXkP9XegFtjDGfGmO2GmMeb7Z0oakh78mvgT5ALrAL+L611ts88aQOjv2uD2+OTxKgTC33XXupaEOOkabV4O+5MeYOfCVsjF8TSUPek18CP7TWenz/wJdm0JD3JRwYCkwEYoAvjDFfWmsP+jtciGrIe3I3sB2YAHQH1hhjPrfWFvk5m9TNsd/1oVzCTgKda9zuhO9fJo09RppWg77nxpiBwCvAFGttfjNlC1UNeU+GAQurC1gyMNUY47bWLmuWhKGpoT/DzltrS4ASY8xnwCBAJcw/GvKePAn8zPrWh8o2xhwFMoHNzRNRauHY7/pQPh35FdDTGJNRPSnyIWDFNcesAB6vvnJiFFBorT3d3EFDzHXfF2NMOrAUmKt/0TeL674n1toMa21Xa21XYAnwHRUwv2vIz7DlwO3GmHBjTCwwEtjXzDlDSUPekxP4RiYxxqQCvYEjzZpSruXY7/qQHQmz1rqNMc/hu5LLBbxqrd1jjPlW9eMvAu8DU4FsoBTfv2DEjxr4vvwTkAS8UD3y4tbGuP7TwPdEmllD3hdr7T5jzGpgJ+AFXrHW1nqZvty8Bv6/8hPgNWPMLnynwX5orT3vWOgQYIx5C9+VqMnGmJPAj4EIcP53vVbMFxEREXFAKJ+OFBEREXGMSpiIiIiIA1TCRERERBygEiYiIiLiAJUwEREREQeohIlISDDGeIwx22v86WqMGW+MKTTGfG2M2WeM+XH1sTXv32+M+W+n84tI8AnZdcJEJOSUWWsH17zDGNMV+Nxae48xphWw3Rizsvrhy/fHAF8bY96x1m5o3sgiEsw0EiYiAlRv7bMV335+Ne8vw7fXX7Ns6CsioUMlTERCRUyNU5HvXPugMSYJGAXsueb+NkBP4LPmiSkioUKnI0UkVHzjdGS1240xX+Pb1udn1dvMjK++fye+vf1+Zq0902xJRSQkqISJSKj73Fp7T133G2N6Aeur54Rtb+ZsIhLEdDpSRKQe1tqDwH8AP3Q6i4gEF5UwEZHrexEYa4zJcDqIiAQPY611OoOIiIhIyNFImIiIiIgDVMJEREREHKASJiIiIuIAlTARERERB6iEiYiIiDhAJUxERETEASphIiIiIg5QCRMRERFxwP8PVIT8kpEcXoAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots(1, 1, figsize=(10, 8))\n",
    "plot_roc(\n",
    "    y_test['h1n1_vaccine'], \n",
    "    predictions_h1, \n",
    "    'h1n1_vaccine',\n",
    "    ax=ax\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.8710964730481472"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "roc_auc_score(y_test.h1n1_vaccine, predictions_h1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Second part"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_dataset_se = Pool(data=X_train,\n",
    "                     label=y_train.seasonal_vaccine,\n",
    "                     cat_features = categorical_features_indices)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I made a second objective function in case I'd want to change some tuning"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "def objective2(trial):\n",
    "    param = {\n",
    "        'iterations':trial.suggest_categorical('iterations', [100,200,300,500,1000,1200,1500]),\n",
    "        'learning_rate':trial.suggest_float(\"learning_rate\", 0.001, 0.3),\n",
    "        'random_strength':trial.suggest_int(\"random_strength\", 1,10),\n",
    "        'bagging_temperature':trial.suggest_int(\"bagging_temperature\", 0,10),\n",
    "        'max_bin':trial.suggest_categorical('max_bin', [4,5,6,8,10,20,30]),\n",
    "        'grow_policy':trial.suggest_categorical('grow_policy', ['SymmetricTree', 'Depthwise', 'Lossguide']),\n",
    "        'min_data_in_leaf':trial.suggest_int(\"min_data_in_leaf\", 1,10),\n",
    "        'od_type' : \"Iter\",\n",
    "        'od_wait' : 100,\n",
    "        \"depth\": trial.suggest_int(\"max_depth\", 2,10),\n",
    "        \"l2_leaf_reg\": trial.suggest_loguniform(\"l2_leaf_reg\", 1e-8, 100),\n",
    "         'one_hot_max_size':trial.suggest_categorical('one_hot_max_size', [5,10,12,100,500,1024]),\n",
    "        'custom_metric' : ['AUC'],\n",
    "        \"loss_function\": \"Logloss\",\n",
    "        'auto_class_weights':trial.suggest_categorical('auto_class_weights', ['Balanced', 'SqrtBalanced']),\n",
    "        }\n",
    "\n",
    "    scores = cv(train_dataset_se,\n",
    "            param,\n",
    "            fold_count=5, \n",
    "            early_stopping_rounds=10,         \n",
    "            plot=False, verbose=False) \n",
    "\n",
    "    return scores['test-AUC-mean'].max()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 01:58:46,608]\u001b[0m A new study created in memory with name: no-name-03913ebf-5557-4268-9f2c-43f40117f5aa\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 01:59:12,754]\u001b[0m Trial 0 finished with value: 0.8592774453063841 and parameters: {'iterations': 300, 'learning_rate': 0.013964954297408176, 'random_strength': 1, 'bagging_temperature': 8, 'max_bin': 10, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 4, 'max_depth': 5, 'l2_leaf_reg': 21.328495943450676, 'one_hot_max_size': 100, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 0 with value: 0.8592774453063841.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 01:59:26,032]\u001b[0m Trial 1 finished with value: 0.8599969087296045 and parameters: {'iterations': 1200, 'learning_rate': 0.11477165079768124, 'random_strength': 9, 'bagging_temperature': 6, 'max_bin': 5, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 5, 'max_depth': 7, 'l2_leaf_reg': 0.5714362138520529, 'one_hot_max_size': 500, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 1 with value: 0.8599969087296045.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:00:01,309]\u001b[0m Trial 2 finished with value: 0.861608374321975 and parameters: {'iterations': 1500, 'learning_rate': 0.06018181691194878, 'random_strength': 6, 'bagging_temperature': 4, 'max_bin': 10, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 6, 'max_depth': 4, 'l2_leaf_reg': 7.937278880272323e-06, 'one_hot_max_size': 12, 'auto_class_weights': 'Balanced'}. Best is trial 2 with value: 0.861608374321975.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:00:14,015]\u001b[0m Trial 3 finished with value: 0.8586019282142363 and parameters: {'iterations': 100, 'learning_rate': 0.04917655399574397, 'random_strength': 1, 'bagging_temperature': 5, 'max_bin': 30, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 3, 'max_depth': 6, 'l2_leaf_reg': 1.253388039132331e-06, 'one_hot_max_size': 500, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 2 with value: 0.861608374321975.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 02:00:27,447]\u001b[0m Trial 4 finished with value: 0.8602753191026151 and parameters: {'iterations': 1500, 'learning_rate': 0.20567786125414012, 'random_strength': 8, 'bagging_temperature': 7, 'max_bin': 20, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 7, 'max_depth': 2, 'l2_leaf_reg': 0.0011659140576640084, 'one_hot_max_size': 12, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 2 with value: 0.861608374321975.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:00:34,719]\u001b[0m Trial 5 finished with value: 0.8567640003480698 and parameters: {'iterations': 100, 'learning_rate': 0.21103536986772822, 'random_strength': 4, 'bagging_temperature': 4, 'max_bin': 6, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 6, 'max_depth': 4, 'l2_leaf_reg': 1.5248267732768012e-08, 'one_hot_max_size': 100, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 2 with value: 0.861608374321975.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:01:10,069]\u001b[0m Trial 6 finished with value: 0.8617699180315392 and parameters: {'iterations': 1000, 'learning_rate': 0.039222305770230614, 'random_strength': 5, 'bagging_temperature': 3, 'max_bin': 20, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 10, 'max_depth': 4, 'l2_leaf_reg': 3.544948380552023e-06, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 6 with value: 0.8617699180315392.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:01:44,172]\u001b[0m Trial 7 finished with value: 0.860718479761989 and parameters: {'iterations': 1000, 'learning_rate': 0.045869610193466345, 'random_strength': 9, 'bagging_temperature': 10, 'max_bin': 30, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 8, 'max_depth': 5, 'l2_leaf_reg': 6.629274905463984e-06, 'one_hot_max_size': 12, 'auto_class_weights': 'Balanced'}. Best is trial 6 with value: 0.8617699180315392.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:01:48,686]\u001b[0m Trial 8 finished with value: 0.8518659419954195 and parameters: {'iterations': 1500, 'learning_rate': 0.2775338323862574, 'random_strength': 6, 'bagging_temperature': 4, 'max_bin': 4, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 9, 'max_depth': 9, 'l2_leaf_reg': 0.0010943835695463693, 'one_hot_max_size': 500, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 6 with value: 0.8617699180315392.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:01:52,676]\u001b[0m Trial 9 finished with value: 0.8576354127969263 and parameters: {'iterations': 1000, 'learning_rate': 0.16527427195333977, 'random_strength': 1, 'bagging_temperature': 9, 'max_bin': 8, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 2, 'max_depth': 5, 'l2_leaf_reg': 5.547808519611033e-05, 'one_hot_max_size': 100, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 6 with value: 0.8617699180315392.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:02:11,155]\u001b[0m Trial 10 finished with value: 0.8616278814855303 and parameters: {'iterations': 500, 'learning_rate': 0.11039932312091873, 'random_strength': 4, 'bagging_temperature': 0, 'max_bin': 20, 'grow_policy': 'Depthwise', 'min_data_in_leaf': 10, 'max_depth': 2, 'l2_leaf_reg': 1.2159249519874814e-08, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 6 with value: 0.8617699180315392.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:02:30,680]\u001b[0m Trial 11 finished with value: 0.8613644301746106 and parameters: {'iterations': 500, 'learning_rate': 0.1084207966325659, 'random_strength': 4, 'bagging_temperature': 0, 'max_bin': 20, 'grow_policy': 'Depthwise', 'min_data_in_leaf': 10, 'max_depth': 2, 'l2_leaf_reg': 1.2894289326228455e-08, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 6 with value: 0.8617699180315392.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:02:55,635]\u001b[0m Trial 12 finished with value: 0.8357434471221705 and parameters: {'iterations': 500, 'learning_rate': 0.004806416713353634, 'random_strength': 4, 'bagging_temperature': 0, 'max_bin': 20, 'grow_policy': 'Depthwise', 'min_data_in_leaf': 10, 'max_depth': 2, 'l2_leaf_reg': 6.39944753546505e-08, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 6 with value: 0.8617699180315392.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 02:03:11,141]\u001b[0m Trial 13 finished with value: 0.8616457635070617 and parameters: {'iterations': 1000, 'learning_rate': 0.10411608463872443, 'random_strength': 3, 'bagging_temperature': 2, 'max_bin': 20, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 10, 'max_depth': 3, 'l2_leaf_reg': 1.276003870120915e-07, 'one_hot_max_size': 5, 'auto_class_weights': 'Balanced'}. Best is trial 6 with value: 0.8617699180315392.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:03:38,197]\u001b[0m Trial 14 finished with value: 0.8619148959330752 and parameters: {'iterations': 1000, 'learning_rate': 0.07618156061546405, 'random_strength': 3, 'bagging_temperature': 2, 'max_bin': 20, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 9, 'max_depth': 3, 'l2_leaf_reg': 3.5865647557736247e-07, 'one_hot_max_size': 5, 'auto_class_weights': 'Balanced'}. Best is trial 14 with value: 0.8619148959330752.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:03:50,054]\u001b[0m Trial 15 finished with value: 0.8549497979408294 and parameters: {'iterations': 1000, 'learning_rate': 0.06969554503907559, 'random_strength': 7, 'bagging_temperature': 2, 'max_bin': 20, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 8, 'max_depth': 7, 'l2_leaf_reg': 0.0002906071529974482, 'one_hot_max_size': 1024, 'auto_class_weights': 'Balanced'}. Best is trial 14 with value: 0.8619148959330752.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:04:00,196]\u001b[0m Trial 16 finished with value: 0.8609895930428383 and parameters: {'iterations': 200, 'learning_rate': 0.15325494851429844, 'random_strength': 2, 'bagging_temperature': 2, 'max_bin': 5, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 8, 'max_depth': 3, 'l2_leaf_reg': 6.938536064744217e-07, 'one_hot_max_size': 5, 'auto_class_weights': 'Balanced'}. Best is trial 14 with value: 0.8619148959330752.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:05:03,898]\u001b[0m Trial 17 finished with value: 0.8564992332206849 and parameters: {'iterations': 1000, 'learning_rate': 0.009317706201861467, 'random_strength': 5, 'bagging_temperature': 2, 'max_bin': 4, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 9, 'max_depth': 4, 'l2_leaf_reg': 0.04373352118950405, 'one_hot_max_size': 5, 'auto_class_weights': 'Balanced'}. Best is trial 14 with value: 0.8619148959330752.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 02:05:09,478]\u001b[0m Trial 18 finished with value: 0.8367012309165419 and parameters: {'iterations': 1000, 'learning_rate': 0.07767821367355207, 'random_strength': 3, 'bagging_temperature': 3, 'max_bin': 6, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 1, 'max_depth': 10, 'l2_leaf_reg': 2.5602616797376855e-05, 'one_hot_max_size': 1024, 'auto_class_weights': 'Balanced'}. Best is trial 14 with value: 0.8619148959330752.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:05:26,289]\u001b[0m Trial 19 finished with value: 0.8498049035164978 and parameters: {'iterations': 300, 'learning_rate': 0.017984347045110226, 'random_strength': 5, 'bagging_temperature': 1, 'max_bin': 8, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 9, 'max_depth': 3, 'l2_leaf_reg': 1.2398896982274805e-06, 'one_hot_max_size': 5, 'auto_class_weights': 'Balanced'}. Best is trial 14 with value: 0.8619148959330752.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 02:06:08,776]\u001b[0m Trial 20 finished with value: 0.8619727265448246 and parameters: {'iterations': 1200, 'learning_rate': 0.03236712628327796, 'random_strength': 7, 'bagging_temperature': 5, 'max_bin': 20, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 7, 'max_depth': 6, 'l2_leaf_reg': 0.046219859585272895, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 20 with value: 0.8619727265448246.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:06:47,953]\u001b[0m Trial 21 finished with value: 0.8597350906668979 and parameters: {'iterations': 1200, 'learning_rate': 0.03452823865248107, 'random_strength': 7, 'bagging_temperature': 5, 'max_bin': 20, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 7, 'max_depth': 7, 'l2_leaf_reg': 0.018537719843351754, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 20 with value: 0.8619727265448246.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:07:08,229]\u001b[0m Trial 22 finished with value: 0.8583524447935437 and parameters: {'iterations': 1200, 'learning_rate': 0.08575309147911239, 'random_strength': 7, 'bagging_temperature': 3, 'max_bin': 20, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 8, 'max_depth': 8, 'l2_leaf_reg': 0.3397573285506683, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 20 with value: 0.8619727265448246.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:08:17,337]\u001b[0m Trial 23 finished with value: 0.8624260361198551 and parameters: {'iterations': 1200, 'learning_rate': 0.027247202810574488, 'random_strength': 10, 'bagging_temperature': 6, 'max_bin': 20, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 9, 'max_depth': 6, 'l2_leaf_reg': 39.55970394954269, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 23 with value: 0.8624260361198551.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:08:31,360]\u001b[0m Trial 24 finished with value: 0.8609916595255 and parameters: {'iterations': 1200, 'learning_rate': 0.13650969502800792, 'random_strength': 10, 'bagging_temperature': 6, 'max_bin': 20, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 7, 'max_depth': 6, 'l2_leaf_reg': 14.889821880428762, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 23 with value: 0.8624260361198551.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:10:14,239]\u001b[0m Trial 25 finished with value: 0.861997483772733 and parameters: {'iterations': 1200, 'learning_rate': 0.02712686924609191, 'random_strength': 10, 'bagging_temperature': 6, 'max_bin': 20, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 9, 'max_depth': 8, 'l2_leaf_reg': 88.84655832391311, 'one_hot_max_size': 5, 'auto_class_weights': 'Balanced'}. Best is trial 23 with value: 0.8624260361198551.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:11:59,242]\u001b[0m Trial 26 finished with value: 0.8622215414530657 and parameters: {'iterations': 1200, 'learning_rate': 0.027279570694869436, 'random_strength': 10, 'bagging_temperature': 7, 'max_bin': 20, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 5, 'max_depth': 8, 'l2_leaf_reg': 95.65012826306524, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 23 with value: 0.8624260361198551.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:12:09,964]\u001b[0m Trial 27 finished with value: 0.8582590962360153 and parameters: {'iterations': 1200, 'learning_rate': 0.29935905333625645, 'random_strength': 10, 'bagging_temperature': 7, 'max_bin': 5, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 5, 'max_depth': 9, 'l2_leaf_reg': 88.42202477486413, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 23 with value: 0.8624260361198551.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:14:34,841]\u001b[0m Trial 28 finished with value: 0.8448933869047659 and parameters: {'iterations': 1200, 'learning_rate': 0.0012883179018759842, 'random_strength': 9, 'bagging_temperature': 8, 'max_bin': 4, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 4, 'max_depth': 8, 'l2_leaf_reg': 4.507225065456655, 'one_hot_max_size': 5, 'auto_class_weights': 'Balanced'}. Best is trial 23 with value: 0.8624260361198551.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 02:15:00,735]\u001b[0m Trial 29 finished with value: 0.8511466699775211 and parameters: {'iterations': 200, 'learning_rate': 0.018121978525421818, 'random_strength': 10, 'bagging_temperature': 8, 'max_bin': 10, 'grow_policy': 'Depthwise', 'min_data_in_leaf': 3, 'max_depth': 8, 'l2_leaf_reg': 82.80146360137286, 'one_hot_max_size': 100, 'auto_class_weights': 'Balanced'}. Best is trial 23 with value: 0.8624260361198551.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 02:16:10,806]\u001b[0m Trial 30 finished with value: 0.8606369200848867 and parameters: {'iterations': 1200, 'learning_rate': 0.025343709715977606, 'random_strength': 8, 'bagging_temperature': 7, 'max_bin': 6, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 4, 'max_depth': 10, 'l2_leaf_reg': 4.231408300407245, 'one_hot_max_size': 1024, 'auto_class_weights': 'Balanced'}. Best is trial 23 with value: 0.8624260361198551.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:16:51,352]\u001b[0m Trial 31 finished with value: 0.8624760408843091 and parameters: {'iterations': 1200, 'learning_rate': 0.05571026924312778, 'random_strength': 10, 'bagging_temperature': 6, 'max_bin': 20, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 6, 'max_depth': 6, 'l2_leaf_reg': 92.84631319598631, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 31 with value: 0.8624760408843091.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:18:56,698]\u001b[0m Trial 32 finished with value: 0.8422039202853909 and parameters: {'iterations': 1200, 'learning_rate': 0.0010313180718820245, 'random_strength': 9, 'bagging_temperature': 6, 'max_bin': 20, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 5, 'max_depth': 7, 'l2_leaf_reg': 24.67310566534338, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 31 with value: 0.8624760408843091.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 02:19:31,048]\u001b[0m Trial 33 finished with value: 0.859487821029903 and parameters: {'iterations': 1200, 'learning_rate': 0.062100494868209254, 'random_strength': 10, 'bagging_temperature': 6, 'max_bin': 10, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 6, 'max_depth': 9, 'l2_leaf_reg': 1.2952184669897353, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 31 with value: 0.8624760408843091.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:20:06,984]\u001b[0m Trial 34 finished with value: 0.8617578410670788 and parameters: {'iterations': 300, 'learning_rate': 0.052470154488746826, 'random_strength': 9, 'bagging_temperature': 7, 'max_bin': 30, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 3, 'max_depth': 8, 'l2_leaf_reg': 45.787192748037945, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 31 with value: 0.8624760408843091.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 02:20:19,957]\u001b[0m Trial 35 finished with value: 0.8616715967736706 and parameters: {'iterations': 1200, 'learning_rate': 0.09350163378814949, 'random_strength': 8, 'bagging_temperature': 9, 'max_bin': 20, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 5, 'max_depth': 6, 'l2_leaf_reg': 8.394192250872125, 'one_hot_max_size': 500, 'auto_class_weights': 'Balanced'}. Best is trial 31 with value: 0.8624760408843091.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:20:57,820]\u001b[0m Trial 36 finished with value: 0.8611326533509001 and parameters: {'iterations': 1200, 'learning_rate': 0.051467883369440526, 'random_strength': 10, 'bagging_temperature': 6, 'max_bin': 20, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 6, 'max_depth': 7, 'l2_leaf_reg': 1.2592120964345, 'one_hot_max_size': 12, 'auto_class_weights': 'Balanced'}. Best is trial 31 with value: 0.8624760408843091.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:21:04,326]\u001b[0m Trial 37 finished with value: 0.8448587047236252 and parameters: {'iterations': 100, 'learning_rate': 0.02568221228798362, 'random_strength': 8, 'bagging_temperature': 7, 'max_bin': 8, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 4, 'max_depth': 5, 'l2_leaf_reg': 87.48517216184811, 'one_hot_max_size': 10, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 31 with value: 0.8624760408843091.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 02:21:20,064]\u001b[0m Trial 38 finished with value: 0.853128473568199 and parameters: {'iterations': 1200, 'learning_rate': 0.12773638178556146, 'random_strength': 9, 'bagging_temperature': 5, 'max_bin': 20, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 7, 'max_depth': 9, 'l2_leaf_reg': 0.22026399083527087, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 31 with value: 0.8624760408843091.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:21:29,547]\u001b[0m Trial 39 finished with value: 0.860400768400553 and parameters: {'iterations': 1200, 'learning_rate': 0.18799555523262435, 'random_strength': 10, 'bagging_temperature': 8, 'max_bin': 30, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 6, 'max_depth': 5, 'l2_leaf_reg': 22.52332464723313, 'one_hot_max_size': 500, 'auto_class_weights': 'Balanced'}. Best is trial 31 with value: 0.8624760408843091.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:22:06,265]\u001b[0m Trial 40 finished with value: 0.861516799464793 and parameters: {'iterations': 1500, 'learning_rate': 0.042578219791789336, 'random_strength': 9, 'bagging_temperature': 4, 'max_bin': 20, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 9, 'max_depth': 7, 'l2_leaf_reg': 2.087481458571183, 'one_hot_max_size': 12, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 31 with value: 0.8624760408843091.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:22:50,192]\u001b[0m Trial 41 finished with value: 0.8609410005750432 and parameters: {'iterations': 1200, 'learning_rate': 0.030632757759002652, 'random_strength': 10, 'bagging_temperature': 5, 'max_bin': 20, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 5, 'max_depth': 6, 'l2_leaf_reg': 0.010120344084021772, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 31 with value: 0.8624760408843091.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:23:14,283]\u001b[0m Trial 42 finished with value: 0.8610182421679324 and parameters: {'iterations': 1200, 'learning_rate': 0.06461226321948788, 'random_strength': 8, 'bagging_temperature': 6, 'max_bin': 20, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 7, 'max_depth': 6, 'l2_leaf_reg': 0.054234637196588546, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 31 with value: 0.8624760408843091.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:24:20,167]\u001b[0m Trial 43 finished with value: 0.8628189010522164 and parameters: {'iterations': 1200, 'learning_rate': 0.04020226312212685, 'random_strength': 9, 'bagging_temperature': 5, 'max_bin': 20, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 8, 'max_depth': 7, 'l2_leaf_reg': 90.13854739728647, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 43 with value: 0.8628189010522164.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:24:28,825]\u001b[0m Trial 44 finished with value: 0.8535210908812981 and parameters: {'iterations': 100, 'learning_rate': 0.04924979597860921, 'random_strength': 9, 'bagging_temperature': 4, 'max_bin': 20, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 8, 'max_depth': 8, 'l2_leaf_reg': 40.94570190854328, 'one_hot_max_size': 100, 'auto_class_weights': 'Balanced'}. Best is trial 43 with value: 0.8628189010522164.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 02:26:30,637]\u001b[0m Trial 45 finished with value: 0.8592894008103542 and parameters: {'iterations': 1200, 'learning_rate': 0.009427441664574057, 'random_strength': 10, 'bagging_temperature': 7, 'max_bin': 5, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 9, 'max_depth': 7, 'l2_leaf_reg': 10.287509791866576, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 43 with value: 0.8628189010522164.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 02:26:39,562]\u001b[0m Trial 46 finished with value: 0.8602585888878375 and parameters: {'iterations': 1200, 'learning_rate': 0.2533464487929816, 'random_strength': 10, 'bagging_temperature': 6, 'max_bin': 20, 'grow_policy': 'Depthwise', 'min_data_in_leaf': 8, 'max_depth': 5, 'l2_leaf_reg': 63.01880613962544, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 43 with value: 0.8628189010522164.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:27:05,906]\u001b[0m Trial 47 finished with value: 0.8614314481756443 and parameters: {'iterations': 1500, 'learning_rate': 0.09233047273917006, 'random_strength': 9, 'bagging_temperature': 9, 'max_bin': 10, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 10, 'max_depth': 8, 'l2_leaf_reg': 95.2433470499676, 'one_hot_max_size': 5, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 43 with value: 0.8628189010522164.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:27:42,278]\u001b[0m Trial 48 finished with value: 0.8620117436511997 and parameters: {'iterations': 500, 'learning_rate': 0.05910864313515036, 'random_strength': 10, 'bagging_temperature': 5, 'max_bin': 20, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 9, 'max_depth': 7, 'l2_leaf_reg': 4.8256119684481265, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 43 with value: 0.8628189010522164.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:28:13,440]\u001b[0m Trial 49 finished with value: 0.8620452402442842 and parameters: {'iterations': 500, 'learning_rate': 0.056721079608529255, 'random_strength': 9, 'bagging_temperature': 4, 'max_bin': 6, 'grow_policy': 'SymmetricTree', 'min_data_in_leaf': 6, 'max_depth': 6, 'l2_leaf_reg': 5.3512189414574705, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 43 with value: 0.8628189010522164.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:28:50,569]\u001b[0m Trial 50 finished with value: 0.8622923239683138 and parameters: {'iterations': 500, 'learning_rate': 0.04192853723185702, 'random_strength': 8, 'bagging_temperature': 4, 'max_bin': 6, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 6, 'max_depth': 5, 'l2_leaf_reg': 0.6513400670834688, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 43 with value: 0.8628189010522164.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:29:09,173]\u001b[0m Trial 51 finished with value: 0.8568569561359218 and parameters: {'iterations': 200, 'learning_rate': 0.04160025060921315, 'random_strength': 8, 'bagging_temperature': 4, 'max_bin': 6, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 6, 'max_depth': 5, 'l2_leaf_reg': 0.7555161140528094, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 43 with value: 0.8628189010522164.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 02:29:37,594]\u001b[0m Trial 52 finished with value: 0.8621902474273602 and parameters: {'iterations': 500, 'learning_rate': 0.07329192392784331, 'random_strength': 9, 'bagging_temperature': 3, 'max_bin': 6, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 5, 'max_depth': 4, 'l2_leaf_reg': 14.58670726905145, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 43 with value: 0.8628189010522164.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:30:02,870]\u001b[0m Trial 53 finished with value: 0.8624964492575723 and parameters: {'iterations': 500, 'learning_rate': 0.07253083471785571, 'random_strength': 6, 'bagging_temperature': 5, 'max_bin': 6, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 5, 'max_depth': 4, 'l2_leaf_reg': 22.98843930329072, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 43 with value: 0.8628189010522164.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:30:43,400]\u001b[0m Trial 54 finished with value: 0.8576171145804865 and parameters: {'iterations': 500, 'learning_rate': 0.017936089217252614, 'random_strength': 6, 'bagging_temperature': 5, 'max_bin': 6, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 4, 'max_depth': 4, 'l2_leaf_reg': 31.27361535509004, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 43 with value: 0.8628189010522164.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 02:31:04,463]\u001b[0m Trial 55 finished with value: 0.8623314647519738 and parameters: {'iterations': 500, 'learning_rate': 0.08222021943922227, 'random_strength': 6, 'bagging_temperature': 5, 'max_bin': 6, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 5, 'max_depth': 4, 'l2_leaf_reg': 2.5013696924716973, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 43 with value: 0.8628189010522164.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:31:18,140]\u001b[0m Trial 56 finished with value: 0.8610512242661583 and parameters: {'iterations': 500, 'learning_rate': 0.11412893766081442, 'random_strength': 6, 'bagging_temperature': 4, 'max_bin': 6, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 3, 'max_depth': 4, 'l2_leaf_reg': 0.15433506787685775, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 43 with value: 0.8628189010522164.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:31:36,692]\u001b[0m Trial 57 finished with value: 0.8613923673116105 and parameters: {'iterations': 500, 'learning_rate': 0.085141915740434, 'random_strength': 7, 'bagging_temperature': 5, 'max_bin': 6, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 6, 'max_depth': 5, 'l2_leaf_reg': 2.5141006580046037, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 43 with value: 0.8628189010522164.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:31:52,378]\u001b[0m Trial 58 finished with value: 0.8612630878227835 and parameters: {'iterations': 500, 'learning_rate': 0.09589713030939544, 'random_strength': 6, 'bagging_temperature': 3, 'max_bin': 6, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 7, 'max_depth': 4, 'l2_leaf_reg': 0.43878407336956, 'one_hot_max_size': 1024, 'auto_class_weights': 'Balanced'}. Best is trial 43 with value: 0.8628189010522164.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:32:07,086]\u001b[0m Trial 59 finished with value: 0.8615545933637773 and parameters: {'iterations': 500, 'learning_rate': 0.12723347031619192, 'random_strength': 5, 'bagging_temperature': 5, 'max_bin': 6, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 8, 'max_depth': 3, 'l2_leaf_reg': 15.782010297515773, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 43 with value: 0.8628189010522164.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:32:24,763]\u001b[0m Trial 60 finished with value: 0.8603807833688792 and parameters: {'iterations': 500, 'learning_rate': 0.06767390848361435, 'random_strength': 6, 'bagging_temperature': 6, 'max_bin': 6, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 6, 'max_depth': 5, 'l2_leaf_reg': 0.0038331250495747165, 'one_hot_max_size': 500, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 43 with value: 0.8628189010522164.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:32:48,765]\u001b[0m Trial 61 finished with value: 0.8593328958417565 and parameters: {'iterations': 300, 'learning_rate': 0.04002795336574851, 'random_strength': 7, 'bagging_temperature': 6, 'max_bin': 4, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 5, 'max_depth': 4, 'l2_leaf_reg': 34.160901732588805, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 43 with value: 0.8628189010522164.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 02:33:12,403]\u001b[0m Trial 62 finished with value: 0.8612478466653123 and parameters: {'iterations': 500, 'learning_rate': 0.08298667194068889, 'random_strength': 8, 'bagging_temperature': 4, 'max_bin': 6, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 5, 'max_depth': 6, 'l2_leaf_reg': 8.990875597184502, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 43 with value: 0.8628189010522164.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:33:58,208]\u001b[0m Trial 63 finished with value: 0.8629142688449043 and parameters: {'iterations': 500, 'learning_rate': 0.03491751310099027, 'random_strength': 5, 'bagging_temperature': 5, 'max_bin': 8, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 4, 'max_depth': 5, 'l2_leaf_reg': 1.8211023402465396, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 63 with value: 0.8629142688449043.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:34:31,979]\u001b[0m Trial 64 finished with value: 0.8623054972865773 and parameters: {'iterations': 500, 'learning_rate': 0.04107559164106448, 'random_strength': 4, 'bagging_temperature': 5, 'max_bin': 8, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 4, 'max_depth': 5, 'l2_leaf_reg': 0.7995434990095341, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 63 with value: 0.8629142688449043.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:34:54,473]\u001b[0m Trial 65 finished with value: 0.8625410372679563 and parameters: {'iterations': 500, 'learning_rate': 0.07106096822317107, 'random_strength': 4, 'bagging_temperature': 5, 'max_bin': 8, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 4, 'max_depth': 5, 'l2_leaf_reg': 2.273919878847142, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 63 with value: 0.8629142688449043.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:35:12,578]\u001b[0m Trial 66 finished with value: 0.8619768911739152 and parameters: {'iterations': 500, 'learning_rate': 0.07435748178285664, 'random_strength': 3, 'bagging_temperature': 5, 'max_bin': 8, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 3, 'max_depth': 3, 'l2_leaf_reg': 2.341213045555641, 'one_hot_max_size': 100, 'auto_class_weights': 'Balanced'}. Best is trial 63 with value: 0.8629142688449043.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:35:27,906]\u001b[0m Trial 67 finished with value: 0.8613733031099311 and parameters: {'iterations': 500, 'learning_rate': 0.10061698498467272, 'random_strength': 4, 'bagging_temperature': 5, 'max_bin': 8, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 4, 'max_depth': 6, 'l2_leaf_reg': 0.10944962662679808, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 63 with value: 0.8629142688449043.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:35:51,483]\u001b[0m Trial 68 finished with value: 0.8613503283172014 and parameters: {'iterations': 500, 'learning_rate': 0.05331669990575415, 'random_strength': 5, 'bagging_temperature': 6, 'max_bin': 8, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 3, 'max_depth': 4, 'l2_leaf_reg': 0.00018285984431075052, 'one_hot_max_size': 12, 'auto_class_weights': 'Balanced'}. Best is trial 63 with value: 0.8629142688449043.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:35:59,589]\u001b[0m Trial 69 finished with value: 0.8566885276248998 and parameters: {'iterations': 100, 'learning_rate': 0.0641831119378872, 'random_strength': 5, 'bagging_temperature': 6, 'max_bin': 8, 'grow_policy': 'Depthwise', 'min_data_in_leaf': 4, 'max_depth': 5, 'l2_leaf_reg': 6.043144943879615, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 63 with value: 0.8629142688449043.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 02:36:50,508]\u001b[0m Trial 70 finished with value: 0.8604960386211935 and parameters: {'iterations': 500, 'learning_rate': 0.013594721532242872, 'random_strength': 2, 'bagging_temperature': 4, 'max_bin': 8, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 2, 'max_depth': 6, 'l2_leaf_reg': 1.77312277797755, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 63 with value: 0.8629142688449043.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 02:37:41,634]\u001b[0m Trial 71 finished with value: 0.862710225312353 and parameters: {'iterations': 500, 'learning_rate': 0.0350151166318715, 'random_strength': 4, 'bagging_temperature': 5, 'max_bin': 8, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 4, 'max_depth': 5, 'l2_leaf_reg': 0.9621617246015488, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 63 with value: 0.8629142688449043.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:38:21,519]\u001b[0m Trial 72 finished with value: 0.8627817365905435 and parameters: {'iterations': 500, 'learning_rate': 0.035001908536129056, 'random_strength': 4, 'bagging_temperature': 5, 'max_bin': 8, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 4, 'max_depth': 4, 'l2_leaf_reg': 3.13127627723157, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 63 with value: 0.8629142688449043.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 02:38:40,970]\u001b[0m Trial 73 finished with value: 0.8567171783050588 and parameters: {'iterations': 200, 'learning_rate': 0.021780985189808935, 'random_strength': 3, 'bagging_temperature': 5, 'max_bin': 8, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 2, 'max_depth': 5, 'l2_leaf_reg': 16.13766517425775, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 63 with value: 0.8629142688449043.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 02:39:20,994]\u001b[0m Trial 74 finished with value: 0.8625779304204432 and parameters: {'iterations': 500, 'learning_rate': 0.03326560521656461, 'random_strength': 4, 'bagging_temperature': 6, 'max_bin': 8, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 4, 'max_depth': 4, 'l2_leaf_reg': 0.336795984690881, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 63 with value: 0.8629142688449043.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 02:40:00,152]\u001b[0m Trial 75 finished with value: 0.8479172875566358 and parameters: {'iterations': 500, 'learning_rate': 0.005173235743062648, 'random_strength': 4, 'bagging_temperature': 4, 'max_bin': 8, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 4, 'max_depth': 4, 'l2_leaf_reg': 0.2896101424017571, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 63 with value: 0.8629142688449043.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 02:40:27,883]\u001b[0m Trial 76 finished with value: 0.8614776343294792 and parameters: {'iterations': 500, 'learning_rate': 0.03457634623751632, 'random_strength': 4, 'bagging_temperature': 5, 'max_bin': 8, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 4, 'max_depth': 3, 'l2_leaf_reg': 0.07457086425340669, 'one_hot_max_size': 1024, 'auto_class_weights': 'Balanced'}. Best is trial 63 with value: 0.8629142688449043.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 02:41:03,744]\u001b[0m Trial 77 finished with value: 0.8625625729970599 and parameters: {'iterations': 500, 'learning_rate': 0.0503357960741875, 'random_strength': 4, 'bagging_temperature': 6, 'max_bin': 8, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 4, 'max_depth': 4, 'l2_leaf_reg': 1.2106627375724541, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 63 with value: 0.8629142688449043.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:41:35,123]\u001b[0m Trial 78 finished with value: 0.861661843671647 and parameters: {'iterations': 500, 'learning_rate': 0.03262756198451842, 'random_strength': 3, 'bagging_temperature': 7, 'max_bin': 8, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 4, 'max_depth': 3, 'l2_leaf_reg': 0.447397179748981, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 63 with value: 0.8629142688449043.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 02:42:05,616]\u001b[0m Trial 79 finished with value: 0.8624566134591856 and parameters: {'iterations': 500, 'learning_rate': 0.0452970152697529, 'random_strength': 4, 'bagging_temperature': 6, 'max_bin': 8, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 3, 'max_depth': 4, 'l2_leaf_reg': 0.023319854557215667, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 63 with value: 0.8629142688449043.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:42:44,809]\u001b[0m Trial 80 finished with value: 0.8540335533364083 and parameters: {'iterations': 500, 'learning_rate': 0.010032703148291439, 'random_strength': 5, 'bagging_temperature': 5, 'max_bin': 8, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 3, 'max_depth': 4, 'l2_leaf_reg': 1.027664406808666, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 63 with value: 0.8629142688449043.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 02:43:19,053]\u001b[0m Trial 81 finished with value: 0.8629017607304437 and parameters: {'iterations': 500, 'learning_rate': 0.0494773083884324, 'random_strength': 4, 'bagging_temperature': 6, 'max_bin': 8, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 4, 'max_depth': 4, 'l2_leaf_reg': 3.3307922351471193, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 63 with value: 0.8629142688449043.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:43:50,426]\u001b[0m Trial 82 finished with value: 0.8624013913198265 and parameters: {'iterations': 500, 'learning_rate': 0.049542492974861296, 'random_strength': 4, 'bagging_temperature': 6, 'max_bin': 8, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 4, 'max_depth': 3, 'l2_leaf_reg': 0.20418092319233863, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 63 with value: 0.8629142688449043.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 02:44:13,826]\u001b[0m Trial 83 finished with value: 0.8619764751113526 and parameters: {'iterations': 500, 'learning_rate': 0.06873506936751295, 'random_strength': 5, 'bagging_temperature': 7, 'max_bin': 8, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 4, 'max_depth': 4, 'l2_leaf_reg': 3.5171984820667594, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 63 with value: 0.8629142688449043.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:44:53,678]\u001b[0m Trial 84 finished with value: 0.8627432322254434 and parameters: {'iterations': 500, 'learning_rate': 0.03421553291010362, 'random_strength': 4, 'bagging_temperature': 6, 'max_bin': 8, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 3, 'max_depth': 4, 'l2_leaf_reg': 1.279551978103478, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 63 with value: 0.8629142688449043.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 02:45:36,135]\u001b[0m Trial 85 finished with value: 0.8625860809785129 and parameters: {'iterations': 500, 'learning_rate': 0.033269810078548415, 'random_strength': 4, 'bagging_temperature': 6, 'max_bin': 8, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 3, 'max_depth': 5, 'l2_leaf_reg': 0.10983660655047413, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 63 with value: 0.8629142688449043.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:46:15,406]\u001b[0m Trial 86 finished with value: 0.8626428835931395 and parameters: {'iterations': 1500, 'learning_rate': 0.03404153880322145, 'random_strength': 3, 'bagging_temperature': 6, 'max_bin': 8, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 3, 'max_depth': 4, 'l2_leaf_reg': 0.11690785082262126, 'one_hot_max_size': 10, 'auto_class_weights': 'Balanced'}. Best is trial 63 with value: 0.8629142688449043.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:47:08,187]\u001b[0m Trial 87 finished with value: 0.8626456622293277 and parameters: {'iterations': 1500, 'learning_rate': 0.020650041705929434, 'random_strength': 3, 'bagging_temperature': 7, 'max_bin': 8, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 2, 'max_depth': 5, 'l2_leaf_reg': 0.027931864894058415, 'one_hot_max_size': 500, 'auto_class_weights': 'Balanced'}. Best is trial 63 with value: 0.8629142688449043.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:48:24,535]\u001b[0m Trial 88 finished with value: 0.8626965497081441 and parameters: {'iterations': 1500, 'learning_rate': 0.014966381186391205, 'random_strength': 2, 'bagging_temperature': 7, 'max_bin': 8, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 1, 'max_depth': 5, 'l2_leaf_reg': 0.02907660759142206, 'one_hot_max_size': 500, 'auto_class_weights': 'Balanced'}. Best is trial 63 with value: 0.8629142688449043.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:50:30,038]\u001b[0m Trial 89 finished with value: 0.8510351179301624 and parameters: {'iterations': 1500, 'learning_rate': 0.0013992145042054163, 'random_strength': 2, 'bagging_temperature': 8, 'max_bin': 8, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 1, 'max_depth': 5, 'l2_leaf_reg': 0.012752600162201205, 'one_hot_max_size': 500, 'auto_class_weights': 'SqrtBalanced'}. Best is trial 63 with value: 0.8629142688449043.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 02:51:20,823]\u001b[0m Trial 90 finished with value: 0.862443623043017 and parameters: {'iterations': 1500, 'learning_rate': 0.020255318965288774, 'random_strength': 2, 'bagging_temperature': 7, 'max_bin': 8, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 2, 'max_depth': 5, 'l2_leaf_reg': 0.0032786412356445942, 'one_hot_max_size': 500, 'auto_class_weights': 'Balanced'}. Best is trial 63 with value: 0.8629142688449043.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:52:32,738]\u001b[0m Trial 91 finished with value: 0.8627868022639097 and parameters: {'iterations': 1500, 'learning_rate': 0.016208026164651508, 'random_strength': 3, 'bagging_temperature': 7, 'max_bin': 8, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 2, 'max_depth': 5, 'l2_leaf_reg': 0.03229965403267229, 'one_hot_max_size': 500, 'auto_class_weights': 'Balanced'}. Best is trial 63 with value: 0.8629142688449043.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:53:40,379]\u001b[0m Trial 92 finished with value: 0.8623905719396159 and parameters: {'iterations': 1500, 'learning_rate': 0.016219596687074282, 'random_strength': 3, 'bagging_temperature': 8, 'max_bin': 8, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 2, 'max_depth': 5, 'l2_leaf_reg': 0.020324000210090877, 'one_hot_max_size': 500, 'auto_class_weights': 'Balanced'}. Best is trial 63 with value: 0.8629142688449043.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:54:17,055]\u001b[0m Trial 93 finished with value: 0.8623952786907809 and parameters: {'iterations': 1500, 'learning_rate': 0.024683334828658334, 'random_strength': 1, 'bagging_temperature': 7, 'max_bin': 8, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 1, 'max_depth': 5, 'l2_leaf_reg': 0.005206609735610267, 'one_hot_max_size': 500, 'auto_class_weights': 'Balanced'}. Best is trial 63 with value: 0.8629142688449043.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:55:59,178]\u001b[0m Trial 94 finished with value: 0.8601159023908957 and parameters: {'iterations': 1500, 'learning_rate': 0.006850829104537047, 'random_strength': 3, 'bagging_temperature': 7, 'max_bin': 8, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 2, 'max_depth': 4, 'l2_leaf_reg': 0.037270252908201226, 'one_hot_max_size': 500, 'auto_class_weights': 'Balanced'}. Best is trial 63 with value: 0.8629142688449043.\u001b[0m\n",
      "\u001b[32m[I 2021-02-17 02:57:17,634]\u001b[0m Trial 95 finished with value: 0.863132450431481 and parameters: {'iterations': 1500, 'learning_rate': 0.015393853845985255, 'random_strength': 3, 'bagging_temperature': 7, 'max_bin': 8, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 1, 'max_depth': 5, 'l2_leaf_reg': 0.03977425239611938, 'one_hot_max_size': 500, 'auto_class_weights': 'Balanced'}. Best is trial 95 with value: 0.863132450431481.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:58:40,984]\u001b[0m Trial 96 finished with value: 0.8626726388656365 and parameters: {'iterations': 1500, 'learning_rate': 0.012469721707158859, 'random_strength': 1, 'bagging_temperature': 8, 'max_bin': 30, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 1, 'max_depth': 5, 'l2_leaf_reg': 0.0005650382037853742, 'one_hot_max_size': 500, 'auto_class_weights': 'Balanced'}. Best is trial 95 with value: 0.863132450431481.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 02:59:33,794]\u001b[0m Trial 97 finished with value: 0.8622695225487712 and parameters: {'iterations': 1500, 'learning_rate': 0.014165527219236959, 'random_strength': 1, 'bagging_temperature': 8, 'max_bin': 30, 'grow_policy': 'Depthwise', 'min_data_in_leaf': 1, 'max_depth': 5, 'l2_leaf_reg': 0.00020647121997590114, 'one_hot_max_size': 500, 'auto_class_weights': 'Balanced'}. Best is trial 95 with value: 0.863132450431481.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 03:00:14,194]\u001b[0m Trial 98 finished with value: 0.8616028724032899 and parameters: {'iterations': 1500, 'learning_rate': 0.02661898255265924, 'random_strength': 2, 'bagging_temperature': 8, 'max_bin': 30, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 1, 'max_depth': 6, 'l2_leaf_reg': 0.0005062657170811357, 'one_hot_max_size': 500, 'auto_class_weights': 'Balanced'}. Best is trial 95 with value: 0.863132450431481.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped by overfitting detector  (10 iterations wait)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2021-02-17 03:02:18,705]\u001b[0m Trial 99 finished with value: 0.8587725661746266 and parameters: {'iterations': 1500, 'learning_rate': 0.0033222800504134065, 'random_strength': 2, 'bagging_temperature': 9, 'max_bin': 30, 'grow_policy': 'Lossguide', 'min_data_in_leaf': 1, 'max_depth': 5, 'l2_leaf_reg': 0.0014617936344147216, 'one_hot_max_size': 500, 'auto_class_weights': 'Balanced'}. Best is trial 95 with value: 0.863132450431481.\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "sampler = optuna.samplers.TPESampler(seed=68)  # Make the sampler behave in a deterministic way.\n",
    "study2 = optuna.create_study(direction=\"maximize\", sampler=sampler)\n",
    "study2.optimize(objective2, n_trials=100)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of finished trials: 100\n",
      "Best trial:\n",
      "  Value: 0.863132450431481\n",
      "  Params: \n",
      "    iterations=1500,\n",
      "    learning_rate=0.015393853845985255,\n",
      "    random_strength=3,\n",
      "    bagging_temperature=7,\n",
      "    max_bin=8,\n",
      "    grow_policy=Lossguide,\n",
      "    min_data_in_leaf=1,\n",
      "    max_depth=5,\n",
      "    l2_leaf_reg=0.03977425239611938,\n",
      "    one_hot_max_size=500,\n",
      "    auto_class_weights=Balanced,\n"
     ]
    }
   ],
   "source": [
    "print(\"Number of finished trials: {}\".format(len(study2.trials)))\n",
    "print(\"Best trial:\")\n",
    "trial2 = study2.best_trial\n",
    "print(\"  Value: {}\".format(trial2.value))\n",
    "print(\"  Params: \")\n",
    "for key, value in trial2.params.items():\n",
    "    print(\"    {}={},\".format(key, value))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "final_model_se = CatBoostClassifier(verbose=False,  cat_features=categorical_features_indices, \n",
    "                                    **trial2.params)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<catboost.core.CatBoostClassifier at 0x7f87e2537d50>"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "final_model_se.fit(X_train, y_train.seasonal_vaccine)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictions_se = final_model_se.predict_proba(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictions_se = predictions_se[:,1].reshape(-1,1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmEAAAHwCAYAAADuJ7gwAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAABa50lEQVR4nO3deXiUVYL2/+9JZV9IIIGwBsK+r2ERUBBQWRRQWWwV0bbH7mnt6f71LN3z9mzv9CzO9s50T2u7t23bKgEREEFFcQVBFgXZCWsgEEIgCSFrVZ3fH1XEiAECpHKS1P25rlyk6nlSdYeC1J3znOc8xlqLiIiIiDSuCNcBRERERMKRSpiIiIiIAyphIiIiIg6ohImIiIg4oBImIiIi4oBKmIiIiIgDKmEiIiIiDqiEiUiDMMYcNsZMcZ2jNmPMTmPMRNc5RETqohImIi2WtXaAtfbDxno+Y8w/GGOsMWZUHfe/XMf+1hjTs9bt24wxHxtjzhljCowxHxljZjZwRmOM+TdjTGHw49+NMeYy+88zxuwOZtpljJl90fbhwcylxph8Y8yPa237pTHmK2OM1xjzD3U89r3GmCPGmPPGmGXGmDYN+b2KNHUqYSIiDSBYZBYAZ4CF1/D1c4DFwEtAZyAd+DvgjgaMCfAIMBsYAgwGbge+f4lMnYCXgZ8CrYC/BF4xxrQLbk8D3gaeBlKBnsC7tR4iB/gr4K06HntA8OsWEPhey4Anr/ebE2lOVMJEHDPG/MwYczw40rDXGDM5eH+EMebnxpgDwRGL7NojBcaYxcaYk8aY4uBIxIBa26YHRy3OBR/7L2pt+xNjTI4x5owxZoUxpmOtbdYY8wNjzH5jzFljzBMXRkmMMT2MMWuDWU4bY/5ojEm5iu+zozGm/KLvYVjwsaKu9PjGmC7GmKXBEaJCY8xvLvqeao/WDA/eX3OINDgalW2MeSm4305jTNZF+V4PPv4hY8yf1fd7C7oR6Aj8GLjHGBN9FX83Bvh/wC+ttc9Za4uttX5r7UfW2j+5yhxXshD4L2vtMWvtceC/gAcvsW9noMhau9oGvAWcB3oEt/8UeMda+0drbaW19py1dveFL7bW/t5auxo4V8dj3we8aa392FpbCvwtcJcxJqlBvkuRZkAlTMQhY0wf4DFgpLU2CbgNOBzc/GcERiwmEHhzPws8UevLVwO9gHbAVuCPtbY9D3w/+JgDgbXB55sE/CswD+gAHAFeuyjW7cBIAiMl84KZAEzwazsC/YAuwD/U93u11uYBnwF317r7XmCJtbb6co9vjPEAK4N5uwGdLuQ2xswN7vcAgdGamUDhJWLMDH5dCrAC+E3wMSKAN4FtwceeDPzEGHNbcPt4Y0zRFb7FhcHHWBS8ffsV9q+tD4Hvd0l9vyB4KK/oMh8Zl/jSAQS+zwu2Be+ry2ZgtzFmpjHGEzwUWQlsD24fA5wxxqw3xpwyxrx5mee9bA5r7QGgCuhdz68Xaf6stfrQhz4cfRA4fHMKmAJEXbRtNzC51u0OQDUQWcfjpAAWSA7ePkrgEFOri/Z7Hvj3WrcTg4/ZLXjbAuNrbc8Gfn6J7LOBL2rdPgxMucL3+z1gbfBzA+QCN13p8YEbgIJLfO/vAD++xGPUZCJQ1N6rta0/UB78fDRw9KKv/Wvgd/V8HeOBEmB28PbTwPJa2/8BeLmOr7PBfwPjgp/HNsK/OR/Qt9btXsHnNpfY/2GgFPASOGQ4o9a2fUARgdIeC/waWFfHY7wM/MNF970P/OCi+44DE0P9d6APfTSVD42EiThkrc0BfkLgTfqUMea1WocHuwJvXBjZIFDKfEB6cFTi8eChyhK+Hj1LC/55NzAdOGICk7tvCN7fkcBo0oXnLyUwatSpVqyTtT4vI1DUMMa0C+Y7HnzOl2s9X30tAW4Ifo83EXjz/6Qej98FOGKt9dbxmF2AA/V8/ou/t1hjTCSBv+uOtUeSgP9DYK5SfdxJoKSsCt7+IzDNGNM2eNsLRNX+AmPMhdvVfD1y16Gez3c9SgmMGF7QCii11tqLdwweyv13YCIQTWBU9jljzNDgLuXAG9baTdbaCuD/AmONMcnXkONClroOXYq0SCphIo5Za1+x1o4nUAQs8G/BTbnANGttSq2PWBuYx3MvMIvACFoygUN0EBhdIvimOIvAocplBEa0APKCzxPY2ZgEAhOqj9cj6r8G8w221rYC7r/wfFfxvRYRmLg9L/g9vFrrzf9yj58LZAQL08Vy+XqO0rXKBQ5d9HedZK2dXs+vX0igrB41xpwkMME+CvhOcPtRvn6NLsgkUKqPA3uDGe6mnowx95nAGYmX+rjUYcGdBA41XzAkeF9dhgIfW2s328ActU3ARgL/7iBwWLJ2ebvweX3+XXwjhzGmOxBDYHRNJCyohIk4ZIzpY4yZZIyJASoIjCz4gpufAv7ZGNM1uG9bY8ys4LYkAnNzCgkcCvuXWo8ZHXyDTraBuVYltR7zFeAhY8zQ4HP+C7DRWnu4HnGTCIxeFJnAWXN/eY3f9isE5m/dHfy8Po//OXACeNwYk2CMiTXGjAtuew74C2PMCBPQ88Lf2VX4HCgxgZMk4oIjjQONMSOv9IXBrJMJzAEbGvwYQqBMXzhL8m2gjzFmgQmchNCGwN/9EmutN1hEfwr8rTHmIWNMKxM4MWO8MeaZup7XBibDJ17m4+glIr8E/NQY0yk4IvnnwIuX2HcTcOOFkS9jzDACJyBcmBP2O+DO4L+nKAKT6z8Nlm2C32ssgfeayODr5gl+7R+BO4wxNwZ/GfhHYKm1ViNhEj5cHw/Vhz7C+YPAEgGfEzgEc4bA5POOwW0RBN6Y9wa3HwD+JbgtEVgevP8IgVJzYX5RNIE3/bMECtgmvjnP6wfBx7rwfJ1rbbNAz1q3XwT+Kfj5AGALgaL0JYE372O19j3MFeaEBfeLC+beedH9V3r8DAKjeoXAaeDXF31Pe4NfuwMYdnEmLpqXRWBkyhKcZ0bgUO2rBA5ZngU21PraGwkcsqvr+/k5sKWO+zsSONQ4MHh7LPBp8LHzCMzPa33R10wlcHi2lMAcuA+pNQergf7NGQKHGM8EP/6dWvPBCIxQ3Vfr9mMElpo4BxwE/vyix/tTAqN5ZwmcmNDlon8/9qKPB2ttv5fAKOF5Av+e27j+P6kPfTTmh7H2W9MARERERCTEdDhSRERExAGVMBFpUMaY1ZeYKP5/XGcTEWlKdDhSRERExAGNhImIiIg4UNeaO01aWlqa7datm+sYIiIiIle0ZcuW09batnVta3YlrFu3bmzevNl1DBEREZErMsYcudQ2HY4UERERcUAlTERERMQBlTARERERB1TCRERERBxQCRMRERFxQCVMRERExAGVMBEREREHVMJEREREHFAJExEREXFAJUxERETEAZUwEREREQdUwkREREQcUAkTERERcUAlTERERMSBkJUwY8wLxphTxpgdl9hujDG/NsbkGGO2G2OGhyqLiIiISFMTypGwF4Gpl9k+DegV/HgE+G0Is4iIiIg0KZGhemBr7cfGmG6X2WUW8JK11gIbjDEpxpgO1toTocokIiIi4aei2kel1/+t+6M9EcRFexwkCghZCauHTkBurdvHgvephImIiIQhr8+P12/xW4vXb/H5LD5r8fkt+SUVVHr9eH2B216/H5/fUlhaxZmyKgCqvX6qfH4WbcolyhNBtc9PaaWXsipfnc93/5gM/mn2oMb8Fr/BZQkzddxn69zRmEcIHLIkIyMjlJlERESklpKKao4WlrH35Llg+bH4/P7gn5ZKr58dx4tJiY/G7w+Upgt/+oKFqqLaz868YlITYvAH77+w3ecP7J9XXNFgmaMjI4iKMNwxpCMxkRFU+y2pCdEkx0VRmrsbrCWp6wD6tU9qsOe8Fi5L2DGgS63bnYG8una01j4DPAOQlZVVZ1ETEREJd/5gKar0+sg9U06Vz1dr5ChYfHyWfafOkZNfGihENvB1gZElakqS31o+2X/6qp4/vVUMHmOIiDB4IszXnxtDXFTgsF9Gm/jAtjr2OV/lpV+HVkTW2h4ZEdju9Vk6JMeSFBtFpOfrbZ4IQ1JMFG2TYogK3m/Mt8d5KisrWbFiBcf376J///7MGdetzv0ak8sStgJ4zBjzGjAaKNZ8MBERCUellV5OFJXjsxavz349WhT8yD1bTu6ZMvJLKvBbS5XXz5ajZ2kVG4XXZ6ny+Tl0+vw1PXdmWgIRBiIjIoLliZpilNW1NSnxUYzrmUavdkl0SIklMSbyGwUoMiKCSI8hytN0V70qKCggOzubwsJCpkyZwtixY50XMAhhCTPGvApMBNKMMceAvweiAKy1TwGrgOlADlAGPBSqLCIiIo3NBuc1VXr9nKuo5r1d+Ww5chZjDFU+P16fn2qfZe2eU1f1uKkJ0STERBLtieDs+SoGd04hKjKCYRkplJRXM6JrG2KjIvD5Ld3bJhAb6SHiotIUEQE92iYSG+VuUnpjKS8v5/nnnycyMpIFCxaQmZnpOlKNUJ4d+Z0rbLfAo6F6fhERkatlgyNQ1T5LXnE5Xp+lOjhZ3BscbfL6LV8dL8bns6w/eJqUuOiaEasLk8XzSyopr657MjhA97YJRHsCI0hDuqTg8/uZ1Dedfu2Tag7NeTzBPyMMEcbQPjmWzq3jmvSIU1NircUYQ1xcHDNmzKBr1660atXKdaxvcHk4UkREpEFdGH2q8vrZf6qUZz4+wLkK77fOqPP6LXtOngvMNzLmGxPFr0bn1nGcr/LSPS0xMNLk+XrEqaS8mn4dWpEUG0lMpIfUxGhu7d+e6EiVqFArLS3l9ddfZ+zYsfTq1YtBg9ydAXk5KmEiItKkVVT7OHa2jIpqP0cKy6io9lHl81NR7ePL3CLioyPZdaKEg6dKOVfprfMxsrq2JsoTQUzU1/OZOreO48z5KoZ2ScETEVHrcF2gTFVU++mTnhSc7xQ4jOeJMHRKiaN1QjRJMZFERLifVyTfdPToURYvXkxFRQVVVVWu41yWSpiIiDSaSq+PorJqKqv9nK/ycrDgPF8cPUt0ZETNYUCf309ZlY/1Bwo5UVxOfQanOreOIybKw9CMFEZntiEm0kN0ZAQDOrYiq1ub0H9j4py1lo0bN7JmzRpSUlK4//77SU9Pdx3rslTCRESkwX2ZW8Sn+wvw+i2bD59lX/45/NZyuvTSIxMxkRFEBedJRRiDAbqlJTC2RyrtW8XSOz2J2CgPbRKiSU2MJibSQ1yUx+mK59J05OTk8M4779CnTx9mz55NbGys60hXpBImIiLXJL+kgm25RWw7VsTGg2coLq+umbhemyfCEB/loXu7RGYN7URclIfMtARiogKla3DnZDokxzn6LqS583q9REZG0rNnT+bPn0+fPn2axPIT9aESJiIil1RSUc2pkgoOny6jpKKa9/ecotrr591d+d/at0NyLMMzWtMrPZG0xBgm9W3HwE7JeDRvSkJk9+7drF69mgceeIC0tDT69u3rOtJVUQkTERFOnatgx/Fi9uWXUlrhZf2B02w/VvytUa0Lsrq2JtJjuG90VwZ1SiajTbwmqUuj8fv9vP/++6xfv55OnToRFRXlOtI1UQkTEQkj1lre+OI4e06eI+dUKTuOF+O3cLq08hv7JcVEkpYYw7CMFLK6taFbajypiTF0bh1HakJ0szncIy3P+fPnWbJkCYcPHyYrK4vbbruNyMjmWWeaZ2oREbkiv99SXu3jSGEZy788zopteZy46CLJ3dMSSIiJZPbQjnRLS+CGHql0SI4lPlpvD9I0rV+/nmPHjjF79myGDBniOs510f8yEZFmzuvz82nOab7MLWLLkbN8sv80xoCt40jikC4p9GufxC9m9CMptnkewpHwY62lvLyc+Ph4br75ZoYMGUK7du1cx7puKmEiIs2E32/Zf6qULUfOsiOvGGste0+eY+vRom/sF2FgdGYqo7sH1ssCGNipFV1ax9MtLcFBcpFrV1VVxcqVKzl+/DiPPPIIMTExLaKAgUqYiEiTVeX1s+1YEX/47AgrtuXVuU/X1Hg6t45jct923DemK11T42uKl0hzV1hYSHZ2NqdOneLmm28mOjradaQGpRImItIEFJVV8e7OfN7fk4/XZ9mRV0x+yTcny9/aP50BHZMZmdmaAR2TaRUbqQny0mLt2bOHZcuWERERwX333UfPnj1dR2pwKmEiIg74/YGzFJ/4MIfDp89/69I8ozLb0Co2ipv7tmP6oA4M6ZyswiVhw1rL+vXradOmDfPmzSMlJcV1pJBQCRMRaSQ+v+Wpjw6w6fAZPtxbUHN/WmI0E3q3Y1hGChP7tKVz63iHKUXcOX/+PMYY4uPjmT9/PjExMc12+Yn6aLnfmYiIYzmnzvHRvtNsPnyG7ceKOV5UXrOte1oCcdEeXntkjM5SFAGOHz9OdnY2HTp04J577iEhoeWfRKISJiLSQKy1FJdX8+fZ23h/z6lvbOuaGk+PtgncO7orD9zQlShPhKOUIk2LtZYtW7bw9ttvk5SUxIQJE1xHajQqYSIi12j3iRLW5Zxm8+GzREdGfOsMxtGZbfjRpF50TY2nSxsdYhS5WHV1NW+99Rbbtm2jZ8+e3HXXXcTFhc/F3FXCRETqoaisioJzlRwoOM8Lnx7i88NnvrE9ymPonZ6IwfDDm3twx+COupaiyBVUVVVx6NAhJkyYwIQJE8Lu5BOVMBERoKLax5Itxyg4V0mVz0+V18+uvBIslg0Hz3xr/4RoDz3bJfLjKb24sVdbHV4UuQpHjhyhc+fOJCQk8MMf/pCYmBjXkZxQCRORsHSgIHDx6s8OFHLw9Hk+P/R10YryGKI9EXgiDBXVfm7slUaruCjGBq+rmJmWSKZWnhe5an6/n48++oiPP/6YW265hbFjx4ZtAQOVMBEJA1VeP+/uOslTHx2g2mvJPVtGWZXvG/t0SI7l7uGd+cHEHiTG6EejSEMrKytj6dKlHDhwgKFDhzJy5EjXkZzTTxoRaVFKKqr57YcHKCqr5qO9p8grrvjWPrOHdsQTEcFtA9IZldmGlPiWdSkUkabmxIkTLFq0iNLSUm6//XaGDx8edvO/6qISJiLNWrXPz4d7C9h85Axnz1eRvflYzbY2CdGkJUZzS/90eqcnMXNIR1ITw/fQh4hLkZGRPPTQQ3Tq1Ml1lCZDJUxEmp19+ed444vjrM85zbZjxd/Y1qtdIjf1bsvfzOin37RFHKqurmb37t0MHjyYDh068MMf/pCICJ3AUptKmIg0WcXl1ezKK6G82su5Ci8rt59gw8FCzlV4a/bp0iaOG3u15f7RXenTPgmPloUQca6oqIjs7GxOnDhBu3btaN++vQpYHVTCRKRJOVFczo9e+YLNR85ecp8be6XxpxN6cEOPVI12iTQxOTk5LF26FL/fzz333EP79u1dR2qyVMJExLnC0kp+/9kR3t+dz868kpr7b+yVxtSB7emTnkRibCQpcdG0T451mFRELmfdunW89957pKenM2/ePNq0aeM6UpOmEiYiTlhrOVxYxt8s+4p1OYU19/dtn8R3x2UyN6uzRrlEmpnk5GQGDx7M7bffTlSULkx/JSphItJo1h84zf+s2U9ZtZcdx0u+se2/5w9hxqCOREdq3ohIc3LixAlOnz7NoEGDGDhwIAMHDnQdqdlQCRORkCiv8vHapqPkl1Ty8oYjlFZ+PZm+bVIMdwzpSGJMJLf2T2dczzSVL5Fm6Msvv+Stt94iKSmJfv36ERmpWnE19LclIg3mXEU1z396iJc3HOV0aWXN/a1iAz9q7hudwV3DOzGiq+aJiDRnXq+X1atXs3XrVjIzM7n77rtVwK6B/sZE5LoUl1fz9EcHWHfRml3jeqYypV863xmVQWyUx2FCEWlIPp+PF198kePHjzNu3DgmTZqk5SeukUqYiFyT9Tmn+e1HB/hk/+ma+zLaxDNraEcevbmnipdIC+XxeOjduzfjx4+nb9++ruM0ayphInJVqrx+/nrpV7y+NXB5oKTYSOZldeEX0/sRoYVSRVokay2ffvopGRkZdO3alZtuusl1pBZBJUxE6mXjwUL+ZdXubxxy/K+5Q7h7RGeHqUQk1CoqKnjjjTfYt28fo0ePpmvXrq4jtRgqYSJyWcVl1Xz/5c1sOHgGgOS4KP7kxkwevbmn1vESaeHy8/NZtGgRxcXFTJs2jZEjR7qO1KKohInINxSXVfPCukN8mVvEzrxiTpdW1Wx77ZExjOme6jCdiDSW/Px8nnvuOeLi4njwwQfp0qWL60gtjkqYiADg91teXH+Yf1y5q+a+vu2TGJ2Zyg09UvnOqAxdHFskjLRr146xY8cycuRIEhMTXcdpkVTCRMJctc/P0x8d4D/f3Vdz38PjM/nL2/roDEeRMFNSUsLKlSuZPn06KSkp3Hzzza4jtWgqYSJhylrL1qNF3P3b9TX3jejamv/9zjA6psQ5TCYiLhw6dIglS5bg9XopLCwkJSXFdaQWTyVMJAwdLSzjpv/4oOZ2p5Q4Vv/kRlrF6oK7IuHGWsu6detYu3YtqampzJ8/n7S0NNexwoJKmEgYyD1TxqJNuby26ShFZdV4/bZm20vfHcVNvds6TCciLm3YsIH333+fAQMGMHPmTKKjo11HChsqYSItkLWWdTmFrNl1klc35VLl9ddsS0uMYc6IzgzLSOHW/ulaZkIkTFlrMcYwfPhwYmJiGDZsmH4eNDKVMJEW4tS5Ct7Zmc+v399PwbnKb2zr36EVP5vWl3E9Uon06BpvIuHuq6++4vPPP+eBBx4gJiaG4cOHu44UllTCRJo5ay1PfJDzjbMbM9rEMyqzDX9yY3e6t00gSsVLRAhcfPvdd9/l888/JyMjg6qqKqKiNBfUFZUwkWZs94kSpv3qk5rbU/q143/uGUZijP5ri8g3nTt3jsWLF5Obm8uYMWOYMmUKHo+WoXFJP6lFmiFrLT97fTvZmwMX0b6xVxrPLxxJdKRGvESkbitWrODkyZPcfffdDBw40HUcQSVMpFmx1nLmfBVZ//weNniC44xBHXjiPs3nEJFvs9bi8/mIjIxk+vTpeL1e2rbV2dBNhUqYSBNWUe3jf97bT+6ZMt766sS3tu/8v7eRoEOPIlKHyspKVqxYgbWWuXPn0rp1a9eR5CL66S3SxCzenMunOaf5eF8BZ8uqa+7vnpZAfIyHKf3SyUxLYOaQjjqdXETqVFBQQHZ2NoWFhUyePNl1HLkElTCRJsBayx83HuWf3tpFRXVgTa8hnZPpne7hpt5teeCGriRpNXsRqYedO3eyfPlyoqOjeeCBB+jWrZvrSHIJKmEiDhw6fZ7frz9MSXk1FV4fa3blU+0LTPIalpHCv909mN7pSY5TikhzU1FRwapVq2jfvj1z5syhVatWriPJZaiEiTQiay3j/+0DjheV19zXo20CfdonERvp4akFI0hLjHGYUESao7KyMuLi4oiNjeXBBx+kTZs2Wn6iGVAJEwkxv9+y8qsTLN6cyyf7T9fc/4eHRzGuRxoREZrXJSLX7ujRoyxevJiRI0dy00036ezHZkQlTCQErLV8dbyYVzYe5bVNuTX3J0R76JmexLMLRtCuVazDhCLS3Flr2bhxI2vWrCElJYU+ffq4jiRXSSVMpIEdLSxjyv/7iCrf1xfNvmtYJ34xox+pOtQoIg2gqqqKFStWsHPnTvr06cPs2bOJjdUvds2NSphIA3rigxz+4529APRtn8RfTe3DyG5tdGajiDSoU6dOsWfPHiZPnsy4ceO0XE0zpRIm0gBOFJdzw7+urbn96M09+Mvb+jpMJCItUUFBAW3btqVz5878+Mc/JilJZ1E3ZyphIg3gF2/sACCjTTxPLxhBvw46LVxEGo7f7+e9997js88+Y8GCBXTv3l0FrAVQCRO5DtZafr/+MGv3nALg47+62XEiEWlpSktLef311zl8+DBZWVlkZGS4jiQNRCVM5BpVef2Mffx9TpdWAfDg2G5uA4lIi5Obm8vixYspLy9n9uzZDBkyxHUkaUAqYSJXqdLrY+y/rqXwfKB8xUV5+ORnN2uRVRFpcAUFBURGRvLwww/Tvn1713GkgamEidRTUVkVq3ec5K+XflVz39/M6MfD4zN1ZpKINJiqqipOnjxJRkYGw4cPZ+DAgURHR7uOJSGgEiZyGdZanv74INtyi1i942TN/aMz2/D7744iNkqXBRGRhlNYWEh2djZFRUX85Cc/IS4uTgWsBVMJE7mM5z89xOOr9wDQs10itw/uwINju5ESrx+KItKw9uzZw7Jly4iIiGDevHnExcW5jiQhphImUoeKah/ffXET6w8UAvDu/3cTvdN1OriINDxrLWvXruXTTz+lY8eOzJ07l5SUFNexpBGohInUcrSwjPUHTvPzWvO+fj6trwqYiISMMYbKykqGDx/OtGnTiIzUW3O40CstEjT/6c/YeOjMN+7b88upmvclIiFx7NgxPB4PHTp0YNq0aTrBJwyphElYq6j28egft/J+cLFVgP/9zjBu6tWW5Hhd71FEGp61ls2bN/P222+TkZHBwoULVcDCVEhLmDFmKvArwAM8Z619/KLtycDLQEYwy39aa38XykwitQ38+3fw+i0Agzsn84fvjlb5EpGQqa6u5q233mLbtm307NmTu+66y3UkcShkJcwY4wGeAG4BjgGbjDErrLW7au32KLDLWnuHMaYtsNcY80drbVWocokAbMstYtYT62pu5/zzNCI9EQ4TiUhLV1payssvv0x+fj4TJkxgwoQJGgELc6EcCRsF5FhrDwIYY14DZgG1S5gFkkzgX2EicAbwhjCTCG/vOMEPXt4KwMBOrVj0yA0qYCIScvHx8bRp04bJkyfTq1cv13GkCQhlCesE5Na6fQwYfdE+vwFWAHlAEjDfWusPYSYJc89+fJB/XrUbgL+a2ocf3NSDiAj9JioioeH3+1m/fj1Dhw4lMTGRefPmuY4kTUgoS1hd72z2otu3AV8Ck4AewBpjzCfW2pJvPJAxjwCPALp6vFyzvSfP1RSwv7ytDz+c2NNxIhFpycrKyli6dCkHDhzA4/Fwww03uI4kTUwoS9gxoEut250JjHjV9hDwuLXWAjnGmENAX+Dz2jtZa58BngHIysq6uMiJXJbPb/k/S79i0ebAwOx/zh3CnBGdHacSkZYsLy+P7OxsSktLueOOOxg+fLjrSNIEhbKEbQJ6GWMygePAPcC9F+1zFJgMfGKMSQf6AAdDmEnCSFmVl39ZtZuXNxytue/7E7qrgIlISOXk5PDaa6+RmJjId7/7XTp27Og6kjRRISth1lqvMeYx4B0CS1S8YK3daYz5QXD7U8AvgReNMV8ROHz5M2vt6VBlkpbP57e8svEIv1y5myrf19MLJ/RuyzMPjCAmUguvikhoderUiUGDBnHLLbcQHx/vOo40YSZwJLD5yMrKsps3b3YdQ5qg7M25/NWS7TW3u6bG89DYbtw3pitROvtRRELo7NmzfPrpp0yfPh2PR7/sydeMMVustVl1bdOK+dIirNmVX1PAFt7QlV/M6E90pIqXiITe/v37Wbp0KQAjR46kffv2jhNJc6ESJs3eTxd9ydIvjgOBZSd01qOINAZrLR999BEfffQR6enpzJs3jzZt2riOJc2ISpg0W2v35POnL2+l0huY+/XKn4xmbI80x6lEJFysXr2aTZs2MWTIEGbMmEFUlC55JldHJUyapQ0HC/nui4G5gSnxUXz0Fzfrmo8i0qiysrJo164dI0aM0OWH5JqohEmz8/RHB/jX1XsA+Jc7B3HvaC3gKyKN44svviAvL4/p06fTrl072rVr5zqSNGMqYdJsWGt56MVNfLi3AIDH7xrEPaNUwEQk9LxeL6tXr2br1q1kZmbi9Xp1+FGum0qYNBv3P7+RdTmFAPzHnMHMzepyha8QEbl+RUVFLF68mLy8PMaPH8/NN99MRITOvpbrpxImTd6+/HPMe/ozisqqAdj+D7fSKla/gYpI6Pn9fl566SXKysqYP38+ffv2dR1JWhCVMGmy1uzK50evbqWiOnD2Y6eUOJb+cKwKmIiE3IWFzCMiIpgxYwYpKSmkpqY6TiUtjUqYNEkPvPA5H+8LzP0a0LEVv5jRT8tPiEijqKio4I033iAzM5MxY8bQo0cP15GkhVIJkybnxXWHagrYqj+7kf4dWzlOJCLh4uTJk2RnZ1NcXEzPnlr4WUJLJUyaDL/fMuZf3+fUuUoAXvruKBUwEWk027ZtY+XKlcTFxfHggw/SpYtO/pHQUgmTJsFay9jH19YUsKU/HMvwjNaOU4lIuCgoKGDZsmV069aNu+++m8TERNeRJAyohEmTMPuJdZwsqSAxJpLPfzGZ+Gj90xSR0KuuriYqKoq2bduyYMECunXrpuUnpNHoX5o4Za3lzifXse1YMQCf/uxmFTARaRQHDx7kV7/6FQcOHACge/fuKmDSqPRuJ07d//xGvjhaBMDHf3kzKfHRbgOJSItnrWXdunWsXbuWtLQ0kpOTXUeSMKUSJs689NnhmhXwN//NFNISYxwnEpGWrqKiguXLl7Nnzx4GDBjAzJkziY7WL3/ihkqYOOHzW/5u+U4A/vDwKBUwEWkUO3bsYN++fdx2222MHj0aY4zrSBLGVMKk0ZVWehn49+8AML5nGjf2aus4kYi0dOfOnSMpKYkRI0aQkZFBu3btXEcS0cR8aXwT/+PDms9ffGikuyAi0uL5fD5WrVrFk08+SVFREcYYFTBpMjQSJo3qV+/t53RpYC2ww4/PcJxGRFqykpISFi9ezLFjxxgzZgxJSUmuI4l8g0qYNJpqn5//fm8fAK98b7TjNCLSkh0+fJglS5ZQVVXFnDlzGDBggOtIIt+iEiaNZuZv1gEwomtrxvbUxbhFJHS2bdtGbGwsCxcupG1bzTuVpkklTBrF/Kc/Y/eJEgCeun+E4zQi0hJVVlZSVlZG69atmT59On6/n5gYnXktTZdKmITcbf/9MXvzzwHwwoNZtE3SD0URaVgFBQUsWrQIj8fD97//faKiolxHErkilTAJqbyi8poCtu3vbyU5Tj8YRaRh7dixgxUrVhAdHc2cOXN06SFpNlTCJGSstYx9fC0AP5zYQwVMRBqUz+djzZo1bNy4kS5dujBnzhxatWrlOpZIvamESUgcO1vG9F99UnP7p7f0dphGRFoiay25ubmMGjWKW2+9FY/H4zqSyFVRCZOQeOCFzymp8ALwxd/eQqRHhwdEpGHk5uaSlpZGXFwcDz74oOZ/SbOld0ZpcKdLKzlYcB4ILMjaOkEXxxWR62et5bPPPuN3v/sda9cGpjqogElzppEwaVBen5+sf3oPgPlZXRynEZGWoqqqihUrVrBz50769u3L5MmTXUcSuW4qYdJgzld6GRC8MHef9CQev3uQ40Qi0hKcOXOGV199lcLCQiZPnsy4ceMwxriOJXLdVMKkwfzv2pyaz1f9+Eb9kBSRBhEVFYXH42HBggVkZma6jiPSYDQnTBrEieJynvroAABf/cOteCJUwETk2vn9fjZv3ozf7ycpKYnvf//7KmDS4mgkTK5b7pkybvz3DwC4bUA6SbGaKCsi1660tJTXX3+dw4cPk5iYSN++fTWyLi2SSphct4W/+xyAyX3b8fSCLMdpRKQ5y83NZfHixZSXlzN79mz69u3rOpJIyKiEyXXZcuRMzXIUzz840nEaEWnOvvzyS958802Sk5N5+OGHad++vetIIiGlEibX7LXPj/LzpV8B8OPJvRynEZHmrm3btvTt25c77riD2NhY13FEQk4T8+Wa/OObu2oK2N/e3p//T5clEpFrUFhYyGeffQZAp06dmDt3rgqYhA2NhMlV+4cVO3lx/WEAfnPvMG4f3NFtIBFplnbv3s3y5cvxeDwMHjyYhIQE15FEGpVKmFyVimpfTQHb8NeTaZ+s31hF5Or4/X7Wrl3LunXr6NixI/PmzVMBk7CkEib1drK4gtv/9xMApvRrpwImIlfNWsuiRYvYt28fI0aMYOrUqURG6q1IwpP+5Uu93fyfH1Je7aN72wT+a95Q13FEpBkyxjBw4ED69u3LsGHDXMcRcUolTOplz8kSyqt9AKz984luw4hIs2KtZfPmzURHRzNkyBAGDdJ1ZUVAZ0dKPU39n8BhyP+aO8RxEhFpTqqrq1m2bBmrVq1i7969WGtdRxJpMjQSJle0LbcIgL7tk7h7RGe3YUSk2Thz5gzZ2dnk5+czceJEbrrpJl1+SKQWlTC5ollPrAPgFzP6OU4iIs1FaWkpzz77LMYY7rvvPnr27Ok6kkiToxIml/XURwdqPr+xV1uHSUSkOUlMTOTmm2+md+/epKSkuI4j0iRpTphcUn5JBY+v3gPAez+d4DiNiDR1ZWVlvPrqqxw7dgyAUaNGqYCJXIZKmNTJWsvof3kfgPlZXejZLtFxIhFpyo4fP84zzzzDgQMHOHv2rOs4Is2CDkdKnX65cnfN5/82Z7DDJCLSlFlr2bp1K6tXryYxMZHvfve7dOyoS5mJ1IdKmHyDz2/5s1e/4K2vTgCw7e9udZxIRJqyPXv2sHLlSnr06MFdd91FfHy860gizYZKmNTIL6moOQQJ8L/fGUZyfJTDRCLSVPn9fiIiIujTpw+zZs1i8ODBRERohovI1dD/GAHA7/96DtiQzsns+eVU7hiiQwoi8m379+/nt7/9LSUlJURERDB06FAVMJFroJEw4blPDvJPb309B2z5Y+MdphGRpsrv9/PRRx/x8ccfk56ejs/ncx1JpFlTCQtza/fk1xSwSX3b8dwDWY4TiUhTVF5eztKlS8nJyWHIkCHMmDGDqChNVxC5HiphYcxay09e+xKAVX92I/07tnIbSESarLVr13Lo0CFmzJjBiBEjdPkhkQagEhamKqp93PG/n1JS4SU6MkIFTETqVFVVRXR0NJMnT2bYsGFafkKkAWkmZZia+B8fsv9UKQCf/tXNjtOISFPj9XpZsWIFL774Il6vl9jYWBUwkQamkbAw9Oa2PE6WVABw6F+n67CCiHxDUVER2dnZnDhxgvHjx+vMR5EQUQkLM+/vzudHr34BwNIfjlUBE5FvyMnJYenSpfj9fu655x769OnjOpJIi6USFkZe+PQQ/7hyFwA/mtST4RmtHScSkabE7/fz3nvvkZSUxLx580hNTXUdSaRFUwkLEwXnKmsK2FP3j2DqwPaOE4lIU1FeXo7H4yE6OprvfOc7xMXFER0d7TqWSIunA/1h4s4n1wGw8IauKmAiUuPkyZM8++yzrFq1CoDk5GQVMJFGopGwMOD1+Tl2thyA/ztroOM0ItJUbNu2jZUrVxIXF8eIESNcxxEJOyphYeDCUhRzRnR2nEREmgKv18vbb7/Nli1b6NatG3PmzCEhIcF1LJGwoxLWwllrmfarTwA0EV9EADh//jy7du1i3LhxTJo0SUtQiDiiEtbCPffJoZrP54/s4jCJiLh24sQJ2rdvT3JyMo899hjx8fGuI4mENf3608K9s/MkAF/87S14IrQmmEg4stbyySef8Oyzz7JlyxYAFTCRJkAjYS1YXlE5m4+cBaB1gs52EglHFRUVLFu2jL179zJw4EAGDx7sOpKIBIW0hBljpgK/AjzAc9bax+vYZyLwP0AUcNpaOyGUmcLJsi+PA/DX0/o6TiIiLuTn55OdnU1RURFTp05l1KhRukqGSBMSshJmjPEATwC3AMeATcaYFdbaXbX2SQGeBKZaa48aY9qFKk84+ve39wIwe1gnx0lExIWysjK8Xi8LFy4kIyPDdRwRuUgo54SNAnKstQettVXAa8Csi/a5F1hqrT0KYK09FcI8YSU/eIFugPRWsQ6TiEhj8vl87N+/H4DMzEx+9KMfqYCJNFGhLGGdgNxat48F76utN9DaGPOhMWaLMeaBEOYJK794YwcAP5jQw3ESEWksJSUlvPjii7zyyisUFBQAEBmpqb8iTVUo/3fWNfHA1vH8I4DJQBzwmTFmg7V23zceyJhHgEcA/UZXD36/5b3d+QD8XPPBRMLCoUOHeP3116mqqmLOnDm0bdvWdSQRuYJQlrBjQO2FqToDeXXsc9paex44b4z5GBgCfKOEWWufAZ4ByMrKurjIyUXueWYDAKMz2zhOIiKN4bPPPmPNmjWkpqaycOFCFTCRZiKUhyM3Ab2MMZnGmGjgHmDFRfssB240xkQaY+KB0cDuEGZq8cqrfHx++AwAf3h4tOM0ItIYPB4P/fr143vf+54KmEgzErKRMGut1xjzGPAOgSUqXrDW7jTG/CC4/Slr7W5jzNvAdsBPYBmLHaHKFA7mPLU+8OeIzkRHai1ekZbq1KlTFBcX06tXL0aOHMnIkSO1/IRIMxPSGZvW2lXAqovue+qi2/8B/Ecoc4SLXXkl7MwrAeAfZw1wnEZEQmXHjh2sWLGCpKQkevTooWs/ijRTOm2mBZn+68CFuv/ytj7ER+ulFWlpfD4fa9asYePGjXTp0oW5c+eqgIk0Y3qnbiEOFpQCEBsVwaM393ScRkQaWnV1NS+//DJHjx5l9OjR3HLLLXg8HtexROQ6qIS1EI++8gUAf3e7DkOKtERRUVF06NCBrKwsBg0a5DqOiDQAlbAWYveJwFyw74zqcoU9RaS5sNayceNGMjMzSU9PZ+rUqa4jiUgD0mSCFuDPs7cB0Ld9ks6OEmkhKisrWbJkCe+88w5ffPGF6zgiEgIaCWvmFjy/kU/2nwbgxYdGOU4jIg2hoKCA7OxsCgsLmTJlCmPHjnUdSURCQCWsGTt8+nxNAXv/zyfQPlkX6hZp7k6cOMGLL75IZGQkCxYsIDMz03UkEQkRlbBm7CeLvgTg4fGZ9Gib6DaMiDSIdu3aMWTIEMaPH0+rVq1cxxGRENKcsGaqyuvny9wiAP729v5uw4jIdSktLWXp0qWUlZXh8XiYPn26CphIGFAJa6b+YnFgMv7soR0dJxGR63H06FGefvppdu/ezYkTJ1zHEZFGpMORzdSek4ElKf75Tq0XJNIcWWv5/PPPeffdd0lJSeH+++8nPT3ddSwRaUQqYc1QRbWPffmBFfITYvQSijRH69ev57333qNPnz7Mnj2b2FidWCMSbvQO3sy8veMEP3h5KwD9O2jOiEhzNXToUDweD6NHj9b6fiJhSnPCmpGdecU1BWzawPas/NF4x4lE5Grs3r2bV199FZ/PR0JCAmPGjFEBEwljGglrJqy1zPj1pwD8/R39eWic1g4SaS78fj9r165l3bp1dOrUiYqKChISElzHEhHHVMKaiQ0Hz9R8/uDYbu6CiMhVOX/+PEuWLOHw4cOMGDGCqVOnEhmpH70iohLWbFxYE+z5hVk6fCHSjCxevJjjx48za9Yshg4d6jqOiDQhKmHNxMsbjgAwuHOK2yAickXWWqy1REREMHXqVADat2/vOJWINDUqYc1ElMeQmhBN26QY11FE5DKqq6tZuXIl0dHRzJgxQ+VLRC5JZ0c2A3tOlnC4sIx2rbSOkEhTdubMGZ5//nm2b99OYmIi1lrXkUSkCdNIWDPw/T9sAeCOIR0cJxGRS9m7dy9vvPEGERER3HffffTs2dN1JBFp4lTCmrhKr48jhWUA/HCifqiLNEVlZWUsXbqU1NRU5s2bR0pKiutIItIMqIQ1cZ/sOw3ATb3bOk4iIherrKwkOjqa+Ph4HnjgAdLT07X8hIjUm+aENXF5xeUA/PW0vo6TiEhtx48f58knn2TLlsB0gU6dOqmAichV0U+MJu6XK3cB0Do+2nESEYHA8hNbtmzh7bffJjExkY4dO7qOJCLNlEpYE+bzW6p9gbOr2ifrzEgR16qrq3nrrbfYtm0bPXv25M477yQ+Pt51LBFppq66hBljPMA91to/hiCP1PK7dYcAuKF7quMkIgKQm5vL9u3bmTBhAjfddBMREZrRISLX7pI/QYwxrYwxf22M+Y0x5lYT8CPgIDCv8SKGr+LyagCeuG+44yQi4a24uBiA7t278+ijjzJx4kQVMBG5bpf7KfIHoA/wFfA94F1gDjDLWjurEbKFvT0nzxFhoE2C5oOJuOD3+/nggw/49a9/zbFjxwBITdXItIg0jMsdjuxurR0EYIx5DjgNZFhrzzVKsjBXXuVjza580hJVwERcuLD214EDBxgyZAjp6emuI4lIC3O5ElZ94RNrrc8Yc0gFrPHMfXo9AON7pjlOIhJ+8vLyyM7OprS0lBkzZjBixAiMMa5jiUgLc7kSNsQYUwJc+MkTV+u2tda2Cnm6MLbjeAkA/z1/qNsgImHo4MGDWGt56KGH6NSpk+s4ItJCXbKEWWs9jRlEvvbwi5sAuHNYJ/32LdJIvF4vhYWFpKenM27cOEaMGEFcXJzrWCLSgl3u7MhYY8xPgmdHPmKM0ZpijeT9PacA+JsZ/RwnEQkPRUVFvPDCC7z00ktUVlZijFEBE5GQu1yx+j2BeWGfANOBAcCPGyNUODtXEZiKlxDtITUxxnEakZYvJyeHpUuX4vf7ufPOO4mJ0f87EWkclyth/WudHfk88HnjRApv+/ID5z48OK6b2yAiLZy1lo8//pgPP/yQ9PR05s2bR5s2bVzHEpEwUt+zI72am9Q4Xt96HIAbuuusSJFQKygoYPDgwdx+++1ERUW5jiMiYeZyJWxo8GxICJwRqbMjG8FnBwoBGNG1teMkIi3TyZMniY6Opk2bNtx5551EREToBBgRceJyK+Zvs9a2Cn4kWWsja32uAhYCBecqOXT6PABx0To5VaShffnllzz//POsXr0aAI/HowImIs5cbiTMNloKoaLax8h/fg+AmUM6Ok4j0rJ4vV5Wr17N1q1b6datG7Nnz3YdSUTksiWsnTHmp5faaK39fyHIE7ZmP7EOCFwn8tffGeY4jUjLUVpayquvvkpeXh7jxo1j0qRJuvi2iDQJlythHiCRr1fMlxDaczJwVuSWv5niOIlIyxITE0NUVBTz5s2jXz+tvSciTcflStgJa+0/NlqSMHZhMv64nqmanyLSAKy1bNq0iSFDhhATE8PChQv1f0tEmpzLlTD9xGokf710OwCPTuzpOIlI81dRUcEbb7zBvn37sNYyevRoFTARaZIuV8ImN1qKMLZiWx6HC8sAGN091XEakeYtPz+fRYsWUVxczNSpUxk1apTrSCIil3S5C3ifacwg4eqJtTkAvPOTm/BE6Ld1kWu1f/9+srOziYuLY+HChWRkZLiOJCJyWboot2N7g5cp6tM+yXESkeYtPT2dXr16MX36dBITE13HERG5Ip2n7dCfvrwFgCytji9yTUpKSnjvvfew1tKqVSvmzZunAiYizYZGwhyx1rJ6x0kAfnv/CMdpRJqfQ4cOsWTJErxeL4MHD6Zdu3auI4mIXBWVMEcuLEsxtkcqbZNiHKcRaT6staxfv57333+f1NRU5s+fT1qaLngvIs2PSpgjb3xxHID5I7s4TiLSvKxatYrNmzfTv39/Zs6cSUyMfokRkeZJJcyR5V/mATB1YHvHSUSal8GDB9OmTRvGjBmj9b9EpFlTCXOkyufHGIiJ9LiOItLkffXVVxQWFjJx4kS6dOlCly4aQRaR5k9nRzqQeyawOGv/Dq0cJxFp2nw+H6tXr2bp0qUcOnQIn8/nOpKISIPRSJgD9zyzAYCFY7u5DSLShJ07d47FixeTm5vLmDFjmDJlCh6PRo5FpOVQCWtkx4vKOV5UDsC8LB1SEamL1+vl+eefp6ysjLvvvpuBAwe6jiQi0uBUwhrZf72zF4AFY7o6TiLS9FhrMcYQGRnJrbfeSlpamtb/EpEWS3PCGtmx4CjYL2frN3uR2iorK1myZAnbt28HoH///ipgItKiaSSskX1+SNdFF7lYQUEB2dnZFBYW6sLbIhI2VMIaUbXPD0CnlDjHSUSajp07d7JixQqioqJ44IEH6Natm+tIIiKNQiWsEa3dcwqAKf10iEUE4MSJEyxZsoTOnTszd+5cWrXSsi0iEj5UwhrR9/+wBYB7Rulwi4Q3n8+Hx+OhQ4cOzJ07lz59+mj5CREJO5qY30h+mv0lAG0SoumnRVoljB09epTf/OY35OUFLt3Vv39/FTARCUsaCWsEb20/wdKtgQt2P3HvcMdpRNyw1rJx40bWrFlDSkoKkZH68SMi4U0/BRvBsi8DBezNx8YzqHOy4zQija+qqooVK1awc+dO+vTpw+zZs4mNjXUdS0TEKZWwRrBmVz6ACpiErc2bN7Nr1y4mT57MuHHjMMa4jiQi4pxKWCOIjoygV7tE1zFEGl1FRQWxsbGMGTOGrl270qlTJ9eRRESaDE3MD7HC0kqqvH46JOvQi4QPv9/PmjVrePLJJyktLSUiIkIFTETkIhoJC7ER//Re4M+ubRwnEWkcpaWlvP766xw+fJisrCzN/RIRuQSVsBCq9PpqPv/TiT0cJhFpHLm5uSxevJjy8nJmzZrF0KFDXUcSEWmyQno40hgz1Riz1xiTY4z5+WX2G2mM8Rlj5oQyT2P7eN9pAL4/obvjJCKNY/369URGRvLwww+rgImIXEHIRsKMMR7gCeAW4BiwyRizwlq7q479/g14J1RZXHll4xEAbumX7jiJSOhUVVVRVVVFYmIiM2fOBCAuTtdHFRG5klCOhI0Ccqy1B621VcBrwKw69vsR8DpwKoRZnPhgbwEAWd00H0xapsLCQp5//nmys7Ox1hIXF6cCJiJST6GcE9YJyK11+xgwuvYOxphOwJ3AJGBkCLM0Omut6wgiIbVnzx6WLVtGREQEt956q9b+EhG5SqEsYXX9RL64mfwP8DNrre9yP8CNMY8AjwBkZDSPi18fO1sOwMwhHR0nEWlYfr+fDz74gE8//ZSOHTsyd+5cUlJSXMcSEWl2QlnCjgFdat3uDORdtE8W8FqwgKUB040xXmvtsto7WWufAZ4ByMrKahZDTP7gSNjEPm0dJxFpWNXV1ezevZvhw4czbdo0XQNSROQahfKn5yaglzEmEzgO3APcW3sHa23mhc+NMS8CKy8uYCLSNJw4cYK0tDRiYmL43ve+p/W/RESuU8gm5ltrvcBjBM563A1kW2t3GmN+YIz5Qaiet6nYn1/qOoJIg7DWsnnzZp577jk+/PBDABUwEZEGENLjCNbaVcCqi+576hL7PhjKLI3tbFkVAL3TkxwnEbl21dXVvPXWW2zbto2ePXsyfvx415FERFoMTeYIkb9csh2AtMQYx0lErs3Zs2dZtGgR+fn5TJgwgQkTJugMSBGRBqQSFmLtdeFuaab8fj+VlZXce++99OrVy3UcEZEWJ6SXLQpX7+w8CUC/Dq0cJxG5On6/nx07dmCtJTU1lccee0wFTEQkRDQSFgL/+Gbgyky/umeo2yAiV6GsrIylS5dy4MABYmNj6dmzJx6Px3UsEZEWSyUsBI4XBRZq1aR8aS7y8vLIzs6mtLSU22+/nR49eriOJCLS4qmENbBKrw+AIZ2THScRqZ9t27bx5ptvkpiYyEMPPUSnTp1cRxIRCQsqYQ1s06GzANw6oL3jJCL1k5iYSGZmJnfeeSfx8fGu44iIhA2VsAb28O83ATA8o7XjJCKXVlRUxOHDhxk6dCg9evSge/fuWn5CRKSRqYQ1oHd2nqTS6ycxJpIbeqS6jiNSp5ycHJYuXQpAnz59iIuLUwETEXFAJayBlFf5+P4ftgDw73MGO04j8m3WWj7++GM+/PBD0tPTmTdvHnFxca5jiYiELZWwBuL1+wH4yZReTB/UwXEakW+y1rJo0SL27t3L4MGDuf3224mKinIdS0QkrKmENZCTxRUA+PzWcRKRbzPGkJmZSY8ePcjKytLhRxGRJkAlrIHkni0DtDaYNC1ffPEFCQkJ9O7dm9GjR7uOIyIitaiENZD/XrMfgNSEaMdJRMDr9bJ69Wq2bt1Kv3796N27t+tIIiJyEZWwBpKRGs9Xx4t1VqQ4V1xcTHZ2Nnl5eYwbN45Jkya5jiQiInVQCWsgb20/QUabeM21EadKSkp4+umn8fv9zJ8/n759+7qOJCIil6AS1kDSEqM5eqbMdQwJc0lJSYwaNYpBgwaRmqpRWRGRpizCdYCWoKSimtOlVcwd0dl1FAlDFRUVvP766xQUFGCMYeLEiSpgIiLNgEbCGsCR04ERsG5pCY6TSLg5efIk2dnZFBcX06tXL9q2bes6koiI1JNKWAPYkVcMQLukGMdJJJxs27aNlStXEhcXx4MPPkiXLl1cRxIRkaugEtYA9pwoAWB4V120WxrHjh07WLZsGV27dmXOnDkkJia6jiQiIldJJawBnCwJrJbfubWuwyehZa3FGEPfvn2ZOnUqI0eOJCJCUztFRJoj/fRuAO/szAcgJtLjOIm0ZIcOHeKFF16gvLycyMhIRo8erQImItKMaSTsOm08WAjAgI6tHCeRlspay7p161i7di2pqalUVFQQF6dRVxGR5k4l7Dqt3nESgL+9vb/jJNISVVRUsHz5cvbs2cOAAQOYOXMm0dG6NJaISEugEnadjhSeB2CEJuVLCLz99tvs27eP2267jdGjR+uKDCIiLYhK2HX6YG8BAFEezc2RhuPz+fB4PEyZMoXhw4eTkZHhOpKIiDQwlbDrUFHtcx1BWhifz8e7775LQUEB999/P4mJiVp+QkSkhVIJuw4bD50BYNrA9o6TSEtQUlLCkiVLyM3NZfTo0VhrXUcSEZEQUgm7Di+uOwTAvaN1qEiuz+HDh1myZAlVVVXcfffdDBw40HUkEREJMZWw63DodGBS/o29dL0+uXY+n48VK1YQGxvLwoULdf1HEZEwoRJ2HQ4XluGJ0Nlqcm0qKyuJjIzE4/Fw7733kpSUREyMrj8qIhIudErfNSoqqwKgR9sEx0mkOSooKODZZ59lzZo1AKSlpamAiYiEGY2EXaPDhWUAfGeU5oPJ1dm5cyfLly8nOjqavn37uo4jIiKOqIRdo4/3BdYH65Siy8dI/fh8Pt577z02bNhAly5dmDNnDq1a6XJXIiLhSiXsGsVFBS7WPbhzitsg0mycPXuWLVu2MGrUKG699VY8Hl3wXUQknKmEXaPszbkAJMbqr1Aur7CwkNTUVNLS0nj00UdJTk52HUlERJoATcy/Bj6/Zf+pUgASojWaIXWz1rJhwwaefPJJvvrqKwAVMBERqaFhnGtQWFoJwNwRnXVBZalTVVUVK1asYOfOnfTt25devXq5jiQiIk2MStg1eGfnSQA6tdakfPm206dPs2jRIgoLC5k8eTLjxo1TWRcRkW9RCbsGi4LzwWYO6eg4iTRFBQUFlJWVsWDBAjIzM13HERGRJkol7Cr5/JYdx0sAyEzTQq0S4Pf7OXbsGBkZGfTr14/u3btr8VUREbksTcy/SidLKgAY1a2NDjEJAKWlpfzhD3/g97//PWfPngVQARMRkSvSSNhV2nIk8CY7a5gORQrk5uayePFiysvLmTlzJq1bt3YdSUREmgmVsKv0xdFACRvRVW+24e7zzz/nnXfeITk5mYcffpj27du7jiQiIs2ISthV+uOGowB0S9V8sHBXVlZGz549ufPOO4mNjXUdR0REmhmVsKtQ5fVT5fMDEBulRVrDUWFhIWVlZXTp0oUJEyYAaG6giIhcE03Mvwrl1T4Avjdeyw6Eoz179vDss8/y5ptvYq3FGKMCJiIi10wjYVfh+NlyANJb6dBTOPH7/axdu5Z169bRsWNH5s6dq/IlIiLXTSXsKuw6EVgfLCZKA4jhoqqqitdee41Dhw4xfPhwpk2bRmSk/tuIiMj107vJVdh+rAiAG7qnug0ijSYqKorExERmzpzJsGHDXMcREZEWRCXsKuw4XgxAlzbxjpNIKFlr2bp1K927d6d169bcddddriOJiEgLpONqV2HbsWJioyJ0ZmQLVl1dzfLly1m5ciWbNm1yHUdERFowjYRdhdbx0bSK1V9ZS3XmzBmys7PJz89n4sSJ3HTTTa4jiYhIC6ZGcRU8ETAqs43rGBICeXl5vPTSSxhjuO++++jZs6frSCIi0sKphF2F/JJK1xEkRNLS0ujTpw8TJ07U9R9FRKRRaE5YPZ0qqQDg4OnzjpNIQykrK+Ott96iqqqK6Oho7rzzThUwERFpNBoJq6cLq+XPGd7ZcRJpCHl5eWRnZ1NaWkq/fv3o3r2760giIhJmVMLq6VhwtfxKr89xErkeF5afWL16NYmJiXz3u9+lY8eOrmOJiEgYUgmrp1PnAocje7RLdJxErsenn37K2rVr6dGjB3fddRfx8VrzTURE3FAJqyefP/BnYoz+ypqzQYMGYa1l/PjxRERoSqSIiLijd6F6+jL3LAApcdGOk8jV2r9/P8uXL8daS0pKCjfddJMKmIiIOKdhnXqq9loA0pNjHCeR+rLW8tFHH/HRRx+Rnp5OeXm5Dj+KiEiToRJWT8ZAeqsYYiJ1yaLmoLy8nKVLl5KTk8OQIUOYMWMGUVFRrmOJiIjUUAmTFsdayyuvvEJeXh4zZsxgxIgRGGNcxxIREfkGlTBpUay1GGOYMmUKHo+Hzp21rpuIiDRNKmHSIni93pq1v26++Wa6du3qOpKIiMhl6RQxafaKiop44YUX2Lp1K9Za13FERETqRSNh9XS6tLJmrTBpOg4cOMDrr7+O3+9n/vz59O3b13UkERGRelEJq6ftx4o5c77SdQyppbS0lNdee402bdowb948UlNTXUcSERGpt5CWMGPMVOBXgAd4zlr7+EXb7wN+FrxZCvyptXZbKDNdq8LzVSTFaomDpsDr9RIZGUliYiL33HMPXbp0ITpai+iKiEjzErI5YcYYD/AEMA3oD3zHGNP/ot0OAROstYOBXwLPhCrP9fL5LT3aJriOEfZOnjzJk08+ya5duwDo0aOHCpiIiDRLoRwJGwXkWGsPAhhjXgNmAbsu7GCtXV9r/w1Ak1xPwO8PTPYe3DnFbZAwt23bNlauXElcXBxJSUmu44iIiFyXUJawTkBurdvHgNGX2f9hYHUI81yz40XlAHj9mpnvgtfr5Z133mHz5s1069aNu+++m8TERNexRERErksoS1hdS5TXuX6AMeZmAiVs/CW2PwI8ApCRkdFQ+ertq+PFAPRoqzd+F3Jycti8eTNjx45l8uTJuvi2iIi0CKEsYceALrVudwbyLt7JGDMYeA6YZq0trOuBrLXPEJwvlpWV1egLQS374jgAAzslN/ZTh7WysjLi4+Pp27cvjzzyCB06dHAdSUREpMGEckhhE9DLGJNpjIkG7gFW1N7BGJMBLAUWWGv3hTDLddl1ogSAYV1S3AYJE9ZaPv30U371q1+Rn58PoAImIiItTshGwqy1XmPMY8A7BJaoeMFau9MY84Pg9qeAvwNSgSeDF1j2WmuzQpXpWhWcq2RI52QiPToMFmoVFRUsW7aMvXv3MnDgQFq3bu06koiISEiEdJ0wa+0qYNVF9z1V6/PvAd8LZYaGEB0ZQbVPl8MJtfz8fLKzsykqKmLq1KmMGjWKYDkXERFpcbRifj3EREYwLCPFdYwWb/v27VRVVbFw4UInJ2CIiIg0JpUwccrn81FcXEybNm2YNGkSN9xwg5afEBGRsKBJTvWgQ5GhUVJSwosvvsjvf/97qqur8Xg8KmAiIhI2NBJ2Becqqikur6a8yuc6Soty+PBhlixZQlVVFbNmzSIqStflFBGR8KISdgUF5yoBaBWnktAQrLWsX7+e999/n9TUVBYuXEjbtm1dxxIREWl0KmH1pIn5DcNay8GDB+nXrx8zZ84kJibGdSQREREnVMKuoKi82nWEFqGgoIDY2FiSkpKYP38+UVFRWn5CRETCmibmX0FphRdAheE67Nixg2effZbVqwPXZ4+Ojtbfp4iIhD2NhF3Bha7QMTnWbZBmyOfzsWbNGjZu3EiXLl2YNm2a60giIiJNhkrYFRw9UwaAFqm4OqWlpSxevJijR48yatQobr31Vjwej+tYIiIiTYZK2BVERgSGwlrHRztO0rxERkZSWVnJXXfdxaBBg1zHERERaXJUwq7gYMF5AOKjNYpzJdZatm/fzoABA4iNjeWRRx4hIkLTDkVEROqiEnYFMVGB8tUuSUspXE5lZSVvvvkmO3fupLq6mqysLBUwERGRy1AJu4LfrTsEQKRHheJSCgoKyM7OprCwkClTpjBixAjXkURERJo8lbArSIyJ5FxwmQr5tv3797NkyRIiIyNZsGABmZmZriOJiIg0CyphVxDlieDOYZ1cx2iyUlJS6Ny5M7NmzaJVq1au44iIiDQbOsZ2BReWqJCvlZaWsm7dOqy1tG3blgULFqiAiYiIXCWNhF1GzqlzABSVVTlO0nTk5uayePFiysvL6du3L6mpqa4jiYiINEsqYZexLqcQgPkjMxwncc9ay+eff867775LcnIy3/ve91TAREREroNK2GV8sv80AMMzUtwGaQJWrVrF5s2b6d27N3feeSexsbqMk4iIyPVQCbuMTimBotGulQpHr169aNWqFePHj9fFt0VERBqAStgVpMRHuY7gzJ49eygpKWHUqFH07t2b3r17u44kIiLSYujsSPkWv9/Pe++9x6JFi/jqq6/w+/2uI4mIiLQ4GgmTbzh//jyvv/46hw4dYsSIEUydOlWXHxIREQkBlTCpUV1dzbPPPsv58+eZNWsWQ4cOdR1JRESkxVIJkxpRUVHceOONdOzYkQ4dOriOIyIi0qLpONNlnK/y4fNZ1zFCqrq6mmXLlpGTkwPAiBEjVMBEREQagUrYZRwoKKW0quVevPvMmTM8//zzbNu2jYKCAtdxREREwooOR15G7pky0hJjXMcIib179/LGG28QERHBfffdR8+ePV1HEhERCSsqYZdxurSKDsktb6HW3NxcXnvtNTp06MC8efNISUlxHUlERCTsqIRdQWZagusIDcZaizGGzp07c8cddzB48GAiI/VPQERExAXNCbuEUyUVALRLahmHI48fP85TTz1FYWEhxhiGDx+uAiYiIuKQ3oUvoai8GoABHZMdJ7k+1lq2bNnC22+/TWJiItXV1a4jiYiICCphV9QxJc51hGtWXV3NW2+9xbZt2+jRowd33XUX8fHxrmOJiIgIKmEt2rp169i2bRs33XQTEyZM0OWHREREmhCVsEs4dPo8AJVen+MkV6+6upqoqCjGjRtH165dyczMdB1JRERELqKhkUu4UMK6pjafsyP9fj8ffPABzzzzDBUVFURFRamAiYiINFEaCbuEffnnAOia2jzmUJWVlfHGG2+Qk5PDkCFD8Hg8riOJiIjIZaiEXULBuUoAUhOiHSe5sry8PLKzsyktLWXGjBmMGDECY4zrWCIiInIZKmGXcLK4gsSYyCZfZqy1rFmzBmstDz30EJ06dXIdSUREROpBJewSkuOiOHa23HWMS/J6vXi9XmJjY7nrrrvweDxafkJERKQZUQm7BGNgWEaK6xh1KioqIjs7m4SEBO69916SkpJcRxIREZGrpLMjL+Gr48WuI9QpJyeHZ555hjNnzmjul4iISDOmkbBLSImLprC0ynWMGtZaPv74Yz788EPatWvH/PnzadOmjetYIiIico1Uwi7hZEkF43uluY5Ro7y8nC1btjB48GBuv/12oqKiXEcSERGR66ASdhlnzrsfCSsoKCA1NZX4+HgeeeQREhISdAhSRESkBdCcsMsY1CnZ6fN/+eWXPPPMM3zyyScAJCYmqoCJiIi0EBoJa4K8Xi9vv/02W7ZsoVu3bmRlZbmOJCIiIg1MJawOx86WAeDz20Z/7uLiYhYvXszx48cZN24ckyZNIiJCA5YiIiItjUpYHdbnFAKQ0abxFz8tKyujqKiIefPm0a9fv0Z/fhEREWkcKmF12HAwUMIm9GnbKM9nreXAgQP07NmTDh068OMf/1hnP4qIiLRwOs5Vh5goDwDtkmJC/lwVFRUsWrSIP/7xjxw5cgRABUxERCQMaCSsDsZAWmJMyM9EzM/PJzs7m6KiIqZOnUpGRkZIn09ERESaDpWwOlgbOEQYSjt27GD58uXExsaycOFCFTAREZEwoxJWh61HzlLl9Yf0Oay1dOrUiTlz5pCYmBjS5xIREZGmRyWsDh1SYjlZUtHgj1tSUkJeXh59+/Zl0KBBDBw4UIuvioiIhCmVsEvoltqwy1McOnSIJUuWYK0lMzOTmJjQzzkTERGRpkslrA5+Cw01I8xay/r163n//fdJTU1l3rx5xMSE/qxLERERadpUwuqw5fAZujTAQq3WWhYvXszu3bvp378/M2fOVAETERERQCWsTh1S4hrkUKExhrS0NG699VbGjBmjw48iIiJSQyWsDjmnSrltQPo1f/1XX31FcnIyGRkZTJo0qQGTiYiISEuhElaH6MiIa1qiwufzsWbNGjZu3Ej//v219peIiIhckkpYHaIiDD3aXt3aXefOnWPx4sXk5uYyevRobrnllhClExERkZZAJawBnD17lueff56qqiruvvtuBg4c6DqSiIiINHEqYQ0gJSWF/v37k5WVRbt27VzHERERkWYgwnWApsZay/kq3xX3q6ys5M0336S4uBhjDNOnT1cBExERkXpTCbtISbkXgHMV3kvuU1BQwHPPPccXX3zBkSNHGiuaiIiItCA6HHkJfdon1Xn/rl27WL58OZGRkSxYsIDMzMxGTiYiIiItgUrYVdi+fTtvvPEGnTt3Zu7cubRq1cp1JBEREWmmVMKuQp8+fZg4cSLjx4/H4/G4jiMiIiLNWEjnhBljphpj9hpjcowxP69juzHG/Dq4fbsxZngo89THqXMVAJyvDMwJO3r0KH/84x+prq4mJiaGCRMmqICJiIjIdQtZCTPGeIAngGlAf+A7xpj+F+02DegV/HgE+G2o8tRXlS+wUn7X1Hg2bNjA73//e86cOUNpaanjZCIiItKShPJw5Cggx1p7EMAY8xowC9hVa59ZwEvWWgtsMMakGGM6WGtPhDDXFUXi4+jWD/gi9wB9+vRh9uzZxMbGuowkIiIiLUwoS1gnILfW7WPA6Hrs0wlwWsLGRR2hMPcskyZNYvz48RhjXMYRERGRFiiUJayu5mKvYR+MMY8QOFwZ8otix0V5KG/bl6wRbblx3NCQPpeIiIiEr1CWsGNAl1q3OwN517AP1tpngGcAsrKyvlXSGlL3tolk/5kuvi0iIiKhFcqzIzcBvYwxmcaYaOAeYMVF+6wAHgieJTkGKHY9H0xERESkMYRsJMxa6zXGPAa8A3iAF6y1O40xPwhufwpYBUwHcoAy4KFQ5RERERFpSkK6WKu1dhWBolX7vqdqfW6BR0OZQURERKQp0gW8RURERBxQCRMRERFxQCVMRERExAGVMBEREREHVMJEREREHFAJExEREXFAJUxERETEAZUwEREREQdUwkREREQcUAkTERERcUAlTERERMQBlTARERERB1TCRERERBxQCRMRERFxQCVMRERExAFjrXWd4aoYYwqAI43wVGnA6UZ4Hqk/vSZNj16TpkmvS9Oj16RpaozXpau1tm1dG5pdCWssxpjN1tos1znka3pNmh69Jk2TXpemR69J0+T6ddHhSBEREREHVMJEREREHFAJu7RnXAeQb9Fr0vToNWma9Lo0PXpNmianr4vmhImIiIg4oJEwEREREQfCuoQZY6YaY/YaY3KMMT+vY7sxxvw6uH27MWa4i5zhph6vy33B12O7MWa9MWaIi5zh5EqvSa39RhpjfMaYOY2ZL1zV53Uxxkw0xnxpjNlpjPmosTOGm3r8/Eo2xrxpjNkWfE0ecpEznBhjXjDGnDLG7LjEdmfv9WFbwowxHuAJYBrQH/iOMab/RbtNA3oFPx4BftuoIcNQPV+XQ8AEa+1g4JdorkVI1fM1ubDfvwHvNG7C8FSf18UYkwI8Ccy01g4A5jZ2znBSz/8rjwK7rLVDgInAfxljohs1aPh5EZh6me3O3uvDtoQBo4Aca+1Ba20V8Bow66J9ZgEv2YANQIoxpkNjBw0zV3xdrLXrrbVngzc3AJ0bOWO4qc//FYAfAa8DpxozXBirz+tyL7DUWnsUwFqr1ya06vOaWCDJGGOAROAM4G3cmOHFWvsxgb/nS3H2Xh/OJawTkFvr9rHgfVe7jzSsq/07fxhYHdJEcsXXxBjTCbgTeKoRc4W7+vxf6Q20NsZ8aIzZYox5oNHShaf6vCa/AfoBecBXwI+ttf7GiSeX4Oy9PrIxnqSJMnXcd/GpovXZRxpWvf/OjTE3Eyhh40OaSOrzmvwP8DNrrS/wC740gvq8LpHACGAyEAd8ZozZYK3dF+pwYao+r8ltwJfAJKAHsMYY84m1tiTE2eTSnL3Xh3MJOwZ0qXW7M4HfTK52H2lY9fo7N8YMBp4DpllrCxspW7iqz2uSBbwWLGBpwHRjjNdau6xREoan+v4MO22tPQ+cN8Z8DAwBVMJCoz6vyUPA4zawPlSOMeYQ0Bf4vHEiSh2cvdeH8+HITUAvY0xmcFLkPcCKi/ZZATwQPHNiDFBsrT3R2EHDzBVfF2NMBrAUWKDf6BvFFV8Ta22mtbabtbYbsAT4oQpYyNXnZ9hy4EZjTKQxJh4YDexu5JzhpD6vyVECI5MYY9KBPsDBRk0pF3P2Xh+2I2HWWq8x5jECZ3J5gBestTuNMT8Ibn8KWAVMB3KAMgK/wUgI1fN1+TsgFXgyOPLi1YVxQ6eer4k0svq8Ltba3caYt4HtgB94zlpb52n6cv3q+X/ll8CLxpivCBwG+5m19rSz0GHAGPMqgTNR04wxx4C/B6LA/Xu9VswXERERcSCcD0eKiIiIOKMSJiIiIuKASpiIiIiIAyphIiIiIg6ohImIiIg4oBImImHBGOMzxnxZ66ObMWaiMabYGPOFMWa3Mebvg/vWvn+PMeY/XecXkZYnbNcJE5GwU26tHVr7DmNMN+ATa+3txpgE4EtjzMrg5gv3xwFfGGPesNaua9zIItKSaSRMRAQIXtpnC4Hr+dW+v5zAtf4a5YK+IhI+VMJEJFzE1ToU+cbFG40xqcAYYOdF97cGegEfN05MEQkXOhwpIuHiW4cjg240xnxB4LI+jwcvMzMxeP92Atf2e9xae7LRkopIWFAJE5Fw94m19vZL3W+M6Q18GpwT9mUjZxORFkyHI0VELsNauw/4V+BnrrOISMuiEiYicmVPATcZYzJdBxGRlsNYa11nEBEREQk7GgkTERERcUAlTERERMQBlTARERERB1TCRERERBxQCRMRERFxQCVMRERExAGVMBEREREHVMJEREREHPj/AeEx61aSGTI+AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots(1, 1, figsize=(10, 8))\n",
    "plot_roc(\n",
    "    y_test['seasonal_vaccine'], \n",
    "    predictions_se, \n",
    "    'seasonal_vaccine',\n",
    "    ax=ax\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.8610242609729017"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "roc_auc_score(y_test.seasonal_vaccine, predictions_se)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Combined score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.8660603670105245"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Let's see the score combined of the two best predictions\n",
    "roc_auc_score(y_test, np.hstack((predictions_h1, predictions_se)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Retrain on full dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Seasonal"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [],
   "source": [
    "final_model_se.fit(train, labels.seasonal_vaccine)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "final_se = final_model_se.predict_proba(test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [],
   "source": [
    "final_se = final_se[:,1].reshape(-1,1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### H1N1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<catboost.core.CatBoostClassifier at 0x7f87e2007bd0>"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "final_model.fit(train, labels.h1n1_vaccine)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [],
   "source": [
    "final_h1 = final_model.predict_proba(test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [],
   "source": [
    "final_h1 = final_h1[:,1].reshape(-1,1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Make submission"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [],
   "source": [
    "submission_df = pd.read_csv(\"./submission_format.csv\", \n",
    "                            index_col=\"respondent_id\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>h1n1_vaccine</th>\n",
       "      <th>seasonal_vaccine</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>respondent_id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>26707</th>\n",
       "      <td>0.370448</td>\n",
       "      <td>0.273193</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26708</th>\n",
       "      <td>0.079469</td>\n",
       "      <td>0.044629</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26709</th>\n",
       "      <td>0.504887</td>\n",
       "      <td>0.735957</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26710</th>\n",
       "      <td>0.879725</td>\n",
       "      <td>0.891950</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26711</th>\n",
       "      <td>0.687898</td>\n",
       "      <td>0.509354</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               h1n1_vaccine  seasonal_vaccine\n",
       "respondent_id                                \n",
       "26707              0.370448          0.273193\n",
       "26708              0.079469          0.044629\n",
       "26709              0.504887          0.735957\n",
       "26710              0.879725          0.891950\n",
       "26711              0.687898          0.509354"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Make sure we have the rows in the same order\n",
    "np.testing.assert_array_equal(test.index.values, \n",
    "                              submission_df.index.values)\n",
    "\n",
    "# Save predictions to submission data frame\n",
    "submission_df[\"h1n1_vaccine\"] = final_h1\n",
    "submission_df[\"seasonal_vaccine\"] = final_se\n",
    "\n",
    "submission_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [],
   "source": [
    "date = pd.Timestamp.now().strftime(format='%Y-%m-%d_%H-%M_')\n",
    "submission_df.to_csv(f'predictions/{date}submssion_catboost_optunacvi.csv', index=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The model scored `0.8608` not too bad, much better than the benchmark. There is a lot of room for improvment (my present score is `0.8638`, ranking **7th** on the 16th February 2021). We can verify some more parameters, CatBoost has a lot of [them](https://catboost.ai/docs/concepts/parameter-tuning.html), we may try some feature engineering (I tried some basice ones but the score did not improve so I omitted them in the analysis), change the model (LightGBM, XGBoost for example), try different imputations, increase the rounds for `optuna` a little bit and so on. Spend some time analysing the dataset and the correlation among the variables using EDA notebooks."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "© Andrea Dalseno 2021"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "joseml",
   "language": "python",
   "name": "joseml"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}