Optuna.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Optuna: A hyperparameter optimization framework\n",
    "\n",
    "Optuna es un software de optimización automática de hiperparámetros, diseñado particularmente para machine learning. Gracias a su API, el código escrito en Optuna posee alta modularidad, y el usuario de Optuna puede construir dinámicamente los espacios de búsqueda para los hiperparámetros.\n",
    "\n",
    "## ¿Qué son los hiperparámetros?\n",
    "\n",
    "Los *hiperparámetros* son parámetros que determinan el comportamiento de los algoritmos, determinan el desempeño y generalmente se establecen de forma manual. A diferencia de los parámetros del modelo, que son aprendidos durante el entrenamiento, los hiperparámetros deben ser establecidos antes del proceso de entrenamiento. Algunos ejemplos comunes de hiperparámetros incluyen el *learning rate* (*tasa de aprendizaje*), el número de capas y unidades en una red neuronal, o la profundidad máxima en un árbol de decisión.\n",
    "\n",
    "## Búsqueda de hiperparámetros\n",
    "\n",
    "El problema de la búsqueda de hiperparámetros es el proceso de seleccionar un conjunto de hiperparámetros óptimos para un algoritmo de aprendizaje. Dado que el desempeño del modelo puede depender significativamente de los hiperparámetros seleccionados, siendo este un paso importante en el proceso de modelado.\n",
    "\n",
    "La búsqueda de hiperparámetros es desafiante por varias razones, entre ellas:\n",
    "\n",
    " 1. Espacio de búsqueda grande: Usualmente, el espacio de búsqueda de hiperparámetros puede ser muy grande, especialmente para modelos complejos como las redes neuronales profundas.\n",
    "\n",
    " 2. Evaluaciones costosas: Evaluar una configuración de hiperparámetros dado puede ser muy costoso en términos de tiempo y recursos computacionales, especialmente si el modelo es complejo o si el conjunto de datos es muy grande.\n",
    "\n",
    " 3. Interacciones complejas: Los hiperparámetros pueden interactuar entre sí de formas complejas, lo que implica que el efecto de un hiperparámetro podría depender de otro.\n",
    "\n",
    " 4. Falta de gradientes: A diferencia de los parámetros del modelo, los hiperparámetros a menudo no tienen gradientes que puedan ser utilizados para guiar la búsqueda."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Instalación de Optuna"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "#pip install optuna # Instalamos Optuna\n",
    "#pip install optuna-dashboard # Instalación del Optuna dashboard (similar a Tensorboard)\n",
    "#pip install plotly # En caso de no tener instalado Plotly (para graficar)\n",
    "#pip install nbformat # Requerido por plotly\n",
    "#pip install lightgbm # Para utilizar LightGBM (Light Gradient-Boosting Machine)\n",
    "#pip install mlflow"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Ejemplo: Función cuadrática\n",
    "\n",
    "Generalmente, Optuna se utiliza para optimizar hiperparámetros, pero como ejemplo, vamos a optimizar una función cuadrática simple: $(x-2)^2$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\Users\\franz\\miniconda3\\envs\\tf-gpu\\lib\\site-packages\\tqdm\\auto.py:22: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
      "  from .autonotebook import tqdm as notebook_tqdm\n"
     ]
    }
   ],
   "source": [
    "import optuna"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "En Optuna, las funciones a ser optimizadas llevan el nombre `objective`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "import optuna\n",
    "\n",
    "def objective(trial):\n",
    "\n",
    "    x = trial.suggest_float(\"x\", -10, 10)\n",
    "\n",
    "    return (x - 2)**2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Esta función devuelve el valor de $(x-2)^2$. Nuestro objetivo es el encontrar el valor de `x` que minimíce la salida de la función `objective`. Durante la optimización, Optuna repetidamente llama y evalua la función objetivo con distintos valores de `x`.\n",
    "\n",
    "Un objeto `trial` corresponde a una sola ejecución de la función objetivo y es internamente inicializado sobre cada invocación de la función.\n",
    "\n",
    "Los `suggest` APIs (por ejemplo, `suggest_float()`) son llamados dentro de la función objetivo para obtener parámetros para un intento. `suggest_float()` selecciona los parámetros uniformemente dentro del rango establecido.\n",
    "\n",
    "Para comenzar la optimización, creamos un objeto `study` y pasamos la función objetivo al método `optimize()` como sigue."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:06,666] A new study created in memory with name: no-name-eab50edd-f37c-403b-bfc5-48f0578f2dba\n",
      "[I 2023-09-21 11:36:06,666] Trial 0 finished with value: 10.439138179196712 and parameters: {'x': -1.230965518107043}. Best is trial 0 with value: 10.439138179196712.\n",
      "[I 2023-09-21 11:36:06,666] Trial 1 finished with value: 67.67353109134189 and parameters: {'x': -6.22639234970846}. Best is trial 0 with value: 10.439138179196712.\n",
      "[I 2023-09-21 11:36:06,674] Trial 2 finished with value: 6.198398748162254 and parameters: {'x': -0.48965835972774663}. Best is trial 2 with value: 6.198398748162254.\n",
      "[I 2023-09-21 11:36:06,674] Trial 3 finished with value: 103.8159255741395 and parameters: {'x': -8.189010038965488}. Best is trial 2 with value: 6.198398748162254.\n",
      "[I 2023-09-21 11:36:06,682] Trial 4 finished with value: 99.56679843591786 and parameters: {'x': -7.978316412898414}. Best is trial 2 with value: 6.198398748162254.\n",
      "[I 2023-09-21 11:36:06,682] Trial 5 finished with value: 14.973549558184745 and parameters: {'x': 5.869567102168503}. Best is trial 2 with value: 6.198398748162254.\n",
      "[I 2023-09-21 11:36:06,682] Trial 6 finished with value: 23.13482872775385 and parameters: {'x': -2.809867849302499}. Best is trial 2 with value: 6.198398748162254.\n",
      "[I 2023-09-21 11:36:06,682] Trial 7 finished with value: 0.000882572326693118 and parameters: {'x': 2.0297081188683013}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:06,690] Trial 8 finished with value: 19.17431168596135 and parameters: {'x': -2.378848214537854}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:06,690] Trial 9 finished with value: 38.92290352186729 and parameters: {'x': 8.238822286446961}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:06,699] Trial 10 finished with value: 0.7584233106852957 and parameters: {'x': 2.870875025870702}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:06,706] Trial 11 finished with value: 2.1759940772394843 and parameters: {'x': 3.4751251056230736}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:06,717] Trial 12 finished with value: 2.6296807887221085 and parameters: {'x': 3.6216290539830953}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:06,725] Trial 13 finished with value: 0.4712933778097054 and parameters: {'x': 2.6865081046933863}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:06,733] Trial 14 finished with value: 63.46381346024229 and parameters: {'x': 9.96641785624143}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:06,741] Trial 15 finished with value: 0.4108942644824816 and parameters: {'x': 1.3589896533733004}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:06,749] Trial 16 finished with value: 1.1167522742463818 and parameters: {'x': 0.9432349957316046}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:06,759] Trial 17 finished with value: 18.364242800185945 and parameters: {'x': 6.28535212090978}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:06,767] Trial 18 finished with value: 38.810625218122055 and parameters: {'x': -4.229817430561032}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:06,775] Trial 19 finished with value: 5.074868957983369 and parameters: {'x': -0.2527469804626014}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:06,783] Trial 20 finished with value: 2.1617151628135534 and parameters: {'x': 0.5297227598804526}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:06,791] Trial 21 finished with value: 0.0063975268397884946 and parameters: {'x': 2.079984541255098}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:06,791] Trial 22 finished with value: 0.16609597411028634 and parameters: {'x': 1.592451261675016}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:06,799] Trial 23 finished with value: 7.658442299110139 and parameters: {'x': 4.767389076207055}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:06,807] Trial 24 finished with value: 0.20549370569612097 and parameters: {'x': 1.5466858642220376}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:06,815] Trial 25 finished with value: 0.13358188633062618 and parameters: {'x': 2.3654885584127445}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:06,823] Trial 26 finished with value: 6.074168630113221 and parameters: {'x': 4.464582851135912}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:06,831] Trial 27 finished with value: 0.46752448891038945 and parameters: {'x': 2.68375762438922}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:06,839] Trial 28 finished with value: 12.760655704987576 and parameters: {'x': -1.5722059997972648}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:06,852] Trial 29 finished with value: 8.038513165732384 and parameters: {'x': -0.8352271806210493}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:06,856] Trial 30 finished with value: 1.6000822517235642 and parameters: {'x': 0.735056423501995}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:06,864] Trial 31 finished with value: 0.03041411058137389 and parameters: {'x': 1.8256035820856005}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:06,872] Trial 32 finished with value: 0.12019580160289318 and parameters: {'x': 2.346692661593656}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:06,880] Trial 33 finished with value: 4.975332327972062 and parameters: {'x': -0.2305452983456895}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:06,896] Trial 34 finished with value: 4.96793833509741 and parameters: {'x': 4.228887241449735}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:06,904] Trial 35 finished with value: 0.03871884870550896 and parameters: {'x': 2.1967710565746623}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:06,912] Trial 36 finished with value: 1.7039871327017388 and parameters: {'x': 0.6946314188315474}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:06,920] Trial 37 finished with value: 10.938932199640012 and parameters: {'x': 5.307405659975808}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:06,920] Trial 38 finished with value: 140.68912199056348 and parameters: {'x': -9.861244538013853}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:06,932] Trial 39 finished with value: 12.031223991002035 and parameters: {'x': -1.4686054821789742}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:06,936] Trial 40 finished with value: 4.132352171513821 and parameters: {'x': -0.03281877488226215}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:06,944] Trial 41 finished with value: 0.008153281425288367 and parameters: {'x': 2.090295522731132}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:06,954] Trial 42 finished with value: 0.014905459451316865 and parameters: {'x': 1.8779120831068166}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:06,960] Trial 43 finished with value: 2.3733623524875536 and parameters: {'x': 3.540572086105533}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:06,968] Trial 44 finished with value: 0.2614653344648994 and parameters: {'x': 1.4886631888227688}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:06,976] Trial 45 finished with value: 2.358205065027381 and parameters: {'x': 3.535644836877128}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:06,984] Trial 46 finished with value: 2.910325905365255 and parameters: {'x': 0.29403226719692754}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:06,992] Trial 47 finished with value: 1.0275557167884826 and parameters: {'x': 3.0136842293280894}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:07,001] Trial 48 finished with value: 0.003983811291556887 and parameters: {'x': 1.9368825595294226}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:07,008] Trial 49 finished with value: 20.94019573309133 and parameters: {'x': 6.576045862214596}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:07,016] Trial 50 finished with value: 4.464489768649394 and parameters: {'x': 4.112933924345339}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:07,024] Trial 51 finished with value: 0.13335083422814148 and parameters: {'x': 1.634827665029042}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:07,032] Trial 52 finished with value: 0.10851177202837063 and parameters: {'x': 2.3294112506098883}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:07,043] Trial 53 finished with value: 1.1309151347643565 and parameters: {'x': 3.0634449373448334}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:07,051] Trial 54 finished with value: 1.2607463242920554 and parameters: {'x': 0.8771703939189814}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:07,057] Trial 55 finished with value: 8.303924052178209 and parameters: {'x': -0.8816530069004158}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:07,065] Trial 56 finished with value: 0.22059796939360243 and parameters: {'x': 1.5303214190602232}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:07,074] Trial 57 finished with value: 3.6068011472581185 and parameters: {'x': 0.10084198991813254}. Best is trial 7 with value: 0.000882572326693118.\n",
      "[I 2023-09-21 11:36:07,086] Trial 58 finished with value: 0.00046402615813302087 and parameters: {'x': 2.0215412664004004}. Best is trial 58 with value: 0.00046402615813302087.\n",
      "[I 2023-09-21 11:36:07,092] Trial 59 finished with value: 1.2777576193233762 and parameters: {'x': 3.13037941387986}. Best is trial 58 with value: 0.00046402615813302087.\n",
      "[I 2023-09-21 11:36:07,099] Trial 60 finished with value: 2.898982832103563 and parameters: {'x': 3.7026399596225748}. Best is trial 58 with value: 0.00046402615813302087.\n",
      "[I 2023-09-21 11:36:07,113] Trial 61 finished with value: 0.0003638539715663391 and parameters: {'x': 2.019074956659619}. Best is trial 61 with value: 0.0003638539715663391.\n",
      "[I 2023-09-21 11:36:07,119] Trial 62 finished with value: 1.2900262503416315 and parameters: {'x': 0.8642067748301931}. Best is trial 61 with value: 0.0003638539715663391.\n",
      "[I 2023-09-21 11:36:07,127] Trial 63 finished with value: 0.037746154760492395 and parameters: {'x': 2.1942836965895296}. Best is trial 61 with value: 0.0003638539715663391.\n",
      "[I 2023-09-21 11:36:07,136] Trial 64 finished with value: 0.6103751785117684 and parameters: {'x': 2.7812651141013327}. Best is trial 61 with value: 0.0003638539715663391.\n",
      "[I 2023-09-21 11:36:07,144] Trial 65 finished with value: 0.4849080849252817 and parameters: {'x': 1.303646580445474}. Best is trial 61 with value: 0.0003638539715663391.\n",
      "[I 2023-09-21 11:36:07,152] Trial 66 finished with value: 0.005044977380088013 and parameters: {'x': 2.071028004196148}. Best is trial 61 with value: 0.0003638539715663391.\n",
      "[I 2023-09-21 11:36:07,164] Trial 67 finished with value: 2.5923129626949133 and parameters: {'x': 0.3899338638754888}. Best is trial 61 with value: 0.0003638539715663391.\n",
      "[I 2023-09-21 11:36:07,170] Trial 68 finished with value: 0.34993530044919735 and parameters: {'x': 2.5915532946820576}. Best is trial 61 with value: 0.0003638539715663391.\n",
      "[I 2023-09-21 11:36:07,176] Trial 69 finished with value: 0.848247567481609 and parameters: {'x': 1.0789964345988616}. Best is trial 61 with value: 0.0003638539715663391.\n",
      "[I 2023-09-21 11:36:07,201] Trial 70 finished with value: 4.4732669770662605 and parameters: {'x': 4.115009923633045}. Best is trial 61 with value: 0.0003638539715663391.\n",
      "[I 2023-09-21 11:36:07,217] Trial 71 finished with value: 0.011299343716541616 and parameters: {'x': 1.8937016288151995}. Best is trial 61 with value: 0.0003638539715663391.\n",
      "[I 2023-09-21 11:36:07,225] Trial 72 finished with value: 0.0007775252982715578 and parameters: {'x': 2.0278841406227905}. Best is trial 61 with value: 0.0003638539715663391.\n",
      "[I 2023-09-21 11:36:07,241] Trial 73 finished with value: 0.19004676432402792 and parameters: {'x': 2.4359435334123307}. Best is trial 61 with value: 0.0003638539715663391.\n",
      "[I 2023-09-21 11:36:07,249] Trial 74 finished with value: 0.6660324912165962 and parameters: {'x': 1.1838918630373814}. Best is trial 61 with value: 0.0003638539715663391.\n",
      "[I 2023-09-21 11:36:07,258] Trial 75 finished with value: 1.248903058629068 and parameters: {'x': 3.1175433139834303}. Best is trial 61 with value: 0.0003638539715663391.\n",
      "[I 2023-09-21 11:36:07,272] Trial 76 finished with value: 0.0007221578577954465 and parameters: {'x': 1.973127005046042}. Best is trial 61 with value: 0.0003638539715663391.\n",
      "[I 2023-09-21 11:36:07,281] Trial 77 finished with value: 5.040170630229351 and parameters: {'x': -0.24503243411522924}. Best is trial 61 with value: 0.0003638539715663391.\n",
      "[I 2023-09-21 11:36:07,289] Trial 78 finished with value: 7.026860793653229 and parameters: {'x': -0.6508226635618666}. Best is trial 61 with value: 0.0003638539715663391.\n",
      "[I 2023-09-21 11:36:07,299] Trial 79 finished with value: 2.2231962163588856 and parameters: {'x': 0.5089613632239822}. Best is trial 61 with value: 0.0003638539715663391.\n",
      "[I 2023-09-21 11:36:07,307] Trial 80 finished with value: 0.00629847379803711 and parameters: {'x': 1.920637075419078}. Best is trial 61 with value: 0.0003638539715663391.\n",
      "[I 2023-09-21 11:36:07,315] Trial 81 finished with value: 0.0004213433777420987 and parameters: {'x': 1.9794733495732475}. Best is trial 61 with value: 0.0003638539715663391.\n",
      "[I 2023-09-21 11:36:07,323] Trial 82 finished with value: 0.07148573167762692 and parameters: {'x': 1.7326318424388818}. Best is trial 61 with value: 0.0003638539715663391.\n",
      "[I 2023-09-21 11:36:07,331] Trial 83 finished with value: 0.6774599416740607 and parameters: {'x': 2.823079547597959}. Best is trial 61 with value: 0.0003638539715663391.\n",
      "[I 2023-09-21 11:36:07,339] Trial 84 finished with value: 0.6794997718271018 and parameters: {'x': 1.1756822385589998}. Best is trial 61 with value: 0.0003638539715663391.\n",
      "[I 2023-09-21 11:36:07,348] Trial 85 finished with value: 2.169836244305411 and parameters: {'x': 3.4730364029125047}. Best is trial 61 with value: 0.0003638539715663391.\n",
      "[I 2023-09-21 11:36:07,356] Trial 86 finished with value: 0.0285501373207259 and parameters: {'x': 2.1689678588392654}. Best is trial 61 with value: 0.0003638539715663391.\n",
      "[I 2023-09-21 11:36:07,364] Trial 87 finished with value: 2.2665530368532107 and parameters: {'x': 0.4944924321501367}. Best is trial 61 with value: 0.0003638539715663391.\n",
      "[I 2023-09-21 11:36:07,372] Trial 88 finished with value: 0.08029455389367757 and parameters: {'x': 1.7166370633027714}. Best is trial 61 with value: 0.0003638539715663391.\n",
      "[I 2023-09-21 11:36:07,380] Trial 89 finished with value: 0.4927884012536763 and parameters: {'x': 2.7019888896939013}. Best is trial 61 with value: 0.0003638539715663391.\n",
      "[I 2023-09-21 11:36:07,391] Trial 90 finished with value: 0.6966061220337325 and parameters: {'x': 1.1653706678808056}. Best is trial 61 with value: 0.0003638539715663391.\n",
      "[I 2023-09-21 11:36:07,405] Trial 91 finished with value: 0.002429277862764719 and parameters: {'x': 2.0492877049857743}. Best is trial 61 with value: 0.0003638539715663391.\n",
      "[I 2023-09-21 11:36:07,413] Trial 92 finished with value: 0.02761915886023366 and parameters: {'x': 2.1661901286485863}. Best is trial 61 with value: 0.0003638539715663391.\n",
      "[I 2023-09-21 11:36:07,421] Trial 93 finished with value: 1.481496607308758 and parameters: {'x': 3.2171674524521094}. Best is trial 61 with value: 0.0003638539715663391.\n",
      "[I 2023-09-21 11:36:07,429] Trial 94 finished with value: 0.3727024622483324 and parameters: {'x': 2.610493621791688}. Best is trial 61 with value: 0.0003638539715663391.\n",
      "[I 2023-09-21 11:36:07,439] Trial 95 finished with value: 1.5101748807735993 and parameters: {'x': 0.7711082713381137}. Best is trial 61 with value: 0.0003638539715663391.\n",
      "[I 2023-09-21 11:36:07,453] Trial 96 finished with value: 3.326312742968295 and parameters: {'x': 3.8238181770583095}. Best is trial 61 with value: 0.0003638539715663391.\n",
      "[I 2023-09-21 11:36:07,461] Trial 97 finished with value: 3.6880398748295535 and parameters: {'x': 0.07957299674537133}. Best is trial 61 with value: 0.0003638539715663391.\n",
      "[I 2023-09-21 11:36:07,469] Trial 98 finished with value: 0.17041078996662665 and parameters: {'x': 1.5871915820060998}. Best is trial 61 with value: 0.0003638539715663391.\n",
      "[I 2023-09-21 11:36:07,477] Trial 99 finished with value: 7.745150897196233 and parameters: {'x': 4.783011120566397}. Best is trial 61 with value: 0.0003638539715663391.\n"
     ]
    }
   ],
   "source": [
    "study = optuna.create_study()\n",
    "\n",
    "study.optimize(objective, n_trials=100)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Podemos obtener el mejor parámetro de la siguiente manera."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found x: 2.019074956659619, (x-2)^2: 0.0003638539715663391\n"
     ]
    }
   ],
   "source": [
    "best_params = study.best_params\n",
    "\n",
    "found_x = best_params[\"x\"]\n",
    "\n",
    "print(\"Found x: {}, (x-2)^2: {}\".format(found_x, (found_x - 2)**2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## El objeto `study`\n",
    "\n",
    "Clarifiquemos la terminología en Optuna:\n",
    "\n",
    " - `Trial`: una sola llamada de la función objetivo\n",
    " - `Study`: una sesión de optimización, que consiste en un conjunto de trials\n",
    " - `Parameter`: una variable cuyo valor debe ser optimizado, como `x` en el ejemplo anterior.\n",
    "\n",
    "En Optuna, utilizamos el objeto `study` para lidiar con la optimización. El método `create_study()` regresa un objeto study. Este objeto tiene propiedades útiles para analizar el resultado de la optimización.\n",
    "\n",
    "Para obtener el diccionario de los nombres de los parámetros y sus valores utilizamos:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'x': 2.019074956659619}"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "study.best_params"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Para obtener el mejor valor observado de la función objetivo:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.0003638539715663391"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "study.best_value"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Y para obtener el mejor intento:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "FrozenTrial(number=61, state=TrialState.COMPLETE, values=[0.0003638539715663391], datetime_start=datetime.datetime(2023, 9, 21, 11, 36, 7, 99673), datetime_complete=datetime.datetime(2023, 9, 21, 11, 36, 7, 113052), params={'x': 2.019074956659619}, user_attrs={}, system_attrs={}, intermediate_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial_id=61, value=None)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "study.best_trial"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Si queremos todos los intentos:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "FrozenTrial(number=0, state=TrialState.COMPLETE, values=[10.439138179196712], datetime_start=datetime.datetime(2023, 9, 21, 11, 36, 6, 666218), datetime_complete=datetime.datetime(2023, 9, 21, 11, 36, 6, 666218), params={'x': -1.230965518107043}, user_attrs={}, system_attrs={}, intermediate_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial_id=0, value=None)\n",
      "FrozenTrial(number=1, state=TrialState.COMPLETE, values=[67.67353109134189], datetime_start=datetime.datetime(2023, 9, 21, 11, 36, 6, 666218), datetime_complete=datetime.datetime(2023, 9, 21, 11, 36, 6, 666218), params={'x': -6.22639234970846}, user_attrs={}, system_attrs={}, intermediate_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial_id=1, value=None)\n"
     ]
    }
   ],
   "source": [
    "study.trials\n",
    "\n",
    "for trial in study.trials[:2]: # Muestra los primeros 2 intentos\n",
    "\n",
    "    print (trial)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "El número de intentos se obtiene como:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "100"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(study.trials)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Algo importante en Optuna es que si ejecutamos nuevamente `optimize()`, podemos continuar con la optimización:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:07,623] Trial 100 finished with value: 1.6779901354999374 and parameters: {'x': 3.295372585590701}. Best is trial 61 with value: 0.0003638539715663391.\n",
      "[I 2023-09-21 11:36:07,631] Trial 101 finished with value: 0.00013646101936064974 and parameters: {'x': 1.9883183468909298}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:07,640] Trial 102 finished with value: 0.010412290722878784 and parameters: {'x': 1.8979593672947939}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:07,647] Trial 103 finished with value: 0.35331961992189004 and parameters: {'x': 1.405593051923272}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:07,663] Trial 104 finished with value: 0.18104713576853557 and parameters: {'x': 2.4254963404878302}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:07,671] Trial 105 finished with value: 0.003129880198192888 and parameters: {'x': 1.9440546677711814}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:07,679] Trial 106 finished with value: 1.3030648774500513 and parameters: {'x': 0.8584813284706854}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:07,687] Trial 107 finished with value: 0.9227622877320525 and parameters: {'x': 2.9606051674502134}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:07,695] Trial 108 finished with value: 0.33823319594081275 and parameters: {'x': 2.5815781941758242}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:07,703] Trial 109 finished with value: 0.7384090749900415 and parameters: {'x': 1.1406926772160955}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:07,711] Trial 110 finished with value: 3.232084730673866 and parameters: {'x': 0.20220003040553314}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:07,719] Trial 111 finished with value: 0.00043892707307480693 and parameters: {'x': 2.0209505864613573}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:07,727] Trial 112 finished with value: 0.00040743586486333524 and parameters: {'x': 2.0201850406207997}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:07,735] Trial 113 finished with value: 0.1677185695560272 and parameters: {'x': 1.5904654232472828}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:07,744] Trial 114 finished with value: 0.18465799971842345 and parameters: {'x': 2.429718512189577}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:07,751] Trial 115 finished with value: 3.219594067333083 and parameters: {'x': 3.7943227322120965}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:07,759] Trial 116 finished with value: 0.9179435472771746 and parameters: {'x': 2.958093704852075}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:07,767] Trial 117 finished with value: 0.361252572959702 and parameters: {'x': 1.3989570955749482}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:07,775] Trial 118 finished with value: 0.002063690402476242 and parameters: {'x': 1.9545721406791357}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:07,786] Trial 119 finished with value: 1.4737622927398566 and parameters: {'x': 0.786013882805962}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:07,792] Trial 120 finished with value: 0.05675654301934788 and parameters: {'x': 2.238236317591059}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:07,800] Trial 121 finished with value: 0.005277800549780447 and parameters: {'x': 1.92735152754682}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:07,807] Trial 122 finished with value: 1.9762572277114099 and parameters: {'x': 3.4057941626395416}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:07,820] Trial 123 finished with value: 0.5263820102484836 and parameters: {'x': 2.7255218881939287}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:07,828] Trial 124 finished with value: 0.036916929686622524 and parameters: {'x': 1.8078622117161163}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:07,836] Trial 125 finished with value: 2.2103676602136555 and parameters: {'x': 0.5132694728991216}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:07,844] Trial 126 finished with value: 0.5168381310395452 and parameters: {'x': 1.2810854494172863}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:07,852] Trial 127 finished with value: 0.036348035930599314 and parameters: {'x': 2.190651608780517}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:07,862] Trial 128 finished with value: 0.22204415987865297 and parameters: {'x': 1.5287843806932404}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:07,868] Trial 129 finished with value: 1.0175197233804034 and parameters: {'x': 3.0087218265609224}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:07,876] Trial 130 finished with value: 1.2163830246549312 and parameters: {'x': 0.8971024414502808}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:07,884] Trial 131 finished with value: 0.03865755254682653 and parameters: {'x': 2.1966152398641228}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:07,900] Trial 132 finished with value: 0.003924197578715816 and parameters: {'x': 1.9373565839156595}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:07,910] Trial 133 finished with value: 0.5002422233834276 and parameters: {'x': 2.7072780382448105}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:07,924] Trial 134 finished with value: 0.021703832096239883 and parameters: {'x': 1.8526777949654571}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:07,932] Trial 135 finished with value: 0.6174442323383905 and parameters: {'x': 1.2142238026394598}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:07,940] Trial 136 finished with value: 1.6219697149754206 and parameters: {'x': 3.2735657481949727}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:07,952] Trial 137 finished with value: 0.37143094277082744 and parameters: {'x': 2.609451345696133}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:07,956] Trial 138 finished with value: 0.1490755787202544 and parameters: {'x': 1.6138969325163883}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:07,964] Trial 139 finished with value: 0.05776816325657007 and parameters: {'x': 2.24035008478586}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:07,972] Trial 140 finished with value: 2.9650595296916578 and parameters: {'x': 3.7219348215573254}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:07,980] Trial 141 finished with value: 0.002521572301137346 and parameters: {'x': 1.9497847403557709}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:07,988] Trial 142 finished with value: 0.006756309599474582 and parameters: {'x': 1.9178032263438851}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,004] Trial 143 finished with value: 0.6807431857402614 and parameters: {'x': 1.174928375387772}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,012] Trial 144 finished with value: 2.207380188970622 and parameters: {'x': 0.5142745243583451}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,021] Trial 145 finished with value: 0.21195806833755498 and parameters: {'x': 2.4603890402013877}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,029] Trial 146 finished with value: 0.04469473623867666 and parameters: {'x': 1.7885887036162054}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,037] Trial 147 finished with value: 0.7479339704703517 and parameters: {'x': 2.8648317584769605}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,045] Trial 148 finished with value: 0.33612248975223596 and parameters: {'x': 1.4202392823308603}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,061] Trial 149 finished with value: 1.122971868375219 and parameters: {'x': 0.9402963299227378}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,069] Trial 150 finished with value: 0.022590537984474253 and parameters: {'x': 2.1503014902935904}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,077] Trial 151 finished with value: 0.1945629935431319 and parameters: {'x': 2.441092953404531}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,085] Trial 152 finished with value: 0.03045032176290311 and parameters: {'x': 1.825499794375757}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,093] Trial 153 finished with value: 1.4212007986930069 and parameters: {'x': 3.1921412662486803}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,101] Trial 154 finished with value: 0.006424315438743029 and parameters: {'x': 2.08015182741986}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,117] Trial 155 finished with value: 0.2714479110852764 and parameters: {'x': 1.4789933675227576}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,125] Trial 156 finished with value: 0.6037592896219385 and parameters: {'x': 2.77701949114674}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,133] Trial 157 finished with value: 1.0647884529120426 and parameters: {'x': 0.9681141279617971}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,141] Trial 158 finished with value: 5.4963597784767275 and parameters: {'x': -0.34443165361601613}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,149] Trial 159 finished with value: 0.19871389017076807 and parameters: {'x': 2.445773361890062}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,157] Trial 160 finished with value: 0.001870789907931953 and parameters: {'x': 2.043252628913535}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,173] Trial 161 finished with value: 0.0022625240239369394 and parameters: {'x': 1.9524340034905507}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,181] Trial 162 finished with value: 0.1992702464233616 and parameters: {'x': 1.5536030394107039}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,189] Trial 163 finished with value: 0.014097122714840576 and parameters: {'x': 1.8812686953038897}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,201] Trial 164 finished with value: 0.08912130113000809 and parameters: {'x': 2.2985319097349697}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,213] Trial 165 finished with value: 0.8261081356533891 and parameters: {'x': 2.9089049101272306}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,221] Trial 166 finished with value: 0.6213993010030346 and parameters: {'x': 1.211711156362698}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,229] Trial 167 finished with value: 1.7864322901588643 and parameters: {'x': 0.6634251647742038}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,237] Trial 168 finished with value: 1.8062061577451154 and parameters: {'x': 3.3439516947216203}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,245] Trial 169 finished with value: 3.4743123756504306 and parameters: {'x': 0.1360492561093707}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,253] Trial 170 finished with value: 0.001060210185148926 and parameters: {'x': 2.0325608689249677}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,264] Trial 171 finished with value: 0.0010731356311582528 and parameters: {'x': 2.032758748925413}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,272] Trial 172 finished with value: 0.003322277770569254 and parameters: {'x': 2.057639203417199}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,280] Trial 173 finished with value: 0.24714652987453817 and parameters: {'x': 2.497138340781053}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,288] Trial 174 finished with value: 0.2564569509241506 and parameters: {'x': 1.493584211418966}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,304] Trial 175 finished with value: 0.02301237462450992 and parameters: {'x': 2.1516983013237456}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,312] Trial 176 finished with value: 0.5338594911647507 and parameters: {'x': 2.7306568901781128}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,320] Trial 177 finished with value: 0.13005369930767097 and parameters: {'x': 1.6393704126008641}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,328] Trial 178 finished with value: 0.7922131822412389 and parameters: {'x': 1.1099364167424672}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,336] Trial 179 finished with value: 0.025221366593538956 and parameters: {'x': 2.1588123628485483}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,344] Trial 180 finished with value: 0.30580278829934504 and parameters: {'x': 2.552994383605607}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,352] Trial 181 finished with value: 0.01248613795172917 and parameters: {'x': 1.8882586112860182}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,368] Trial 182 finished with value: 0.04328251841886352 and parameters: {'x': 1.7919554893325385}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,376] Trial 183 finished with value: 0.41061021883768256 and parameters: {'x': 1.3592112525662747}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,384] Trial 184 finished with value: 0.908769609944152 and parameters: {'x': 2.9532940836615698}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,392] Trial 185 finished with value: 0.007571461701691143 and parameters: {'x': 2.087014146560724}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,403] Trial 186 finished with value: 0.2843833465882091 and parameters: {'x': 2.533276051016928}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,410] Trial 187 finished with value: 0.14966780901436247 and parameters: {'x': 1.6131307597981426}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,426] Trial 188 finished with value: 1.0105796811782222 and parameters: {'x': 0.9947240770921537}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,434] Trial 189 finished with value: 0.02043108078308041 and parameters: {'x': 2.1429373316634965}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,447] Trial 190 finished with value: 0.8034812052663266 and parameters: {'x': 2.8963711314329164}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,458] Trial 191 finished with value: 0.0017444098258819546 and parameters: {'x': 1.9582338674775608}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,460] Trial 192 finished with value: 0.022834002147147238 and parameters: {'x': 1.8488907608809202}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,474] Trial 193 finished with value: 0.1925017948991259 and parameters: {'x': 2.438750264842229}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,488] Trial 194 finished with value: 0.3773708527333096 and parameters: {'x': 1.3856948211732953}. Best is trial 101 with value: 0.00013646101936064974.\n",
      "[I 2023-09-21 11:36:08,496] Trial 195 finished with value: 5.227406154742039e-05 and parameters: {'x': 1.9927699196721322}. Best is trial 195 with value: 5.227406154742039e-05.\n",
      "[I 2023-09-21 11:36:08,504] Trial 196 finished with value: 1.8227039532059544 and parameters: {'x': 3.350075536111204}. Best is trial 195 with value: 5.227406154742039e-05.\n",
      "[I 2023-09-21 11:36:08,515] Trial 197 finished with value: 0.046725562615113236 and parameters: {'x': 2.2161609645960927}. Best is trial 195 with value: 5.227406154742039e-05.\n",
      "[I 2023-09-21 11:36:08,522] Trial 198 finished with value: 0.17843029565825994 and parameters: {'x': 1.5775898963586927}. Best is trial 195 with value: 5.227406154742039e-05.\n",
      "[I 2023-09-21 11:36:08,536] Trial 199 finished with value: 1.7977448014436097 and parameters: {'x': 0.6591999397957913}. Best is trial 195 with value: 5.227406154742039e-05.\n"
     ]
    }
   ],
   "source": [
    "study.optimize(objective, n_trials = 100)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Podemos volver a revisar el resultado:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found x: 1.9927699196721322, (x-2)^2: 5.227406154742039e-05\n"
     ]
    }
   ],
   "source": [
    "best_params = study.best_params\n",
    "\n",
    "found_x = best_params[\"x\"]\n",
    "\n",
    "print(\"Found x: {}, (x-2)^2: {}\".format(found_x, (found_x - 2)**2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Espacio de búsqueda\n",
    "\n",
    "Para el muestreo de hiperparámetros, Optuna nos ofrece las siguientes funciones:\n",
    "\n",
    " - `optuna.trial.Trial.suggest_categorical()` para parámetros categóricos\n",
    " - `optuna.trial.Trial.suggest_int()` para parámetros enteros\n",
    " - `optuna.trial.Trial.suggest_float()` para parámetros float\n",
    "\n",
    "Con argumentos opcionales de `step` y `log`, podemos discretizar o calcular el logaritmo de los parámetros enteros y flotantes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "def objective(trial):\n",
    "\n",
    "    # Parámetro categórico\n",
    "\n",
    "    optimizer = trial.suggest_categorical(\"optimizer\", [\"SGD\", \"Adam\"])\n",
    "\n",
    "    # Parámetro entero\n",
    "\n",
    "    num_layers = trial.suggest_int(\"num_layers\", 1, 3)\n",
    "\n",
    "    # Parámetro entero (con log)\n",
    "\n",
    "    num_channels = trial.suggest_int(\"num_channels\", 32, 512, log = True)\n",
    "\n",
    "    # Parámetro entero (discretizado)\n",
    "\n",
    "    num_units = trial.suggest_int(\"num_units\", 10, 100, step = 5)\n",
    "\n",
    "    # Parámetro de punto flotante\n",
    "\n",
    "    dropout_rate = trial.suggest_float(\"dropout_rate\", 0.0, 1.0)\n",
    "\n",
    "    # Parámetro de punto flotante (log)\n",
    "\n",
    "    learning_rate = trial.suggest_float(\"learning_rate\", 1e-5, 1e-2, log = True)\n",
    "\n",
    "    # Parámetro de punto flotante (discretizado)\n",
    "\n",
    "    drop_path_rate = trial.suggest_float(\"drop_path_rate\", 0.0, 1.0, step = 0.1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Definiendo espacios de parámetros\n",
    "\n",
    "En Optuna, podemos definir espacios de búsqueda utilizando la sintaxis de Python, que incluye condicionales y loops. También se pueden utilizar branches o loops, dependiendo de los valores de los parámetros.\n",
    "\n",
    "\n",
    "#### Branches"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sklearn.ensemble\n",
    "import sklearn.svm\n",
    "\n",
    "def objective(trial):\n",
    "\n",
    "    classifier_name = trial.suggest_categorical(\"classifier\", [\"SVC\", \"RandomForest\"])\n",
    "\n",
    "    if classifier_name == \"SVC\":\n",
    "\n",
    "        svc_c = trial.suggest_float(\"svc_c\", 1e-10, 1e10, log = True)\n",
    "\n",
    "        classifier_obj = sklearn.svm.SVC(C = svc_c)\n",
    "\n",
    "    else:\n",
    "\n",
    "        rf_max_depth = trial.suggest_int(\"rf_max_depth\", 2, 32, log = True)\n",
    "\n",
    "        classifier_obj = sklearn.ensemble.RandomForestClassifier(max_depth = rf_max_depth)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Loops"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "from tensorflow.keras import layers, models \n",
    "\n",
    "def create_model(trial, input_size):\n",
    "\n",
    "    n_layers = trial.suggest_int(\"n_layers\", 1, 3)\n",
    "\n",
    "    layers_list = []\n",
    "\n",
    "    for i in range(n_layers):\n",
    "\n",
    "        n_units = trial.suggest_int(\"n_units_l{}\".format(i), 4, 128, log = True)\n",
    "\n",
    "        layers_list.append(layers.Dense(n_units, input_shape = (input_size, ), activation = 'relu'))\n",
    "\n",
    "        input_size = n_units\n",
    "\n",
    "    layers_list.append(layers.Dense(10))\n",
    "\n",
    "    model = models.Sequential(layers_list)\n",
    "\n",
    "    return model\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Sobre el número de parámetros**: La dificultad del proceso de optimización incrementa casi exponencialmente con respecto al número de parámetros. Es decir, el número de trials necesarios incrementa exponencialmente conforme incrementamos el número de parámetros, así que es recomendable no añadir parámetros poco importantes.\n",
    "\n",
    "\n",
    "## Algoritmos de optimización eficientes\n",
    "\n",
    "Optuna permite realizar una optimización de hiperparámetros eficiente al utilizar algoritmos de última generación para el muestreo de hiperperámetros y al descartar eficientemente ensayos poco prometedores.\n",
    "\n",
    "### Algoritmos de muestreo\n",
    "\n",
    "Básicamente, los samplers reducen continuamente el espacio de búsqueda utilizando los registros de los valores de los parámetros sugeridos y los valores objetivo evaluados, lo que conduce a un espacio de búsqueda óptimo que otorga parámetros que conducen a mejores valores de la función objetivo.\n",
    "\n",
    "Optuna provee los siguientes algoritmos de muestreo:\n",
    "\n",
    " - Grid Search se implementa en `GridSampler`\n",
    " - Random Search se implementa en `RandomSampler`\n",
    " - Tree-structured Parzen Estimator Algorithm se implementa en `TPESampler`\n",
    " - CMA-ES Based Algorithm se implementa en `CmaEsSampler`\n",
    " - El algoritmo para permitir valores parciales fijos se implementa en `PartialFixedSampler`\n",
    " - Non-dominated Sorting Genetic Algorithm II se implementa en `NSGAIISampler`\n",
    " - Quasi Monte Carlo Sampling Algorithm se implementa en `QMCSampler`\n",
    "\n",
    "El sampler por defecto es `TPESampler`.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:16,336] A new study created in memory with name: no-name-33cdf731-bcde-4ea9-8343-dd293dfb2b3f\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sampler is: TPESampler\n"
     ]
    }
   ],
   "source": [
    "study = optuna.create_study()\n",
    "\n",
    "print(f\"Sampler is: {study.sampler.__class__.__name__}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "En caso de que deseemos usar distintos samplers, como `RandomSampler` y `CmaEsSampler`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:16,377] A new study created in memory with name: no-name-9f849da8-8783-4f14-87d5-0a998d9ccd77\n",
      "[I 2023-09-21 11:36:16,385] A new study created in memory with name: no-name-804fdb6f-bac6-45b3-859f-7eb547c6240d\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sampler is RandomSampler\n",
      "Sampler is CmaEsSampler\n"
     ]
    }
   ],
   "source": [
    "study = optuna.create_study(sampler = optuna.samplers.RandomSampler())\n",
    "print(f\"Sampler is {study.sampler.__class__.__name__}\")\n",
    "\n",
    "study = optuna.create_study(sampler = optuna.samplers.CmaEsSampler())\n",
    "print(f\"Sampler is {study.sampler.__class__.__name__}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Pruning Algorithms\n",
    "\n",
    "Los `Pruners` automáticamente detienen las pruebas poco prometedoras en las primeras fases del entrenamiento (también conocido como *early-stopping*).\n",
    "\n",
    "Optuna ofrece los siguientes algoritmos:\n",
    "\n",
    " - Median Pruning Algorithm se implementa en `MedianPruner`\n",
    " - Non-Pruning Algorithm se implementa en `NopPruner`\n",
    " - Un algoritmo para operar un pruner con tolerancia se implementa en `PatientPruner`\n",
    " - Un algoritmo para depurar el percentil específicado de ensayos se implementa en `PercentilPruner`\n",
    " - Assynchronous Successive Halving Algorithm se implementa en `SuccessiveHalvingPruner`\n",
    " - Hyperband Algorithm se implementa en `HyperbandPruner`\n",
    " - Treshold Pruning Algorithm se implementa en `TresholdPruner`\n",
    "\n",
    "\n",
    "### Activando Pruners\n",
    "\n",
    "Para activar la función de pruning, necesitamos llamar a los métodos `report()` y `should_prune()` después de cada paso de entrenamiento. `report()` monitorea periódicamente los valores objetivos intermedios. `should_prune()` decide la terminación del ensayo que no cumpla con una condición pre establecida.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "import logging\n",
    "import sys \n",
    "\n",
    "import sklearn.datasets\n",
    "import sklearn.linear_model\n",
    "import sklearn.model_selection\n",
    "\n",
    "def objective(trial):\n",
    "\n",
    "    iris = sklearn.datasets.load_iris()\n",
    "\n",
    "    classes = list(set(iris.target))\n",
    "\n",
    "    train_x, valid_x, train_y, valid_y = sklearn.model_selection.train_test_split(\n",
    "        iris.data, iris.target, test_size = 0.25, random_state = 42 \n",
    "    )\n",
    "\n",
    "    alpha = trial.suggest_float(\"alpha\", 1e-5, 1e-1, log = True)\n",
    "\n",
    "    clf = sklearn.linear_model.SGDClassifier(alpha = alpha)\n",
    "\n",
    "    for step in range(100):\n",
    "\n",
    "        clf.partial_fit(train_x, train_y, classes = classes)\n",
    "\n",
    "        # Reporte del valor objetivo intermedio\n",
    "\n",
    "        intermediate_value = 1.0 - clf.score(valid_x, valid_y)\n",
    "\n",
    "        trial.report(intermediate_value, step)\n",
    "\n",
    "        # Pruning basado en el valor intermedio\n",
    "\n",
    "        if trial.should_prune():\n",
    "\n",
    "            raise optuna.TrialPruned()\n",
    "        \n",
    "    return 1.0 - clf.score(valid_x, valid_y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:16,572] A new study created in memory with name: no-name-04528ff5-be5f-45a3-a66f-7a699d429c2d\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "A new study created in memory with name: no-name-04528ff5-be5f-45a3-a66f-7a699d429c2d\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:16,836] Trial 0 finished with value: 0.052631578947368474 and parameters: {'alpha': 0.001583681405390258}. Best is trial 0 with value: 0.052631578947368474.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 0 finished with value: 0.052631578947368474 and parameters: {'alpha': 0.001583681405390258}. Best is trial 0 with value: 0.052631578947368474.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:17,096] Trial 1 finished with value: 0.02631578947368418 and parameters: {'alpha': 0.007850586105704724}. Best is trial 1 with value: 0.02631578947368418.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 1 finished with value: 0.02631578947368418 and parameters: {'alpha': 0.007850586105704724}. Best is trial 1 with value: 0.02631578947368418.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:17,359] Trial 2 finished with value: 0.052631578947368474 and parameters: {'alpha': 0.0011859014470015467}. Best is trial 1 with value: 0.02631578947368418.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 2 finished with value: 0.052631578947368474 and parameters: {'alpha': 0.0011859014470015467}. Best is trial 1 with value: 0.02631578947368418.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:17,603] Trial 3 finished with value: 0.02631578947368418 and parameters: {'alpha': 0.03150859209518395}. Best is trial 1 with value: 0.02631578947368418.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 3 finished with value: 0.02631578947368418 and parameters: {'alpha': 0.03150859209518395}. Best is trial 1 with value: 0.02631578947368418.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:17,850] Trial 4 finished with value: 0.02631578947368418 and parameters: {'alpha': 0.0036598653895681603}. Best is trial 1 with value: 0.02631578947368418.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 4 finished with value: 0.02631578947368418 and parameters: {'alpha': 0.0036598653895681603}. Best is trial 1 with value: 0.02631578947368418.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:17,850] Trial 5 pruned. \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 5 pruned. \n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:17,881] Trial 6 pruned. \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 6 pruned. \n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:18,131] Trial 7 finished with value: 0.052631578947368474 and parameters: {'alpha': 0.008892554126735792}. Best is trial 1 with value: 0.02631578947368418.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 7 finished with value: 0.052631578947368474 and parameters: {'alpha': 0.008892554126735792}. Best is trial 1 with value: 0.02631578947368418.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:18,165] Trial 8 pruned. \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 8 pruned. \n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:18,480] Trial 9 finished with value: 0.1842105263157895 and parameters: {'alpha': 0.043199806881960554}. Best is trial 1 with value: 0.02631578947368418.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 9 finished with value: 0.1842105263157895 and parameters: {'alpha': 0.043199806881960554}. Best is trial 1 with value: 0.02631578947368418.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:18,495] Trial 10 pruned. \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 10 pruned. \n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:18,758] Trial 11 finished with value: 0.02631578947368418 and parameters: {'alpha': 0.015909739038452074}. Best is trial 1 with value: 0.02631578947368418.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 11 finished with value: 0.02631578947368418 and parameters: {'alpha': 0.015909739038452074}. Best is trial 1 with value: 0.02631578947368418.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:19,006] Trial 12 finished with value: 0.23684210526315785 and parameters: {'alpha': 0.08559381206196237}. Best is trial 1 with value: 0.02631578947368418.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 12 finished with value: 0.23684210526315785 and parameters: {'alpha': 0.08559381206196237}. Best is trial 1 with value: 0.02631578947368418.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:19,022] Trial 13 pruned. \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 13 pruned. \n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:19,278] Trial 14 finished with value: 0.02631578947368418 and parameters: {'alpha': 0.005347413609214803}. Best is trial 1 with value: 0.02631578947368418.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 14 finished with value: 0.02631578947368418 and parameters: {'alpha': 0.005347413609214803}. Best is trial 1 with value: 0.02631578947368418.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:19,317] Trial 15 pruned. \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 15 pruned. \n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:19,562] Trial 16 finished with value: 0.23684210526315785 and parameters: {'alpha': 0.027410475900033985}. Best is trial 1 with value: 0.02631578947368418.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 16 finished with value: 0.23684210526315785 and parameters: {'alpha': 0.027410475900033985}. Best is trial 1 with value: 0.02631578947368418.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:19,818] Trial 17 finished with value: 0.02631578947368418 and parameters: {'alpha': 0.004723407285569162}. Best is trial 1 with value: 0.02631578947368418.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 17 finished with value: 0.02631578947368418 and parameters: {'alpha': 0.004723407285569162}. Best is trial 1 with value: 0.02631578947368418.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:20,074] Trial 18 finished with value: 0.21052631578947367 and parameters: {'alpha': 0.09552054491541963}. Best is trial 1 with value: 0.02631578947368418.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 18 finished with value: 0.21052631578947367 and parameters: {'alpha': 0.09552054491541963}. Best is trial 1 with value: 0.02631578947368418.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:20,331] Trial 19 finished with value: 0.052631578947368474 and parameters: {'alpha': 0.030960934313489924}. Best is trial 1 with value: 0.02631578947368418.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 19 finished with value: 0.052631578947368474 and parameters: {'alpha': 0.030960934313489924}. Best is trial 1 with value: 0.02631578947368418.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:20,619] Trial 20 finished with value: 0.0 and parameters: {'alpha': 0.010216154422302231}. Best is trial 20 with value: 0.0.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 20 finished with value: 0.0 and parameters: {'alpha': 0.010216154422302231}. Best is trial 20 with value: 0.0.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:20,656] Trial 21 pruned. \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 21 pruned. \n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:20,665] Trial 22 pruned. \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 22 pruned. \n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:21,037] Trial 23 finished with value: 0.052631578947368474 and parameters: {'alpha': 0.03829470396119289}. Best is trial 20 with value: 0.0.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 23 finished with value: 0.052631578947368474 and parameters: {'alpha': 0.03829470396119289}. Best is trial 20 with value: 0.0.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:21,321] Trial 24 finished with value: 0.26315789473684215 and parameters: {'alpha': 0.0026536373200147564}. Best is trial 20 with value: 0.0.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 24 finished with value: 0.26315789473684215 and parameters: {'alpha': 0.0026536373200147564}. Best is trial 20 with value: 0.0.\n"
     ]
    }
   ],
   "source": [
    "optuna.logging.get_logger(\"optuna\").addHandler(logging.StreamHandler(sys.stdout))\n",
    "\n",
    "study = optuna.create_study(pruner = optuna.pruners.MedianPruner())\n",
    "\n",
    "study.optimize(objective, n_trials = 25)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Como podemos observar, distintos intentos fueron detenidos antes de que terminaran todas las iteraciones.\n",
    "\n",
    "### ¿Qué sampler y pruner debería usar?\n",
    "\n",
    "Para tareas **distintas** a deep learning, se recomienda:\n",
    "\n",
    " - Para `RandomSampler`, `MedianPruner` es el mejor.\n",
    " - Para `TPESampler`, `HyperbandPruner` es el mejor.\n",
    "\n",
    "Para tareas de deep learning:\n",
    "\n",
    "<table>\n",
    "    <tr>\n",
    "        <th>Recurso de cómputo en paralelo</th>\n",
    "        <th>Hiperparámetros Categóricos/Condicionales</th>\n",
    "        <th>Algoritmos recomendados</th>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td rowspan=\"2\">Limitado</td>\n",
    "        <td>No</td>\n",
    "        <td>TPE, GP-EI si el espacio de búsqueda es de baja dimensión y continuo</td>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td>Sí</td>\n",
    "        <td>TPE, GP-EI si el espacio de búsqueda es de baja dimensión y continuo</td>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td rowspan=\"2\">Suficiente</td>\n",
    "        <td>No</td>\n",
    "        <td>CMA-ES, Random Searchc</td>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td>Sí</td>\n",
    "        <td>Random Search o Genetic Algorithm</td>\n",
    "    </tr>\n",
    "</table>\n",
    "\n",
    "## Visualización para el análisis de optimización de hiperparámetros\n",
    "\n",
    "\n",
    "Optuna ofrece distintas funciones de visualización en el módulo `optuna.visualization` para analizar los resultados de la optimización de forma visual.\n",
    "\n",
    "Para mostrar cómo hacerlo, veremos cómo visualizar el historial del modelo `lightgbm` para el dataset `breast cancer`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "#import optuna\n",
    "import lightgbm as lgb\n",
    "import numpy as np\n",
    "import sklearn.datasets\n",
    "import sklearn.metrics\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "# Se puede utilizar Matplotlib en vez de Plotly para visualizar, al reemplazar 'optuna.visualization' \n",
    "# con 'optuna.visualization.matplotlib'\n",
    "\n",
    "from optuna.visualization import plot_contour\n",
    "from optuna.visualization import plot_edf\n",
    "from optuna.visualization import plot_intermediate_values\n",
    "from optuna.visualization import plot_optimization_history\n",
    "from optuna.visualization import plot_parallel_coordinate\n",
    "from optuna.visualization import plot_param_importances\n",
    "from optuna.visualization import plot_rank\n",
    "from optuna.visualization import plot_slice\n",
    "from optuna.visualization import plot_timeline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "SEED = 42\n",
    "np.random.seed(SEED)\n",
    "\n",
    "def objective(trial):\n",
    "\n",
    "    data, target = sklearn.datasets.load_breast_cancer(return_X_y = True)\n",
    "\n",
    "    train_x, valid_x, train_y, valid_y = train_test_split(data, target, test_size=0.25)\n",
    "\n",
    "    dtrain = lgb.Dataset(train_x, label = train_y)\n",
    "    dvalid = lgb.Dataset(valid_x, label = valid_y)\n",
    "\n",
    "    param = {\n",
    "        \"objective\": \"binary\",\n",
    "        \"metric\": \"auc\",\n",
    "        \"verbosity\": -1,\n",
    "        \"boosting_type\": \"gbdt\",\n",
    "        \"bagging_fraction\": trial.suggest_float(\"bagging_fraction\", 0.4, 1.0),\n",
    "        \"bagging_freq\": trial.suggest_int(\"bagging_freq\", 1, 7),\n",
    "        \"min_child_samples\": trial.suggest_int(\"min_child_samples\", 5, 100),\n",
    "    }\n",
    "\n",
    "    # Callback para pruning\n",
    "\n",
    "    pruning_callback = optuna.integration.LightGBMPruningCallback(trial, \"auc\")\n",
    "\n",
    "    gbm = lgb.train(param, dtrain, valid_sets = [dvalid], callbacks = [pruning_callback])\n",
    "\n",
    "    preds = gbm.predict(valid_x)\n",
    "\n",
    "    pred_labels = np.rint(preds)\n",
    "\n",
    "    accuracy = sklearn.metrics.accuracy_score(valid_y, pred_labels)\n",
    "\n",
    "    return accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:22,036] A new study created in memory with name: no-name-5e62f812-e50e-44c9-aca4-4e78b4154ca1\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "A new study created in memory with name: no-name-5e62f812-e50e-44c9-aca4-4e78b4154ca1\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:22,101] Trial 0 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.6247240713084175, 'bagging_freq': 7, 'min_child_samples': 75}. Best is trial 0 with value: 0.972027972027972.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 0 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.6247240713084175, 'bagging_freq': 7, 'min_child_samples': 75}. Best is trial 0 with value: 0.972027972027972.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:22,229] Trial 1 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.759195090518222, 'bagging_freq': 2, 'min_child_samples': 19}. Best is trial 0 with value: 0.972027972027972.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 1 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.759195090518222, 'bagging_freq': 2, 'min_child_samples': 19}. Best is trial 0 with value: 0.972027972027972.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:22,278] Trial 2 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.4348501673009197, 'bagging_freq': 7, 'min_child_samples': 62}. Best is trial 0 with value: 0.972027972027972.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 2 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.4348501673009197, 'bagging_freq': 7, 'min_child_samples': 62}. Best is trial 0 with value: 0.972027972027972.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:22,326] Trial 3 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.8248435466776274, 'bagging_freq': 1, 'min_child_samples': 98}. Best is trial 3 with value: 0.9790209790209791.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 3 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.8248435466776274, 'bagging_freq': 1, 'min_child_samples': 98}. Best is trial 3 with value: 0.9790209790209791.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:22,449] Trial 4 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.899465584480253, 'bagging_freq': 2, 'min_child_samples': 22}. Best is trial 3 with value: 0.9790209790209791.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 4 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.899465584480253, 'bagging_freq': 2, 'min_child_samples': 22}. Best is trial 3 with value: 0.9790209790209791.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:22,528] Trial 5 finished with value: 0.993006993006993 and parameters: {'bagging_fraction': 0.5100427059120604, 'bagging_freq': 3, 'min_child_samples': 55}. Best is trial 5 with value: 0.993006993006993.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 5 finished with value: 0.993006993006993 and parameters: {'bagging_fraction': 0.5100427059120604, 'bagging_freq': 3, 'min_child_samples': 55}. Best is trial 5 with value: 0.993006993006993.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:22,563] Trial 6 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 6 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:22,591] Trial 7 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 7 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:22,622] Trial 8 pruned. Trial was pruned at iteration 14.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 8 pruned. Trial was pruned at iteration 14.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:22,662] Trial 9 pruned. Trial was pruned at iteration 18.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 9 pruned. Trial was pruned at iteration 18.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:22,725] Trial 10 pruned. Trial was pruned at iteration 28.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 10 pruned. Trial was pruned at iteration 28.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:22,772] Trial 11 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 11 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:22,881] Trial 12 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.9733907188988407, 'bagging_freq': 1, 'min_child_samples': 98}. Best is trial 5 with value: 0.993006993006993.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 12 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.9733907188988407, 'bagging_freq': 1, 'min_child_samples': 98}. Best is trial 5 with value: 0.993006993006993.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:22,923] Trial 13 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 13 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:22,971] Trial 14 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 14 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:23,032] Trial 15 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 15 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:23,082] Trial 16 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 16 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:23,142] Trial 17 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 17 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:23,260] Trial 18 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.9905077333938093, 'bagging_freq': 2, 'min_child_samples': 54}. Best is trial 5 with value: 0.993006993006993.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 18 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.9905077333938093, 'bagging_freq': 2, 'min_child_samples': 54}. Best is trial 5 with value: 0.993006993006993.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:23,311] Trial 19 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 19 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:23,362] Trial 20 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 20 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:23,466] Trial 21 pruned. Trial was pruned at iteration 70.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 21 pruned. Trial was pruned at iteration 70.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:23,523] Trial 22 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 22 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:23,576] Trial 23 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 23 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:23,631] Trial 24 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 24 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:23,823] Trial 25 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.9539760625736883, 'bagging_freq': 1, 'min_child_samples': 5}. Best is trial 5 with value: 0.993006993006993.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 25 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.9539760625736883, 'bagging_freq': 1, 'min_child_samples': 5}. Best is trial 5 with value: 0.993006993006993.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:23,875] Trial 26 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 26 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:23,923] Trial 27 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 27 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:23,978] Trial 28 pruned. Trial was pruned at iteration 11.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 28 pruned. Trial was pruned at iteration 11.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:24,040] Trial 29 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 29 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:24,087] Trial 30 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 30 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:24,150] Trial 31 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 31 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:24,213] Trial 32 pruned. Trial was pruned at iteration 18.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 32 pruned. Trial was pruned at iteration 18.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:24,261] Trial 33 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 33 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:24,324] Trial 34 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 34 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:24,386] Trial 35 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 35 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:24,449] Trial 36 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 36 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:24,512] Trial 37 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 37 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:24,574] Trial 38 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 38 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:24,638] Trial 39 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 39 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:24,701] Trial 40 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 40 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:24,756] Trial 41 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 41 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:24,824] Trial 42 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 42 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:24,893] Trial 43 pruned. Trial was pruned at iteration 18.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 43 pruned. Trial was pruned at iteration 18.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:25,025] Trial 44 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.8823409487656906, 'bagging_freq': 1, 'min_child_samples': 48}. Best is trial 5 with value: 0.993006993006993.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 44 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.8823409487656906, 'bagging_freq': 1, 'min_child_samples': 48}. Best is trial 5 with value: 0.993006993006993.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:25,080] Trial 45 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 45 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:25,136] Trial 46 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 46 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:25,207] Trial 47 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 47 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:25,262] Trial 48 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 48 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:25,325] Trial 49 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 49 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:25,380] Trial 50 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 50 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:25,443] Trial 51 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 51 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:25,506] Trial 52 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 52 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:25,569] Trial 53 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 53 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:25,633] Trial 54 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 54 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:25,764] Trial 55 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.973380947872035, 'bagging_freq': 2, 'min_child_samples': 39}. Best is trial 5 with value: 0.993006993006993.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 55 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.973380947872035, 'bagging_freq': 2, 'min_child_samples': 39}. Best is trial 5 with value: 0.993006993006993.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:25,827] Trial 56 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 56 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:25,890] Trial 57 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 57 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:25,946] Trial 58 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 58 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:26,017] Trial 59 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 59 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:26,064] Trial 60 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 60 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:26,143] Trial 61 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 61 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:26,190] Trial 62 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 62 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:26,261] Trial 63 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 63 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:26,325] Trial 64 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 64 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:26,380] Trial 65 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 65 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:26,449] Trial 66 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 66 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:26,511] Trial 67 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 67 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:26,575] Trial 68 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 68 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:26,625] Trial 69 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 69 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:26,685] Trial 70 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 70 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:26,754] Trial 71 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 71 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:26,817] Trial 72 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 72 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:26,870] Trial 73 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 73 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:26,933] Trial 74 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 74 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:26,996] Trial 75 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 75 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:27,059] Trial 76 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 76 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:27,125] Trial 77 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 77 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:27,191] Trial 78 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 78 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:27,254] Trial 79 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 79 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:27,317] Trial 80 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 80 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:27,388] Trial 81 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 81 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:27,570] Trial 82 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.818345732780162, 'bagging_freq': 3, 'min_child_samples': 22}. Best is trial 5 with value: 0.993006993006993.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 82 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.818345732780162, 'bagging_freq': 3, 'min_child_samples': 22}. Best is trial 5 with value: 0.993006993006993.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:27,633] Trial 83 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 83 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:27,696] Trial 84 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 84 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:27,759] Trial 85 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 85 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:27,826] Trial 86 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 86 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:27,901] Trial 87 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 87 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:27,972] Trial 88 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 88 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:28,035] Trial 89 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 89 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:28,098] Trial 90 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 90 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:28,177] Trial 91 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 91 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:28,245] Trial 92 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 92 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:28,308] Trial 93 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 93 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:28,379] Trial 94 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 94 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:28,442] Trial 95 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 95 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:28,509] Trial 96 pruned. Trial was pruned at iteration 21.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 96 pruned. Trial was pruned at iteration 21.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:28,588] Trial 97 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 97 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:28,641] Trial 98 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 98 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:28,704] Trial 99 pruned. Trial was pruned at iteration 10.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 99 pruned. Trial was pruned at iteration 10.\n"
     ]
    }
   ],
   "source": [
    "study = optuna.create_study(\n",
    "    direction=\"maximize\",\n",
    "    sampler=optuna.samplers.TPESampler(seed=SEED),\n",
    "    pruner=optuna.pruners.MedianPruner(n_warmup_steps=10),\n",
    ")\n",
    "\n",
    "study.optimize(objective, n_trials=100, timeout=600)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Gráfica de funciones\n",
    "\n",
    "Para visualizar el historial de optimización:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "config": {
        "plotlyServerURL": "https://plot.ly"
       },
       "data": [
        {
         "mode": "markers",
         "name": "Objective Value",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          12,
          18,
          25,
          44,
          55,
          82
         ],
         "y": [
          0.972027972027972,
          0.972027972027972,
          0.958041958041958,
          0.9790209790209791,
          0.958041958041958,
          0.993006993006993,
          0.986013986013986,
          0.986013986013986,
          0.9790209790209791,
          0.986013986013986,
          0.986013986013986,
          0.9790209790209791
         ]
        },
        {
         "mode": "lines",
         "name": "Best Value",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35,
          36,
          37,
          38,
          39,
          40,
          41,
          42,
          43,
          44,
          45,
          46,
          47,
          48,
          49,
          50,
          51,
          52,
          53,
          54,
          55,
          56,
          57,
          58,
          59,
          60,
          61,
          62,
          63,
          64,
          65,
          66,
          67,
          68,
          69,
          70,
          71,
          72,
          73,
          74,
          75,
          76,
          77,
          78,
          79,
          80,
          81,
          82,
          83,
          84,
          85,
          86,
          87,
          88,
          89,
          90,
          91,
          92,
          93,
          94,
          95,
          96,
          97,
          98,
          99
         ],
         "y": [
          0.972027972027972,
          0.972027972027972,
          0.972027972027972,
          0.9790209790209791,
          0.9790209790209791,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993,
          0.993006993006993
         ]
        },
        {
         "marker": {
          "color": "#cccccc"
         },
         "mode": "markers",
         "name": "Infeasible Trial",
         "showlegend": false,
         "type": "scatter",
         "x": [],
         "y": []
        }
       ],
       "layout": {
        "template": {
         "data": {
          "bar": [
           {
            "error_x": {
             "color": "#2a3f5f"
            },
            "error_y": {
             "color": "#2a3f5f"
            },
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "bar"
           }
          ],
          "barpolar": [
           {
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "barpolar"
           }
          ],
          "carpet": [
           {
            "aaxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "baxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "type": "carpet"
           }
          ],
          "choropleth": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "choropleth"
           }
          ],
          "contour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "contour"
           }
          ],
          "contourcarpet": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "contourcarpet"
           }
          ],
          "heatmap": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmap"
           }
          ],
          "heatmapgl": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmapgl"
           }
          ],
          "histogram": [
           {
            "marker": {
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "histogram"
           }
          ],
          "histogram2d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2d"
           }
          ],
          "histogram2dcontour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2dcontour"
           }
          ],
          "mesh3d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "mesh3d"
           }
          ],
          "parcoords": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "parcoords"
           }
          ],
          "pie": [
           {
            "automargin": true,
            "type": "pie"
           }
          ],
          "scatter": [
           {
            "fillpattern": {
             "fillmode": "overlay",
             "size": 10,
             "solidity": 0.2
            },
            "type": "scatter"
           }
          ],
          "scatter3d": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatter3d"
           }
          ],
          "scattercarpet": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattercarpet"
           }
          ],
          "scattergeo": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergeo"
           }
          ],
          "scattergl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergl"
           }
          ],
          "scattermapbox": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattermapbox"
           }
          ],
          "scatterpolar": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolar"
           }
          ],
          "scatterpolargl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolargl"
           }
          ],
          "scatterternary": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterternary"
           }
          ],
          "surface": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "surface"
           }
          ],
          "table": [
           {
            "cells": {
             "fill": {
              "color": "#EBF0F8"
             },
             "line": {
              "color": "white"
             }
            },
            "header": {
             "fill": {
              "color": "#C8D4E3"
             },
             "line": {
              "color": "white"
             }
            },
            "type": "table"
           }
          ]
         },
         "layout": {
          "annotationdefaults": {
           "arrowcolor": "#2a3f5f",
           "arrowhead": 0,
           "arrowwidth": 1
          },
          "autotypenumbers": "strict",
          "coloraxis": {
           "colorbar": {
            "outlinewidth": 0,
            "ticks": ""
           }
          },
          "colorscale": {
           "diverging": [
            [
             0,
             "#8e0152"
            ],
            [
             0.1,
             "#c51b7d"
            ],
            [
             0.2,
             "#de77ae"
            ],
            [
             0.3,
             "#f1b6da"
            ],
            [
             0.4,
             "#fde0ef"
            ],
            [
             0.5,
             "#f7f7f7"
            ],
            [
             0.6,
             "#e6f5d0"
            ],
            [
             0.7,
             "#b8e186"
            ],
            [
             0.8,
             "#7fbc41"
            ],
            [
             0.9,
             "#4d9221"
            ],
            [
             1,
             "#276419"
            ]
           ],
           "sequential": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ],
           "sequentialminus": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ]
          },
          "colorway": [
           "#636efa",
           "#EF553B",
           "#00cc96",
           "#ab63fa",
           "#FFA15A",
           "#19d3f3",
           "#FF6692",
           "#B6E880",
           "#FF97FF",
           "#FECB52"
          ],
          "font": {
           "color": "#2a3f5f"
          },
          "geo": {
           "bgcolor": "white",
           "lakecolor": "white",
           "landcolor": "#E5ECF6",
           "showlakes": true,
           "showland": true,
           "subunitcolor": "white"
          },
          "hoverlabel": {
           "align": "left"
          },
          "hovermode": "closest",
          "mapbox": {
           "style": "light"
          },
          "paper_bgcolor": "white",
          "plot_bgcolor": "#E5ECF6",
          "polar": {
           "angularaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "radialaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "scene": {
           "xaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "yaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "zaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           }
          },
          "shapedefaults": {
           "line": {
            "color": "#2a3f5f"
           }
          },
          "ternary": {
           "aaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "baxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "caxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "title": {
           "x": 0.05
          },
          "xaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          },
          "yaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          }
         }
        },
        "title": {
         "text": "Optimization History Plot"
        },
        "xaxis": {
         "title": {
          "text": "Trial"
         }
        },
        "yaxis": {
         "title": {
          "text": "Objective Value"
         }
        }
       }
      }
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_optimization_history(study)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Para ver las curvas de aprendizaje de las pruebas:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "config": {
        "plotlyServerURL": "https://plot.ly"
       },
       "data": [
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial0",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35,
          36,
          37,
          38,
          39,
          40,
          41,
          42,
          43,
          44,
          45,
          46,
          47,
          48,
          49,
          50,
          51,
          52,
          53,
          54,
          55,
          56,
          57,
          58,
          59,
          60,
          61,
          62,
          63,
          64,
          65,
          66,
          67,
          68,
          69,
          70,
          71,
          72,
          73,
          74,
          75,
          76,
          77,
          78,
          79,
          80,
          81,
          82,
          83,
          84,
          85,
          86,
          87,
          88,
          89,
          90,
          91,
          92,
          93,
          94,
          95,
          96,
          97,
          98,
          99
         ],
         "y": [
          0.9527673741156887,
          0.9534956304619226,
          0.9534956304619226,
          0.9675405742821473,
          0.9675405742821473,
          0.9882438618393674,
          0.9863712026633374,
          0.9884519350811486,
          0.9901165210153974,
          0.9888680815647107,
          0.990220557636288,
          0.9891801914273824,
          0.990220557636288,
          0.9906367041198502,
          0.9916770703287557,
          0.992093216812318,
          0.9924053266749896,
          0.993237619642114,
          0.993237619642114,
          0.993237619642114,
          0.9931335830212235,
          0.9937578027465668,
          0.994173949230129,
          0.9937578027465668,
          0.9943820224719101,
          0.9937578027465668,
          0.9943820224719101,
          0.9952143154390345,
          0.9956304619225967,
          0.9956304619225967,
          0.9958385351643778,
          0.9958385351643778,
          0.996046608406159,
          0.996046608406159,
          0.996046608406159,
          0.9962546816479401,
          0.9958385351643778,
          0.9958385351643778,
          0.9958385351643778,
          0.9956304619225967,
          0.9956304619225967,
          0.9956304619225967,
          0.9958385351643778,
          0.996046608406159,
          0.9958385351643778,
          0.9956304619225967,
          0.996046608406159,
          0.9958385351643778,
          0.9956304619225967,
          0.996046608406159,
          0.9958385351643778,
          0.9958385351643778,
          0.9958385351643778,
          0.996046608406159,
          0.996046608406159,
          0.996046608406159,
          0.996046608406159,
          0.9962546816479401,
          0.9962546816479401,
          0.9956304619225967,
          0.9956304619225967,
          0.996046608406159,
          0.996046608406159,
          0.9964627548897211,
          0.9964627548897211,
          0.9966708281315023,
          0.9964627548897211,
          0.9970869746150645,
          0.9970869746150645,
          0.9970869746150645,
          0.9970869746150645,
          0.9970869746150645,
          0.9970869746150645,
          0.9970869746150645,
          0.9970869746150645,
          0.9970869746150645,
          0.9970869746150645,
          0.9970869746150645,
          0.9970869746150645,
          0.9970869746150645,
          0.9970869746150645,
          0.9968789013732834,
          0.9968789013732834,
          0.9966708281315023,
          0.9964627548897211,
          0.996046608406159,
          0.996046608406159,
          0.9956304619225967,
          0.9956304619225967,
          0.9958385351643778,
          0.9956304619225967,
          0.9952143154390345,
          0.9950062421972534,
          0.9952143154390345,
          0.9956304619225967,
          0.9950062421972534,
          0.9954223886808157,
          0.9950062421972534,
          0.9954223886808157,
          0.9954223886808157
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial1",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35,
          36,
          37,
          38,
          39,
          40,
          41,
          42,
          43,
          44,
          45,
          46,
          47,
          48,
          49,
          50,
          51,
          52,
          53,
          54,
          55,
          56,
          57,
          58,
          59,
          60,
          61,
          62,
          63,
          64,
          65,
          66,
          67,
          68,
          69,
          70,
          71,
          72,
          73,
          74,
          75,
          76,
          77,
          78,
          79,
          80,
          81,
          82,
          83,
          84,
          85,
          86,
          87,
          88,
          89,
          90,
          91,
          92,
          93,
          94,
          95,
          96,
          97,
          98,
          99
         ],
         "y": [
          0.9746376811594203,
          0.9822037510656436,
          0.985080988917306,
          0.9860400682011935,
          0.9888107416879796,
          0.9905157715260017,
          0.9921142369991475,
          0.9928601875532822,
          0.9928601875532822,
          0.993925831202046,
          0.994458653026428,
          0.9946717817561808,
          0.9946717817561808,
          0.9948849104859335,
          0.9948849104859335,
          0.9953111679454391,
          0.9953111679454391,
          0.9955242966751918,
          0.9961636828644501,
          0.9961636828644501,
          0.9957374254049446,
          0.9955242966751918,
          0.9955242966751918,
          0.9959505541346974,
          0.9963768115942029,
          0.9961636828644501,
          0.9963768115942029,
          0.9968030690537084,
          0.9970161977834612,
          0.9974424552429667,
          0.9974424552429667,
          0.9968030690537084,
          0.9972293265132139,
          0.9974424552429667,
          0.9972293265132139,
          0.9972293265132139,
          0.9974424552429667,
          0.9976555839727195,
          0.9976555839727195,
          0.9976555839727195,
          0.9976555839727195,
          0.9972293265132139,
          0.9974424552429667,
          0.9974424552429667,
          0.9974424552429667,
          0.9972293265132139,
          0.9970161977834612,
          0.9972293265132139,
          0.9970161977834612,
          0.9970161977834612,
          0.9974424552429667,
          0.9972293265132139,
          0.9968030690537084,
          0.9968030690537084,
          0.9968030690537084,
          0.9974424552429667,
          0.9970161977834612,
          0.9972293265132139,
          0.9974424552429667,
          0.9974424552429667,
          0.9974424552429667,
          0.9974424552429667,
          0.9972293265132139,
          0.9972293265132139,
          0.9972293265132139,
          0.9972293265132139,
          0.9972293265132139,
          0.9970161977834612,
          0.9974424552429667,
          0.9974424552429667,
          0.9978687127024722,
          0.9978687127024722,
          0.9978687127024722,
          0.9978687127024722,
          0.9978687127024722,
          0.9980818414322251,
          0.9978687127024722,
          0.9978687127024722,
          0.9978687127024722,
          0.9980818414322251,
          0.9978687127024722,
          0.9978687127024722,
          0.9978687127024722,
          0.9978687127024722,
          0.9978687127024722,
          0.9976555839727195,
          0.9974424552429667,
          0.9978687127024722,
          0.9980818414322251,
          0.9982949701619779,
          0.9980818414322251,
          0.9980818414322251,
          0.9982949701619779,
          0.9980818414322251,
          0.9980818414322251,
          0.9980818414322251,
          0.9978687127024722,
          0.9978687127024722,
          0.9978687127024722,
          0.9978687127024722
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial2",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35,
          36,
          37,
          38,
          39,
          40,
          41,
          42,
          43,
          44,
          45,
          46,
          47,
          48,
          49,
          50,
          51,
          52,
          53,
          54,
          55,
          56,
          57,
          58,
          59,
          60,
          61,
          62,
          63,
          64,
          65,
          66,
          67,
          68,
          69,
          70,
          71,
          72,
          73,
          74,
          75,
          76,
          77,
          78,
          79,
          80,
          81,
          82,
          83,
          84,
          85,
          86,
          87,
          88,
          89,
          90,
          91,
          92,
          93,
          94,
          95,
          96,
          97,
          98,
          99
         ],
         "y": [
          0.8978858784893268,
          0.9536124794745484,
          0.9613095238095238,
          0.9727011494252874,
          0.972495894909688,
          0.9770114942528736,
          0.9778325123152709,
          0.9762931034482759,
          0.9767036124794746,
          0.9779351395730707,
          0.979371921182266,
          0.9811165845648604,
          0.9819376026272578,
          0.9828612479474549,
          0.9846059113300493,
          0.9835796387520526,
          0.9848111658456487,
          0.9846059113300493,
          0.986863711001642,
          0.9866584564860427,
          0.9863505747126436,
          0.9847085385878489,
          0.9838875205254516,
          0.9836822660098522,
          0.9834770114942529,
          0.9824507389162561,
          0.9818349753694581,
          0.9818349753694581,
          0.9818349753694581,
          0.9816297208538588,
          0.9814244663382594,
          0.9812192118226601,
          0.9814244663382594,
          0.9820402298850575,
          0.9823481116584565,
          0.9823481116584565,
          0.9821428571428571,
          0.9821428571428571,
          0.9813218390804598,
          0.9809113300492611,
          0.9809113300492611,
          0.9809113300492611,
          0.9800903119868637,
          0.9800903119868637,
          0.9805008210180624,
          0.9800903119868637,
          0.9802955665024631,
          0.9805008210180624,
          0.979679802955665,
          0.9798850574712644,
          0.979679802955665,
          0.9798850574712644,
          0.9802955665024631,
          0.9802955665024631,
          0.9802955665024631,
          0.9813218390804598,
          0.9819376026272578,
          0.9823481116584565,
          0.9825533661740559,
          0.9833743842364532,
          0.9831691297208539,
          0.9835796387520526,
          0.9841954022988506,
          0.9837848932676518,
          0.9835796387520526,
          0.9829638752052545,
          0.9831691297208539,
          0.9829638752052545,
          0.9825533661740559,
          0.9821428571428571,
          0.9827586206896551,
          0.9823481116584565,
          0.9829638752052545,
          0.9831691297208539,
          0.9829638752052545,
          0.9829638752052545,
          0.9831691297208539,
          0.9835796387520526,
          0.9844006568144499,
          0.9850164203612479,
          0.9844006568144499,
          0.9844006568144499,
          0.9837848932676518,
          0.9835796387520526,
          0.9835796387520526,
          0.9837848932676518,
          0.9837848932676518,
          0.9833743842364532,
          0.9837848932676518,
          0.9839901477832512,
          0.9833743842364532,
          0.9827586206896551,
          0.9827586206896551,
          0.9827586206896551,
          0.9823481116584565,
          0.9823481116584565,
          0.9829638752052545,
          0.9829638752052545,
          0.9829638752052545,
          0.9829638752052545
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial3",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35,
          36,
          37,
          38,
          39,
          40,
          41,
          42,
          43,
          44,
          45,
          46,
          47,
          48,
          49,
          50,
          51,
          52,
          53,
          54,
          55,
          56,
          57,
          58,
          59,
          60,
          61,
          62,
          63,
          64,
          65,
          66,
          67,
          68,
          69,
          70,
          71,
          72,
          73,
          74,
          75,
          76,
          77,
          78,
          79,
          80,
          81,
          82,
          83,
          84,
          85,
          86,
          87,
          88,
          89,
          90,
          91,
          92,
          93,
          94,
          95,
          96,
          97,
          98,
          99
         ],
         "y": [
          0.9180191427382438,
          0.9369538077403246,
          0.9660840615896795,
          0.9683728672492717,
          0.9682688306283812,
          0.9771119434040783,
          0.9812734082397003,
          0.9827299209321682,
          0.9850187265917603,
          0.9857469829379941,
          0.9842904702455264,
          0.9846025801081981,
          0.9860590928006658,
          0.9868913857677902,
          0.9885559717020391,
          0.9881398252184769,
          0.9875156054931336,
          0.9864752392842281,
          0.9887640449438202,
          0.9893882646691635,
          0.9900124843945068,
          0.990220557636288,
          0.9906367041198502,
          0.9910528506034124,
          0.9912609238451935,
          0.9918851435705368,
          0.992093216812318,
          0.9931335830212235,
          0.9939658759883478,
          0.9939658759883478,
          0.9939658759883478,
          0.9947981689554724,
          0.9947981689554724,
          0.9952143154390345,
          0.9954223886808157,
          0.9962546816479401,
          0.9962546816479401,
          0.996046608406159,
          0.9958385351643778,
          0.9962546816479401,
          0.9958385351643778,
          0.9958385351643778,
          0.9958385351643778,
          0.9964627548897211,
          0.9962546816479401,
          0.9964627548897211,
          0.9968789013732834,
          0.9970869746150645,
          0.9972950478568456,
          0.9968789013732834,
          0.9972950478568456,
          0.9970869746150645,
          0.9968789013732834,
          0.9970869746150645,
          0.9966708281315023,
          0.9970869746150645,
          0.9972950478568456,
          0.9972950478568456,
          0.9975031210986267,
          0.9975031210986267,
          0.9975031210986267,
          0.9977111943404078,
          0.9977111943404078,
          0.99812734082397,
          0.99812734082397,
          0.9983354140657511,
          0.9983354140657511,
          0.9983354140657511,
          0.9983354140657511,
          0.9983354140657511,
          0.9983354140657511,
          0.99812734082397,
          0.99812734082397,
          0.99812734082397,
          0.99812734082397,
          0.99812734082397,
          0.99812734082397,
          0.99812734082397,
          0.99812734082397,
          0.997919267582189,
          0.997919267582189,
          0.997919267582189,
          0.997919267582189,
          0.99812734082397,
          0.99812734082397,
          0.99812734082397,
          0.9983354140657511,
          0.9983354140657511,
          0.99812734082397,
          0.99812734082397,
          0.99812734082397,
          0.9985434873075323,
          0.9985434873075323,
          0.9985434873075323,
          0.9985434873075323,
          0.9985434873075323,
          0.9985434873075323,
          0.9987515605493134,
          0.9987515605493134,
          0.9987515605493134
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial4",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35,
          36,
          37,
          38,
          39,
          40,
          41,
          42,
          43,
          44,
          45,
          46,
          47,
          48,
          49,
          50,
          51,
          52,
          53,
          54,
          55,
          56,
          57,
          58,
          59,
          60,
          61,
          62,
          63,
          64,
          65,
          66,
          67,
          68,
          69,
          70,
          71,
          72,
          73,
          74,
          75,
          76,
          77,
          78,
          79,
          80,
          81,
          82,
          83,
          84,
          85,
          86,
          87,
          88,
          89,
          90,
          91,
          92,
          93,
          94,
          95,
          96,
          97,
          98,
          99
         ],
         "y": [
          0.9710743801652892,
          0.9729338842975207,
          0.9694214876033058,
          0.9693181818181819,
          0.9724173553719008,
          0.9733471074380166,
          0.9737603305785124,
          0.9735537190082645,
          0.9725206611570248,
          0.9725206611570248,
          0.9723140495867768,
          0.9723140495867768,
          0.9861570247933884,
          0.9878099173553719,
          0.987603305785124,
          0.9888429752066116,
          0.9880165289256199,
          0.9880165289256199,
          0.9913223140495868,
          0.9917355371900827,
          0.9915289256198347,
          0.9911157024793389,
          0.990909090909091,
          0.9902892561983471,
          0.9900826446280991,
          0.9898760330578512,
          0.9911157024793389,
          0.9907024793388429,
          0.9902892561983471,
          0.9898760330578512,
          0.990909090909091,
          0.9911157024793389,
          0.990909090909091,
          0.9915289256198347,
          0.9913223140495868,
          0.990495867768595,
          0.9898760330578512,
          0.9900826446280991,
          0.9913223140495868,
          0.9921487603305785,
          0.9913223140495868,
          0.9911157024793389,
          0.9919421487603306,
          0.9927685950413223,
          0.9923553719008265,
          0.9931818181818182,
          0.9929752066115702,
          0.9931818181818182,
          0.993595041322314,
          0.993595041322314,
          0.993595041322314,
          0.993595041322314,
          0.993595041322314,
          0.9933884297520661,
          0.9929752066115702,
          0.9927685950413223,
          0.9929752066115702,
          0.9931818181818182,
          0.9929752066115702,
          0.9927685950413223,
          0.9925619834710744,
          0.9925619834710744,
          0.9923553719008265,
          0.9921487603305785,
          0.9921487603305785,
          0.9919421487603306,
          0.9917355371900827,
          0.9915289256198347,
          0.9913223140495868,
          0.9911157024793389,
          0.9917355371900827,
          0.9921487603305785,
          0.9921487603305785,
          0.9919421487603306,
          0.9919421487603306,
          0.9923553719008265,
          0.9919421487603306,
          0.9917355371900827,
          0.9917355371900827,
          0.9921487603305785,
          0.9919421487603306,
          0.9917355371900827,
          0.9917355371900827,
          0.9919421487603306,
          0.9915289256198347,
          0.9913223140495868,
          0.9915289256198347,
          0.9915289256198347,
          0.9915289256198347,
          0.9913223140495868,
          0.990909090909091,
          0.9907024793388429,
          0.9902892561983471,
          0.9902892561983471,
          0.9902892561983471,
          0.9907024793388429,
          0.9913223140495868,
          0.9919421487603306,
          0.9921487603305785,
          0.9919421487603306
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial5",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35,
          36,
          37,
          38,
          39,
          40,
          41,
          42,
          43,
          44,
          45,
          46,
          47,
          48,
          49,
          50,
          51,
          52,
          53,
          54,
          55,
          56,
          57,
          58,
          59,
          60,
          61,
          62,
          63,
          64,
          65,
          66,
          67,
          68,
          69,
          70,
          71,
          72,
          73,
          74,
          75,
          76,
          77,
          78,
          79,
          80,
          81,
          82,
          83,
          84,
          85,
          86,
          87,
          88,
          89,
          90,
          91,
          92,
          93,
          94,
          95,
          96,
          97,
          98,
          99
         ],
         "y": [
          0.9682389937106918,
          0.9818658280922432,
          0.9832285115303984,
          0.9819706498951782,
          0.9788259958071279,
          0.990880503144654,
          0.9917190775681342,
          0.9941299790356394,
          0.9945492662473795,
          0.9949685534591195,
          0.9958071278825996,
          0.9962264150943396,
          0.9966457023060796,
          0.9968553459119497,
          0.9974842767295597,
          0.9968553459119497,
          0.9964360587002097,
          0.9964360587002097,
          0.9966457023060796,
          0.9968553459119497,
          0.9974842767295597,
          0.9981132075471698,
          0.9983228511530399,
          0.9983228511530399,
          0.9983228511530399,
          0.9983228511530399,
          0.9985324947589098,
          0.9987421383647799,
          0.9987421383647799,
          0.9985324947589098,
          0.9985324947589098,
          0.9985324947589098,
          0.9985324947589098,
          0.9989517819706499,
          0.9991614255765199,
          0.9991614255765199,
          0.9991614255765199,
          0.9991614255765199,
          0.9991614255765199,
          0.9991614255765199,
          0.9991614255765199,
          0.9993710691823899,
          0.9989517819706499,
          0.9989517819706499,
          0.9989517819706499,
          0.9989517819706499,
          0.9991614255765199,
          0.9991614255765199,
          0.9991614255765199,
          0.9989517819706499,
          0.9989517819706499,
          0.9989517819706499,
          0.9989517819706499,
          0.9989517819706499,
          0.9991614255765199,
          0.9993710691823899,
          0.9993710691823899,
          0.9993710691823899,
          0.9993710691823899,
          0.9991614255765199,
          0.9993710691823899,
          0.9993710691823899,
          0.9993710691823899,
          0.9993710691823899,
          0.9993710691823899,
          0.9993710691823899,
          0.9993710691823899,
          0.9993710691823899,
          0.9993710691823899,
          0.9993710691823899,
          0.9993710691823899,
          0.9993710691823899,
          0.9993710691823899,
          0.9993710691823899,
          0.9993710691823899,
          0.9991614255765199,
          0.9989517819706499,
          0.9989517819706499,
          0.9991614255765199,
          0.9993710691823899,
          0.9993710691823899,
          0.9993710691823899,
          0.9993710691823899,
          0.9993710691823899,
          0.99958071278826,
          0.99958071278826,
          0.99958071278826,
          0.99958071278826,
          0.99958071278826,
          0.99958071278826,
          0.9997903563941299,
          0.9997903563941299,
          0.9997903563941299,
          0.99958071278826,
          0.99958071278826,
          0.99958071278826,
          0.99958071278826,
          0.9997903563941299,
          0.99958071278826,
          0.99958071278826
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial6",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.937061403508772,
          0.9438596491228071,
          0.9685307017543859,
          0.9671052631578947,
          0.9805921052631579,
          0.9850877192982456,
          0.9838815789473684,
          0.9867324561403509,
          0.9868421052631579,
          0.9846491228070176,
          0.9861842105263158
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial7",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9564516129032258,
          0.974516129032258,
          0.9751612903225807,
          0.9709677419354839,
          0.9690322580645161,
          0.9689247311827957,
          0.9713978494623656,
          0.9729032258064516,
          0.9735483870967742,
          0.9746236559139785,
          0.9739784946236559
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial8",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14
         ],
         "y": [
          0.9660303776683087,
          0.9689039408866995,
          0.9707512315270936,
          0.9705459770114943,
          0.9709564860426929,
          0.9833743842364532,
          0.9827586206896551,
          0.9874794745484401,
          0.9887110016420362,
          0.9895320197044335,
          0.9885057471264368,
          0.9866584564860427,
          0.9883004926108374,
          0.9889162561576355,
          0.9883004926108374
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial9",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18
         ],
         "y": [
          0.9431444991789819,
          0.9674671592775042,
          0.9916871921182266,
          0.9911740558292282,
          0.9899425287356322,
          0.9889162561576355,
          0.9880952380952381,
          0.9870689655172413,
          0.9864532019704434,
          0.9860426929392446,
          0.9854269293924466,
          0.9850164203612479,
          0.9866584564860427,
          0.9870689655172413,
          0.9878899835796388,
          0.986863711001642,
          0.986863711001642,
          0.986863711001642,
          0.9866584564860427
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial10",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28
         ],
         "y": [
          0.9515532544378699,
          0.9765779092702169,
          0.9738658777120316,
          0.9776873767258383,
          0.9875493096646942,
          0.9885355029585798,
          0.9927268244575936,
          0.9924802761341223,
          0.9919871794871795,
          0.9932199211045365,
          0.9932199211045365,
          0.9942061143984221,
          0.9944526627218935,
          0.9939595660749507,
          0.9934664694280079,
          0.9924802761341223,
          0.9922337278106509,
          0.9912475345167653,
          0.9914940828402367,
          0.9912475345167653,
          0.9907544378698225,
          0.990507889546351,
          0.9900147928994083,
          0.9895216962524654,
          0.9902613412228797,
          0.9916173570019724,
          0.992603550295858,
          0.9930966469428008,
          0.9928500986193294
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial11",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9044624746450304,
          0.943002028397566,
          0.9692697768762677,
          0.9774847870182556,
          0.9818458417849899,
          0.981237322515213,
          0.9797160243407708,
          0.9807302231237323,
          0.9795131845841785,
          0.9817444219066938,
          0.981947261663286
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial12",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35,
          36,
          37,
          38,
          39,
          40,
          41,
          42,
          43,
          44,
          45,
          46,
          47,
          48,
          49,
          50,
          51,
          52,
          53,
          54,
          55,
          56,
          57,
          58,
          59,
          60,
          61,
          62,
          63,
          64,
          65,
          66,
          67,
          68,
          69,
          70,
          71,
          72,
          73,
          74,
          75,
          76,
          77,
          78,
          79,
          80,
          81,
          82,
          83,
          84,
          85,
          86,
          87,
          88,
          89,
          90,
          91,
          92,
          93,
          94,
          95,
          96,
          97,
          98,
          99
         ],
         "y": [
          0.9467178175618073,
          0.9854006820119352,
          0.9854006820119352,
          0.9859335038363172,
          0.9909420289855072,
          0.9952046035805626,
          0.9969096334185849,
          0.9969096334185849,
          0.9969096334185849,
          0.9970161977834612,
          0.9974424552429667,
          0.9978687127024722,
          0.9972293265132139,
          0.9984015345268542,
          0.9973358908780904,
          0.9981884057971014,
          0.9978687127024722,
          0.9985080988917306,
          0.9980818414322251,
          0.9978687127024722,
          0.9985080988917306,
          0.9985080988917306,
          0.9989343563512362,
          0.9987212276214834,
          0.9989343563512362,
          0.9991474850809889,
          0.9991474850809889,
          0.9991474850809889,
          0.9993606138107417,
          0.9993606138107417,
          0.9993606138107417,
          0.9993606138107417,
          0.9993606138107417,
          0.9993606138107417,
          0.9993606138107417,
          0.9993606138107417,
          0.9993606138107417,
          0.9993606138107417,
          0.9991474850809889,
          0.9991474850809889,
          0.9991474850809889,
          0.9993606138107417,
          0.9993606138107417,
          0.9991474850809889,
          0.9991474850809889,
          0.9991474850809889,
          0.9991474850809889,
          0.9993606138107417,
          0.9991474850809889,
          0.9989343563512362,
          0.9991474850809889,
          0.9993606138107417,
          0.9993606138107417,
          0.9993606138107417,
          0.9993606138107417,
          0.9993606138107417,
          0.9993606138107417,
          0.9991474850809889,
          0.9991474850809889,
          0.9989343563512362,
          0.9991474850809889,
          0.9991474850809889,
          0.9989343563512362,
          0.9991474850809889,
          0.9993606138107417,
          0.9993606138107417,
          0.9991474850809889,
          0.9991474850809889,
          0.9989343563512362,
          0.9991474850809889,
          0.9991474850809889,
          0.9989343563512362,
          0.9989343563512362,
          0.9989343563512362,
          0.9989343563512362,
          0.9989343563512362,
          0.9989343563512362,
          0.9989343563512362,
          0.9989343563512362,
          0.9989343563512362,
          0.9987212276214834,
          0.9987212276214834,
          0.9987212276214834,
          0.9987212276214834,
          0.9987212276214834,
          0.9987212276214834,
          0.9987212276214834,
          0.9987212276214834,
          0.9987212276214834,
          0.9987212276214834,
          0.9987212276214834,
          0.9987212276214834,
          0.9987212276214834,
          0.9987212276214834,
          0.9987212276214834,
          0.9987212276214834,
          0.9987212276214834,
          0.9989343563512362,
          0.9989343563512362,
          0.9989343563512362
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial13",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9495698924731183,
          0.9704301075268817,
          0.9721505376344086,
          0.9690322580645161,
          0.9710752688172043,
          0.9697849462365591,
          0.9762365591397849,
          0.9775268817204301,
          0.979247311827957,
          0.9780645161290322,
          0.9790322580645161
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial14",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9078313253012048,
          0.9142570281124498,
          0.9559236947791164,
          0.9569277108433735,
          0.9617469879518072,
          0.981425702811245,
          0.9812248995983935,
          0.9794176706827309,
          0.9837349397590361,
          0.9843373493975903,
          0.9847389558232932
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial15",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9660215053763441,
          0.9743010752688172,
          0.9843010752688172,
          0.9847311827956989,
          0.9808602150537634,
          0.9772043010752688,
          0.9805376344086022,
          0.9766666666666667,
          0.9776344086021506,
          0.9782795698924731,
          0.9780645161290322
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial16",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.8978858784893268,
          0.9682881773399015,
          0.9735221674876847,
          0.9757799671592775,
          0.9764983579638752,
          0.9780377668308703,
          0.9772167487684729,
          0.9772167487684729,
          0.9785509031198686,
          0.9838875205254516,
          0.9834770114942529
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial17",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.972920892494929,
          0.9621703853955376,
          0.9609533468559838,
          0.9663286004056795,
          0.9675456389452333,
          0.9667342799188641,
          0.9689655172413794,
          0.9697768762677484,
          0.9699797160243407,
          0.9807302231237323,
          0.9809330628803246
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial18",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35,
          36,
          37,
          38,
          39,
          40,
          41,
          42,
          43,
          44,
          45,
          46,
          47,
          48,
          49,
          50,
          51,
          52,
          53,
          54,
          55,
          56,
          57,
          58,
          59,
          60,
          61,
          62,
          63,
          64,
          65,
          66,
          67,
          68,
          69,
          70,
          71,
          72,
          73,
          74,
          75,
          76,
          77,
          78,
          79,
          80,
          81,
          82,
          83,
          84,
          85,
          86,
          87,
          88,
          89,
          90,
          91,
          92,
          93,
          94,
          95,
          96,
          97,
          98,
          99
         ],
         "y": [
          0.9698082415340677,
          0.9882700938392492,
          0.9882700938392492,
          0.9912280701754386,
          0.9961240310077519,
          0.996328029375765,
          0.996532027743778,
          0.9971440228478172,
          0.9971440228478172,
          0.9971440228478172,
          0.9969400244798041,
          0.9969400244798041,
          0.9973480212158303,
          0.9971440228478172,
          0.9977560179518564,
          0.9981640146878825,
          0.9983680130558955,
          0.9987760097919217,
          0.9991840065279478,
          0.9991840065279478,
          0.9987760097919217,
          0.9989800081599347,
          0.9989800081599347,
          0.9991840065279478,
          0.9991840065279478,
          0.9989800081599347,
          0.9989800081599347,
          0.9989800081599347,
          0.9989800081599347,
          0.9993880048959608,
          0.9995920032639739,
          0.9991840065279478,
          0.9991840065279478,
          0.9991840065279478,
          0.9991840065279478,
          0.9991840065279478,
          0.9991840065279478,
          0.9993880048959608,
          0.9993880048959608,
          0.9993880048959608,
          0.9993880048959608,
          0.9993880048959608,
          0.9993880048959608,
          0.9993880048959608,
          0.9993880048959608,
          0.9993880048959608,
          0.9993880048959608,
          0.9993880048959608,
          0.9993880048959608,
          0.9993880048959608,
          0.9993880048959608,
          0.9993880048959608,
          0.9993880048959608,
          0.9995920032639739,
          0.9995920032639739,
          0.9995920032639739,
          0.9995920032639739,
          0.9995920032639739,
          0.9995920032639739,
          0.9993880048959608,
          0.9997960016319869,
          0.9993880048959608,
          0.9993880048959608,
          0.9993880048959608,
          0.9993880048959608,
          0.9995920032639739,
          0.9993880048959608,
          0.9993880048959608,
          0.9993880048959608,
          0.9993880048959608,
          0.9993880048959608,
          0.9993880048959608,
          0.9993880048959608,
          0.9993880048959608,
          0.9993880048959608,
          0.9991840065279478,
          0.9993880048959608,
          0.9991840065279478,
          0.9991840065279478,
          0.9991840065279478,
          0.9991840065279478,
          0.9991840065279478,
          0.9991840065279478,
          0.9991840065279478,
          0.9991840065279478,
          0.9991840065279478,
          0.9991840065279478,
          0.9991840065279478,
          0.9989800081599347,
          0.9991840065279478,
          0.9991840065279478,
          0.9991840065279478,
          0.9991840065279478,
          0.9989800081599347,
          0.9989800081599347,
          0.9989800081599347,
          0.9989800081599347,
          0.9989800081599347,
          0.9989800081599347,
          0.9989800081599347
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial19",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9108056265984654,
          0.953537936913896,
          0.953537936913896,
          0.9585464620630861,
          0.9574808184143222,
          0.9555626598465473,
          0.9628090366581415,
          0.9630221653878943,
          0.9629156010230179,
          0.9631287297527706,
          0.9695225916453538
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial20",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.8621514773200166,
          0.938722430295464,
          0.938722430295464,
          0.938722430295464,
          0.951310861423221,
          0.9526633374947981,
          0.9528714107365793,
          0.9624427798585102,
          0.9624427798585102,
          0.9634831460674157,
          0.965980024968789
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial21",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35,
          36,
          37,
          38,
          39,
          40,
          41,
          42,
          43,
          44,
          45,
          46,
          47,
          48,
          49,
          50,
          51,
          52,
          53,
          54,
          55,
          56,
          57,
          58,
          59,
          60,
          61,
          62,
          63,
          64,
          65,
          66,
          67,
          68,
          69,
          70
         ],
         "y": [
          0.9760715771951727,
          0.9810653349979193,
          0.9909488139825219,
          0.9923012900540991,
          0.9927174365376612,
          0.9926133999167707,
          0.992093216812318,
          0.9925093632958801,
          0.992093216812318,
          0.9923012900540991,
          0.9925093632958801,
          0.9924053266749896,
          0.9925093632958801,
          0.9933416562630045,
          0.9929255097794424,
          0.994173949230129,
          0.9945900957136912,
          0.9950062421972534,
          0.9950062421972534,
          0.9950062421972534,
          0.9950062421972534,
          0.9956304619225967,
          0.996046608406159,
          0.9958385351643778,
          0.9964627548897211,
          0.9968789013732834,
          0.9968789013732834,
          0.9970869746150645,
          0.9975031210986267,
          0.9975031210986267,
          0.9975031210986267,
          0.9975031210986267,
          0.9975031210986267,
          0.9977111943404078,
          0.9977111943404078,
          0.9977111943404078,
          0.997919267582189,
          0.997919267582189,
          0.997919267582189,
          0.997919267582189,
          0.997919267582189,
          0.997919267582189,
          0.997919267582189,
          0.997919267582189,
          0.997919267582189,
          0.997919267582189,
          0.997919267582189,
          0.997919267582189,
          0.997919267582189,
          0.997919267582189,
          0.997919267582189,
          0.997919267582189,
          0.997919267582189,
          0.9977111943404078,
          0.9977111943404078,
          0.9975031210986267,
          0.9977111943404078,
          0.9975031210986267,
          0.9977111943404078,
          0.9977111943404078,
          0.9975031210986267,
          0.9975031210986267,
          0.9975031210986267,
          0.9975031210986267,
          0.9975031210986267,
          0.9975031210986267,
          0.9972950478568456,
          0.9972950478568456,
          0.9975031210986267,
          0.9975031210986267,
          0.9975031210986267
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial22",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9531746031746032,
          0.948015873015873,
          0.971031746031746,
          0.9743055555555555,
          0.9754960317460317,
          0.9753968253968254,
          0.9742063492063492,
          0.9736111111111111,
          0.975,
          0.9748015873015873,
          0.9757936507936508
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial23",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9519916142557652,
          0.9626834381551362,
          0.9657232704402515,
          0.9759958071278826,
          0.9720125786163522,
          0.9740041928721174,
          0.9721174004192872,
          0.9722222222222222,
          0.9737945492662474,
          0.9730607966457023,
          0.9788259958071279
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial24",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9619611158072696,
          0.9630177514792899,
          0.9635460693153001,
          0.9684065934065934,
          0.9822485207100592,
          0.985207100591716,
          0.9842561284868977,
          0.9808748943364328,
          0.9796069315300084,
          0.9842561284868977,
          0.9867920540997465
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial25",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35,
          36,
          37,
          38,
          39,
          40,
          41,
          42,
          43,
          44,
          45,
          46,
          47,
          48,
          49,
          50,
          51,
          52,
          53,
          54,
          55,
          56,
          57,
          58,
          59,
          60,
          61,
          62,
          63,
          64,
          65,
          66,
          67,
          68,
          69,
          70,
          71,
          72,
          73,
          74,
          75,
          76,
          77,
          78,
          79,
          80,
          81,
          82,
          83,
          84,
          85,
          86,
          87,
          88,
          89,
          90,
          91,
          92,
          93,
          94,
          95,
          96,
          97,
          98,
          99
         ],
         "y": [
          0.9952451394759088,
          0.9952451394759088,
          0.9951394759087067,
          0.9975697379543533,
          0.9976754015215553,
          0.9983093829247676,
          0.9980980557903635,
          0.9978867286559594,
          0.9978867286559594,
          0.9972527472527473,
          0.9972527472527473,
          0.9980980557903635,
          0.9980980557903635,
          0.9980980557903635,
          0.9983093829247676,
          0.9983093829247676,
          0.9985207100591716,
          0.9987320371935756,
          0.9987320371935756,
          0.9991546914623838,
          0.9989433643279797,
          0.9989433643279797,
          0.9989433643279797,
          0.9989433643279797,
          0.9991546914623838,
          0.9991546914623838,
          0.9991546914623838,
          0.9991546914623838,
          0.9989433643279797,
          0.9989433643279797,
          0.9989433643279797,
          0.9989433643279797,
          0.9991546914623838,
          0.9989433643279797,
          0.9989433643279797,
          0.9989433643279797,
          0.9989433643279797,
          0.9987320371935756,
          0.9987320371935756,
          0.9987320371935756,
          0.9987320371935756,
          0.9989433643279797,
          0.9989433643279797,
          0.9989433643279797,
          0.9989433643279797,
          0.9989433643279797,
          0.9989433643279797,
          0.9989433643279797,
          0.9989433643279797,
          0.9989433643279797,
          0.9989433643279797,
          0.9989433643279797,
          0.9989433643279797,
          0.9987320371935756,
          0.9987320371935756,
          0.9987320371935756,
          0.9987320371935756,
          0.9987320371935756,
          0.9987320371935756,
          0.9987320371935756,
          0.9987320371935756,
          0.9987320371935756,
          0.9987320371935756,
          0.9987320371935756,
          0.9987320371935756,
          0.9987320371935756,
          0.9987320371935756,
          0.9989433643279797,
          0.9989433643279797,
          0.9989433643279797,
          0.9989433643279797,
          0.9989433643279797,
          0.9987320371935756,
          0.9989433643279797,
          0.9989433643279797,
          0.9989433643279797,
          0.9989433643279797,
          0.9989433643279797,
          0.9989433643279797,
          0.9989433643279797,
          0.9987320371935756,
          0.9987320371935756,
          0.9987320371935756,
          0.9987320371935756,
          0.9987320371935756,
          0.9987320371935756,
          0.9987320371935756,
          0.9987320371935756,
          0.9987320371935756,
          0.9987320371935756,
          0.9987320371935756,
          0.9987320371935756,
          0.9987320371935756,
          0.9987320371935756,
          0.9987320371935756,
          0.9987320371935756,
          0.9987320371935756,
          0.9987320371935756,
          0.9987320371935756,
          0.9987320371935756
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial26",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9626506024096385,
          0.9883534136546185,
          0.986144578313253,
          0.989558232931727,
          0.9886546184738956,
          0.9910642570281124,
          0.9909638554216867,
          0.9897590361445783,
          0.9913654618473896,
          0.9907630522088353,
          0.989558232931727
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial27",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9774793388429752,
          0.984607438016529,
          0.9873966942148761,
          0.9886363636363636,
          0.9883264462809918,
          0.987706611570248,
          0.9889462809917355,
          0.9880165289256199,
          0.9886363636363636,
          0.9888429752066116,
          0.9896694214876033
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial28",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11
         ],
         "y": [
          0.9464876033057851,
          0.946590909090909,
          0.9725206611570248,
          0.9740702479338843,
          0.9857438016528925,
          0.987603305785124,
          0.9899793388429752,
          0.990599173553719,
          0.9916322314049587,
          0.9926652892561983,
          0.9945247933884298,
          0.9934917355371901
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial29",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9386546184738955,
          0.9497991967871486,
          0.9662650602409638,
          0.9635542168674699,
          0.9629518072289157,
          0.9694779116465864,
          0.9734939759036144,
          0.9758032128514056,
          0.9813253012048193,
          0.9809236947791165,
          0.9843373493975903
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial30",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9583853516437786,
          0.9839783603828548,
          0.9863712026633374,
          0.9923012900540991,
          0.9925093632958801,
          0.9900124843945068,
          0.9918851435705368,
          0.9918851435705368,
          0.9893882646691635,
          0.992093216812318,
          0.9918851435705368
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial31",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9328798185941043,
          0.9630385487528345,
          0.9630385487528345,
          0.9616780045351474,
          0.9781179138321996,
          0.9811791383219954,
          0.9802721088435374,
          0.9836734693877551,
          0.9832199546485261,
          0.98140589569161,
          0.9839002267573697
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial32",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18
         ],
         "y": [
          0.9634297520661157,
          0.9885330578512397,
          0.9920454545454546,
          0.9920454545454546,
          0.9941115702479338,
          0.9930785123966942,
          0.9945247933884298,
          0.993801652892562,
          0.9948347107438017,
          0.993801652892562,
          0.993698347107438,
          0.993698347107438,
          0.9954545454545455,
          0.993801652892562,
          0.9958677685950413,
          0.9956611570247934,
          0.9954545454545455,
          0.9952479338842976,
          0.9956611570247934
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial33",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.8703703703703703,
          0.9588014981273408,
          0.9721181856013317,
          0.9707657095297545,
          0.9757594673325011,
          0.9805451518934665,
          0.9786724927174365,
          0.9814814814814815,
          0.9796088223054515,
          0.9816895547232626,
          0.982521847690387
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial34",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9325738916256158,
          0.9481732348111659,
          0.9510467980295566,
          0.9785509031198686,
          0.9826559934318555,
          0.9802955665024631,
          0.9833743842364532,
          0.9821428571428571,
          0.9806034482758621,
          0.9809113300492611,
          0.9792692939244664
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial35",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9456391875746715,
          0.9468339307048984,
          0.9594782954998009,
          0.9582835523695739,
          0.9586818000796495,
          0.9732178414974114,
          0.9732178414974114,
          0.9741138988450816,
          0.9775985663082437,
          0.9730187176423736,
          0.9771007566706491
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial36",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.927194018103109,
          0.9530696576151122,
          0.9718614718614719,
          0.9695985832349469,
          0.9713695395513577,
          0.9720582447855175,
          0.9742227469500196,
          0.9759937032664305,
          0.9750098386462023,
          0.9752066115702479,
          0.9752066115702479
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial37",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.953514739229025,
          0.9616780045351474,
          0.9713151927437642,
          0.9706349206349206,
          0.9717687074829932,
          0.9701814058956916,
          0.97437641723356,
          0.9767573696145124,
          0.9787981859410431,
          0.9785714285714285,
          0.9781179138321996
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial38",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9733552631578948,
          0.962938596491228,
          0.9650219298245614,
          0.9661184210526316,
          0.9736842105263158,
          0.9754385964912281,
          0.9771929824561404,
          0.9802631578947368,
          0.9802631578947368,
          0.9802631578947368,
          0.9800438596491228
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial39",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9066937119675457,
          0.9066937119675457,
          0.971501014198783,
          0.9764705882352941,
          0.9811359026369169,
          0.9853955375253549,
          0.982657200811359,
          0.9822515212981744,
          0.9844827586206897,
          0.9890466531440162,
          0.9879310344827587
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial40",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9798182586644125,
          0.9900676246830094,
          0.9908072696534235,
          0.9920752324598479,
          0.9914412510566357,
          0.9916525781910397,
          0.9928148774302621,
          0.9919695688926459,
          0.99270921386306,
          0.9931318681318682,
          0.9943998309382924
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial41",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9708677685950413,
          0.9683884297520661,
          0.9692148760330579,
          0.9752066115702479,
          0.9817148760330578,
          0.9839876033057852,
          0.9891528925619835,
          0.9885330578512397,
          0.990599173553719,
          0.9891528925619835,
          0.9883264462809918
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial42",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.945124716553288,
          0.9692743764172336,
          0.9723356009070295,
          0.969954648526077,
          0.9727891156462585,
          0.973015873015873,
          0.9724489795918367,
          0.9706349206349206,
          0.969047619047619,
          0.9665532879818595,
          0.9660997732426304
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial43",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18
         ],
         "y": [
          0.9819727891156462,
          0.9753968253968254,
          0.9947845804988662,
          0.9947845804988662,
          0.9941043083900227,
          0.9950113378684807,
          0.9959183673469387,
          0.9961451247165533,
          0.9956916099773243,
          0.9943310657596371,
          0.9956916099773243,
          0.9952380952380953,
          0.9952380952380953,
          0.9954648526077098,
          0.9959183673469387,
          0.9950113378684807,
          0.9950113378684807,
          0.9956916099773243,
          0.9956916099773243
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial44",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35,
          36,
          37,
          38,
          39,
          40,
          41,
          42,
          43,
          44,
          45,
          46,
          47,
          48,
          49,
          50,
          51,
          52,
          53,
          54,
          55,
          56,
          57,
          58,
          59,
          60,
          61,
          62,
          63,
          64,
          65,
          66,
          67,
          68,
          69,
          70,
          71,
          72,
          73,
          74,
          75,
          76,
          77,
          78,
          79,
          80,
          81,
          82,
          83,
          84,
          85,
          86,
          87,
          88,
          89,
          90,
          91,
          92,
          93,
          94,
          95,
          96,
          97,
          98,
          99
         ],
         "y": [
          0.9732182877633349,
          0.9863290004482295,
          0.9910354101299865,
          0.9890183774092335,
          0.9908112953832362,
          0.9917077543702376,
          0.9932765575974899,
          0.9928283281039892,
          0.9950694755714926,
          0.9968623935454953,
          0.9961900493052442,
          0.9970865082922457,
          0.997982967279247,
          0.997982967279247,
          0.9982070820259973,
          0.9982070820259973,
          0.9977588525324966,
          0.9984311967727476,
          0.9982070820259973,
          0.9982070820259973,
          0.997982967279247,
          0.9986553115194979,
          0.9988794262662484,
          0.9991035410129987,
          0.9988794262662484,
          0.9991035410129987,
          0.9991035410129987,
          0.9991035410129987,
          0.9991035410129987,
          0.9991035410129987,
          0.9986553115194979,
          0.9991035410129987,
          0.9988794262662484,
          0.9988794262662484,
          0.9991035410129987,
          0.9991035410129987,
          0.9991035410129987,
          0.9991035410129987,
          0.9991035410129987,
          0.9991035410129987,
          0.999327655759749,
          0.999327655759749,
          0.999327655759749,
          0.999327655759749,
          0.999327655759749,
          0.9995517705064993,
          0.9995517705064993,
          0.9995517705064993,
          0.9995517705064993,
          0.9995517705064993,
          0.999327655759749,
          0.999327655759749,
          0.9995517705064993,
          0.999327655759749,
          0.9995517705064993,
          0.9995517705064993,
          0.9995517705064993,
          0.9995517705064993,
          0.9995517705064993,
          0.9995517705064993,
          0.9995517705064993,
          0.9995517705064993,
          0.9995517705064993,
          0.9995517705064993,
          0.9995517705064993,
          0.9995517705064993,
          0.9995517705064993,
          0.9995517705064993,
          0.9995517705064993,
          0.9995517705064993,
          0.9995517705064993,
          0.9995517705064993,
          0.9995517705064993,
          0.9995517705064993,
          0.9995517705064993,
          0.9995517705064993,
          0.9995517705064993,
          0.9995517705064993,
          0.9995517705064993,
          0.9995517705064993,
          0.9995517705064993,
          0.9995517705064993,
          0.9995517705064993,
          0.9995517705064993,
          0.9995517705064993,
          0.9995517705064993,
          0.9995517705064993,
          0.9995517705064993,
          0.9995517705064993,
          0.9995517705064993,
          0.9995517705064993,
          0.9995517705064993,
          0.9995517705064993,
          0.9995517705064993,
          0.9995517705064993,
          0.9995517705064993,
          0.9995517705064993,
          0.9995517705064993,
          0.9995517705064993,
          0.9995517705064993
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial45",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9444444444444444,
          0.9506064162754304,
          0.9663536776212832,
          0.9745696400625978,
          0.9728090766823161,
          0.977112676056338,
          0.9821987480438185,
          0.9839593114241002,
          0.9853286384976526,
          0.9843505477308294,
          0.9827856025039123
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial46",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9551362683438155,
          0.970859538784067,
          0.970125786163522,
          0.969706498951782,
          0.9716981132075472,
          0.9712788259958072,
          0.9772536687631027,
          0.9766247379454926,
          0.9783018867924528,
          0.9786163522012579,
          0.9781970649895179
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial47",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9615702479338843,
          0.9646694214876033,
          0.9702479338842975,
          0.9824380165289256,
          0.9806818181818182,
          0.9848140495867769,
          0.9879132231404959,
          0.9902892561983471,
          0.9890495867768595,
          0.9913223140495868,
          0.9917355371900827
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial48",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9259031198686372,
          0.9412972085385879,
          0.9727011494252874,
          0.9713669950738916,
          0.9711617405582923,
          0.9686986863711001,
          0.9734195402298851,
          0.9736247947454844,
          0.9753694581280788,
          0.9760878489326765,
          0.9813218390804598
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial49",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9630541871921182,
          0.9670566502463054,
          0.9757799671592775,
          0.9754720853858785,
          0.9770114942528736,
          0.979064039408867,
          0.979679802955665,
          0.9813218390804598,
          0.9831691297208539,
          0.9838875205254516,
          0.9846059113300493
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial50",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9389264581572274,
          0.9660819949281487,
          0.9562552831783601,
          0.963229078613694,
          0.9604818258664413,
          0.9647083685545224,
          0.963229078613694,
          0.9628064243448858,
          0.9648140321217245,
          0.9665046491969569,
          0.9656593406593407
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial51",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9144088669950738,
          0.9338054187192119,
          0.9615147783251231,
          0.9652093596059114,
          0.9657224958949097,
          0.9695197044334976,
          0.9698275862068966,
          0.9734195402298851,
          0.9758825944170771,
          0.9738300492610837,
          0.9769088669950738
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial52",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.971031746031746,
          0.9799603174603174,
          0.9798611111111111,
          0.9833333333333333,
          0.983829365079365,
          0.9834325396825396,
          0.9819444444444444,
          0.9829365079365079,
          0.9853174603174604,
          0.9878968253968254,
          0.9892857142857143
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial53",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9429006085192698,
          0.9852941176470589,
          0.9864097363083164,
          0.988133874239351,
          0.9896551724137931,
          0.9905679513184584,
          0.9907707910750507,
          0.9908722109533469,
          0.9906693711967546,
          0.9918864097363083,
          0.9918864097363083
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial54",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9554507337526206,
          0.9654088050314465,
          0.9626834381551362,
          0.9621593291404612,
          0.9724318658280923,
          0.9748427672955975,
          0.9792452830188679,
          0.980503144654088,
          0.9825995807127883,
          0.980293501048218,
          0.979874213836478
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial55",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35,
          36,
          37,
          38,
          39,
          40,
          41,
          42,
          43,
          44,
          45,
          46,
          47,
          48,
          49,
          50,
          51,
          52,
          53,
          54,
          55,
          56,
          57,
          58,
          59,
          60,
          61,
          62,
          63,
          64,
          65,
          66,
          67,
          68,
          69,
          70,
          71,
          72,
          73,
          74,
          75,
          76,
          77,
          78,
          79,
          80,
          81,
          82,
          83,
          84,
          85,
          86,
          87,
          88,
          89,
          90,
          91,
          92,
          93,
          94,
          95,
          96,
          97,
          98,
          99
         ],
         "y": [
          0.9811075569772091,
          0.9890043982407037,
          0.9919032387045182,
          0.9936025589764095,
          0.9967013194722111,
          0.9965013994402239,
          0.9972011195521792,
          0.9954018392642943,
          0.9974010395841664,
          0.9972011195521792,
          0.997001199520192,
          0.9962015193922431,
          0.9960015993602559,
          0.9956017592962815,
          0.9964014394242303,
          0.9964014394242303,
          0.9972011195521792,
          0.9974010395841664,
          0.997001199520192,
          0.9974010395841664,
          0.9982007197121151,
          0.9986005597760895,
          0.9988004798080767,
          0.9986005597760895,
          0.9984006397441023,
          0.9984006397441023,
          0.9986005597760895,
          0.9986005597760895,
          0.9986005597760895,
          0.9988004798080767,
          0.9988004798080767,
          0.999000399840064,
          0.999000399840064,
          0.999000399840064,
          0.999000399840064,
          0.999000399840064,
          0.9992003198720512,
          0.999000399840064,
          0.9992003198720512,
          0.9994002399040384,
          0.9996001599360256,
          0.9994002399040384,
          0.9994002399040384,
          0.9994002399040384,
          0.9996001599360256,
          0.9996001599360256,
          0.9996001599360256,
          0.9998000799680128,
          1,
          1,
          1,
          1,
          1,
          1,
          0.9998000799680128,
          0.9998000799680128,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial56",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9509127789046653,
          0.9535496957403651,
          0.957606490872211,
          0.9578093306288032,
          0.9609533468559838,
          0.9609533468559838,
          0.9628803245436105,
          0.961764705882353,
          0.9656186612576065,
          0.9614604462474645,
          0.9611561866125761
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial57",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9496569814366425,
          0.9677158999192897,
          0.9666061339790153,
          0.9649919289749799,
          0.9674132364810331,
          0.9689265536723164,
          0.9693301049233253,
          0.9759887005649718,
          0.9753833736884584,
          0.9798224374495561,
          0.9808313155770783
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial58",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9028147163120568,
          0.9688608156028369,
          0.9700797872340425,
          0.9685283687943262,
          0.9700797872340425,
          0.9736258865248227,
          0.9738475177304965,
          0.9725177304964538,
          0.9730718085106383,
          0.972406914893617,
          0.9728501773049646
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial59",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9656462585034014,
          0.9714285714285714,
          0.9812925170068028,
          0.9886621315192744,
          0.9892290249433107,
          0.9893424036281179,
          0.9850340136054422,
          0.9848072562358277,
          0.9848072562358277,
          0.9852607709750567,
          0.9861678004535147
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial60",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9721852836879432,
          0.9863696808510638,
          0.9888076241134752,
          0.990802304964539,
          0.9917996453900709,
          0.993572695035461,
          0.9940159574468085,
          0.9937943262411347,
          0.993572695035461,
          0.9933510638297872,
          0.9940159574468085
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial61",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.984400826446281,
          0.9837809917355372,
          0.9879132231404959,
          0.9873966942148761,
          0.9880165289256199,
          0.9878099173553719,
          0.9883264462809918,
          0.9884297520661157,
          0.9890495867768595,
          0.9890495867768595,
          0.9890495867768595
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial62",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9353325173398613,
          0.9356385148918809,
          0.9425744594043247,
          0.941656466748266,
          0.9452264381884945,
          0.9511423908608732,
          0.9521623827009384,
          0.9575683394532843,
          0.9587923296613627,
          0.9577723378212974,
          0.9557323541411669
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial63",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9175191815856778,
          0.9347826086956522,
          0.9539641943734015,
          0.9542838874680307,
          0.9557757885763001,
          0.956308610400682,
          0.9568414322250639,
          0.9564151747655584,
          0.9552429667519181,
          0.9552429667519181,
          0.9565217391304348
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial64",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9666739036039947,
          0.9706904038211029,
          0.97405557967868,
          0.9813287016934433,
          0.9833912288319583,
          0.9823056882327399,
          0.9841511072514112,
          0.9832826747720365,
          0.9802431610942249,
          0.9807859313938341,
          0.9814372557533652
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial65",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9278559249786871,
          0.9337169650468883,
          0.9337169650468883,
          0.9576939471440751,
          0.9591858482523444,
          0.9598252344416027,
          0.9591858482523444,
          0.9607843137254902,
          0.9597186700767263,
          0.9628090366581415,
          0.9602514919011083
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial66",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9173710904480136,
          0.9106086221470837,
          0.9467455621301775,
          0.9578402366863905,
          0.9572062552831784,
          0.9575232459847844,
          0.9584742180896028,
          0.9602704987320372,
          0.9584742180896028,
          0.9600591715976331,
          0.9604818258664413
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial67",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9760230179028133,
          0.9844416027280477,
          0.986466325660699,
          0.9879582267689685,
          0.9884910485933504,
          0.9876385336743393,
          0.987531969309463,
          0.9866794543904518,
          0.9876385336743393,
          0.9884910485933504,
          0.9874254049445865
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial68",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9117647058823529,
          0.9595334685598377,
          0.9768762677484787,
          0.9842799188640974,
          0.9835699797160243,
          0.986815415821501,
          0.9836713995943205,
          0.9842799188640974,
          0.9859026369168357,
          0.986105476673428,
          0.9869168356997972
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial69",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9869735128093791,
          0.9913156752062527,
          0.992509769865393,
          0.9946808510638298,
          0.9928354320451585,
          0.9924012158054711,
          0.991098567086409,
          0.991098567086409,
          0.9919669995657837,
          0.9945722970039079,
          0.9946808510638298
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial70",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9537943696450428,
          0.9928600571195431,
          0.9946960424316605,
          0.9938800489596084,
          0.992452060383517,
          0.99265605875153,
          0.9929620563035496,
          0.994390044879641,
          0.9929620563035496,
          0.9925540595675234,
          0.9933700530395757
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial71",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.932258064516129,
          0.9451612903225807,
          0.9582795698924731,
          0.9582795698924731,
          0.9615053763440861,
          0.9601075268817204,
          0.9687096774193549,
          0.9687096774193549,
          0.9747311827956989,
          0.9770967741935483,
          0.9802150537634409
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial72",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.8869892473118279,
          0.9549462365591398,
          0.9639784946236559,
          0.9674193548387097,
          0.9654838709677419,
          0.968494623655914,
          0.9670967741935483,
          0.9704301075268817,
          0.969247311827957,
          0.9760215053763441,
          0.9783870967741936
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial73",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9532182438192668,
          0.9751705029838023,
          0.9751705029838023,
          0.9751705029838023,
          0.9845481670929241,
          0.9845481670929241,
          0.9836956521739131,
          0.9839087809036658,
          0.9843350383631714,
          0.9844416027280477,
          0.985080988917306
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial74",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9010989010989011,
          0.9010989010989011,
          0.9624894336432798,
          0.9680896027049873,
          0.9680896027049873,
          0.9787616229923922,
          0.9791842772612004,
          0.9804522400676247,
          0.981825866441251,
          0.981614539306847,
          0.9807692307692307
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial75",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9734784446322908,
          0.9792899408284024,
          0.9825655114116653,
          0.9876373626373627,
          0.9878486897717667,
          0.9921808960270498,
          0.9937658495350803,
          0.9939771766694844,
          0.9933431952662722,
          0.9943998309382924,
          0.9939771766694844
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial76",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9706486042692939,
          0.9828612479474549,
          0.9888136288998358,
          0.9886083743842364,
          0.9902504105090312,
          0.9901477832512315,
          0.9914819376026273,
          0.9916871921182266,
          0.9920977011494253,
          0.993123973727422,
          0.9919950738916257
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial77",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9608555286521389,
          0.9815375302663438,
          0.9822437449556094,
          0.981638418079096,
          0.9824455205811138,
          0.981638418079096,
          0.9856739305891848,
          0.9854721549636803,
          0.9822437449556094,
          0.9830508474576272,
          0.9828490718321227
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial78",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9392543859649123,
          0.969078947368421,
          0.968421052631579,
          0.9707236842105263,
          0.9709429824561403,
          0.9707236842105263,
          0.9791666666666666,
          0.9799342105263158,
          0.980701754385965,
          0.9802631578947368,
          0.9763157894736842
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial79",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9758635039533916,
          0.9858510195588848,
          0.9854348730753225,
          0.987411568872243,
          0.9878277153558053,
          0.9882438618393674,
          0.9897003745318352,
          0.9915730337078652,
          0.9929255097794424,
          0.9929255097794424,
          0.9927174365376612
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial80",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9267554479418886,
          0.9591404358353511,
          0.9534907183212268,
          0.9579297820823245,
          0.9584342211460856,
          0.9598466505246166,
          0.9621670702179177,
          0.9620661824051655,
          0.9624697336561744,
          0.9633777239709443,
          0.9653954802259888
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial81",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9863801452784504,
          0.9887005649717514,
          0.9952582728006457,
          0.9943502824858758,
          0.9941485068603713,
          0.9933414043583535,
          0.9937449556093624,
          0.994955609362389,
          0.9943502824858758,
          0.9933414043583535,
          0.9943502824858758
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial82",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35,
          36,
          37,
          38,
          39,
          40,
          41,
          42,
          43,
          44,
          45,
          46,
          47,
          48,
          49,
          50,
          51,
          52,
          53,
          54,
          55,
          56,
          57,
          58,
          59,
          60,
          61,
          62,
          63,
          64,
          65,
          66,
          67,
          68,
          69,
          70,
          71,
          72,
          73,
          74,
          75,
          76,
          77,
          78,
          79,
          80,
          81,
          82,
          83,
          84,
          85,
          86,
          87,
          88,
          89,
          90,
          91,
          92,
          93,
          94,
          95,
          96,
          97,
          98,
          99
         ],
         "y": [
          0.9745283018867924,
          0.9791404612159329,
          0.9875262054507338,
          0.9882599580712789,
          0.990880503144654,
          0.9929769392033543,
          0.9954926624737945,
          0.9968553459119497,
          0.9970649895178197,
          0.9962264150943396,
          0.9962264150943396,
          0.9964360587002097,
          0.9972746331236897,
          0.9974842767295597,
          0.9972746331236897,
          0.9976939203354298,
          0.9981132075471698,
          0.9981132075471698,
          0.9983228511530399,
          0.9983228511530399,
          0.9985324947589098,
          0.9985324947589098,
          0.9989517819706499,
          0.9989517819706499,
          0.9987421383647799,
          0.9989517819706499,
          0.9989517819706499,
          0.9993710691823899,
          1,
          1,
          1,
          1,
          1,
          0.9997903563941299,
          0.9997903563941299,
          0.9997903563941299,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          1,
          0.9997903563941299,
          0.9997903563941299,
          0.9997903563941299,
          1,
          0.9997903563941299,
          0.9997903563941299,
          0.9997903563941299,
          0.9997903563941299,
          0.9997903563941299,
          0.9997903563941299,
          0.9997903563941299,
          0.9997903563941299,
          0.9997903563941299,
          0.9997903563941299,
          0.9997903563941299,
          0.9997903563941299,
          0.9997903563941299,
          0.9997903563941299,
          0.9997903563941299,
          0.99958071278826,
          0.99958071278826,
          0.9991614255765199,
          0.9991614255765199,
          0.9993710691823899,
          0.9991614255765199,
          0.9991614255765199,
          0.9991614255765199,
          0.9993710691823899,
          0.9993710691823899,
          0.99958071278826,
          0.99958071278826,
          0.99958071278826,
          0.99958071278826,
          0.9997903563941299,
          1,
          0.9997903563941299,
          1,
          0.9997903563941299,
          0.9997903563941299,
          0.9997903563941299,
          0.99958071278826,
          0.99958071278826,
          0.99958071278826,
          0.99958071278826,
          0.9997903563941299,
          0.9997903563941299,
          1,
          1,
          1,
          1
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial83",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9916142557651991,
          0.9946540880503144,
          0.9946540880503144,
          0.9892033542976939,
          0.9881551362683438,
          0.9935010482180293,
          0.9953878406708595,
          0.9958071278825996,
          0.9958071278825996,
          0.9955974842767296,
          0.9955974842767296
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial84",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9517263427109974,
          0.9689897698209718,
          0.9760230179028133,
          0.9815643648763853,
          0.9764492753623188,
          0.9819906223358909,
          0.9766624040920716,
          0.9770886615515771,
          0.9794330775788577,
          0.9792199488491049,
          0.9820971867007673
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial85",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9631944444444445,
          0.9634920634920635,
          0.9638888888888889,
          0.9625992063492064,
          0.9734126984126984,
          0.9738095238095238,
          0.9680555555555556,
          0.9677579365079365,
          0.9683531746031746,
          0.9675595238095238,
          0.9679563492063492
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial86",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9538177339901478,
          0.9742405582922824,
          0.9761904761904762,
          0.9743431855500821,
          0.9743431855500821,
          0.9737274220032841,
          0.9758825944170771,
          0.9736247947454844,
          0.9756773399014779,
          0.9786535303776683,
          0.979064039408867
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial87",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9634646962233169,
          0.9682881773399015,
          0.9817323481116584,
          0.9910714285714286,
          0.9890188834154351,
          0.993431855500821,
          0.9940476190476191,
          0.9942528735632183,
          0.993431855500821,
          0.9932266009852216,
          0.9919950738916257
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial88",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9797052154195012,
          0.9870748299319728,
          0.9918367346938776,
          0.992517006802721,
          0.9929705215419501,
          0.992517006802721,
          0.9934240362811791,
          0.9929705215419501,
          0.9927437641723356,
          0.9920634920634921,
          0.9947845804988662
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial89",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9327655759748992,
          0.9438592559390407,
          0.9398251904975348,
          0.9406095921111609,
          0.9468848050201704,
          0.9536082474226805,
          0.9463245181532944,
          0.9418422232182878,
          0.9514791573285523,
          0.9530479605558045,
          0.9685118780815778
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial90",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9681647940074907,
          0.9694132334581773,
          0.9860590928006658,
          0.9893882646691635,
          0.992093216812318,
          0.992093216812318,
          0.9933416562630045,
          0.9942779858510196,
          0.9945900957136912,
          0.9937578027465668,
          0.9947981689554724
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial91",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9278530377668309,
          0.9363711001642037,
          0.950636288998358,
          0.9499178981937603,
          0.9493021346469622,
          0.9499178981937603,
          0.9516625615763546,
          0.9510467980295566,
          0.9623357963875205,
          0.9627463054187192,
          0.964696223316913
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial92",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9581572273879966,
          0.9658706677937448,
          0.9683009298393914,
          0.9749577345731192,
          0.9737954353338969,
          0.970625528317836,
          0.9766483516483516,
          0.9802409129332206,
          0.9838334742180896,
          0.985207100591716,
          0.9859467455621301
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial93",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9621593291404612,
          0.9582809224318658,
          0.9858490566037735,
          0.9861635220125786,
          0.9886792452830189,
          0.9897274633123689,
          0.9890985324947589,
          0.9880503144654088,
          0.9874213836477987,
          0.9876310272536688,
          0.9886792452830189
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial94",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9784090909090909,
          0.9852272727272727,
          0.984607438016529,
          0.9849173553719008,
          0.9889462809917355,
          0.9886363636363636,
          0.9829545454545454,
          0.9933884297520661,
          0.9940082644628099,
          0.993595041322314,
          0.9946280991735538
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial95",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.8545771756978654,
          0.9349343185550082,
          0.944068144499179,
          0.9529967159277504,
          0.9623357963875205,
          0.9631568144499179,
          0.9760878489326765,
          0.9760878489326765,
          0.9789614121510674,
          0.9829638752052545,
          0.9848111658456487
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial96",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21
         ],
         "y": [
          0.9693877551020408,
          0.9859965262700825,
          0.9888189318280504,
          0.9905557967867998,
          0.990447242726878,
          0.9915327833260964,
          0.9977203647416414,
          0.9975032566217976,
          0.9964177160225792,
          0.997286148501954,
          0.9976118106817194,
          0.9971775944420321,
          0.996526270082501,
          0.9967433782023448,
          0.9969604863221885,
          0.9971775944420321,
          0.9973947025618758,
          0.9971775944420321,
          0.9973947025618758,
          0.9976118106817194,
          0.9982631350412505,
          0.9976118106817194
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial97",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9546703296703297,
          0.9646027049873204,
          0.9714708368554522,
          0.9762256973795436,
          0.977916314454776,
          0.9804522400676247,
          0.9810862214708369,
          0.9803465765004227,
          0.9811918850380389,
          0.9807692307692307,
          0.9826711749788672
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial98",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9726544082979727,
          0.9799622819424799,
          0.9829090051862329,
          0.9818481848184818,
          0.9837340876944838,
          0.9846770391324847,
          0.9836162187647336,
          0.9833804809052333,
          0.9839698255539839,
          0.9856199905704857,
          0.9851485148514851
         ]
        },
        {
         "marker": {
          "maxdisplayed": 10
         },
         "mode": "lines+markers",
         "name": "Trial99",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10
         ],
         "y": [
          0.9492324561403509,
          0.9611842105263158,
          0.9598684210526316,
          0.95625,
          0.9747807017543859,
          0.9787280701754386,
          0.9848684210526316,
          0.9857456140350878,
          0.9875,
          0.9899122807017544,
          0.9890350877192983
         ]
        }
       ],
       "layout": {
        "showlegend": false,
        "template": {
         "data": {
          "bar": [
           {
            "error_x": {
             "color": "#2a3f5f"
            },
            "error_y": {
             "color": "#2a3f5f"
            },
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "bar"
           }
          ],
          "barpolar": [
           {
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "barpolar"
           }
          ],
          "carpet": [
           {
            "aaxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "baxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "type": "carpet"
           }
          ],
          "choropleth": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "choropleth"
           }
          ],
          "contour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "contour"
           }
          ],
          "contourcarpet": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "contourcarpet"
           }
          ],
          "heatmap": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmap"
           }
          ],
          "heatmapgl": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmapgl"
           }
          ],
          "histogram": [
           {
            "marker": {
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "histogram"
           }
          ],
          "histogram2d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2d"
           }
          ],
          "histogram2dcontour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2dcontour"
           }
          ],
          "mesh3d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "mesh3d"
           }
          ],
          "parcoords": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "parcoords"
           }
          ],
          "pie": [
           {
            "automargin": true,
            "type": "pie"
           }
          ],
          "scatter": [
           {
            "fillpattern": {
             "fillmode": "overlay",
             "size": 10,
             "solidity": 0.2
            },
            "type": "scatter"
           }
          ],
          "scatter3d": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatter3d"
           }
          ],
          "scattercarpet": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattercarpet"
           }
          ],
          "scattergeo": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergeo"
           }
          ],
          "scattergl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergl"
           }
          ],
          "scattermapbox": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattermapbox"
           }
          ],
          "scatterpolar": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolar"
           }
          ],
          "scatterpolargl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolargl"
           }
          ],
          "scatterternary": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterternary"
           }
          ],
          "surface": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "surface"
           }
          ],
          "table": [
           {
            "cells": {
             "fill": {
              "color": "#EBF0F8"
             },
             "line": {
              "color": "white"
             }
            },
            "header": {
             "fill": {
              "color": "#C8D4E3"
             },
             "line": {
              "color": "white"
             }
            },
            "type": "table"
           }
          ]
         },
         "layout": {
          "annotationdefaults": {
           "arrowcolor": "#2a3f5f",
           "arrowhead": 0,
           "arrowwidth": 1
          },
          "autotypenumbers": "strict",
          "coloraxis": {
           "colorbar": {
            "outlinewidth": 0,
            "ticks": ""
           }
          },
          "colorscale": {
           "diverging": [
            [
             0,
             "#8e0152"
            ],
            [
             0.1,
             "#c51b7d"
            ],
            [
             0.2,
             "#de77ae"
            ],
            [
             0.3,
             "#f1b6da"
            ],
            [
             0.4,
             "#fde0ef"
            ],
            [
             0.5,
             "#f7f7f7"
            ],
            [
             0.6,
             "#e6f5d0"
            ],
            [
             0.7,
             "#b8e186"
            ],
            [
             0.8,
             "#7fbc41"
            ],
            [
             0.9,
             "#4d9221"
            ],
            [
             1,
             "#276419"
            ]
           ],
           "sequential": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ],
           "sequentialminus": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ]
          },
          "colorway": [
           "#636efa",
           "#EF553B",
           "#00cc96",
           "#ab63fa",
           "#FFA15A",
           "#19d3f3",
           "#FF6692",
           "#B6E880",
           "#FF97FF",
           "#FECB52"
          ],
          "font": {
           "color": "#2a3f5f"
          },
          "geo": {
           "bgcolor": "white",
           "lakecolor": "white",
           "landcolor": "#E5ECF6",
           "showlakes": true,
           "showland": true,
           "subunitcolor": "white"
          },
          "hoverlabel": {
           "align": "left"
          },
          "hovermode": "closest",
          "mapbox": {
           "style": "light"
          },
          "paper_bgcolor": "white",
          "plot_bgcolor": "#E5ECF6",
          "polar": {
           "angularaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "radialaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "scene": {
           "xaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "yaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "zaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           }
          },
          "shapedefaults": {
           "line": {
            "color": "#2a3f5f"
           }
          },
          "ternary": {
           "aaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "baxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "caxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "title": {
           "x": 0.05
          },
          "xaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          },
          "yaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          }
         }
        },
        "title": {
         "text": "Intermediate Values Plot"
        },
        "xaxis": {
         "title": {
          "text": "Step"
         }
        },
        "yaxis": {
         "title": {
          "text": "Intermediate Value"
         }
        }
       }
      }
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_intermediate_values(study)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Para visualizar relaciones de parámetros de alta dimensión: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "config": {
        "plotlyServerURL": "https://plot.ly"
       },
       "data": [
        {
         "dimensions": [
          {
           "label": "Objective Value",
           "range": [
            0.958041958041958,
            0.993006993006993
           ],
           "values": [
            0.972027972027972,
            0.972027972027972,
            0.958041958041958,
            0.9790209790209791,
            0.958041958041958,
            0.993006993006993,
            0.986013986013986,
            0.986013986013986,
            0.9790209790209791,
            0.986013986013986,
            0.986013986013986,
            0.9790209790209791
           ]
          },
          {
           "label": "bagging_fraction",
           "range": [
            0.4348501673009197,
            0.9905077333938093
           ],
           "values": [
            0.6247240713084175,
            0.759195090518222,
            0.4348501673009197,
            0.8248435466776274,
            0.899465584480253,
            0.5100427059120604,
            0.9733907188988407,
            0.9905077333938093,
            0.9539760625736883,
            0.8823409487656906,
            0.973380947872035,
            0.818345732780162
           ]
          },
          {
           "label": "bagging_freq",
           "range": [
            1,
            7
           ],
           "values": [
            7,
            2,
            7,
            1,
            2,
            3,
            1,
            2,
            1,
            1,
            2,
            3
           ]
          },
          {
           "label": "min_child_samples",
           "range": [
            5,
            98
           ],
           "values": [
            75,
            19,
            62,
            98,
            22,
            55,
            98,
            54,
            5,
            48,
            39,
            22
           ]
          }
         ],
         "labelangle": 30,
         "labelside": "bottom",
         "line": {
          "color": [
           0.972027972027972,
           0.972027972027972,
           0.958041958041958,
           0.9790209790209791,
           0.958041958041958,
           0.993006993006993,
           0.986013986013986,
           0.986013986013986,
           0.9790209790209791,
           0.986013986013986,
           0.986013986013986,
           0.9790209790209791
          ],
          "colorbar": {
           "title": {
            "text": "Objective Value"
           }
          },
          "colorscale": [
           [
            0,
            "rgb(247,251,255)"
           ],
           [
            0.125,
            "rgb(222,235,247)"
           ],
           [
            0.25,
            "rgb(198,219,239)"
           ],
           [
            0.375,
            "rgb(158,202,225)"
           ],
           [
            0.5,
            "rgb(107,174,214)"
           ],
           [
            0.625,
            "rgb(66,146,198)"
           ],
           [
            0.75,
            "rgb(33,113,181)"
           ],
           [
            0.875,
            "rgb(8,81,156)"
           ],
           [
            1,
            "rgb(8,48,107)"
           ]
          ],
          "reversescale": false,
          "showscale": true
         },
         "type": "parcoords"
        }
       ],
       "layout": {
        "template": {
         "data": {
          "bar": [
           {
            "error_x": {
             "color": "#2a3f5f"
            },
            "error_y": {
             "color": "#2a3f5f"
            },
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "bar"
           }
          ],
          "barpolar": [
           {
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "barpolar"
           }
          ],
          "carpet": [
           {
            "aaxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "baxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "type": "carpet"
           }
          ],
          "choropleth": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "choropleth"
           }
          ],
          "contour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "contour"
           }
          ],
          "contourcarpet": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "contourcarpet"
           }
          ],
          "heatmap": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmap"
           }
          ],
          "heatmapgl": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmapgl"
           }
          ],
          "histogram": [
           {
            "marker": {
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "histogram"
           }
          ],
          "histogram2d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2d"
           }
          ],
          "histogram2dcontour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2dcontour"
           }
          ],
          "mesh3d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "mesh3d"
           }
          ],
          "parcoords": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "parcoords"
           }
          ],
          "pie": [
           {
            "automargin": true,
            "type": "pie"
           }
          ],
          "scatter": [
           {
            "fillpattern": {
             "fillmode": "overlay",
             "size": 10,
             "solidity": 0.2
            },
            "type": "scatter"
           }
          ],
          "scatter3d": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatter3d"
           }
          ],
          "scattercarpet": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattercarpet"
           }
          ],
          "scattergeo": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergeo"
           }
          ],
          "scattergl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergl"
           }
          ],
          "scattermapbox": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattermapbox"
           }
          ],
          "scatterpolar": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolar"
           }
          ],
          "scatterpolargl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolargl"
           }
          ],
          "scatterternary": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterternary"
           }
          ],
          "surface": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "surface"
           }
          ],
          "table": [
           {
            "cells": {
             "fill": {
              "color": "#EBF0F8"
             },
             "line": {
              "color": "white"
             }
            },
            "header": {
             "fill": {
              "color": "#C8D4E3"
             },
             "line": {
              "color": "white"
             }
            },
            "type": "table"
           }
          ]
         },
         "layout": {
          "annotationdefaults": {
           "arrowcolor": "#2a3f5f",
           "arrowhead": 0,
           "arrowwidth": 1
          },
          "autotypenumbers": "strict",
          "coloraxis": {
           "colorbar": {
            "outlinewidth": 0,
            "ticks": ""
           }
          },
          "colorscale": {
           "diverging": [
            [
             0,
             "#8e0152"
            ],
            [
             0.1,
             "#c51b7d"
            ],
            [
             0.2,
             "#de77ae"
            ],
            [
             0.3,
             "#f1b6da"
            ],
            [
             0.4,
             "#fde0ef"
            ],
            [
             0.5,
             "#f7f7f7"
            ],
            [
             0.6,
             "#e6f5d0"
            ],
            [
             0.7,
             "#b8e186"
            ],
            [
             0.8,
             "#7fbc41"
            ],
            [
             0.9,
             "#4d9221"
            ],
            [
             1,
             "#276419"
            ]
           ],
           "sequential": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ],
           "sequentialminus": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ]
          },
          "colorway": [
           "#636efa",
           "#EF553B",
           "#00cc96",
           "#ab63fa",
           "#FFA15A",
           "#19d3f3",
           "#FF6692",
           "#B6E880",
           "#FF97FF",
           "#FECB52"
          ],
          "font": {
           "color": "#2a3f5f"
          },
          "geo": {
           "bgcolor": "white",
           "lakecolor": "white",
           "landcolor": "#E5ECF6",
           "showlakes": true,
           "showland": true,
           "subunitcolor": "white"
          },
          "hoverlabel": {
           "align": "left"
          },
          "hovermode": "closest",
          "mapbox": {
           "style": "light"
          },
          "paper_bgcolor": "white",
          "plot_bgcolor": "#E5ECF6",
          "polar": {
           "angularaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "radialaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "scene": {
           "xaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "yaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "zaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           }
          },
          "shapedefaults": {
           "line": {
            "color": "#2a3f5f"
           }
          },
          "ternary": {
           "aaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "baxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "caxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "title": {
           "x": 0.05
          },
          "xaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          },
          "yaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          }
         }
        },
        "title": {
         "text": "Parallel Coordinate Plot"
        }
       }
      }
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_parallel_coordinate(study)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "O si queremos seleccionar sólo algunos parámetros específicos para visualizar:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "config": {
        "plotlyServerURL": "https://plot.ly"
       },
       "data": [
        {
         "dimensions": [
          {
           "label": "Objective Value",
           "range": [
            0.958041958041958,
            0.993006993006993
           ],
           "values": [
            0.972027972027972,
            0.972027972027972,
            0.958041958041958,
            0.9790209790209791,
            0.958041958041958,
            0.993006993006993,
            0.986013986013986,
            0.986013986013986,
            0.9790209790209791,
            0.986013986013986,
            0.986013986013986,
            0.9790209790209791
           ]
          },
          {
           "label": "bagging_fraction",
           "range": [
            0.4348501673009197,
            0.9905077333938093
           ],
           "values": [
            0.6247240713084175,
            0.759195090518222,
            0.4348501673009197,
            0.8248435466776274,
            0.899465584480253,
            0.5100427059120604,
            0.9733907188988407,
            0.9905077333938093,
            0.9539760625736883,
            0.8823409487656906,
            0.973380947872035,
            0.818345732780162
           ]
          },
          {
           "label": "bagging_freq",
           "range": [
            1,
            7
           ],
           "values": [
            7,
            2,
            7,
            1,
            2,
            3,
            1,
            2,
            1,
            1,
            2,
            3
           ]
          }
         ],
         "labelangle": 30,
         "labelside": "bottom",
         "line": {
          "color": [
           0.972027972027972,
           0.972027972027972,
           0.958041958041958,
           0.9790209790209791,
           0.958041958041958,
           0.993006993006993,
           0.986013986013986,
           0.986013986013986,
           0.9790209790209791,
           0.986013986013986,
           0.986013986013986,
           0.9790209790209791
          ],
          "colorbar": {
           "title": {
            "text": "Objective Value"
           }
          },
          "colorscale": [
           [
            0,
            "rgb(247,251,255)"
           ],
           [
            0.125,
            "rgb(222,235,247)"
           ],
           [
            0.25,
            "rgb(198,219,239)"
           ],
           [
            0.375,
            "rgb(158,202,225)"
           ],
           [
            0.5,
            "rgb(107,174,214)"
           ],
           [
            0.625,
            "rgb(66,146,198)"
           ],
           [
            0.75,
            "rgb(33,113,181)"
           ],
           [
            0.875,
            "rgb(8,81,156)"
           ],
           [
            1,
            "rgb(8,48,107)"
           ]
          ],
          "reversescale": false,
          "showscale": true
         },
         "type": "parcoords"
        }
       ],
       "layout": {
        "template": {
         "data": {
          "bar": [
           {
            "error_x": {
             "color": "#2a3f5f"
            },
            "error_y": {
             "color": "#2a3f5f"
            },
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "bar"
           }
          ],
          "barpolar": [
           {
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "barpolar"
           }
          ],
          "carpet": [
           {
            "aaxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "baxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "type": "carpet"
           }
          ],
          "choropleth": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "choropleth"
           }
          ],
          "contour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "contour"
           }
          ],
          "contourcarpet": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "contourcarpet"
           }
          ],
          "heatmap": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmap"
           }
          ],
          "heatmapgl": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmapgl"
           }
          ],
          "histogram": [
           {
            "marker": {
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "histogram"
           }
          ],
          "histogram2d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2d"
           }
          ],
          "histogram2dcontour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2dcontour"
           }
          ],
          "mesh3d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "mesh3d"
           }
          ],
          "parcoords": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "parcoords"
           }
          ],
          "pie": [
           {
            "automargin": true,
            "type": "pie"
           }
          ],
          "scatter": [
           {
            "fillpattern": {
             "fillmode": "overlay",
             "size": 10,
             "solidity": 0.2
            },
            "type": "scatter"
           }
          ],
          "scatter3d": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatter3d"
           }
          ],
          "scattercarpet": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattercarpet"
           }
          ],
          "scattergeo": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergeo"
           }
          ],
          "scattergl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergl"
           }
          ],
          "scattermapbox": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattermapbox"
           }
          ],
          "scatterpolar": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolar"
           }
          ],
          "scatterpolargl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolargl"
           }
          ],
          "scatterternary": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterternary"
           }
          ],
          "surface": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "surface"
           }
          ],
          "table": [
           {
            "cells": {
             "fill": {
              "color": "#EBF0F8"
             },
             "line": {
              "color": "white"
             }
            },
            "header": {
             "fill": {
              "color": "#C8D4E3"
             },
             "line": {
              "color": "white"
             }
            },
            "type": "table"
           }
          ]
         },
         "layout": {
          "annotationdefaults": {
           "arrowcolor": "#2a3f5f",
           "arrowhead": 0,
           "arrowwidth": 1
          },
          "autotypenumbers": "strict",
          "coloraxis": {
           "colorbar": {
            "outlinewidth": 0,
            "ticks": ""
           }
          },
          "colorscale": {
           "diverging": [
            [
             0,
             "#8e0152"
            ],
            [
             0.1,
             "#c51b7d"
            ],
            [
             0.2,
             "#de77ae"
            ],
            [
             0.3,
             "#f1b6da"
            ],
            [
             0.4,
             "#fde0ef"
            ],
            [
             0.5,
             "#f7f7f7"
            ],
            [
             0.6,
             "#e6f5d0"
            ],
            [
             0.7,
             "#b8e186"
            ],
            [
             0.8,
             "#7fbc41"
            ],
            [
             0.9,
             "#4d9221"
            ],
            [
             1,
             "#276419"
            ]
           ],
           "sequential": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ],
           "sequentialminus": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ]
          },
          "colorway": [
           "#636efa",
           "#EF553B",
           "#00cc96",
           "#ab63fa",
           "#FFA15A",
           "#19d3f3",
           "#FF6692",
           "#B6E880",
           "#FF97FF",
           "#FECB52"
          ],
          "font": {
           "color": "#2a3f5f"
          },
          "geo": {
           "bgcolor": "white",
           "lakecolor": "white",
           "landcolor": "#E5ECF6",
           "showlakes": true,
           "showland": true,
           "subunitcolor": "white"
          },
          "hoverlabel": {
           "align": "left"
          },
          "hovermode": "closest",
          "mapbox": {
           "style": "light"
          },
          "paper_bgcolor": "white",
          "plot_bgcolor": "#E5ECF6",
          "polar": {
           "angularaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "radialaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "scene": {
           "xaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "yaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "zaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           }
          },
          "shapedefaults": {
           "line": {
            "color": "#2a3f5f"
           }
          },
          "ternary": {
           "aaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "baxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "caxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "title": {
           "x": 0.05
          },
          "xaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          },
          "yaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          }
         }
        },
        "title": {
         "text": "Parallel Coordinate Plot"
        }
       }
      }
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_parallel_coordinate(study, params = [\"bagging_freq\", \"bagging_fraction\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Para visualizar relaciones entre los hiperparámetros:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "config": {
        "plotlyServerURL": "https://plot.ly"
       },
       "data": [
        {
         "type": "scatter",
         "xaxis": "x",
         "yaxis": "y"
        },
        {
         "colorbar": {
          "title": {
           "text": "Objective Value"
          }
         },
         "colorscale": [
          [
           0,
           "rgb(247,251,255)"
          ],
          [
           0.125,
           "rgb(222,235,247)"
          ],
          [
           0.25,
           "rgb(198,219,239)"
          ],
          [
           0.375,
           "rgb(158,202,225)"
          ],
          [
           0.5,
           "rgb(107,174,214)"
          ],
          [
           0.625,
           "rgb(66,146,198)"
          ],
          [
           0.75,
           "rgb(33,113,181)"
          ],
          [
           0.875,
           "rgb(8,81,156)"
          ],
          [
           1,
           "rgb(8,48,107)"
          ]
         ],
         "connectgaps": true,
         "contours": {
          "coloring": "heatmap"
         },
         "hoverinfo": "none",
         "line": {
          "smoothing": 1.3
         },
         "reversescale": false,
         "showscale": true,
         "type": "contour",
         "x": [
          0.4070672889962752,
          0.4348501673009197,
          0.5100427059120604,
          0.6247240713084175,
          0.759195090518222,
          0.818345732780162,
          0.8248435466776274,
          0.8823409487656906,
          0.899465584480253,
          0.9539760625736883,
          0.973380947872035,
          0.9733907188988407,
          0.9905077333938093,
          1.0182906116984538
         ],
         "xaxis": "x4",
         "y": [
          0.7,
          1,
          2,
          3,
          7,
          7.3
         ],
         "yaxis": "y4",
         "z": [
          [
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null
          ],
          [
           null,
           null,
           null,
           null,
           null,
           null,
           0.9790209790209791,
           0.986013986013986,
           null,
           0.9790209790209791,
           null,
           0.986013986013986,
           null,
           null
          ],
          [
           null,
           null,
           null,
           null,
           0.972027972027972,
           null,
           null,
           null,
           0.958041958041958,
           null,
           0.986013986013986,
           null,
           0.986013986013986,
           null
          ],
          [
           null,
           null,
           0.993006993006993,
           null,
           null,
           0.9790209790209791,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null
          ],
          [
           null,
           0.958041958041958,
           null,
           0.972027972027972,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null
          ],
          [
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null
          ]
         ]
        },
        {
         "marker": {
          "color": "black",
          "line": {
           "color": "Grey",
           "width": 2
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          0.6247240713084175,
          0.759195090518222,
          0.4348501673009197,
          0.8248435466776274,
          0.899465584480253,
          0.5100427059120604,
          0.9733907188988407,
          0.9905077333938093,
          0.9539760625736883,
          0.8823409487656906,
          0.973380947872035,
          0.818345732780162
         ],
         "xaxis": "x4",
         "y": [
          7,
          2,
          7,
          1,
          2,
          3,
          1,
          2,
          1,
          1,
          2,
          3
         ],
         "yaxis": "y4"
        },
        {
         "colorbar": {
          "title": {
           "text": "Objective Value"
          }
         },
         "colorscale": [
          [
           0,
           "rgb(247,251,255)"
          ],
          [
           0.125,
           "rgb(222,235,247)"
          ],
          [
           0.25,
           "rgb(198,219,239)"
          ],
          [
           0.375,
           "rgb(158,202,225)"
          ],
          [
           0.5,
           "rgb(107,174,214)"
          ],
          [
           0.625,
           "rgb(66,146,198)"
          ],
          [
           0.75,
           "rgb(33,113,181)"
          ],
          [
           0.875,
           "rgb(8,81,156)"
          ],
          [
           1,
           "rgb(8,48,107)"
          ]
         ],
         "connectgaps": true,
         "contours": {
          "coloring": "heatmap"
         },
         "hoverinfo": "none",
         "line": {
          "smoothing": 1.3
         },
         "reversescale": false,
         "showscale": false,
         "type": "contour",
         "x": [
          0.4070672889962752,
          0.4348501673009197,
          0.5100427059120604,
          0.6247240713084175,
          0.759195090518222,
          0.818345732780162,
          0.8248435466776274,
          0.8823409487656906,
          0.899465584480253,
          0.9539760625736883,
          0.973380947872035,
          0.9733907188988407,
          0.9905077333938093,
          1.0182906116984538
         ],
         "xaxis": "x7",
         "y": [
          0.34999999999999964,
          5,
          19,
          22,
          39,
          48,
          54,
          55,
          62,
          75,
          98,
          102.65
         ],
         "yaxis": "y7",
         "z": [
          [
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null
          ],
          [
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           0.9790209790209791,
           null,
           null,
           null,
           null
          ],
          [
           null,
           null,
           null,
           null,
           0.972027972027972,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null
          ],
          [
           null,
           null,
           null,
           null,
           null,
           0.9790209790209791,
           null,
           null,
           0.958041958041958,
           null,
           null,
           null,
           null,
           null
          ],
          [
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           0.986013986013986,
           null,
           null,
           null
          ],
          [
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           0.986013986013986,
           null,
           null,
           null,
           null,
           null,
           null
          ],
          [
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           0.986013986013986,
           null
          ],
          [
           null,
           null,
           0.993006993006993,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null
          ],
          [
           null,
           0.958041958041958,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null
          ],
          [
           null,
           null,
           null,
           0.972027972027972,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null
          ],
          [
           null,
           null,
           null,
           null,
           null,
           null,
           0.9790209790209791,
           null,
           null,
           null,
           null,
           0.986013986013986,
           null,
           null
          ],
          [
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null
          ]
         ]
        },
        {
         "marker": {
          "color": "black",
          "line": {
           "color": "Grey",
           "width": 2
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          0.6247240713084175,
          0.759195090518222,
          0.4348501673009197,
          0.8248435466776274,
          0.899465584480253,
          0.5100427059120604,
          0.9733907188988407,
          0.9905077333938093,
          0.9539760625736883,
          0.8823409487656906,
          0.973380947872035,
          0.818345732780162
         ],
         "xaxis": "x7",
         "y": [
          75,
          19,
          62,
          98,
          22,
          55,
          98,
          54,
          5,
          48,
          39,
          22
         ],
         "yaxis": "y7"
        },
        {
         "colorbar": {
          "title": {
           "text": "Objective Value"
          }
         },
         "colorscale": [
          [
           0,
           "rgb(247,251,255)"
          ],
          [
           0.125,
           "rgb(222,235,247)"
          ],
          [
           0.25,
           "rgb(198,219,239)"
          ],
          [
           0.375,
           "rgb(158,202,225)"
          ],
          [
           0.5,
           "rgb(107,174,214)"
          ],
          [
           0.625,
           "rgb(66,146,198)"
          ],
          [
           0.75,
           "rgb(33,113,181)"
          ],
          [
           0.875,
           "rgb(8,81,156)"
          ],
          [
           1,
           "rgb(8,48,107)"
          ]
         ],
         "connectgaps": true,
         "contours": {
          "coloring": "heatmap"
         },
         "hoverinfo": "none",
         "line": {
          "smoothing": 1.3
         },
         "reversescale": false,
         "showscale": false,
         "type": "contour",
         "x": [
          0.7,
          1,
          2,
          3,
          7,
          7.3
         ],
         "xaxis": "x2",
         "y": [
          0.4070672889962752,
          0.4348501673009197,
          0.5100427059120604,
          0.6247240713084175,
          0.759195090518222,
          0.818345732780162,
          0.8248435466776274,
          0.8823409487656906,
          0.899465584480253,
          0.9539760625736883,
          0.973380947872035,
          0.9733907188988407,
          0.9905077333938093,
          1.0182906116984538
         ],
         "yaxis": "y2",
         "z": [
          [
           null,
           null,
           null,
           null,
           null,
           null
          ],
          [
           null,
           null,
           null,
           null,
           0.958041958041958,
           null
          ],
          [
           null,
           null,
           null,
           0.993006993006993,
           null,
           null
          ],
          [
           null,
           null,
           null,
           null,
           0.972027972027972,
           null
          ],
          [
           null,
           null,
           0.972027972027972,
           null,
           null,
           null
          ],
          [
           null,
           null,
           null,
           0.9790209790209791,
           null,
           null
          ],
          [
           null,
           0.9790209790209791,
           null,
           null,
           null,
           null
          ],
          [
           null,
           0.986013986013986,
           null,
           null,
           null,
           null
          ],
          [
           null,
           null,
           0.958041958041958,
           null,
           null,
           null
          ],
          [
           null,
           0.9790209790209791,
           null,
           null,
           null,
           null
          ],
          [
           null,
           null,
           0.986013986013986,
           null,
           null,
           null
          ],
          [
           null,
           0.986013986013986,
           null,
           null,
           null,
           null
          ],
          [
           null,
           null,
           0.986013986013986,
           null,
           null,
           null
          ],
          [
           null,
           null,
           null,
           null,
           null,
           null
          ]
         ]
        },
        {
         "marker": {
          "color": "black",
          "line": {
           "color": "Grey",
           "width": 2
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          7,
          2,
          7,
          1,
          2,
          3,
          1,
          2,
          1,
          1,
          2,
          3
         ],
         "xaxis": "x2",
         "y": [
          0.6247240713084175,
          0.759195090518222,
          0.4348501673009197,
          0.8248435466776274,
          0.899465584480253,
          0.5100427059120604,
          0.9733907188988407,
          0.9905077333938093,
          0.9539760625736883,
          0.8823409487656906,
          0.973380947872035,
          0.818345732780162
         ],
         "yaxis": "y2"
        },
        {
         "type": "scatter",
         "xaxis": "x5",
         "yaxis": "y5"
        },
        {
         "colorbar": {
          "title": {
           "text": "Objective Value"
          }
         },
         "colorscale": [
          [
           0,
           "rgb(247,251,255)"
          ],
          [
           0.125,
           "rgb(222,235,247)"
          ],
          [
           0.25,
           "rgb(198,219,239)"
          ],
          [
           0.375,
           "rgb(158,202,225)"
          ],
          [
           0.5,
           "rgb(107,174,214)"
          ],
          [
           0.625,
           "rgb(66,146,198)"
          ],
          [
           0.75,
           "rgb(33,113,181)"
          ],
          [
           0.875,
           "rgb(8,81,156)"
          ],
          [
           1,
           "rgb(8,48,107)"
          ]
         ],
         "connectgaps": true,
         "contours": {
          "coloring": "heatmap"
         },
         "hoverinfo": "none",
         "line": {
          "smoothing": 1.3
         },
         "reversescale": false,
         "showscale": false,
         "type": "contour",
         "x": [
          0.7,
          1,
          2,
          3,
          7,
          7.3
         ],
         "xaxis": "x8",
         "y": [
          0.34999999999999964,
          5,
          19,
          22,
          39,
          48,
          54,
          55,
          62,
          75,
          98,
          102.65
         ],
         "yaxis": "y8",
         "z": [
          [
           null,
           null,
           null,
           null,
           null,
           null
          ],
          [
           null,
           0.9790209790209791,
           null,
           null,
           null,
           null
          ],
          [
           null,
           null,
           0.972027972027972,
           null,
           null,
           null
          ],
          [
           null,
           null,
           0.958041958041958,
           0.9790209790209791,
           null,
           null
          ],
          [
           null,
           null,
           0.986013986013986,
           null,
           null,
           null
          ],
          [
           null,
           0.986013986013986,
           null,
           null,
           null,
           null
          ],
          [
           null,
           null,
           0.986013986013986,
           null,
           null,
           null
          ],
          [
           null,
           null,
           null,
           0.993006993006993,
           null,
           null
          ],
          [
           null,
           null,
           null,
           null,
           0.958041958041958,
           null
          ],
          [
           null,
           null,
           null,
           null,
           0.972027972027972,
           null
          ],
          [
           null,
           0.986013986013986,
           null,
           null,
           null,
           null
          ],
          [
           null,
           null,
           null,
           null,
           null,
           null
          ]
         ]
        },
        {
         "marker": {
          "color": "black",
          "line": {
           "color": "Grey",
           "width": 2
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          7,
          2,
          7,
          1,
          2,
          3,
          1,
          2,
          1,
          1,
          2,
          3
         ],
         "xaxis": "x8",
         "y": [
          75,
          19,
          62,
          98,
          22,
          55,
          98,
          54,
          5,
          48,
          39,
          22
         ],
         "yaxis": "y8"
        },
        {
         "colorbar": {
          "title": {
           "text": "Objective Value"
          }
         },
         "colorscale": [
          [
           0,
           "rgb(247,251,255)"
          ],
          [
           0.125,
           "rgb(222,235,247)"
          ],
          [
           0.25,
           "rgb(198,219,239)"
          ],
          [
           0.375,
           "rgb(158,202,225)"
          ],
          [
           0.5,
           "rgb(107,174,214)"
          ],
          [
           0.625,
           "rgb(66,146,198)"
          ],
          [
           0.75,
           "rgb(33,113,181)"
          ],
          [
           0.875,
           "rgb(8,81,156)"
          ],
          [
           1,
           "rgb(8,48,107)"
          ]
         ],
         "connectgaps": true,
         "contours": {
          "coloring": "heatmap"
         },
         "hoverinfo": "none",
         "line": {
          "smoothing": 1.3
         },
         "reversescale": false,
         "showscale": false,
         "type": "contour",
         "x": [
          0.34999999999999964,
          5,
          19,
          22,
          39,
          48,
          54,
          55,
          62,
          75,
          98,
          102.65
         ],
         "xaxis": "x3",
         "y": [
          0.4070672889962752,
          0.4348501673009197,
          0.5100427059120604,
          0.6247240713084175,
          0.759195090518222,
          0.818345732780162,
          0.8248435466776274,
          0.8823409487656906,
          0.899465584480253,
          0.9539760625736883,
          0.973380947872035,
          0.9733907188988407,
          0.9905077333938093,
          1.0182906116984538
         ],
         "yaxis": "y3",
         "z": [
          [
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null
          ],
          [
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           0.958041958041958,
           null,
           null,
           null
          ],
          [
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           0.993006993006993,
           null,
           null,
           null,
           null
          ],
          [
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           0.972027972027972,
           null,
           null
          ],
          [
           null,
           null,
           0.972027972027972,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null
          ],
          [
           null,
           null,
           null,
           0.9790209790209791,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null
          ],
          [
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           0.9790209790209791,
           null
          ],
          [
           null,
           null,
           null,
           null,
           null,
           0.986013986013986,
           null,
           null,
           null,
           null,
           null,
           null
          ],
          [
           null,
           null,
           null,
           0.958041958041958,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null
          ],
          [
           null,
           0.9790209790209791,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null
          ],
          [
           null,
           null,
           null,
           null,
           0.986013986013986,
           null,
           null,
           null,
           null,
           null,
           null,
           null
          ],
          [
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           0.986013986013986,
           null
          ],
          [
           null,
           null,
           null,
           null,
           null,
           null,
           0.986013986013986,
           null,
           null,
           null,
           null,
           null
          ],
          [
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null
          ]
         ]
        },
        {
         "marker": {
          "color": "black",
          "line": {
           "color": "Grey",
           "width": 2
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          75,
          19,
          62,
          98,
          22,
          55,
          98,
          54,
          5,
          48,
          39,
          22
         ],
         "xaxis": "x3",
         "y": [
          0.6247240713084175,
          0.759195090518222,
          0.4348501673009197,
          0.8248435466776274,
          0.899465584480253,
          0.5100427059120604,
          0.9733907188988407,
          0.9905077333938093,
          0.9539760625736883,
          0.8823409487656906,
          0.973380947872035,
          0.818345732780162
         ],
         "yaxis": "y3"
        },
        {
         "colorbar": {
          "title": {
           "text": "Objective Value"
          }
         },
         "colorscale": [
          [
           0,
           "rgb(247,251,255)"
          ],
          [
           0.125,
           "rgb(222,235,247)"
          ],
          [
           0.25,
           "rgb(198,219,239)"
          ],
          [
           0.375,
           "rgb(158,202,225)"
          ],
          [
           0.5,
           "rgb(107,174,214)"
          ],
          [
           0.625,
           "rgb(66,146,198)"
          ],
          [
           0.75,
           "rgb(33,113,181)"
          ],
          [
           0.875,
           "rgb(8,81,156)"
          ],
          [
           1,
           "rgb(8,48,107)"
          ]
         ],
         "connectgaps": true,
         "contours": {
          "coloring": "heatmap"
         },
         "hoverinfo": "none",
         "line": {
          "smoothing": 1.3
         },
         "reversescale": false,
         "showscale": false,
         "type": "contour",
         "x": [
          0.34999999999999964,
          5,
          19,
          22,
          39,
          48,
          54,
          55,
          62,
          75,
          98,
          102.65
         ],
         "xaxis": "x6",
         "y": [
          0.7,
          1,
          2,
          3,
          7,
          7.3
         ],
         "yaxis": "y6",
         "z": [
          [
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null
          ],
          [
           null,
           0.9790209790209791,
           null,
           null,
           null,
           0.986013986013986,
           null,
           null,
           null,
           null,
           0.986013986013986,
           null
          ],
          [
           null,
           null,
           0.972027972027972,
           0.958041958041958,
           0.986013986013986,
           null,
           0.986013986013986,
           null,
           null,
           null,
           null,
           null
          ],
          [
           null,
           null,
           null,
           0.9790209790209791,
           null,
           null,
           null,
           0.993006993006993,
           null,
           null,
           null,
           null
          ],
          [
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           0.958041958041958,
           0.972027972027972,
           null,
           null
          ],
          [
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null
          ]
         ]
        },
        {
         "marker": {
          "color": "black",
          "line": {
           "color": "Grey",
           "width": 2
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          75,
          19,
          62,
          98,
          22,
          55,
          98,
          54,
          5,
          48,
          39,
          22
         ],
         "xaxis": "x6",
         "y": [
          7,
          2,
          7,
          1,
          2,
          3,
          1,
          2,
          1,
          1,
          2,
          3
         ],
         "yaxis": "y6"
        },
        {
         "type": "scatter",
         "xaxis": "x9",
         "yaxis": "y9"
        }
       ],
       "layout": {
        "template": {
         "data": {
          "bar": [
           {
            "error_x": {
             "color": "#2a3f5f"
            },
            "error_y": {
             "color": "#2a3f5f"
            },
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "bar"
           }
          ],
          "barpolar": [
           {
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "barpolar"
           }
          ],
          "carpet": [
           {
            "aaxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "baxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "type": "carpet"
           }
          ],
          "choropleth": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "choropleth"
           }
          ],
          "contour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "contour"
           }
          ],
          "contourcarpet": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "contourcarpet"
           }
          ],
          "heatmap": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmap"
           }
          ],
          "heatmapgl": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmapgl"
           }
          ],
          "histogram": [
           {
            "marker": {
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "histogram"
           }
          ],
          "histogram2d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2d"
           }
          ],
          "histogram2dcontour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2dcontour"
           }
          ],
          "mesh3d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "mesh3d"
           }
          ],
          "parcoords": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "parcoords"
           }
          ],
          "pie": [
           {
            "automargin": true,
            "type": "pie"
           }
          ],
          "scatter": [
           {
            "fillpattern": {
             "fillmode": "overlay",
             "size": 10,
             "solidity": 0.2
            },
            "type": "scatter"
           }
          ],
          "scatter3d": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatter3d"
           }
          ],
          "scattercarpet": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattercarpet"
           }
          ],
          "scattergeo": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergeo"
           }
          ],
          "scattergl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergl"
           }
          ],
          "scattermapbox": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattermapbox"
           }
          ],
          "scatterpolar": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolar"
           }
          ],
          "scatterpolargl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolargl"
           }
          ],
          "scatterternary": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterternary"
           }
          ],
          "surface": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "surface"
           }
          ],
          "table": [
           {
            "cells": {
             "fill": {
              "color": "#EBF0F8"
             },
             "line": {
              "color": "white"
             }
            },
            "header": {
             "fill": {
              "color": "#C8D4E3"
             },
             "line": {
              "color": "white"
             }
            },
            "type": "table"
           }
          ]
         },
         "layout": {
          "annotationdefaults": {
           "arrowcolor": "#2a3f5f",
           "arrowhead": 0,
           "arrowwidth": 1
          },
          "autotypenumbers": "strict",
          "coloraxis": {
           "colorbar": {
            "outlinewidth": 0,
            "ticks": ""
           }
          },
          "colorscale": {
           "diverging": [
            [
             0,
             "#8e0152"
            ],
            [
             0.1,
             "#c51b7d"
            ],
            [
             0.2,
             "#de77ae"
            ],
            [
             0.3,
             "#f1b6da"
            ],
            [
             0.4,
             "#fde0ef"
            ],
            [
             0.5,
             "#f7f7f7"
            ],
            [
             0.6,
             "#e6f5d0"
            ],
            [
             0.7,
             "#b8e186"
            ],
            [
             0.8,
             "#7fbc41"
            ],
            [
             0.9,
             "#4d9221"
            ],
            [
             1,
             "#276419"
            ]
           ],
           "sequential": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ],
           "sequentialminus": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ]
          },
          "colorway": [
           "#636efa",
           "#EF553B",
           "#00cc96",
           "#ab63fa",
           "#FFA15A",
           "#19d3f3",
           "#FF6692",
           "#B6E880",
           "#FF97FF",
           "#FECB52"
          ],
          "font": {
           "color": "#2a3f5f"
          },
          "geo": {
           "bgcolor": "white",
           "lakecolor": "white",
           "landcolor": "#E5ECF6",
           "showlakes": true,
           "showland": true,
           "subunitcolor": "white"
          },
          "hoverlabel": {
           "align": "left"
          },
          "hovermode": "closest",
          "mapbox": {
           "style": "light"
          },
          "paper_bgcolor": "white",
          "plot_bgcolor": "#E5ECF6",
          "polar": {
           "angularaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "radialaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "scene": {
           "xaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "yaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "zaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           }
          },
          "shapedefaults": {
           "line": {
            "color": "#2a3f5f"
           }
          },
          "ternary": {
           "aaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "baxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "caxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "title": {
           "x": 0.05
          },
          "xaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          },
          "yaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          }
         }
        },
        "title": {
         "text": "Contour Plot"
        },
        "xaxis": {
         "anchor": "y",
         "domain": [
          0,
          0.2888888888888889
         ],
         "matches": "x7",
         "range": [
          0.4070672889962752,
          1.0182906116984538
         ],
         "showticklabels": false
        },
        "xaxis2": {
         "anchor": "y2",
         "domain": [
          0.35555555555555557,
          0.6444444444444445
         ],
         "matches": "x8",
         "range": [
          0.7,
          7.3
         ],
         "showticklabels": false
        },
        "xaxis3": {
         "anchor": "y3",
         "domain": [
          0.7111111111111111,
          1
         ],
         "matches": "x9",
         "range": [
          0.34999999999999964,
          102.65
         ],
         "showticklabels": false
        },
        "xaxis4": {
         "anchor": "y4",
         "domain": [
          0,
          0.2888888888888889
         ],
         "matches": "x7",
         "range": [
          0.4070672889962752,
          1.0182906116984538
         ],
         "showticklabels": false
        },
        "xaxis5": {
         "anchor": "y5",
         "domain": [
          0.35555555555555557,
          0.6444444444444445
         ],
         "matches": "x8",
         "range": [
          0.7,
          7.3
         ],
         "showticklabels": false
        },
        "xaxis6": {
         "anchor": "y6",
         "domain": [
          0.7111111111111111,
          1
         ],
         "matches": "x9",
         "range": [
          0.34999999999999964,
          102.65
         ],
         "showticklabels": false
        },
        "xaxis7": {
         "anchor": "y7",
         "domain": [
          0,
          0.2888888888888889
         ],
         "range": [
          0.4070672889962752,
          1.0182906116984538
         ],
         "title": {
          "text": "bagging_fraction"
         }
        },
        "xaxis8": {
         "anchor": "y8",
         "domain": [
          0.35555555555555557,
          0.6444444444444445
         ],
         "range": [
          0.7,
          7.3
         ],
         "title": {
          "text": "bagging_freq"
         }
        },
        "xaxis9": {
         "anchor": "y9",
         "domain": [
          0.7111111111111111,
          1
         ],
         "range": [
          0.34999999999999964,
          102.65
         ],
         "title": {
          "text": "min_child_samples"
         }
        },
        "yaxis": {
         "anchor": "x",
         "domain": [
          0.7333333333333333,
          1
         ],
         "range": [
          0.4070672889962752,
          1.0182906116984538
         ],
         "title": {
          "text": "bagging_fraction"
         }
        },
        "yaxis2": {
         "anchor": "x2",
         "domain": [
          0.7333333333333333,
          1
         ],
         "matches": "y",
         "range": [
          0.4070672889962752,
          1.0182906116984538
         ],
         "showticklabels": false
        },
        "yaxis3": {
         "anchor": "x3",
         "domain": [
          0.7333333333333333,
          1
         ],
         "matches": "y",
         "range": [
          0.4070672889962752,
          1.0182906116984538
         ],
         "showticklabels": false
        },
        "yaxis4": {
         "anchor": "x4",
         "domain": [
          0.36666666666666664,
          0.6333333333333333
         ],
         "range": [
          0.7,
          7.3
         ],
         "title": {
          "text": "bagging_freq"
         }
        },
        "yaxis5": {
         "anchor": "x5",
         "domain": [
          0.36666666666666664,
          0.6333333333333333
         ],
         "matches": "y4",
         "range": [
          0.7,
          7.3
         ],
         "showticklabels": false
        },
        "yaxis6": {
         "anchor": "x6",
         "domain": [
          0.36666666666666664,
          0.6333333333333333
         ],
         "matches": "y4",
         "range": [
          0.7,
          7.3
         ],
         "showticklabels": false
        },
        "yaxis7": {
         "anchor": "x7",
         "domain": [
          0,
          0.26666666666666666
         ],
         "range": [
          0.34999999999999964,
          102.65
         ],
         "title": {
          "text": "min_child_samples"
         }
        },
        "yaxis8": {
         "anchor": "x8",
         "domain": [
          0,
          0.26666666666666666
         ],
         "matches": "y7",
         "range": [
          0.34999999999999964,
          102.65
         ],
         "showticklabels": false
        },
        "yaxis9": {
         "anchor": "x9",
         "domain": [
          0,
          0.26666666666666666
         ],
         "matches": "y7",
         "range": [
          0.34999999999999964,
          102.65
         ],
         "showticklabels": false
        }
       }
      }
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_contour(study)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "config": {
        "plotlyServerURL": "https://plot.ly"
       },
       "data": [
        {
         "colorbar": {
          "title": {
           "text": "Objective Value"
          }
         },
         "colorscale": [
          [
           0,
           "rgb(247,251,255)"
          ],
          [
           0.125,
           "rgb(222,235,247)"
          ],
          [
           0.25,
           "rgb(198,219,239)"
          ],
          [
           0.375,
           "rgb(158,202,225)"
          ],
          [
           0.5,
           "rgb(107,174,214)"
          ],
          [
           0.625,
           "rgb(66,146,198)"
          ],
          [
           0.75,
           "rgb(33,113,181)"
          ],
          [
           0.875,
           "rgb(8,81,156)"
          ],
          [
           1,
           "rgb(8,48,107)"
          ]
         ],
         "connectgaps": true,
         "contours": {
          "coloring": "heatmap"
         },
         "hoverinfo": "none",
         "line": {
          "smoothing": 1.3
         },
         "reversescale": false,
         "type": "contour",
         "x": [
          0.4070672889962752,
          0.4348501673009197,
          0.5100427059120604,
          0.6247240713084175,
          0.759195090518222,
          0.818345732780162,
          0.8248435466776274,
          0.8823409487656906,
          0.899465584480253,
          0.9539760625736883,
          0.973380947872035,
          0.9733907188988407,
          0.9905077333938093,
          1.0182906116984538
         ],
         "y": [
          0.7,
          1,
          2,
          3,
          7,
          7.3
         ],
         "z": [
          [
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null
          ],
          [
           null,
           null,
           null,
           null,
           null,
           null,
           0.9790209790209791,
           0.986013986013986,
           null,
           0.9790209790209791,
           null,
           0.986013986013986,
           null,
           null
          ],
          [
           null,
           null,
           null,
           null,
           0.972027972027972,
           null,
           null,
           null,
           0.958041958041958,
           null,
           0.986013986013986,
           null,
           0.986013986013986,
           null
          ],
          [
           null,
           null,
           0.993006993006993,
           null,
           null,
           0.9790209790209791,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null
          ],
          [
           null,
           0.958041958041958,
           null,
           0.972027972027972,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null
          ],
          [
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null,
           null
          ]
         ]
        },
        {
         "marker": {
          "color": "black",
          "line": {
           "color": "Grey",
           "width": 2
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          0.6247240713084175,
          0.759195090518222,
          0.4348501673009197,
          0.8248435466776274,
          0.899465584480253,
          0.5100427059120604,
          0.9733907188988407,
          0.9905077333938093,
          0.9539760625736883,
          0.8823409487656906,
          0.973380947872035,
          0.818345732780162
         ],
         "y": [
          7,
          2,
          7,
          1,
          2,
          3,
          1,
          2,
          1,
          1,
          2,
          3
         ]
        }
       ],
       "layout": {
        "template": {
         "data": {
          "bar": [
           {
            "error_x": {
             "color": "#2a3f5f"
            },
            "error_y": {
             "color": "#2a3f5f"
            },
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "bar"
           }
          ],
          "barpolar": [
           {
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "barpolar"
           }
          ],
          "carpet": [
           {
            "aaxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "baxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "type": "carpet"
           }
          ],
          "choropleth": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "choropleth"
           }
          ],
          "contour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "contour"
           }
          ],
          "contourcarpet": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "contourcarpet"
           }
          ],
          "heatmap": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmap"
           }
          ],
          "heatmapgl": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmapgl"
           }
          ],
          "histogram": [
           {
            "marker": {
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "histogram"
           }
          ],
          "histogram2d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2d"
           }
          ],
          "histogram2dcontour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2dcontour"
           }
          ],
          "mesh3d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "mesh3d"
           }
          ],
          "parcoords": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "parcoords"
           }
          ],
          "pie": [
           {
            "automargin": true,
            "type": "pie"
           }
          ],
          "scatter": [
           {
            "fillpattern": {
             "fillmode": "overlay",
             "size": 10,
             "solidity": 0.2
            },
            "type": "scatter"
           }
          ],
          "scatter3d": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatter3d"
           }
          ],
          "scattercarpet": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattercarpet"
           }
          ],
          "scattergeo": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergeo"
           }
          ],
          "scattergl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergl"
           }
          ],
          "scattermapbox": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattermapbox"
           }
          ],
          "scatterpolar": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolar"
           }
          ],
          "scatterpolargl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolargl"
           }
          ],
          "scatterternary": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterternary"
           }
          ],
          "surface": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "surface"
           }
          ],
          "table": [
           {
            "cells": {
             "fill": {
              "color": "#EBF0F8"
             },
             "line": {
              "color": "white"
             }
            },
            "header": {
             "fill": {
              "color": "#C8D4E3"
             },
             "line": {
              "color": "white"
             }
            },
            "type": "table"
           }
          ]
         },
         "layout": {
          "annotationdefaults": {
           "arrowcolor": "#2a3f5f",
           "arrowhead": 0,
           "arrowwidth": 1
          },
          "autotypenumbers": "strict",
          "coloraxis": {
           "colorbar": {
            "outlinewidth": 0,
            "ticks": ""
           }
          },
          "colorscale": {
           "diverging": [
            [
             0,
             "#8e0152"
            ],
            [
             0.1,
             "#c51b7d"
            ],
            [
             0.2,
             "#de77ae"
            ],
            [
             0.3,
             "#f1b6da"
            ],
            [
             0.4,
             "#fde0ef"
            ],
            [
             0.5,
             "#f7f7f7"
            ],
            [
             0.6,
             "#e6f5d0"
            ],
            [
             0.7,
             "#b8e186"
            ],
            [
             0.8,
             "#7fbc41"
            ],
            [
             0.9,
             "#4d9221"
            ],
            [
             1,
             "#276419"
            ]
           ],
           "sequential": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ],
           "sequentialminus": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ]
          },
          "colorway": [
           "#636efa",
           "#EF553B",
           "#00cc96",
           "#ab63fa",
           "#FFA15A",
           "#19d3f3",
           "#FF6692",
           "#B6E880",
           "#FF97FF",
           "#FECB52"
          ],
          "font": {
           "color": "#2a3f5f"
          },
          "geo": {
           "bgcolor": "white",
           "lakecolor": "white",
           "landcolor": "#E5ECF6",
           "showlakes": true,
           "showland": true,
           "subunitcolor": "white"
          },
          "hoverlabel": {
           "align": "left"
          },
          "hovermode": "closest",
          "mapbox": {
           "style": "light"
          },
          "paper_bgcolor": "white",
          "plot_bgcolor": "#E5ECF6",
          "polar": {
           "angularaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "radialaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "scene": {
           "xaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "yaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "zaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           }
          },
          "shapedefaults": {
           "line": {
            "color": "#2a3f5f"
           }
          },
          "ternary": {
           "aaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "baxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "caxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "title": {
           "x": 0.05
          },
          "xaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          },
          "yaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          }
         }
        },
        "title": {
         "text": "Contour Plot"
        },
        "xaxis": {
         "range": [
          0.4070672889962752,
          1.0182906116984538
         ],
         "title": {
          "text": "bagging_fraction"
         }
        },
        "yaxis": {
         "range": [
          0.7,
          7.3
         ],
         "title": {
          "text": "bagging_freq"
         }
        }
       }
      }
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_contour(study, params = [\"bagging_freq\", \"bagging_fraction\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Visualizar hiperparámetros individuales como gráfico de slice:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "config": {
        "plotlyServerURL": "https://plot.ly"
       },
       "data": [
        {
         "marker": {
          "color": [
           0,
           1,
           2,
           3,
           4,
           5,
           12,
           18,
           25,
           44,
           55,
           82
          ],
          "colorbar": {
           "title": {
            "text": "Trial"
           },
           "x": 1,
           "xpad": 40
          },
          "colorscale": [
           [
            0,
            "rgb(247,251,255)"
           ],
           [
            0.125,
            "rgb(222,235,247)"
           ],
           [
            0.25,
            "rgb(198,219,239)"
           ],
           [
            0.375,
            "rgb(158,202,225)"
           ],
           [
            0.5,
            "rgb(107,174,214)"
           ],
           [
            0.625,
            "rgb(66,146,198)"
           ],
           [
            0.75,
            "rgb(33,113,181)"
           ],
           [
            0.875,
            "rgb(8,81,156)"
           ],
           [
            1,
            "rgb(8,48,107)"
           ]
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          },
          "showscale": true
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          0.6247240713084175,
          0.759195090518222,
          0.4348501673009197,
          0.8248435466776274,
          0.899465584480253,
          0.5100427059120604,
          0.9733907188988407,
          0.9905077333938093,
          0.9539760625736883,
          0.8823409487656906,
          0.973380947872035,
          0.818345732780162
         ],
         "xaxis": "x",
         "y": [
          0.972027972027972,
          0.972027972027972,
          0.958041958041958,
          0.9790209790209791,
          0.958041958041958,
          0.993006993006993,
          0.986013986013986,
          0.986013986013986,
          0.9790209790209791,
          0.986013986013986,
          0.986013986013986,
          0.9790209790209791
         ],
         "yaxis": "y"
        },
        {
         "marker": {
          "color": [
           0,
           1,
           2,
           3,
           4,
           5,
           12,
           18,
           25,
           44,
           55,
           82
          ],
          "colorbar": {
           "title": {
            "text": "Trial"
           },
           "x": 1,
           "xpad": 40
          },
          "colorscale": [
           [
            0,
            "rgb(247,251,255)"
           ],
           [
            0.125,
            "rgb(222,235,247)"
           ],
           [
            0.25,
            "rgb(198,219,239)"
           ],
           [
            0.375,
            "rgb(158,202,225)"
           ],
           [
            0.5,
            "rgb(107,174,214)"
           ],
           [
            0.625,
            "rgb(66,146,198)"
           ],
           [
            0.75,
            "rgb(33,113,181)"
           ],
           [
            0.875,
            "rgb(8,81,156)"
           ],
           [
            1,
            "rgb(8,48,107)"
           ]
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          },
          "showscale": false
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          7,
          2,
          7,
          1,
          2,
          3,
          1,
          2,
          1,
          1,
          2,
          3
         ],
         "xaxis": "x2",
         "y": [
          0.972027972027972,
          0.972027972027972,
          0.958041958041958,
          0.9790209790209791,
          0.958041958041958,
          0.993006993006993,
          0.986013986013986,
          0.986013986013986,
          0.9790209790209791,
          0.986013986013986,
          0.986013986013986,
          0.9790209790209791
         ],
         "yaxis": "y2"
        },
        {
         "marker": {
          "color": [
           0,
           1,
           2,
           3,
           4,
           5,
           12,
           18,
           25,
           44,
           55,
           82
          ],
          "colorbar": {
           "title": {
            "text": "Trial"
           },
           "x": 1,
           "xpad": 40
          },
          "colorscale": [
           [
            0,
            "rgb(247,251,255)"
           ],
           [
            0.125,
            "rgb(222,235,247)"
           ],
           [
            0.25,
            "rgb(198,219,239)"
           ],
           [
            0.375,
            "rgb(158,202,225)"
           ],
           [
            0.5,
            "rgb(107,174,214)"
           ],
           [
            0.625,
            "rgb(66,146,198)"
           ],
           [
            0.75,
            "rgb(33,113,181)"
           ],
           [
            0.875,
            "rgb(8,81,156)"
           ],
           [
            1,
            "rgb(8,48,107)"
           ]
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          },
          "showscale": false
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          75,
          19,
          62,
          98,
          22,
          55,
          98,
          54,
          5,
          48,
          39,
          22
         ],
         "xaxis": "x3",
         "y": [
          0.972027972027972,
          0.972027972027972,
          0.958041958041958,
          0.9790209790209791,
          0.958041958041958,
          0.993006993006993,
          0.986013986013986,
          0.986013986013986,
          0.9790209790209791,
          0.986013986013986,
          0.986013986013986,
          0.9790209790209791
         ],
         "yaxis": "y3"
        }
       ],
       "layout": {
        "template": {
         "data": {
          "bar": [
           {
            "error_x": {
             "color": "#2a3f5f"
            },
            "error_y": {
             "color": "#2a3f5f"
            },
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "bar"
           }
          ],
          "barpolar": [
           {
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "barpolar"
           }
          ],
          "carpet": [
           {
            "aaxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "baxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "type": "carpet"
           }
          ],
          "choropleth": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "choropleth"
           }
          ],
          "contour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "contour"
           }
          ],
          "contourcarpet": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "contourcarpet"
           }
          ],
          "heatmap": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmap"
           }
          ],
          "heatmapgl": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmapgl"
           }
          ],
          "histogram": [
           {
            "marker": {
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "histogram"
           }
          ],
          "histogram2d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2d"
           }
          ],
          "histogram2dcontour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2dcontour"
           }
          ],
          "mesh3d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "mesh3d"
           }
          ],
          "parcoords": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "parcoords"
           }
          ],
          "pie": [
           {
            "automargin": true,
            "type": "pie"
           }
          ],
          "scatter": [
           {
            "fillpattern": {
             "fillmode": "overlay",
             "size": 10,
             "solidity": 0.2
            },
            "type": "scatter"
           }
          ],
          "scatter3d": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatter3d"
           }
          ],
          "scattercarpet": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattercarpet"
           }
          ],
          "scattergeo": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergeo"
           }
          ],
          "scattergl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergl"
           }
          ],
          "scattermapbox": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattermapbox"
           }
          ],
          "scatterpolar": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolar"
           }
          ],
          "scatterpolargl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolargl"
           }
          ],
          "scatterternary": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterternary"
           }
          ],
          "surface": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "surface"
           }
          ],
          "table": [
           {
            "cells": {
             "fill": {
              "color": "#EBF0F8"
             },
             "line": {
              "color": "white"
             }
            },
            "header": {
             "fill": {
              "color": "#C8D4E3"
             },
             "line": {
              "color": "white"
             }
            },
            "type": "table"
           }
          ]
         },
         "layout": {
          "annotationdefaults": {
           "arrowcolor": "#2a3f5f",
           "arrowhead": 0,
           "arrowwidth": 1
          },
          "autotypenumbers": "strict",
          "coloraxis": {
           "colorbar": {
            "outlinewidth": 0,
            "ticks": ""
           }
          },
          "colorscale": {
           "diverging": [
            [
             0,
             "#8e0152"
            ],
            [
             0.1,
             "#c51b7d"
            ],
            [
             0.2,
             "#de77ae"
            ],
            [
             0.3,
             "#f1b6da"
            ],
            [
             0.4,
             "#fde0ef"
            ],
            [
             0.5,
             "#f7f7f7"
            ],
            [
             0.6,
             "#e6f5d0"
            ],
            [
             0.7,
             "#b8e186"
            ],
            [
             0.8,
             "#7fbc41"
            ],
            [
             0.9,
             "#4d9221"
            ],
            [
             1,
             "#276419"
            ]
           ],
           "sequential": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ],
           "sequentialminus": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ]
          },
          "colorway": [
           "#636efa",
           "#EF553B",
           "#00cc96",
           "#ab63fa",
           "#FFA15A",
           "#19d3f3",
           "#FF6692",
           "#B6E880",
           "#FF97FF",
           "#FECB52"
          ],
          "font": {
           "color": "#2a3f5f"
          },
          "geo": {
           "bgcolor": "white",
           "lakecolor": "white",
           "landcolor": "#E5ECF6",
           "showlakes": true,
           "showland": true,
           "subunitcolor": "white"
          },
          "hoverlabel": {
           "align": "left"
          },
          "hovermode": "closest",
          "mapbox": {
           "style": "light"
          },
          "paper_bgcolor": "white",
          "plot_bgcolor": "#E5ECF6",
          "polar": {
           "angularaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "radialaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "scene": {
           "xaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "yaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "zaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           }
          },
          "shapedefaults": {
           "line": {
            "color": "#2a3f5f"
           }
          },
          "ternary": {
           "aaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "baxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "caxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "title": {
           "x": 0.05
          },
          "xaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          },
          "yaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          }
         }
        },
        "title": {
         "text": "Slice Plot"
        },
        "xaxis": {
         "anchor": "y",
         "domain": [
          0,
          0.2888888888888889
         ],
         "title": {
          "text": "bagging_fraction"
         }
        },
        "xaxis2": {
         "anchor": "y2",
         "domain": [
          0.35555555555555557,
          0.6444444444444445
         ],
         "title": {
          "text": "bagging_freq"
         }
        },
        "xaxis3": {
         "anchor": "y3",
         "domain": [
          0.7111111111111111,
          1
         ],
         "title": {
          "text": "min_child_samples"
         }
        },
        "yaxis": {
         "anchor": "x",
         "domain": [
          0,
          1
         ],
         "title": {
          "text": "Objective Value"
         }
        },
        "yaxis2": {
         "anchor": "x2",
         "domain": [
          0,
          1
         ],
         "matches": "y",
         "showticklabels": false
        },
        "yaxis3": {
         "anchor": "x3",
         "domain": [
          0,
          1
         ],
         "matches": "y",
         "showticklabels": false
        }
       }
      }
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_slice(study)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Para visualizar la importancia de los parámetros."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "config": {
        "plotlyServerURL": "https://plot.ly"
       },
       "data": [
        {
         "cliponaxis": false,
         "hovertemplate": [
          "bagging_freq (IntDistribution): 0.2573919908384002<extra></extra>",
          "min_child_samples (IntDistribution): 0.27920131238824286<extra></extra>",
          "bagging_fraction (FloatDistribution): 0.46340669677335694<extra></extra>"
         ],
         "marker": {
          "color": "rgb(66,146,198)"
         },
         "orientation": "h",
         "text": [
          "0.26",
          "0.28",
          "0.46"
         ],
         "textposition": "outside",
         "type": "bar",
         "x": [
          0.2573919908384002,
          0.27920131238824286,
          0.46340669677335694
         ],
         "y": [
          "bagging_freq",
          "min_child_samples",
          "bagging_fraction"
         ]
        }
       ],
       "layout": {
        "showlegend": false,
        "template": {
         "data": {
          "bar": [
           {
            "error_x": {
             "color": "#2a3f5f"
            },
            "error_y": {
             "color": "#2a3f5f"
            },
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "bar"
           }
          ],
          "barpolar": [
           {
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "barpolar"
           }
          ],
          "carpet": [
           {
            "aaxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "baxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "type": "carpet"
           }
          ],
          "choropleth": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "choropleth"
           }
          ],
          "contour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "contour"
           }
          ],
          "contourcarpet": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "contourcarpet"
           }
          ],
          "heatmap": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmap"
           }
          ],
          "heatmapgl": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmapgl"
           }
          ],
          "histogram": [
           {
            "marker": {
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "histogram"
           }
          ],
          "histogram2d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2d"
           }
          ],
          "histogram2dcontour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2dcontour"
           }
          ],
          "mesh3d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "mesh3d"
           }
          ],
          "parcoords": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "parcoords"
           }
          ],
          "pie": [
           {
            "automargin": true,
            "type": "pie"
           }
          ],
          "scatter": [
           {
            "fillpattern": {
             "fillmode": "overlay",
             "size": 10,
             "solidity": 0.2
            },
            "type": "scatter"
           }
          ],
          "scatter3d": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatter3d"
           }
          ],
          "scattercarpet": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattercarpet"
           }
          ],
          "scattergeo": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergeo"
           }
          ],
          "scattergl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergl"
           }
          ],
          "scattermapbox": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattermapbox"
           }
          ],
          "scatterpolar": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolar"
           }
          ],
          "scatterpolargl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolargl"
           }
          ],
          "scatterternary": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterternary"
           }
          ],
          "surface": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "surface"
           }
          ],
          "table": [
           {
            "cells": {
             "fill": {
              "color": "#EBF0F8"
             },
             "line": {
              "color": "white"
             }
            },
            "header": {
             "fill": {
              "color": "#C8D4E3"
             },
             "line": {
              "color": "white"
             }
            },
            "type": "table"
           }
          ]
         },
         "layout": {
          "annotationdefaults": {
           "arrowcolor": "#2a3f5f",
           "arrowhead": 0,
           "arrowwidth": 1
          },
          "autotypenumbers": "strict",
          "coloraxis": {
           "colorbar": {
            "outlinewidth": 0,
            "ticks": ""
           }
          },
          "colorscale": {
           "diverging": [
            [
             0,
             "#8e0152"
            ],
            [
             0.1,
             "#c51b7d"
            ],
            [
             0.2,
             "#de77ae"
            ],
            [
             0.3,
             "#f1b6da"
            ],
            [
             0.4,
             "#fde0ef"
            ],
            [
             0.5,
             "#f7f7f7"
            ],
            [
             0.6,
             "#e6f5d0"
            ],
            [
             0.7,
             "#b8e186"
            ],
            [
             0.8,
             "#7fbc41"
            ],
            [
             0.9,
             "#4d9221"
            ],
            [
             1,
             "#276419"
            ]
           ],
           "sequential": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ],
           "sequentialminus": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ]
          },
          "colorway": [
           "#636efa",
           "#EF553B",
           "#00cc96",
           "#ab63fa",
           "#FFA15A",
           "#19d3f3",
           "#FF6692",
           "#B6E880",
           "#FF97FF",
           "#FECB52"
          ],
          "font": {
           "color": "#2a3f5f"
          },
          "geo": {
           "bgcolor": "white",
           "lakecolor": "white",
           "landcolor": "#E5ECF6",
           "showlakes": true,
           "showland": true,
           "subunitcolor": "white"
          },
          "hoverlabel": {
           "align": "left"
          },
          "hovermode": "closest",
          "mapbox": {
           "style": "light"
          },
          "paper_bgcolor": "white",
          "plot_bgcolor": "#E5ECF6",
          "polar": {
           "angularaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "radialaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "scene": {
           "xaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "yaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "zaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           }
          },
          "shapedefaults": {
           "line": {
            "color": "#2a3f5f"
           }
          },
          "ternary": {
           "aaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "baxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "caxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "title": {
           "x": 0.05
          },
          "xaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          },
          "yaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          }
         }
        },
        "title": {
         "text": "Hyperparameter Importances"
        },
        "xaxis": {
         "title": {
          "text": "Importance for Objective Value"
         }
        },
        "yaxis": {
         "title": {
          "text": "Hyperparameter"
         }
        }
       }
      }
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_param_importances(study)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Si queremos aprender cuáles hiperparámetros están afectando la duración de la prueba con la importancia de los hiperparámetros:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "config": {
        "plotlyServerURL": "https://plot.ly"
       },
       "data": [
        {
         "cliponaxis": false,
         "hovertemplate": [
          "bagging_freq (IntDistribution): 0.11352435648252507<extra></extra>",
          "bagging_fraction (FloatDistribution): 0.24179897730835495<extra></extra>",
          "min_child_samples (IntDistribution): 0.64467666620912<extra></extra>"
         ],
         "marker": {
          "color": "rgb(66,146,198)"
         },
         "orientation": "h",
         "text": [
          "0.11",
          "0.24",
          "0.64"
         ],
         "textposition": "outside",
         "type": "bar",
         "x": [
          0.11352435648252507,
          0.24179897730835495,
          0.64467666620912
         ],
         "y": [
          "bagging_freq",
          "bagging_fraction",
          "min_child_samples"
         ]
        }
       ],
       "layout": {
        "showlegend": false,
        "template": {
         "data": {
          "bar": [
           {
            "error_x": {
             "color": "#2a3f5f"
            },
            "error_y": {
             "color": "#2a3f5f"
            },
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "bar"
           }
          ],
          "barpolar": [
           {
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "barpolar"
           }
          ],
          "carpet": [
           {
            "aaxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "baxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "type": "carpet"
           }
          ],
          "choropleth": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "choropleth"
           }
          ],
          "contour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "contour"
           }
          ],
          "contourcarpet": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "contourcarpet"
           }
          ],
          "heatmap": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmap"
           }
          ],
          "heatmapgl": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmapgl"
           }
          ],
          "histogram": [
           {
            "marker": {
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "histogram"
           }
          ],
          "histogram2d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2d"
           }
          ],
          "histogram2dcontour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2dcontour"
           }
          ],
          "mesh3d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "mesh3d"
           }
          ],
          "parcoords": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "parcoords"
           }
          ],
          "pie": [
           {
            "automargin": true,
            "type": "pie"
           }
          ],
          "scatter": [
           {
            "fillpattern": {
             "fillmode": "overlay",
             "size": 10,
             "solidity": 0.2
            },
            "type": "scatter"
           }
          ],
          "scatter3d": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatter3d"
           }
          ],
          "scattercarpet": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattercarpet"
           }
          ],
          "scattergeo": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergeo"
           }
          ],
          "scattergl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergl"
           }
          ],
          "scattermapbox": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattermapbox"
           }
          ],
          "scatterpolar": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolar"
           }
          ],
          "scatterpolargl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolargl"
           }
          ],
          "scatterternary": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterternary"
           }
          ],
          "surface": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "surface"
           }
          ],
          "table": [
           {
            "cells": {
             "fill": {
              "color": "#EBF0F8"
             },
             "line": {
              "color": "white"
             }
            },
            "header": {
             "fill": {
              "color": "#C8D4E3"
             },
             "line": {
              "color": "white"
             }
            },
            "type": "table"
           }
          ]
         },
         "layout": {
          "annotationdefaults": {
           "arrowcolor": "#2a3f5f",
           "arrowhead": 0,
           "arrowwidth": 1
          },
          "autotypenumbers": "strict",
          "coloraxis": {
           "colorbar": {
            "outlinewidth": 0,
            "ticks": ""
           }
          },
          "colorscale": {
           "diverging": [
            [
             0,
             "#8e0152"
            ],
            [
             0.1,
             "#c51b7d"
            ],
            [
             0.2,
             "#de77ae"
            ],
            [
             0.3,
             "#f1b6da"
            ],
            [
             0.4,
             "#fde0ef"
            ],
            [
             0.5,
             "#f7f7f7"
            ],
            [
             0.6,
             "#e6f5d0"
            ],
            [
             0.7,
             "#b8e186"
            ],
            [
             0.8,
             "#7fbc41"
            ],
            [
             0.9,
             "#4d9221"
            ],
            [
             1,
             "#276419"
            ]
           ],
           "sequential": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ],
           "sequentialminus": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ]
          },
          "colorway": [
           "#636efa",
           "#EF553B",
           "#00cc96",
           "#ab63fa",
           "#FFA15A",
           "#19d3f3",
           "#FF6692",
           "#B6E880",
           "#FF97FF",
           "#FECB52"
          ],
          "font": {
           "color": "#2a3f5f"
          },
          "geo": {
           "bgcolor": "white",
           "lakecolor": "white",
           "landcolor": "#E5ECF6",
           "showlakes": true,
           "showland": true,
           "subunitcolor": "white"
          },
          "hoverlabel": {
           "align": "left"
          },
          "hovermode": "closest",
          "mapbox": {
           "style": "light"
          },
          "paper_bgcolor": "white",
          "plot_bgcolor": "#E5ECF6",
          "polar": {
           "angularaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "radialaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "scene": {
           "xaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "yaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "zaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           }
          },
          "shapedefaults": {
           "line": {
            "color": "#2a3f5f"
           }
          },
          "ternary": {
           "aaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "baxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "caxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "title": {
           "x": 0.05
          },
          "xaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          },
          "yaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          }
         }
        },
        "title": {
         "text": "Hyperparameter Importances"
        },
        "xaxis": {
         "title": {
          "text": "Importance for duration"
         }
        },
        "yaxis": {
         "title": {
          "text": "Hyperparameter"
         }
        }
       }
      }
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "optuna.visualization.plot_param_importances(study, \n",
    "                                            target = lambda t: t.duration.total_seconds(),\n",
    "                                            target_name = \"duration\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Para visualizar las relaciones entre los parámetros con gráficos de dispersión coloreados por los valores objetivos:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\franz\\AppData\\Local\\Temp\\ipykernel_23396\\3430271029.py:1: ExperimentalWarning:\n",
      "\n",
      "plot_rank is experimental (supported from v3.2.0). The interface can change in the future.\n",
      "\n"
     ]
    },
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "config": {
        "plotlyServerURL": "https://plot.ly"
       },
       "data": [
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #0<br>bagging_fraction: 0.6247240713084175<br>bagging_freq: 7<br>min_child_samples: 75<br><b>Objective Value: 0.972027972027972</b>",
          "Trial #1<br>bagging_fraction: 0.759195090518222<br>bagging_freq: 2<br>min_child_samples: 19<br><b>Objective Value: 0.972027972027972</b>",
          "Trial #2<br>bagging_fraction: 0.4348501673009197<br>bagging_freq: 7<br>min_child_samples: 62<br><b>Objective Value: 0.958041958041958</b>",
          "Trial #3<br>bagging_fraction: 0.8248435466776274<br>bagging_freq: 1<br>min_child_samples: 98<br><b>Objective Value: 0.9790209790209791</b>",
          "Trial #4<br>bagging_fraction: 0.899465584480253<br>bagging_freq: 2<br>min_child_samples: 22<br><b>Objective Value: 0.958041958041958</b>",
          "Trial #5<br>bagging_fraction: 0.5100427059120604<br>bagging_freq: 3<br>min_child_samples: 55<br><b>Objective Value: 0.993006993006993</b>",
          "Trial #12<br>bagging_fraction: 0.9733907188988407<br>bagging_freq: 1<br>min_child_samples: 98<br><b>Objective Value: 0.986013986013986</b>",
          "Trial #18<br>bagging_fraction: 0.9905077333938093<br>bagging_freq: 2<br>min_child_samples: 54<br><b>Objective Value: 0.986013986013986</b>",
          "Trial #25<br>bagging_fraction: 0.9539760625736883<br>bagging_freq: 1<br>min_child_samples: 5<br><b>Objective Value: 0.9790209790209791</b>",
          "Trial #44<br>bagging_fraction: 0.8823409487656906<br>bagging_freq: 1<br>min_child_samples: 48<br><b>Objective Value: 0.986013986013986</b>",
          "Trial #55<br>bagging_fraction: 0.973380947872035<br>bagging_freq: 2<br>min_child_samples: 39<br><b>Objective Value: 0.986013986013986</b>",
          "Trial #82<br>bagging_fraction: 0.818345732780162<br>bagging_freq: 3<br>min_child_samples: 22<br><b>Objective Value: 0.9790209790209791</b>"
         ],
         "marker": {
          "color": [
           "rgb(131, 185, 216)",
           "rgb(131, 185, 216)",
           "rgb(58, 83, 163)",
           "rgb(241, 250, 217)",
           "rgb(58, 83, 163)",
           "rgb(165, 0, 38)",
           "rgb(246, 127, 75)",
           "rgb(246, 127, 75)",
           "rgb(241, 250, 217)",
           "rgb(246, 127, 75)",
           "rgb(246, 127, 75)",
           "rgb(241, 250, 217)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          0.6247240713084175,
          0.759195090518222,
          0.4348501673009197,
          0.8248435466776274,
          0.899465584480253,
          0.5100427059120604,
          0.9733907188988407,
          0.9905077333938093,
          0.9539760625736883,
          0.8823409487656906,
          0.973380947872035,
          0.818345732780162
         ],
         "xaxis": "x",
         "y": [
          0.6247240713084175,
          0.759195090518222,
          0.4348501673009197,
          0.8248435466776274,
          0.899465584480253,
          0.5100427059120604,
          0.9733907188988407,
          0.9905077333938093,
          0.9539760625736883,
          0.8823409487656906,
          0.973380947872035,
          0.818345732780162
         ],
         "yaxis": "y"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #0<br>bagging_fraction: 0.6247240713084175<br>bagging_freq: 7<br>min_child_samples: 75<br><b>Objective Value: 0.972027972027972</b>",
          "Trial #1<br>bagging_fraction: 0.759195090518222<br>bagging_freq: 2<br>min_child_samples: 19<br><b>Objective Value: 0.972027972027972</b>",
          "Trial #2<br>bagging_fraction: 0.4348501673009197<br>bagging_freq: 7<br>min_child_samples: 62<br><b>Objective Value: 0.958041958041958</b>",
          "Trial #3<br>bagging_fraction: 0.8248435466776274<br>bagging_freq: 1<br>min_child_samples: 98<br><b>Objective Value: 0.9790209790209791</b>",
          "Trial #4<br>bagging_fraction: 0.899465584480253<br>bagging_freq: 2<br>min_child_samples: 22<br><b>Objective Value: 0.958041958041958</b>",
          "Trial #5<br>bagging_fraction: 0.5100427059120604<br>bagging_freq: 3<br>min_child_samples: 55<br><b>Objective Value: 0.993006993006993</b>",
          "Trial #12<br>bagging_fraction: 0.9733907188988407<br>bagging_freq: 1<br>min_child_samples: 98<br><b>Objective Value: 0.986013986013986</b>",
          "Trial #18<br>bagging_fraction: 0.9905077333938093<br>bagging_freq: 2<br>min_child_samples: 54<br><b>Objective Value: 0.986013986013986</b>",
          "Trial #25<br>bagging_fraction: 0.9539760625736883<br>bagging_freq: 1<br>min_child_samples: 5<br><b>Objective Value: 0.9790209790209791</b>",
          "Trial #44<br>bagging_fraction: 0.8823409487656906<br>bagging_freq: 1<br>min_child_samples: 48<br><b>Objective Value: 0.986013986013986</b>",
          "Trial #55<br>bagging_fraction: 0.973380947872035<br>bagging_freq: 2<br>min_child_samples: 39<br><b>Objective Value: 0.986013986013986</b>",
          "Trial #82<br>bagging_fraction: 0.818345732780162<br>bagging_freq: 3<br>min_child_samples: 22<br><b>Objective Value: 0.9790209790209791</b>"
         ],
         "marker": {
          "color": [
           "rgb(131, 185, 216)",
           "rgb(131, 185, 216)",
           "rgb(58, 83, 163)",
           "rgb(241, 250, 217)",
           "rgb(58, 83, 163)",
           "rgb(165, 0, 38)",
           "rgb(246, 127, 75)",
           "rgb(246, 127, 75)",
           "rgb(241, 250, 217)",
           "rgb(246, 127, 75)",
           "rgb(246, 127, 75)",
           "rgb(241, 250, 217)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          0.6247240713084175,
          0.759195090518222,
          0.4348501673009197,
          0.8248435466776274,
          0.899465584480253,
          0.5100427059120604,
          0.9733907188988407,
          0.9905077333938093,
          0.9539760625736883,
          0.8823409487656906,
          0.973380947872035,
          0.818345732780162
         ],
         "xaxis": "x4",
         "y": [
          7,
          2,
          7,
          1,
          2,
          3,
          1,
          2,
          1,
          1,
          2,
          3
         ],
         "yaxis": "y4"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #0<br>bagging_fraction: 0.6247240713084175<br>bagging_freq: 7<br>min_child_samples: 75<br><b>Objective Value: 0.972027972027972</b>",
          "Trial #1<br>bagging_fraction: 0.759195090518222<br>bagging_freq: 2<br>min_child_samples: 19<br><b>Objective Value: 0.972027972027972</b>",
          "Trial #2<br>bagging_fraction: 0.4348501673009197<br>bagging_freq: 7<br>min_child_samples: 62<br><b>Objective Value: 0.958041958041958</b>",
          "Trial #3<br>bagging_fraction: 0.8248435466776274<br>bagging_freq: 1<br>min_child_samples: 98<br><b>Objective Value: 0.9790209790209791</b>",
          "Trial #4<br>bagging_fraction: 0.899465584480253<br>bagging_freq: 2<br>min_child_samples: 22<br><b>Objective Value: 0.958041958041958</b>",
          "Trial #5<br>bagging_fraction: 0.5100427059120604<br>bagging_freq: 3<br>min_child_samples: 55<br><b>Objective Value: 0.993006993006993</b>",
          "Trial #12<br>bagging_fraction: 0.9733907188988407<br>bagging_freq: 1<br>min_child_samples: 98<br><b>Objective Value: 0.986013986013986</b>",
          "Trial #18<br>bagging_fraction: 0.9905077333938093<br>bagging_freq: 2<br>min_child_samples: 54<br><b>Objective Value: 0.986013986013986</b>",
          "Trial #25<br>bagging_fraction: 0.9539760625736883<br>bagging_freq: 1<br>min_child_samples: 5<br><b>Objective Value: 0.9790209790209791</b>",
          "Trial #44<br>bagging_fraction: 0.8823409487656906<br>bagging_freq: 1<br>min_child_samples: 48<br><b>Objective Value: 0.986013986013986</b>",
          "Trial #55<br>bagging_fraction: 0.973380947872035<br>bagging_freq: 2<br>min_child_samples: 39<br><b>Objective Value: 0.986013986013986</b>",
          "Trial #82<br>bagging_fraction: 0.818345732780162<br>bagging_freq: 3<br>min_child_samples: 22<br><b>Objective Value: 0.9790209790209791</b>"
         ],
         "marker": {
          "color": [
           "rgb(131, 185, 216)",
           "rgb(131, 185, 216)",
           "rgb(58, 83, 163)",
           "rgb(241, 250, 217)",
           "rgb(58, 83, 163)",
           "rgb(165, 0, 38)",
           "rgb(246, 127, 75)",
           "rgb(246, 127, 75)",
           "rgb(241, 250, 217)",
           "rgb(246, 127, 75)",
           "rgb(246, 127, 75)",
           "rgb(241, 250, 217)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          0.6247240713084175,
          0.759195090518222,
          0.4348501673009197,
          0.8248435466776274,
          0.899465584480253,
          0.5100427059120604,
          0.9733907188988407,
          0.9905077333938093,
          0.9539760625736883,
          0.8823409487656906,
          0.973380947872035,
          0.818345732780162
         ],
         "xaxis": "x7",
         "y": [
          75,
          19,
          62,
          98,
          22,
          55,
          98,
          54,
          5,
          48,
          39,
          22
         ],
         "yaxis": "y7"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #0<br>bagging_fraction: 0.6247240713084175<br>bagging_freq: 7<br>min_child_samples: 75<br><b>Objective Value: 0.972027972027972</b>",
          "Trial #1<br>bagging_fraction: 0.759195090518222<br>bagging_freq: 2<br>min_child_samples: 19<br><b>Objective Value: 0.972027972027972</b>",
          "Trial #2<br>bagging_fraction: 0.4348501673009197<br>bagging_freq: 7<br>min_child_samples: 62<br><b>Objective Value: 0.958041958041958</b>",
          "Trial #3<br>bagging_fraction: 0.8248435466776274<br>bagging_freq: 1<br>min_child_samples: 98<br><b>Objective Value: 0.9790209790209791</b>",
          "Trial #4<br>bagging_fraction: 0.899465584480253<br>bagging_freq: 2<br>min_child_samples: 22<br><b>Objective Value: 0.958041958041958</b>",
          "Trial #5<br>bagging_fraction: 0.5100427059120604<br>bagging_freq: 3<br>min_child_samples: 55<br><b>Objective Value: 0.993006993006993</b>",
          "Trial #12<br>bagging_fraction: 0.9733907188988407<br>bagging_freq: 1<br>min_child_samples: 98<br><b>Objective Value: 0.986013986013986</b>",
          "Trial #18<br>bagging_fraction: 0.9905077333938093<br>bagging_freq: 2<br>min_child_samples: 54<br><b>Objective Value: 0.986013986013986</b>",
          "Trial #25<br>bagging_fraction: 0.9539760625736883<br>bagging_freq: 1<br>min_child_samples: 5<br><b>Objective Value: 0.9790209790209791</b>",
          "Trial #44<br>bagging_fraction: 0.8823409487656906<br>bagging_freq: 1<br>min_child_samples: 48<br><b>Objective Value: 0.986013986013986</b>",
          "Trial #55<br>bagging_fraction: 0.973380947872035<br>bagging_freq: 2<br>min_child_samples: 39<br><b>Objective Value: 0.986013986013986</b>",
          "Trial #82<br>bagging_fraction: 0.818345732780162<br>bagging_freq: 3<br>min_child_samples: 22<br><b>Objective Value: 0.9790209790209791</b>"
         ],
         "marker": {
          "color": [
           "rgb(131, 185, 216)",
           "rgb(131, 185, 216)",
           "rgb(58, 83, 163)",
           "rgb(241, 250, 217)",
           "rgb(58, 83, 163)",
           "rgb(165, 0, 38)",
           "rgb(246, 127, 75)",
           "rgb(246, 127, 75)",
           "rgb(241, 250, 217)",
           "rgb(246, 127, 75)",
           "rgb(246, 127, 75)",
           "rgb(241, 250, 217)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          7,
          2,
          7,
          1,
          2,
          3,
          1,
          2,
          1,
          1,
          2,
          3
         ],
         "xaxis": "x2",
         "y": [
          0.6247240713084175,
          0.759195090518222,
          0.4348501673009197,
          0.8248435466776274,
          0.899465584480253,
          0.5100427059120604,
          0.9733907188988407,
          0.9905077333938093,
          0.9539760625736883,
          0.8823409487656906,
          0.973380947872035,
          0.818345732780162
         ],
         "yaxis": "y2"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #0<br>bagging_fraction: 0.6247240713084175<br>bagging_freq: 7<br>min_child_samples: 75<br><b>Objective Value: 0.972027972027972</b>",
          "Trial #1<br>bagging_fraction: 0.759195090518222<br>bagging_freq: 2<br>min_child_samples: 19<br><b>Objective Value: 0.972027972027972</b>",
          "Trial #2<br>bagging_fraction: 0.4348501673009197<br>bagging_freq: 7<br>min_child_samples: 62<br><b>Objective Value: 0.958041958041958</b>",
          "Trial #3<br>bagging_fraction: 0.8248435466776274<br>bagging_freq: 1<br>min_child_samples: 98<br><b>Objective Value: 0.9790209790209791</b>",
          "Trial #4<br>bagging_fraction: 0.899465584480253<br>bagging_freq: 2<br>min_child_samples: 22<br><b>Objective Value: 0.958041958041958</b>",
          "Trial #5<br>bagging_fraction: 0.5100427059120604<br>bagging_freq: 3<br>min_child_samples: 55<br><b>Objective Value: 0.993006993006993</b>",
          "Trial #12<br>bagging_fraction: 0.9733907188988407<br>bagging_freq: 1<br>min_child_samples: 98<br><b>Objective Value: 0.986013986013986</b>",
          "Trial #18<br>bagging_fraction: 0.9905077333938093<br>bagging_freq: 2<br>min_child_samples: 54<br><b>Objective Value: 0.986013986013986</b>",
          "Trial #25<br>bagging_fraction: 0.9539760625736883<br>bagging_freq: 1<br>min_child_samples: 5<br><b>Objective Value: 0.9790209790209791</b>",
          "Trial #44<br>bagging_fraction: 0.8823409487656906<br>bagging_freq: 1<br>min_child_samples: 48<br><b>Objective Value: 0.986013986013986</b>",
          "Trial #55<br>bagging_fraction: 0.973380947872035<br>bagging_freq: 2<br>min_child_samples: 39<br><b>Objective Value: 0.986013986013986</b>",
          "Trial #82<br>bagging_fraction: 0.818345732780162<br>bagging_freq: 3<br>min_child_samples: 22<br><b>Objective Value: 0.9790209790209791</b>"
         ],
         "marker": {
          "color": [
           "rgb(131, 185, 216)",
           "rgb(131, 185, 216)",
           "rgb(58, 83, 163)",
           "rgb(241, 250, 217)",
           "rgb(58, 83, 163)",
           "rgb(165, 0, 38)",
           "rgb(246, 127, 75)",
           "rgb(246, 127, 75)",
           "rgb(241, 250, 217)",
           "rgb(246, 127, 75)",
           "rgb(246, 127, 75)",
           "rgb(241, 250, 217)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          7,
          2,
          7,
          1,
          2,
          3,
          1,
          2,
          1,
          1,
          2,
          3
         ],
         "xaxis": "x5",
         "y": [
          7,
          2,
          7,
          1,
          2,
          3,
          1,
          2,
          1,
          1,
          2,
          3
         ],
         "yaxis": "y5"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #0<br>bagging_fraction: 0.6247240713084175<br>bagging_freq: 7<br>min_child_samples: 75<br><b>Objective Value: 0.972027972027972</b>",
          "Trial #1<br>bagging_fraction: 0.759195090518222<br>bagging_freq: 2<br>min_child_samples: 19<br><b>Objective Value: 0.972027972027972</b>",
          "Trial #2<br>bagging_fraction: 0.4348501673009197<br>bagging_freq: 7<br>min_child_samples: 62<br><b>Objective Value: 0.958041958041958</b>",
          "Trial #3<br>bagging_fraction: 0.8248435466776274<br>bagging_freq: 1<br>min_child_samples: 98<br><b>Objective Value: 0.9790209790209791</b>",
          "Trial #4<br>bagging_fraction: 0.899465584480253<br>bagging_freq: 2<br>min_child_samples: 22<br><b>Objective Value: 0.958041958041958</b>",
          "Trial #5<br>bagging_fraction: 0.5100427059120604<br>bagging_freq: 3<br>min_child_samples: 55<br><b>Objective Value: 0.993006993006993</b>",
          "Trial #12<br>bagging_fraction: 0.9733907188988407<br>bagging_freq: 1<br>min_child_samples: 98<br><b>Objective Value: 0.986013986013986</b>",
          "Trial #18<br>bagging_fraction: 0.9905077333938093<br>bagging_freq: 2<br>min_child_samples: 54<br><b>Objective Value: 0.986013986013986</b>",
          "Trial #25<br>bagging_fraction: 0.9539760625736883<br>bagging_freq: 1<br>min_child_samples: 5<br><b>Objective Value: 0.9790209790209791</b>",
          "Trial #44<br>bagging_fraction: 0.8823409487656906<br>bagging_freq: 1<br>min_child_samples: 48<br><b>Objective Value: 0.986013986013986</b>",
          "Trial #55<br>bagging_fraction: 0.973380947872035<br>bagging_freq: 2<br>min_child_samples: 39<br><b>Objective Value: 0.986013986013986</b>",
          "Trial #82<br>bagging_fraction: 0.818345732780162<br>bagging_freq: 3<br>min_child_samples: 22<br><b>Objective Value: 0.9790209790209791</b>"
         ],
         "marker": {
          "color": [
           "rgb(131, 185, 216)",
           "rgb(131, 185, 216)",
           "rgb(58, 83, 163)",
           "rgb(241, 250, 217)",
           "rgb(58, 83, 163)",
           "rgb(165, 0, 38)",
           "rgb(246, 127, 75)",
           "rgb(246, 127, 75)",
           "rgb(241, 250, 217)",
           "rgb(246, 127, 75)",
           "rgb(246, 127, 75)",
           "rgb(241, 250, 217)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          7,
          2,
          7,
          1,
          2,
          3,
          1,
          2,
          1,
          1,
          2,
          3
         ],
         "xaxis": "x8",
         "y": [
          75,
          19,
          62,
          98,
          22,
          55,
          98,
          54,
          5,
          48,
          39,
          22
         ],
         "yaxis": "y8"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #0<br>bagging_fraction: 0.6247240713084175<br>bagging_freq: 7<br>min_child_samples: 75<br><b>Objective Value: 0.972027972027972</b>",
          "Trial #1<br>bagging_fraction: 0.759195090518222<br>bagging_freq: 2<br>min_child_samples: 19<br><b>Objective Value: 0.972027972027972</b>",
          "Trial #2<br>bagging_fraction: 0.4348501673009197<br>bagging_freq: 7<br>min_child_samples: 62<br><b>Objective Value: 0.958041958041958</b>",
          "Trial #3<br>bagging_fraction: 0.8248435466776274<br>bagging_freq: 1<br>min_child_samples: 98<br><b>Objective Value: 0.9790209790209791</b>",
          "Trial #4<br>bagging_fraction: 0.899465584480253<br>bagging_freq: 2<br>min_child_samples: 22<br><b>Objective Value: 0.958041958041958</b>",
          "Trial #5<br>bagging_fraction: 0.5100427059120604<br>bagging_freq: 3<br>min_child_samples: 55<br><b>Objective Value: 0.993006993006993</b>",
          "Trial #12<br>bagging_fraction: 0.9733907188988407<br>bagging_freq: 1<br>min_child_samples: 98<br><b>Objective Value: 0.986013986013986</b>",
          "Trial #18<br>bagging_fraction: 0.9905077333938093<br>bagging_freq: 2<br>min_child_samples: 54<br><b>Objective Value: 0.986013986013986</b>",
          "Trial #25<br>bagging_fraction: 0.9539760625736883<br>bagging_freq: 1<br>min_child_samples: 5<br><b>Objective Value: 0.9790209790209791</b>",
          "Trial #44<br>bagging_fraction: 0.8823409487656906<br>bagging_freq: 1<br>min_child_samples: 48<br><b>Objective Value: 0.986013986013986</b>",
          "Trial #55<br>bagging_fraction: 0.973380947872035<br>bagging_freq: 2<br>min_child_samples: 39<br><b>Objective Value: 0.986013986013986</b>",
          "Trial #82<br>bagging_fraction: 0.818345732780162<br>bagging_freq: 3<br>min_child_samples: 22<br><b>Objective Value: 0.9790209790209791</b>"
         ],
         "marker": {
          "color": [
           "rgb(131, 185, 216)",
           "rgb(131, 185, 216)",
           "rgb(58, 83, 163)",
           "rgb(241, 250, 217)",
           "rgb(58, 83, 163)",
           "rgb(165, 0, 38)",
           "rgb(246, 127, 75)",
           "rgb(246, 127, 75)",
           "rgb(241, 250, 217)",
           "rgb(246, 127, 75)",
           "rgb(246, 127, 75)",
           "rgb(241, 250, 217)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          75,
          19,
          62,
          98,
          22,
          55,
          98,
          54,
          5,
          48,
          39,
          22
         ],
         "xaxis": "x3",
         "y": [
          0.6247240713084175,
          0.759195090518222,
          0.4348501673009197,
          0.8248435466776274,
          0.899465584480253,
          0.5100427059120604,
          0.9733907188988407,
          0.9905077333938093,
          0.9539760625736883,
          0.8823409487656906,
          0.973380947872035,
          0.818345732780162
         ],
         "yaxis": "y3"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #0<br>bagging_fraction: 0.6247240713084175<br>bagging_freq: 7<br>min_child_samples: 75<br><b>Objective Value: 0.972027972027972</b>",
          "Trial #1<br>bagging_fraction: 0.759195090518222<br>bagging_freq: 2<br>min_child_samples: 19<br><b>Objective Value: 0.972027972027972</b>",
          "Trial #2<br>bagging_fraction: 0.4348501673009197<br>bagging_freq: 7<br>min_child_samples: 62<br><b>Objective Value: 0.958041958041958</b>",
          "Trial #3<br>bagging_fraction: 0.8248435466776274<br>bagging_freq: 1<br>min_child_samples: 98<br><b>Objective Value: 0.9790209790209791</b>",
          "Trial #4<br>bagging_fraction: 0.899465584480253<br>bagging_freq: 2<br>min_child_samples: 22<br><b>Objective Value: 0.958041958041958</b>",
          "Trial #5<br>bagging_fraction: 0.5100427059120604<br>bagging_freq: 3<br>min_child_samples: 55<br><b>Objective Value: 0.993006993006993</b>",
          "Trial #12<br>bagging_fraction: 0.9733907188988407<br>bagging_freq: 1<br>min_child_samples: 98<br><b>Objective Value: 0.986013986013986</b>",
          "Trial #18<br>bagging_fraction: 0.9905077333938093<br>bagging_freq: 2<br>min_child_samples: 54<br><b>Objective Value: 0.986013986013986</b>",
          "Trial #25<br>bagging_fraction: 0.9539760625736883<br>bagging_freq: 1<br>min_child_samples: 5<br><b>Objective Value: 0.9790209790209791</b>",
          "Trial #44<br>bagging_fraction: 0.8823409487656906<br>bagging_freq: 1<br>min_child_samples: 48<br><b>Objective Value: 0.986013986013986</b>",
          "Trial #55<br>bagging_fraction: 0.973380947872035<br>bagging_freq: 2<br>min_child_samples: 39<br><b>Objective Value: 0.986013986013986</b>",
          "Trial #82<br>bagging_fraction: 0.818345732780162<br>bagging_freq: 3<br>min_child_samples: 22<br><b>Objective Value: 0.9790209790209791</b>"
         ],
         "marker": {
          "color": [
           "rgb(131, 185, 216)",
           "rgb(131, 185, 216)",
           "rgb(58, 83, 163)",
           "rgb(241, 250, 217)",
           "rgb(58, 83, 163)",
           "rgb(165, 0, 38)",
           "rgb(246, 127, 75)",
           "rgb(246, 127, 75)",
           "rgb(241, 250, 217)",
           "rgb(246, 127, 75)",
           "rgb(246, 127, 75)",
           "rgb(241, 250, 217)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          75,
          19,
          62,
          98,
          22,
          55,
          98,
          54,
          5,
          48,
          39,
          22
         ],
         "xaxis": "x6",
         "y": [
          7,
          2,
          7,
          1,
          2,
          3,
          1,
          2,
          1,
          1,
          2,
          3
         ],
         "yaxis": "y6"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #0<br>bagging_fraction: 0.6247240713084175<br>bagging_freq: 7<br>min_child_samples: 75<br><b>Objective Value: 0.972027972027972</b>",
          "Trial #1<br>bagging_fraction: 0.759195090518222<br>bagging_freq: 2<br>min_child_samples: 19<br><b>Objective Value: 0.972027972027972</b>",
          "Trial #2<br>bagging_fraction: 0.4348501673009197<br>bagging_freq: 7<br>min_child_samples: 62<br><b>Objective Value: 0.958041958041958</b>",
          "Trial #3<br>bagging_fraction: 0.8248435466776274<br>bagging_freq: 1<br>min_child_samples: 98<br><b>Objective Value: 0.9790209790209791</b>",
          "Trial #4<br>bagging_fraction: 0.899465584480253<br>bagging_freq: 2<br>min_child_samples: 22<br><b>Objective Value: 0.958041958041958</b>",
          "Trial #5<br>bagging_fraction: 0.5100427059120604<br>bagging_freq: 3<br>min_child_samples: 55<br><b>Objective Value: 0.993006993006993</b>",
          "Trial #12<br>bagging_fraction: 0.9733907188988407<br>bagging_freq: 1<br>min_child_samples: 98<br><b>Objective Value: 0.986013986013986</b>",
          "Trial #18<br>bagging_fraction: 0.9905077333938093<br>bagging_freq: 2<br>min_child_samples: 54<br><b>Objective Value: 0.986013986013986</b>",
          "Trial #25<br>bagging_fraction: 0.9539760625736883<br>bagging_freq: 1<br>min_child_samples: 5<br><b>Objective Value: 0.9790209790209791</b>",
          "Trial #44<br>bagging_fraction: 0.8823409487656906<br>bagging_freq: 1<br>min_child_samples: 48<br><b>Objective Value: 0.986013986013986</b>",
          "Trial #55<br>bagging_fraction: 0.973380947872035<br>bagging_freq: 2<br>min_child_samples: 39<br><b>Objective Value: 0.986013986013986</b>",
          "Trial #82<br>bagging_fraction: 0.818345732780162<br>bagging_freq: 3<br>min_child_samples: 22<br><b>Objective Value: 0.9790209790209791</b>"
         ],
         "marker": {
          "color": [
           "rgb(131, 185, 216)",
           "rgb(131, 185, 216)",
           "rgb(58, 83, 163)",
           "rgb(241, 250, 217)",
           "rgb(58, 83, 163)",
           "rgb(165, 0, 38)",
           "rgb(246, 127, 75)",
           "rgb(246, 127, 75)",
           "rgb(241, 250, 217)",
           "rgb(246, 127, 75)",
           "rgb(246, 127, 75)",
           "rgb(241, 250, 217)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          75,
          19,
          62,
          98,
          22,
          55,
          98,
          54,
          5,
          48,
          39,
          22
         ],
         "xaxis": "x9",
         "y": [
          75,
          19,
          62,
          98,
          22,
          55,
          98,
          54,
          5,
          48,
          39,
          22
         ],
         "yaxis": "y9"
        },
        {
         "hoverinfo": "none",
         "marker": {
          "cmax": 1,
          "cmin": 0,
          "colorbar": {
           "thickness": 10,
           "ticktext": [
            "min. (0.958042)",
            "25% (0.972028)",
            "50% (0.979021)",
            "75% (0.986014)",
            "max. (0.993007)"
           ],
           "tickvals": [
            0,
            0.25,
            0.5,
            0.75,
            1
           ]
          },
          "colorscale": [
           [
            0,
            "rgb(49,54,149)"
           ],
           [
            0.1,
            "rgb(69,117,180)"
           ],
           [
            0.2,
            "rgb(116,173,209)"
           ],
           [
            0.3,
            "rgb(171,217,233)"
           ],
           [
            0.4,
            "rgb(224,243,248)"
           ],
           [
            0.5,
            "rgb(255,255,191)"
           ],
           [
            0.6,
            "rgb(254,224,144)"
           ],
           [
            0.7,
            "rgb(253,174,97)"
           ],
           [
            0.8,
            "rgb(244,109,67)"
           ],
           [
            0.9,
            "rgb(215,48,39)"
           ],
           [
            1,
            "rgb(165,0,38)"
           ]
          ],
          "showscale": true
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          null
         ],
         "y": [
          null
         ]
        }
       ],
       "layout": {
        "template": {
         "data": {
          "bar": [
           {
            "error_x": {
             "color": "#2a3f5f"
            },
            "error_y": {
             "color": "#2a3f5f"
            },
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "bar"
           }
          ],
          "barpolar": [
           {
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "barpolar"
           }
          ],
          "carpet": [
           {
            "aaxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "baxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "type": "carpet"
           }
          ],
          "choropleth": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "choropleth"
           }
          ],
          "contour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "contour"
           }
          ],
          "contourcarpet": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "contourcarpet"
           }
          ],
          "heatmap": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmap"
           }
          ],
          "heatmapgl": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmapgl"
           }
          ],
          "histogram": [
           {
            "marker": {
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "histogram"
           }
          ],
          "histogram2d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2d"
           }
          ],
          "histogram2dcontour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2dcontour"
           }
          ],
          "mesh3d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "mesh3d"
           }
          ],
          "parcoords": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "parcoords"
           }
          ],
          "pie": [
           {
            "automargin": true,
            "type": "pie"
           }
          ],
          "scatter": [
           {
            "fillpattern": {
             "fillmode": "overlay",
             "size": 10,
             "solidity": 0.2
            },
            "type": "scatter"
           }
          ],
          "scatter3d": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatter3d"
           }
          ],
          "scattercarpet": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattercarpet"
           }
          ],
          "scattergeo": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergeo"
           }
          ],
          "scattergl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergl"
           }
          ],
          "scattermapbox": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattermapbox"
           }
          ],
          "scatterpolar": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolar"
           }
          ],
          "scatterpolargl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolargl"
           }
          ],
          "scatterternary": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterternary"
           }
          ],
          "surface": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "surface"
           }
          ],
          "table": [
           {
            "cells": {
             "fill": {
              "color": "#EBF0F8"
             },
             "line": {
              "color": "white"
             }
            },
            "header": {
             "fill": {
              "color": "#C8D4E3"
             },
             "line": {
              "color": "white"
             }
            },
            "type": "table"
           }
          ]
         },
         "layout": {
          "annotationdefaults": {
           "arrowcolor": "#2a3f5f",
           "arrowhead": 0,
           "arrowwidth": 1
          },
          "autotypenumbers": "strict",
          "coloraxis": {
           "colorbar": {
            "outlinewidth": 0,
            "ticks": ""
           }
          },
          "colorscale": {
           "diverging": [
            [
             0,
             "#8e0152"
            ],
            [
             0.1,
             "#c51b7d"
            ],
            [
             0.2,
             "#de77ae"
            ],
            [
             0.3,
             "#f1b6da"
            ],
            [
             0.4,
             "#fde0ef"
            ],
            [
             0.5,
             "#f7f7f7"
            ],
            [
             0.6,
             "#e6f5d0"
            ],
            [
             0.7,
             "#b8e186"
            ],
            [
             0.8,
             "#7fbc41"
            ],
            [
             0.9,
             "#4d9221"
            ],
            [
             1,
             "#276419"
            ]
           ],
           "sequential": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ],
           "sequentialminus": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ]
          },
          "colorway": [
           "#636efa",
           "#EF553B",
           "#00cc96",
           "#ab63fa",
           "#FFA15A",
           "#19d3f3",
           "#FF6692",
           "#B6E880",
           "#FF97FF",
           "#FECB52"
          ],
          "font": {
           "color": "#2a3f5f"
          },
          "geo": {
           "bgcolor": "white",
           "lakecolor": "white",
           "landcolor": "#E5ECF6",
           "showlakes": true,
           "showland": true,
           "subunitcolor": "white"
          },
          "hoverlabel": {
           "align": "left"
          },
          "hovermode": "closest",
          "mapbox": {
           "style": "light"
          },
          "paper_bgcolor": "white",
          "plot_bgcolor": "#E5ECF6",
          "polar": {
           "angularaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "radialaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "scene": {
           "xaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "yaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "zaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           }
          },
          "shapedefaults": {
           "line": {
            "color": "#2a3f5f"
           }
          },
          "ternary": {
           "aaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "baxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "caxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "title": {
           "x": 0.05
          },
          "xaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          },
          "yaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          }
         }
        },
        "title": {
         "text": "Rank (Objective Value)"
        },
        "xaxis": {
         "anchor": "y",
         "domain": [
          0,
          0.31555555555555553
         ],
         "matches": "x7",
         "range": [
          0.4070672889962752,
          1.0182906116984538
         ],
         "showticklabels": false
        },
        "xaxis2": {
         "anchor": "y2",
         "domain": [
          0.3422222222222222,
          0.6577777777777778
         ],
         "matches": "x8",
         "range": [
          0.7,
          7.3
         ],
         "showticklabels": false
        },
        "xaxis3": {
         "anchor": "y3",
         "domain": [
          0.6844444444444444,
          1
         ],
         "matches": "x9",
         "range": [
          0.34999999999999964,
          102.65
         ],
         "showticklabels": false
        },
        "xaxis4": {
         "anchor": "y4",
         "domain": [
          0,
          0.31555555555555553
         ],
         "matches": "x7",
         "range": [
          0.4070672889962752,
          1.0182906116984538
         ],
         "showticklabels": false
        },
        "xaxis5": {
         "anchor": "y5",
         "domain": [
          0.3422222222222222,
          0.6577777777777778
         ],
         "matches": "x8",
         "range": [
          0.7,
          7.3
         ],
         "showticklabels": false
        },
        "xaxis6": {
         "anchor": "y6",
         "domain": [
          0.6844444444444444,
          1
         ],
         "matches": "x9",
         "range": [
          0.34999999999999964,
          102.65
         ],
         "showticklabels": false
        },
        "xaxis7": {
         "anchor": "y7",
         "domain": [
          0,
          0.31555555555555553
         ],
         "range": [
          0.4070672889962752,
          1.0182906116984538
         ],
         "title": {
          "text": "bagging_fraction"
         }
        },
        "xaxis8": {
         "anchor": "y8",
         "domain": [
          0.3422222222222222,
          0.6577777777777778
         ],
         "range": [
          0.7,
          7.3
         ],
         "title": {
          "text": "bagging_freq"
         }
        },
        "xaxis9": {
         "anchor": "y9",
         "domain": [
          0.6844444444444444,
          1
         ],
         "range": [
          0.34999999999999964,
          102.65
         ],
         "title": {
          "text": "min_child_samples"
         }
        },
        "yaxis": {
         "anchor": "x",
         "domain": [
          0.6844444444444444,
          1
         ],
         "range": [
          0.4070672889962752,
          1.0182906116984538
         ],
         "title": {
          "text": "bagging_fraction"
         }
        },
        "yaxis2": {
         "anchor": "x2",
         "domain": [
          0.6844444444444444,
          1
         ],
         "matches": "y",
         "range": [
          0.4070672889962752,
          1.0182906116984538
         ],
         "showticklabels": false
        },
        "yaxis3": {
         "anchor": "x3",
         "domain": [
          0.6844444444444444,
          1
         ],
         "matches": "y",
         "range": [
          0.4070672889962752,
          1.0182906116984538
         ],
         "showticklabels": false
        },
        "yaxis4": {
         "anchor": "x4",
         "domain": [
          0.3422222222222222,
          0.6577777777777778
         ],
         "range": [
          0.7,
          7.3
         ],
         "title": {
          "text": "bagging_freq"
         }
        },
        "yaxis5": {
         "anchor": "x5",
         "domain": [
          0.3422222222222222,
          0.6577777777777778
         ],
         "matches": "y4",
         "range": [
          0.7,
          7.3
         ],
         "showticklabels": false
        },
        "yaxis6": {
         "anchor": "x6",
         "domain": [
          0.3422222222222222,
          0.6577777777777778
         ],
         "matches": "y4",
         "range": [
          0.7,
          7.3
         ],
         "showticklabels": false
        },
        "yaxis7": {
         "anchor": "x7",
         "domain": [
          0,
          0.31555555555555553
         ],
         "range": [
          0.34999999999999964,
          102.65
         ],
         "title": {
          "text": "min_child_samples"
         }
        },
        "yaxis8": {
         "anchor": "x8",
         "domain": [
          0,
          0.31555555555555553
         ],
         "matches": "y7",
         "range": [
          0.34999999999999964,
          102.65
         ],
         "showticklabels": false
        },
        "yaxis9": {
         "anchor": "x9",
         "domain": [
          0,
          0.31555555555555553
         ],
         "matches": "y7",
         "range": [
          0.34999999999999964,
          102.65
         ],
         "showticklabels": false
        }
       }
      }
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_rank(study)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Y finalmente, para visualizar la serie de tiempo de la optimización para las pruebas realizadas:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\franz\\AppData\\Local\\Temp\\ipykernel_23396\\547629393.py:1: ExperimentalWarning:\n",
      "\n",
      "plot_timeline is experimental (supported from v3.2.0). The interface can change in the future.\n",
      "\n"
     ]
    },
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "config": {
        "plotlyServerURL": "https://plot.ly"
       },
       "data": [
        {
         "base": [
          "2023-09-21T11:36:22.036246",
          "2023-09-21T11:36:22.109478",
          "2023-09-21T11:36:22.229442",
          "2023-09-21T11:36:22.278254",
          "2023-09-21T11:36:22.339306",
          "2023-09-21T11:36:22.449261",
          "2023-09-21T11:36:22.772874",
          "2023-09-21T11:36:23.142600",
          "2023-09-21T11:36:23.639038",
          "2023-09-21T11:36:24.907090",
          "2023-09-21T11:36:25.633148",
          "2023-09-21T11:36:27.388759"
         ],
         "hovertemplate": "%{text}<extra>COMPLETE</extra>",
         "marker": {
          "color": "blue"
         },
         "name": "COMPLETE",
         "orientation": "h",
         "text": [
          "{<br>  \"number\": 0,<br>  \"values\": [<br>    0.972027972027972<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.6247240713084175,<br>    \"bagging_freq\": 7,<br>    \"min_child_samples\": 75<br>  }<br>}",
          "{<br>  \"number\": 1,<br>  \"values\": [<br>    0.972027972027972<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.759195090518222,<br>    \"bagging_freq\": 2,<br>    \"min_child_samples\": 19<br>  }<br>}",
          "{<br>  \"number\": 2,<br>  \"values\": [<br>    0.958041958041958<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.4348501673009197,<br>    \"bagging_freq\": 7,<br>    \"min_child_samples\": 62<br>  }<br>}",
          "{<br>  \"number\": 3,<br>  \"values\": [<br>    0.9790209790209791<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.8248435466776274,<br>    \"bagging_freq\": 1,<br>    \"min_child_samples\": 98<br>  }<br>}",
          "{<br>  \"number\": 4,<br>  \"values\": [<br>    0.958041958041958<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.899465584480253,<br>    \"bagging_freq\": 2,<br>    \"min_child_samples\": 22<br>  }<br>}",
          "{<br>  \"number\": 5,<br>  \"values\": [<br>    0.993006993006993<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.5100427059120604,<br>    \"bagging_freq\": 3,<br>    \"min_child_samples\": 55<br>  }<br>}",
          "{<br>  \"number\": 12,<br>  \"values\": [<br>    0.986013986013986<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.9733907188988407,<br>    \"bagging_freq\": 1,<br>    \"min_child_samples\": 98<br>  }<br>}",
          "{<br>  \"number\": 18,<br>  \"values\": [<br>    0.986013986013986<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.9905077333938093,<br>    \"bagging_freq\": 2,<br>    \"min_child_samples\": 54<br>  }<br>}",
          "{<br>  \"number\": 25,<br>  \"values\": [<br>    0.9790209790209791<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.9539760625736883,<br>    \"bagging_freq\": 1,<br>    \"min_child_samples\": 5<br>  }<br>}",
          "{<br>  \"number\": 44,<br>  \"values\": [<br>    0.986013986013986<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.8823409487656906,<br>    \"bagging_freq\": 1,<br>    \"min_child_samples\": 48<br>  }<br>}",
          "{<br>  \"number\": 55,<br>  \"values\": [<br>    0.986013986013986<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.973380947872035,<br>    \"bagging_freq\": 2,<br>    \"min_child_samples\": 39<br>  }<br>}",
          "{<br>  \"number\": 82,<br>  \"values\": [<br>    0.9790209790209791<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.818345732780162,<br>    \"bagging_freq\": 3,<br>    \"min_child_samples\": 22<br>  }<br>}"
         ],
         "textposition": "none",
         "type": "bar",
         "x": [
          65.229,
          119.964,
          48.812000000000005,
          48.548,
          109.955,
          79.00699999999999,
          109.038,
          118.085,
          173.391,
          118.34100000000001,
          131.53799999999998,
          181.278
         ],
         "y": [
          0,
          1,
          2,
          3,
          4,
          5,
          12,
          18,
          25,
          44,
          55,
          82
         ]
        },
        {
         "base": [
          "2023-09-21T11:36:22.528268",
          "2023-09-21T11:36:22.563745",
          "2023-09-21T11:36:22.591612",
          "2023-09-21T11:36:22.625151",
          "2023-09-21T11:36:22.662675",
          "2023-09-21T11:36:22.725882",
          "2023-09-21T11:36:22.881912",
          "2023-09-21T11:36:22.923141",
          "2023-09-21T11:36:22.985453",
          "2023-09-21T11:36:23.032558",
          "2023-09-21T11:36:23.082953",
          "2023-09-21T11:36:23.260685",
          "2023-09-21T11:36:23.311938",
          "2023-09-21T11:36:23.362840",
          "2023-09-21T11:36:23.466008",
          "2023-09-21T11:36:23.523996",
          "2023-09-21T11:36:23.576131",
          "2023-09-21T11:36:23.823938",
          "2023-09-21T11:36:23.875548",
          "2023-09-21T11:36:23.923828",
          "2023-09-21T11:36:23.978374",
          "2023-09-21T11:36:24.040880",
          "2023-09-21T11:36:24.087778",
          "2023-09-21T11:36:24.150922",
          "2023-09-21T11:36:24.213905",
          "2023-09-21T11:36:24.261248",
          "2023-09-21T11:36:24.324007",
          "2023-09-21T11:36:24.386743",
          "2023-09-21T11:36:24.449536",
          "2023-09-21T11:36:24.512022",
          "2023-09-21T11:36:24.574931",
          "2023-09-21T11:36:24.638180",
          "2023-09-21T11:36:24.701167",
          "2023-09-21T11:36:24.756539",
          "2023-09-21T11:36:24.824286",
          "2023-09-21T11:36:25.025431",
          "2023-09-21T11:36:25.080668",
          "2023-09-21T11:36:25.136397",
          "2023-09-21T11:36:25.207507",
          "2023-09-21T11:36:25.262921",
          "2023-09-21T11:36:25.325860",
          "2023-09-21T11:36:25.380928",
          "2023-09-21T11:36:25.443687",
          "2023-09-21T11:36:25.506539",
          "2023-09-21T11:36:25.569315",
          "2023-09-21T11:36:25.764686",
          "2023-09-21T11:36:25.827769",
          "2023-09-21T11:36:25.890502",
          "2023-09-21T11:36:25.946398",
          "2023-09-21T11:36:26.017167",
          "2023-09-21T11:36:26.080180",
          "2023-09-21T11:36:26.143189",
          "2023-09-21T11:36:26.190490",
          "2023-09-21T11:36:26.261976",
          "2023-09-21T11:36:26.325471",
          "2023-09-21T11:36:26.380543",
          "2023-09-21T11:36:26.449222",
          "2023-09-21T11:36:26.511790",
          "2023-09-21T11:36:26.575263",
          "2023-09-21T11:36:26.638522",
          "2023-09-21T11:36:26.685981",
          "2023-09-21T11:36:26.754450",
          "2023-09-21T11:36:26.825713",
          "2023-09-21T11:36:26.886361",
          "2023-09-21T11:36:26.933647",
          "2023-09-21T11:36:26.996483",
          "2023-09-21T11:36:27.059787",
          "2023-09-21T11:36:27.125870",
          "2023-09-21T11:36:27.191461",
          "2023-09-21T11:36:27.254325",
          "2023-09-21T11:36:27.317112",
          "2023-09-21T11:36:27.570037",
          "2023-09-21T11:36:27.640834",
          "2023-09-21T11:36:27.696386",
          "2023-09-21T11:36:27.759588",
          "2023-09-21T11:36:27.826735",
          "2023-09-21T11:36:27.901182",
          "2023-09-21T11:36:27.972536",
          "2023-09-21T11:36:28.035294",
          "2023-09-21T11:36:28.098253",
          "2023-09-21T11:36:28.177160",
          "2023-09-21T11:36:28.245932",
          "2023-09-21T11:36:28.308756",
          "2023-09-21T11:36:28.379701",
          "2023-09-21T11:36:28.442417",
          "2023-09-21T11:36:28.525251",
          "2023-09-21T11:36:28.588432",
          "2023-09-21T11:36:28.641378"
         ],
         "hovertemplate": "%{text}<extra>PRUNED</extra>",
         "marker": {
          "color": "orange"
         },
         "name": "PRUNED",
         "orientation": "h",
         "text": [
          "{<br>  \"number\": 6,<br>  \"values\": [<br>    0.9861842105263158<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.6591670111852694,<br>    \"bagging_freq\": 3,<br>    \"min_child_samples\": 63<br>  }<br>}",
          "{<br>  \"number\": 7,<br>  \"values\": [<br>    0.9739784946236559<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.4836963163912251,<br>    \"bagging_freq\": 3,<br>    \"min_child_samples\": 40<br>  }<br>}",
          "{<br>  \"number\": 8,<br>  \"values\": [<br>    0.9883004926108374<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.6736419905302216,<br>    \"bagging_freq\": 6,<br>    \"min_child_samples\": 24<br>  }<br>}",
          "{<br>  \"number\": 9,<br>  \"values\": [<br>    0.9866584564860427<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.708540663048167,<br>    \"bagging_freq\": 5,<br>    \"min_child_samples\": 9<br>  }<br>}",
          "{<br>  \"number\": 10,<br>  \"values\": [<br>    0.9928500986193294<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.5245713049729106,<br>    \"bagging_freq\": 4,<br>    \"min_child_samples\": 43<br>  }<br>}",
          "{<br>  \"number\": 11,<br>  \"values\": [<br>    0.981947261663286<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.8658543869864226,<br>    \"bagging_freq\": 1,<br>    \"min_child_samples\": 92<br>  }<br>}",
          "{<br>  \"number\": 13,<br>  \"values\": [<br>    0.9790322580645161<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.9974783900520112,<br>    \"bagging_freq\": 3,<br>    \"min_child_samples\": 79<br>  }<br>}",
          "{<br>  \"number\": 14,<br>  \"values\": [<br>    0.9847389558232932<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.5715125850670596,<br>    \"bagging_freq\": 2,<br>    \"min_child_samples\": 83<br>  }<br>}",
          "{<br>  \"number\": 15,<br>  \"values\": [<br>    0.9780645161290322<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.5909717500748849,<br>    \"bagging_freq\": 4,<br>    \"min_child_samples\": 51<br>  }<br>}",
          "{<br>  \"number\": 16,<br>  \"values\": [<br>    0.9834770114942529<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.40518624324379326,<br>    \"bagging_freq\": 1,<br>    \"min_child_samples\": 67<br>  }<br>}",
          "{<br>  \"number\": 17,<br>  \"values\": [<br>    0.9809330628803246<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.743911900574529,<br>    \"bagging_freq\": 5,<br>    \"min_child_samples\": 34<br>  }<br>}",
          "{<br>  \"number\": 19,<br>  \"values\": [<br>    0.9695225916453538<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.5362834069905272,<br>    \"bagging_freq\": 3,<br>    \"min_child_samples\": 99<br>  }<br>}",
          "{<br>  \"number\": 20,<br>  \"values\": [<br>    0.965980024968789<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.4761909064006522,<br>    \"bagging_freq\": 5,<br>    \"min_child_samples\": 83<br>  }<br>}",
          "{<br>  \"number\": 21,<br>  \"values\": [<br>    0.9975031210986267<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.9808066047665708,<br>    \"bagging_freq\": 2,<br>    \"min_child_samples\": 49<br>  }<br>}",
          "{<br>  \"number\": 22,<br>  \"values\": [<br>    0.9757936507936508<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.9314174573371937,<br>    \"bagging_freq\": 1,<br>    \"min_child_samples\": 57<br>  }<br>}",
          "{<br>  \"number\": 23,<br>  \"values\": [<br>    0.9788259958071279<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.9368590564802465,<br>    \"bagging_freq\": 2,<br>    \"min_child_samples\": 68<br>  }<br>}",
          "{<br>  \"number\": 24,<br>  \"values\": [<br>    0.9867920540997465<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.8308488248889416,<br>    \"bagging_freq\": 3,<br>    \"min_child_samples\": 32<br>  }<br>}",
          "{<br>  \"number\": 26,<br>  \"values\": [<br>    0.989558232931727<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.8878580266516762,<br>    \"bagging_freq\": 2,<br>    \"min_child_samples\": 91<br>  }<br>}",
          "{<br>  \"number\": 27,<br>  \"values\": [<br>    0.9896694214876033<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.9876553550636469,<br>    \"bagging_freq\": 4,<br>    \"min_child_samples\": 54<br>  }<br>}",
          "{<br>  \"number\": 28,<br>  \"values\": [<br>    0.9934917355371901<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.7914629164612188,<br>    \"bagging_freq\": 2,<br>    \"min_child_samples\": 74<br>  }<br>}",
          "{<br>  \"number\": 29,<br>  \"values\": [<br>    0.9843373493975903<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.6296260493825359,<br>    \"bagging_freq\": 1,<br>    \"min_child_samples\": 43<br>  }<br>}",
          "{<br>  \"number\": 30,<br>  \"values\": [<br>    0.9918851435705368<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.9344372889727264,<br>    \"bagging_freq\": 4,<br>    \"min_child_samples\": 76<br>  }<br>}",
          "{<br>  \"number\": 31,<br>  \"values\": [<br>    0.9839002267573697<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.8434238777773336,<br>    \"bagging_freq\": 1,<br>    \"min_child_samples\": 99<br>  }<br>}",
          "{<br>  \"number\": 32,<br>  \"values\": [<br>    0.9956611570247934<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.902944123412818,<br>    \"bagging_freq\": 1,<br>    \"min_child_samples\": 85<br>  }<br>}",
          "{<br>  \"number\": 33,<br>  \"values\": [<br>    0.982521847690387<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.8170369447371449,<br>    \"bagging_freq\": 2,<br>    \"min_child_samples\": 94<br>  }<br>}",
          "{<br>  \"number\": 34,<br>  \"values\": [<br>    0.9792692939244664<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.7722112958853592,<br>    \"bagging_freq\": 1,<br>    \"min_child_samples\": 88<br>  }<br>}",
          "{<br>  \"number\": 35,<br>  \"values\": [<br>    0.9771007566706491<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.9675079204616536,<br>    \"bagging_freq\": 3,<br>    \"min_child_samples\": 60<br>  }<br>}",
          "{<br>  \"number\": 36,<br>  \"values\": [<br>    0.9752066115702479<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.9049108440631194,<br>    \"bagging_freq\": 2,<br>    \"min_child_samples\": 73<br>  }<br>}",
          "{<br>  \"number\": 37,<br>  \"values\": [<br>    0.9781179138321996<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.8615225918405333,<br>    \"bagging_freq\": 7,<br>    \"min_child_samples\": 67<br>  }<br>}",
          "{<br>  \"number\": 38,<br>  \"values\": [<br>    0.9800438596491228<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.9971801450009377,<br>    \"bagging_freq\": 2,<br>    \"min_child_samples\": 16<br>  }<br>}",
          "{<br>  \"number\": 39,<br>  \"values\": [<br>    0.9879310344827587<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.7349589579118172,<br>    \"bagging_freq\": 3,<br>    \"min_child_samples\": 100<br>  }<br>}",
          "{<br>  \"number\": 40,<br>  \"values\": [<br>    0.9943998309382924<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.8032010286966313,<br>    \"bagging_freq\": 1,<br>    \"min_child_samples\": 31<br>  }<br>}",
          "{<br>  \"number\": 41,<br>  \"values\": [<br>    0.9883264462809918<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.9535680222813515,<br>    \"bagging_freq\": 1,<br>    \"min_child_samples\": 15<br>  }<br>}",
          "{<br>  \"number\": 42,<br>  \"values\": [<br>    0.9660997732426304<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.966647745265634,<br>    \"bagging_freq\": 1,<br>    \"min_child_samples\": 25<br>  }<br>}",
          "{<br>  \"number\": 43,<br>  \"values\": [<br>    0.9956916099773243<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.918133828513899,<br>    \"bagging_freq\": 2,<br>    \"min_child_samples\": 8<br>  }<br>}",
          "{<br>  \"number\": 45,<br>  \"values\": [<br>    0.9827856025039123<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.8780331843267022,<br>    \"bagging_freq\": 3,<br>    \"min_child_samples\": 45<br>  }<br>}",
          "{<br>  \"number\": 46,<br>  \"values\": [<br>    0.9781970649895179<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.8584099121851536,<br>    \"bagging_freq\": 2,<br>    \"min_child_samples\": 37<br>  }<br>}",
          "{<br>  \"number\": 47,<br>  \"values\": [<br>    0.9917355371900827<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.8889322836859961,<br>    \"bagging_freq\": 1,<br>    \"min_child_samples\": 47<br>  }<br>}",
          "{<br>  \"number\": 48,<br>  \"values\": [<br>    0.9813218390804598<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.6907439156972484,<br>    \"bagging_freq\": 2,<br>    \"min_child_samples\": 61<br>  }<br>}",
          "{<br>  \"number\": 49,<br>  \"values\": [<br>    0.9846059113300493<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.9206614784340134,<br>    \"bagging_freq\": 6,<br>    \"min_child_samples\": 54<br>  }<br>}",
          "{<br>  \"number\": 50,<br>  \"values\": [<br>    0.9656593406593407<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.9995760862502638,<br>    \"bagging_freq\": 3,<br>    \"min_child_samples\": 79<br>  }<br>}",
          "{<br>  \"number\": 51,<br>  \"values\": [<br>    0.9769088669950738<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.9539834671106007,<br>    \"bagging_freq\": 1,<br>    \"min_child_samples\": 95<br>  }<br>}",
          "{<br>  \"number\": 52,<br>  \"values\": [<br>    0.9892857142857143<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.9439812147929513,<br>    \"bagging_freq\": 1,<br>    \"min_child_samples\": 27<br>  }<br>}",
          "{<br>  \"number\": 53,<br>  \"values\": [<br>    0.9918864097363083<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.9574855588488546,<br>    \"bagging_freq\": 1,<br>    \"min_child_samples\": 14<br>  }<br>}",
          "{<br>  \"number\": 54,<br>  \"values\": [<br>    0.979874213836478<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.9144677608930759,<br>    \"bagging_freq\": 1,<br>    \"min_child_samples\": 21<br>  }<br>}",
          "{<br>  \"number\": 56,<br>  \"values\": [<br>    0.9611561866125761<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.9735990997429657,<br>    \"bagging_freq\": 2,<br>    \"min_child_samples\": 38<br>  }<br>}",
          "{<br>  \"number\": 57,<br>  \"values\": [<br>    0.9808313155770783<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.8744245929618839,<br>    \"bagging_freq\": 3,<br>    \"min_child_samples\": 51<br>  }<br>}",
          "{<br>  \"number\": 58,<br>  \"values\": [<br>    0.9728501773049646<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.9777715835777594,<br>    \"bagging_freq\": 2,<br>    \"min_child_samples\": 64<br>  }<br>}",
          "{<br>  \"number\": 59,<br>  \"values\": [<br>    0.9861678004535147<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.898821814557613,<br>    \"bagging_freq\": 2,<br>    \"min_child_samples\": 57<br>  }<br>}",
          "{<br>  \"number\": 60,<br>  \"values\": [<br>    0.9940159574468085<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.9344563886418518,<br>    \"bagging_freq\": 2,<br>    \"min_child_samples\": 44<br>  }<br>}",
          "{<br>  \"number\": 61,<br>  \"values\": [<br>    0.9890495867768595<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.9542965951935196,<br>    \"bagging_freq\": 1,<br>    \"min_child_samples\": 5<br>  }<br>}",
          "{<br>  \"number\": 62,<br>  \"values\": [<br>    0.9557323541411669<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.9818633762803768,<br>    \"bagging_freq\": 1,<br>    \"min_child_samples\": 41<br>  }<br>}",
          "{<br>  \"number\": 63,<br>  \"values\": [<br>    0.9565217391304348<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.9340477124241735,<br>    \"bagging_freq\": 1,<br>    \"min_child_samples\": 90<br>  }<br>}",
          "{<br>  \"number\": 64,<br>  \"values\": [<br>    0.9814372557533652<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.8424701484296954,<br>    \"bagging_freq\": 1,<br>    \"min_child_samples\": 57<br>  }<br>}",
          "{<br>  \"number\": 65,<br>  \"values\": [<br>    0.9602514919011083<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.9130467166440498,<br>    \"bagging_freq\": 4,<br>    \"min_child_samples\": 95<br>  }<br>}",
          "{<br>  \"number\": 66,<br>  \"values\": [<br>    0.9604818258664413<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.9980995769230847,<br>    \"bagging_freq\": 3,<br>    \"min_child_samples\": 70<br>  }<br>}",
          "{<br>  \"number\": 67,<br>  \"values\": [<br>    0.9874254049445865<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.97030374081496,<br>    \"bagging_freq\": 2,<br>    \"min_child_samples\": 49<br>  }<br>}",
          "{<br>  \"number\": 68,<br>  \"values\": [<br>    0.9869168356997972<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.8911259630401749,<br>    \"bagging_freq\": 1,<br>    \"min_child_samples\": 85<br>  }<br>}",
          "{<br>  \"number\": 69,<br>  \"values\": [<br>    0.9946808510638298<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.947233077817559,<br>    \"bagging_freq\": 2,<br>    \"min_child_samples\": 29<br>  }<br>}",
          "{<br>  \"number\": 70,<br>  \"values\": [<br>    0.9933700530395757<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.4538899767883604,<br>    \"bagging_freq\": 1,<br>    \"min_child_samples\": 37<br>  }<br>}",
          "{<br>  \"number\": 71,<br>  \"values\": [<br>    0.9802150537634409<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.6472559969314469,<br>    \"bagging_freq\": 6,<br>    \"min_child_samples\": 63<br>  }<br>}",
          "{<br>  \"number\": 72,<br>  \"values\": [<br>    0.9783870967741936<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.5174617427269833,<br>    \"bagging_freq\": 7,<br>    \"min_child_samples\": 80<br>  }<br>}",
          "{<br>  \"number\": 73,<br>  \"values\": [<br>    0.985080988917306<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.6214242874578974,<br>    \"bagging_freq\": 4,<br>    \"min_child_samples\": 97<br>  }<br>}",
          "{<br>  \"number\": 74,<br>  \"values\": [<br>    0.9807692307692307<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.5885896781778261,<br>    \"bagging_freq\": 5,<br>    \"min_child_samples\": 89<br>  }<br>}",
          "{<br>  \"number\": 75,<br>  \"values\": [<br>    0.9939771766694844<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.9868032103218984,<br>    \"bagging_freq\": 3,<br>    \"min_child_samples\": 54<br>  }<br>}",
          "{<br>  \"number\": 76,<br>  \"values\": [<br>    0.9919950738916257<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.9274515086910508,<br>    \"bagging_freq\": 1,<br>    \"min_child_samples\": 47<br>  }<br>}",
          "{<br>  \"number\": 77,<br>  \"values\": [<br>    0.9828490718321227<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.7066925339440039,<br>    \"bagging_freq\": 2,<br>    \"min_child_samples\": 93<br>  }<br>}",
          "{<br>  \"number\": 78,<br>  \"values\": [<br>    0.9763157894736842<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.9649220321474548,<br>    \"bagging_freq\": 7,<br>    \"min_child_samples\": 71<br>  }<br>}",
          "{<br>  \"number\": 79,<br>  \"values\": [<br>    0.9927174365376612<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.9461741058370005,<br>    \"bagging_freq\": 4,<br>    \"min_child_samples\": 59<br>  }<br>}",
          "{<br>  \"number\": 80,<br>  \"values\": [<br>    0.9653954802259888<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.9837349585102458,<br>    \"bagging_freq\": 1,<br>    \"min_child_samples\": 86<br>  }<br>}",
          "{<br>  \"number\": 81,<br>  \"values\": [<br>    0.9943502824858758<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.7920988685270793,<br>    \"bagging_freq\": 2,<br>    \"min_child_samples\": 12<br>  }<br>}",
          "{<br>  \"number\": 83,<br>  \"values\": [<br>    0.9955974842767296<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.8301963812079555,<br>    \"bagging_freq\": 3,<br>    \"min_child_samples\": 18<br>  }<br>}",
          "{<br>  \"number\": 84,<br>  \"values\": [<br>    0.9820971867007673<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.7689634136256347,<br>    \"bagging_freq\": 3,<br>    \"min_child_samples\": 41<br>  }<br>}",
          "{<br>  \"number\": 85,<br>  \"values\": [<br>    0.9679563492063492<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.906639724233196,<br>    \"bagging_freq\": 6,<br>    \"min_child_samples\": 24<br>  }<br>}",
          "{<br>  \"number\": 86,<br>  \"values\": [<br>    0.979064039408867<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.8617088447319218,<br>    \"bagging_freq\": 4,<br>    \"min_child_samples\": 32<br>  }<br>}",
          "{<br>  \"number\": 87,<br>  \"values\": [<br>    0.9919950738916257<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.7400623474371266,<br>    \"bagging_freq\": 5,<br>    \"min_child_samples\": 51<br>  }<br>}",
          "{<br>  \"number\": 88,<br>  \"values\": [<br>    0.9947845804988662<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.9238182573982916,<br>    \"bagging_freq\": 1,<br>    \"min_child_samples\": 10<br>  }<br>}",
          "{<br>  \"number\": 89,<br>  \"values\": [<br>    0.9685118780815778<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.8133386623656621,<br>    \"bagging_freq\": 1,<br>    \"min_child_samples\": 19<br>  }<br>}",
          "{<br>  \"number\": 90,<br>  \"values\": [<br>    0.9947981689554724<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.8776271206706433,<br>    \"bagging_freq\": 2,<br>    \"min_child_samples\": 66<br>  }<br>}",
          "{<br>  \"number\": 91,<br>  \"values\": [<br>    0.964696223316913<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.7739265829501584,<br>    \"bagging_freq\": 2,<br>    \"min_child_samples\": 7<br>  }<br>}",
          "{<br>  \"number\": 92,<br>  \"values\": [<br>    0.9859467455621301<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.7191238408079187,<br>    \"bagging_freq\": 3,<br>    \"min_child_samples\": 34<br>  }<br>}",
          "{<br>  \"number\": 93,<br>  \"values\": [<br>    0.9886792452830189<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.8505243292091138,<br>    \"bagging_freq\": 2,<br>    \"min_child_samples\": 23<br>  }<br>}",
          "{<br>  \"number\": 94,<br>  \"values\": [<br>    0.9946280991735538<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.9630463582824942,<br>    \"bagging_freq\": 1,<br>    \"min_child_samples\": 11<br>  }<br>}",
          "{<br>  \"number\": 95,<br>  \"values\": [<br>    0.9848111658456487<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.6894184231263091,<br>    \"bagging_freq\": 3,<br>    \"min_child_samples\": 97<br>  }<br>}",
          "{<br>  \"number\": 96,<br>  \"values\": [<br>    0.9976118106817194<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.8291575843630097,<br>    \"bagging_freq\": 2,<br>    \"min_child_samples\": 76<br>  }<br>}",
          "{<br>  \"number\": 97,<br>  \"values\": [<br>    0.9826711749788672<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.9419769524574453,<br>    \"bagging_freq\": 1,<br>    \"min_child_samples\": 35<br>  }<br>}",
          "{<br>  \"number\": 98,<br>  \"values\": [<br>    0.9851485148514851<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.9762270802333125,<br>    \"bagging_freq\": 2,<br>    \"min_child_samples\": 46<br>  }<br>}",
          "{<br>  \"number\": 99,<br>  \"values\": [<br>    0.9890350877192983<br>  ],<br>  \"params\": {<br>    \"bagging_fraction\": 0.8952095283668161,<br>    \"bagging_freq\": 1,<br>    \"min_child_samples\": 26<br>  }<br>}"
         ],
         "textposition": "none",
         "type": "bar",
         "x": [
          35.477000000000004,
          27.867,
          31.091,
          37.524,
          63.207,
          46.992,
          41.229,
          48.805,
          47.105000000000004,
          50.395,
          48.641999999999996,
          51.253,
          50.902,
          103.16799999999999,
          57.988,
          52.135,
          54.904,
          51.61000000000001,
          48.28,
          54.546,
          62.50600000000001,
          46.898,
          63.144000000000005,
          62.983,
          47.343,
          62.75899999999999,
          62.736,
          62.793,
          62.486,
          62.909000000000006,
          63.249,
          62.987,
          55.372,
          67.747,
          69.297,
          55.237,
          55.729,
          71.11000000000001,
          55.414,
          62.93899999999999,
          55.068,
          62.75899999999999,
          62.852000000000004,
          62.775999999999996,
          63.833,
          63.083,
          62.733,
          55.896,
          55.163,
          47.391000000000005,
          63.00899999999999,
          47.301,
          71.48599999999999,
          55.489999999999995,
          55.072,
          68.679,
          62.568,
          47.848,
          50.253,
          47.459,
          68.46900000000001,
          62.748,
          45.027,
          47.286,
          62.836000000000006,
          63.304,
          66.083,
          65.591,
          62.864000000000004,
          62.787,
          71.647,
          62.995999999999995,
          55.552,
          63.20199999999999,
          67.14699999999999,
          74.447,
          71.354,
          62.757999999999996,
          62.959,
          78.90700000000001,
          68.772,
          62.824000000000005,
          70.945,
          62.715999999999994,
          67.209,
          63.181000000000004,
          52.946,
          62.82600000000001
         ],
         "y": [
          6,
          7,
          8,
          9,
          10,
          11,
          13,
          14,
          15,
          16,
          17,
          19,
          20,
          21,
          22,
          23,
          24,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35,
          36,
          37,
          38,
          39,
          40,
          41,
          42,
          43,
          45,
          46,
          47,
          48,
          49,
          50,
          51,
          52,
          53,
          54,
          56,
          57,
          58,
          59,
          60,
          61,
          62,
          63,
          64,
          65,
          66,
          67,
          68,
          69,
          70,
          71,
          72,
          73,
          74,
          75,
          76,
          77,
          78,
          79,
          80,
          81,
          83,
          84,
          85,
          86,
          87,
          88,
          89,
          90,
          91,
          92,
          93,
          94,
          95,
          96,
          97,
          98,
          99
         ]
        }
       ],
       "layout": {
        "showlegend": true,
        "template": {
         "data": {
          "bar": [
           {
            "error_x": {
             "color": "#2a3f5f"
            },
            "error_y": {
             "color": "#2a3f5f"
            },
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "bar"
           }
          ],
          "barpolar": [
           {
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "barpolar"
           }
          ],
          "carpet": [
           {
            "aaxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "baxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "type": "carpet"
           }
          ],
          "choropleth": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "choropleth"
           }
          ],
          "contour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "contour"
           }
          ],
          "contourcarpet": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "contourcarpet"
           }
          ],
          "heatmap": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmap"
           }
          ],
          "heatmapgl": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmapgl"
           }
          ],
          "histogram": [
           {
            "marker": {
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "histogram"
           }
          ],
          "histogram2d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2d"
           }
          ],
          "histogram2dcontour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2dcontour"
           }
          ],
          "mesh3d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "mesh3d"
           }
          ],
          "parcoords": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "parcoords"
           }
          ],
          "pie": [
           {
            "automargin": true,
            "type": "pie"
           }
          ],
          "scatter": [
           {
            "fillpattern": {
             "fillmode": "overlay",
             "size": 10,
             "solidity": 0.2
            },
            "type": "scatter"
           }
          ],
          "scatter3d": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatter3d"
           }
          ],
          "scattercarpet": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattercarpet"
           }
          ],
          "scattergeo": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergeo"
           }
          ],
          "scattergl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergl"
           }
          ],
          "scattermapbox": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattermapbox"
           }
          ],
          "scatterpolar": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolar"
           }
          ],
          "scatterpolargl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolargl"
           }
          ],
          "scatterternary": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterternary"
           }
          ],
          "surface": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "surface"
           }
          ],
          "table": [
           {
            "cells": {
             "fill": {
              "color": "#EBF0F8"
             },
             "line": {
              "color": "white"
             }
            },
            "header": {
             "fill": {
              "color": "#C8D4E3"
             },
             "line": {
              "color": "white"
             }
            },
            "type": "table"
           }
          ]
         },
         "layout": {
          "annotationdefaults": {
           "arrowcolor": "#2a3f5f",
           "arrowhead": 0,
           "arrowwidth": 1
          },
          "autotypenumbers": "strict",
          "coloraxis": {
           "colorbar": {
            "outlinewidth": 0,
            "ticks": ""
           }
          },
          "colorscale": {
           "diverging": [
            [
             0,
             "#8e0152"
            ],
            [
             0.1,
             "#c51b7d"
            ],
            [
             0.2,
             "#de77ae"
            ],
            [
             0.3,
             "#f1b6da"
            ],
            [
             0.4,
             "#fde0ef"
            ],
            [
             0.5,
             "#f7f7f7"
            ],
            [
             0.6,
             "#e6f5d0"
            ],
            [
             0.7,
             "#b8e186"
            ],
            [
             0.8,
             "#7fbc41"
            ],
            [
             0.9,
             "#4d9221"
            ],
            [
             1,
             "#276419"
            ]
           ],
           "sequential": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ],
           "sequentialminus": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ]
          },
          "colorway": [
           "#636efa",
           "#EF553B",
           "#00cc96",
           "#ab63fa",
           "#FFA15A",
           "#19d3f3",
           "#FF6692",
           "#B6E880",
           "#FF97FF",
           "#FECB52"
          ],
          "font": {
           "color": "#2a3f5f"
          },
          "geo": {
           "bgcolor": "white",
           "lakecolor": "white",
           "landcolor": "#E5ECF6",
           "showlakes": true,
           "showland": true,
           "subunitcolor": "white"
          },
          "hoverlabel": {
           "align": "left"
          },
          "hovermode": "closest",
          "mapbox": {
           "style": "light"
          },
          "paper_bgcolor": "white",
          "plot_bgcolor": "#E5ECF6",
          "polar": {
           "angularaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "radialaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "scene": {
           "xaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "yaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "zaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           }
          },
          "shapedefaults": {
           "line": {
            "color": "#2a3f5f"
           }
          },
          "ternary": {
           "aaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "baxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "caxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "title": {
           "x": 0.05
          },
          "xaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          },
          "yaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          }
         }
        },
        "title": {
         "text": "Timeline Plot"
        },
        "xaxis": {
         "title": {
          "text": "Datetime"
         },
         "type": "date"
        },
        "yaxis": {
         "title": {
          "text": "Trial"
         }
        }
       }
      }
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_timeline(study)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Ejemplo con Tensorflow: MNIST\n",
    "\n",
    "Pongamos en práctica lo que hemos aprendido sobre Optuna. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "import optuna\n",
    "import tensorflow as tf\n",
    "import mlflow\n",
    "from tensorflow.keras import datasets, layers, models\n",
    "from tensorflow.keras.utils import to_categorical"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Haremos el clásico ejemplo de clasificar imágenes de dígitos escritos a mano. Para ello, usaremos el dataset MNIST. \n",
    "\n",
    "Podemos dividir los datos en los conjuntos de entrenamiento y prueba, así como normalizarlos y convirtiendo las etiquetas a una representación categórica (*one-hot encoding*)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 1. Cargamos el dataset MNIST\n",
    "\n",
    "(x_train, y_train), (x_test, y_test) = datasets.mnist.load_data()\n",
    "\n",
    "x_train, x_test = x_train/255., x_test/255.\n",
    "\n",
    "y_train, y_test = to_categorical(y_train), to_categorical(y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Después, definimos la función `create_model()` la cual recibe el objeto `trial` y define y compila un modelo de red neuronal a partir de un conjunto de hiperparámetros. \n",
    "\n",
    "`trial` sugiere los valores a utilizar durante el estudio."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Definimos la función create_model\n",
    "\n",
    "def create_model(trial):\n",
    "\n",
    "    model = models.Sequential()\n",
    "\n",
    "    model.add(layers.Flatten(input_shape = (28, 28)))\n",
    "\n",
    "    n_layers = trial.suggest_int(\"n_layers\", 1, 5)\n",
    "\n",
    "    # Optuna sugiere: Número de unidades por capa\n",
    "\n",
    "    for i in range(n_layers):\n",
    "\n",
    "        n_units = trial.suggest_int(f\"n_units_l{i}\", 32, 128)\n",
    "\n",
    "        model.add(layers.Dense(n_units, activation = 'relu'))\n",
    "\n",
    "    model.add(layers.Dense(10, activation = 'softmax'))\n",
    "\n",
    "    # Optuna sugiere: Valor de learning rate\n",
    "\n",
    "    lr = trial.suggest_float(\"lr\", 1e-5, 1e-1, log = True)\n",
    "\n",
    "    # Optuna sugiere: Optimizador\n",
    "\n",
    "    optimizer_options = [\"sgd\", \"adam\", \"rmsprop\"]\n",
    "\n",
    "    optimizer_selected = trial.suggest_categorical(\"optimizer\", optimizer_options)\n",
    "\n",
    "    if optimizer_selected == \"adam\":\n",
    "\n",
    "        optimizer = tf.keras.optimizers.Adam(learning_rate = lr)\n",
    "    \n",
    "    elif optimizer_selected == \"sgd\":\n",
    "\n",
    "        optimizer = tf.keras.optimizers.SGD(learning_rate = lr)\n",
    "\n",
    "    elif optimizer_selected == \"rmsprop\":\n",
    "\n",
    "        optimizer = tf.keras.optimizers.RMSprop(learning_rate = lr)\n",
    "\n",
    "    # Compilamos el modelo\n",
    "\n",
    "    model.compile(optimizer = optimizer,\n",
    "                  loss = \"categorical_crossentropy\",\n",
    "                  metrics = [\"accuracy\"])\n",
    "    \n",
    "    return model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "También, como lo hicimos anteriormente, definimos la función `objective`, la cual define el objetivo de la optimización. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "def objective(trial):\n",
    "\n",
    "    modelo = create_model(trial)\n",
    "\n",
    "    modelo.fit(x_train, y_train, epochs = 5, batch_size = 64, verbose = 0)\n",
    "\n",
    "    loss, accuracy = modelo.evaluate(x_test, y_test, verbose = 0)\n",
    "        \n",
    "    return loss"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finalmente, creamos un objeto `study`, pasando el argumento `direction = \"minimize\"`, que indica que la meta es minimizar la función `objective()`, indicando que realice 15 pruebas con los parámetros sugeridos, para encontrar el conjunto óptimo."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:36:31,862] A new study created in memory with name: no-name-e7ea1e13-f33f-4bae-b8e0-aae9fe57af4c\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "A new study created in memory with name: no-name-e7ea1e13-f33f-4bae-b8e0-aae9fe57af4c\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:37:35,466] Trial 0 finished with value: 0.13771341741085052 and parameters: {'n_layers': 2, 'n_units_l0': 101, 'n_units_l1': 48, 'lr': 0.007711398857368533, 'optimizer': 'adam'}. Best is trial 0 with value: 0.13771341741085052.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 0 finished with value: 0.13771341741085052 and parameters: {'n_layers': 2, 'n_units_l0': 101, 'n_units_l1': 48, 'lr': 0.007711398857368533, 'optimizer': 'adam'}. Best is trial 0 with value: 0.13771341741085052.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:38:35,001] Trial 1 finished with value: 0.12340866029262543 and parameters: {'n_layers': 2, 'n_units_l0': 70, 'n_units_l1': 41, 'lr': 0.003012808172117605, 'optimizer': 'rmsprop'}. Best is trial 1 with value: 0.12340866029262543.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 1 finished with value: 0.12340866029262543 and parameters: {'n_layers': 2, 'n_units_l0': 70, 'n_units_l1': 41, 'lr': 0.003012808172117605, 'optimizer': 'rmsprop'}. Best is trial 1 with value: 0.12340866029262543.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:39:13,948] Trial 2 finished with value: 0.1751943975687027 and parameters: {'n_layers': 3, 'n_units_l0': 124, 'n_units_l1': 48, 'n_units_l2': 58, 'lr': 0.005442268439829251, 'optimizer': 'rmsprop'}. Best is trial 1 with value: 0.12340866029262543.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 2 finished with value: 0.1751943975687027 and parameters: {'n_layers': 3, 'n_units_l0': 124, 'n_units_l1': 48, 'n_units_l2': 58, 'lr': 0.005442268439829251, 'optimizer': 'rmsprop'}. Best is trial 1 with value: 0.12340866029262543.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:40:18,749] Trial 3 finished with value: 0.11885558068752289 and parameters: {'n_layers': 3, 'n_units_l0': 122, 'n_units_l1': 97, 'n_units_l2': 66, 'lr': 0.00013421360666602302, 'optimizer': 'adam'}. Best is trial 3 with value: 0.11885558068752289.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 3 finished with value: 0.11885558068752289 and parameters: {'n_layers': 3, 'n_units_l0': 122, 'n_units_l1': 97, 'n_units_l2': 66, 'lr': 0.00013421360666602302, 'optimizer': 'adam'}. Best is trial 3 with value: 0.11885558068752289.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:41:25,615] Trial 4 finished with value: 0.9541876316070557 and parameters: {'n_layers': 3, 'n_units_l0': 37, 'n_units_l1': 43, 'n_units_l2': 65, 'lr': 0.05696244653419064, 'optimizer': 'rmsprop'}. Best is trial 3 with value: 0.11885558068752289.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 4 finished with value: 0.9541876316070557 and parameters: {'n_layers': 3, 'n_units_l0': 37, 'n_units_l1': 43, 'n_units_l2': 65, 'lr': 0.05696244653419064, 'optimizer': 'rmsprop'}. Best is trial 3 with value: 0.11885558068752289.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:42:14,592] Trial 5 finished with value: 0.09808095544576645 and parameters: {'n_layers': 3, 'n_units_l0': 78, 'n_units_l1': 126, 'n_units_l2': 60, 'lr': 0.05752434764095089, 'optimizer': 'sgd'}. Best is trial 5 with value: 0.09808095544576645.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 5 finished with value: 0.09808095544576645 and parameters: {'n_layers': 3, 'n_units_l0': 78, 'n_units_l1': 126, 'n_units_l2': 60, 'lr': 0.05752434764095089, 'optimizer': 'sgd'}. Best is trial 5 with value: 0.09808095544576645.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:43:16,771] Trial 6 finished with value: 0.08969108015298843 and parameters: {'n_layers': 4, 'n_units_l0': 119, 'n_units_l1': 86, 'n_units_l2': 112, 'n_units_l3': 91, 'lr': 0.00024040850246942775, 'optimizer': 'adam'}. Best is trial 6 with value: 0.08969108015298843.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 6 finished with value: 0.08969108015298843 and parameters: {'n_layers': 4, 'n_units_l0': 119, 'n_units_l1': 86, 'n_units_l2': 112, 'n_units_l3': 91, 'lr': 0.00024040850246942775, 'optimizer': 'adam'}. Best is trial 6 with value: 0.08969108015298843.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:44:07,795] Trial 7 finished with value: 0.37900349497795105 and parameters: {'n_layers': 2, 'n_units_l0': 114, 'n_units_l1': 51, 'lr': 0.0023572885912189617, 'optimizer': 'sgd'}. Best is trial 6 with value: 0.08969108015298843.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 7 finished with value: 0.37900349497795105 and parameters: {'n_layers': 2, 'n_units_l0': 114, 'n_units_l1': 51, 'lr': 0.0023572885912189617, 'optimizer': 'sgd'}. Best is trial 6 with value: 0.08969108015298843.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:45:08,302] Trial 8 finished with value: 0.1021626815199852 and parameters: {'n_layers': 5, 'n_units_l0': 116, 'n_units_l1': 52, 'n_units_l2': 75, 'n_units_l3': 33, 'n_units_l4': 99, 'lr': 0.0005769984149991314, 'optimizer': 'adam'}. Best is trial 6 with value: 0.08969108015298843.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 8 finished with value: 0.1021626815199852 and parameters: {'n_layers': 5, 'n_units_l0': 116, 'n_units_l1': 52, 'n_units_l2': 75, 'n_units_l3': 33, 'n_units_l4': 99, 'lr': 0.0005769984149991314, 'optimizer': 'adam'}. Best is trial 6 with value: 0.08969108015298843.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:45:46,243] Trial 9 finished with value: 0.6329582333564758 and parameters: {'n_layers': 1, 'n_units_l0': 38, 'lr': 0.08245600062993334, 'optimizer': 'adam'}. Best is trial 6 with value: 0.08969108015298843.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 9 finished with value: 0.6329582333564758 and parameters: {'n_layers': 1, 'n_units_l0': 38, 'lr': 0.08245600062993334, 'optimizer': 'adam'}. Best is trial 6 with value: 0.08969108015298843.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:46:43,635] Trial 10 finished with value: 0.3321899473667145 and parameters: {'n_layers': 5, 'n_units_l0': 99, 'n_units_l1': 82, 'n_units_l2': 122, 'n_units_l3': 110, 'n_units_l4': 32, 'lr': 1.785199667793032e-05, 'optimizer': 'adam'}. Best is trial 6 with value: 0.08969108015298843.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 10 finished with value: 0.3321899473667145 and parameters: {'n_layers': 5, 'n_units_l0': 99, 'n_units_l1': 82, 'n_units_l2': 122, 'n_units_l3': 110, 'n_units_l4': 32, 'lr': 1.785199667793032e-05, 'optimizer': 'adam'}. Best is trial 6 with value: 0.08969108015298843.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:47:35,436] Trial 11 finished with value: 1.9311206340789795 and parameters: {'n_layers': 4, 'n_units_l0': 72, 'n_units_l1': 127, 'n_units_l2': 113, 'n_units_l3': 81, 'lr': 0.00037332091663808175, 'optimizer': 'sgd'}. Best is trial 6 with value: 0.08969108015298843.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 11 finished with value: 1.9311206340789795 and parameters: {'n_layers': 4, 'n_units_l0': 72, 'n_units_l1': 127, 'n_units_l2': 113, 'n_units_l3': 81, 'lr': 0.00037332091663808175, 'optimizer': 'sgd'}. Best is trial 6 with value: 0.08969108015298843.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:48:28,832] Trial 12 finished with value: 0.1834155023097992 and parameters: {'n_layers': 4, 'n_units_l0': 88, 'n_units_l1': 125, 'n_units_l2': 33, 'n_units_l3': 127, 'lr': 0.018851800842778852, 'optimizer': 'sgd'}. Best is trial 6 with value: 0.08969108015298843.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 12 finished with value: 0.1834155023097992 and parameters: {'n_layers': 4, 'n_units_l0': 88, 'n_units_l1': 125, 'n_units_l2': 33, 'n_units_l3': 127, 'lr': 0.018851800842778852, 'optimizer': 'sgd'}. Best is trial 6 with value: 0.08969108015298843.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:49:21,509] Trial 13 finished with value: 0.14339764416217804 and parameters: {'n_layers': 4, 'n_units_l0': 55, 'n_units_l1': 104, 'n_units_l2': 96, 'n_units_l3': 77, 'lr': 0.022091351249891724, 'optimizer': 'sgd'}. Best is trial 6 with value: 0.08969108015298843.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 13 finished with value: 0.14339764416217804 and parameters: {'n_layers': 4, 'n_units_l0': 55, 'n_units_l1': 104, 'n_units_l2': 96, 'n_units_l3': 77, 'lr': 0.022091351249891724, 'optimizer': 'sgd'}. Best is trial 6 with value: 0.08969108015298843.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2023-09-21 11:50:18,432] Trial 14 finished with value: 0.7806804180145264 and parameters: {'n_layers': 4, 'n_units_l0': 85, 'n_units_l1': 71, 'n_units_l2': 96, 'n_units_l3': 81, 'lr': 0.0009877185587115079, 'optimizer': 'sgd'}. Best is trial 6 with value: 0.08969108015298843.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trial 14 finished with value: 0.7806804180145264 and parameters: {'n_layers': 4, 'n_units_l0': 85, 'n_units_l1': 71, 'n_units_l2': 96, 'n_units_l3': 81, 'lr': 0.0009877185587115079, 'optimizer': 'sgd'}. Best is trial 6 with value: 0.08969108015298843.\n"
     ]
    }
   ],
   "source": [
    "\n",
    "study = optuna.create_study(direction = \"minimize\")\n",
    "\n",
    "study.optimize(objective, n_trials = 15)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Número de pruebas terminadas:  15\n",
      "Mejor intento:  FrozenTrial(number=6, state=TrialState.COMPLETE, values=[0.08969108015298843], datetime_start=datetime.datetime(2023, 9, 21, 11, 42, 14, 592961), datetime_complete=datetime.datetime(2023, 9, 21, 11, 43, 16, 769889), params={'n_layers': 4, 'n_units_l0': 119, 'n_units_l1': 86, 'n_units_l2': 112, 'n_units_l3': 91, 'lr': 0.00024040850246942775, 'optimizer': 'adam'}, user_attrs={}, system_attrs={}, intermediate_values={}, distributions={'n_layers': IntDistribution(high=5, log=False, low=1, step=1), 'n_units_l0': IntDistribution(high=128, log=False, low=32, step=1), 'n_units_l1': IntDistribution(high=128, log=False, low=32, step=1), 'n_units_l2': IntDistribution(high=128, log=False, low=32, step=1), 'n_units_l3': IntDistribution(high=128, log=False, low=32, step=1), 'lr': FloatDistribution(high=0.1, log=True, low=1e-05, step=None), 'optimizer': CategoricalDistribution(choices=('sgd', 'adam', 'rmsprop'))}, trial_id=6, value=None)\n",
      "Valor:  0.08969108015298843\n",
      "Hiperparámetros:  {'n_layers': 4, 'n_units_l0': 119, 'n_units_l1': 86, 'n_units_l2': 112, 'n_units_l3': 91, 'lr': 0.00024040850246942775, 'optimizer': 'adam'}\n"
     ]
    }
   ],
   "source": [
    "print(\"Número de pruebas terminadas: \", len(study.trials))\n",
    "\n",
    "trial = study.best_trial\n",
    "\n",
    "print(\"Mejor intento: \", trial)\n",
    "\n",
    "\n",
    "print(\"Valor: \", trial.value)\n",
    "print(\"Hiperparámetros: \", trial.params)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "config": {
        "plotlyServerURL": "https://plot.ly"
       },
       "data": [
        {
         "mode": "markers",
         "name": "Objective Value",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14
         ],
         "y": [
          0.13771341741085052,
          0.12340866029262543,
          0.1751943975687027,
          0.11885558068752289,
          0.9541876316070557,
          0.09808095544576645,
          0.08969108015298843,
          0.37900349497795105,
          0.1021626815199852,
          0.6329582333564758,
          0.3321899473667145,
          1.9311206340789795,
          0.1834155023097992,
          0.14339764416217804,
          0.7806804180145264
         ]
        },
        {
         "mode": "lines",
         "name": "Best Value",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14
         ],
         "y": [
          0.13771341741085052,
          0.12340866029262543,
          0.12340866029262543,
          0.11885558068752289,
          0.11885558068752289,
          0.09808095544576645,
          0.08969108015298843,
          0.08969108015298843,
          0.08969108015298843,
          0.08969108015298843,
          0.08969108015298843,
          0.08969108015298843,
          0.08969108015298843,
          0.08969108015298843,
          0.08969108015298843
         ]
        },
        {
         "marker": {
          "color": "#cccccc"
         },
         "mode": "markers",
         "name": "Infeasible Trial",
         "showlegend": false,
         "type": "scatter",
         "x": [],
         "y": []
        }
       ],
       "layout": {
        "template": {
         "data": {
          "bar": [
           {
            "error_x": {
             "color": "#2a3f5f"
            },
            "error_y": {
             "color": "#2a3f5f"
            },
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "bar"
           }
          ],
          "barpolar": [
           {
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "barpolar"
           }
          ],
          "carpet": [
           {
            "aaxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "baxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "type": "carpet"
           }
          ],
          "choropleth": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "choropleth"
           }
          ],
          "contour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "contour"
           }
          ],
          "contourcarpet": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "contourcarpet"
           }
          ],
          "heatmap": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmap"
           }
          ],
          "heatmapgl": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmapgl"
           }
          ],
          "histogram": [
           {
            "marker": {
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "histogram"
           }
          ],
          "histogram2d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2d"
           }
          ],
          "histogram2dcontour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2dcontour"
           }
          ],
          "mesh3d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "mesh3d"
           }
          ],
          "parcoords": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "parcoords"
           }
          ],
          "pie": [
           {
            "automargin": true,
            "type": "pie"
           }
          ],
          "scatter": [
           {
            "fillpattern": {
             "fillmode": "overlay",
             "size": 10,
             "solidity": 0.2
            },
            "type": "scatter"
           }
          ],
          "scatter3d": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatter3d"
           }
          ],
          "scattercarpet": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattercarpet"
           }
          ],
          "scattergeo": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergeo"
           }
          ],
          "scattergl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergl"
           }
          ],
          "scattermapbox": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattermapbox"
           }
          ],
          "scatterpolar": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolar"
           }
          ],
          "scatterpolargl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolargl"
           }
          ],
          "scatterternary": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterternary"
           }
          ],
          "surface": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "surface"
           }
          ],
          "table": [
           {
            "cells": {
             "fill": {
              "color": "#EBF0F8"
             },
             "line": {
              "color": "white"
             }
            },
            "header": {
             "fill": {
              "color": "#C8D4E3"
             },
             "line": {
              "color": "white"
             }
            },
            "type": "table"
           }
          ]
         },
         "layout": {
          "annotationdefaults": {
           "arrowcolor": "#2a3f5f",
           "arrowhead": 0,
           "arrowwidth": 1
          },
          "autotypenumbers": "strict",
          "coloraxis": {
           "colorbar": {
            "outlinewidth": 0,
            "ticks": ""
           }
          },
          "colorscale": {
           "diverging": [
            [
             0,
             "#8e0152"
            ],
            [
             0.1,
             "#c51b7d"
            ],
            [
             0.2,
             "#de77ae"
            ],
            [
             0.3,
             "#f1b6da"
            ],
            [
             0.4,
             "#fde0ef"
            ],
            [
             0.5,
             "#f7f7f7"
            ],
            [
             0.6,
             "#e6f5d0"
            ],
            [
             0.7,
             "#b8e186"
            ],
            [
             0.8,
             "#7fbc41"
            ],
            [
             0.9,
             "#4d9221"
            ],
            [
             1,
             "#276419"
            ]
           ],
           "sequential": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ],
           "sequentialminus": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ]
          },
          "colorway": [
           "#636efa",
           "#EF553B",
           "#00cc96",
           "#ab63fa",
           "#FFA15A",
           "#19d3f3",
           "#FF6692",
           "#B6E880",
           "#FF97FF",
           "#FECB52"
          ],
          "font": {
           "color": "#2a3f5f"
          },
          "geo": {
           "bgcolor": "white",
           "lakecolor": "white",
           "landcolor": "#E5ECF6",
           "showlakes": true,
           "showland": true,
           "subunitcolor": "white"
          },
          "hoverlabel": {
           "align": "left"
          },
          "hovermode": "closest",
          "mapbox": {
           "style": "light"
          },
          "paper_bgcolor": "white",
          "plot_bgcolor": "#E5ECF6",
          "polar": {
           "angularaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "radialaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "scene": {
           "xaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "yaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "zaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           }
          },
          "shapedefaults": {
           "line": {
            "color": "#2a3f5f"
           }
          },
          "ternary": {
           "aaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "baxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "caxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "title": {
           "x": 0.05
          },
          "xaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          },
          "yaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          }
         }
        },
        "title": {
         "text": "Optimization History Plot"
        },
        "xaxis": {
         "title": {
          "text": "Trial"
         }
        },
        "yaxis": {
         "title": {
          "text": "Objective Value"
         }
        }
       }
      }
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_optimization_history(study)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "config": {
        "plotlyServerURL": "https://plot.ly"
       },
       "data": [
        {
         "dimensions": [
          {
           "label": "Objective Value",
           "range": [
            0.1021626815199852,
            0.3321899473667145
           ],
           "values": [
            0.1021626815199852,
            0.3321899473667145
           ]
          },
          {
           "label": "lr",
           "range": [
            -4.748313202649157,
            -3.2388253798392017
           ],
           "ticktext": [
            "1.79e-05",
            "0.0001",
            "0.000577"
           ],
           "tickvals": [
            -4.748313202649157,
            -4,
            -3.2388253798392017
           ],
           "values": [
            -3.2388253798392017,
            -4.748313202649157
           ]
          },
          {
           "label": "n_layers",
           "range": [
            5,
            5
           ],
           "values": [
            5,
            5
           ]
          },
          {
           "label": "n_units_l0",
           "range": [
            99,
            116
           ],
           "values": [
            116,
            99
           ]
          },
          {
           "label": "n_units_l1",
           "range": [
            52,
            82
           ],
           "values": [
            52,
            82
           ]
          },
          {
           "label": "n_units_l2",
           "range": [
            75,
            122
           ],
           "values": [
            75,
            122
           ]
          },
          {
           "label": "n_units_l3",
           "range": [
            33,
            110
           ],
           "values": [
            33,
            110
           ]
          },
          {
           "label": "n_units_l4",
           "range": [
            32,
            99
           ],
           "values": [
            99,
            32
           ]
          },
          {
           "label": "optimizer",
           "range": [
            0,
            0
           ],
           "ticktext": [
            "adam"
           ],
           "tickvals": [
            0
           ],
           "values": [
            0,
            0
           ]
          }
         ],
         "labelangle": 30,
         "labelside": "bottom",
         "line": {
          "color": [
           0.1021626815199852,
           0.3321899473667145
          ],
          "colorbar": {
           "title": {
            "text": "Objective Value"
           }
          },
          "colorscale": [
           [
            0,
            "rgb(247,251,255)"
           ],
           [
            0.125,
            "rgb(222,235,247)"
           ],
           [
            0.25,
            "rgb(198,219,239)"
           ],
           [
            0.375,
            "rgb(158,202,225)"
           ],
           [
            0.5,
            "rgb(107,174,214)"
           ],
           [
            0.625,
            "rgb(66,146,198)"
           ],
           [
            0.75,
            "rgb(33,113,181)"
           ],
           [
            0.875,
            "rgb(8,81,156)"
           ],
           [
            1,
            "rgb(8,48,107)"
           ]
          ],
          "reversescale": true,
          "showscale": true
         },
         "type": "parcoords"
        }
       ],
       "layout": {
        "template": {
         "data": {
          "bar": [
           {
            "error_x": {
             "color": "#2a3f5f"
            },
            "error_y": {
             "color": "#2a3f5f"
            },
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "bar"
           }
          ],
          "barpolar": [
           {
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "barpolar"
           }
          ],
          "carpet": [
           {
            "aaxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "baxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "type": "carpet"
           }
          ],
          "choropleth": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "choropleth"
           }
          ],
          "contour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "contour"
           }
          ],
          "contourcarpet": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "contourcarpet"
           }
          ],
          "heatmap": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmap"
           }
          ],
          "heatmapgl": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmapgl"
           }
          ],
          "histogram": [
           {
            "marker": {
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "histogram"
           }
          ],
          "histogram2d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2d"
           }
          ],
          "histogram2dcontour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2dcontour"
           }
          ],
          "mesh3d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "mesh3d"
           }
          ],
          "parcoords": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "parcoords"
           }
          ],
          "pie": [
           {
            "automargin": true,
            "type": "pie"
           }
          ],
          "scatter": [
           {
            "fillpattern": {
             "fillmode": "overlay",
             "size": 10,
             "solidity": 0.2
            },
            "type": "scatter"
           }
          ],
          "scatter3d": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatter3d"
           }
          ],
          "scattercarpet": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattercarpet"
           }
          ],
          "scattergeo": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergeo"
           }
          ],
          "scattergl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergl"
           }
          ],
          "scattermapbox": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattermapbox"
           }
          ],
          "scatterpolar": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolar"
           }
          ],
          "scatterpolargl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolargl"
           }
          ],
          "scatterternary": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterternary"
           }
          ],
          "surface": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "surface"
           }
          ],
          "table": [
           {
            "cells": {
             "fill": {
              "color": "#EBF0F8"
             },
             "line": {
              "color": "white"
             }
            },
            "header": {
             "fill": {
              "color": "#C8D4E3"
             },
             "line": {
              "color": "white"
             }
            },
            "type": "table"
           }
          ]
         },
         "layout": {
          "annotationdefaults": {
           "arrowcolor": "#2a3f5f",
           "arrowhead": 0,
           "arrowwidth": 1
          },
          "autotypenumbers": "strict",
          "coloraxis": {
           "colorbar": {
            "outlinewidth": 0,
            "ticks": ""
           }
          },
          "colorscale": {
           "diverging": [
            [
             0,
             "#8e0152"
            ],
            [
             0.1,
             "#c51b7d"
            ],
            [
             0.2,
             "#de77ae"
            ],
            [
             0.3,
             "#f1b6da"
            ],
            [
             0.4,
             "#fde0ef"
            ],
            [
             0.5,
             "#f7f7f7"
            ],
            [
             0.6,
             "#e6f5d0"
            ],
            [
             0.7,
             "#b8e186"
            ],
            [
             0.8,
             "#7fbc41"
            ],
            [
             0.9,
             "#4d9221"
            ],
            [
             1,
             "#276419"
            ]
           ],
           "sequential": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ],
           "sequentialminus": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ]
          },
          "colorway": [
           "#636efa",
           "#EF553B",
           "#00cc96",
           "#ab63fa",
           "#FFA15A",
           "#19d3f3",
           "#FF6692",
           "#B6E880",
           "#FF97FF",
           "#FECB52"
          ],
          "font": {
           "color": "#2a3f5f"
          },
          "geo": {
           "bgcolor": "white",
           "lakecolor": "white",
           "landcolor": "#E5ECF6",
           "showlakes": true,
           "showland": true,
           "subunitcolor": "white"
          },
          "hoverlabel": {
           "align": "left"
          },
          "hovermode": "closest",
          "mapbox": {
           "style": "light"
          },
          "paper_bgcolor": "white",
          "plot_bgcolor": "#E5ECF6",
          "polar": {
           "angularaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "radialaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "scene": {
           "xaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "yaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "zaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           }
          },
          "shapedefaults": {
           "line": {
            "color": "#2a3f5f"
           }
          },
          "ternary": {
           "aaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "baxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "caxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "title": {
           "x": 0.05
          },
          "xaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          },
          "yaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          }
         }
        },
        "title": {
         "text": "Parallel Coordinate Plot"
        }
       }
      }
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_parallel_coordinate(study)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "config": {
        "plotlyServerURL": "https://plot.ly"
       },
       "data": [
        {
         "marker": {
          "color": [
           0,
           1,
           2,
           3,
           4,
           5,
           6,
           7,
           8,
           9,
           10,
           11,
           12,
           13,
           14
          ],
          "colorbar": {
           "title": {
            "text": "Trial"
           },
           "x": 1,
           "xpad": 40
          },
          "colorscale": [
           [
            0,
            "rgb(247,251,255)"
           ],
           [
            0.125,
            "rgb(222,235,247)"
           ],
           [
            0.25,
            "rgb(198,219,239)"
           ],
           [
            0.375,
            "rgb(158,202,225)"
           ],
           [
            0.5,
            "rgb(107,174,214)"
           ],
           [
            0.625,
            "rgb(66,146,198)"
           ],
           [
            0.75,
            "rgb(33,113,181)"
           ],
           [
            0.875,
            "rgb(8,81,156)"
           ],
           [
            1,
            "rgb(8,48,107)"
           ]
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          },
          "showscale": true
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          0.007711398857368533,
          0.003012808172117605,
          0.005442268439829251,
          0.00013421360666602302,
          0.05696244653419064,
          0.05752434764095089,
          0.00024040850246942775,
          0.0023572885912189617,
          0.0005769984149991314,
          0.08245600062993334,
          0.00001785199667793032,
          0.00037332091663808175,
          0.018851800842778852,
          0.022091351249891724,
          0.0009877185587115079
         ],
         "xaxis": "x",
         "y": [
          0.13771341741085052,
          0.12340866029262543,
          0.1751943975687027,
          0.11885558068752289,
          0.9541876316070557,
          0.09808095544576645,
          0.08969108015298843,
          0.37900349497795105,
          0.1021626815199852,
          0.6329582333564758,
          0.3321899473667145,
          1.9311206340789795,
          0.1834155023097992,
          0.14339764416217804,
          0.7806804180145264
         ],
         "yaxis": "y"
        },
        {
         "marker": {
          "color": [
           0,
           1,
           2,
           3,
           4,
           5,
           6,
           7,
           8,
           9,
           10,
           11,
           12,
           13,
           14
          ],
          "colorbar": {
           "title": {
            "text": "Trial"
           },
           "x": 1,
           "xpad": 40
          },
          "colorscale": [
           [
            0,
            "rgb(247,251,255)"
           ],
           [
            0.125,
            "rgb(222,235,247)"
           ],
           [
            0.25,
            "rgb(198,219,239)"
           ],
           [
            0.375,
            "rgb(158,202,225)"
           ],
           [
            0.5,
            "rgb(107,174,214)"
           ],
           [
            0.625,
            "rgb(66,146,198)"
           ],
           [
            0.75,
            "rgb(33,113,181)"
           ],
           [
            0.875,
            "rgb(8,81,156)"
           ],
           [
            1,
            "rgb(8,48,107)"
           ]
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          },
          "showscale": false
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          2,
          2,
          3,
          3,
          3,
          3,
          4,
          2,
          5,
          1,
          5,
          4,
          4,
          4,
          4
         ],
         "xaxis": "x2",
         "y": [
          0.13771341741085052,
          0.12340866029262543,
          0.1751943975687027,
          0.11885558068752289,
          0.9541876316070557,
          0.09808095544576645,
          0.08969108015298843,
          0.37900349497795105,
          0.1021626815199852,
          0.6329582333564758,
          0.3321899473667145,
          1.9311206340789795,
          0.1834155023097992,
          0.14339764416217804,
          0.7806804180145264
         ],
         "yaxis": "y2"
        },
        {
         "marker": {
          "color": [
           0,
           1,
           2,
           3,
           4,
           5,
           6,
           7,
           8,
           9,
           10,
           11,
           12,
           13,
           14
          ],
          "colorbar": {
           "title": {
            "text": "Trial"
           },
           "x": 1,
           "xpad": 40
          },
          "colorscale": [
           [
            0,
            "rgb(247,251,255)"
           ],
           [
            0.125,
            "rgb(222,235,247)"
           ],
           [
            0.25,
            "rgb(198,219,239)"
           ],
           [
            0.375,
            "rgb(158,202,225)"
           ],
           [
            0.5,
            "rgb(107,174,214)"
           ],
           [
            0.625,
            "rgb(66,146,198)"
           ],
           [
            0.75,
            "rgb(33,113,181)"
           ],
           [
            0.875,
            "rgb(8,81,156)"
           ],
           [
            1,
            "rgb(8,48,107)"
           ]
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          },
          "showscale": false
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          101,
          70,
          124,
          122,
          37,
          78,
          119,
          114,
          116,
          38,
          99,
          72,
          88,
          55,
          85
         ],
         "xaxis": "x3",
         "y": [
          0.13771341741085052,
          0.12340866029262543,
          0.1751943975687027,
          0.11885558068752289,
          0.9541876316070557,
          0.09808095544576645,
          0.08969108015298843,
          0.37900349497795105,
          0.1021626815199852,
          0.6329582333564758,
          0.3321899473667145,
          1.9311206340789795,
          0.1834155023097992,
          0.14339764416217804,
          0.7806804180145264
         ],
         "yaxis": "y3"
        },
        {
         "marker": {
          "color": [
           0,
           1,
           2,
           3,
           4,
           5,
           6,
           7,
           8,
           10,
           11,
           12,
           13,
           14
          ],
          "colorbar": {
           "title": {
            "text": "Trial"
           },
           "x": 1,
           "xpad": 40
          },
          "colorscale": [
           [
            0,
            "rgb(247,251,255)"
           ],
           [
            0.125,
            "rgb(222,235,247)"
           ],
           [
            0.25,
            "rgb(198,219,239)"
           ],
           [
            0.375,
            "rgb(158,202,225)"
           ],
           [
            0.5,
            "rgb(107,174,214)"
           ],
           [
            0.625,
            "rgb(66,146,198)"
           ],
           [
            0.75,
            "rgb(33,113,181)"
           ],
           [
            0.875,
            "rgb(8,81,156)"
           ],
           [
            1,
            "rgb(8,48,107)"
           ]
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          },
          "showscale": false
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          48,
          41,
          48,
          97,
          43,
          126,
          86,
          51,
          52,
          82,
          127,
          125,
          104,
          71
         ],
         "xaxis": "x4",
         "y": [
          0.13771341741085052,
          0.12340866029262543,
          0.1751943975687027,
          0.11885558068752289,
          0.9541876316070557,
          0.09808095544576645,
          0.08969108015298843,
          0.37900349497795105,
          0.1021626815199852,
          0.3321899473667145,
          1.9311206340789795,
          0.1834155023097992,
          0.14339764416217804,
          0.7806804180145264
         ],
         "yaxis": "y4"
        },
        {
         "marker": {
          "color": [
           2,
           3,
           4,
           5,
           6,
           8,
           10,
           11,
           12,
           13,
           14
          ],
          "colorbar": {
           "title": {
            "text": "Trial"
           },
           "x": 1,
           "xpad": 40
          },
          "colorscale": [
           [
            0,
            "rgb(247,251,255)"
           ],
           [
            0.125,
            "rgb(222,235,247)"
           ],
           [
            0.25,
            "rgb(198,219,239)"
           ],
           [
            0.375,
            "rgb(158,202,225)"
           ],
           [
            0.5,
            "rgb(107,174,214)"
           ],
           [
            0.625,
            "rgb(66,146,198)"
           ],
           [
            0.75,
            "rgb(33,113,181)"
           ],
           [
            0.875,
            "rgb(8,81,156)"
           ],
           [
            1,
            "rgb(8,48,107)"
           ]
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          },
          "showscale": false
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          58,
          66,
          65,
          60,
          112,
          75,
          122,
          113,
          33,
          96,
          96
         ],
         "xaxis": "x5",
         "y": [
          0.1751943975687027,
          0.11885558068752289,
          0.9541876316070557,
          0.09808095544576645,
          0.08969108015298843,
          0.1021626815199852,
          0.3321899473667145,
          1.9311206340789795,
          0.1834155023097992,
          0.14339764416217804,
          0.7806804180145264
         ],
         "yaxis": "y5"
        },
        {
         "marker": {
          "color": [
           6,
           8,
           10,
           11,
           12,
           13,
           14
          ],
          "colorbar": {
           "title": {
            "text": "Trial"
           },
           "x": 1,
           "xpad": 40
          },
          "colorscale": [
           [
            0,
            "rgb(247,251,255)"
           ],
           [
            0.125,
            "rgb(222,235,247)"
           ],
           [
            0.25,
            "rgb(198,219,239)"
           ],
           [
            0.375,
            "rgb(158,202,225)"
           ],
           [
            0.5,
            "rgb(107,174,214)"
           ],
           [
            0.625,
            "rgb(66,146,198)"
           ],
           [
            0.75,
            "rgb(33,113,181)"
           ],
           [
            0.875,
            "rgb(8,81,156)"
           ],
           [
            1,
            "rgb(8,48,107)"
           ]
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          },
          "showscale": false
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          91,
          33,
          110,
          81,
          127,
          77,
          81
         ],
         "xaxis": "x6",
         "y": [
          0.08969108015298843,
          0.1021626815199852,
          0.3321899473667145,
          1.9311206340789795,
          0.1834155023097992,
          0.14339764416217804,
          0.7806804180145264
         ],
         "yaxis": "y6"
        },
        {
         "marker": {
          "color": [
           8,
           10
          ],
          "colorbar": {
           "title": {
            "text": "Trial"
           },
           "x": 1,
           "xpad": 40
          },
          "colorscale": [
           [
            0,
            "rgb(247,251,255)"
           ],
           [
            0.125,
            "rgb(222,235,247)"
           ],
           [
            0.25,
            "rgb(198,219,239)"
           ],
           [
            0.375,
            "rgb(158,202,225)"
           ],
           [
            0.5,
            "rgb(107,174,214)"
           ],
           [
            0.625,
            "rgb(66,146,198)"
           ],
           [
            0.75,
            "rgb(33,113,181)"
           ],
           [
            0.875,
            "rgb(8,81,156)"
           ],
           [
            1,
            "rgb(8,48,107)"
           ]
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          },
          "showscale": false
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          99,
          32
         ],
         "xaxis": "x7",
         "y": [
          0.1021626815199852,
          0.3321899473667145
         ],
         "yaxis": "y7"
        },
        {
         "marker": {
          "color": [
           0,
           1,
           2,
           3,
           4,
           5,
           6,
           7,
           8,
           9,
           10,
           11,
           12,
           13,
           14
          ],
          "colorbar": {
           "title": {
            "text": "Trial"
           },
           "x": 1,
           "xpad": 40
          },
          "colorscale": [
           [
            0,
            "rgb(247,251,255)"
           ],
           [
            0.125,
            "rgb(222,235,247)"
           ],
           [
            0.25,
            "rgb(198,219,239)"
           ],
           [
            0.375,
            "rgb(158,202,225)"
           ],
           [
            0.5,
            "rgb(107,174,214)"
           ],
           [
            0.625,
            "rgb(66,146,198)"
           ],
           [
            0.75,
            "rgb(33,113,181)"
           ],
           [
            0.875,
            "rgb(8,81,156)"
           ],
           [
            1,
            "rgb(8,48,107)"
           ]
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          },
          "showscale": false
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          "adam",
          "rmsprop",
          "rmsprop",
          "adam",
          "rmsprop",
          "sgd",
          "adam",
          "sgd",
          "adam",
          "adam",
          "adam",
          "sgd",
          "sgd",
          "sgd",
          "sgd"
         ],
         "xaxis": "x8",
         "y": [
          0.13771341741085052,
          0.12340866029262543,
          0.1751943975687027,
          0.11885558068752289,
          0.9541876316070557,
          0.09808095544576645,
          0.08969108015298843,
          0.37900349497795105,
          0.1021626815199852,
          0.6329582333564758,
          0.3321899473667145,
          1.9311206340789795,
          0.1834155023097992,
          0.14339764416217804,
          0.7806804180145264
         ],
         "yaxis": "y8"
        }
       ],
       "layout": {
        "template": {
         "data": {
          "bar": [
           {
            "error_x": {
             "color": "#2a3f5f"
            },
            "error_y": {
             "color": "#2a3f5f"
            },
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "bar"
           }
          ],
          "barpolar": [
           {
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "barpolar"
           }
          ],
          "carpet": [
           {
            "aaxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "baxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "type": "carpet"
           }
          ],
          "choropleth": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "choropleth"
           }
          ],
          "contour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "contour"
           }
          ],
          "contourcarpet": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "contourcarpet"
           }
          ],
          "heatmap": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmap"
           }
          ],
          "heatmapgl": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmapgl"
           }
          ],
          "histogram": [
           {
            "marker": {
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "histogram"
           }
          ],
          "histogram2d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2d"
           }
          ],
          "histogram2dcontour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2dcontour"
           }
          ],
          "mesh3d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "mesh3d"
           }
          ],
          "parcoords": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "parcoords"
           }
          ],
          "pie": [
           {
            "automargin": true,
            "type": "pie"
           }
          ],
          "scatter": [
           {
            "fillpattern": {
             "fillmode": "overlay",
             "size": 10,
             "solidity": 0.2
            },
            "type": "scatter"
           }
          ],
          "scatter3d": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatter3d"
           }
          ],
          "scattercarpet": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattercarpet"
           }
          ],
          "scattergeo": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergeo"
           }
          ],
          "scattergl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergl"
           }
          ],
          "scattermapbox": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattermapbox"
           }
          ],
          "scatterpolar": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolar"
           }
          ],
          "scatterpolargl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolargl"
           }
          ],
          "scatterternary": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterternary"
           }
          ],
          "surface": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "surface"
           }
          ],
          "table": [
           {
            "cells": {
             "fill": {
              "color": "#EBF0F8"
             },
             "line": {
              "color": "white"
             }
            },
            "header": {
             "fill": {
              "color": "#C8D4E3"
             },
             "line": {
              "color": "white"
             }
            },
            "type": "table"
           }
          ]
         },
         "layout": {
          "annotationdefaults": {
           "arrowcolor": "#2a3f5f",
           "arrowhead": 0,
           "arrowwidth": 1
          },
          "autotypenumbers": "strict",
          "coloraxis": {
           "colorbar": {
            "outlinewidth": 0,
            "ticks": ""
           }
          },
          "colorscale": {
           "diverging": [
            [
             0,
             "#8e0152"
            ],
            [
             0.1,
             "#c51b7d"
            ],
            [
             0.2,
             "#de77ae"
            ],
            [
             0.3,
             "#f1b6da"
            ],
            [
             0.4,
             "#fde0ef"
            ],
            [
             0.5,
             "#f7f7f7"
            ],
            [
             0.6,
             "#e6f5d0"
            ],
            [
             0.7,
             "#b8e186"
            ],
            [
             0.8,
             "#7fbc41"
            ],
            [
             0.9,
             "#4d9221"
            ],
            [
             1,
             "#276419"
            ]
           ],
           "sequential": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ],
           "sequentialminus": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ]
          },
          "colorway": [
           "#636efa",
           "#EF553B",
           "#00cc96",
           "#ab63fa",
           "#FFA15A",
           "#19d3f3",
           "#FF6692",
           "#B6E880",
           "#FF97FF",
           "#FECB52"
          ],
          "font": {
           "color": "#2a3f5f"
          },
          "geo": {
           "bgcolor": "white",
           "lakecolor": "white",
           "landcolor": "#E5ECF6",
           "showlakes": true,
           "showland": true,
           "subunitcolor": "white"
          },
          "hoverlabel": {
           "align": "left"
          },
          "hovermode": "closest",
          "mapbox": {
           "style": "light"
          },
          "paper_bgcolor": "white",
          "plot_bgcolor": "#E5ECF6",
          "polar": {
           "angularaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "radialaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "scene": {
           "xaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "yaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "zaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           }
          },
          "shapedefaults": {
           "line": {
            "color": "#2a3f5f"
           }
          },
          "ternary": {
           "aaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "baxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "caxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "title": {
           "x": 0.05
          },
          "xaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          },
          "yaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          }
         }
        },
        "title": {
         "text": "Slice Plot"
        },
        "width": 2400,
        "xaxis": {
         "anchor": "y",
         "domain": [
          0,
          0.103125
         ],
         "title": {
          "text": "lr"
         },
         "type": "log"
        },
        "xaxis2": {
         "anchor": "y2",
         "domain": [
          0.128125,
          0.23124999999999998
         ],
         "title": {
          "text": "n_layers"
         }
        },
        "xaxis3": {
         "anchor": "y3",
         "domain": [
          0.25625,
          0.359375
         ],
         "title": {
          "text": "n_units_l0"
         }
        },
        "xaxis4": {
         "anchor": "y4",
         "domain": [
          0.38437499999999997,
          0.48749999999999993
         ],
         "title": {
          "text": "n_units_l1"
         }
        },
        "xaxis5": {
         "anchor": "y5",
         "domain": [
          0.5125,
          0.615625
         ],
         "title": {
          "text": "n_units_l2"
         }
        },
        "xaxis6": {
         "anchor": "y6",
         "domain": [
          0.640625,
          0.74375
         ],
         "title": {
          "text": "n_units_l3"
         }
        },
        "xaxis7": {
         "anchor": "y7",
         "domain": [
          0.76875,
          0.8718750000000001
         ],
         "title": {
          "text": "n_units_l4"
         }
        },
        "xaxis8": {
         "anchor": "y8",
         "categoryarray": [
          "sgd",
          "adam",
          "rmsprop"
         ],
         "categoryorder": "array",
         "domain": [
          0.8968750000000001,
          1
         ],
         "title": {
          "text": "optimizer"
         },
         "type": "category"
        },
        "yaxis": {
         "anchor": "x",
         "domain": [
          0,
          1
         ],
         "title": {
          "text": "Objective Value"
         }
        },
        "yaxis2": {
         "anchor": "x2",
         "domain": [
          0,
          1
         ],
         "matches": "y",
         "showticklabels": false
        },
        "yaxis3": {
         "anchor": "x3",
         "domain": [
          0,
          1
         ],
         "matches": "y",
         "showticklabels": false
        },
        "yaxis4": {
         "anchor": "x4",
         "domain": [
          0,
          1
         ],
         "matches": "y",
         "showticklabels": false
        },
        "yaxis5": {
         "anchor": "x5",
         "domain": [
          0,
          1
         ],
         "matches": "y",
         "showticklabels": false
        },
        "yaxis6": {
         "anchor": "x6",
         "domain": [
          0,
          1
         ],
         "matches": "y",
         "showticklabels": false
        },
        "yaxis7": {
         "anchor": "x7",
         "domain": [
          0,
          1
         ],
         "matches": "y",
         "showticklabels": false
        },
        "yaxis8": {
         "anchor": "x8",
         "domain": [
          0,
          1
         ],
         "matches": "y",
         "showticklabels": false
        }
       }
      }
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_slice(study)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "config": {
        "plotlyServerURL": "https://plot.ly"
       },
       "data": [
        {
         "cliponaxis": false,
         "hovertemplate": [
          "n_layers (IntDistribution): 0.03917666629655613<extra></extra>",
          "optimizer (CategoricalDistribution): 0.07096149207067703<extra></extra>",
          "lr (FloatDistribution): 0.18502616273346137<extra></extra>",
          "n_units_l0 (IntDistribution): 0.7048356788993054<extra></extra>"
         ],
         "marker": {
          "color": "rgb(66,146,198)"
         },
         "orientation": "h",
         "text": [
          "0.04",
          "0.07",
          "0.19",
          "0.70"
         ],
         "textposition": "outside",
         "type": "bar",
         "x": [
          0.03917666629655613,
          0.07096149207067703,
          0.18502616273346137,
          0.7048356788993054
         ],
         "y": [
          "n_layers",
          "optimizer",
          "lr",
          "n_units_l0"
         ]
        }
       ],
       "layout": {
        "showlegend": false,
        "template": {
         "data": {
          "bar": [
           {
            "error_x": {
             "color": "#2a3f5f"
            },
            "error_y": {
             "color": "#2a3f5f"
            },
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "bar"
           }
          ],
          "barpolar": [
           {
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "barpolar"
           }
          ],
          "carpet": [
           {
            "aaxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "baxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "type": "carpet"
           }
          ],
          "choropleth": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "choropleth"
           }
          ],
          "contour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "contour"
           }
          ],
          "contourcarpet": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "contourcarpet"
           }
          ],
          "heatmap": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmap"
           }
          ],
          "heatmapgl": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmapgl"
           }
          ],
          "histogram": [
           {
            "marker": {
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "histogram"
           }
          ],
          "histogram2d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2d"
           }
          ],
          "histogram2dcontour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2dcontour"
           }
          ],
          "mesh3d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "mesh3d"
           }
          ],
          "parcoords": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "parcoords"
           }
          ],
          "pie": [
           {
            "automargin": true,
            "type": "pie"
           }
          ],
          "scatter": [
           {
            "fillpattern": {
             "fillmode": "overlay",
             "size": 10,
             "solidity": 0.2
            },
            "type": "scatter"
           }
          ],
          "scatter3d": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatter3d"
           }
          ],
          "scattercarpet": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattercarpet"
           }
          ],
          "scattergeo": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergeo"
           }
          ],
          "scattergl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergl"
           }
          ],
          "scattermapbox": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattermapbox"
           }
          ],
          "scatterpolar": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolar"
           }
          ],
          "scatterpolargl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolargl"
           }
          ],
          "scatterternary": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterternary"
           }
          ],
          "surface": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "surface"
           }
          ],
          "table": [
           {
            "cells": {
             "fill": {
              "color": "#EBF0F8"
             },
             "line": {
              "color": "white"
             }
            },
            "header": {
             "fill": {
              "color": "#C8D4E3"
             },
             "line": {
              "color": "white"
             }
            },
            "type": "table"
           }
          ]
         },
         "layout": {
          "annotationdefaults": {
           "arrowcolor": "#2a3f5f",
           "arrowhead": 0,
           "arrowwidth": 1
          },
          "autotypenumbers": "strict",
          "coloraxis": {
           "colorbar": {
            "outlinewidth": 0,
            "ticks": ""
           }
          },
          "colorscale": {
           "diverging": [
            [
             0,
             "#8e0152"
            ],
            [
             0.1,
             "#c51b7d"
            ],
            [
             0.2,
             "#de77ae"
            ],
            [
             0.3,
             "#f1b6da"
            ],
            [
             0.4,
             "#fde0ef"
            ],
            [
             0.5,
             "#f7f7f7"
            ],
            [
             0.6,
             "#e6f5d0"
            ],
            [
             0.7,
             "#b8e186"
            ],
            [
             0.8,
             "#7fbc41"
            ],
            [
             0.9,
             "#4d9221"
            ],
            [
             1,
             "#276419"
            ]
           ],
           "sequential": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ],
           "sequentialminus": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ]
          },
          "colorway": [
           "#636efa",
           "#EF553B",
           "#00cc96",
           "#ab63fa",
           "#FFA15A",
           "#19d3f3",
           "#FF6692",
           "#B6E880",
           "#FF97FF",
           "#FECB52"
          ],
          "font": {
           "color": "#2a3f5f"
          },
          "geo": {
           "bgcolor": "white",
           "lakecolor": "white",
           "landcolor": "#E5ECF6",
           "showlakes": true,
           "showland": true,
           "subunitcolor": "white"
          },
          "hoverlabel": {
           "align": "left"
          },
          "hovermode": "closest",
          "mapbox": {
           "style": "light"
          },
          "paper_bgcolor": "white",
          "plot_bgcolor": "#E5ECF6",
          "polar": {
           "angularaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "radialaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "scene": {
           "xaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "yaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "zaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           }
          },
          "shapedefaults": {
           "line": {
            "color": "#2a3f5f"
           }
          },
          "ternary": {
           "aaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "baxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "caxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "title": {
           "x": 0.05
          },
          "xaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          },
          "yaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          }
         }
        },
        "title": {
         "text": "Hyperparameter Importances"
        },
        "xaxis": {
         "title": {
          "text": "Importance for Objective Value"
         }
        },
        "yaxis": {
         "title": {
          "text": "Hyperparameter"
         }
        }
       }
      }
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_param_importances(study)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\franz\\AppData\\Local\\Temp\\ipykernel_23396\\3430271029.py:1: ExperimentalWarning:\n",
      "\n",
      "plot_rank is experimental (supported from v3.2.0). The interface can change in the future.\n",
      "\n"
     ]
    },
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "config": {
        "plotlyServerURL": "https://plot.ly"
       },
       "data": [
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #0<br>n_layers: 2<br>n_units_l0: 101<br>n_units_l1: 48<br>lr: 0.007711398857368533<br>optimizer: adam<br><b>Objective Value: 0.13771341741085052</b>",
          "Trial #1<br>n_layers: 2<br>n_units_l0: 70<br>n_units_l1: 41<br>lr: 0.003012808172117605<br>optimizer: rmsprop<br><b>Objective Value: 0.12340866029262543</b>",
          "Trial #2<br>n_layers: 3<br>n_units_l0: 124<br>n_units_l1: 48<br>n_units_l2: 58<br>lr: 0.005442268439829251<br>optimizer: rmsprop<br><b>Objective Value: 0.1751943975687027</b>",
          "Trial #3<br>n_layers: 3<br>n_units_l0: 122<br>n_units_l1: 97<br>n_units_l2: 66<br>lr: 0.00013421360666602302<br>optimizer: adam<br><b>Objective Value: 0.11885558068752289</b>",
          "Trial #4<br>n_layers: 3<br>n_units_l0: 37<br>n_units_l1: 43<br>n_units_l2: 65<br>lr: 0.05696244653419064<br>optimizer: rmsprop<br><b>Objective Value: 0.9541876316070557</b>",
          "Trial #5<br>n_layers: 3<br>n_units_l0: 78<br>n_units_l1: 126<br>n_units_l2: 60<br>lr: 0.05752434764095089<br>optimizer: sgd<br><b>Objective Value: 0.09808095544576645</b>",
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #7<br>n_layers: 2<br>n_units_l0: 114<br>n_units_l1: 51<br>lr: 0.0023572885912189617<br>optimizer: sgd<br><b>Objective Value: 0.37900349497795105</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #9<br>n_layers: 1<br>n_units_l0: 38<br>lr: 0.08245600062993334<br>optimizer: adam<br><b>Objective Value: 0.6329582333564758</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(201, 232, 242)",
           "rgb(163, 211, 230)",
           "rgb(255, 255, 191)",
           "rgb(124, 179, 212)",
           "rgb(201, 34, 39)",
           "rgb(63, 99, 171)",
           "rgb(49, 54, 149)",
           "rgb(252, 165, 93)",
           "rgb(89, 141, 192)",
           "rgb(245, 118, 71)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          0.007711398857368533,
          0.003012808172117605,
          0.005442268439829251,
          0.00013421360666602302,
          0.05696244653419064,
          0.05752434764095089,
          0.00024040850246942775,
          0.0023572885912189617,
          0.0005769984149991314,
          0.08245600062993334,
          0.00001785199667793032,
          0.00037332091663808175,
          0.018851800842778852,
          0.022091351249891724,
          0.0009877185587115079
         ],
         "xaxis": "x",
         "y": [
          0.007711398857368533,
          0.003012808172117605,
          0.005442268439829251,
          0.00013421360666602302,
          0.05696244653419064,
          0.05752434764095089,
          0.00024040850246942775,
          0.0023572885912189617,
          0.0005769984149991314,
          0.08245600062993334,
          0.00001785199667793032,
          0.00037332091663808175,
          0.018851800842778852,
          0.022091351249891724,
          0.0009877185587115079
         ],
         "yaxis": "y"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #0<br>n_layers: 2<br>n_units_l0: 101<br>n_units_l1: 48<br>lr: 0.007711398857368533<br>optimizer: adam<br><b>Objective Value: 0.13771341741085052</b>",
          "Trial #1<br>n_layers: 2<br>n_units_l0: 70<br>n_units_l1: 41<br>lr: 0.003012808172117605<br>optimizer: rmsprop<br><b>Objective Value: 0.12340866029262543</b>",
          "Trial #2<br>n_layers: 3<br>n_units_l0: 124<br>n_units_l1: 48<br>n_units_l2: 58<br>lr: 0.005442268439829251<br>optimizer: rmsprop<br><b>Objective Value: 0.1751943975687027</b>",
          "Trial #3<br>n_layers: 3<br>n_units_l0: 122<br>n_units_l1: 97<br>n_units_l2: 66<br>lr: 0.00013421360666602302<br>optimizer: adam<br><b>Objective Value: 0.11885558068752289</b>",
          "Trial #4<br>n_layers: 3<br>n_units_l0: 37<br>n_units_l1: 43<br>n_units_l2: 65<br>lr: 0.05696244653419064<br>optimizer: rmsprop<br><b>Objective Value: 0.9541876316070557</b>",
          "Trial #5<br>n_layers: 3<br>n_units_l0: 78<br>n_units_l1: 126<br>n_units_l2: 60<br>lr: 0.05752434764095089<br>optimizer: sgd<br><b>Objective Value: 0.09808095544576645</b>",
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #7<br>n_layers: 2<br>n_units_l0: 114<br>n_units_l1: 51<br>lr: 0.0023572885912189617<br>optimizer: sgd<br><b>Objective Value: 0.37900349497795105</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #9<br>n_layers: 1<br>n_units_l0: 38<br>lr: 0.08245600062993334<br>optimizer: adam<br><b>Objective Value: 0.6329582333564758</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(201, 232, 242)",
           "rgb(163, 211, 230)",
           "rgb(255, 255, 191)",
           "rgb(124, 179, 212)",
           "rgb(201, 34, 39)",
           "rgb(63, 99, 171)",
           "rgb(49, 54, 149)",
           "rgb(252, 165, 93)",
           "rgb(89, 141, 192)",
           "rgb(245, 118, 71)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          0.007711398857368533,
          0.003012808172117605,
          0.005442268439829251,
          0.00013421360666602302,
          0.05696244653419064,
          0.05752434764095089,
          0.00024040850246942775,
          0.0023572885912189617,
          0.0005769984149991314,
          0.08245600062993334,
          0.00001785199667793032,
          0.00037332091663808175,
          0.018851800842778852,
          0.022091351249891724,
          0.0009877185587115079
         ],
         "xaxis": "x9",
         "y": [
          2,
          2,
          3,
          3,
          3,
          3,
          4,
          2,
          5,
          1,
          5,
          4,
          4,
          4,
          4
         ],
         "yaxis": "y9"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #0<br>n_layers: 2<br>n_units_l0: 101<br>n_units_l1: 48<br>lr: 0.007711398857368533<br>optimizer: adam<br><b>Objective Value: 0.13771341741085052</b>",
          "Trial #1<br>n_layers: 2<br>n_units_l0: 70<br>n_units_l1: 41<br>lr: 0.003012808172117605<br>optimizer: rmsprop<br><b>Objective Value: 0.12340866029262543</b>",
          "Trial #2<br>n_layers: 3<br>n_units_l0: 124<br>n_units_l1: 48<br>n_units_l2: 58<br>lr: 0.005442268439829251<br>optimizer: rmsprop<br><b>Objective Value: 0.1751943975687027</b>",
          "Trial #3<br>n_layers: 3<br>n_units_l0: 122<br>n_units_l1: 97<br>n_units_l2: 66<br>lr: 0.00013421360666602302<br>optimizer: adam<br><b>Objective Value: 0.11885558068752289</b>",
          "Trial #4<br>n_layers: 3<br>n_units_l0: 37<br>n_units_l1: 43<br>n_units_l2: 65<br>lr: 0.05696244653419064<br>optimizer: rmsprop<br><b>Objective Value: 0.9541876316070557</b>",
          "Trial #5<br>n_layers: 3<br>n_units_l0: 78<br>n_units_l1: 126<br>n_units_l2: 60<br>lr: 0.05752434764095089<br>optimizer: sgd<br><b>Objective Value: 0.09808095544576645</b>",
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #7<br>n_layers: 2<br>n_units_l0: 114<br>n_units_l1: 51<br>lr: 0.0023572885912189617<br>optimizer: sgd<br><b>Objective Value: 0.37900349497795105</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #9<br>n_layers: 1<br>n_units_l0: 38<br>lr: 0.08245600062993334<br>optimizer: adam<br><b>Objective Value: 0.6329582333564758</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(201, 232, 242)",
           "rgb(163, 211, 230)",
           "rgb(255, 255, 191)",
           "rgb(124, 179, 212)",
           "rgb(201, 34, 39)",
           "rgb(63, 99, 171)",
           "rgb(49, 54, 149)",
           "rgb(252, 165, 93)",
           "rgb(89, 141, 192)",
           "rgb(245, 118, 71)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          0.007711398857368533,
          0.003012808172117605,
          0.005442268439829251,
          0.00013421360666602302,
          0.05696244653419064,
          0.05752434764095089,
          0.00024040850246942775,
          0.0023572885912189617,
          0.0005769984149991314,
          0.08245600062993334,
          0.00001785199667793032,
          0.00037332091663808175,
          0.018851800842778852,
          0.022091351249891724,
          0.0009877185587115079
         ],
         "xaxis": "x17",
         "y": [
          101,
          70,
          124,
          122,
          37,
          78,
          119,
          114,
          116,
          38,
          99,
          72,
          88,
          55,
          85
         ],
         "yaxis": "y17"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #0<br>n_layers: 2<br>n_units_l0: 101<br>n_units_l1: 48<br>lr: 0.007711398857368533<br>optimizer: adam<br><b>Objective Value: 0.13771341741085052</b>",
          "Trial #1<br>n_layers: 2<br>n_units_l0: 70<br>n_units_l1: 41<br>lr: 0.003012808172117605<br>optimizer: rmsprop<br><b>Objective Value: 0.12340866029262543</b>",
          "Trial #2<br>n_layers: 3<br>n_units_l0: 124<br>n_units_l1: 48<br>n_units_l2: 58<br>lr: 0.005442268439829251<br>optimizer: rmsprop<br><b>Objective Value: 0.1751943975687027</b>",
          "Trial #3<br>n_layers: 3<br>n_units_l0: 122<br>n_units_l1: 97<br>n_units_l2: 66<br>lr: 0.00013421360666602302<br>optimizer: adam<br><b>Objective Value: 0.11885558068752289</b>",
          "Trial #4<br>n_layers: 3<br>n_units_l0: 37<br>n_units_l1: 43<br>n_units_l2: 65<br>lr: 0.05696244653419064<br>optimizer: rmsprop<br><b>Objective Value: 0.9541876316070557</b>",
          "Trial #5<br>n_layers: 3<br>n_units_l0: 78<br>n_units_l1: 126<br>n_units_l2: 60<br>lr: 0.05752434764095089<br>optimizer: sgd<br><b>Objective Value: 0.09808095544576645</b>",
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #7<br>n_layers: 2<br>n_units_l0: 114<br>n_units_l1: 51<br>lr: 0.0023572885912189617<br>optimizer: sgd<br><b>Objective Value: 0.37900349497795105</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(201, 232, 242)",
           "rgb(163, 211, 230)",
           "rgb(255, 255, 191)",
           "rgb(124, 179, 212)",
           "rgb(201, 34, 39)",
           "rgb(63, 99, 171)",
           "rgb(49, 54, 149)",
           "rgb(252, 165, 93)",
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          0.007711398857368533,
          0.003012808172117605,
          0.005442268439829251,
          0.00013421360666602302,
          0.05696244653419064,
          0.05752434764095089,
          0.00024040850246942775,
          0.0023572885912189617,
          0.0005769984149991314,
          0.00001785199667793032,
          0.00037332091663808175,
          0.018851800842778852,
          0.022091351249891724,
          0.0009877185587115079
         ],
         "xaxis": "x25",
         "y": [
          48,
          41,
          48,
          97,
          43,
          126,
          86,
          51,
          52,
          82,
          127,
          125,
          104,
          71
         ],
         "yaxis": "y25"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #2<br>n_layers: 3<br>n_units_l0: 124<br>n_units_l1: 48<br>n_units_l2: 58<br>lr: 0.005442268439829251<br>optimizer: rmsprop<br><b>Objective Value: 0.1751943975687027</b>",
          "Trial #3<br>n_layers: 3<br>n_units_l0: 122<br>n_units_l1: 97<br>n_units_l2: 66<br>lr: 0.00013421360666602302<br>optimizer: adam<br><b>Objective Value: 0.11885558068752289</b>",
          "Trial #4<br>n_layers: 3<br>n_units_l0: 37<br>n_units_l1: 43<br>n_units_l2: 65<br>lr: 0.05696244653419064<br>optimizer: rmsprop<br><b>Objective Value: 0.9541876316070557</b>",
          "Trial #5<br>n_layers: 3<br>n_units_l0: 78<br>n_units_l1: 126<br>n_units_l2: 60<br>lr: 0.05752434764095089<br>optimizer: sgd<br><b>Objective Value: 0.09808095544576645</b>",
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(255, 255, 191)",
           "rgb(124, 179, 212)",
           "rgb(201, 34, 39)",
           "rgb(63, 99, 171)",
           "rgb(49, 54, 149)",
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          0.005442268439829251,
          0.00013421360666602302,
          0.05696244653419064,
          0.05752434764095089,
          0.00024040850246942775,
          0.0005769984149991314,
          0.00001785199667793032,
          0.00037332091663808175,
          0.018851800842778852,
          0.022091351249891724,
          0.0009877185587115079
         ],
         "xaxis": "x33",
         "y": [
          58,
          66,
          65,
          60,
          112,
          75,
          122,
          113,
          33,
          96,
          96
         ],
         "yaxis": "y33"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(49, 54, 149)",
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          0.00024040850246942775,
          0.0005769984149991314,
          0.00001785199667793032,
          0.00037332091663808175,
          0.018851800842778852,
          0.022091351249891724,
          0.0009877185587115079
         ],
         "xaxis": "x41",
         "y": [
          91,
          33,
          110,
          81,
          127,
          77,
          81
         ],
         "yaxis": "y41"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>"
         ],
         "marker": {
          "color": [
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          0.0005769984149991314,
          0.00001785199667793032
         ],
         "xaxis": "x49",
         "y": [
          99,
          32
         ],
         "yaxis": "y49"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #0<br>n_layers: 2<br>n_units_l0: 101<br>n_units_l1: 48<br>lr: 0.007711398857368533<br>optimizer: adam<br><b>Objective Value: 0.13771341741085052</b>",
          "Trial #1<br>n_layers: 2<br>n_units_l0: 70<br>n_units_l1: 41<br>lr: 0.003012808172117605<br>optimizer: rmsprop<br><b>Objective Value: 0.12340866029262543</b>",
          "Trial #2<br>n_layers: 3<br>n_units_l0: 124<br>n_units_l1: 48<br>n_units_l2: 58<br>lr: 0.005442268439829251<br>optimizer: rmsprop<br><b>Objective Value: 0.1751943975687027</b>",
          "Trial #3<br>n_layers: 3<br>n_units_l0: 122<br>n_units_l1: 97<br>n_units_l2: 66<br>lr: 0.00013421360666602302<br>optimizer: adam<br><b>Objective Value: 0.11885558068752289</b>",
          "Trial #4<br>n_layers: 3<br>n_units_l0: 37<br>n_units_l1: 43<br>n_units_l2: 65<br>lr: 0.05696244653419064<br>optimizer: rmsprop<br><b>Objective Value: 0.9541876316070557</b>",
          "Trial #5<br>n_layers: 3<br>n_units_l0: 78<br>n_units_l1: 126<br>n_units_l2: 60<br>lr: 0.05752434764095089<br>optimizer: sgd<br><b>Objective Value: 0.09808095544576645</b>",
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #7<br>n_layers: 2<br>n_units_l0: 114<br>n_units_l1: 51<br>lr: 0.0023572885912189617<br>optimizer: sgd<br><b>Objective Value: 0.37900349497795105</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #9<br>n_layers: 1<br>n_units_l0: 38<br>lr: 0.08245600062993334<br>optimizer: adam<br><b>Objective Value: 0.6329582333564758</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(201, 232, 242)",
           "rgb(163, 211, 230)",
           "rgb(255, 255, 191)",
           "rgb(124, 179, 212)",
           "rgb(201, 34, 39)",
           "rgb(63, 99, 171)",
           "rgb(49, 54, 149)",
           "rgb(252, 165, 93)",
           "rgb(89, 141, 192)",
           "rgb(245, 118, 71)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          0.007711398857368533,
          0.003012808172117605,
          0.005442268439829251,
          0.00013421360666602302,
          0.05696244653419064,
          0.05752434764095089,
          0.00024040850246942775,
          0.0023572885912189617,
          0.0005769984149991314,
          0.08245600062993334,
          0.00001785199667793032,
          0.00037332091663808175,
          0.018851800842778852,
          0.022091351249891724,
          0.0009877185587115079
         ],
         "xaxis": "x57",
         "y": [
          "adam",
          "rmsprop",
          "rmsprop",
          "adam",
          "rmsprop",
          "sgd",
          "adam",
          "sgd",
          "adam",
          "adam",
          "adam",
          "sgd",
          "sgd",
          "sgd",
          "sgd"
         ],
         "yaxis": "y57"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #0<br>n_layers: 2<br>n_units_l0: 101<br>n_units_l1: 48<br>lr: 0.007711398857368533<br>optimizer: adam<br><b>Objective Value: 0.13771341741085052</b>",
          "Trial #1<br>n_layers: 2<br>n_units_l0: 70<br>n_units_l1: 41<br>lr: 0.003012808172117605<br>optimizer: rmsprop<br><b>Objective Value: 0.12340866029262543</b>",
          "Trial #2<br>n_layers: 3<br>n_units_l0: 124<br>n_units_l1: 48<br>n_units_l2: 58<br>lr: 0.005442268439829251<br>optimizer: rmsprop<br><b>Objective Value: 0.1751943975687027</b>",
          "Trial #3<br>n_layers: 3<br>n_units_l0: 122<br>n_units_l1: 97<br>n_units_l2: 66<br>lr: 0.00013421360666602302<br>optimizer: adam<br><b>Objective Value: 0.11885558068752289</b>",
          "Trial #4<br>n_layers: 3<br>n_units_l0: 37<br>n_units_l1: 43<br>n_units_l2: 65<br>lr: 0.05696244653419064<br>optimizer: rmsprop<br><b>Objective Value: 0.9541876316070557</b>",
          "Trial #5<br>n_layers: 3<br>n_units_l0: 78<br>n_units_l1: 126<br>n_units_l2: 60<br>lr: 0.05752434764095089<br>optimizer: sgd<br><b>Objective Value: 0.09808095544576645</b>",
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #7<br>n_layers: 2<br>n_units_l0: 114<br>n_units_l1: 51<br>lr: 0.0023572885912189617<br>optimizer: sgd<br><b>Objective Value: 0.37900349497795105</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #9<br>n_layers: 1<br>n_units_l0: 38<br>lr: 0.08245600062993334<br>optimizer: adam<br><b>Objective Value: 0.6329582333564758</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(201, 232, 242)",
           "rgb(163, 211, 230)",
           "rgb(255, 255, 191)",
           "rgb(124, 179, 212)",
           "rgb(201, 34, 39)",
           "rgb(63, 99, 171)",
           "rgb(49, 54, 149)",
           "rgb(252, 165, 93)",
           "rgb(89, 141, 192)",
           "rgb(245, 118, 71)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          2,
          2,
          3,
          3,
          3,
          3,
          4,
          2,
          5,
          1,
          5,
          4,
          4,
          4,
          4
         ],
         "xaxis": "x2",
         "y": [
          0.007711398857368533,
          0.003012808172117605,
          0.005442268439829251,
          0.00013421360666602302,
          0.05696244653419064,
          0.05752434764095089,
          0.00024040850246942775,
          0.0023572885912189617,
          0.0005769984149991314,
          0.08245600062993334,
          0.00001785199667793032,
          0.00037332091663808175,
          0.018851800842778852,
          0.022091351249891724,
          0.0009877185587115079
         ],
         "yaxis": "y2"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #0<br>n_layers: 2<br>n_units_l0: 101<br>n_units_l1: 48<br>lr: 0.007711398857368533<br>optimizer: adam<br><b>Objective Value: 0.13771341741085052</b>",
          "Trial #1<br>n_layers: 2<br>n_units_l0: 70<br>n_units_l1: 41<br>lr: 0.003012808172117605<br>optimizer: rmsprop<br><b>Objective Value: 0.12340866029262543</b>",
          "Trial #2<br>n_layers: 3<br>n_units_l0: 124<br>n_units_l1: 48<br>n_units_l2: 58<br>lr: 0.005442268439829251<br>optimizer: rmsprop<br><b>Objective Value: 0.1751943975687027</b>",
          "Trial #3<br>n_layers: 3<br>n_units_l0: 122<br>n_units_l1: 97<br>n_units_l2: 66<br>lr: 0.00013421360666602302<br>optimizer: adam<br><b>Objective Value: 0.11885558068752289</b>",
          "Trial #4<br>n_layers: 3<br>n_units_l0: 37<br>n_units_l1: 43<br>n_units_l2: 65<br>lr: 0.05696244653419064<br>optimizer: rmsprop<br><b>Objective Value: 0.9541876316070557</b>",
          "Trial #5<br>n_layers: 3<br>n_units_l0: 78<br>n_units_l1: 126<br>n_units_l2: 60<br>lr: 0.05752434764095089<br>optimizer: sgd<br><b>Objective Value: 0.09808095544576645</b>",
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #7<br>n_layers: 2<br>n_units_l0: 114<br>n_units_l1: 51<br>lr: 0.0023572885912189617<br>optimizer: sgd<br><b>Objective Value: 0.37900349497795105</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #9<br>n_layers: 1<br>n_units_l0: 38<br>lr: 0.08245600062993334<br>optimizer: adam<br><b>Objective Value: 0.6329582333564758</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(201, 232, 242)",
           "rgb(163, 211, 230)",
           "rgb(255, 255, 191)",
           "rgb(124, 179, 212)",
           "rgb(201, 34, 39)",
           "rgb(63, 99, 171)",
           "rgb(49, 54, 149)",
           "rgb(252, 165, 93)",
           "rgb(89, 141, 192)",
           "rgb(245, 118, 71)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          2,
          2,
          3,
          3,
          3,
          3,
          4,
          2,
          5,
          1,
          5,
          4,
          4,
          4,
          4
         ],
         "xaxis": "x10",
         "y": [
          2,
          2,
          3,
          3,
          3,
          3,
          4,
          2,
          5,
          1,
          5,
          4,
          4,
          4,
          4
         ],
         "yaxis": "y10"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #0<br>n_layers: 2<br>n_units_l0: 101<br>n_units_l1: 48<br>lr: 0.007711398857368533<br>optimizer: adam<br><b>Objective Value: 0.13771341741085052</b>",
          "Trial #1<br>n_layers: 2<br>n_units_l0: 70<br>n_units_l1: 41<br>lr: 0.003012808172117605<br>optimizer: rmsprop<br><b>Objective Value: 0.12340866029262543</b>",
          "Trial #2<br>n_layers: 3<br>n_units_l0: 124<br>n_units_l1: 48<br>n_units_l2: 58<br>lr: 0.005442268439829251<br>optimizer: rmsprop<br><b>Objective Value: 0.1751943975687027</b>",
          "Trial #3<br>n_layers: 3<br>n_units_l0: 122<br>n_units_l1: 97<br>n_units_l2: 66<br>lr: 0.00013421360666602302<br>optimizer: adam<br><b>Objective Value: 0.11885558068752289</b>",
          "Trial #4<br>n_layers: 3<br>n_units_l0: 37<br>n_units_l1: 43<br>n_units_l2: 65<br>lr: 0.05696244653419064<br>optimizer: rmsprop<br><b>Objective Value: 0.9541876316070557</b>",
          "Trial #5<br>n_layers: 3<br>n_units_l0: 78<br>n_units_l1: 126<br>n_units_l2: 60<br>lr: 0.05752434764095089<br>optimizer: sgd<br><b>Objective Value: 0.09808095544576645</b>",
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #7<br>n_layers: 2<br>n_units_l0: 114<br>n_units_l1: 51<br>lr: 0.0023572885912189617<br>optimizer: sgd<br><b>Objective Value: 0.37900349497795105</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #9<br>n_layers: 1<br>n_units_l0: 38<br>lr: 0.08245600062993334<br>optimizer: adam<br><b>Objective Value: 0.6329582333564758</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(201, 232, 242)",
           "rgb(163, 211, 230)",
           "rgb(255, 255, 191)",
           "rgb(124, 179, 212)",
           "rgb(201, 34, 39)",
           "rgb(63, 99, 171)",
           "rgb(49, 54, 149)",
           "rgb(252, 165, 93)",
           "rgb(89, 141, 192)",
           "rgb(245, 118, 71)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          2,
          2,
          3,
          3,
          3,
          3,
          4,
          2,
          5,
          1,
          5,
          4,
          4,
          4,
          4
         ],
         "xaxis": "x18",
         "y": [
          101,
          70,
          124,
          122,
          37,
          78,
          119,
          114,
          116,
          38,
          99,
          72,
          88,
          55,
          85
         ],
         "yaxis": "y18"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #0<br>n_layers: 2<br>n_units_l0: 101<br>n_units_l1: 48<br>lr: 0.007711398857368533<br>optimizer: adam<br><b>Objective Value: 0.13771341741085052</b>",
          "Trial #1<br>n_layers: 2<br>n_units_l0: 70<br>n_units_l1: 41<br>lr: 0.003012808172117605<br>optimizer: rmsprop<br><b>Objective Value: 0.12340866029262543</b>",
          "Trial #2<br>n_layers: 3<br>n_units_l0: 124<br>n_units_l1: 48<br>n_units_l2: 58<br>lr: 0.005442268439829251<br>optimizer: rmsprop<br><b>Objective Value: 0.1751943975687027</b>",
          "Trial #3<br>n_layers: 3<br>n_units_l0: 122<br>n_units_l1: 97<br>n_units_l2: 66<br>lr: 0.00013421360666602302<br>optimizer: adam<br><b>Objective Value: 0.11885558068752289</b>",
          "Trial #4<br>n_layers: 3<br>n_units_l0: 37<br>n_units_l1: 43<br>n_units_l2: 65<br>lr: 0.05696244653419064<br>optimizer: rmsprop<br><b>Objective Value: 0.9541876316070557</b>",
          "Trial #5<br>n_layers: 3<br>n_units_l0: 78<br>n_units_l1: 126<br>n_units_l2: 60<br>lr: 0.05752434764095089<br>optimizer: sgd<br><b>Objective Value: 0.09808095544576645</b>",
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #7<br>n_layers: 2<br>n_units_l0: 114<br>n_units_l1: 51<br>lr: 0.0023572885912189617<br>optimizer: sgd<br><b>Objective Value: 0.37900349497795105</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(201, 232, 242)",
           "rgb(163, 211, 230)",
           "rgb(255, 255, 191)",
           "rgb(124, 179, 212)",
           "rgb(201, 34, 39)",
           "rgb(63, 99, 171)",
           "rgb(49, 54, 149)",
           "rgb(252, 165, 93)",
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          2,
          2,
          3,
          3,
          3,
          3,
          4,
          2,
          5,
          5,
          4,
          4,
          4,
          4
         ],
         "xaxis": "x26",
         "y": [
          48,
          41,
          48,
          97,
          43,
          126,
          86,
          51,
          52,
          82,
          127,
          125,
          104,
          71
         ],
         "yaxis": "y26"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #2<br>n_layers: 3<br>n_units_l0: 124<br>n_units_l1: 48<br>n_units_l2: 58<br>lr: 0.005442268439829251<br>optimizer: rmsprop<br><b>Objective Value: 0.1751943975687027</b>",
          "Trial #3<br>n_layers: 3<br>n_units_l0: 122<br>n_units_l1: 97<br>n_units_l2: 66<br>lr: 0.00013421360666602302<br>optimizer: adam<br><b>Objective Value: 0.11885558068752289</b>",
          "Trial #4<br>n_layers: 3<br>n_units_l0: 37<br>n_units_l1: 43<br>n_units_l2: 65<br>lr: 0.05696244653419064<br>optimizer: rmsprop<br><b>Objective Value: 0.9541876316070557</b>",
          "Trial #5<br>n_layers: 3<br>n_units_l0: 78<br>n_units_l1: 126<br>n_units_l2: 60<br>lr: 0.05752434764095089<br>optimizer: sgd<br><b>Objective Value: 0.09808095544576645</b>",
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(255, 255, 191)",
           "rgb(124, 179, 212)",
           "rgb(201, 34, 39)",
           "rgb(63, 99, 171)",
           "rgb(49, 54, 149)",
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          3,
          3,
          3,
          3,
          4,
          5,
          5,
          4,
          4,
          4,
          4
         ],
         "xaxis": "x34",
         "y": [
          58,
          66,
          65,
          60,
          112,
          75,
          122,
          113,
          33,
          96,
          96
         ],
         "yaxis": "y34"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(49, 54, 149)",
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          4,
          5,
          5,
          4,
          4,
          4,
          4
         ],
         "xaxis": "x42",
         "y": [
          91,
          33,
          110,
          81,
          127,
          77,
          81
         ],
         "yaxis": "y42"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>"
         ],
         "marker": {
          "color": [
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          5,
          5
         ],
         "xaxis": "x50",
         "y": [
          99,
          32
         ],
         "yaxis": "y50"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #0<br>n_layers: 2<br>n_units_l0: 101<br>n_units_l1: 48<br>lr: 0.007711398857368533<br>optimizer: adam<br><b>Objective Value: 0.13771341741085052</b>",
          "Trial #1<br>n_layers: 2<br>n_units_l0: 70<br>n_units_l1: 41<br>lr: 0.003012808172117605<br>optimizer: rmsprop<br><b>Objective Value: 0.12340866029262543</b>",
          "Trial #2<br>n_layers: 3<br>n_units_l0: 124<br>n_units_l1: 48<br>n_units_l2: 58<br>lr: 0.005442268439829251<br>optimizer: rmsprop<br><b>Objective Value: 0.1751943975687027</b>",
          "Trial #3<br>n_layers: 3<br>n_units_l0: 122<br>n_units_l1: 97<br>n_units_l2: 66<br>lr: 0.00013421360666602302<br>optimizer: adam<br><b>Objective Value: 0.11885558068752289</b>",
          "Trial #4<br>n_layers: 3<br>n_units_l0: 37<br>n_units_l1: 43<br>n_units_l2: 65<br>lr: 0.05696244653419064<br>optimizer: rmsprop<br><b>Objective Value: 0.9541876316070557</b>",
          "Trial #5<br>n_layers: 3<br>n_units_l0: 78<br>n_units_l1: 126<br>n_units_l2: 60<br>lr: 0.05752434764095089<br>optimizer: sgd<br><b>Objective Value: 0.09808095544576645</b>",
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #7<br>n_layers: 2<br>n_units_l0: 114<br>n_units_l1: 51<br>lr: 0.0023572885912189617<br>optimizer: sgd<br><b>Objective Value: 0.37900349497795105</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #9<br>n_layers: 1<br>n_units_l0: 38<br>lr: 0.08245600062993334<br>optimizer: adam<br><b>Objective Value: 0.6329582333564758</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(201, 232, 242)",
           "rgb(163, 211, 230)",
           "rgb(255, 255, 191)",
           "rgb(124, 179, 212)",
           "rgb(201, 34, 39)",
           "rgb(63, 99, 171)",
           "rgb(49, 54, 149)",
           "rgb(252, 165, 93)",
           "rgb(89, 141, 192)",
           "rgb(245, 118, 71)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          2,
          2,
          3,
          3,
          3,
          3,
          4,
          2,
          5,
          1,
          5,
          4,
          4,
          4,
          4
         ],
         "xaxis": "x58",
         "y": [
          "adam",
          "rmsprop",
          "rmsprop",
          "adam",
          "rmsprop",
          "sgd",
          "adam",
          "sgd",
          "adam",
          "adam",
          "adam",
          "sgd",
          "sgd",
          "sgd",
          "sgd"
         ],
         "yaxis": "y58"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #0<br>n_layers: 2<br>n_units_l0: 101<br>n_units_l1: 48<br>lr: 0.007711398857368533<br>optimizer: adam<br><b>Objective Value: 0.13771341741085052</b>",
          "Trial #1<br>n_layers: 2<br>n_units_l0: 70<br>n_units_l1: 41<br>lr: 0.003012808172117605<br>optimizer: rmsprop<br><b>Objective Value: 0.12340866029262543</b>",
          "Trial #2<br>n_layers: 3<br>n_units_l0: 124<br>n_units_l1: 48<br>n_units_l2: 58<br>lr: 0.005442268439829251<br>optimizer: rmsprop<br><b>Objective Value: 0.1751943975687027</b>",
          "Trial #3<br>n_layers: 3<br>n_units_l0: 122<br>n_units_l1: 97<br>n_units_l2: 66<br>lr: 0.00013421360666602302<br>optimizer: adam<br><b>Objective Value: 0.11885558068752289</b>",
          "Trial #4<br>n_layers: 3<br>n_units_l0: 37<br>n_units_l1: 43<br>n_units_l2: 65<br>lr: 0.05696244653419064<br>optimizer: rmsprop<br><b>Objective Value: 0.9541876316070557</b>",
          "Trial #5<br>n_layers: 3<br>n_units_l0: 78<br>n_units_l1: 126<br>n_units_l2: 60<br>lr: 0.05752434764095089<br>optimizer: sgd<br><b>Objective Value: 0.09808095544576645</b>",
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #7<br>n_layers: 2<br>n_units_l0: 114<br>n_units_l1: 51<br>lr: 0.0023572885912189617<br>optimizer: sgd<br><b>Objective Value: 0.37900349497795105</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #9<br>n_layers: 1<br>n_units_l0: 38<br>lr: 0.08245600062993334<br>optimizer: adam<br><b>Objective Value: 0.6329582333564758</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(201, 232, 242)",
           "rgb(163, 211, 230)",
           "rgb(255, 255, 191)",
           "rgb(124, 179, 212)",
           "rgb(201, 34, 39)",
           "rgb(63, 99, 171)",
           "rgb(49, 54, 149)",
           "rgb(252, 165, 93)",
           "rgb(89, 141, 192)",
           "rgb(245, 118, 71)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          101,
          70,
          124,
          122,
          37,
          78,
          119,
          114,
          116,
          38,
          99,
          72,
          88,
          55,
          85
         ],
         "xaxis": "x3",
         "y": [
          0.007711398857368533,
          0.003012808172117605,
          0.005442268439829251,
          0.00013421360666602302,
          0.05696244653419064,
          0.05752434764095089,
          0.00024040850246942775,
          0.0023572885912189617,
          0.0005769984149991314,
          0.08245600062993334,
          0.00001785199667793032,
          0.00037332091663808175,
          0.018851800842778852,
          0.022091351249891724,
          0.0009877185587115079
         ],
         "yaxis": "y3"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #0<br>n_layers: 2<br>n_units_l0: 101<br>n_units_l1: 48<br>lr: 0.007711398857368533<br>optimizer: adam<br><b>Objective Value: 0.13771341741085052</b>",
          "Trial #1<br>n_layers: 2<br>n_units_l0: 70<br>n_units_l1: 41<br>lr: 0.003012808172117605<br>optimizer: rmsprop<br><b>Objective Value: 0.12340866029262543</b>",
          "Trial #2<br>n_layers: 3<br>n_units_l0: 124<br>n_units_l1: 48<br>n_units_l2: 58<br>lr: 0.005442268439829251<br>optimizer: rmsprop<br><b>Objective Value: 0.1751943975687027</b>",
          "Trial #3<br>n_layers: 3<br>n_units_l0: 122<br>n_units_l1: 97<br>n_units_l2: 66<br>lr: 0.00013421360666602302<br>optimizer: adam<br><b>Objective Value: 0.11885558068752289</b>",
          "Trial #4<br>n_layers: 3<br>n_units_l0: 37<br>n_units_l1: 43<br>n_units_l2: 65<br>lr: 0.05696244653419064<br>optimizer: rmsprop<br><b>Objective Value: 0.9541876316070557</b>",
          "Trial #5<br>n_layers: 3<br>n_units_l0: 78<br>n_units_l1: 126<br>n_units_l2: 60<br>lr: 0.05752434764095089<br>optimizer: sgd<br><b>Objective Value: 0.09808095544576645</b>",
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #7<br>n_layers: 2<br>n_units_l0: 114<br>n_units_l1: 51<br>lr: 0.0023572885912189617<br>optimizer: sgd<br><b>Objective Value: 0.37900349497795105</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #9<br>n_layers: 1<br>n_units_l0: 38<br>lr: 0.08245600062993334<br>optimizer: adam<br><b>Objective Value: 0.6329582333564758</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(201, 232, 242)",
           "rgb(163, 211, 230)",
           "rgb(255, 255, 191)",
           "rgb(124, 179, 212)",
           "rgb(201, 34, 39)",
           "rgb(63, 99, 171)",
           "rgb(49, 54, 149)",
           "rgb(252, 165, 93)",
           "rgb(89, 141, 192)",
           "rgb(245, 118, 71)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          101,
          70,
          124,
          122,
          37,
          78,
          119,
          114,
          116,
          38,
          99,
          72,
          88,
          55,
          85
         ],
         "xaxis": "x11",
         "y": [
          2,
          2,
          3,
          3,
          3,
          3,
          4,
          2,
          5,
          1,
          5,
          4,
          4,
          4,
          4
         ],
         "yaxis": "y11"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #0<br>n_layers: 2<br>n_units_l0: 101<br>n_units_l1: 48<br>lr: 0.007711398857368533<br>optimizer: adam<br><b>Objective Value: 0.13771341741085052</b>",
          "Trial #1<br>n_layers: 2<br>n_units_l0: 70<br>n_units_l1: 41<br>lr: 0.003012808172117605<br>optimizer: rmsprop<br><b>Objective Value: 0.12340866029262543</b>",
          "Trial #2<br>n_layers: 3<br>n_units_l0: 124<br>n_units_l1: 48<br>n_units_l2: 58<br>lr: 0.005442268439829251<br>optimizer: rmsprop<br><b>Objective Value: 0.1751943975687027</b>",
          "Trial #3<br>n_layers: 3<br>n_units_l0: 122<br>n_units_l1: 97<br>n_units_l2: 66<br>lr: 0.00013421360666602302<br>optimizer: adam<br><b>Objective Value: 0.11885558068752289</b>",
          "Trial #4<br>n_layers: 3<br>n_units_l0: 37<br>n_units_l1: 43<br>n_units_l2: 65<br>lr: 0.05696244653419064<br>optimizer: rmsprop<br><b>Objective Value: 0.9541876316070557</b>",
          "Trial #5<br>n_layers: 3<br>n_units_l0: 78<br>n_units_l1: 126<br>n_units_l2: 60<br>lr: 0.05752434764095089<br>optimizer: sgd<br><b>Objective Value: 0.09808095544576645</b>",
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #7<br>n_layers: 2<br>n_units_l0: 114<br>n_units_l1: 51<br>lr: 0.0023572885912189617<br>optimizer: sgd<br><b>Objective Value: 0.37900349497795105</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #9<br>n_layers: 1<br>n_units_l0: 38<br>lr: 0.08245600062993334<br>optimizer: adam<br><b>Objective Value: 0.6329582333564758</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(201, 232, 242)",
           "rgb(163, 211, 230)",
           "rgb(255, 255, 191)",
           "rgb(124, 179, 212)",
           "rgb(201, 34, 39)",
           "rgb(63, 99, 171)",
           "rgb(49, 54, 149)",
           "rgb(252, 165, 93)",
           "rgb(89, 141, 192)",
           "rgb(245, 118, 71)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          101,
          70,
          124,
          122,
          37,
          78,
          119,
          114,
          116,
          38,
          99,
          72,
          88,
          55,
          85
         ],
         "xaxis": "x19",
         "y": [
          101,
          70,
          124,
          122,
          37,
          78,
          119,
          114,
          116,
          38,
          99,
          72,
          88,
          55,
          85
         ],
         "yaxis": "y19"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #0<br>n_layers: 2<br>n_units_l0: 101<br>n_units_l1: 48<br>lr: 0.007711398857368533<br>optimizer: adam<br><b>Objective Value: 0.13771341741085052</b>",
          "Trial #1<br>n_layers: 2<br>n_units_l0: 70<br>n_units_l1: 41<br>lr: 0.003012808172117605<br>optimizer: rmsprop<br><b>Objective Value: 0.12340866029262543</b>",
          "Trial #2<br>n_layers: 3<br>n_units_l0: 124<br>n_units_l1: 48<br>n_units_l2: 58<br>lr: 0.005442268439829251<br>optimizer: rmsprop<br><b>Objective Value: 0.1751943975687027</b>",
          "Trial #3<br>n_layers: 3<br>n_units_l0: 122<br>n_units_l1: 97<br>n_units_l2: 66<br>lr: 0.00013421360666602302<br>optimizer: adam<br><b>Objective Value: 0.11885558068752289</b>",
          "Trial #4<br>n_layers: 3<br>n_units_l0: 37<br>n_units_l1: 43<br>n_units_l2: 65<br>lr: 0.05696244653419064<br>optimizer: rmsprop<br><b>Objective Value: 0.9541876316070557</b>",
          "Trial #5<br>n_layers: 3<br>n_units_l0: 78<br>n_units_l1: 126<br>n_units_l2: 60<br>lr: 0.05752434764095089<br>optimizer: sgd<br><b>Objective Value: 0.09808095544576645</b>",
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #7<br>n_layers: 2<br>n_units_l0: 114<br>n_units_l1: 51<br>lr: 0.0023572885912189617<br>optimizer: sgd<br><b>Objective Value: 0.37900349497795105</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(201, 232, 242)",
           "rgb(163, 211, 230)",
           "rgb(255, 255, 191)",
           "rgb(124, 179, 212)",
           "rgb(201, 34, 39)",
           "rgb(63, 99, 171)",
           "rgb(49, 54, 149)",
           "rgb(252, 165, 93)",
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          101,
          70,
          124,
          122,
          37,
          78,
          119,
          114,
          116,
          99,
          72,
          88,
          55,
          85
         ],
         "xaxis": "x27",
         "y": [
          48,
          41,
          48,
          97,
          43,
          126,
          86,
          51,
          52,
          82,
          127,
          125,
          104,
          71
         ],
         "yaxis": "y27"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #2<br>n_layers: 3<br>n_units_l0: 124<br>n_units_l1: 48<br>n_units_l2: 58<br>lr: 0.005442268439829251<br>optimizer: rmsprop<br><b>Objective Value: 0.1751943975687027</b>",
          "Trial #3<br>n_layers: 3<br>n_units_l0: 122<br>n_units_l1: 97<br>n_units_l2: 66<br>lr: 0.00013421360666602302<br>optimizer: adam<br><b>Objective Value: 0.11885558068752289</b>",
          "Trial #4<br>n_layers: 3<br>n_units_l0: 37<br>n_units_l1: 43<br>n_units_l2: 65<br>lr: 0.05696244653419064<br>optimizer: rmsprop<br><b>Objective Value: 0.9541876316070557</b>",
          "Trial #5<br>n_layers: 3<br>n_units_l0: 78<br>n_units_l1: 126<br>n_units_l2: 60<br>lr: 0.05752434764095089<br>optimizer: sgd<br><b>Objective Value: 0.09808095544576645</b>",
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(255, 255, 191)",
           "rgb(124, 179, 212)",
           "rgb(201, 34, 39)",
           "rgb(63, 99, 171)",
           "rgb(49, 54, 149)",
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          124,
          122,
          37,
          78,
          119,
          116,
          99,
          72,
          88,
          55,
          85
         ],
         "xaxis": "x35",
         "y": [
          58,
          66,
          65,
          60,
          112,
          75,
          122,
          113,
          33,
          96,
          96
         ],
         "yaxis": "y35"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(49, 54, 149)",
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          119,
          116,
          99,
          72,
          88,
          55,
          85
         ],
         "xaxis": "x43",
         "y": [
          91,
          33,
          110,
          81,
          127,
          77,
          81
         ],
         "yaxis": "y43"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>"
         ],
         "marker": {
          "color": [
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          116,
          99
         ],
         "xaxis": "x51",
         "y": [
          99,
          32
         ],
         "yaxis": "y51"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #0<br>n_layers: 2<br>n_units_l0: 101<br>n_units_l1: 48<br>lr: 0.007711398857368533<br>optimizer: adam<br><b>Objective Value: 0.13771341741085052</b>",
          "Trial #1<br>n_layers: 2<br>n_units_l0: 70<br>n_units_l1: 41<br>lr: 0.003012808172117605<br>optimizer: rmsprop<br><b>Objective Value: 0.12340866029262543</b>",
          "Trial #2<br>n_layers: 3<br>n_units_l0: 124<br>n_units_l1: 48<br>n_units_l2: 58<br>lr: 0.005442268439829251<br>optimizer: rmsprop<br><b>Objective Value: 0.1751943975687027</b>",
          "Trial #3<br>n_layers: 3<br>n_units_l0: 122<br>n_units_l1: 97<br>n_units_l2: 66<br>lr: 0.00013421360666602302<br>optimizer: adam<br><b>Objective Value: 0.11885558068752289</b>",
          "Trial #4<br>n_layers: 3<br>n_units_l0: 37<br>n_units_l1: 43<br>n_units_l2: 65<br>lr: 0.05696244653419064<br>optimizer: rmsprop<br><b>Objective Value: 0.9541876316070557</b>",
          "Trial #5<br>n_layers: 3<br>n_units_l0: 78<br>n_units_l1: 126<br>n_units_l2: 60<br>lr: 0.05752434764095089<br>optimizer: sgd<br><b>Objective Value: 0.09808095544576645</b>",
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #7<br>n_layers: 2<br>n_units_l0: 114<br>n_units_l1: 51<br>lr: 0.0023572885912189617<br>optimizer: sgd<br><b>Objective Value: 0.37900349497795105</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #9<br>n_layers: 1<br>n_units_l0: 38<br>lr: 0.08245600062993334<br>optimizer: adam<br><b>Objective Value: 0.6329582333564758</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(201, 232, 242)",
           "rgb(163, 211, 230)",
           "rgb(255, 255, 191)",
           "rgb(124, 179, 212)",
           "rgb(201, 34, 39)",
           "rgb(63, 99, 171)",
           "rgb(49, 54, 149)",
           "rgb(252, 165, 93)",
           "rgb(89, 141, 192)",
           "rgb(245, 118, 71)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          101,
          70,
          124,
          122,
          37,
          78,
          119,
          114,
          116,
          38,
          99,
          72,
          88,
          55,
          85
         ],
         "xaxis": "x59",
         "y": [
          "adam",
          "rmsprop",
          "rmsprop",
          "adam",
          "rmsprop",
          "sgd",
          "adam",
          "sgd",
          "adam",
          "adam",
          "adam",
          "sgd",
          "sgd",
          "sgd",
          "sgd"
         ],
         "yaxis": "y59"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #0<br>n_layers: 2<br>n_units_l0: 101<br>n_units_l1: 48<br>lr: 0.007711398857368533<br>optimizer: adam<br><b>Objective Value: 0.13771341741085052</b>",
          "Trial #1<br>n_layers: 2<br>n_units_l0: 70<br>n_units_l1: 41<br>lr: 0.003012808172117605<br>optimizer: rmsprop<br><b>Objective Value: 0.12340866029262543</b>",
          "Trial #2<br>n_layers: 3<br>n_units_l0: 124<br>n_units_l1: 48<br>n_units_l2: 58<br>lr: 0.005442268439829251<br>optimizer: rmsprop<br><b>Objective Value: 0.1751943975687027</b>",
          "Trial #3<br>n_layers: 3<br>n_units_l0: 122<br>n_units_l1: 97<br>n_units_l2: 66<br>lr: 0.00013421360666602302<br>optimizer: adam<br><b>Objective Value: 0.11885558068752289</b>",
          "Trial #4<br>n_layers: 3<br>n_units_l0: 37<br>n_units_l1: 43<br>n_units_l2: 65<br>lr: 0.05696244653419064<br>optimizer: rmsprop<br><b>Objective Value: 0.9541876316070557</b>",
          "Trial #5<br>n_layers: 3<br>n_units_l0: 78<br>n_units_l1: 126<br>n_units_l2: 60<br>lr: 0.05752434764095089<br>optimizer: sgd<br><b>Objective Value: 0.09808095544576645</b>",
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #7<br>n_layers: 2<br>n_units_l0: 114<br>n_units_l1: 51<br>lr: 0.0023572885912189617<br>optimizer: sgd<br><b>Objective Value: 0.37900349497795105</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(201, 232, 242)",
           "rgb(163, 211, 230)",
           "rgb(255, 255, 191)",
           "rgb(124, 179, 212)",
           "rgb(201, 34, 39)",
           "rgb(63, 99, 171)",
           "rgb(49, 54, 149)",
           "rgb(252, 165, 93)",
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          48,
          41,
          48,
          97,
          43,
          126,
          86,
          51,
          52,
          82,
          127,
          125,
          104,
          71
         ],
         "xaxis": "x4",
         "y": [
          0.007711398857368533,
          0.003012808172117605,
          0.005442268439829251,
          0.00013421360666602302,
          0.05696244653419064,
          0.05752434764095089,
          0.00024040850246942775,
          0.0023572885912189617,
          0.0005769984149991314,
          0.00001785199667793032,
          0.00037332091663808175,
          0.018851800842778852,
          0.022091351249891724,
          0.0009877185587115079
         ],
         "yaxis": "y4"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #0<br>n_layers: 2<br>n_units_l0: 101<br>n_units_l1: 48<br>lr: 0.007711398857368533<br>optimizer: adam<br><b>Objective Value: 0.13771341741085052</b>",
          "Trial #1<br>n_layers: 2<br>n_units_l0: 70<br>n_units_l1: 41<br>lr: 0.003012808172117605<br>optimizer: rmsprop<br><b>Objective Value: 0.12340866029262543</b>",
          "Trial #2<br>n_layers: 3<br>n_units_l0: 124<br>n_units_l1: 48<br>n_units_l2: 58<br>lr: 0.005442268439829251<br>optimizer: rmsprop<br><b>Objective Value: 0.1751943975687027</b>",
          "Trial #3<br>n_layers: 3<br>n_units_l0: 122<br>n_units_l1: 97<br>n_units_l2: 66<br>lr: 0.00013421360666602302<br>optimizer: adam<br><b>Objective Value: 0.11885558068752289</b>",
          "Trial #4<br>n_layers: 3<br>n_units_l0: 37<br>n_units_l1: 43<br>n_units_l2: 65<br>lr: 0.05696244653419064<br>optimizer: rmsprop<br><b>Objective Value: 0.9541876316070557</b>",
          "Trial #5<br>n_layers: 3<br>n_units_l0: 78<br>n_units_l1: 126<br>n_units_l2: 60<br>lr: 0.05752434764095089<br>optimizer: sgd<br><b>Objective Value: 0.09808095544576645</b>",
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #7<br>n_layers: 2<br>n_units_l0: 114<br>n_units_l1: 51<br>lr: 0.0023572885912189617<br>optimizer: sgd<br><b>Objective Value: 0.37900349497795105</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(201, 232, 242)",
           "rgb(163, 211, 230)",
           "rgb(255, 255, 191)",
           "rgb(124, 179, 212)",
           "rgb(201, 34, 39)",
           "rgb(63, 99, 171)",
           "rgb(49, 54, 149)",
           "rgb(252, 165, 93)",
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          48,
          41,
          48,
          97,
          43,
          126,
          86,
          51,
          52,
          82,
          127,
          125,
          104,
          71
         ],
         "xaxis": "x12",
         "y": [
          2,
          2,
          3,
          3,
          3,
          3,
          4,
          2,
          5,
          5,
          4,
          4,
          4,
          4
         ],
         "yaxis": "y12"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #0<br>n_layers: 2<br>n_units_l0: 101<br>n_units_l1: 48<br>lr: 0.007711398857368533<br>optimizer: adam<br><b>Objective Value: 0.13771341741085052</b>",
          "Trial #1<br>n_layers: 2<br>n_units_l0: 70<br>n_units_l1: 41<br>lr: 0.003012808172117605<br>optimizer: rmsprop<br><b>Objective Value: 0.12340866029262543</b>",
          "Trial #2<br>n_layers: 3<br>n_units_l0: 124<br>n_units_l1: 48<br>n_units_l2: 58<br>lr: 0.005442268439829251<br>optimizer: rmsprop<br><b>Objective Value: 0.1751943975687027</b>",
          "Trial #3<br>n_layers: 3<br>n_units_l0: 122<br>n_units_l1: 97<br>n_units_l2: 66<br>lr: 0.00013421360666602302<br>optimizer: adam<br><b>Objective Value: 0.11885558068752289</b>",
          "Trial #4<br>n_layers: 3<br>n_units_l0: 37<br>n_units_l1: 43<br>n_units_l2: 65<br>lr: 0.05696244653419064<br>optimizer: rmsprop<br><b>Objective Value: 0.9541876316070557</b>",
          "Trial #5<br>n_layers: 3<br>n_units_l0: 78<br>n_units_l1: 126<br>n_units_l2: 60<br>lr: 0.05752434764095089<br>optimizer: sgd<br><b>Objective Value: 0.09808095544576645</b>",
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #7<br>n_layers: 2<br>n_units_l0: 114<br>n_units_l1: 51<br>lr: 0.0023572885912189617<br>optimizer: sgd<br><b>Objective Value: 0.37900349497795105</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(201, 232, 242)",
           "rgb(163, 211, 230)",
           "rgb(255, 255, 191)",
           "rgb(124, 179, 212)",
           "rgb(201, 34, 39)",
           "rgb(63, 99, 171)",
           "rgb(49, 54, 149)",
           "rgb(252, 165, 93)",
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          48,
          41,
          48,
          97,
          43,
          126,
          86,
          51,
          52,
          82,
          127,
          125,
          104,
          71
         ],
         "xaxis": "x20",
         "y": [
          101,
          70,
          124,
          122,
          37,
          78,
          119,
          114,
          116,
          99,
          72,
          88,
          55,
          85
         ],
         "yaxis": "y20"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #0<br>n_layers: 2<br>n_units_l0: 101<br>n_units_l1: 48<br>lr: 0.007711398857368533<br>optimizer: adam<br><b>Objective Value: 0.13771341741085052</b>",
          "Trial #1<br>n_layers: 2<br>n_units_l0: 70<br>n_units_l1: 41<br>lr: 0.003012808172117605<br>optimizer: rmsprop<br><b>Objective Value: 0.12340866029262543</b>",
          "Trial #2<br>n_layers: 3<br>n_units_l0: 124<br>n_units_l1: 48<br>n_units_l2: 58<br>lr: 0.005442268439829251<br>optimizer: rmsprop<br><b>Objective Value: 0.1751943975687027</b>",
          "Trial #3<br>n_layers: 3<br>n_units_l0: 122<br>n_units_l1: 97<br>n_units_l2: 66<br>lr: 0.00013421360666602302<br>optimizer: adam<br><b>Objective Value: 0.11885558068752289</b>",
          "Trial #4<br>n_layers: 3<br>n_units_l0: 37<br>n_units_l1: 43<br>n_units_l2: 65<br>lr: 0.05696244653419064<br>optimizer: rmsprop<br><b>Objective Value: 0.9541876316070557</b>",
          "Trial #5<br>n_layers: 3<br>n_units_l0: 78<br>n_units_l1: 126<br>n_units_l2: 60<br>lr: 0.05752434764095089<br>optimizer: sgd<br><b>Objective Value: 0.09808095544576645</b>",
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #7<br>n_layers: 2<br>n_units_l0: 114<br>n_units_l1: 51<br>lr: 0.0023572885912189617<br>optimizer: sgd<br><b>Objective Value: 0.37900349497795105</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(201, 232, 242)",
           "rgb(163, 211, 230)",
           "rgb(255, 255, 191)",
           "rgb(124, 179, 212)",
           "rgb(201, 34, 39)",
           "rgb(63, 99, 171)",
           "rgb(49, 54, 149)",
           "rgb(252, 165, 93)",
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          48,
          41,
          48,
          97,
          43,
          126,
          86,
          51,
          52,
          82,
          127,
          125,
          104,
          71
         ],
         "xaxis": "x28",
         "y": [
          48,
          41,
          48,
          97,
          43,
          126,
          86,
          51,
          52,
          82,
          127,
          125,
          104,
          71
         ],
         "yaxis": "y28"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #2<br>n_layers: 3<br>n_units_l0: 124<br>n_units_l1: 48<br>n_units_l2: 58<br>lr: 0.005442268439829251<br>optimizer: rmsprop<br><b>Objective Value: 0.1751943975687027</b>",
          "Trial #3<br>n_layers: 3<br>n_units_l0: 122<br>n_units_l1: 97<br>n_units_l2: 66<br>lr: 0.00013421360666602302<br>optimizer: adam<br><b>Objective Value: 0.11885558068752289</b>",
          "Trial #4<br>n_layers: 3<br>n_units_l0: 37<br>n_units_l1: 43<br>n_units_l2: 65<br>lr: 0.05696244653419064<br>optimizer: rmsprop<br><b>Objective Value: 0.9541876316070557</b>",
          "Trial #5<br>n_layers: 3<br>n_units_l0: 78<br>n_units_l1: 126<br>n_units_l2: 60<br>lr: 0.05752434764095089<br>optimizer: sgd<br><b>Objective Value: 0.09808095544576645</b>",
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(255, 255, 191)",
           "rgb(124, 179, 212)",
           "rgb(201, 34, 39)",
           "rgb(63, 99, 171)",
           "rgb(49, 54, 149)",
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          48,
          97,
          43,
          126,
          86,
          52,
          82,
          127,
          125,
          104,
          71
         ],
         "xaxis": "x36",
         "y": [
          58,
          66,
          65,
          60,
          112,
          75,
          122,
          113,
          33,
          96,
          96
         ],
         "yaxis": "y36"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(49, 54, 149)",
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          86,
          52,
          82,
          127,
          125,
          104,
          71
         ],
         "xaxis": "x44",
         "y": [
          91,
          33,
          110,
          81,
          127,
          77,
          81
         ],
         "yaxis": "y44"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>"
         ],
         "marker": {
          "color": [
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          52,
          82
         ],
         "xaxis": "x52",
         "y": [
          99,
          32
         ],
         "yaxis": "y52"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #0<br>n_layers: 2<br>n_units_l0: 101<br>n_units_l1: 48<br>lr: 0.007711398857368533<br>optimizer: adam<br><b>Objective Value: 0.13771341741085052</b>",
          "Trial #1<br>n_layers: 2<br>n_units_l0: 70<br>n_units_l1: 41<br>lr: 0.003012808172117605<br>optimizer: rmsprop<br><b>Objective Value: 0.12340866029262543</b>",
          "Trial #2<br>n_layers: 3<br>n_units_l0: 124<br>n_units_l1: 48<br>n_units_l2: 58<br>lr: 0.005442268439829251<br>optimizer: rmsprop<br><b>Objective Value: 0.1751943975687027</b>",
          "Trial #3<br>n_layers: 3<br>n_units_l0: 122<br>n_units_l1: 97<br>n_units_l2: 66<br>lr: 0.00013421360666602302<br>optimizer: adam<br><b>Objective Value: 0.11885558068752289</b>",
          "Trial #4<br>n_layers: 3<br>n_units_l0: 37<br>n_units_l1: 43<br>n_units_l2: 65<br>lr: 0.05696244653419064<br>optimizer: rmsprop<br><b>Objective Value: 0.9541876316070557</b>",
          "Trial #5<br>n_layers: 3<br>n_units_l0: 78<br>n_units_l1: 126<br>n_units_l2: 60<br>lr: 0.05752434764095089<br>optimizer: sgd<br><b>Objective Value: 0.09808095544576645</b>",
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #7<br>n_layers: 2<br>n_units_l0: 114<br>n_units_l1: 51<br>lr: 0.0023572885912189617<br>optimizer: sgd<br><b>Objective Value: 0.37900349497795105</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(201, 232, 242)",
           "rgb(163, 211, 230)",
           "rgb(255, 255, 191)",
           "rgb(124, 179, 212)",
           "rgb(201, 34, 39)",
           "rgb(63, 99, 171)",
           "rgb(49, 54, 149)",
           "rgb(252, 165, 93)",
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          48,
          41,
          48,
          97,
          43,
          126,
          86,
          51,
          52,
          82,
          127,
          125,
          104,
          71
         ],
         "xaxis": "x60",
         "y": [
          "adam",
          "rmsprop",
          "rmsprop",
          "adam",
          "rmsprop",
          "sgd",
          "adam",
          "sgd",
          "adam",
          "adam",
          "sgd",
          "sgd",
          "sgd",
          "sgd"
         ],
         "yaxis": "y60"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #2<br>n_layers: 3<br>n_units_l0: 124<br>n_units_l1: 48<br>n_units_l2: 58<br>lr: 0.005442268439829251<br>optimizer: rmsprop<br><b>Objective Value: 0.1751943975687027</b>",
          "Trial #3<br>n_layers: 3<br>n_units_l0: 122<br>n_units_l1: 97<br>n_units_l2: 66<br>lr: 0.00013421360666602302<br>optimizer: adam<br><b>Objective Value: 0.11885558068752289</b>",
          "Trial #4<br>n_layers: 3<br>n_units_l0: 37<br>n_units_l1: 43<br>n_units_l2: 65<br>lr: 0.05696244653419064<br>optimizer: rmsprop<br><b>Objective Value: 0.9541876316070557</b>",
          "Trial #5<br>n_layers: 3<br>n_units_l0: 78<br>n_units_l1: 126<br>n_units_l2: 60<br>lr: 0.05752434764095089<br>optimizer: sgd<br><b>Objective Value: 0.09808095544576645</b>",
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(255, 255, 191)",
           "rgb(124, 179, 212)",
           "rgb(201, 34, 39)",
           "rgb(63, 99, 171)",
           "rgb(49, 54, 149)",
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          58,
          66,
          65,
          60,
          112,
          75,
          122,
          113,
          33,
          96,
          96
         ],
         "xaxis": "x5",
         "y": [
          0.005442268439829251,
          0.00013421360666602302,
          0.05696244653419064,
          0.05752434764095089,
          0.00024040850246942775,
          0.0005769984149991314,
          0.00001785199667793032,
          0.00037332091663808175,
          0.018851800842778852,
          0.022091351249891724,
          0.0009877185587115079
         ],
         "yaxis": "y5"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #2<br>n_layers: 3<br>n_units_l0: 124<br>n_units_l1: 48<br>n_units_l2: 58<br>lr: 0.005442268439829251<br>optimizer: rmsprop<br><b>Objective Value: 0.1751943975687027</b>",
          "Trial #3<br>n_layers: 3<br>n_units_l0: 122<br>n_units_l1: 97<br>n_units_l2: 66<br>lr: 0.00013421360666602302<br>optimizer: adam<br><b>Objective Value: 0.11885558068752289</b>",
          "Trial #4<br>n_layers: 3<br>n_units_l0: 37<br>n_units_l1: 43<br>n_units_l2: 65<br>lr: 0.05696244653419064<br>optimizer: rmsprop<br><b>Objective Value: 0.9541876316070557</b>",
          "Trial #5<br>n_layers: 3<br>n_units_l0: 78<br>n_units_l1: 126<br>n_units_l2: 60<br>lr: 0.05752434764095089<br>optimizer: sgd<br><b>Objective Value: 0.09808095544576645</b>",
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(255, 255, 191)",
           "rgb(124, 179, 212)",
           "rgb(201, 34, 39)",
           "rgb(63, 99, 171)",
           "rgb(49, 54, 149)",
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          58,
          66,
          65,
          60,
          112,
          75,
          122,
          113,
          33,
          96,
          96
         ],
         "xaxis": "x13",
         "y": [
          3,
          3,
          3,
          3,
          4,
          5,
          5,
          4,
          4,
          4,
          4
         ],
         "yaxis": "y13"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #2<br>n_layers: 3<br>n_units_l0: 124<br>n_units_l1: 48<br>n_units_l2: 58<br>lr: 0.005442268439829251<br>optimizer: rmsprop<br><b>Objective Value: 0.1751943975687027</b>",
          "Trial #3<br>n_layers: 3<br>n_units_l0: 122<br>n_units_l1: 97<br>n_units_l2: 66<br>lr: 0.00013421360666602302<br>optimizer: adam<br><b>Objective Value: 0.11885558068752289</b>",
          "Trial #4<br>n_layers: 3<br>n_units_l0: 37<br>n_units_l1: 43<br>n_units_l2: 65<br>lr: 0.05696244653419064<br>optimizer: rmsprop<br><b>Objective Value: 0.9541876316070557</b>",
          "Trial #5<br>n_layers: 3<br>n_units_l0: 78<br>n_units_l1: 126<br>n_units_l2: 60<br>lr: 0.05752434764095089<br>optimizer: sgd<br><b>Objective Value: 0.09808095544576645</b>",
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(255, 255, 191)",
           "rgb(124, 179, 212)",
           "rgb(201, 34, 39)",
           "rgb(63, 99, 171)",
           "rgb(49, 54, 149)",
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          58,
          66,
          65,
          60,
          112,
          75,
          122,
          113,
          33,
          96,
          96
         ],
         "xaxis": "x21",
         "y": [
          124,
          122,
          37,
          78,
          119,
          116,
          99,
          72,
          88,
          55,
          85
         ],
         "yaxis": "y21"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #2<br>n_layers: 3<br>n_units_l0: 124<br>n_units_l1: 48<br>n_units_l2: 58<br>lr: 0.005442268439829251<br>optimizer: rmsprop<br><b>Objective Value: 0.1751943975687027</b>",
          "Trial #3<br>n_layers: 3<br>n_units_l0: 122<br>n_units_l1: 97<br>n_units_l2: 66<br>lr: 0.00013421360666602302<br>optimizer: adam<br><b>Objective Value: 0.11885558068752289</b>",
          "Trial #4<br>n_layers: 3<br>n_units_l0: 37<br>n_units_l1: 43<br>n_units_l2: 65<br>lr: 0.05696244653419064<br>optimizer: rmsprop<br><b>Objective Value: 0.9541876316070557</b>",
          "Trial #5<br>n_layers: 3<br>n_units_l0: 78<br>n_units_l1: 126<br>n_units_l2: 60<br>lr: 0.05752434764095089<br>optimizer: sgd<br><b>Objective Value: 0.09808095544576645</b>",
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(255, 255, 191)",
           "rgb(124, 179, 212)",
           "rgb(201, 34, 39)",
           "rgb(63, 99, 171)",
           "rgb(49, 54, 149)",
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          58,
          66,
          65,
          60,
          112,
          75,
          122,
          113,
          33,
          96,
          96
         ],
         "xaxis": "x29",
         "y": [
          48,
          97,
          43,
          126,
          86,
          52,
          82,
          127,
          125,
          104,
          71
         ],
         "yaxis": "y29"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #2<br>n_layers: 3<br>n_units_l0: 124<br>n_units_l1: 48<br>n_units_l2: 58<br>lr: 0.005442268439829251<br>optimizer: rmsprop<br><b>Objective Value: 0.1751943975687027</b>",
          "Trial #3<br>n_layers: 3<br>n_units_l0: 122<br>n_units_l1: 97<br>n_units_l2: 66<br>lr: 0.00013421360666602302<br>optimizer: adam<br><b>Objective Value: 0.11885558068752289</b>",
          "Trial #4<br>n_layers: 3<br>n_units_l0: 37<br>n_units_l1: 43<br>n_units_l2: 65<br>lr: 0.05696244653419064<br>optimizer: rmsprop<br><b>Objective Value: 0.9541876316070557</b>",
          "Trial #5<br>n_layers: 3<br>n_units_l0: 78<br>n_units_l1: 126<br>n_units_l2: 60<br>lr: 0.05752434764095089<br>optimizer: sgd<br><b>Objective Value: 0.09808095544576645</b>",
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(255, 255, 191)",
           "rgb(124, 179, 212)",
           "rgb(201, 34, 39)",
           "rgb(63, 99, 171)",
           "rgb(49, 54, 149)",
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          58,
          66,
          65,
          60,
          112,
          75,
          122,
          113,
          33,
          96,
          96
         ],
         "xaxis": "x37",
         "y": [
          58,
          66,
          65,
          60,
          112,
          75,
          122,
          113,
          33,
          96,
          96
         ],
         "yaxis": "y37"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(49, 54, 149)",
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          112,
          75,
          122,
          113,
          33,
          96,
          96
         ],
         "xaxis": "x45",
         "y": [
          91,
          33,
          110,
          81,
          127,
          77,
          81
         ],
         "yaxis": "y45"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>"
         ],
         "marker": {
          "color": [
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          75,
          122
         ],
         "xaxis": "x53",
         "y": [
          99,
          32
         ],
         "yaxis": "y53"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #2<br>n_layers: 3<br>n_units_l0: 124<br>n_units_l1: 48<br>n_units_l2: 58<br>lr: 0.005442268439829251<br>optimizer: rmsprop<br><b>Objective Value: 0.1751943975687027</b>",
          "Trial #3<br>n_layers: 3<br>n_units_l0: 122<br>n_units_l1: 97<br>n_units_l2: 66<br>lr: 0.00013421360666602302<br>optimizer: adam<br><b>Objective Value: 0.11885558068752289</b>",
          "Trial #4<br>n_layers: 3<br>n_units_l0: 37<br>n_units_l1: 43<br>n_units_l2: 65<br>lr: 0.05696244653419064<br>optimizer: rmsprop<br><b>Objective Value: 0.9541876316070557</b>",
          "Trial #5<br>n_layers: 3<br>n_units_l0: 78<br>n_units_l1: 126<br>n_units_l2: 60<br>lr: 0.05752434764095089<br>optimizer: sgd<br><b>Objective Value: 0.09808095544576645</b>",
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(255, 255, 191)",
           "rgb(124, 179, 212)",
           "rgb(201, 34, 39)",
           "rgb(63, 99, 171)",
           "rgb(49, 54, 149)",
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          58,
          66,
          65,
          60,
          112,
          75,
          122,
          113,
          33,
          96,
          96
         ],
         "xaxis": "x61",
         "y": [
          "rmsprop",
          "adam",
          "rmsprop",
          "sgd",
          "adam",
          "adam",
          "adam",
          "sgd",
          "sgd",
          "sgd",
          "sgd"
         ],
         "yaxis": "y61"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(49, 54, 149)",
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          91,
          33,
          110,
          81,
          127,
          77,
          81
         ],
         "xaxis": "x6",
         "y": [
          0.00024040850246942775,
          0.0005769984149991314,
          0.00001785199667793032,
          0.00037332091663808175,
          0.018851800842778852,
          0.022091351249891724,
          0.0009877185587115079
         ],
         "yaxis": "y6"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(49, 54, 149)",
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          91,
          33,
          110,
          81,
          127,
          77,
          81
         ],
         "xaxis": "x14",
         "y": [
          4,
          5,
          5,
          4,
          4,
          4,
          4
         ],
         "yaxis": "y14"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(49, 54, 149)",
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          91,
          33,
          110,
          81,
          127,
          77,
          81
         ],
         "xaxis": "x22",
         "y": [
          119,
          116,
          99,
          72,
          88,
          55,
          85
         ],
         "yaxis": "y22"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(49, 54, 149)",
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          91,
          33,
          110,
          81,
          127,
          77,
          81
         ],
         "xaxis": "x30",
         "y": [
          86,
          52,
          82,
          127,
          125,
          104,
          71
         ],
         "yaxis": "y30"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(49, 54, 149)",
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          91,
          33,
          110,
          81,
          127,
          77,
          81
         ],
         "xaxis": "x38",
         "y": [
          112,
          75,
          122,
          113,
          33,
          96,
          96
         ],
         "yaxis": "y38"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(49, 54, 149)",
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          91,
          33,
          110,
          81,
          127,
          77,
          81
         ],
         "xaxis": "x46",
         "y": [
          91,
          33,
          110,
          81,
          127,
          77,
          81
         ],
         "yaxis": "y46"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>"
         ],
         "marker": {
          "color": [
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          33,
          110
         ],
         "xaxis": "x54",
         "y": [
          99,
          32
         ],
         "yaxis": "y54"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(49, 54, 149)",
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          91,
          33,
          110,
          81,
          127,
          77,
          81
         ],
         "xaxis": "x62",
         "y": [
          "adam",
          "adam",
          "adam",
          "sgd",
          "sgd",
          "sgd",
          "sgd"
         ],
         "yaxis": "y62"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>"
         ],
         "marker": {
          "color": [
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          99,
          32
         ],
         "xaxis": "x7",
         "y": [
          0.0005769984149991314,
          0.00001785199667793032
         ],
         "yaxis": "y7"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>"
         ],
         "marker": {
          "color": [
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          99,
          32
         ],
         "xaxis": "x15",
         "y": [
          5,
          5
         ],
         "yaxis": "y15"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>"
         ],
         "marker": {
          "color": [
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          99,
          32
         ],
         "xaxis": "x23",
         "y": [
          116,
          99
         ],
         "yaxis": "y23"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>"
         ],
         "marker": {
          "color": [
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          99,
          32
         ],
         "xaxis": "x31",
         "y": [
          52,
          82
         ],
         "yaxis": "y31"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>"
         ],
         "marker": {
          "color": [
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          99,
          32
         ],
         "xaxis": "x39",
         "y": [
          75,
          122
         ],
         "yaxis": "y39"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>"
         ],
         "marker": {
          "color": [
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          99,
          32
         ],
         "xaxis": "x47",
         "y": [
          33,
          110
         ],
         "yaxis": "y47"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>"
         ],
         "marker": {
          "color": [
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          99,
          32
         ],
         "xaxis": "x55",
         "y": [
          99,
          32
         ],
         "yaxis": "y55"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>"
         ],
         "marker": {
          "color": [
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          99,
          32
         ],
         "xaxis": "x63",
         "y": [
          "adam",
          "adam"
         ],
         "yaxis": "y63"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #0<br>n_layers: 2<br>n_units_l0: 101<br>n_units_l1: 48<br>lr: 0.007711398857368533<br>optimizer: adam<br><b>Objective Value: 0.13771341741085052</b>",
          "Trial #1<br>n_layers: 2<br>n_units_l0: 70<br>n_units_l1: 41<br>lr: 0.003012808172117605<br>optimizer: rmsprop<br><b>Objective Value: 0.12340866029262543</b>",
          "Trial #2<br>n_layers: 3<br>n_units_l0: 124<br>n_units_l1: 48<br>n_units_l2: 58<br>lr: 0.005442268439829251<br>optimizer: rmsprop<br><b>Objective Value: 0.1751943975687027</b>",
          "Trial #3<br>n_layers: 3<br>n_units_l0: 122<br>n_units_l1: 97<br>n_units_l2: 66<br>lr: 0.00013421360666602302<br>optimizer: adam<br><b>Objective Value: 0.11885558068752289</b>",
          "Trial #4<br>n_layers: 3<br>n_units_l0: 37<br>n_units_l1: 43<br>n_units_l2: 65<br>lr: 0.05696244653419064<br>optimizer: rmsprop<br><b>Objective Value: 0.9541876316070557</b>",
          "Trial #5<br>n_layers: 3<br>n_units_l0: 78<br>n_units_l1: 126<br>n_units_l2: 60<br>lr: 0.05752434764095089<br>optimizer: sgd<br><b>Objective Value: 0.09808095544576645</b>",
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #7<br>n_layers: 2<br>n_units_l0: 114<br>n_units_l1: 51<br>lr: 0.0023572885912189617<br>optimizer: sgd<br><b>Objective Value: 0.37900349497795105</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #9<br>n_layers: 1<br>n_units_l0: 38<br>lr: 0.08245600062993334<br>optimizer: adam<br><b>Objective Value: 0.6329582333564758</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(201, 232, 242)",
           "rgb(163, 211, 230)",
           "rgb(255, 255, 191)",
           "rgb(124, 179, 212)",
           "rgb(201, 34, 39)",
           "rgb(63, 99, 171)",
           "rgb(49, 54, 149)",
           "rgb(252, 165, 93)",
           "rgb(89, 141, 192)",
           "rgb(245, 118, 71)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          "adam",
          "rmsprop",
          "rmsprop",
          "adam",
          "rmsprop",
          "sgd",
          "adam",
          "sgd",
          "adam",
          "adam",
          "adam",
          "sgd",
          "sgd",
          "sgd",
          "sgd"
         ],
         "xaxis": "x8",
         "y": [
          0.007711398857368533,
          0.003012808172117605,
          0.005442268439829251,
          0.00013421360666602302,
          0.05696244653419064,
          0.05752434764095089,
          0.00024040850246942775,
          0.0023572885912189617,
          0.0005769984149991314,
          0.08245600062993334,
          0.00001785199667793032,
          0.00037332091663808175,
          0.018851800842778852,
          0.022091351249891724,
          0.0009877185587115079
         ],
         "yaxis": "y8"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #0<br>n_layers: 2<br>n_units_l0: 101<br>n_units_l1: 48<br>lr: 0.007711398857368533<br>optimizer: adam<br><b>Objective Value: 0.13771341741085052</b>",
          "Trial #1<br>n_layers: 2<br>n_units_l0: 70<br>n_units_l1: 41<br>lr: 0.003012808172117605<br>optimizer: rmsprop<br><b>Objective Value: 0.12340866029262543</b>",
          "Trial #2<br>n_layers: 3<br>n_units_l0: 124<br>n_units_l1: 48<br>n_units_l2: 58<br>lr: 0.005442268439829251<br>optimizer: rmsprop<br><b>Objective Value: 0.1751943975687027</b>",
          "Trial #3<br>n_layers: 3<br>n_units_l0: 122<br>n_units_l1: 97<br>n_units_l2: 66<br>lr: 0.00013421360666602302<br>optimizer: adam<br><b>Objective Value: 0.11885558068752289</b>",
          "Trial #4<br>n_layers: 3<br>n_units_l0: 37<br>n_units_l1: 43<br>n_units_l2: 65<br>lr: 0.05696244653419064<br>optimizer: rmsprop<br><b>Objective Value: 0.9541876316070557</b>",
          "Trial #5<br>n_layers: 3<br>n_units_l0: 78<br>n_units_l1: 126<br>n_units_l2: 60<br>lr: 0.05752434764095089<br>optimizer: sgd<br><b>Objective Value: 0.09808095544576645</b>",
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #7<br>n_layers: 2<br>n_units_l0: 114<br>n_units_l1: 51<br>lr: 0.0023572885912189617<br>optimizer: sgd<br><b>Objective Value: 0.37900349497795105</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #9<br>n_layers: 1<br>n_units_l0: 38<br>lr: 0.08245600062993334<br>optimizer: adam<br><b>Objective Value: 0.6329582333564758</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(201, 232, 242)",
           "rgb(163, 211, 230)",
           "rgb(255, 255, 191)",
           "rgb(124, 179, 212)",
           "rgb(201, 34, 39)",
           "rgb(63, 99, 171)",
           "rgb(49, 54, 149)",
           "rgb(252, 165, 93)",
           "rgb(89, 141, 192)",
           "rgb(245, 118, 71)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          "adam",
          "rmsprop",
          "rmsprop",
          "adam",
          "rmsprop",
          "sgd",
          "adam",
          "sgd",
          "adam",
          "adam",
          "adam",
          "sgd",
          "sgd",
          "sgd",
          "sgd"
         ],
         "xaxis": "x16",
         "y": [
          2,
          2,
          3,
          3,
          3,
          3,
          4,
          2,
          5,
          1,
          5,
          4,
          4,
          4,
          4
         ],
         "yaxis": "y16"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #0<br>n_layers: 2<br>n_units_l0: 101<br>n_units_l1: 48<br>lr: 0.007711398857368533<br>optimizer: adam<br><b>Objective Value: 0.13771341741085052</b>",
          "Trial #1<br>n_layers: 2<br>n_units_l0: 70<br>n_units_l1: 41<br>lr: 0.003012808172117605<br>optimizer: rmsprop<br><b>Objective Value: 0.12340866029262543</b>",
          "Trial #2<br>n_layers: 3<br>n_units_l0: 124<br>n_units_l1: 48<br>n_units_l2: 58<br>lr: 0.005442268439829251<br>optimizer: rmsprop<br><b>Objective Value: 0.1751943975687027</b>",
          "Trial #3<br>n_layers: 3<br>n_units_l0: 122<br>n_units_l1: 97<br>n_units_l2: 66<br>lr: 0.00013421360666602302<br>optimizer: adam<br><b>Objective Value: 0.11885558068752289</b>",
          "Trial #4<br>n_layers: 3<br>n_units_l0: 37<br>n_units_l1: 43<br>n_units_l2: 65<br>lr: 0.05696244653419064<br>optimizer: rmsprop<br><b>Objective Value: 0.9541876316070557</b>",
          "Trial #5<br>n_layers: 3<br>n_units_l0: 78<br>n_units_l1: 126<br>n_units_l2: 60<br>lr: 0.05752434764095089<br>optimizer: sgd<br><b>Objective Value: 0.09808095544576645</b>",
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #7<br>n_layers: 2<br>n_units_l0: 114<br>n_units_l1: 51<br>lr: 0.0023572885912189617<br>optimizer: sgd<br><b>Objective Value: 0.37900349497795105</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #9<br>n_layers: 1<br>n_units_l0: 38<br>lr: 0.08245600062993334<br>optimizer: adam<br><b>Objective Value: 0.6329582333564758</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(201, 232, 242)",
           "rgb(163, 211, 230)",
           "rgb(255, 255, 191)",
           "rgb(124, 179, 212)",
           "rgb(201, 34, 39)",
           "rgb(63, 99, 171)",
           "rgb(49, 54, 149)",
           "rgb(252, 165, 93)",
           "rgb(89, 141, 192)",
           "rgb(245, 118, 71)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          "adam",
          "rmsprop",
          "rmsprop",
          "adam",
          "rmsprop",
          "sgd",
          "adam",
          "sgd",
          "adam",
          "adam",
          "adam",
          "sgd",
          "sgd",
          "sgd",
          "sgd"
         ],
         "xaxis": "x24",
         "y": [
          101,
          70,
          124,
          122,
          37,
          78,
          119,
          114,
          116,
          38,
          99,
          72,
          88,
          55,
          85
         ],
         "yaxis": "y24"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #0<br>n_layers: 2<br>n_units_l0: 101<br>n_units_l1: 48<br>lr: 0.007711398857368533<br>optimizer: adam<br><b>Objective Value: 0.13771341741085052</b>",
          "Trial #1<br>n_layers: 2<br>n_units_l0: 70<br>n_units_l1: 41<br>lr: 0.003012808172117605<br>optimizer: rmsprop<br><b>Objective Value: 0.12340866029262543</b>",
          "Trial #2<br>n_layers: 3<br>n_units_l0: 124<br>n_units_l1: 48<br>n_units_l2: 58<br>lr: 0.005442268439829251<br>optimizer: rmsprop<br><b>Objective Value: 0.1751943975687027</b>",
          "Trial #3<br>n_layers: 3<br>n_units_l0: 122<br>n_units_l1: 97<br>n_units_l2: 66<br>lr: 0.00013421360666602302<br>optimizer: adam<br><b>Objective Value: 0.11885558068752289</b>",
          "Trial #4<br>n_layers: 3<br>n_units_l0: 37<br>n_units_l1: 43<br>n_units_l2: 65<br>lr: 0.05696244653419064<br>optimizer: rmsprop<br><b>Objective Value: 0.9541876316070557</b>",
          "Trial #5<br>n_layers: 3<br>n_units_l0: 78<br>n_units_l1: 126<br>n_units_l2: 60<br>lr: 0.05752434764095089<br>optimizer: sgd<br><b>Objective Value: 0.09808095544576645</b>",
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #7<br>n_layers: 2<br>n_units_l0: 114<br>n_units_l1: 51<br>lr: 0.0023572885912189617<br>optimizer: sgd<br><b>Objective Value: 0.37900349497795105</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(201, 232, 242)",
           "rgb(163, 211, 230)",
           "rgb(255, 255, 191)",
           "rgb(124, 179, 212)",
           "rgb(201, 34, 39)",
           "rgb(63, 99, 171)",
           "rgb(49, 54, 149)",
           "rgb(252, 165, 93)",
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          "adam",
          "rmsprop",
          "rmsprop",
          "adam",
          "rmsprop",
          "sgd",
          "adam",
          "sgd",
          "adam",
          "adam",
          "sgd",
          "sgd",
          "sgd",
          "sgd"
         ],
         "xaxis": "x32",
         "y": [
          48,
          41,
          48,
          97,
          43,
          126,
          86,
          51,
          52,
          82,
          127,
          125,
          104,
          71
         ],
         "yaxis": "y32"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #2<br>n_layers: 3<br>n_units_l0: 124<br>n_units_l1: 48<br>n_units_l2: 58<br>lr: 0.005442268439829251<br>optimizer: rmsprop<br><b>Objective Value: 0.1751943975687027</b>",
          "Trial #3<br>n_layers: 3<br>n_units_l0: 122<br>n_units_l1: 97<br>n_units_l2: 66<br>lr: 0.00013421360666602302<br>optimizer: adam<br><b>Objective Value: 0.11885558068752289</b>",
          "Trial #4<br>n_layers: 3<br>n_units_l0: 37<br>n_units_l1: 43<br>n_units_l2: 65<br>lr: 0.05696244653419064<br>optimizer: rmsprop<br><b>Objective Value: 0.9541876316070557</b>",
          "Trial #5<br>n_layers: 3<br>n_units_l0: 78<br>n_units_l1: 126<br>n_units_l2: 60<br>lr: 0.05752434764095089<br>optimizer: sgd<br><b>Objective Value: 0.09808095544576645</b>",
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(255, 255, 191)",
           "rgb(124, 179, 212)",
           "rgb(201, 34, 39)",
           "rgb(63, 99, 171)",
           "rgb(49, 54, 149)",
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          "rmsprop",
          "adam",
          "rmsprop",
          "sgd",
          "adam",
          "adam",
          "adam",
          "sgd",
          "sgd",
          "sgd",
          "sgd"
         ],
         "xaxis": "x40",
         "y": [
          58,
          66,
          65,
          60,
          112,
          75,
          122,
          113,
          33,
          96,
          96
         ],
         "yaxis": "y40"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(49, 54, 149)",
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          "adam",
          "adam",
          "adam",
          "sgd",
          "sgd",
          "sgd",
          "sgd"
         ],
         "xaxis": "x48",
         "y": [
          91,
          33,
          110,
          81,
          127,
          77,
          81
         ],
         "yaxis": "y48"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>"
         ],
         "marker": {
          "color": [
           "rgb(89, 141, 192)",
           "rgb(254, 203, 124)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          "adam",
          "adam"
         ],
         "xaxis": "x56",
         "y": [
          99,
          32
         ],
         "yaxis": "y56"
        },
        {
         "hovertemplate": "%{hovertext}<extra></extra>",
         "hovertext": [
          "Trial #0<br>n_layers: 2<br>n_units_l0: 101<br>n_units_l1: 48<br>lr: 0.007711398857368533<br>optimizer: adam<br><b>Objective Value: 0.13771341741085052</b>",
          "Trial #1<br>n_layers: 2<br>n_units_l0: 70<br>n_units_l1: 41<br>lr: 0.003012808172117605<br>optimizer: rmsprop<br><b>Objective Value: 0.12340866029262543</b>",
          "Trial #2<br>n_layers: 3<br>n_units_l0: 124<br>n_units_l1: 48<br>n_units_l2: 58<br>lr: 0.005442268439829251<br>optimizer: rmsprop<br><b>Objective Value: 0.1751943975687027</b>",
          "Trial #3<br>n_layers: 3<br>n_units_l0: 122<br>n_units_l1: 97<br>n_units_l2: 66<br>lr: 0.00013421360666602302<br>optimizer: adam<br><b>Objective Value: 0.11885558068752289</b>",
          "Trial #4<br>n_layers: 3<br>n_units_l0: 37<br>n_units_l1: 43<br>n_units_l2: 65<br>lr: 0.05696244653419064<br>optimizer: rmsprop<br><b>Objective Value: 0.9541876316070557</b>",
          "Trial #5<br>n_layers: 3<br>n_units_l0: 78<br>n_units_l1: 126<br>n_units_l2: 60<br>lr: 0.05752434764095089<br>optimizer: sgd<br><b>Objective Value: 0.09808095544576645</b>",
          "Trial #6<br>n_layers: 4<br>n_units_l0: 119<br>n_units_l1: 86<br>n_units_l2: 112<br>n_units_l3: 91<br>lr: 0.00024040850246942775<br>optimizer: adam<br><b>Objective Value: 0.08969108015298843</b>",
          "Trial #7<br>n_layers: 2<br>n_units_l0: 114<br>n_units_l1: 51<br>lr: 0.0023572885912189617<br>optimizer: sgd<br><b>Objective Value: 0.37900349497795105</b>",
          "Trial #8<br>n_layers: 5<br>n_units_l0: 116<br>n_units_l1: 52<br>n_units_l2: 75<br>n_units_l3: 33<br>n_units_l4: 99<br>lr: 0.0005769984149991314<br>optimizer: adam<br><b>Objective Value: 0.1021626815199852</b>",
          "Trial #9<br>n_layers: 1<br>n_units_l0: 38<br>lr: 0.08245600062993334<br>optimizer: adam<br><b>Objective Value: 0.6329582333564758</b>",
          "Trial #10<br>n_layers: 5<br>n_units_l0: 99<br>n_units_l1: 82<br>n_units_l2: 122<br>n_units_l3: 110<br>n_units_l4: 32<br>lr: 1.785199667793032e-05<br>optimizer: adam<br><b>Objective Value: 0.3321899473667145</b>",
          "Trial #11<br>n_layers: 4<br>n_units_l0: 72<br>n_units_l1: 127<br>n_units_l2: 113<br>n_units_l3: 81<br>lr: 0.00037332091663808175<br>optimizer: sgd<br><b>Objective Value: 1.9311206340789795</b>",
          "Trial #12<br>n_layers: 4<br>n_units_l0: 88<br>n_units_l1: 125<br>n_units_l2: 33<br>n_units_l3: 127<br>lr: 0.018851800842778852<br>optimizer: sgd<br><b>Objective Value: 0.1834155023097992</b>",
          "Trial #13<br>n_layers: 4<br>n_units_l0: 55<br>n_units_l1: 104<br>n_units_l2: 96<br>n_units_l3: 77<br>lr: 0.022091351249891724<br>optimizer: sgd<br><b>Objective Value: 0.14339764416217804</b>",
          "Trial #14<br>n_layers: 4<br>n_units_l0: 85<br>n_units_l1: 71<br>n_units_l2: 96<br>n_units_l3: 81<br>lr: 0.0009877185587115079<br>optimizer: sgd<br><b>Objective Value: 0.7806804180145264</b>"
         ],
         "marker": {
          "color": [
           "rgb(201, 232, 242)",
           "rgb(163, 211, 230)",
           "rgb(255, 255, 191)",
           "rgb(124, 179, 212)",
           "rgb(201, 34, 39)",
           "rgb(63, 99, 171)",
           "rgb(49, 54, 149)",
           "rgb(252, 165, 93)",
           "rgb(89, 141, 192)",
           "rgb(245, 118, 71)",
           "rgb(254, 203, 124)",
           "rgb(165, 0, 38)",
           "rgb(254, 233, 157)",
           "rgb(233, 246, 232)",
           "rgb(227, 74, 51)"
          ],
          "line": {
           "color": "Grey",
           "width": 0.5
          }
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          "adam",
          "rmsprop",
          "rmsprop",
          "adam",
          "rmsprop",
          "sgd",
          "adam",
          "sgd",
          "adam",
          "adam",
          "adam",
          "sgd",
          "sgd",
          "sgd",
          "sgd"
         ],
         "xaxis": "x64",
         "y": [
          "adam",
          "rmsprop",
          "rmsprop",
          "adam",
          "rmsprop",
          "sgd",
          "adam",
          "sgd",
          "adam",
          "adam",
          "adam",
          "sgd",
          "sgd",
          "sgd",
          "sgd"
         ],
         "yaxis": "y64"
        },
        {
         "hoverinfo": "none",
         "marker": {
          "cmax": 1,
          "cmin": 0,
          "colorbar": {
           "thickness": 10,
           "ticktext": [
            "min. (0.0896911)",
            "25% (0.121132)",
            "50% (0.175194)",
            "75% (0.505981)",
            "max. (1.93112)"
           ],
           "tickvals": [
            0,
            0.25,
            0.5,
            0.75,
            1
           ]
          },
          "colorscale": [
           [
            0,
            "rgb(49,54,149)"
           ],
           [
            0.1,
            "rgb(69,117,180)"
           ],
           [
            0.2,
            "rgb(116,173,209)"
           ],
           [
            0.3,
            "rgb(171,217,233)"
           ],
           [
            0.4,
            "rgb(224,243,248)"
           ],
           [
            0.5,
            "rgb(255,255,191)"
           ],
           [
            0.6,
            "rgb(254,224,144)"
           ],
           [
            0.7,
            "rgb(253,174,97)"
           ],
           [
            0.8,
            "rgb(244,109,67)"
           ],
           [
            0.9,
            "rgb(215,48,39)"
           ],
           [
            1,
            "rgb(165,0,38)"
           ]
          ],
          "showscale": true
         },
         "mode": "markers",
         "showlegend": false,
         "type": "scatter",
         "x": [
          null
         ],
         "y": [
          null
         ]
        }
       ],
       "layout": {
        "template": {
         "data": {
          "bar": [
           {
            "error_x": {
             "color": "#2a3f5f"
            },
            "error_y": {
             "color": "#2a3f5f"
            },
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "bar"
           }
          ],
          "barpolar": [
           {
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "barpolar"
           }
          ],
          "carpet": [
           {
            "aaxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "baxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "type": "carpet"
           }
          ],
          "choropleth": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "choropleth"
           }
          ],
          "contour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "contour"
           }
          ],
          "contourcarpet": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "contourcarpet"
           }
          ],
          "heatmap": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmap"
           }
          ],
          "heatmapgl": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmapgl"
           }
          ],
          "histogram": [
           {
            "marker": {
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "histogram"
           }
          ],
          "histogram2d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2d"
           }
          ],
          "histogram2dcontour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2dcontour"
           }
          ],
          "mesh3d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "mesh3d"
           }
          ],
          "parcoords": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "parcoords"
           }
          ],
          "pie": [
           {
            "automargin": true,
            "type": "pie"
           }
          ],
          "scatter": [
           {
            "fillpattern": {
             "fillmode": "overlay",
             "size": 10,
             "solidity": 0.2
            },
            "type": "scatter"
           }
          ],
          "scatter3d": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatter3d"
           }
          ],
          "scattercarpet": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattercarpet"
           }
          ],
          "scattergeo": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergeo"
           }
          ],
          "scattergl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergl"
           }
          ],
          "scattermapbox": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattermapbox"
           }
          ],
          "scatterpolar": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolar"
           }
          ],
          "scatterpolargl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolargl"
           }
          ],
          "scatterternary": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterternary"
           }
          ],
          "surface": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "surface"
           }
          ],
          "table": [
           {
            "cells": {
             "fill": {
              "color": "#EBF0F8"
             },
             "line": {
              "color": "white"
             }
            },
            "header": {
             "fill": {
              "color": "#C8D4E3"
             },
             "line": {
              "color": "white"
             }
            },
            "type": "table"
           }
          ]
         },
         "layout": {
          "annotationdefaults": {
           "arrowcolor": "#2a3f5f",
           "arrowhead": 0,
           "arrowwidth": 1
          },
          "autotypenumbers": "strict",
          "coloraxis": {
           "colorbar": {
            "outlinewidth": 0,
            "ticks": ""
           }
          },
          "colorscale": {
           "diverging": [
            [
             0,
             "#8e0152"
            ],
            [
             0.1,
             "#c51b7d"
            ],
            [
             0.2,
             "#de77ae"
            ],
            [
             0.3,
             "#f1b6da"
            ],
            [
             0.4,
             "#fde0ef"
            ],
            [
             0.5,
             "#f7f7f7"
            ],
            [
             0.6,
             "#e6f5d0"
            ],
            [
             0.7,
             "#b8e186"
            ],
            [
             0.8,
             "#7fbc41"
            ],
            [
             0.9,
             "#4d9221"
            ],
            [
             1,
             "#276419"
            ]
           ],
           "sequential": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ],
           "sequentialminus": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ]
          },
          "colorway": [
           "#636efa",
           "#EF553B",
           "#00cc96",
           "#ab63fa",
           "#FFA15A",
           "#19d3f3",
           "#FF6692",
           "#B6E880",
           "#FF97FF",
           "#FECB52"
          ],
          "font": {
           "color": "#2a3f5f"
          },
          "geo": {
           "bgcolor": "white",
           "lakecolor": "white",
           "landcolor": "#E5ECF6",
           "showlakes": true,
           "showland": true,
           "subunitcolor": "white"
          },
          "hoverlabel": {
           "align": "left"
          },
          "hovermode": "closest",
          "mapbox": {
           "style": "light"
          },
          "paper_bgcolor": "white",
          "plot_bgcolor": "#E5ECF6",
          "polar": {
           "angularaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "radialaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "scene": {
           "xaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "yaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "zaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           }
          },
          "shapedefaults": {
           "line": {
            "color": "#2a3f5f"
           }
          },
          "ternary": {
           "aaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "baxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "caxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "title": {
           "x": 0.05
          },
          "xaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          },
          "yaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          }
         }
        },
        "title": {
         "text": "Rank (Objective Value)"
        },
        "xaxis": {
         "anchor": "y",
         "domain": [
          0,
          0.11624999999999999
         ],
         "matches": "x57",
         "range": [
          -4.931539976099404,
          -0.9005509601939562
         ],
         "showticklabels": false,
         "type": "log"
        },
        "xaxis10": {
         "anchor": "y10",
         "domain": [
          0.12625,
          0.2425
         ],
         "matches": "x58",
         "range": [
          0.8,
          5.2
         ],
         "showticklabels": false
        },
        "xaxis11": {
         "anchor": "y11",
         "domain": [
          0.2525,
          0.36875
         ],
         "matches": "x59",
         "range": [
          32.65,
          128.35
         ],
         "showticklabels": false
        },
        "xaxis12": {
         "anchor": "y12",
         "domain": [
          0.37875000000000003,
          0.495
         ],
         "matches": "x60",
         "range": [
          36.7,
          131.3
         ],
         "showticklabels": false
        },
        "xaxis13": {
         "anchor": "y13",
         "domain": [
          0.505,
          0.62125
         ],
         "matches": "x61",
         "range": [
          28.55,
          126.45
         ],
         "showticklabels": false
        },
        "xaxis14": {
         "anchor": "y14",
         "domain": [
          0.63125,
          0.7474999999999999
         ],
         "matches": "x62",
         "range": [
          28.3,
          131.7
         ],
         "showticklabels": false
        },
        "xaxis15": {
         "anchor": "y15",
         "domain": [
          0.7574999999999998,
          0.8737499999999998
         ],
         "matches": "x63",
         "range": [
          28.65,
          102.35
         ],
         "showticklabels": false
        },
        "xaxis16": {
         "anchor": "y16",
         "domain": [
          0.8837499999999998,
          0.9999999999999998
         ],
         "matches": "x64",
         "range": [
          -0.1,
          2.1
         ],
         "showticklabels": false,
         "type": "category"
        },
        "xaxis17": {
         "anchor": "y17",
         "domain": [
          0,
          0.11624999999999999
         ],
         "matches": "x57",
         "range": [
          -4.931539976099404,
          -0.9005509601939562
         ],
         "showticklabels": false,
         "type": "log"
        },
        "xaxis18": {
         "anchor": "y18",
         "domain": [
          0.12625,
          0.2425
         ],
         "matches": "x58",
         "range": [
          0.8,
          5.2
         ],
         "showticklabels": false
        },
        "xaxis19": {
         "anchor": "y19",
         "domain": [
          0.2525,
          0.36875
         ],
         "matches": "x59",
         "range": [
          32.65,
          128.35
         ],
         "showticklabels": false
        },
        "xaxis2": {
         "anchor": "y2",
         "domain": [
          0.12625,
          0.2425
         ],
         "matches": "x58",
         "range": [
          0.8,
          5.2
         ],
         "showticklabels": false
        },
        "xaxis20": {
         "anchor": "y20",
         "domain": [
          0.37875000000000003,
          0.495
         ],
         "matches": "x60",
         "range": [
          36.7,
          131.3
         ],
         "showticklabels": false
        },
        "xaxis21": {
         "anchor": "y21",
         "domain": [
          0.505,
          0.62125
         ],
         "matches": "x61",
         "range": [
          28.55,
          126.45
         ],
         "showticklabels": false
        },
        "xaxis22": {
         "anchor": "y22",
         "domain": [
          0.63125,
          0.7474999999999999
         ],
         "matches": "x62",
         "range": [
          28.3,
          131.7
         ],
         "showticklabels": false
        },
        "xaxis23": {
         "anchor": "y23",
         "domain": [
          0.7574999999999998,
          0.8737499999999998
         ],
         "matches": "x63",
         "range": [
          28.65,
          102.35
         ],
         "showticklabels": false
        },
        "xaxis24": {
         "anchor": "y24",
         "domain": [
          0.8837499999999998,
          0.9999999999999998
         ],
         "matches": "x64",
         "range": [
          -0.1,
          2.1
         ],
         "showticklabels": false,
         "type": "category"
        },
        "xaxis25": {
         "anchor": "y25",
         "domain": [
          0,
          0.11624999999999999
         ],
         "matches": "x57",
         "range": [
          -4.931539976099404,
          -0.9005509601939562
         ],
         "showticklabels": false,
         "type": "log"
        },
        "xaxis26": {
         "anchor": "y26",
         "domain": [
          0.12625,
          0.2425
         ],
         "matches": "x58",
         "range": [
          0.8,
          5.2
         ],
         "showticklabels": false
        },
        "xaxis27": {
         "anchor": "y27",
         "domain": [
          0.2525,
          0.36875
         ],
         "matches": "x59",
         "range": [
          32.65,
          128.35
         ],
         "showticklabels": false
        },
        "xaxis28": {
         "anchor": "y28",
         "domain": [
          0.37875000000000003,
          0.495
         ],
         "matches": "x60",
         "range": [
          36.7,
          131.3
         ],
         "showticklabels": false
        },
        "xaxis29": {
         "anchor": "y29",
         "domain": [
          0.505,
          0.62125
         ],
         "matches": "x61",
         "range": [
          28.55,
          126.45
         ],
         "showticklabels": false
        },
        "xaxis3": {
         "anchor": "y3",
         "domain": [
          0.2525,
          0.36875
         ],
         "matches": "x59",
         "range": [
          32.65,
          128.35
         ],
         "showticklabels": false
        },
        "xaxis30": {
         "anchor": "y30",
         "domain": [
          0.63125,
          0.7474999999999999
         ],
         "matches": "x62",
         "range": [
          28.3,
          131.7
         ],
         "showticklabels": false
        },
        "xaxis31": {
         "anchor": "y31",
         "domain": [
          0.7574999999999998,
          0.8737499999999998
         ],
         "matches": "x63",
         "range": [
          28.65,
          102.35
         ],
         "showticklabels": false
        },
        "xaxis32": {
         "anchor": "y32",
         "domain": [
          0.8837499999999998,
          0.9999999999999998
         ],
         "matches": "x64",
         "range": [
          -0.1,
          2.1
         ],
         "showticklabels": false,
         "type": "category"
        },
        "xaxis33": {
         "anchor": "y33",
         "domain": [
          0,
          0.11624999999999999
         ],
         "matches": "x57",
         "range": [
          -4.931539976099404,
          -0.9005509601939562
         ],
         "showticklabels": false,
         "type": "log"
        },
        "xaxis34": {
         "anchor": "y34",
         "domain": [
          0.12625,
          0.2425
         ],
         "matches": "x58",
         "range": [
          0.8,
          5.2
         ],
         "showticklabels": false
        },
        "xaxis35": {
         "anchor": "y35",
         "domain": [
          0.2525,
          0.36875
         ],
         "matches": "x59",
         "range": [
          32.65,
          128.35
         ],
         "showticklabels": false
        },
        "xaxis36": {
         "anchor": "y36",
         "domain": [
          0.37875000000000003,
          0.495
         ],
         "matches": "x60",
         "range": [
          36.7,
          131.3
         ],
         "showticklabels": false
        },
        "xaxis37": {
         "anchor": "y37",
         "domain": [
          0.505,
          0.62125
         ],
         "matches": "x61",
         "range": [
          28.55,
          126.45
         ],
         "showticklabels": false
        },
        "xaxis38": {
         "anchor": "y38",
         "domain": [
          0.63125,
          0.7474999999999999
         ],
         "matches": "x62",
         "range": [
          28.3,
          131.7
         ],
         "showticklabels": false
        },
        "xaxis39": {
         "anchor": "y39",
         "domain": [
          0.7574999999999998,
          0.8737499999999998
         ],
         "matches": "x63",
         "range": [
          28.65,
          102.35
         ],
         "showticklabels": false
        },
        "xaxis4": {
         "anchor": "y4",
         "domain": [
          0.37875000000000003,
          0.495
         ],
         "matches": "x60",
         "range": [
          36.7,
          131.3
         ],
         "showticklabels": false
        },
        "xaxis40": {
         "anchor": "y40",
         "domain": [
          0.8837499999999998,
          0.9999999999999998
         ],
         "matches": "x64",
         "range": [
          -0.1,
          2.1
         ],
         "showticklabels": false,
         "type": "category"
        },
        "xaxis41": {
         "anchor": "y41",
         "domain": [
          0,
          0.11624999999999999
         ],
         "matches": "x57",
         "range": [
          -4.931539976099404,
          -0.9005509601939562
         ],
         "showticklabels": false,
         "type": "log"
        },
        "xaxis42": {
         "anchor": "y42",
         "domain": [
          0.12625,
          0.2425
         ],
         "matches": "x58",
         "range": [
          0.8,
          5.2
         ],
         "showticklabels": false
        },
        "xaxis43": {
         "anchor": "y43",
         "domain": [
          0.2525,
          0.36875
         ],
         "matches": "x59",
         "range": [
          32.65,
          128.35
         ],
         "showticklabels": false
        },
        "xaxis44": {
         "anchor": "y44",
         "domain": [
          0.37875000000000003,
          0.495
         ],
         "matches": "x60",
         "range": [
          36.7,
          131.3
         ],
         "showticklabels": false
        },
        "xaxis45": {
         "anchor": "y45",
         "domain": [
          0.505,
          0.62125
         ],
         "matches": "x61",
         "range": [
          28.55,
          126.45
         ],
         "showticklabels": false
        },
        "xaxis46": {
         "anchor": "y46",
         "domain": [
          0.63125,
          0.7474999999999999
         ],
         "matches": "x62",
         "range": [
          28.3,
          131.7
         ],
         "showticklabels": false
        },
        "xaxis47": {
         "anchor": "y47",
         "domain": [
          0.7574999999999998,
          0.8737499999999998
         ],
         "matches": "x63",
         "range": [
          28.65,
          102.35
         ],
         "showticklabels": false
        },
        "xaxis48": {
         "anchor": "y48",
         "domain": [
          0.8837499999999998,
          0.9999999999999998
         ],
         "matches": "x64",
         "range": [
          -0.1,
          2.1
         ],
         "showticklabels": false,
         "type": "category"
        },
        "xaxis49": {
         "anchor": "y49",
         "domain": [
          0,
          0.11624999999999999
         ],
         "matches": "x57",
         "range": [
          -4.931539976099404,
          -0.9005509601939562
         ],
         "showticklabels": false,
         "type": "log"
        },
        "xaxis5": {
         "anchor": "y5",
         "domain": [
          0.505,
          0.62125
         ],
         "matches": "x61",
         "range": [
          28.55,
          126.45
         ],
         "showticklabels": false
        },
        "xaxis50": {
         "anchor": "y50",
         "domain": [
          0.12625,
          0.2425
         ],
         "matches": "x58",
         "range": [
          0.8,
          5.2
         ],
         "showticklabels": false
        },
        "xaxis51": {
         "anchor": "y51",
         "domain": [
          0.2525,
          0.36875
         ],
         "matches": "x59",
         "range": [
          32.65,
          128.35
         ],
         "showticklabels": false
        },
        "xaxis52": {
         "anchor": "y52",
         "domain": [
          0.37875000000000003,
          0.495
         ],
         "matches": "x60",
         "range": [
          36.7,
          131.3
         ],
         "showticklabels": false
        },
        "xaxis53": {
         "anchor": "y53",
         "domain": [
          0.505,
          0.62125
         ],
         "matches": "x61",
         "range": [
          28.55,
          126.45
         ],
         "showticklabels": false
        },
        "xaxis54": {
         "anchor": "y54",
         "domain": [
          0.63125,
          0.7474999999999999
         ],
         "matches": "x62",
         "range": [
          28.3,
          131.7
         ],
         "showticklabels": false
        },
        "xaxis55": {
         "anchor": "y55",
         "domain": [
          0.7574999999999998,
          0.8737499999999998
         ],
         "matches": "x63",
         "range": [
          28.65,
          102.35
         ],
         "showticklabels": false
        },
        "xaxis56": {
         "anchor": "y56",
         "domain": [
          0.8837499999999998,
          0.9999999999999998
         ],
         "matches": "x64",
         "range": [
          -0.1,
          2.1
         ],
         "showticklabels": false,
         "type": "category"
        },
        "xaxis57": {
         "anchor": "y57",
         "domain": [
          0,
          0.11624999999999999
         ],
         "range": [
          -4.931539976099404,
          -0.9005509601939562
         ],
         "title": {
          "text": "lr"
         },
         "type": "log"
        },
        "xaxis58": {
         "anchor": "y58",
         "domain": [
          0.12625,
          0.2425
         ],
         "range": [
          0.8,
          5.2
         ],
         "title": {
          "text": "n_layers"
         }
        },
        "xaxis59": {
         "anchor": "y59",
         "domain": [
          0.2525,
          0.36875
         ],
         "range": [
          32.65,
          128.35
         ],
         "title": {
          "text": "n_units_l0"
         }
        },
        "xaxis6": {
         "anchor": "y6",
         "domain": [
          0.63125,
          0.7474999999999999
         ],
         "matches": "x62",
         "range": [
          28.3,
          131.7
         ],
         "showticklabels": false
        },
        "xaxis60": {
         "anchor": "y60",
         "domain": [
          0.37875000000000003,
          0.495
         ],
         "range": [
          36.7,
          131.3
         ],
         "title": {
          "text": "n_units_l1"
         }
        },
        "xaxis61": {
         "anchor": "y61",
         "domain": [
          0.505,
          0.62125
         ],
         "range": [
          28.55,
          126.45
         ],
         "title": {
          "text": "n_units_l2"
         }
        },
        "xaxis62": {
         "anchor": "y62",
         "domain": [
          0.63125,
          0.7474999999999999
         ],
         "range": [
          28.3,
          131.7
         ],
         "title": {
          "text": "n_units_l3"
         }
        },
        "xaxis63": {
         "anchor": "y63",
         "domain": [
          0.7574999999999998,
          0.8737499999999998
         ],
         "range": [
          28.65,
          102.35
         ],
         "title": {
          "text": "n_units_l4"
         }
        },
        "xaxis64": {
         "anchor": "y64",
         "domain": [
          0.8837499999999998,
          0.9999999999999998
         ],
         "range": [
          -0.1,
          2.1
         ],
         "title": {
          "text": "optimizer"
         },
         "type": "category"
        },
        "xaxis7": {
         "anchor": "y7",
         "domain": [
          0.7574999999999998,
          0.8737499999999998
         ],
         "matches": "x63",
         "range": [
          28.65,
          102.35
         ],
         "showticklabels": false
        },
        "xaxis8": {
         "anchor": "y8",
         "domain": [
          0.8837499999999998,
          0.9999999999999998
         ],
         "matches": "x64",
         "range": [
          -0.1,
          2.1
         ],
         "showticklabels": false,
         "type": "category"
        },
        "xaxis9": {
         "anchor": "y9",
         "domain": [
          0,
          0.11624999999999999
         ],
         "matches": "x57",
         "range": [
          -4.931539976099404,
          -0.9005509601939562
         ],
         "showticklabels": false,
         "type": "log"
        },
        "yaxis": {
         "anchor": "x",
         "domain": [
          0.8837499999999998,
          0.9999999999999998
         ],
         "range": [
          -4.931539976099404,
          -0.9005509601939562
         ],
         "title": {
          "text": "lr"
         },
         "type": "log"
        },
        "yaxis10": {
         "anchor": "x10",
         "domain": [
          0.7574999999999998,
          0.8737499999999998
         ],
         "matches": "y9",
         "range": [
          0.8,
          5.2
         ],
         "showticklabels": false
        },
        "yaxis11": {
         "anchor": "x11",
         "domain": [
          0.7574999999999998,
          0.8737499999999998
         ],
         "matches": "y9",
         "range": [
          0.8,
          5.2
         ],
         "showticklabels": false
        },
        "yaxis12": {
         "anchor": "x12",
         "domain": [
          0.7574999999999998,
          0.8737499999999998
         ],
         "matches": "y9",
         "range": [
          0.8,
          5.2
         ],
         "showticklabels": false
        },
        "yaxis13": {
         "anchor": "x13",
         "domain": [
          0.7574999999999998,
          0.8737499999999998
         ],
         "matches": "y9",
         "range": [
          0.8,
          5.2
         ],
         "showticklabels": false
        },
        "yaxis14": {
         "anchor": "x14",
         "domain": [
          0.7574999999999998,
          0.8737499999999998
         ],
         "matches": "y9",
         "range": [
          0.8,
          5.2
         ],
         "showticklabels": false
        },
        "yaxis15": {
         "anchor": "x15",
         "domain": [
          0.7574999999999998,
          0.8737499999999998
         ],
         "matches": "y9",
         "range": [
          0.8,
          5.2
         ],
         "showticklabels": false
        },
        "yaxis16": {
         "anchor": "x16",
         "domain": [
          0.7574999999999998,
          0.8737499999999998
         ],
         "matches": "y9",
         "range": [
          0.8,
          5.2
         ],
         "showticklabels": false
        },
        "yaxis17": {
         "anchor": "x17",
         "domain": [
          0.63125,
          0.7474999999999999
         ],
         "range": [
          32.65,
          128.35
         ],
         "title": {
          "text": "n_units_l0"
         }
        },
        "yaxis18": {
         "anchor": "x18",
         "domain": [
          0.63125,
          0.7474999999999999
         ],
         "matches": "y17",
         "range": [
          32.65,
          128.35
         ],
         "showticklabels": false
        },
        "yaxis19": {
         "anchor": "x19",
         "domain": [
          0.63125,
          0.7474999999999999
         ],
         "matches": "y17",
         "range": [
          32.65,
          128.35
         ],
         "showticklabels": false
        },
        "yaxis2": {
         "anchor": "x2",
         "domain": [
          0.8837499999999998,
          0.9999999999999998
         ],
         "matches": "y",
         "range": [
          -4.931539976099404,
          -0.9005509601939562
         ],
         "showticklabels": false,
         "type": "log"
        },
        "yaxis20": {
         "anchor": "x20",
         "domain": [
          0.63125,
          0.7474999999999999
         ],
         "matches": "y17",
         "range": [
          32.65,
          128.35
         ],
         "showticklabels": false
        },
        "yaxis21": {
         "anchor": "x21",
         "domain": [
          0.63125,
          0.7474999999999999
         ],
         "matches": "y17",
         "range": [
          32.65,
          128.35
         ],
         "showticklabels": false
        },
        "yaxis22": {
         "anchor": "x22",
         "domain": [
          0.63125,
          0.7474999999999999
         ],
         "matches": "y17",
         "range": [
          32.65,
          128.35
         ],
         "showticklabels": false
        },
        "yaxis23": {
         "anchor": "x23",
         "domain": [
          0.63125,
          0.7474999999999999
         ],
         "matches": "y17",
         "range": [
          32.65,
          128.35
         ],
         "showticklabels": false
        },
        "yaxis24": {
         "anchor": "x24",
         "domain": [
          0.63125,
          0.7474999999999999
         ],
         "matches": "y17",
         "range": [
          32.65,
          128.35
         ],
         "showticklabels": false
        },
        "yaxis25": {
         "anchor": "x25",
         "domain": [
          0.505,
          0.62125
         ],
         "range": [
          36.7,
          131.3
         ],
         "title": {
          "text": "n_units_l1"
         }
        },
        "yaxis26": {
         "anchor": "x26",
         "domain": [
          0.505,
          0.62125
         ],
         "matches": "y25",
         "range": [
          36.7,
          131.3
         ],
         "showticklabels": false
        },
        "yaxis27": {
         "anchor": "x27",
         "domain": [
          0.505,
          0.62125
         ],
         "matches": "y25",
         "range": [
          36.7,
          131.3
         ],
         "showticklabels": false
        },
        "yaxis28": {
         "anchor": "x28",
         "domain": [
          0.505,
          0.62125
         ],
         "matches": "y25",
         "range": [
          36.7,
          131.3
         ],
         "showticklabels": false
        },
        "yaxis29": {
         "anchor": "x29",
         "domain": [
          0.505,
          0.62125
         ],
         "matches": "y25",
         "range": [
          36.7,
          131.3
         ],
         "showticklabels": false
        },
        "yaxis3": {
         "anchor": "x3",
         "domain": [
          0.8837499999999998,
          0.9999999999999998
         ],
         "matches": "y",
         "range": [
          -4.931539976099404,
          -0.9005509601939562
         ],
         "showticklabels": false,
         "type": "log"
        },
        "yaxis30": {
         "anchor": "x30",
         "domain": [
          0.505,
          0.62125
         ],
         "matches": "y25",
         "range": [
          36.7,
          131.3
         ],
         "showticklabels": false
        },
        "yaxis31": {
         "anchor": "x31",
         "domain": [
          0.505,
          0.62125
         ],
         "matches": "y25",
         "range": [
          36.7,
          131.3
         ],
         "showticklabels": false
        },
        "yaxis32": {
         "anchor": "x32",
         "domain": [
          0.505,
          0.62125
         ],
         "matches": "y25",
         "range": [
          36.7,
          131.3
         ],
         "showticklabels": false
        },
        "yaxis33": {
         "anchor": "x33",
         "domain": [
          0.37875000000000003,
          0.495
         ],
         "range": [
          28.55,
          126.45
         ],
         "title": {
          "text": "n_units_l2"
         }
        },
        "yaxis34": {
         "anchor": "x34",
         "domain": [
          0.37875000000000003,
          0.495
         ],
         "matches": "y33",
         "range": [
          28.55,
          126.45
         ],
         "showticklabels": false
        },
        "yaxis35": {
         "anchor": "x35",
         "domain": [
          0.37875000000000003,
          0.495
         ],
         "matches": "y33",
         "range": [
          28.55,
          126.45
         ],
         "showticklabels": false
        },
        "yaxis36": {
         "anchor": "x36",
         "domain": [
          0.37875000000000003,
          0.495
         ],
         "matches": "y33",
         "range": [
          28.55,
          126.45
         ],
         "showticklabels": false
        },
        "yaxis37": {
         "anchor": "x37",
         "domain": [
          0.37875000000000003,
          0.495
         ],
         "matches": "y33",
         "range": [
          28.55,
          126.45
         ],
         "showticklabels": false
        },
        "yaxis38": {
         "anchor": "x38",
         "domain": [
          0.37875000000000003,
          0.495
         ],
         "matches": "y33",
         "range": [
          28.55,
          126.45
         ],
         "showticklabels": false
        },
        "yaxis39": {
         "anchor": "x39",
         "domain": [
          0.37875000000000003,
          0.495
         ],
         "matches": "y33",
         "range": [
          28.55,
          126.45
         ],
         "showticklabels": false
        },
        "yaxis4": {
         "anchor": "x4",
         "domain": [
          0.8837499999999998,
          0.9999999999999998
         ],
         "matches": "y",
         "range": [
          -4.931539976099404,
          -0.9005509601939562
         ],
         "showticklabels": false,
         "type": "log"
        },
        "yaxis40": {
         "anchor": "x40",
         "domain": [
          0.37875000000000003,
          0.495
         ],
         "matches": "y33",
         "range": [
          28.55,
          126.45
         ],
         "showticklabels": false
        },
        "yaxis41": {
         "anchor": "x41",
         "domain": [
          0.2525,
          0.36875
         ],
         "range": [
          28.3,
          131.7
         ],
         "title": {
          "text": "n_units_l3"
         }
        },
        "yaxis42": {
         "anchor": "x42",
         "domain": [
          0.2525,
          0.36875
         ],
         "matches": "y41",
         "range": [
          28.3,
          131.7
         ],
         "showticklabels": false
        },
        "yaxis43": {
         "anchor": "x43",
         "domain": [
          0.2525,
          0.36875
         ],
         "matches": "y41",
         "range": [
          28.3,
          131.7
         ],
         "showticklabels": false
        },
        "yaxis44": {
         "anchor": "x44",
         "domain": [
          0.2525,
          0.36875
         ],
         "matches": "y41",
         "range": [
          28.3,
          131.7
         ],
         "showticklabels": false
        },
        "yaxis45": {
         "anchor": "x45",
         "domain": [
          0.2525,
          0.36875
         ],
         "matches": "y41",
         "range": [
          28.3,
          131.7
         ],
         "showticklabels": false
        },
        "yaxis46": {
         "anchor": "x46",
         "domain": [
          0.2525,
          0.36875
         ],
         "matches": "y41",
         "range": [
          28.3,
          131.7
         ],
         "showticklabels": false
        },
        "yaxis47": {
         "anchor": "x47",
         "domain": [
          0.2525,
          0.36875
         ],
         "matches": "y41",
         "range": [
          28.3,
          131.7
         ],
         "showticklabels": false
        },
        "yaxis48": {
         "anchor": "x48",
         "domain": [
          0.2525,
          0.36875
         ],
         "matches": "y41",
         "range": [
          28.3,
          131.7
         ],
         "showticklabels": false
        },
        "yaxis49": {
         "anchor": "x49",
         "domain": [
          0.12625,
          0.2425
         ],
         "range": [
          28.65,
          102.35
         ],
         "title": {
          "text": "n_units_l4"
         }
        },
        "yaxis5": {
         "anchor": "x5",
         "domain": [
          0.8837499999999998,
          0.9999999999999998
         ],
         "matches": "y",
         "range": [
          -4.931539976099404,
          -0.9005509601939562
         ],
         "showticklabels": false,
         "type": "log"
        },
        "yaxis50": {
         "anchor": "x50",
         "domain": [
          0.12625,
          0.2425
         ],
         "matches": "y49",
         "range": [
          28.65,
          102.35
         ],
         "showticklabels": false
        },
        "yaxis51": {
         "anchor": "x51",
         "domain": [
          0.12625,
          0.2425
         ],
         "matches": "y49",
         "range": [
          28.65,
          102.35
         ],
         "showticklabels": false
        },
        "yaxis52": {
         "anchor": "x52",
         "domain": [
          0.12625,
          0.2425
         ],
         "matches": "y49",
         "range": [
          28.65,
          102.35
         ],
         "showticklabels": false
        },
        "yaxis53": {
         "anchor": "x53",
         "domain": [
          0.12625,
          0.2425
         ],
         "matches": "y49",
         "range": [
          28.65,
          102.35
         ],
         "showticklabels": false
        },
        "yaxis54": {
         "anchor": "x54",
         "domain": [
          0.12625,
          0.2425
         ],
         "matches": "y49",
         "range": [
          28.65,
          102.35
         ],
         "showticklabels": false
        },
        "yaxis55": {
         "anchor": "x55",
         "domain": [
          0.12625,
          0.2425
         ],
         "matches": "y49",
         "range": [
          28.65,
          102.35
         ],
         "showticklabels": false
        },
        "yaxis56": {
         "anchor": "x56",
         "domain": [
          0.12625,
          0.2425
         ],
         "matches": "y49",
         "range": [
          28.65,
          102.35
         ],
         "showticklabels": false
        },
        "yaxis57": {
         "anchor": "x57",
         "domain": [
          0,
          0.11624999999999999
         ],
         "range": [
          -0.1,
          2.1
         ],
         "title": {
          "text": "optimizer"
         },
         "type": "category"
        },
        "yaxis58": {
         "anchor": "x58",
         "domain": [
          0,
          0.11624999999999999
         ],
         "matches": "y57",
         "range": [
          -0.1,
          2.1
         ],
         "showticklabels": false,
         "type": "category"
        },
        "yaxis59": {
         "anchor": "x59",
         "domain": [
          0,
          0.11624999999999999
         ],
         "matches": "y57",
         "range": [
          -0.1,
          2.1
         ],
         "showticklabels": false,
         "type": "category"
        },
        "yaxis6": {
         "anchor": "x6",
         "domain": [
          0.8837499999999998,
          0.9999999999999998
         ],
         "matches": "y",
         "range": [
          -4.931539976099404,
          -0.9005509601939562
         ],
         "showticklabels": false,
         "type": "log"
        },
        "yaxis60": {
         "anchor": "x60",
         "domain": [
          0,
          0.11624999999999999
         ],
         "matches": "y57",
         "range": [
          -0.1,
          2.1
         ],
         "showticklabels": false,
         "type": "category"
        },
        "yaxis61": {
         "anchor": "x61",
         "domain": [
          0,
          0.11624999999999999
         ],
         "matches": "y57",
         "range": [
          -0.1,
          2.1
         ],
         "showticklabels": false,
         "type": "category"
        },
        "yaxis62": {
         "anchor": "x62",
         "domain": [
          0,
          0.11624999999999999
         ],
         "matches": "y57",
         "range": [
          -0.1,
          2.1
         ],
         "showticklabels": false,
         "type": "category"
        },
        "yaxis63": {
         "anchor": "x63",
         "domain": [
          0,
          0.11624999999999999
         ],
         "matches": "y57",
         "range": [
          -0.1,
          2.1
         ],
         "showticklabels": false,
         "type": "category"
        },
        "yaxis64": {
         "anchor": "x64",
         "domain": [
          0,
          0.11624999999999999
         ],
         "matches": "y57",
         "range": [
          -0.1,
          2.1
         ],
         "showticklabels": false,
         "type": "category"
        },
        "yaxis7": {
         "anchor": "x7",
         "domain": [
          0.8837499999999998,
          0.9999999999999998
         ],
         "matches": "y",
         "range": [
          -4.931539976099404,
          -0.9005509601939562
         ],
         "showticklabels": false,
         "type": "log"
        },
        "yaxis8": {
         "anchor": "x8",
         "domain": [
          0.8837499999999998,
          0.9999999999999998
         ],
         "matches": "y",
         "range": [
          -4.931539976099404,
          -0.9005509601939562
         ],
         "showticklabels": false,
         "type": "log"
        },
        "yaxis9": {
         "anchor": "x9",
         "domain": [
          0.7574999999999998,
          0.8737499999999998
         ],
         "range": [
          0.8,
          5.2
         ],
         "title": {
          "text": "n_layers"
         }
        }
       }
      }
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_rank(study)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "tf-gpu",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}