EuclideanCI.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Derivation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Cressie-Read (-2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is computable from averages over the dataset, but calibrates better than Euclidean."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Direct"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This strategy directly integrates the level set condition.  The lower bound is shown here: the upper bound corresponds to negating $r$ everywhere and negating the result.\n",
    "\n",
    "Assume no duplicates for now (infinitesmal futzing, $c_{w,r} = 0$ or $1$).\n",
    "$$\n",
    "\\begin{alignat}{2}\n",
    "&\\!\\min_{Q \\succeq 0} &\\qquad& \\vec{w}^\\top Q \\vec{r},\\label{eq:mleci}\\\\\n",
    "&\\text{subject to} &  & \\vec{w}^\\top Q \\vec{1} = 1, \\tag{$\\beta$} \\label{eq:mlecisumw} \\\\\n",
    "&                  &  & \\vec{1}^\\top Q \\vec{1} = 1, \\tag{$\\gamma$} \\label{eq:mlecisum} \\\\\n",
    "&                  &  & \\sum_{n} \\left( \\left(N Q_{w_n,r_n}\\right)^2 - 1\\right) \\leq \\Delta + \\sum_{n} \\left(\\left(N Q^{\\text{(mle)}}_{w_n,r_n}\\right)^2 - 1\\right), \\tag{$\\kappa$}\n",
    "\\end{alignat}\n",
    "$$\n",
    "Lagrangian dual $\\ldots$\n",
    "$$\n",
    "\\begin{aligned}\n",
    "&L(\\beta, \\gamma, \\kappa, Q) \\\\\n",
    "&=\\vec{w}^\\top Q \\vec{r} + \\kappa \\left(\\sum_{n} \\left( \\left(N Q_{w_n,r_n}\\right)^2 - 1\\right) - \\Delta - \\sum_{n} \\left(\\left(N Q^{\\text{(mle)}}_{w_n,r_n}\\right)^2 - 1\\right) \\right) + \\gamma \\left(\\vec{1}^\\top Q \\vec{1} - 1\\right) + \\beta \\left( \\vec{w}^\\top Q \\vec{1} - 1 \\right) \\\\\n",
    "&= \\kappa \\left( -\\Delta - \\sum_{n} \\left(\\left(N Q^{\\text{(mle)}}_{w_n,r_n}\\right)^2 - 1\\right) \\right) -\\gamma -\\beta + \\sum_{w,r} \\left( \\kappa c_{w,r} \\left( \\left(N Q_{w_n,r_n}\\right)^2 - 1\\right) + \\left(\\gamma + \\beta w + w r\\right)Q_{w,r}\\right), \\\\\n",
    "&= \\kappa \\biggl( 2 N \\phi \\biggr) -\\gamma -\\beta + \\sum_{w,r} \\left( \\kappa c_{w,r} \\left( \\left(N Q_{w_n,r_n}\\right)^2 - 1\\right) + \\left(\\gamma + \\beta w + w r\\right)Q_{w,r}\\right), \\\\\n",
    "\\end{aligned}\n",
    "$$ which is unbounded unless $\\forall w, r: \\gamma + \\beta w + w r \\geq 0$, but in that case $Q = 0$ for observed data points."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Empirical on augmented support"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can place all mass on the sample and discard the dual variable constraint.  We augment the support where the log-likelihood solution potentially places additional mass: at an extreme value of $w_{\\text{fake}}$ associated with $r_{\\min}$.  We can try both $w_{\\min}$ and $w_{\\max}$.\n",
    "\n",
    "$$\n",
    "\\begin{aligned}\n",
    "Q_{w,r} &= \\max\\left\\{0, -\\frac{\\gamma + \\beta w + w r}{2 (N+1)^2 \\kappa}\\right\\}\n",
    "\\end{aligned}\n",
    "$$ Allow signed measures and drop the $\\max$.  Substitute $\\kappa \\leftarrow \\frac{1}{2 (N+1)} \\kappa$ to get $$\n",
    "\\begin{aligned}\n",
    "Q_{w,r} &= -\\frac{\\gamma + \\beta w + w r}{(N+1) \\kappa} \\\\\n",
    "g(\\beta, \\gamma, \\kappa) &= \\kappa \\left(\\phi - \\frac{1}{2} \\right) - \\gamma - \\beta + \\sum_{n \\cup \\text{fake}} \\frac{(N+1)^2}{2 (N + 1)} \\kappa \\left(\\frac{\\gamma + \\beta w_n + w_n r_n}{(N+1)\\kappa}\\right)^2 - \\frac{(\\gamma + \\beta w_n + w_n r_n)^2}{(N+1) \\kappa} \\\\\n",
    "&= \\kappa \\left(\\phi - \\frac{1}{2} \\right) - \\gamma - \\beta - \\frac{1}{2} \\sum_{n \\cup \\text{fake}} \\frac{(\\gamma + \\beta w_n + w_n r_n)^2}{(N+1) \\kappa} \\\\\n",
    "&= \\kappa \\left(\\phi - \\frac{1}{2} \\right) - \\gamma - \\beta - \\frac{1}{2 \\kappa} \\left( \\gamma^2 + 2 \\gamma \\beta \\overline{w} + 2 \\gamma \\overline{w r} + \\beta^2 \\overline{w^2} + 2 \\beta \\overline{w^2 r} + \\overline{w^2 r^2} \\right)\n",
    "\\end{aligned}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\begin{aligned}\n",
    "g(\\beta, \\gamma, \\kappa) &= \\kappa \\left(\\phi - \\frac{1}{2} \\right) - \\frac{1}{2 \\kappa} \\overline{w^2 r^2} - \\left(\\begin{matrix} 1 + \\frac{\\overline{w r}}{\\kappa} \\\\ 1 + \\frac{\\overline{w^2 r}}{\\kappa} \\end{matrix}\\right)^\\top \\left(\\begin{matrix} \\gamma \\\\ \\beta \\end{matrix}\\right) + \\frac{1}{2}  \\left(\\begin{matrix} \\gamma \\\\ \\beta \\end{matrix}\\right)^\\top   \\left(\\begin{matrix} -\\frac{1}{\\kappa} & -\\frac{\\overline{w}}{\\kappa} \\\\ -\\frac{\\overline{w}}{\\kappa} & -\\frac{\\overline{w^2}}{\\kappa}\\end{matrix}\\right) \\left(\\begin{matrix} \\gamma \\\\ \\beta \\end{matrix}\\right) \\\\\n",
    "\\left(\\begin{matrix} \\gamma^* \\\\ \\beta^* \\end{matrix}\\right) &=  \\left(\\begin{matrix} -\\frac{1}{\\kappa} & -\\frac{\\overline{w}}{\\kappa} \\\\ -\\frac{\\overline{w}}{\\kappa} & -\\frac{\\overline{w^2}}{\\kappa}\\end{matrix}\\right)^{-1} \\left(\\begin{matrix} 1 + \\frac{\\overline{w r}}{\\kappa} \\\\ 1 + \\frac{\\overline{w^2 r}}{\\kappa} \\end{matrix}\\right) = -\\left(\\begin{matrix} 1 & \\overline{w} \\\\ \\overline{w} & \\overline{w^2} \\end{matrix}\\right)^{-1} \\left(\\begin{matrix} \\kappa + \\overline{w r} \\\\ \\kappa + \\overline{w^2 r} \\end{matrix}\\right) = \\frac{1}{\\overline{w^2} - \\overline{w}^2} \\left(\\begin{matrix} -\\overline{w^2} & \\overline{w} \\\\ \\overline{w} & -1 \\end{matrix}\\right) \\left(\\begin{matrix} \\kappa + \\overline{w r} \\\\ \\kappa + \\overline{w^2 r} \\end{matrix}\\right)  \\\\\n",
    "&\\doteq \\kappa \\vec{a} + \\vec{b} \\\\\n",
    "a &= \\frac{1}{\\overline{w^2} - \\overline{w}^2} \\left(\\begin{matrix} -\\overline{w^2} & \\overline{w} \\\\ \\overline{w} & -1 \\end{matrix}\\right) \\vec{1} \\\\\n",
    "b &=  \\frac{1}{\\overline{w^2} - \\overline{w}^2} \\left(\\begin{matrix} -\\overline{w^2} & \\overline{w} \\\\ \\overline{w} & -1 \\end{matrix}\\right) \\left(\\begin{matrix} \\overline{w r} \\\\ \\overline{w^2 r} \\end{matrix}\\right)  \\\\\n",
    "g(\\kappa) &= \\kappa \\left(\\phi - \\frac{1}{2} \\right) - \\frac{1}{2 \\kappa} \\overline{w^2 r^2} - \\frac{1}{2} \\left(\\begin{matrix} 1 + \\frac{\\overline{w r}}{\\kappa} \\\\ 1 + \\frac{\\overline{w^2 r}}{\\kappa} \\end{matrix}\\right)^\\top \\left(\\begin{matrix} \\gamma^* \\\\ \\beta^* \\end{matrix}\\right) \\\\\n",
    "&= \\kappa \\left(\\phi - \\frac{1}{2} \\right) - \\frac{1}{2 \\kappa} \\overline{w^2 r^2} - \\frac{1}{2} \\kappa \\vec{1}^\\top \\vec{a} - \\frac{1}{2} \\vec{1}^\\top \\vec{b} -\\frac{1}{2} \\left(\\begin{matrix} \\overline{w r} \\\\ \\overline{w^2 r} \\end{matrix}\\right)^\\top \\vec{a} -\\frac{1}{2 \\kappa} \\left(\\begin{matrix} \\overline{w r} \\\\ \\overline{w^2 r} \\end{matrix}\\right)^\\top \\vec{b} \\\\\n",
    "&= - \\frac{1}{2} \\vec{1}^\\top \\vec{b} -\\frac{1}{2} \\left(\\begin{matrix} \\overline{w r} \\\\ \\overline{w^2 r} \\end{matrix}\\right)^\\top \\vec{a} + \\frac{1}{2 \\kappa} \\left( -\\overline{w^2 r^2} -\\left(\\begin{matrix} \\overline{w r} \\\\ \\overline{w^2 r} \\end{matrix}\\right)^\\top \\vec{b} \\right)  + \\kappa \\left(\\phi - \\frac{1}{2} - \\frac{1}{2} \\vec{1}^\\top \\vec{a} \\right)  \\\\\n",
    "&\\doteq x + y \\frac{1}{2 \\kappa} + z \\kappa \\\\\n",
    "x &= - \\frac{1}{2} \\vec{1}^\\top \\vec{b} -\\frac{1}{2} \\left(\\begin{matrix} \\overline{w r} \\\\ \\overline{w^2 r} \\end{matrix}\\right)^\\top \\vec{a} =\n",
    "\\overline{w r} + \\frac{\\left(1 - \\overline{w}\\right) \\left(\\overline{w^2 r} - \\overline{w}\\ \\overline{w r} \\right)}{\\overline{w^2} - \\overline{w}^2} \\\\\n",
    "y &= -\\overline{w^2 r^2} -\\left(\\begin{matrix} \\overline{w r} \\\\ \\overline{w^2 r} \\end{matrix}\\right)^\\top \\vec{b} = \\frac{\\left(\\overline{w^2 r} - \\overline{w}\\ \\overline{w r}\\right)^2}{\\overline{w^2} - \\overline{w}^2} - \\left( \\overline{w^2 r^2} - \\overline{wr}^2 \\right) \\\\\n",
    "z &= \\phi - \\frac{1}{2} - \\frac{1}{2} \\vec{1}^\\top \\vec{a}  = \\phi + \\frac{\\left(1 - \\overline{w}\\right)^2}{2 \\left(\\overline{w^2} - (\\overline{w})^2\\right)} \\\\\n",
    "\\kappa &= \\sqrt{\\frac{y}{2 z}} & (y \\leq 0, z < 0) \\\\\n",
    "g &= x - \\sqrt{2 y z}\n",
    "\\end{aligned}\n",
    "$$"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "Inverse[{ { 1, wbar }, { wbar, wsqbar } } / \\[Kappa]] // FullSimplify\n",
    "Dot[-%, { 1 + wrbar / \\[Kappa], 1 + wsqrbar / \\[Kappa] }] // FullSimplify\n",
    "-1/2 Dot[%, { 1 + wrbar / \\[Kappa], 1 + wsqrbar / \\[Kappa] }] // FullSimplify // Collect[#, \\[Kappa]]&\n",
    "\\[Kappa] (\\[Phi] - 1/2) - 1/(2\\[Kappa]) wsqrsqbar + % // FullSimplify // Collect[#, \\[Kappa]]&\n",
    "x + y/(2 \\[Kappa]) + z \\[Kappa] == % /. x -> (wrbar + (1 - wbar) (wsqrbar - wbar wrbar) / (wsqbar - wbar^2)) /. y ->  (wsqrbar - wbar wrbar)^2  / (wsqbar - wbar^2) - (wsqrsqbar - wrbar^2) /. z -> \\[Phi] + (1 - wbar)^2 / (2 (wsqbar - wbar^2)) // FullSimplify\n",
    "\n",
    "{{ -wsqbar, wbar } , { wbar, -1 }} / (wsqbar - wbar^2) // MatrixForm\n",
    "a = Dot[%, {1, 1 }] // FullSimplify\n",
    "b = Dot[%%, { wrbar, wsqrbar }] // FullSimplify\n",
    "x = -1/2 Dot[{1, 1}, b] - 1/2 Dot[{wrbar, wsqrbar}, a] // FullSimplify \n",
    "x == wrbar + (1 - wbar) (wsqrbar - wbar wrbar)/(wsqbar - wbar^2) // FullSimplify\n",
    "y = -wsqrsqbar - Dot[{wrbar, wsqrbar}, b] // FullSimplify \n",
    "y == (wsqrbar - wbar wrbar)^2 / (wsqbar - wbar^2) -(wsqrsqbar - wrbar^2) // FullSimplify\n",
    "z = \\[Phi] -1/2 - 1/2 Dot[{1, 1}, a] // FullSimplify"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Limiting behaviour, $w_{\\max} \\to \\infty$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Turns out the confidence interval has a non-vacuous limit as $w_{\\max} \\to \\infty$."
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "Series[wrbar + (1 - wbar) (wsqrbar - wbar wrbar)/(wsqbar - wbar^2) /. wbar -> (sumw + wfake) / (1 + n) /. wsqbar -> (sumwsq + wfake^2) / (1 + n) /. wsqrbar -> (sumwsqr + wfake^2 rfake) / (1 + n) /. wrbar -> (sumwr + wfake rfake) / (1 + n) , { wfake, Infinity, 0 }] // FullSimplify (* x *)\n",
    "Series[(wsqrbar - wbar wrbar)^2 / (wsqbar - wbar^2) -(wsqrsqbar - wrbar^2) /. wbar -> (sumw + wfake) / (1 + n) /. wsqbar -> (sumwsq + wfake^2) / (1 + n) /. wrbar -> (sumwr + wfake rfake) / (1 + n) /. wsqrbar -> (sumwsqr + wfake^2 rfake) / (1 + n) /. wsqrsqbar -> (sumwsqrsq + wfake^2 rfake^2) / (1 + n), { wfake, Infinity, 0 }] // FullSimplify (* y *)\n",
    "{{ -wsqbar, wbar } , { wbar, -1 }} / (wsqbar - wbar^2) /. wbar -> (sumw + wfake) / (1 + n) /. wsqbar -> (sumwsq + wfake^2) / (1 + n) // MatrixForm \n",
    "Series[\\[Phi] -1/2 - 1/2 Dot[{1, 1}, Dot[%, {1, 1 }]], { wfake, Infinity, 0 }] // FullSimplify (* z *)\n",
    "{{ -wsqbar, wbar } , { wbar, -1 }} / (wsqbar - wbar^2) /. wbar -> (sumw + wfake) / (1 + n) /. wsqbar -> (sumwsq + wfake^2) / (1 + n) // MatrixForm\n",
    "Series[Dot[%, {1, 1 }], { wfake, Infinity, 2 }] // FullSimplify (* a *)\n",
    "Series[Dot[%%, { wrbar, wsqrbar }] /. wrbar -> (sumwr + wfake rfake) / (1 + n) /. wsqrbar -> (sumwsqr + wfake^2 rfake) / (1 + n), { wfake, Infinity, 2 }] // FullSimplify (* b *)\n",
    "Dot[\\[Kappa] Dot[%%%, {1 , 1}] + Dot[%%%, { wrbar, wsqrbar }], {1, wfake }] + wfake rfake /. wrbar -> (sumwr + wfake rfake) / (1 + n) /. wsqrbar -> (sumwsqr + wfake^2 rfake) / (1 + n) // FullSimplify\n",
    "Series[wfake (-%) / (\\[Kappa] (n + 1)), { wfake, Infinity, 0 }] // FullSimplify (* wfake Q_{wfake, rfake} *)\n",
    "% // Normal // FullSimplify // Collect[#, { \\[Kappa] }]& // { #[[1]] // FullSimplify, #[[2]] }&"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\begin{aligned}\n",
    "\\lim_{w_{\\text{fake}} \\to \\infty} x &\\to r_{\\min} + \\frac{1}{N} \\sum_n w_n (r_n - r_{\\min}) \\\\\n",
    "\\lim_{w_{\\text{fake}} \\to \\infty} y &\\to \\frac{\\left(r_{\\min} \\sum_n w_n - \\sum_n w_n r_n\\right)^2}{N (N + 1)} - \\frac{r_{\\min}^2 \\sum_n w_n^2 - 2 r_{\\min} \\sum_n w_n^2 r_n + \\sum_n w_n^2 r_n^2}{N + 1} \\\\\n",
    "\\lim_{w_{\\text{fake}} \\to \\infty} z &\\to \\phi + \\frac{1}{2 N} < -\\frac{1}{2 N} \\Delta < 0 \\\\\n",
    "\\lim_{w_{\\text{fake}} \\to \\infty} a &\\to -\\frac{N+1}{N} \\left( \\begin{matrix} 1 \\\\ 0 \\end{matrix} \\right) \\\\\n",
    "\\lim_{w_{\\text{fake}} \\to \\infty} b &\\to \\left( \\begin{matrix} \\frac{r_{\\min} \\sum_n w_n - \\sum_n w_n r_n}{N} \\\\ -r_{\\min} \\end{matrix} \\right) \\\\\n",
    "\\lim_{w_{\\text{fake}} \\to \\infty} \\gamma^* &\\to -\\kappa \\frac{N + 1}{N} + \\frac{r_{\\min} \\sum_n w_n - \\sum_n w_n r_n}{N} \\\\\n",
    "%\\lim_{w_{\\text{fake}} \\to \\infty} \\left(\\beta^* + r_n\\right) w_n &\\to w_n \\left(r_n -r_{\\min}\\right) \\\\\n",
    "%\\lim_{w_{\\text{fake}} \\to \\infty} \\left(\\beta^* + r_{\\min}\\right) w_{\\text{fake}} &\\to \\frac{\\sum_n w_n (r_n - r_{\\min})}{N} \\\\\n",
    "\\lim_{w_{\\text{fake}} \\to \\infty} \\beta^* &\\to -r_{\\min} + \\frac{\\sum_n w_n (r_n - r_{\\min})}{N} \\frac{1}{w_{\\text{fake}}} \\\\\n",
    "\\lim_{w_{\\text{fake}} \\to \\infty} Q_{w_n,r_n} &\\to \\frac{1}{N} - \\frac{1}{\\kappa (N + 1)} \\left(w_n \\left(r_n - r_{\\min}\\right) - \\frac{1}{N} \\sum_n w_n \\left(r_n - r_{\\min}\\right) \\right) \\\\\n",
    "\\lim_{w_{\\text{fake}} \\to \\infty} \\sum_n w_n r_n Q_{w_n,r_n} &\\to \\frac{1}{N} \\sum_n w_n r_n - \\frac{1}{\\kappa (N + 1)} \\left(\\left( \\sum_n w_n^2 \\left(r_n^2 - r_n r_{\\min}\\right) \\right) - \\frac{1}{N} \\left(\\sum_n w_n r_n\\right) \\left( \\sum_n w_n \\left(r_n - r_{\\min}\\right) \\right) \\right) \\\\\n",
    "\\lim_{w_{\\text{fake}} \\to \\infty} \\sum_n w_n Q_{w_n,r_n} &\\to \\frac{1}{N} \\sum_n w_n - \\frac{1}{\\kappa (N + 1)} \\left(\\left( \\sum_n w_n^2 \\left(r_n - r_{\\min}\\right) \\right) - \\frac{1}{N} \\left(\\sum_n w_n\\right) \\left( \\sum_n w_n \\left(r_n - r_{\\min}\\right) \\right) \\right) \\\\\n",
    "\\lim_{w_{\\text{fake}} \\to \\infty} w_{\\text{fake}} Q_{w_{\\text{fake}},r_{\\min}} &\\to 1 - \\frac{1}{N} \\sum_n w_n - \\frac{1}{\\kappa (N + 1)} \\left(\\left( \\sum_n w_n^2 \\left(r_n - r_{\\min}\\right) \\right) - \\frac{1}{N} \\left(\\sum_n w_n\\right) \\left( \\sum_n w_n \\left(r_n - r_{\\min}\\right) \\right) \\right)\n",
    "\\end{aligned}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "heading_collapsed": true
   },
   "source": [
    "## Euclidean"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true
   },
   "source": [
    "The key benefit here is something that can be computed using moments of the data, and is therefore amenable to integration with online optimization or other streaming data scenarios.\n",
    "\n",
    "The interval does not calibrate with $\\chi^2$ asymptotically.  This could be because the Euclidean divergence is not in the Cressie-Read family unless all mass is assigned to the sample."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "heading_collapsed": true,
    "hidden": true
   },
   "source": [
    "### Direct"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true
   },
   "source": [
    "This strategy directly integrates the level set condition.  The lower bound is shown here: the upper bound corresponds to negating $r$ everywhere and negating the result.\n",
    "\n",
    "Assume no duplicates for now (infinitesmal futzing, $c_{w,r} = 0$ or $1$).\n",
    "$$\n",
    "\\begin{alignat}{2}\n",
    "&\\!\\min_{Q \\succeq 0} &\\qquad& \\vec{w}^\\top Q \\vec{r},\\label{eq:mleci}\\\\\n",
    "&\\text{subject to} &  & \\vec{w}^\\top Q \\vec{1} = 1, \\tag{$\\beta$} \\label{eq:mlecisumw} \\\\\n",
    "&                  &  & \\vec{1}^\\top Q \\vec{1} = 1, \\tag{$\\gamma$} \\label{eq:mlecisum} \\\\\n",
    "&                  &  & \\sum_{n} \\frac{1}{2} \\left(N Q_{w_n,r_n} - 1\\right)^2 \\leq \\Delta + \\sum_{n} \\frac{1}{2} \\left(N Q^{\\text{(mle)}}_{w_n,r_n} - 1\\right)^2, \\tag{$\\kappa$}\n",
    "\\end{alignat}\n",
    "$$\n",
    "Lagrangian dual $\\ldots$\n",
    "$$\n",
    "\\begin{aligned}\n",
    "&L(\\beta, \\gamma, \\kappa, Q) \\\\\n",
    "&=\\vec{w}^\\top Q \\vec{r} + \\kappa \\left(\\sum_n \\frac{1}{2} \\left(N Q_{w_n,r_n} - 1\\right)^2 - \\Delta - \\sum_n \\frac{1}{2} \\left(N Q^{\\text{(mle)}}_{w_n,r_n} - 1\\right)^2 \\right) + \\gamma \\left(\\vec{1}^\\top Q \\vec{1} - 1\\right) + \\beta \\left( \\vec{w}^\\top Q \\vec{1} - 1 \\right) \\\\\n",
    "&= \\kappa \\left( -\\Delta - \\sum_n \\frac{1}{2} \\left(N Q^{\\text{(mle)}}_{w_n,r_n} - 1\\right)^2 \\right) -\\gamma -\\beta + \\sum_{w,r} \\left( \\kappa c_{w,r} \\frac{1}{2} \\left(N Q_{w,r} - 1\\right)^2 + \\left(\\gamma + \\beta w + w r\\right)Q_{w,r}\\right), \\\\\n",
    "\\end{aligned}\n",
    "$$ Dual will be unbounded unless $\\forall w, r: \\gamma + \\beta w + w r \\geq 0 \\lor c_{w,r} > 0$.  With infinistemal futzing, this is equivalent to $\\forall w, r: \\gamma + \\beta w + w r \\geq 0$. $\\gamma + \\beta w + w r = 0$ can only happen either everywhere or at an extreme value of $w$, and with $r_{\\min}$ (unless $w_{\\min}=0$ but then $r_{\\min}$ is sufficient to consider).  So we will only potentially place undata on an extreme point of $w$ associated with $r_{\\min}$.  Continuing $\\ldots$\n",
    "<!---\n",
    "\\[Kappa] 1/2 (n q - 1)^2 + (\\[Gamma] + \\[Beta] w + w r) q \n",
    "Solve[D[%, q] == 0, q] // FullSimplify // Collect[#, n]&\n",
    "%% /. %[[1]] // FullSimplify // Collect[#, n]&\n",
    "--->\n",
    "$$\n",
    "\\begin{aligned}\n",
    "Q^*_{w,r} &= \\max\\left\\{ 0, \\frac{1}{N} - \\frac{\\gamma + \\beta w + w r}{\\kappa N^2} \\right\\} & (\\kappa c_{w,r} > 0)\n",
    "\\end{aligned}\n",
    "$$\n",
    "The  max{0,…}  is difficult to deal with so ignore that for the purpose of finding closed-form expressions for the (approximate) dual variables. Continuing $\\ldots$\n",
    "$$\n",
    "\\begin{aligned}\n",
    "g (\\beta, \\gamma, \\kappa) &= \\inf_{Q \\succeq 0} L(Q, \\beta, \\gamma, \\kappa) \\\\\n",
    "&\\geq \\kappa \\left( -\\Delta - \\sum_n -\\frac{1}{2} \\left(N Q^{\\text{(mle)}}_{w_n,r_n} - 1\\right)^2 \\right) -\\gamma -\\beta + \\sum_n \\left( \\frac{\\gamma + \\beta w_n + w_n r_n}{N} - \\frac{\\left(\\gamma + \\beta w_n + w_n r_n\\right)^2}{2 N^2 \\kappa} \\right) \\\\\n",
    "&= \\kappa \\left( -\\Delta - \\sum_n \\frac{1}{2} \\left(N Q^{\\text{(mle)}}_{w_n,r_n} - 1\\right)^2 \\right) -\\beta + \\sum_n \\left(\\frac{\\beta w_n + w_n r_n}{N} - \\frac{\\left(\\gamma + \\beta w_n + w_n r_n\\right)^2}{2 N^2 \\kappa} \\right) \\\\\n",
    "&= \\overline{w r} - \\kappa \\phi + \\beta \\left(-1 + \\overline{w}\\right) - \\sum_n \\frac{\\left(\\gamma + \\beta w_n + w_n r_n\\right)^2}{2 N^2 \\kappa} \n",
    "\\end{aligned}\n",
    "$$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true
   },
   "source": [
    "#### $\\gamma^*$ "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true
   },
   "source": [
    "The unconstrained $\\gamma$ optimum is $-\\frac{1}{N} \\sum_n \\left(\\beta w_n + w_n r_n\\right) = -\\beta \\overline{w} - \\overline{w r}$ but this is infeasible.  Therefore maximizing $\\gamma$ under the constraint is $$\n",
    "\\begin{aligned}\n",
    "\\forall w, r: \\gamma &\\geq -\\beta w - w r \\\\\n",
    "\\gamma^* &= \\max_{w,r} \\left(-\\beta w - w r \\right) \\\\\n",
    "&= \\max_w \\left(-\\beta w - w r_{\\min} \\right) & (w \\geq 0) \\\\\n",
    "&= \\begin{cases} \n",
    "-\\left( \\beta + r_{\\min} \\right) w_{\\max} & \\beta + r_{\\min} < 0 \\\\\n",
    "-\\left( \\beta + r_{\\min} \\right) w_{\\min} & \\beta + r_{\\min} \\geq 0\n",
    "\\end{cases} \\\\\n",
    "&\\doteq -\\left( \\beta + r_{\\min} \\right) w_{\\text{ex}}\n",
    "\\end{aligned}\n",
    "$$ Substituting\n",
    "$$\n",
    "\\begin{aligned}\n",
    "g(\\beta, \\kappa) &= \\overline{w r} - \\kappa \\phi + \\beta \\left(-1 + \\overline{w}\\right) - \\sum_n \\frac{\\left(\\gamma + \\beta w_n + w_n r_n\\right)^2}{2 N^2 \\kappa} \\\\\n",
    "&= \\overline{w r} - \\kappa \\phi + \\beta \\left(-1 + \\overline{w}\\right) - \\sum_n \\frac{\\left(\\beta (w_n - w_{\\text{ex}}) + w_n r_n - w_{\\text{ex}} r_{\\min}\\right)^2}{2 N^2 \\kappa} \\\\\n",
    "\\end{aligned}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true
   },
   "source": [
    "#### $\\kappa^*$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true
   },
   "source": [
    "<!---\n",
    "Maximize[{ a - \\[Phi] \\[Kappa] - c / \\[Kappa], \\[Kappa] >= 0, c >= 0, \\[Phi] > 0 }, \\[Kappa]]\n",
    "--->\n",
    "Rewrite the dual objective\n",
    "$$\n",
    "\\begin{aligned}\n",
    "g(\\beta, \\kappa) &= a(\\beta) - \\kappa \\phi - \\frac{1}{\\kappa} c(\\beta) \\\\\n",
    "a(\\beta) &\\doteq \\overline{w r} + \\beta \\left(-1 + \\overline{w}\\right) \\\\\n",
    "c(\\beta) &\\doteq \\sum_n \\frac{\\left(\\beta (w_n - w_{\\text{ex}}) + w_n r_n - w_{\\text{ex}} r_{\\min}\\right)^2}{2 N^2} \\\\\n",
    "\\end{aligned}\n",
    "$$ where $\\phi > 0$ and $c(\\beta) \\geq 0$ are guaranteed by definition.  Thus\n",
    "$$\n",
    "\\begin{aligned}\n",
    "\\kappa^*(\\beta) &= \\sqrt{\\frac{c(\\beta)}{\\phi}} \\\\\n",
    "g(\\beta) &= a(\\beta) - 2 \\sqrt{c(\\beta) \\phi} \\\\\n",
    "&= \\overline{w r} + \\beta \\left(-1 + \\overline{w}\\right) - 2 \\sqrt{\\phi} \\sqrt{\\sum_n \\frac{\\left(\\beta (w_n - w_{\\text{ex}}) + w_n r_n - w_{\\text{ex}} r_{\\min}\\right)^2}{2 N^2}} \\\\\n",
    "\\end{aligned}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true
   },
   "source": [
    "#### $\\beta^*$ and $g^*$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true
   },
   "source": [
    "$$\n",
    "\\begin{aligned}\n",
    "g(\\beta) &= \\overline{w r} + \\beta \\left(-1 + \\overline{w}\\right) - 2 \\sqrt{\\frac{\\phi}{N}} \\sqrt{\\frac{1}{2} \\frac{1}{N} \\sum_n \\left(\\beta (w_n - w_{\\text{ex}}) + w_n r_n - w_{\\text{ex}} r_{\\min}\\right)^2} \\\\\n",
    "&= \\overline{w r} + \\beta \\left(-1 + \\overline{w}\\right) - 2 \\sqrt{\\frac{\\phi}{N}} \\sqrt{\\frac{1}{2} \\beta^2 x_{\\text{ex}} + \\beta y_{\\text{ex}} + \\frac{1}{2} z_{\\text{ex}}} \\\\\n",
    "x_{ex} &\\doteq \\overline{w^2} - 2 w_{\\text{ex}} \\overline{w} + w_{\\text{ex}}^2 \\\\\n",
    "y_{ex} &\\doteq \\overline{w^2 r} - \\overline{w r} w_{\\text{ex}} - \\overline{w} w_{\\text{ex}} r_{\\min} + w_{\\text{ex}}^2 r_{\\min} \\\\\n",
    "z_{ex} &\\doteq \\overline{w^2 r^2} - 2 \\overline{w r} w_{\\text{ex}} r_{\\min} + w_{\\text{ex}}^2 r_{\\min}^2\n",
    "\\end{aligned}\n",
    "$$ Ignoring the dependence of $x_{ex}$, $y_{ex}$, and $z_{ex}$ on $\\beta$ the maximizer is given by $$\n",
    "\\begin{aligned}\n",
    "\\beta^* &= \n",
    "\\begin{cases}\n",
    "\\frac{-y_{ex}}{x_{ex}} & x_{ex} > 0 \\land \\overline{w} = 1 \\\\\n",
    "\\frac{1}{x_{ex}} \\left( -y_{ex} + \\frac{\\left(-1 + \\overline{w}\\right)\\sqrt{y_{ex}^2 - x_{ex} z_{ex}}}{\\sqrt{\\left(-1 + \\overline{w}\\right)^2 - \\frac{2 x_{ex} \\phi}{N}}} \\right)  & x_{ex} > 0 \\land \\overline{w} \\neq 1 \\\\\n",
    "-\\frac{z_{ex}}{y_{ex}} + \\frac{\\phi}{N} \\frac{y_{ex}}{\\left(-1 + \\overline{w}\\right)^2} & x_{ex} = 0 \\land y_{ex} > 0 \\land \\overline{w} \\neq 1 \\\\\n",
    "\\text{indeterminate} & \\text{otherwise} \\\\\n",
    "\\end{cases}\n",
    "\\end{aligned}\n",
    "$$ Both $\\beta^* \\leq -r_{\\min}$ and $\\beta^* \\geq -r_{\\min}$ have to be checked using the above formulas.  \n",
    "\n",
    "Furthermore $\\beta^* = -r_{\\min}$ needs to be considered.  $\\gamma^* = 0$ in this case thus the value of $w_{\\text{ex}}$ is immaterial and $w_{\\text{ex}} = w_{\\min}$ is sufficient to consider.\n",
    "\n",
    "The largest dual objective is the winner.\n",
    "\n",
    "<!----\n",
    "F + \\[Beta] G -2 H Sqrt[1/2 X \\[Beta]^2 +Y \\[Beta] + 1/2 Z] \n",
    "D[%, \\[Beta]]\n",
    "Solve[% == 0, \\[Beta]] // Simplify // PowerExpand // Simplify // FullSimplify\n",
    "%% /. % // Simplify // PowerExpand\n",
    "((%%%% /. %%[[2]] // Simplify // PowerExpand // Simplify ) /. G^2 -> K + 2 H^2 X // Simplify) /. K -> G^2 - 2 H^2 X // Collect[#, {F, Y}]&\n",
    "% /. F -> wrbar /. G -> (-1 + wbar) /. H^2 -> \\[Phi]/n\n",
    "%%%%[[2]] /. F -> wrbar /. G -> (-1 + wbar) /. H^2 -> \\[Phi]/n\n",
    "\n",
    "F + \\[Beta] G -2 H Sqrt[1/2 X \\[Beta]^2 +Y \\[Beta] + 1/2 Z]  /. X -> 0\n",
    "D[%, \\[Beta]]\n",
    "Solve[% == 0, \\[Beta]] // Simplify // PowerExpand // Simplify // FullSimplify\n",
    "%% /. % // Simplify // PowerExpand\n",
    "(%%%% /. %%[[1]] // FullSimplify // PowerExpand) /. F -> wrbar /. G -> (-1 + wbar) /. H^2 -> \\[Phi]/n \n",
    "%%%[[1]] /. F -> wrbar /. G -> (-1 + wbar) /. H^2 -> \\[Phi]/n \n",
    "--->\n",
    "\n",
    "<!----\n",
    "(\\[Beta] (wn - wex) + wn rn - wex rmin)^2 // Expand // Collect[#, \\[Beta]]& // TextForm\n",
    "    2    2                        2   2\n",
    "rmin  wex  - 2 rmin rn wex wn + rn  wn  + \n",
    "              2                                        2\n",
    "   (2 rmin wex  - 2 rmin wex wn - 2 rn wex wn + 2 rn wn ) \\[Beta] + \n",
    "       2                2   2\n",
    "   (wex  - 2 wex wn + wn ) \\[Beta]\n",
    "\n",
    "---->"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "heading_collapsed": true,
    "hidden": true
   },
   "source": [
    "### Profile"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true
   },
   "source": [
    "One strategy is to search over the profile likelihood using the level set condition. Assume no duplicates for \n",
    "now (infinitesmal futzing, $c_{w,r} = 0$ or $1$).  The profile likelihood is $$\n",
    "\\begin{alignat}{2}\n",
    "&\\!\\min_{Q \\succeq 0} &\\qquad& \\sum_{n} \\frac{1}{2} \\left(N Q_{w_n,r_n} - 1\\right)^2,\\label{eq:mle}\\\\\n",
    "&\\text{subject to} &  & \\vec{w}^\\top Q \\vec{1} = 1, \\tag{$\\beta$} \\label{eq:mlesumw} \\\\\n",
    "&                  &  & \\vec{1}^\\top Q \\vec{1} = 1. \\tag{$\\gamma$} \\label{eq:mlesum} \\\\\n",
    "&                  &  & \\vec{w}^\\top Q \\vec{r} = v. \\tag{$\\kappa$} \\label{eq:mlesumwr}\n",
    "\\end{alignat}\n",
    "$$\n",
    "Langragian:\n",
    "$$\n",
    "\\begin{aligned}\n",
    "L(Q, \\beta, \\gamma) &= \\beta  \\left(\\vec{w}^\\top Q \\vec{1} - 1\\right) + \\gamma \\left(\\vec{1} Q \\vec{1} - 1\\right) + \\kappa \\left( \\vec{w}^\\top Q \\vec{r} - v\\right) + \\sum_{n} \\frac{1}{2} \\left(N Q_{w_n,r_n} - 1\\right)^2. \\\\\n",
    "&= -\\beta - \\gamma - \\kappa v + \\sum_{w,r} \\left( \\left( \\gamma + \\beta w + \\kappa w r \\right) Q_{w,r} + \\frac{1}{2} c_{w,r} \\left(N Q_{w,r} - 1\\right)^2 \\right). \\\\\n",
    "\\frac{\\partial}{\\partial Q_{w,r}} L(Q, \\beta, \\gamma, \\kappa) &= \\gamma + \\beta w + \\kappa w r + c_{w,r} N \\left(N Q_{w,r} - 1\\right) \\\\\n",
    "\\end{aligned}\n",
    "$$ Dual will be unbounded unless $\\forall w, r: \\gamma + \\beta w + \\kappa w r \\geq 0$.  Thus undata can only be placed at extreme points of $w r$.  Continuing $\\ldots$\n",
    "$$\n",
    "\\begin{aligned}\n",
    "Q^*_{w,r} &= \\max\\left\\{0, \\frac{1}{N} - \\frac{\\gamma + \\beta w + \\kappa w r}{N^2}\\right\\} & (c_{w,r} = 1). \\\\\n",
    "\\end{aligned}\n",
    "$$ The $\\max\\{0,\\ldots\\}$ is difficult to deal with so ignore that for the purpose of finding closed-form expressions for the (approximate) dual variables.  Continuing $\\ldots$\n",
    "$$\n",
    "\\begin{aligned}\n",
    "g (\\beta, \\gamma, \\kappa) &= \\inf_{Q \\succeq 0} L(Q, \\beta, \\gamma, \\kappa) \\\\\n",
    "&\\geq -\\beta - \\gamma - \\kappa v + \\sum_n \\left( \\frac{\\gamma + \\beta w_n + \\kappa w_n r_n}{N} - \\frac{(\\gamma + \\beta w_n + \\kappa w_n r_n)^2}{2 N^2} \\right) \\\\\n",
    "&= -\\beta - \\kappa v + \\sum_n \\left( \\frac{\\beta w_n + \\kappa w_n r_n}{N} - \\frac{(\\gamma + \\beta w_n + \\kappa w_n r_n)^2}{2 N^2} \\right) \\\\\n",
    "\\end{aligned}\n",
    "$$ We can expand the lower into sufficient statistics $$\n",
    "\\begin{aligned}\n",
    "&= -\\beta - \\kappa v + \\beta \\frac{1}{N} \\sum_n w_n + \\kappa \\frac{1}{N} \\sum_n w_n r_n -\\frac{\\gamma^2}{2 N} - \\gamma \\beta \\frac{1}{N^2} \\sum_n w_n - \\gamma \\kappa \\frac{1}{N^2} \\sum_n w_n r_n - \\beta^2 \\frac{1}{2 N^2} \\sum_n w_n^2 - \\beta \\kappa \\frac{1}{N^2} \\sum_n w_n^2 r_n - \\kappa^2 \\frac{1}{2 N^2} \\sum_n w_n^2 r_n^2 \\\\\n",
    "&\\doteq -\\beta - \\kappa v + \\beta \\overline{w} + \\kappa \\overline{w r} -\\frac{\\gamma^2}{2 N} - \\gamma \\beta \\frac{1}{N} \\overline{w} - \\gamma \\kappa \\frac{1}{N} \\overline{w r} - \\beta^2 \\frac{1}{2 N} \\overline{w^2} - \\beta \\kappa \\frac{1}{N} \\overline{w^2 r} - \\kappa^2 \\frac{1}{2 N} \\overline{w^2 r^2} \\\\\n",
    "&= \\frac{1}{2} \\left( \\begin{matrix} \\gamma \\\\ \\beta \\\\ \\kappa \\end{matrix} \\right)^\\top \\left(\n",
    "\\begin{matrix} -\\frac{1}{N} & -\\frac{1}{N} \\overline{w} & -\\frac{1}{N} \\overline{w r} \\\\ \n",
    "-\\frac{1}{N} \\overline{w} & -\\frac{1}{N} \\overline{w^2} & -\\frac{1}{N} \\overline{w^2 r} \\\\ \n",
    "-\\frac{1}{N} \\overline{w r} & -\\frac{1}{N} \\overline{w^2 r} & -\\frac{1}{N} \\overline{w^2 r^2}\n",
    "\\end{matrix} \\right) \\left( \\begin{matrix} \\gamma \\\\ \\beta \\\\ \\kappa \\end{matrix} \\right) - \\left(\\begin{matrix} 0 & 1 - \\overline{w} & v - \\overline{w r} \\end{matrix}\\right)^\\top \\left( \\begin{matrix} \\gamma \\\\ \\beta \\\\ \\kappa \\end{matrix} \\right)\n",
    "\\end{aligned}\n",
    "$$ This is a quadratic program with 4 linear inequality constraints $\\{ \\gamma + \\beta w + \\kappa w r \\geq 0 | w \\in \\{w_\\min, w_\\max \\}, r \\in \\{ r_\\min, r_\\max \\} \\}$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Comparison with baselines"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "code_folding": [
     1,
     14,
     15,
     29,
     30,
     39,
     53,
     77,
     106,
     190,
     191,
     203,
     279,
     291,
     306,
     320
    ],
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "class CITest:\n",
    "    class ClopperPearson:\n",
    "        @staticmethod\n",
    "        def interval(data, wmax, alpha, **kwargs):    \n",
    "            from scipy.special import betaincinv\n",
    "\n",
    "            k = sum((c*w/wmax)*r for c, w, r in data)\n",
    "            n = sum(c for c, w, _ in data)\n",
    "\n",
    "            lb = betaincinv(k, n - k + 1, alpha/2) if k > 0 and n - k + 1 > 0 else 0\n",
    "            ub = betaincinv(k + 1, n - k, 1-alpha/2) if n > k else 1\n",
    "\n",
    "            return min(1, lb*wmax), min(1, ub*wmax)  \n",
    "        \n",
    "    class AsymptoticGaussian:\n",
    "        def interval(data, wmax, alpha, **kwargs):\n",
    "            from math import sqrt\n",
    "            from scipy.stats import norm  \n",
    "            \n",
    "            scale = norm.ppf(1-alpha/2)\n",
    " \n",
    "            n = sum(c for c, _, _ in data)\n",
    "            tot = sum(c*w*r for c, w, r in data)\n",
    "            muhat = tot / n\n",
    "            totsq = sum(c*(w*r - muhat)**2 for c, w, r in data)\n",
    "            sigmahat = sqrt(totsq) / n\n",
    "\n",
    "            return max(0, min(1, muhat - scale*sigmahat)), max(0, min(1, muhat + scale*sigmahat))\n",
    " \n",
    "    class CressieRead:\n",
    "        def dualobjective(x, data, phi, n, lam):\n",
    "            (gamma, beta, kappa) = x\n",
    "            lampow = lam / (1 + lam)\n",
    "\n",
    "            dual = - gamma - beta + kappa * phi\n",
    "            dual += kappa / (2 * lampow) * sum((c/n) * (2 * (gamma + beta * w + w * r) / kappa) ** lampow\n",
    "                                               for c, w, r in data)\n",
    "            return -dual\n",
    "        \n",
    "        def jacdualobjective(x, data, phi, n, lam):\n",
    "            (gamma, beta, kappa) = x\n",
    "            lampow = lam / (1 + lam)\n",
    "            \n",
    "            j = [ -1, -1, phi ]\n",
    "            for c, w, r in data:\n",
    "                x = (kappa / (2 * lampow)) * (c/n) * (2 * (gamma + beta * w + w * r) / kappa)**lampow\n",
    "                dx = (c/n) * (2 * (gamma + beta * w + w * r) / kappa)**(lampow - 1)\n",
    "                j[0] += dx\n",
    "                j[1] += dx * w\n",
    "                j[2] += (1 - lampow) * x / kappa\n",
    "                \n",
    "            return -j[0], -j[1], -j[2]\n",
    "        \n",
    "        def hessdualobjective(x, data, phi, n, lam):\n",
    "            import numpy\n",
    "            \n",
    "            (gamma, beta, kappa) = x\n",
    "            lampow = lam / (1 + lam)\n",
    "            \n",
    "            h = numpy.zeros((3,3))\n",
    "            for c, w, r in data:\n",
    "                x = (kappa / (2 * lampow)) * (c/n) * (2 * (gamma + beta * w + w * r) / kappa)**lampow\n",
    "                dx = (c/n) * (2 * (gamma + beta * w + w * r) / kappa)**(lampow - 1)\n",
    "                d2x = (2/kappa) * (lampow - 1) * (c/n) * (2 * (gamma + beta * w + w * r) / kappa)**(lampow - 2)\n",
    "                h[0,0] += d2x\n",
    "                h[0,1] += w * d2x\n",
    "                h[0,2] += (1 - lampow) * dx / kappa\n",
    "                h[1,1] += w * w * d2x\n",
    "                h[1,2] += (1 - lampow) * w * dx / kappa\n",
    "                h[2,2] += lampow * (lampow - 1) * x / kappa**2\n",
    "                \n",
    "            h[1,0] = h[0,1]\n",
    "            h[2,0] = h[0,2]\n",
    "            h[2,1] = h[1,2]\n",
    "            \n",
    "            return -h     \n",
    "            \n",
    "        def interval(data, wmax, alpha, **kwargs):\n",
    "            from cvxopt import solvers, matrix\n",
    "            import MLE.MLE\n",
    "            \n",
    "            wmin = kwargs.pop('wmin', 0)\n",
    "            rmin = kwargs.pop('rmin', 0)\n",
    "            rmax = kwargs.pop('rmax', 1)\n",
    "            lam = kwargs.pop('lam')\n",
    "            \n",
    "            mle = MLE.MLE.CressieRead.estimate(datagen=lambda: data, lam=lam, wmin=wmin, wmax=wmax, rmin=rmin, rmax=rmax)\n",
    "            \n",
    "            G = matrix([ [ 0.0, 0.0, -1.0 ] ] + [ [ -1.0, -float(w), 0.0 ] for w in (wmin, wmax) ])\n",
    "            h = matrix([ 0.0 ] + [ w * rmin for w in (wmin, wmax) ])   \n",
    "            \n",
    "            def F(x=None, z=None):\n",
    "                if x is None: return 0, matrix(x0)\n",
    "                if any(x[0] + x[1] * w + w * r <= 0 for _, w, r in data):\n",
    "                    return None\n",
    "                f = CressieRead.dualobjective(x, data, phi, n, lam)\n",
    "                jf = CressieRead.jacdualobjective(x, data, phi, n, lam)\n",
    "                Df = matrix(jf).T\n",
    "                if z is None: return f, Df\n",
    "                hf = CressieRead.hessdualobjective(x, data, phi, n, lam)\n",
    "                H = z[0] * matrix(hf)\n",
    "                return f, Df, H\n",
    "            \n",
    "            pass\n",
    "\n",
    "    class Euclidean:\n",
    "        class Profile:\n",
    "            def __init__(self, n, barw, barwsq, barwr, barwsqr, barwsqrsq, wmin, wmax, rmin=0, rmax=1):\n",
    "                import numpy\n",
    "                \n",
    "                G = (1/n) * numpy.array( ( \n",
    "                                           ( 1, barw, barwr ),\n",
    "                                           ( barw, barwsq, barwsqr ),\n",
    "                                           ( barwr, barwsqr, barwsqrsq ) \n",
    "                                       ) )\n",
    "                Geigen = numpy.linalg.eigvalsh(G)\n",
    "                maxev = max(Geigen)\n",
    "                minev = min(Geigen)\n",
    "                condfac = 1e-6\n",
    "                targetev = condfac * maxev\n",
    "                if targetev > minev:\n",
    "                    G += (targetev - minev) * numpy.eye(3)\n",
    "                    \n",
    "                self.G = G\n",
    "                self.C = numpy.array([ (1.0, w, w * r) for w in (wmin, wmax) for r in (rmin, rmax) ])\n",
    "                self.b = numpy.array([ 0.0 for w in (wmin, wmax) for r in (rmin, rmax) ])\n",
    "                \n",
    "                self.n = n\n",
    "                self.barw = barw\n",
    "                self.barwr = barwr\n",
    "                self.barwsq = barwsq\n",
    "                self.barwsqr = barwsqr\n",
    "                self.wmin = wmin\n",
    "                self.wmax = wmax\n",
    "                self.rmin = rmin\n",
    "                self.rmax = rmax\n",
    "                \n",
    "            def profile(self, v):\n",
    "                import numpy\n",
    "                from quadprog import solve_qp\n",
    "                \n",
    "                a = numpy.array( (0, self.barw - 1, self.barwr - v ) )\n",
    "                sol = solve_qp(self.G, a, self.C.T, self.b)\n",
    "\n",
    "                return { 'dualobjective': -sol[1], \n",
    "                         'gammastar': sol[0][0], \n",
    "                         'betastar': sol[0][1], \n",
    "                         'kappastar': sol[0][2],\n",
    "                       }\n",
    "            \n",
    "            def solve(self, alpha):\n",
    "                from scipy.stats import f\n",
    "                import numpy\n",
    "\n",
    "                # NB: profile maximizer corresponds to \\kappa^* = 0, can be solved analytically\n",
    "                wextreme = self.wmin if self.barw > 1 else self.wmax\n",
    "                denom = self.barwsq - 2 * wextreme * self.barw + wextreme * wextreme\n",
    "                uncdual = self.n * (self.barw - 1)**2 / (2 * denom)\n",
    "                uncbetastarovern = (self.barw - 1) / denom\n",
    "                uncgammastarovern = -uncbetastarovern * wextreme\n",
    "                vmle = max(self.rmin, min(self.rmax, self.barwr - uncgammastarovern * self.barwr - uncbetastarovern * self.barwsqr))\n",
    "\n",
    "                Delta = 0.5 * f.isf(q=alpha, dfn=1, dfd=self.n-1)\n",
    "\n",
    "                def feasiblefunc(v):\n",
    "                    vprofile = self.profile(v)\n",
    "                    return vprofile['dualobjective'] - uncdual - Delta\n",
    "\n",
    "                if vmle < self.rmax and feasiblefunc(self.rmax) > 0:\n",
    "                    from scipy.optimize import root_scalar\n",
    "                    rootresult = root_scalar(feasiblefunc, bracket=(vmle, self.rmax))\n",
    "                    assert rootresult.converged, print(pformat({\n",
    "                        'rootresult': rootresult,\n",
    "                    }))\n",
    "                    vub = rootresult.root\n",
    "                else:\n",
    "                    vub = self.rmax\n",
    "\n",
    "                if self.rmin < vmle and feasiblefunc(self.rmin) > 0:\n",
    "                    from scipy.optimize import root_scalar\n",
    "                    rootresult = root_scalar(feasiblefunc, bracket=(self.rmin, vmle))\n",
    "                    assert rootresult.converged, print(pformat({\n",
    "                        'rootresult': rootresult,\n",
    "                    }))\n",
    "                    vlb = rootresult.root\n",
    "                else:\n",
    "                    vlb = self.rmin\n",
    "\n",
    "                return vlb, vub\n",
    "\n",
    "        class DirectInterval:\n",
    "            def __init__(self, n, barw, barwsq, barwr, barwsqr, barwsqrsq, wmin, wmax, rmin=0, rmax=1):\n",
    "                self.n = n\n",
    "                self.barw = barw\n",
    "                self.barwsq = barwsq\n",
    "                self.barwr = barwr\n",
    "                self.barwsqr = barwsqr\n",
    "                self.barwsqrsq = barwsqrsq\n",
    "                self.wmin = wmin\n",
    "                self.wmax = wmax\n",
    "                self.rmin = rmin\n",
    "                self.rmax = rmax\n",
    "            \n",
    "            def solve(self, alpha):\n",
    "                lb = self.solvesigned(alpha, 1)\n",
    "                ub = self.solvesigned(alpha, -1)\n",
    "                \n",
    "                return lb['v'], ub['v']\n",
    "            \n",
    "            def solvesigned(self, alpha, sign):\n",
    "                from math import sqrt\n",
    "                from scipy.stats import f\n",
    "                \n",
    "                # NB: profile maximizer corresponds to \\kappa^* = 0, can be solved analytically\n",
    "                wextreme = self.wmin if self.barw > 1 else self.wmax\n",
    "                denom = self.barwsq - 2 * wextreme * self.barw + wextreme * wextreme\n",
    "                uncdual = self.n * (self.barw - 1)**2 / (2 * denom)\n",
    "                uncbetastarovern = (self.barw - 1) / denom\n",
    "                uncgammastarovern = -uncbetastarovern * wextreme\n",
    "                uncsumofone = min(1, max(0, 1 - uncgammastarovern - uncbetastarovern * self.barw))\n",
    "                \n",
    "                Delta = 0.5 * f.isf(q=alpha, dfn=1, dfd=self.n-1)\n",
    "                phi = uncdual + Delta\n",
    "                \n",
    "                rex = self.rmin if sign > 0 else self.rmax\n",
    "                \n",
    "                xplus = self.barwsq - 2 * sign * self.wmin * self.barw + self.wmin**2 \n",
    "                xminus = self.barwsq - 2 * sign * self.wmax * self.barw + self.wmax**2\n",
    "                yplus = self.barwsqr - self.wmin * self.barwr - self.barw * self.wmin * rex + self.wmin**2 * rex\n",
    "                yminus = self.barwsqr - self.wmax * self.barwr - self.barw * self.wmax * rex + self.wmax**2 * rex\n",
    "                zplus = self.barwsqrsq - 2 * self.barwr * self.wmin * rex + self.wmin**2 * rex**2\n",
    "                zminus = self.barwsqrsq - 2 * self.barwr * self.wmax * rex + self.wmax**2 * rex**2\n",
    "                sqrtphiovern = sqrt(phi / self.n)\n",
    "                \n",
    "                def megabetafunc(beta, x, y, z):\n",
    "                    from math import sqrt\n",
    "                    \n",
    "                    sqrtterm = 0.5 * x * beta**2 + sign * y * beta + 0.5 * z\n",
    "                    sqrtterm = max(0, sqrtterm)\n",
    "                    \n",
    "                    return -( sign * self.barwr + beta * (-1 + self.barw) - 2 * sqrtphiovern * sqrt(sqrtterm) )\n",
    "                \n",
    "                def betafunc(beta):\n",
    "                    return megabetafunc(beta, xplus, yplus, zplus) if beta + sign * rex >= 0 else \\\n",
    "                           megabetafunc(beta, xminus, yminus, zminus)\n",
    "                \n",
    "                candidates = [ ( 1, -sign * rex, megabetafunc(-sign * rex, xplus, yplus, zplus) ) ]\n",
    "                for (x, y, z, s) in ( (xplus, yplus, zplus, 1), (xminus, yminus, zminus, -1) ):\n",
    "                    if x > 0:\n",
    "                        numsqrt = y**2 - x * z\n",
    "                        if self.barw == 1 or numsqrt == 0:\n",
    "                            betastar = -sign * y / x\n",
    "                            candidates.append( (s, betastar, megabetafunc(betastar, x, y, z)) )\n",
    "                        else:\n",
    "                            denomsqrt = (-1 + self.barw)**2 - 2 * x * phi / self.n\n",
    "                            if numsqrt * denomsqrt > 0:\n",
    "                                from math import sqrt\n",
    "                                sqrtfac = sqrt(numsqrt / denomsqrt)\n",
    "                                betastar = (-sign * y + (-1 + self.barw) * sqrtfac) / x\n",
    "                                candidates.append( (s, betastar, megabetafunc(betastar, x, y, z)) )\n",
    "                    elif y > 0:\n",
    "                        if self.barw != 1:\n",
    "                            betastar = -z / (2 * sign * y) + (phi / self.n) * (sign * y / (-1 + self.barw)**2)\n",
    "                            candidates.append( (s, betastar, megabetafunc(betastar, x, y, z)) )\n",
    "                 \n",
    "                optimum = min( ( (s, x, fx) for s, x, fx in candidates if s * (x + sign * rex) >= 0 ),\n",
    "                               key = lambda x: x[2] )\n",
    "                plusorminus = optimum[0] > 0\n",
    "                betastar = optimum[1]\n",
    "                vstar = -sign * optimum[2]\n",
    "                \n",
    "                if False:\n",
    "                    from scipy.optimize import minimize_scalar\n",
    "                    optresult = minimize_scalar(betafunc, bracket=(-sign * rex - 0.5, -sign * rex + 0.5))\n",
    "                    from pprint import pformat\n",
    "                    assert optresult.success, pformat(optresult)\n",
    "                    betastar = optresult.x\n",
    "                    import numpy\n",
    "                    assert numpy.allclose(betastar, optimum[1], atol=1e-5), pformat(\n",
    "                    {\n",
    "                        'optresult': optresult,\n",
    "                        'optimum': optimum,\n",
    "                        'candidates': [ (s, x, fx, s * (x + sign * rex) >= 0) for s, x, fx in candidates ],\n",
    "                        'profile': CITest.Euclidean.Profile(self.n, self.barw, self.barwsq, self.barwr, \n",
    "                                                            self.barwsqr, self.barwsqrsq, self.wmin, self.wmax).solve(alpha),\n",
    "                    })\n",
    "                    vstar = -sign * optresult.fun\n",
    "                    plusorminus = betastar + sign * rex >= 0\n",
    "\n",
    "                wex = self.wmin if plusorminus else self.wmax    \n",
    "                gammastar = -(betastar + sign * rex) * wex\n",
    "                cbeta = (\n",
    "                    0.5 * betastar**2 * (xplus if plusorminus else xminus)\n",
    "                    + betastar * sign * (yplus if plusorminus else yminus)\n",
    "                    + 0.5 * (zplus if plusorminus else zminus)\n",
    "                ) / self.n\n",
    "                cbeta = max(0, cbeta)\n",
    "                kappastar = sqrt(cbeta / phi)\n",
    "                \n",
    "                return { 'v': max(self.rmin, min(self.rmax, vstar)),\n",
    "                         'betastar': betastar,\n",
    "                         'gammastar': gammastar,\n",
    "                         'kappastar': kappastar,\n",
    "                       }\n",
    "                \n",
    "        @staticmethod\n",
    "        def interval(data, wmax, alpha, **kwargs):                \n",
    "            n = sum(c for c, _, _ in data)\n",
    "            barw = sum(c * w for c, w, _ in data) / n\n",
    "            barwsq = sum(c * w**2 for c, w, _ in data) / n\n",
    "            barwr = sum(c * w * r for c, w, r in data) / n\n",
    "            barwsqr = sum(c * w**2 * r for c, w, r in data) / n\n",
    "            barwsqrsq = sum(c * w**2 * r**2 for c, w, r in data) / n\n",
    "            \n",
    "#             profile = CITest.Euclidean.Profile(n, barw, barwsq, barwr, barwsqr, barwsqrsq, 0, wmax)\n",
    "#             return profile.solve(alpha)\n",
    "            \n",
    "            direct = CITest.Euclidean.DirectInterval(n, barw, barwsq, barwr, barwsqr, barwsqrsq, 0, wmax)\n",
    "            return direct.solve(alpha)\n",
    "                                                 \n",
    "if False:\n",
    "    for mydata in [\n",
    "        [(18, 0, 0), (1, 2, 1)],\n",
    "        [(2, 1.0101010101010102, 1), (8, 0, 0)],\n",
    "        [(10, 1.0101010101010102, 1), (1, 1.0101010101010102, 0)],\n",
    "        [(1, 0, 0), (99, 1000/999, 1)],\n",
    "        [(21, 0.0, 0), (247, 1.0101010101010102, 0), (1, 1000.0, 0)],\n",
    "        [(10, 2, 0)],\n",
    "        [(6, 0, 0), (4, 2, 1)],\n",
    "        [(5428, 0, 0), (872, 2, 0), (3700, 2, 1)],\n",
    "        [(5516, 0, 0), (4181, 2, 0), (303, 2, 1)],\n",
    "        [(5514, 0, 0), (4073, 2, 0), (411, 2, 1), (2, 1000, 1)],\n",
    "        [(5532, 0, 0), (2936, 2, 0), (1528, 2, 1), (4, 1000, 1)],\n",
    "        [(5532, 0, 0), (2936, 2, 0), (1528, 2, 1), (10, 1000, 1)],\n",
    "    ]:\n",
    "        from math import inf\n",
    "        from importlib import reload\n",
    "        import MLE.MLE\n",
    "        from MLE.MLE import CrMinusTwo as CrMinusTwo\n",
    "\n",
    "        reload(MLE.MLE)\n",
    "\n",
    "        print(\"****\")\n",
    "        print(mydata)\n",
    "#         print(CITest.ClopperPearson.interval(mydata, wmax=1000, alpha=0.05))\n",
    "#         print(CITest.AsymptoticGaussian.interval(mydata, wmax=1000, alpha=0.05))\n",
    "#         print(MLE.MLE.asymptoticconfidenceinterval(datagen=lambda: mydata, wmin=0, wmax=1000, alpha=0.05))\n",
    "#         print(CITest.Euclidean.interval(data=mydata, wmin=0, wmax=1000, alpha=0.05))\n",
    "        print(CrMinusTwo.interval(datagen=lambda: mydata, wmin=0, wmax=100000, alpha=0.05))\n",
    "        print(CrMinusTwo.interval(datagen=lambda: mydata, wmin=0, wmax=inf, alpha=0.05))\n",
    "#         from timeit import timeit\n",
    "#         print(timeit(stmt=lambda: CITest.Euclidean.interval(data=mydata, wmin=0, wmax=1000, alpha=0.05), number=1000))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "code_folding": [
     8
    ],
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "((0, 1000), 99.99999410674329)\n",
      "****** Binomial CI ******\n",
      "(10, {'cov': 0.9998, 'covstd': 0.0001, 'ubcov': 1.0, 'lbcov': 0.9998, 'width': 0.9998, 'widthstd': 0.0001, 'widthlo': 1.0, 'widthhi': 1.0, 'ub': 1.0, 'lb': 0.0002})\n",
      "(19, {'cov': 0.9991, 'covstd': 0.0003, 'ubcov': 1.0, 'lbcov': 0.9991, 'width': 0.9991, 'widthstd': 0.0003, 'widthlo': 1.0, 'widthhi': 1.0, 'ub': 1.0, 'lb': 0.0009})\n",
      "(34, {'cov': 0.9993, 'covstd': 0.0003, 'ubcov': 1.0, 'lbcov': 0.9993, 'width': 0.9991, 'widthstd': 0.0003, 'widthlo': 1.0, 'widthhi': 1.0, 'ub': 1.0, 'lb': 0.0009})\n",
      "(62, {'cov': 0.9988, 'covstd': 0.0003, 'ubcov': 1.0, 'lbcov': 0.9988, 'width': 0.9986, 'widthstd': 0.0003, 'widthlo': 1.0, 'widthhi': 1.0, 'ub': 1.0, 'lb': 0.0014})\n",
      "(113, {'cov': 0.9985, 'covstd': 0.0004, 'ubcov': 1.0, 'lbcov': 0.9985, 'width': 0.9986, 'widthstd': 0.0002, 'widthlo': 1.0, 'widthhi': 1.0, 'ub': 1.0, 'lb': 0.0014})\n",
      "(207, {'cov': 0.9988, 'covstd': 0.0003, 'ubcov': 1.0, 'lbcov': 0.9988, 'width': 0.9981, 'widthstd': 0.0002, 'widthlo': 1.0, 'widthhi': 1.0, 'ub': 1.0, 'lb': 0.0019})\n",
      "(380, {'cov': 0.9996, 'covstd': 0.0002, 'ubcov': 1.0, 'lbcov': 0.9996, 'width': 0.9976, 'widthstd': 0.0002, 'widthlo': 1.0, 'widthhi': 1.0, 'ub': 1.0, 'lb': 0.0024})\n",
      "(696, {'cov': 0.9999, 'covstd': 0.0001, 'ubcov': 1.0, 'lbcov': 0.9999, 'width': 0.9957, 'widthstd': 0.0002, 'widthlo': 0.9962, 'widthhi': 1.0, 'ub': 1.0, 'lb': 0.0043})\n",
      "(1275, {'cov': 0.9996, 'covstd': 0.0002, 'ubcov': 1.0, 'lbcov': 0.9996, 'width': 0.9855, 'widthstd': 0.0004, 'widthlo': 0.9583, 'widthhi': 1.0, 'ub': 1.0, 'lb': 0.0145})\n",
      "(2336, {'cov': 0.9995, 'covstd': 0.0002, 'ubcov': 1.0, 'lbcov': 0.9995, 'width': 0.9548, 'widthstd': 0.0006, 'widthlo': 0.8538, 'widthhi': 1.0, 'ub': 1.0, 'lb': 0.0452})\n",
      "(4282, {'cov': 0.9992, 'covstd': 0.0003, 'ubcov': 1.0, 'lbcov': 0.9992, 'width': 0.8947, 'widthstd': 0.0009, 'widthlo': 0.7258, 'widthhi': 0.9998, 'ub': 0.9916, 'lb': 0.0969})\n",
      "(7848, {'cov': 0.9996, 'covstd': 0.0002, 'ubcov': 1.0, 'lbcov': 0.9996, 'width': 0.7574, 'widthstd': 0.0013, 'widthlo': 0.5051, 'widthhi': 0.9371, 'ub': 0.9168, 'lb': 0.1594})\n",
      "(14385, {'cov': 0.9991, 'covstd': 0.0003, 'ubcov': 1.0, 'lbcov': 0.9991, 'width': 0.6028, 'widthstd': 0.0014, 'widthlo': 0.3205, 'widthhi': 0.7891, 'ub': 0.8293, 'lb': 0.2265})\n",
      "(26367, {'cov': 0.9992, 'covstd': 0.0003, 'ubcov': 1.0, 'lbcov': 0.9992, 'width': 0.4618, 'widthstd': 0.0013, 'widthlo': 0.2144, 'widthhi': 0.6284, 'ub': 0.7454, 'lb': 0.2836})\n",
      "(48330, {'cov': 0.9995, 'covstd': 0.0002, 'ubcov': 1.0, 'lbcov': 0.9995, 'width': 0.3531, 'widthstd': 0.001, 'widthlo': 0.149, 'widthhi': 0.489, 'ub': 0.6862, 'lb': 0.3331})\n",
      "(88587, {'cov': 0.9994, 'covstd': 0.0002, 'ubcov': 1.0, 'lbcov': 0.9994, 'width': 0.264, 'widthstd': 0.0008, 'widthlo': 0.0989, 'widthhi': 0.373, 'ub': 0.6344, 'lb': 0.3704})\n",
      "(162378, {'cov': 0.9985, 'covstd': 0.0004, 'ubcov': 0.9998, 'lbcov': 0.9987, 'width': 0.1978, 'widthstd': 0.0006, 'widthlo': 0.074, 'widthhi': 0.2822, 'ub': 0.5976, 'lb': 0.3998})\n",
      "(297636, {'cov': 0.9993, 'covstd': 0.0003, 'ubcov': 0.9998, 'lbcov': 0.9995, 'width': 0.1487, 'widthstd': 0.0005, 'widthlo': 0.0544, 'widthhi': 0.2121, 'ub': 0.5808, 'lb': 0.4321})\n",
      "(545560, {'cov': 0.9988, 'covstd': 0.0003, 'ubcov': 0.9995, 'lbcov': 0.9993, 'width': 0.1098, 'widthstd': 0.0004, 'widthlo': 0.0384, 'widthhi': 0.1583, 'ub': 0.5538, 'lb': 0.444})\n",
      "(1000000, {'cov': 0.9989, 'covstd': 0.0003, 'ubcov': 0.9997, 'lbcov': 0.9992, 'width': 0.0817, 'widthstd': 0.0003, 'widthlo': 0.0295, 'widthhi': 0.118, 'ub': 0.5412, 'lb': 0.4595})\n",
      "****** Asymptotic Gaussian ******\n",
      "(10, {'cov': 0.7265, 'covstd': 0.0045, 'ubcov': 0.7391, 'lbcov': 0.9874, 'width': 0.5845, 'widthstd': 0.0036, 'widthlo': 0.0, 'widthhi': 0.9681, 'ub': 0.6818, 'lb': 0.0973})\n",
      "(19, {'cov': 0.7789, 'covstd': 0.0042, 'ubcov': 0.7887, 'lbcov': 0.9902, 'width': 0.5085, 'widthstd': 0.0027, 'widthlo': 0.0, 'widthhi': 0.792, 'ub': 0.6629, 'lb': 0.1543})\n",
      "(34, {'cov': 0.8129, 'covstd': 0.0039, 'ubcov': 0.8201, 'lbcov': 0.9928, 'width': 0.4256, 'widthstd': 0.0019, 'widthlo': 0.0, 'widthhi': 0.629, 'ub': 0.6424, 'lb': 0.2168})\n",
      "(62, {'cov': 0.8145, 'covstd': 0.0039, 'ubcov': 0.8201, 'lbcov': 0.9944, 'width': 0.3353, 'widthstd': 0.0014, 'widthlo': 0.0, 'widthhi': 0.4764, 'ub': 0.6049, 'lb': 0.2696})\n",
      "(113, {'cov': 0.7936, 'covstd': 0.004, 'ubcov': 0.7987, 'lbcov': 0.9949, 'width': 0.2614, 'widthstd': 0.0011, 'widthlo': 0.0522, 'widthhi': 0.3596, 'ub': 0.5755, 'lb': 0.3141})\n",
      "(207, {'cov': 0.7441, 'covstd': 0.0044, 'ubcov': 0.7475, 'lbcov': 0.9966, 'width': 0.2056, 'widthstd': 0.0011, 'widthlo': 0.046, 'widthhi': 0.2681, 'ub': 0.5533, 'lb': 0.3477})\n",
      "(380, {'cov': 0.6612, 'covstd': 0.0047, 'ubcov': 0.6648, 'lbcov': 0.9964, 'width': 0.1618, 'widthstd': 0.0012, 'widthlo': 0.0336, 'widthhi': 0.1993, 'ub': 0.5308, 'lb': 0.369})\n",
      "(696, {'cov': 0.5542, 'covstd': 0.005, 'ubcov': 0.5559, 'lbcov': 0.9983, 'width': 0.1376, 'widthstd': 0.0016, 'widthlo': 0.0275, 'widthhi': 0.1479, 'ub': 0.5193, 'lb': 0.3817})\n",
      "(1275, {'cov': 0.4518, 'covstd': 0.005, 'ubcov': 0.4539, 'lbcov': 0.9979, 'width': 0.1334, 'widthstd': 0.0021, 'widthlo': 0.0212, 'widthhi': 0.8987, 'ub': 0.5183, 'lb': 0.3849})\n",
      "(2336, {'cov': 0.3905, 'covstd': 0.0049, 'ubcov': 0.3913, 'lbcov': 0.9992, 'width': 0.1431, 'widthstd': 0.0025, 'widthlo': 0.0164, 'widthhi': 0.9411, 'ub': 0.5368, 'lb': 0.3938})\n",
      "(4282, {'cov': 0.3916, 'covstd': 0.0049, 'ubcov': 0.3933, 'lbcov': 0.9983, 'width': 0.1594, 'widthstd': 0.0026, 'widthlo': 0.0125, 'widthhi': 0.8278, 'ub': 0.5587, 'lb': 0.3994})\n",
      "(7848, {'cov': 0.4471, 'covstd': 0.005, 'ubcov': 0.4477, 'lbcov': 0.9994, 'width': 0.1587, 'widthstd': 0.0021, 'widthlo': 0.0099, 'widthhi': 0.5008, 'ub': 0.5652, 'lb': 0.4065})\n",
      "(14385, {'cov': 0.5726, 'covstd': 0.0049, 'ubcov': 0.5738, 'lbcov': 0.9988, 'width': 0.1502, 'widthstd': 0.0015, 'widthlo': 0.0086, 'widthhi': 0.3862, 'ub': 0.5708, 'lb': 0.4207})\n",
      "(26367, {'cov': 0.7157, 'covstd': 0.0045, 'ubcov': 0.7172, 'lbcov': 0.9985, 'width': 0.1282, 'widthstd': 0.0009, 'widthlo': 0.0071, 'widthhi': 0.2973, 'ub': 0.5596, 'lb': 0.4314})\n",
      "(48330, {'cov': 0.7995, 'covstd': 0.004, 'ubcov': 0.802, 'lbcov': 0.9975, 'width': 0.1042, 'widthstd': 0.0006, 'widthlo': 0.0081, 'widthhi': 0.199, 'ub': 0.5507, 'lb': 0.4464})\n",
      "(88587, {'cov': 0.8556, 'covstd': 0.0035, 'ubcov': 0.8582, 'lbcov': 0.9974, 'width': 0.0807, 'widthstd': 0.0004, 'widthlo': 0.0083, 'widthhi': 0.1403, 'ub': 0.5365, 'lb': 0.4558})\n",
      "(162378, {'cov': 0.8932, 'covstd': 0.0031, 'ubcov': 0.8984, 'lbcov': 0.9948, 'width': 0.0611, 'widthstd': 0.0003, 'widthlo': 0.0087, 'widthhi': 0.0998, 'ub': 0.5256, 'lb': 0.4645})\n",
      "(297636, {'cov': 0.9147, 'covstd': 0.0028, 'ubcov': 0.9232, 'lbcov': 0.9915, 'width': 0.0463, 'widthstd': 0.0002, 'widthlo': 0.0133, 'widthhi': 0.0722, 'ub': 0.5277, 'lb': 0.4814})\n",
      "(545560, {'cov': 0.9318, 'covstd': 0.0025, 'ubcov': 0.9436, 'lbcov': 0.9882, 'width': 0.0344, 'widthstd': 0.0001, 'widthlo': 0.0102, 'widthhi': 0.0525, 'ub': 0.515, 'lb': 0.4806})\n",
      "(1000000, {'cov': 0.9402, 'covstd': 0.0024, 'ubcov': 0.9543, 'lbcov': 0.9859, 'width': 0.0258, 'widthstd': 0.0001, 'widthlo': 0.0085, 'widthhi': 0.0383, 'ub': 0.5126, 'lb': 0.4869})\n",
      "****** CRMinus2 ******\n",
      "(10, {'cov': 0.9979, 'covstd': 0.0005, 'ubcov': 0.9989, 'lbcov': 0.999, 'width': 0.9306, 'widthstd': 0.0013, 'widthlo': 0.5725, 'widthhi': 1.0, 'ub': 0.9647, 'lb': 0.0341})\n",
      "(19, {'cov': 0.995, 'covstd': 0.0007, 'ubcov': 0.998, 'lbcov': 0.997, 'width': 0.809, 'widthstd': 0.0019, 'widthlo': 0.4507, 'widthhi': 1.0, 'ub': 0.9035, 'lb': 0.0945})\n",
      "(34, {'cov': 0.9927, 'covstd': 0.0009, 'ubcov': 0.9968, 'lbcov': 0.9959, 'width': 0.664, 'widthstd': 0.002, 'widthlo': 0.3542, 'widthhi': 1.0, 'ub': 0.831, 'lb': 0.167})\n",
      "(62, {'cov': 0.992, 'covstd': 0.0009, 'ubcov': 0.9956, 'lbcov': 0.9964, 'width': 0.5334, 'widthstd': 0.0021, 'widthlo': 0.2815, 'widthhi': 1.0, 'ub': 0.7653, 'lb': 0.2319})\n",
      "(113, {'cov': 0.9929, 'covstd': 0.0008, 'ubcov': 0.997, 'lbcov': 0.9959, 'width': 0.4238, 'widthstd': 0.0019, 'widthlo': 0.2259, 'widthhi': 1.0, 'ub': 0.7106, 'lb': 0.2868})\n",
      "(207, {'cov': 0.9934, 'covstd': 0.0008, 'ubcov': 0.9968, 'lbcov': 0.9966, 'width': 0.3397, 'widthstd': 0.0017, 'widthlo': 0.1859, 'widthhi': 0.7729, 'ub': 0.6696, 'lb': 0.3298})\n",
      "(380, {'cov': 0.9925, 'covstd': 0.0009, 'ubcov': 0.9962, 'lbcov': 0.9963, 'width': 0.2793, 'widthstd': 0.0015, 'widthlo': 0.1575, 'widthhi': 0.402, 'ub': 0.6404, 'lb': 0.3611})\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(696, {'cov': 0.996, 'covstd': 0.0006, 'ubcov': 0.998, 'lbcov': 0.998, 'width': 0.2437, 'widthstd': 0.0015, 'widthlo': 0.1485, 'widthhi': 0.4961, 'ub': 0.6209, 'lb': 0.3772})\n",
      "(1275, {'cov': 0.9966, 'covstd': 0.0006, 'ubcov': 0.9986, 'lbcov': 0.998, 'width': 0.2327, 'widthstd': 0.0017, 'widthlo': 0.1335, 'widthhi': 0.7147, 'ub': 0.6151, 'lb': 0.3824})\n",
      "(2336, {'cov': 0.9983, 'covstd': 0.0004, 'ubcov': 0.999, 'lbcov': 0.9993, 'width': 0.2287, 'widthstd': 0.0016, 'widthlo': 0.1252, 'widthhi': 0.5901, 'ub': 0.6176, 'lb': 0.3889})\n",
      "(4282, {'cov': 0.9975, 'covstd': 0.0005, 'ubcov': 0.9992, 'lbcov': 0.9983, 'width': 0.2121, 'widthstd': 0.0012, 'widthlo': 0.1198, 'widthhi': 0.3754, 'ub': 0.6064, 'lb': 0.3943})\n",
      "(7848, {'cov': 0.9988, 'covstd': 0.0003, 'ubcov': 0.9993, 'lbcov': 0.9995, 'width': 0.1902, 'widthstd': 0.0008, 'widthlo': 0.1166, 'widthhi': 0.3565, 'ub': 0.5941, 'lb': 0.404})\n",
      "(14385, {'cov': 0.9983, 'covstd': 0.0004, 'ubcov': 0.9994, 'lbcov': 0.9989, 'width': 0.1597, 'widthstd': 0.0005, 'widthlo': 0.1146, 'widthhi': 0.2504, 'ub': 0.5813, 'lb': 0.4216})\n",
      "(26367, {'cov': 0.9993, 'covstd': 0.0003, 'ubcov': 0.9994, 'lbcov': 0.9999, 'width': 0.1221, 'widthstd': 0.0003, 'widthlo': 0.0866, 'widthhi': 0.1685, 'ub': 0.5602, 'lb': 0.4381})\n",
      "(48330, {'cov': 0.993, 'covstd': 0.0008, 'ubcov': 0.9967, 'lbcov': 0.9963, 'width': 0.0883, 'widthstd': 0.0003, 'widthlo': 0.0544, 'widthhi': 0.1168, 'ub': 0.5448, 'lb': 0.4565})\n",
      "(88587, {'cov': 0.9805, 'covstd': 0.0014, 'ubcov': 0.989, 'lbcov': 0.9915, 'width': 0.0626, 'widthstd': 0.0003, 'widthlo': 0.0334, 'widthhi': 0.0808, 'ub': 0.5287, 'lb': 0.4662})\n",
      "(162378, {'cov': 0.973, 'covstd': 0.0016, 'ubcov': 0.9866, 'lbcov': 0.9864, 'width': 0.044, 'widthstd': 0.0002, 'widthlo': 0.0205, 'widthhi': 0.0567, 'ub': 0.518, 'lb': 0.474})\n",
      "(297636, {'cov': 0.9663, 'covstd': 0.0018, 'ubcov': 0.9827, 'lbcov': 0.9836, 'width': 0.0318, 'widthstd': 0.0003, 'widthlo': 0.0133, 'widthhi': 0.0403, 'ub': 0.5209, 'lb': 0.4891})\n",
      "(545560, {'cov': 0.965, 'covstd': 0.0018, 'ubcov': 0.9839, 'lbcov': 0.9811, 'width': 0.0228, 'widthstd': 0.0002, 'widthlo': 0.0092, 'widthhi': 0.0288, 'ub': 0.5095, 'lb': 0.4868})\n",
      "(1000000, {'cov': 0.9617, 'covstd': 0.0019, 'ubcov': 0.9794, 'lbcov': 0.9823, 'width': 0.0177, 'widthstd': 0.0004, 'widthlo': 0.0072, 'widthhi': 0.0208, 'ub': 0.5084, 'lb': 0.4907})\n",
      "****** Euclidean ******\n",
      "(10, {'cov': 1.0, 'covstd': 0.0, 'ubcov': 1.0, 'lbcov': 1.0, 'width': 0.9905, 'widthstd': 0.0003, 'widthlo': 0.9261, 'widthhi': 1.0, 'ub': 0.9953, 'lb': 0.0047})\n",
      "(19, {'cov': 1.0, 'covstd': 0.0, 'ubcov': 1.0, 'lbcov': 1.0, 'width': 0.8515, 'widthstd': 0.0012, 'widthlo': 0.6176, 'widthhi': 1.0, 'ub': 0.9241, 'lb': 0.0726})\n",
      "(34, {'cov': 0.9998, 'covstd': 0.0001, 'ubcov': 0.9998, 'lbcov': 1.0, 'width': 0.684, 'widthstd': 0.0012, 'widthlo': 0.4617, 'widthhi': 0.8784, 'ub': 0.8417, 'lb': 0.1577})\n",
      "(62, {'cov': 0.9995, 'covstd': 0.0002, 'ubcov': 0.9997, 'lbcov': 0.9998, 'width': 0.5356, 'widthstd': 0.001, 'widthlo': 0.3605, 'widthhi': 0.7063, 'ub': 0.7653, 'lb': 0.2297})\n",
      "(113, {'cov': 0.9994, 'covstd': 0.0002, 'ubcov': 0.9998, 'lbcov': 0.9996, 'width': 0.4256, 'widthstd': 0.0009, 'widthlo': 0.2866, 'widthhi': 0.559, 'ub': 0.712, 'lb': 0.2864})\n",
      "(207, {'cov': 0.9992, 'covstd': 0.0003, 'ubcov': 0.9995, 'lbcov': 0.9997, 'width': 0.3452, 'widthstd': 0.0009, 'widthlo': 0.233, 'widthhi': 0.4517, 'ub': 0.6734, 'lb': 0.3283})\n",
      "(380, {'cov': 0.9982, 'covstd': 0.0004, 'ubcov': 0.9992, 'lbcov': 0.999, 'width': 0.2888, 'widthstd': 0.001, 'widthlo': 0.1928, 'widthhi': 0.3732, 'ub': 0.6439, 'lb': 0.3551})\n",
      "(696, {'cov': 0.9985, 'covstd': 0.0004, 'ubcov': 0.9993, 'lbcov': 0.9992, 'width': 0.2551, 'widthstd': 0.0012, 'widthlo': 0.1698, 'widthhi': 0.3668, 'ub': 0.6268, 'lb': 0.3717})\n",
      "(1275, {'cov': 0.9988, 'covstd': 0.0003, 'ubcov': 0.9996, 'lbcov': 0.9992, 'width': 0.2452, 'widthstd': 0.0017, 'widthlo': 0.1498, 'widthhi': 0.7086, 'ub': 0.6214, 'lb': 0.3762})\n",
      "(2336, {'cov': 0.9994, 'covstd': 0.0002, 'ubcov': 0.9996, 'lbcov': 0.9998, 'width': 0.2389, 'widthstd': 0.0016, 'widthlo': 0.1374, 'widthhi': 0.6025, 'ub': 0.6227, 'lb': 0.3837})\n",
      "(4282, {'cov': 0.999, 'covstd': 0.0003, 'ubcov': 0.9997, 'lbcov': 0.9993, 'width': 0.2203, 'widthstd': 0.0011, 'widthlo': 0.1284, 'widthhi': 0.3948, 'ub': 0.6106, 'lb': 0.3903})\n",
      "(7848, {'cov': 0.9992, 'covstd': 0.0003, 'ubcov': 0.9996, 'lbcov': 0.9996, 'width': 0.1993, 'widthstd': 0.0007, 'widthlo': 0.1229, 'widthhi': 0.3458, 'ub': 0.5989, 'lb': 0.3997})\n",
      "(14385, {'cov': 0.9991, 'covstd': 0.0003, 'ubcov': 0.9996, 'lbcov': 0.9995, 'width': 0.173, 'widthstd': 0.0004, 'widthlo': 0.1196, 'widthhi': 0.2538, 'ub': 0.5881, 'lb': 0.415})\n",
      "(26367, {'cov': 0.9997, 'covstd': 0.0002, 'ubcov': 0.9997, 'lbcov': 1.0, 'width': 0.1437, 'widthstd': 0.0002, 'widthlo': 0.1101, 'widthhi': 0.1766, 'ub': 0.571, 'lb': 0.4273})\n",
      "(48330, {'cov': 0.9999, 'covstd': 0.0001, 'ubcov': 1.0, 'lbcov': 0.9999, 'width': 0.1145, 'widthstd': 0.0002, 'widthlo': 0.0809, 'widthhi': 0.1371, 'ub': 0.5578, 'lb': 0.4433})\n",
      "(88587, {'cov': 0.9996, 'covstd': 0.0002, 'ubcov': 0.9997, 'lbcov': 0.9999, 'width': 0.088, 'widthstd': 0.0002, 'widthlo': 0.0608, 'widthhi': 0.1126, 'ub': 0.5415, 'lb': 0.4534})\n",
      "(162378, {'cov': 0.9991, 'covstd': 0.0003, 'ubcov': 0.9995, 'lbcov': 0.9996, 'width': 0.0667, 'widthstd': 0.0001, 'widthlo': 0.0457, 'widthhi': 0.09, 'ub': 0.5295, 'lb': 0.4628})\n",
      "(297636, {'cov': 0.9982, 'covstd': 0.0004, 'ubcov': 0.9989, 'lbcov': 0.9993, 'width': 0.0498, 'widthstd': 0.0001, 'widthlo': 0.033, 'widthhi': 0.07, 'ub': 0.5298, 'lb': 0.4801})\n",
      "(545560, {'cov': 0.9972, 'covstd': 0.0005, 'ubcov': 0.9989, 'lbcov': 0.9983, 'width': 0.0368, 'widthstd': 0.0001, 'widthlo': 0.0243, 'widthhi': 0.0529, 'ub': 0.5164, 'lb': 0.4796})\n",
      "(1000000, {'cov': 0.9961, 'covstd': 0.0006, 'ubcov': 0.9979, 'lbcov': 0.9982, 'width': 0.0275, 'widthstd': 0.0001, 'widthlo': 0.0179, 'widthhi': 0.0403, 'ub': 0.5136, 'lb': 0.4861})\n",
      "****** MLE CI ******\n",
      "(10, {'cov': 0.9964, 'covstd': 0.0006, 'ubcov': 0.9977, 'lbcov': 0.9987, 'width': 0.7722, 'widthstd': 0.0015, 'widthlo': 0.502, 'widthhi': 0.9495, 'ub': 0.8866, 'lb': 0.1144})\n",
      "(19, {'cov': 0.9939, 'covstd': 0.0008, 'ubcov': 0.9976, 'lbcov': 0.9963, 'width': 0.6353, 'widthstd': 0.0015, 'widthlo': 0.3722, 'widthhi': 0.8672, 'ub': 0.8149, 'lb': 0.1796})\n",
      "(34, {'cov': 0.9926, 'covstd': 0.0009, 'ubcov': 0.9967, 'lbcov': 0.9959, 'width': 0.5237, 'widthstd': 0.0013, 'widthlo': 0.2961, 'widthhi': 0.734, 'ub': 0.7614, 'lb': 0.2377})\n",
      "(62, {'cov': 0.9914, 'covstd': 0.0009, 'ubcov': 0.9955, 'lbcov': 0.9959, 'width': 0.425, 'widthstd': 0.0011, 'widthlo': 0.2492, 'widthhi': 0.6047, 'ub': 0.7097, 'lb': 0.2848})\n",
      "(113, {'cov': 0.9936, 'covstd': 0.0008, 'ubcov': 0.9974, 'lbcov': 0.9962, 'width': 0.3478, 'widthstd': 0.0009, 'widthlo': 0.2056, 'widthhi': 0.4871, 'ub': 0.6728, 'lb': 0.325})\n",
      "(207, {'cov': 0.9935, 'covstd': 0.0008, 'ubcov': 0.9967, 'lbcov': 0.9968, 'width': 0.286, 'widthstd': 0.0007, 'widthlo': 0.1754, 'widthhi': 0.3964, 'ub': 0.6437, 'lb': 0.3577})\n",
      "(380, {'cov': 0.9919, 'covstd': 0.0009, 'ubcov': 0.9964, 'lbcov': 0.9955, 'width': 0.2371, 'widthstd': 0.0005, 'widthlo': 0.1504, 'widthhi': 0.3227, 'ub': 0.6178, 'lb': 0.3807})\n",
      "(696, {'cov': 0.9946, 'covstd': 0.0007, 'ubcov': 0.9969, 'lbcov': 0.9977, 'width': 0.201, 'widthstd': 0.0004, 'widthlo': 0.1341, 'widthhi': 0.2648, 'ub': 0.6001, 'lb': 0.3991})\n",
      "(1275, {'cov': 0.9926, 'covstd': 0.0009, 'ubcov': 0.997, 'lbcov': 0.9956, 'width': 0.1721, 'widthstd': 0.0003, 'widthlo': 0.1191, 'widthhi': 0.2232, 'ub': 0.5859, 'lb': 0.4138})\n",
      "(2336, {'cov': 0.9931, 'covstd': 0.0008, 'ubcov': 0.9968, 'lbcov': 0.9963, 'width': 0.1485, 'widthstd': 0.0003, 'widthlo': 0.1029, 'widthhi': 0.1892, 'ub': 0.5775, 'lb': 0.429})\n",
      "(4282, {'cov': 0.9891, 'covstd': 0.001, 'ubcov': 0.9957, 'lbcov': 0.9934, 'width': 0.1292, 'widthstd': 0.0002, 'widthlo': 0.089, 'widthhi': 0.1637, 'ub': 0.5659, 'lb': 0.4366})\n",
      "(7848, {'cov': 0.9806, 'covstd': 0.0014, 'ubcov': 0.9902, 'lbcov': 0.9904, 'width': 0.1112, 'widthstd': 0.0002, 'widthlo': 0.0735, 'widthhi': 0.1438, 'ub': 0.5539, 'lb': 0.4427})\n",
      "(14385, {'cov': 0.9703, 'covstd': 0.0017, 'ubcov': 0.9849, 'lbcov': 0.9854, 'width': 0.0938, 'widthstd': 0.0002, 'widthlo': 0.058, 'widthhi': 0.1286, 'ub': 0.5485, 'lb': 0.4547})\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(26367, {'cov': 0.9635, 'covstd': 0.0019, 'ubcov': 0.9818, 'lbcov': 0.9817, 'width': 0.0767, 'widthstd': 0.0002, 'widthlo': 0.0426, 'widthhi': 0.1123, 'ub': 0.5377, 'lb': 0.461})\n",
      "(48330, {'cov': 0.9492, 'covstd': 0.0022, 'ubcov': 0.9764, 'lbcov': 0.9728, 'width': 0.0604, 'widthstd': 0.0002, 'widthlo': 0.0312, 'widthhi': 0.09, 'ub': 0.5309, 'lb': 0.4705})\n",
      "(88587, {'cov': 0.948, 'covstd': 0.0022, 'ubcov': 0.9706, 'lbcov': 0.9774, 'width': 0.0468, 'widthstd': 0.0002, 'widthlo': 0.0209, 'widthhi': 0.0699, 'ub': 0.5207, 'lb': 0.474})\n",
      "(162378, {'cov': 0.9453, 'covstd': 0.0023, 'ubcov': 0.9722, 'lbcov': 0.9731, 'width': 0.036, 'widthstd': 0.0001, 'widthlo': 0.0137, 'widthhi': 0.0521, 'ub': 0.514, 'lb': 0.478})\n",
      "(297636, {'cov': 0.9462, 'covstd': 0.0023, 'ubcov': 0.9713, 'lbcov': 0.9749, 'width': 0.0272, 'widthstd': 0.0001, 'widthlo': 0.0096, 'widthhi': 0.038, 'ub': 0.5186, 'lb': 0.4914})\n",
      "(545560, {'cov': 0.9486, 'covstd': 0.0022, 'ubcov': 0.9752, 'lbcov': 0.9734, 'width': 0.0201, 'widthstd': 0.0001, 'widthlo': 0.0072, 'widthhi': 0.0274, 'ub': 0.508, 'lb': 0.4879})\n",
      "(1000000, {'cov': 0.9472, 'covstd': 0.0022, 'ubcov': 0.9728, 'lbcov': 0.9744, 'width': 0.0151, 'widthstd': 0.0, 'widthlo': 0.0061, 'widthhi': 0.02, 'ub': 0.5075, 'lb': 0.4924})\n"
     ]
    }
   ],
   "source": [
    "from importlib import reload\n",
    "import environments.ControlledRangeVariance\n",
    "import MLE.MLE\n",
    "from MLE.MLE import CrMinusTwo as CrMinusTwo\n",
    "\n",
    "reload(environments.ControlledRangeVariance)\n",
    "reload(MLE.MLE)\n",
    "\n",
    "def produceresults(env, method, minexp=1, maxexp=6, numpts=15, ndataperpt=100):\n",
    "    from math import ceil\n",
    "    import numpy\n",
    "    \n",
    "    wmin, wmax = env.range()\n",
    "\n",
    "    for ndata in map(ceil, numpy.logspace(minexp, maxexp, numpts)):\n",
    "        bounds=[]\n",
    "        pc=0\n",
    "        for i in range(1, ndataperpt+1):   \n",
    "            (truevalue, data) = env.sample(ndata)\n",
    "\n",
    "            try:\n",
    "                ci = method(data=data, wmin=wmin, wmax=wmax, alpha=0.05)\n",
    "                assert numpy.isfinite(ci[0]) and numpy.isfinite(ci[1])\n",
    "                assert ci[1] >= ci[0]\n",
    "                assert ci[1] <= 1 + 1e-4\n",
    "                assert ci[0] >= -1e-4\n",
    "#                 assert ci[1] >= truevalue\n",
    "#                 assert ci[0] <= truevalue\n",
    "            except:\n",
    "                print('truevalue was {}'.format(truevalue))\n",
    "                print('data was {}'.format(data))\n",
    "                print('wmin, wmax was {} {}'.format(wmin, wmax))\n",
    "                print('ci was {}'.format(ci))\n",
    "                raise\n",
    "                                                \n",
    "            bounds.append(\n",
    "                ( truevalue,\n",
    "                  data,\n",
    "                  [ \n",
    "                      ci[0], 'n/a', ci[1]\n",
    "                  ]\n",
    "                )  \n",
    "            )\n",
    "            \n",
    "        uppers = [ d[2][2] for d in bounds ]\n",
    "        lowers = [ d[2][0] for d in bounds ]\n",
    "        upperbounded = [ 1 if d[0] <= d[2][2] else 0 for d in bounds ]\n",
    "        lowerbounded = [ 1 if d[2][0] <= d[0] else 0 for d in bounds ]\n",
    "        covered = [ 1 if u*l > 0 else 0 for (u, l) in zip(upperbounded, lowerbounded) ]\n",
    "        width = [ d[2][2] - d[2][0] for d in bounds ]    \n",
    "        yield (ndata,\n",
    "                { \n",
    "                    'cov': numpy.mean(covered),\n",
    "                    'covstd': numpy.std(covered, ddof=1) / numpy.sqrt(len(covered) - 1),\n",
    "                    'ubcov': numpy.mean(upperbounded),\n",
    "                    'lbcov': numpy.mean(lowerbounded),\n",
    "                    'width': numpy.mean(width),\n",
    "                    'widthstd': numpy.std(width, ddof=1) / numpy.sqrt(len(width) - 1),\n",
    "                    'widthlo': numpy.quantile(width, q=[0.05])[0],\n",
    "                    'widthhi': numpy.quantile(width, q=[0.95])[0],\n",
    "                    'ub': numpy.mean(uppers),\n",
    "                    'lb': numpy.mean(lowers),\n",
    "                },\n",
    "              )\n",
    "\n",
    "def getenv():\n",
    "    wsupport = [ 0, 2, 1000 ]\n",
    "    env = environments.ControlledRangeVariance.ControlledRangeVariance(seed=45, wsupport=wsupport, expwsq=100)\n",
    "    return env, env.getpw(), env.range(), env.expectedwsq()\n",
    "    \n",
    "print(getenv()[2:])\n",
    "wmax = getenv()[2][1]\n",
    "\n",
    "allres = []\n",
    "for (name, method) in [ \n",
    "                        ('Binomial CI', lambda data, wmax, alpha, **kwargs: CITest.ClopperPearson.interval(data, wmax, alpha)),\n",
    "                        ('Asymptotic Gaussian', lambda data, wmax, alpha, **kwargs: CITest.AsymptoticGaussian.interval(data, wmax, alpha)),\n",
    "                        ('CRMinus2', lambda data, **kwargs: CrMinusTwo.interval(datagen=lambda: data, **kwargs)[0]),\n",
    "                        ('Euclidean', lambda data, wmax, alpha, **kwargs: CITest.Euclidean.interval(data, wmax, alpha)),\n",
    "                        ('MLE CI', lambda data, **kwargs: MLE.MLE.asymptoticconfidenceinterval(datagen=lambda: data, **kwargs)[0]),\n",
    "\n",
    "                      ]:\n",
    "    print('****** {} ******'.format(name))\n",
    "    res = []\n",
    "    for zzz in produceresults(getenv()[0], method, minexp=1, ndataperpt=10000, numpts=20):\n",
    "        wmax = getenv()[2][1]\n",
    "        res.append( (zzz[0] / wmax, zzz[1]) )\n",
    "        print('{}'.format(( zzz[0], { y: round(z, 4) for y, z in zzz[1].items() })), flush=True)\n",
    "        del wmax\n",
    "    allres.append((name, res))\n",
    "import pickle\n",
    "pickle.dump( allres, open( \"epsilongreedy_ci_allres.euclidean.p\", \"wb\" ) )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "code_folding": [
     2
    ]
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZcAAAEdCAYAAADATSqzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOydd5hU5dXAf2fK9r70uhRBQEQRFCwIqKioWDBYQYzdaDQiRk2CLYmixvrZsETEEqzBXhBsCCoYC4JiYZG+vU2fe9/vj9nFLdN32u7e3/PMA3vv+945s+Wee7oopTAwMDAwMIglpmQLYGBgYGDQ+TCUi4GBgYFBzDGUi4GBgYFBzDGUi4GBgYFBzDGUi4GBgYFBzDGUi4GBgYFBzLEkW4Bk061bN1VSUpJsMQwMDAw6FOvWratQSnUPdL7LK5eSkhLWrl2bbDEMDFKSuro6ysrK8Hg8yRbFIIWwWq1cdNFF1cHWdHnlYmBg4J+6ujp2795N3759yczMRESSLZJBCqCUwuFwcN111w348ssvjx47duw7/tYZMRcDAwO/lJWV0bdvX7KysgzFYrAHESErK4sePXroFovlhkDrUkK5iEg/EblfRFaLiF1ElIiUhLnXJCLXiUipiDhF5GsRmRlfiQ0MOj8ej4fMzMxki2GQooiIUkr1CnQ+JZQLMBSYBVQDH0e49xbgRuD/gGOBNcALIjI9lgIaGHRFDIvFIBCNvxsBdUiqxFw+Ukr1BBCR84Fp4WwSkR7A1cBtSqk7Gw+vFJGhwG3Am/EQ1sDAwMAgOClhuSil9Ci3Hg2kAU+3Ov40MFpEBrVLMAMDAwODqEgJ5dIORgEu4KdWx79r/HdkYsUxMDDoCKxevZrTTz+dfv36kZaWRl5eHuPHj+dvf/sbO3fuTLZ4Abnxxhs7jKsyVdxi0VIE1Ki2Q2mqmp2PH3q0BlcnwdTRn00MuiL/+te/mD9/PlOmTOHvf/87gwcPpqGhgU8//ZRFixaxdu1a3nrrrWSL6Zfzzz+fY445JtlihEVHVy5RISIXAhcCDBgwILqL6BpUbY6hVB2Q/H5gzUi2FAYGYbNy5Urmz5/PFVdcwd13393i3PTp07nuuut44YUXkiRdaPr160e/fv2SLUZYdPRHz2qgQNraiU0WSxV+UEotUkqNU0qN6949YPcCg1C46pMtgYFBRCxcuJBu3bqxcOFCv+ezs7OZO3funq9vuOEGxo4dS15eHt26dWPq1KmsWbOmxZ4nn3wSEaG0tLTFcX8urHvvvZcRI0aQmZlJYWEh48aN45VXXtlz/p133uHggw8mPz+fnJwchg8fzs033xz0mv/3f//HxIkTKSoqoqCggAkTJvDGG2+0WFNaWoqI8Mgjj7BgwQJ69+5NQUEBJ5xwAtu2bQv5fYuGjm65fAekA0NoGXdpirVsSLhEXQl3Pahu0EF8wAZdG6/Xy4cffsgpp5xCWlpaWHu2b9/On/70J/r164fNZuPpp59m0qRJrFu3jtGjR0f0/s888wzz5s1jwYIFHHbYYTgcDr755huqqnzPwL/88gszZszg1FNPZcGCBaSlpfHjjz/yyy+/BL1uaWkp559/PiUlJXi9Xl577TWOP/543nrrrTYutFtvvZWDDz6YJ554grKyMubNm8fZZ5/NBx98ENFnCYeOrlzeBjzAWcBNzY6fDaxXSnVxv1Wc0XXw2CEtO9mSGCSIm177jg076pIqw8g+edxwwqiI91VWVuJ0Ov26wr1eb4uvLRbfrfGxxx7bc0zTNI455hhGjRrFY489xr333hvR+69evZp9992XBQsW7Dk2ffpv5Xhffvklbrebhx56iLy8PACmTp0a8rp33nnnnv/rus4RRxzBpk2beOihh9ool5KSEp599tk9X5eXlzN//nx27NhBnz59Ivo8oUgZt5iInCoipwIHNB46tvHY4c3WeEXk8aavlVJlwF3AdSJylYhMFpGHgKnAdYmUv8tiuMYMOji7du3CarW2eDUpm+XLlzNlyhSKi4uxWCxYrVY2bdrEDz/8EPH7jB8/nq+++orLL7+c5cuXY7fbW5zfb7/9sFqtnH766bz44ouUlZWFdd1169Zx/PHH07Nnzz0yvvfee35lbK7MgD3W16+//hrx5wlFKlkuraNoDzb++yEwufH/5sZXc/4CNABXAL2AH4BZSqnX4yOmQQvcNp8FY2SOdQmisRhSheLiYjIyMtrcSLt168YXX3wBwKJFi3j00UcBnyUxffp0jj76aB5//HF69+6N2Wzm/PPPx+l0Rvz+c+bMwel08vjjj/Pggw9itVqZPn06d911FyUlJQwdOpR33nmHhQsXMnv2bFwuFwceeCALFy7k8MMP93vNrVu3csQRRzBy5Ejuv/9+BgwYgMVi4W9/+xsbN25ss76oqGUCbXp6OkBUnycUKaNclFIhHff+1iilNODvja+EoesKpRTmrh5vUArcDZCRl2xJDAyCYrFYmDRpEu+99x5ut3tP3MVisTBu3DgAXn/9t2fSl156CYvFwssvv4zVat1zvLq6moKCgj1fZ2T4MibdbneL96usrGzxtYhw0UUXcdFFF1FdXc27777LvHnzOO200/jss88AmDJlClOmTMHlcrFq1SoWLFjAcccdR2lpKd26dWvzmd5++21qa2t5/vnnW2SRtbaKkoHxuBklCqh3ekOu6xK4G5ItgYFBWFxzzTVUVFTw5z//OeRau92O2WxukZ21YsWKNpbPwIEDAVi/fv2eY16vl3fffTfgtQsLCznttNOYNWtWi31NpKenM3XqVK655hpsNhubN/sPHzcpkebKb9OmTaxatSrk54s3KWO5dEQanB5yMyyG9eK2++p+TK09lgYGqcURRxzBbbfdxrXXXss333zDnDlzGDRoEE6nk02bNvGf//yH7OxsRIRjjjmGe+65h7lz53LuueeyadMmbrnlFvr27dvimuPHj2fIkCHMnz8fXddJT0/nwQcfxOVytVh34YUXkpuby8SJE+nRowebNm1iyZIlTJvma6X48MMP89FHHzF9+nT69+9PRUUFt956K3369GGfffbx+3mOPPJILBYLc+bMYd68eezcuZMbbriBAQMGoCe5yNuwXNqBQtFgWC8+jMC+QQfhmmuu4eOPP6a4uJjrr7+eI488klNPPZXFixdz2mmn8eOPP2I2mzn66KO57777WLVqFccffzxPPPEETz31FEOHDm1xPYvFwrJly+jfvz9z587lD3/4A0cddVSLehmAQw45hHXr1nHppZdy1FFH8Y9//IOzzz6bxYsXAzBmzBhsNhvXXXcd06ZN47LLLmPQoEGsWLEi4OiDUaNG8cwzz7BlyxZmzJjB7bffzm233cakSZPi8r2LBGnbOaVrMW7cOBXNmGO328WPP31BQVoevfMzMXV168WSDgX9ky2FQQzZuHEjI0aMSLYYBinK+vXr7ZqmlY8ZM6bE33nDcmkHdd4Gqt21NLgM6wWvCzRjznq4eDQdr9bFe9MZdGoM5RItjmoszlrqvTa21lWg6NoWIGC4xsLAq+lUNLjYVu2gvMEVeoOBQQfFUC7RoHmwLj6WoeseBKWo89rYUluRbKmSj6FcAqLpiiqbm63VDuocHpRSONwatXbD2jPonBjKJRrMVrRxF1C4+yt6/fI2AGX2esqclXTpGJbmAU/si7E6MrquqLG72Vplp8bubvP7UWV34/JqSZLOwCB+GMolSvQDfk91z/0Z+O0zZNZtRVM65fYGylxVXVvBGNYLAEopau0etlbbqbK50QP8TiilKKtzde3fGYNOiaFcokWEnw+4GM2ayV5f3I9oHmwuDbvXye6ubMG4u7ZyUUpR5/SwtcpBpc2Fpof+PfBoOhUN7pDrDAw6EoZyaQeejAJ+Hnsx2bVbGLBhKZrScXi8ODUXu52V6KoLZgPpuq/fWBekweVlW7WDinoX3ggL2OqdHmxG1qFBJ8JQLu2kuvcB7Bp0JL1/fJ28svXYXBoK9igYTXVBf7qra7WDsbu9bKu2U1bnxNOO9OKKBpeRnmzQaTCUSwzYMno2zpxeDF37AMpZh8PjewJ1aW52O7qggnE3+BpadnKcHo0dNQ521Tpxe9uvFDRdGenJBp0GQ7nEAN2SwY/jL8fqqmXw/x7D1qwljFv3sMtRiVfvQgqmqVNyJ2dnrROnJ7Y/V4dbo8ZuxF8MOj6GcokRtsIhbBvxO7ptX01B6Qc4mt10PLqHXc4KPHoX8ql38qwxrXHkQjyotntirrQMWrJ69WpmzZpFnz59SEtLo7i4mKOOOorFixejaRoffPABIrLnZbFYGDBgAJdeeinV1dUtrtU0n15EWLRoUZv3stls5ObmIiL89a9/3XP8ySefREQoLS2N98cNyKZNm7jiiivYd999ycnJoXfv3syYMYOvv/663dc2lEsM2T78ROqK92bQ1//GXbW9xTmv7m1UMF2kaM7j8HVK7qREGrCPBKUU5fUu9DAyzQwi55577uGQQw6hqqqKhQsXsnz5cp544gmGDRvGJZdc0mKmy3333cfq1at59913mT17NosWLWLOnDl+r5ubm8uSJUvaHH/ppZdatO1v4rjjjmP16tX07t07dh8uQt59911WrlzJOeecw2uvvcaDDz5IeXk5EyZMYN26de26ttFyP5aIiR/HX8aY5fMZ9Pn97Jj2DzLS0/ac1nSNXc5KemUUYzVZg1yoE6CUz3rJLAi9tgMSTopxe/BoOpU2N91z0+P6Pl2Njz76iKuuuorLLruM++67r8W5E088kauuugqbzUZVVRUAI0aMYMKECYBvnn1ZWRmPPfYYu3btolevXi32n3LKKTz11FNs3ryZQYMG7Tn+1FNPMXPmTJ588skW67t370737t3j8CnD5/TTT+cPf/hDC+U3depUSkpKuPfee3nqqaeivrZhucQYd1Z3Nu93HnmVP5D7zYttzmu6xi5HBe6u0OSxE7vGvAmwKoz05NizcOFCioqKuP322/2eHzJkCPvuu2/A/WPHjgX8z5w/9NBDGTRoEE8//fSeY9u2bWPlypV+rR1/brGSkhLOPvts/vOf/zBixAiys7MZN24cn3zySYu9kydPZvLkyW2uWVJS0qLV/65duzjnnHPo06cP6enp9O7dm+OPP56ysjLAN+K5tVWVn5/PsGHD2L69pfclUgzlEgcqBhxGRb9D6LPhedjVdo61pnR2OStwaZ08cNuJOyV7tcS4rIz05NihaRorV65k2rRpe0YTR0ppaSlms5mSkhK/52fPnt1CuTz99NP069fPryIIxMcff8y//vUvbrnlFpYuXYqmaRx//PHU1NRELO/s2bNZvXo1d9xxB++99x733Xcf/fr1CzoGuaqqivXr17d73ILhFosTv+x3HrmV39Prk7vZfeK9KGvLYT+60tntrKRPZg8snXmCo6sesoqSLUXMiWfMpTlN6cm98/0Pi0o4b10Lu75Nrgy9RsOxt0W8raKiAofDsWcscTjouo7X68XhcPD+++/z0EMPceWVV9KjRw+/6+fMmcNNN93EmjVrmDBhAkuWLOHss8/2G3MJRF1dHV999RWFhYUA9OrVi/Hjx/Pmm29y5plnhn0d8CUu/POf/+Sss87ac+x3v/td0D2XX345SimuvPLKiN6rNYZyiRNaWjY/jbuMkR/fTP5nj1Fz6OVt1uhKp9pdS/eMznfz3UMnVS7xjrk0pyk9uSArLfRig5hy9NFHt/j6uOOO44477gi4fvDgwRxyyCEsWbIEi8XChg0bePnllyN6z4kTJ+5RLACjR48G/LviQjF+/HjuuOMOlFJMnTqVffbZJ6iiu/XWW3n22Wd5/PHH20zcjBRDucSRuu4j2TFsBn03LcPdfzz2gRParLF5HeRqLjLMnTRw29Qp2RqdGyJVSZRbrIlqu4cMq5kMa5Kt3CgshlShuLiYzMxMtmzZEvaeBx54gAMPPJDa2loeffRRli5dyi233MKCBQsC7pkzZw7XX389mqZx4IEHMnz48IjkLCpq+TCWnu67NzidkXccX7p0KTfddBO33347V155Jb179+biiy/mr3/9KyZTy6jIww8/zPXXX8/f//53fv/730f8Xq0xYi5xZuvIWTTkl1D86f2Y7VV+11S5ahMsVYLphAWVibRcwEhPjgUWi4XJkyfz3nvv4XKF1wlh2LBhjBs3jiOOOILnnnuOyZMnc+utt7J169aAe2bNmoXNZuPRRx8NmLbcXjIyMnC728Zsm7LcmujRowcPPPAA27dv5/vvv2fu3LnccMMNPPLIIy3WLVmyhEsvvZR58+bxl7/8JSYyGsolziiThZ/GX454XXT75F7w08zSrXuo93TiZo+dLGtM11XAFvrxxKPpVNiM9jDt4dprr6WyspJrrrnG7/nNmzfzzTff+D0nItx99924XC5uuy2wBVdQUMB1113HjBkzOP3002Mid2sGDhzIpk2bWiiYjz76iPr6wH9rw4cP55///CeFhYWsX79+z/FXXnmFc889l/PPP58777wzZjIabrEE4MjrR+k+ZzP46yfI3fgG9SNPaLOmxl1HtiUTk3RCfa9r4LZDWlayJYkJiUhDDkSD00tWmpecdONPNxomTZrEXXfdxVVXXcWGDRuYO3cuAwYMoLq6mvfff5/HHnuMZ599lvz8fL/799tvP2bOnMnjjz/OX/7yF/r06eN3XTC3WSw4/fTTWbRoEb///e+ZO3cumzdv5q677mohd21tLUceeSRnnXUWe++9N1arlWXLllFdXc20adMAn0I644wzGDNmDHPnzmXNmjV79qenp7P//vtHLWMnvJOlJrsHT6O291iK1j6JtbptYE5TOtXuuiRIliA6kfWSaJdYayrqjfTk9nDllVfyySefUFBQwNVXX83UqVOZO3cuGzdu5JFHHuGEE9o+/DXn5ptvxuPxsHDhwgRJ3JYpU6bw8MMP89lnn3HCCSfw73//m6effpqCgt+KljMyMhg7diyPPvoop556KieffDKrV6/mmWee4cQTTwRgxYoVuFwuvvzySw455BAmTpy453XyySe3S0bpskOtGhk3bpxau3ZtxPvcbhfrvl8Z0R6rs4b93r8GPauQHcf/C8wtq/RFhD6Z3Ttn9b4IFA32/dvBqXd6KK9Prnsqw2qmT0F805M3btzY7loHg87L+vXr7ZqmlY8ZM6bE3/mUsFxEpL+IvCgitSJSJyIvi8iAMPcOEJHFIvKriDhEZJOI/F1EsuMtd6R4MgrYcuAlpFWXUvhl2x5ESikqO2twvxN1Sk625QK+dv9Gc0uDVCbpykVEsoAVwN7AOcBsYC9gZSgF0Xh+OTAJ+BswHXgMmAc8EUexo6as+/7UDDuG/O/+S8aOtp1HnZoLu9eRBMkSQCdxjXkSnIYcCEO5GKQyqRAVvAAYDAxXSv0EICLfAD8CFwF3Bdl7CD5FdLRS6t3GYytFpAi4WkSylFKB+xwkiS2jZ5O961u6fXIPO068Dz09t8X5KncdmeaMiKp6OwRNnZI7eEeCVLBcABwejc7ZFtSgM5B0ywWYAaxpUiwASqnNwCrgxBB7m0qWW0fCa/B9tpS8O9uVhZ2HXYXZUUPR54+3Oe/VvdR6OsdTfgs6iWssUa1fQuH06HGbKWNg0F5SQbmMAtb7Of4dMDLE3uX4LJyFIjJSRHJEZCpwBfCwUipli0eqcgZSN2oG2T+vxFq1uc35Wk9D55xe2QlcY6liuSilcMVgvLKBQTxIBeVSBFT7OV4FFPo5vgellBM4FN/n+A6oB94HXgcuC7RPRC4UkbUisra8vDxauduFw6NRuc9M9LSsgMH9ancnDO57nB26U7JSKmWUCxhxF4PUJRWUS9SISAawFOiBLxHgcGA+cBrwQKB9SqlFSqlxSqlxyRzWU68yqR19Klnb1pK+q63xZvM6cGqdsCK7A1svySyg9IfDUC4GKUoqKJdq/FsogSya5pwHTAamK6WeVkp9pJS6E1+22MUiMiamksYYu0ejZu/j8GYVUbhusS8m0YpO2XesAyuXVLJaAFxG3MUgRUkF5fIdvrhLa0YCG0LsHQ1UK6V+bnX888Z/U7wCTFGvmanZ70wyyn8g69fP2qxw6x7qOlvfMc3jGyTWAUk1y0U34i4GKUoqKJdXgQkiMrjpgIiU4EszfjXE3l1AoYi0HjxwUOO/7ZvTGQLdTxPKSLG7NeqGHIE7vx8FXz7lS9VtRa27Dk11MvdHB7VetBSpcWmOEXcxSEVSQbk8CpQCy0TkRBGZASwDtgJ7+kKLyEAR8YpI845wT+IL4r8pIueIyBQRmQ/cCazDl84cc7y6l+tWXcdru96LwdUUDV5FzdizSavdRs5P77dZoSmdGnfHvBkHpIMql1RJQ26O05N6MqUqTXPrRYRNmza1Of/hhx/uOb98+XIAbrzxRkQEr9cb8Lpz587ds6/166STTorb50llkl5EqZSyNaYP3w0swVeb8j5wpVKqeVGEAGaaKUSlVKmITABuBP4OdMOnlBYB/1AqBqaFHywmC2ax8PbudxlfsB99Mnu263p2l5eG/hPJ7zaMgq+ewzb4cJSl5fCweo+NXEs2aeZO0ndM18DrBkvHmq6YajEX8FkuSqnOV3QbR3Jzc1myZAm33HJLi+OLFy8mNzc3aOv6QHTv3p1XX23rbGk9/KurkAqWC0qpX5VSM5VSeUqpXKXUSUqp0lZrSpVSopS6sdXxDUqpWUqp/kqpTKXUMKXU1UqpUMkA7eKqsX8i3ZTG01tfard7TKGwezWqxs3FYq8kd+PrftdVdbbU5A7Y5saTgsrFiLtEzimnnMLTTz/dIhnC4XDw4osvMnPmzKiumZaWxoQJE9q8hg0bFiuxOxQpoVw6IkUZRZza93h+sm1mVeUX7b6ezeXF2Wsf7P3Gkf/ti5hcbSvZnZoLWwe8IQekAwb1UzHmAr6sMYPwmT17Nlu2bOGTTz7Zc+yVV15B1/WolYtBSwzl0g4OKRrPXjmDeWnHG9S1s12LrhR2t5fqsbMxue3kf/ui33VV7trOk3rqjXwmeLLRUvR7b9S7RMbAgQOZNGkSS5b8VsD81FNPcfLJJ5OTkxP1db1eb5tXp/l7jZCkx1w6MiLC2f1P4ebv7+b57a9xfsmZ7bqezaWRVTQI25DJ5G58nboRx6Nld2uxRtM1aj31FKTlteu9UgKvG3QdTB3jGcerpW5NSaIyxhZ+vpDvq75PyHsFYu+ivfnzgX9u93XmzJnDvHnzuO+++6iurmb58uW89dZbUV9v+/btWK1tY6J33HEHV199dXtE7ZB0jL/qFKZ3Rk+O7TmFz6v/x3d1P7TrWprScXi81Ox3JqJ0Cr76j991narvWAeyXlKtxqU5ulJGSnKE/O53v8PlcvHaa6/xzDPP0KtXL4444oior9ejRw+++OKLNq/Zs2fHUOqOg2G5xIBje07li+qveWbry9wwYh7ppugzoGwujazcntQPP5bc79+gbtSJeAr6t1jT1Hese0YnyELxOiEtK9lShEUqZoo1x+XRybDGd5xBLCyGVCE3N5eTTjqJJUuWUFpayllnnYWpHVa01Wpl3LhxMZSwY2NYLjHAarJyVv9TqHBX8eautnUqkeDVdRwejZoxs1CWdAr8NLUEX98xR2foO2ZYLjHDiLtEzpw5c3jjjTf49ttvmTNnTrLF6VQYyiVG7J07lIOLxvHO7g/Y5tjZrmvZXF70jHxq9zmZ7F/XkF7m38dd5eoEwf0OpFxS3XIx3GKRc9RRRzFr1iwuvvhiRo3y14XKIFoMt1gMmdn3eL6u3cDTW1/imr0uxSTR6W6PruP0asjIE8nb+CaF6xaz65h/QqsiOY/u6fjBfV339RrrAMWhqVid3xxfvYtGuqVjT/pMJGazmeeeey6stS+//HIbt1nv3r055JBDAHC73axZs6bNvqysLPbdd9/2C9vBMJRLDMm1ZDOr7wn8+9elfFz5GYd3mxj1tWwujYzsTGr2O53iNQ+TuX0djn5t/bm1ngayLZlYTal/cw6Ix9EhlEuqWy4ATrduKJc4cdppp7U5dtxxx/H6676i5/LyciZObPs3P2rUKNav9zcPsXMjHd6t0k7GjRun1q5dG/E+t9vFuu9XtjmulOLunxaxxbGNm0bMp8AavVVRlJVOukmn7yuXoizp7DjhHr/z5zPM6fTK7ObnCh2EjHzISd5cnXDZWmXHozVaL0qB7gZzevBNCSY73ULPvIyYXGvjxo2MGJHijcUNksb69evtmqaVjxkzpsTf+Yj8NiJiFpEzROQxEXlTRFYEeLUvqt2BERHO6n8KHt3L89tCNXUOToPbCyYL1WNnk1a9hezNH/ld59RcNHjs7XqvpNJB4i7NA/risWGxlfudwZNMjLiLQaoQtltMRAqBd4Gx+JpIBiO1/uISTM+M7hzX6wiW7XyHibUHMDo/uqc/t1fDrelQcgiu9S9T+OUz2EoO9etCqnbXkmlJxywd0CWiuX036RRuvKjpqkXyhMldj+guTK5q9BRKCdd0I+5ikBpEYrn8AzgA2Ab8GTgRmBLgNTW2YnY8ju4xmd4ZPXh22yu4NHfU17G5vSAmqg84B4utjLzv3/S7TlM61R21Lb9SKW+9tAjm615MXp+laHLW+JRjCuF0p3bigUHXIJKA/gx8Y4cPUkrtipM8nQaLycLZ/Wdyx48P8dqudzm17/FRXcfp0fCk69BnPxy9x5D/zfPU73UkKi27zdoGj40cSyYZKRYHCAuvE6yZyZYiIM2D+Sb3b01FBYXZXo6W2zcZYvnF6dXIJ/UTJAw6N5FYLt2ATwzFEj575Qzm0OIDWV72MVvt0Q/FtLl8Q4qqDzgHs6ue/PWvBFzbYWtfPKluubR0iTXHpDl9FkyKYMRdDFKBSJTLDiDwKDYDv8zscxzZliyWtGPui8Oj4VUKd7eh2EoOI2/DMsx2/+Nq3LqHOk/bdv0pT4q7xZpa7YvXieht3WAmZ5WvXicFaIq7xIIO+aBikBAafzcC3tQiUS4vAZNEJHV9FylItiWLWX1nUGrfygcVq6O+js3ZaL2MPQvRvOR/szTg2hpPPR69gz0H6BpoqStzk+Uibv+KW1CYHeWJFCkosRh9bLVacTg60apGQVwAACAASURBVPwgg5iilBIRCejJikS53ITPelkqIj3aLVkX4sDC/RiZO4z/7niL6iinSdo9GppSePP6UD/8aHJ/eAdL3Q6/a5VSHXNqZQoPQtN0BUphCmIVmrwOxFWXQKkCEwvXWI8ePdi+fTt2u92wYAz2oJTCbrdTVlZm8nq9NwVaFzCgLyJP+Dn8M3AS8KOIrAN+xb9ZpJRS50UqdGelqfblxo138p9t/+WSwedEcRWFzeUlL8NK7b6nkfPT+xR++TTlk6/xu9rhdWL3OsiydCBD0+uC9NxkS+EXr64jHjuigt+0zY5KvNYsMCW3+UUslEtenq8AeMeOHXg8qeHyM0gNrFYrt95666/vv//+O4HWBPsLmBvkXC4wOch5BRjKpRnd04s5vtdRvLLzLb6q/Y798iNvkmd3a+SkWyGrkLpRJ1Hw9VLqdh+Hq6f/a1W6a8kwp0fd4yzhpHDcxaspTGFMGxV0zI4KtOxeCZAqMJqucHt10izt+9nn5eXtUTIGBs1ZsWJFUDM9mHI5N8aydHmO6nk4n1X/j+e2vsLeOUPIMEfWpkOhsLm95KZbqB19Kjk/raR49UPsOOFuv4WVmq5R7a6jOL0gVh8hvnhdKVlMqZRC17xYwuyCYPLY0N0NqLTox+XGAodHa7dyMTCIloDKRSm1OJGCdAUsYmb2gJncvulBlu18l9P6zYj4Gna3l+w0CyZLOpUTLqLn+7eQt+FV6kbP9Lu+3mMjx5JFujn6AWYJQymfgrHGpjdWrPBovliLRNB4wuyowGvJ9NsLLlG4PBpkGvUuBskh7McaERkgIiH7XIhIoYgMaJ9YnZch2SVM6jaBFeWfsMW+LeL9ulLYPb6sKkf/8dgGTKDgq+ew1O8OuKfSlTo1GCFJQdeYpquAWWKBEKVhdlTGSaLwMIaHGSSTSGzmzcAdYay7HfglOnG6Bif3PpZcSw5Lfn0RLUSA2B92l7bnGbrqwAtATBR9tihgE0W37qG2o9S+pKBy8XqcmLTI5TJ56pEkNhRtirsYGCSDSJSLELphZfO1BgHIsmRyer8T+dWxnRXln0S8X1M6drfPetFyulOz3xlkbfuCrF8/C7in1l2PV+8AT7IpqFx0R/TpxWZ7OURZPBsLnDEqpjQwiJR4RPsKgE4w3D2+HFCwL2PyR/LKjrfZ4Qjs0gqE3f2b9VI38gTchQMp+nwR4vFfK6IrvWPUvmheX0FlCqE5o1cuorxJdY853an1vTToOgRVLo1xlgHNYig5zY+1eg0WkeOAafhcaGEjIv1F5EURqRWROhF5OZK4jYiMEJEXRKRCRBwi8oOIXBGJDIlGRJjd/1Qyzek8vuVZvBFW1Ht1HUdj7AWThcqJl2KxVVDw1X8C7rF7HdhT0DJoQyrJ6Lajt7NzgMldhySpQDQWlfoGBtEQynIpxacompTFzGZft379CLyKrwbmmXAFEJEsYAWwN3AOMBvYC1gpIm1b/7bdPw74DEgHzgemA/8CUn6gRZ41l9n9T2WrYwev7Xov4v0Nzt+sF1ePEdTvNY28DcuwVpcG3FPl7gCNLVOpiaWrPibjjX3uscR/3726bsRdDJJCqDLiX/lt8NcAwA5UBFjrBrYDrwD/F4EMFwCDgeFKqZ8AROQbfMrqIuCuQBtFxAQ8BbyvlDq52am284dTlP0K9uHQ4gN5e/dK9snbm71yBoW9tyn2kp3m+zFWHzCHrF/XULz6QXYdexv4KZ706l6q3XUUpefH7DPEnFSxXHQd3A17mla2B9E9mJxV6JnFMRAsMpxeo97FIPEE/Y1TSpUopQYppQbhC9K/0PS1n9dwpdRUpdT9KrJH4xnAmibF0vi+m4FV+AaSBWMyMIIgCqgjMKvvCRSnFfLEludwRJiV1ODy0vRgrWfkUT3uXDLKvifnx+UB99R7bbhTpIOvX1JFubjrUUpHi1FA3uyqQZLw2Yy4i0EyiORx5lzg8TjIMApY7+f4d8DIEHsPbfw3Q0TWiIhHRMpE5L6O1L05w5zBeQPPoMpdw9Jtr0a0V1e+qv0mGoZOxdlzFIXrFmMKEIhWSlHpTuHal6ZiymQTI5dYc8z2ioS7x4y4i0EyCFu5KKUWK6VWxUGGInwTLltTBRSG2Nun8d+lwLvAUfjqbM4Hno2VgIlgSE4Jx/acyqdVX/C/mm8j2mtrZr0gQuWESzC57RSu/XfAPS7NTZ3H1g6J40yyrRfNAx5nzJWL6C5MLv+zeOKFV9fxaIaCMUgsHd0R2yT/00qpBUqpD5RSd+IbD3CSiIzwt0lELhSRtSKytrw8dWZwHN/rSAZk9mXJ1peo9YSf/qpQNLh+c3N5CgdQu89J5P70Pum7/BmFPqrdtXj0FHWPJdtycfmaVHpjrFwA39RKre3AsXhiVOsbJJqAykVEtHa8IsndrMa/hRLIomlOUwFB61Srdxv/3d/fJqXUIqXUOKXUuO7du4ctaLyxmCycV3IGLs3F4l9fiCiry+b2zXtponbM6Xize1C85qGAExKVUlSkamuYAPU6CaNxLoseB+UiKF/2WAIxRh8bJJpglou04xWJRfQdvrhLa0YCG8LYG4y4+AKUUtz8xve880PsL987oycz+x7H+rrv+ahyTSRS0eD8TaerxsaWaTVbydsQOI7j0tzUukO3kk84mseXrZUMPI49UzHjYbkAmDRnQlvDON2GW8wgsQRUAkopU+sXcDe+dOS78FkFhY2v/fHVltiAuxrXhsurwAQRGdx0QERKgEMazwXjLXzdAI5udfyYxn/XRiBH2Hh1xc5aJ0+uUzz7Px09xgHayd0OZmTuMF7Y/hq7neE/4do9Gt5msoTb2LLGU5+a2WPJiru4flO2sf7ZNsfkTtzUSiPuYpBoIumKfB7wR+BYpdTVSqmvlVK1ja+vlVLzgWOBK0TkgghkeBRfseYyETlRRGYAy4CtwCPN3n+giHhFZEHTMaVUJXArcLGI/FNEjhSRa4EFwOLm6c2xxGo2cf9p+3LkUOG1jYoHP1V4YlAL0YRJTJwzYBYWsfD4lufwht3cUtHgbKkkqg4K3djS5x6rTr3iymQoF6VaKJd4WS6Az3JJYKsbwzVmkEgisTAuBT5WSn0caIFS6hPgY+CScC+qlLIBU4FNwBJ81f2bgalKqeatfAVf1X1rmW8GrgFmAW82vvcd+Ioz44bFYuH3B1o4fYywaovitg907O7Y3YgK0/I5u/9MSu1beWvX+2Hvc3g0PM3cSVp2eI0t3bqHmjAmLSaUZCgXd0MLJRyPmEsTgm9OTKIwgvoGiSQS5TIc2BnGup3AsEiEUEr9qpSaqZTKU0rlKqVOUkqVtlpTqpQSpdSNrY4rpdRdSqmhSqk0pdTAxsyx+Pp5RNAzijhxlIlLJwo/lMGNy3Uq7bG7GY0rHMNBhWN5Y9f7/GL7Nex99c6W+RThNLYEX+dkV4KzmIKSDOXiaqlg42m5AJhciVPoLqPexSCBRKJcXATIvmrF/nSRrsjKmoWyZHPYIBN/nmKiogEWvKuztSZ2N6Qz+p9EgTWPJ7Y8F/aN3+XVcDf3r4fZ2BKgwlWTOu4xXQdvApWd5gX3b0F2X/ZdfL8XortAS8yfi0cz4i4GiSMS5fIRMFxEbhFpO+RcfNyMrwHlR7ESMNXRMgvBZGF0L2HBUSZ0BTe+p/Pd7tjclLLMmZw78HTKXZW8uOO1sPe1tl7CbWzp0T2p1Zo/kdZLq6w5LUHZaqYIp1y2ByPuYpAoIlEufwOcwPXADyJys4ic2/i6Cfge+AvgwBdQ7yKY0DJ8059LCoVbppkozITbVup8Whqbm9Pw3CEc1WMSH1as4dvajWHtcWtam0FR1QfMQU/LoXj1g0EHWNV7bDgS9DQdkkQWU7Zql+ONYZJGMEzu+oS1hDHiLgaJIpL2L+vxtbPfCQzFp0gea3z9FV+b/F3A8UqpyPqXdHCUOQM9zddluFu2cOM0E0OL4f5PFa9v1GPiZjqx9zH0zejF4l+fpz7MIHBDK+sl3MaWABWuavQkTlDcQ6LmoHicbYpNtQTd8EVpCat5MeIuBokiovYvSqkP8SmWOcAT+Crh3wX+jW8Wy1ClVIdpdx9L9PQ8lCkdgJw04bqpJg4aIDzzP8VT61S7s46sjdX7ds3Bkq0vhqWwPLre5km1ZWPLwO4vTdeocqWAe0zzJOap3k9gPZHhCVOCMvU8mo7XiLsYJICIe4sppZxKqaeVUhcopY5tfJ2vlFqilEpyz45kImiZRfgypiHNLPzxEGH6cOHtTYp7V+m4ve27SfbL7MNJvY/hq9rv+LQqvPrQBlerTjwiVE70NbYs+iJwY0uABq8de5ImKO5BqfjHXZRqE2+BxMVcILE1L4ZrzCARdPTGlamFyYqW8VubNJMIsw8wMXus8MVW+OdKnQZX+xTMkT0mMSxnCP/ZtowKV1XI9V5dx+5pqWA8BQOoHT2TnJ9XkFX6adD9la4atLCLOONEvPuMuW1+W80kyi0GjTUvCWrDY7TgN0gEhnKJMcqagzK3HCUzfW8TfzzUxM+VcMN7OuUN0d+0TGLi3IGnIUjY1fvNxyE3UTPmNFzdhlH86f2YG8oC7tWUTmWy3WPxDuoHqDWJxQTKSEhU1piRMWaQCIJ1Rf5FRH4WkUHNvg739XPiPkLqoWUWgZhbHJswQLh+qolah68WZnNV9Deu4rRCzu5/Cj/bSnlxe+j05KZxyC0wWyk//GpEKbp/eCfogRtZ270ObMl0j8XTLaZrECCYHutZLqEQ3ZWQSZVG3MUgEQSzXEoaX9ZWX4f76rqIeU96cnNG9BBuOMqE2QQ3L9f5Zmf0N68Di/bnqO6TWFG+ik8qPw+5vvk45Ca8ub2oOPgPZJR/T8FXzwXdX+WqwZvAPlgt0LWAYwPajct/GrCuFCrOBZT+kERZL15DuRjEl2DKZRAwGPil2dfhvga3vlhXQ1ky0a25bY73LxBunmaiZw7c/oHOqnbUwpzSdzojcvfima0v83NDadC1rcchN2EfdBj1ex1F/jcvkrHj64D7NaUndzRyvJ7oA7jE4t32JRAmT0NCsuMcbsM1ZhBfgrXc39L48rb6OqxX4j5C6qJnFIDJ2uZ4UZavmn94D3hgteKLrdHdTMxi5sKSsymyFvDQ5sVUh7j52/xYLwBVB16AJ78f3T6+C5Mj8DUcXif1yRqNHI+4i9cd8LqJzBRrjq/mJf7fYyPuYhBvjIB+lJjEhElCffuk0T3WplsOWVZh/iQTQ4vgvlU63+6KTsFkW7L4w+C5uHQPD25ejDvI2OLW45D3HLdmUD55PmZXA90+uTdo9X61uw5PkPhM3IhHxliQppGJzBRrTSKyxow+YwbxJpJ5Lv8VkT8Emkvf1RAR8tP9TWduiTKno6fn+T2XYRWumWKidx7860OdTeXR3dD6ZPbivJIz2GLfxpJfgxdYth6H3ISnsISqA88ja/s68r5bFnC/rnQqkzEaWXPH1l2k63tGGft9uyQ+2Ju89qAJFrGioiFFWvwYdEoisVxmAPcB60Vku4g8JSLniEi/OMmW8mRZssmwZIZcp6fl76neb01OmnD9FF8/sts/0NlSHd0NdL/8UZzY+2g+q/6S98qC9Q1VbdrCNFE//FhsAyZQ+OUS0ip+DHgFp+aiNoFzSIDGYsoY3QyVgvqdQYsWE50p1ppEWC8Ot0atPQUnkBp0CiJRLsfhG3P8DdALOBtfC5gtIvK9iDwgIqeISEEc5ExZCtKLwnCPgZZZTKBvd0GmcP0RJtKtvkLLnXXR3dim9zyCsQX78tKON/iu7oeA6+weDY+/m6cIlYdcjpZZQPcP70Dcgftd1bjrEj/7JVbp0LbykG62ZLrFIHE1L1V2Ny6vEX8xiD2RNK58q3G88f5AD+A0fE0rN+MbDnYJ8AJQLiKhc2M7CWYxk58W2j2GydKier813bN9FgwK/rlCp9IW+c1NRJg7YBZ9M3qxqPRpdjvLA6xU2PzEXgD09FzKJ12NpaGM4jUPBR2NXOasxBMkxhNzYmG52KvadD/2R7IC+k2I7k5IzYtSirI6V+rM8DHoNEQV0FdKVSqlXlBKXaSUGoqvruVOfEPCzMABsRMx9cmyZpNhyQi5TlmzUZasgOf75gvXTjFh88A/VujUOiP/g88wp3Pp4LmYMPHAL0/i0PzfoByeVgPFmuHqOZKa/c4g55cPyflpRcD30pTObmdV4trDtPdm66r3KZcwSLZbDBLjGgNfcL/SlkITSA06BVFni4lITxE5S0T+DXwCzAMyAB34IkbydRgK0oowtZ2h1gZ/1fvNGVQkXHO4iUo73LpSx+aO/CbXLb2IiwfNocxVweOlzwZsnV/r8AQsE6wdfSqOXqMp+uxhrDXbAr6XV/dS5qxKTHt+zRt9c0ePE4K0uWmOQqGnwJO8uBNT8wJQ5/C07eJgYNAOIskWyxaR40TkbhH5FtgBLMHXat8GPAScAhQrpSbERdoUxmyykJ/Wtiq/LSa0jOKgK/buIVx1mIlttXDHhzrOKLopD88dwmn9ZvBN3UaW7XzH7xqvrlPvDODWMpmpmHQVypzui78EGTfs0tyUu6oT41qJJiVZ80D9jrBv1MkqoGyNoCek5qWJ8npXSlhsBp2DSCyXKuBV4I9AEfAMMBfor5QaoZS6TCn1X6VUCgwBSQ5hu8csGehp/tOTmxjTR7j8YBObKuDuj3Q8UTRRnNztYA4tPpC3dq/gi+qv/K6xub24ArQC0bKKqTj0CtKqN1O47smg7+XwOhOTohxp3EXXoW6H367Hgbekzg3W5A4dH4oVmq4orzfSkw1iQyTKpanU/FvgBmCBUuoppdT22IvVcQnXPaan54MpLeiagwYIFx4ofLML/u9TPeKnShHhjH4nMyR7IE9ueZ6tdv8/qlqHx2/lPoCj/3hqR55I3sbXyfx1TdD3a/DaqY73zTCSuEtTynGEfck8SQ7mN8fkdSSk5qUJu9tLXSBr1sAgAiJRLvfgUyyjgUeAn0XkJxF5RER+JyLBfT1dBLPJQl442WMI3iDpyU1MHmJizljh863w6OeRxwKsJgsXD5pDjiWLB3550u+IZE3p1DkDu72qD5iDq3gI3Vbdj9kWKAPNR627nrp4unIiUS4NZVG50VJItwCJC+w3Udngxm00tjRoJ5GkIl+llNoP6AmcCTzeeOoCYCmwW0S+FJE7ROTo2Ivacci25pBuDu0e8w0XC62Tj93bxMzRwoe/KJZ8qSKObeRb87hk8DnUext4uHQJXj9Pwg6PFrjflNlK+eHzEc1L94/+FTKoXuWqid8Ey3CLKe1VQdu7BCNVYi5NJFq5KKUoq3ca6ckG7SKaMccVSqmlSqkLG9OQBwHn41MwewNXAW/EVsyOh6+4MrR7TFkz0dPyQ66buY9w7HDh7R8UL30b+R99SVZ/5gz4HT82/MLS7a/6XVPrDOwe8+b1oXLixWTs3kDB10tDvl+5qxqnFif/fSjrJYKUY3+kQqZYc0T3IAmep+P26lQb1fsG7aBdjStFpDdwGDCp8ZWOr0tj6LtqJ8dispCbFl6zAj09v830ytaICGePFQ4fLLy0XvHm95G7LQ4qGsu0HofzYcVqPqxY3VYOpagN4h6zDZlCw5Ap5H/zPOm71gd9L6UU5c6q+BRZeoIoF48j7JTjQKRixlSiKvabU2N3G92TDaImIuUiInkiMkNE7hOR74BtwGJgDtAX2ICv/9hJMZe0A5JjzQ3PPYYvM8tfe/7mmMQX4D+oPyz5UrHy58gVzCl9pjMqdzjPbf0vPzb80ua806Nh9wQOIFdOuBhvbi+6f/QvTCEq3TWls8tZGfshY4EsF83jC+C30/JI9HjjcPDVvCQ+DlJW50qp7DmDjkMkdS6rgQrgFeAyYAQ+5fIkvj5jvZVSo5VSVyqlQs/ebXnt/iLyoojUikidiLwsIgMiuUbjda4VESUin0S6N16E6x4DE97MboT6kZhMwh8ONrFvb1+Af+22yP7wTWLigpKz6JZexIO/LGaLvW2BZJ3DG7C3lrJmUn74fMzOWl/8JUQmlqZrlDkrY1tkqXnaxn10LeKU44CXT8JNPBSJrnlpwqvrRvdkg6iIxHI5CKgDXsbXR2yYUmqgUuo8pdSzSqnd0QggIlnACnzxmnOA2cBewEoRyY7gOoOBvwLt84nEmEjcY5isjQ0ug2M1C386zMTgQrh/lc6PFZEpmCxLJn8cch7ppjTu+ukRfraVtjivUNQ4AisNd/EQKideQuaO/9H9wztCBvjduocyZ1VsA8TNrZcoU479XjbVUsWakejAfhMNLm/gYlsDgwBEolwOALorpX6nlHpEKfVTjGS4AN9Y5JMaizCX4WvvPxC4KILrPISvsHNjjOSKGTnWXNLM/lvut0ZZMn01MCHIsAjzJ5soyoTbP4y8k3KP9G7MH3YpuZYc7vnpUX6o/7nFebdX8zsWuYmGvY6i8sDzyf51Dd0+vjukgnFqLipc1RHJGJTmGWMNZcHjMBGQivGWJkxeR0wUaDRUNriN4WIGERFJKvL/VHxyE2cAa5orK6XUZmAVcGI4FxCRM4GxwHVxkC8mFKYXI2G5xxrnv4QxJyY/w9fo0gTctlKnxhHZj6c4rZCr97qEorRC7vv5sTZt+uucXv+t+RupHzmD6rFzyNn8EcWf/l/ImIDN66DKFaMGDk2WSztSjv2RysoFwORJjvWiK6N63yAyUmHM8SjAX+rRd8DIUJtFpBDfnJlrlFLR55/GGYvJQl4YKcdNaJnFISv4AXrmCtdMNlHr9A0bc3giuzkWWPO4eujF9MzozgO//Juvapr/KBR1QZpbAtTueyo1Y04n96f3KfpsUchgep2nITaDxrxOX+v8dqQc+yPllUsSssaacHo0auxG92SD8EgF5VIE+POXVAHhlLrfAWzCl1iQ0uRY88J2j+0J8AfpoNzEkGLhikNNbKmBez7WIy4CzLXmMG/oxfTL7MPDm5e06EPm1jQaXMHbj9Tsdwa1+5xM3vdvUrj23yEVTLWrlgZP4EFkYaHr7U459keqFVC2RnQPEk3zzhhRbfcY6ckGYZEKyiVqROQwfGnQl0TishORC0VkrYisLS8P3s4k1hSkF4XtHvMNGAuvq87+fYXzG/uQLfos8ir+bEsWfxp6IUOyB/JY6bN8Wrl2z7kGlzfg7BcARKg+YC51ex9H/nf/peCrZ0O+X6U7jlX87SDVCij9kazAPjTWL9Ub6ckGoUkF5VKNfwslkEXTnEfwtaHZJiIFjSOWLYC58Wu/ZoJSapFSapxSalz37t3bI3vEWE3WiNxjypKBnh5ettmUISZOHS18vFnx/DeR//FnmjP445Dz2Tt3KE/+urRZoaUKOvsFABGqDrqA+r2mUfD1UvK/eSHoe/kmWVZRn4T02mCkuuUCIJ7k1Lw0YQwXMwiHVFAu3+GLu7RmJL6izGCMAC7Gp4SaXocAExr/f0nsxIwdPvdY6HhKE3paXtAJls05ZR9h6hDhv98p3vsx8htQujmNywafy755I3hm68u8V/YREGL2SxNionLiJTQMPpzCL5eQ992ykO9X6aqhJoFt5UPREZ7IBeUrqkwi9U6PkZ5sEJRUUC6vAhMa61QAEJESfErCfxOs35ji5/U1vgSBKcCLsRc3NhREkD0G4Qf4RYTfjxfG9oF/r1V8sTXym6XVZOXiQXMYW7AvL2x/jTd2vQ8En/2yB5OZikOvxDbwYIq+eJzc798K+X417noqEjELJgw6guUCyXWNNVHR4DamVxoEJBWUy6NAKbBMRE4UkRnAMmArPrcXACIyUES8IrKg6ZhS6oPWL6AGqG38OvB83iRjNVnJtQYfGNYSwZsVXoDfbBIuP9TEkCK4/1OdTeWR3zAtJgsXlJzJQYVjWbbzbV7Z8RZKqaCzX/ZgMlM+aR72fuMpXvMQ2T+9H/L9Gjy22BdaRoivK0EHUS6aE5OzyhfcT5KLTCnF7jqXEeA38EvSlYtSygZMxZfxtQRfIeRmYKpSqrntL4CZFJA5VuSm5ZNlDbsJAYilsYI/tMWTYRHmH26iONM3KnlHhEWWAGYxc+7A0zis+CDe2r2C57e/hlfXgs5++W2zlfLJf8bRewzdVt1P1uaPQ26xex3sjnWrmAjQUrg63x9mZzUW2w4staWY67djclT63GUJHC7mUzBOY/6LQRsk0JOiiDzRjusqpdR57difMMaNG6fWrl0bemErNF2xpTI2wehqVyX2CALbJk89Jmd41e676xUL3tNJN8NN00wUZkbesFopxdLtr7Ki/BMmFU/gzP4nU5yVQYY1tBUlHic9l99Ietn3lE25FseACSH3pJms9MgoxmIKff1Y4vB4O00fLWWyoswZ6JYMlCUDIojxRYPFZKJPQQYWc6d59jMIgYisU0qNC3g+iHLR8fkIommfr5RSib0zREkqKBeAGlcVtgiKC83OKl/WUBj8XKm45X2dXrmw4EgTWdboFMx/d77NW7tXMLHoAOYOnEXP3CzMYcSNxGOn1zsLSKv6hbKpf8HR74CQeywmCz0zirGaLBHLGi0Nbg/VnTQLSokZZU5HNSobZc6ACGJ+4WA1m+hTkInZ1OUnbnQJ2qNcbmjPGyulbmrP/kSRKsoFIlUwCrOtDNHDe9L+aofijg91RvaEPx9uwmKO7gbwxq7lLNv5DuMKxnDR4LPokZNFOPcSk6uBnu/8FWvtNsqOXICz974h95jFRI+MYtLj/NTdRI3DQ304Lr9OgELQMruh0iOJ+4Um3Wqmd14GJkPBdHqiVi5dhVRSLgC1rmoawu4fpTDbKxAtvGLED3/ReXiN4rAS4ZKJElG2WnPe3f0hL+54nVG5w7hk6GwG5BWGZd6anHX0evt6LA1l7D7qRlw9Q3b3wSQmumcUkRl2Z4PoqbS5ulT2k0Lw5vaFGH9vs9Is9MxLj/r3y6BjEEq5GA7SFCM/vZDctHCfJgUtqzvKmhPW6ZfkPgAAIABJREFU6sMHm/jdvsLHpYqlX0f/UDGt5+HM7n8qPzT8zA3f3cX/ysNrkK1n5LF72i1oWcX0XH4zaRU/ht6jdMqcldgSUM0faIZNZ0VQWGy7Y55tZnd7jSaXBsGVS+MQr31FpGeoC4lIr8a1/WInXtckL60gAgUDWkYRephV/yePEo4YKizboHjnh+hvKod1O4hr9voDADdvvJ8Xt6wMK41Yyypk19G3oKXn0uut68j5cXnIXmRNI5Pr4lzNn4oTKOON6B7M9tj3aGtweansJMkRBtERULmISA6wDvgQCCdfNqtx7WciErpfvEFQfAom/DYxeno+ekboPp8iwrnjhHH94Ml1itc3Rq9gSrL789fhVzIqdxiLS1/i1u+ewB5oBHEztOxu7Jp+O64ee9Nt1X10+/huJIxGllWuGqrjWM2f6h2R44XJY8PkjH0Ra63DY3RR7sIEs1zOAroB/1BKtR223orGNbcAvYEzYiNe1yYvLT+iPmS6NRctoxuhEvzMJuGPh5g4aIDwzP8Uz32lR128mG3J4tLBczm597GsqfyKP315O1tsO0Lu07IK2X3UTVTvfybZmz+iz2vzsFZtDrmvNk7V/LpSqA5SQBkPTM4qJA6uxyqbmzqjTUyXJJhyOQFw4ZvwGC4PN+45qT1CGfxGblo++WE2rgRQ1iy0zO6ECqdZzcIfD/a5yF7doHj0cxV1Xy2TmDi211SuGnoRDR4HV315Byt2fRbGRjO1Y05n19F/R7wO+rx+ta9dTAhFF49q/o7S9iVeSGP2YaiJotFQUe/CFmJsg0HnI9gdaAzwRWMFfVgopezA58B+7RXM4DdyrHnkp4cz2saHsmTgze4RslWMySScN144eZSw8mfFvat03O2IOwzPHcLf9r6SQVkDuPuHJdz/w7O4tNBuEVevfdgx414cvUdTvOYhun94O+IO/mtn9zrY4SjHHaOxvx2hYWW8EeXFbN8dl2uX1RttYroawZRLdyCa3lzbG/caxJAcay4F6UXhbzCl4c3qCSGKEEWEWWNMzBkrfL41ummWzcm35nHl0As4rtcRvLvrU6756i52OkLPzNEz8ik7cgFVB5xD1pbV9HntTyGzyTy6h53O8phMtvQmsYV9KmHyOjA5Yz/QVSnFrlonLq+hYLoKwZSLB4imei0NMGzgOJBtzYlQwVgaFUzoH+Oxe5u4dKKwsQxueV+nzhm9gjGLmRN7H8Ofhp5PubOKK9ct5NPyr0JvFBN1o2ey69hbEd1L7zf/TO6GV4O6yZRSVLtq2eWowNsOl04HaysWV8zO6rhMu9SVYnetC0+wwXMGnYZgymUXsHcU19wbiI9tbUC2NYfC9PCmUwIgZrzZPXztPkJw2CAT8yaZ2FYLN76nU2Frn6toRO5wbhx5Ff2yenLrhsd4/OeXw1IArh4jfG6yvmMp/vwxeqz4JyZX8MJSp+Zip6MsrGw1f3i7YBpyMMz23XFpgOnVdXbVOrtsZl5XIphyWQOMEBF/g7z8IiL74BvytTrUWoPoybJmUxjm+GMfJl+xZRgDx8b2Fa6fYqLWCTe8q7Ottn03gVxzPtcPv4zj+x7Of7et4Pqv76XCFbrppp6eS9nUv1A1/jwyt6+jz6tXkl72fdA9WmPBZaWrJuJgf1croAyFKA2zbXfI5Ipo8Gg6O2sdRpyrkxNMuTyHL6f1YREJ6VcRESu+bDHVuNcgjmRZsume2TOCiZa+XlK6NTfkyr17CAuONKEruOk9nZ8q2ncTcGtwZv+TuWbE7ym1beeKdQv5X9XGMEQW6kadyM7pC1Fiotdb15L37UshK8rrPbaIg/0drd1+ImiaGRMP3F6dMqOKv1MTULkopd4EPgIOBj4QkYCdBkVkDL4CyonAJ417DeJMmjmd7pm9KEgvwiThdfLRMwrR00PXzgwsFG6cZiIrDf6+Quebne1TMA0uDwcU7stdY6+hMC2XG759kKd+eRVnGNlk7m57sWPGPdgHTKBo3WJ6LL8Fk7M26J5Ig/2Gm8Y/ZldNyMy9aLG7vVR10i7UBiEaV4pId+BTYAg+i+Rb4P/be9PoOLLzMPt5q7p6b+wACYAEwU3kUEPOvkrOaLR4RrE1GmuJFSWO4kiWEsVfrOR8jnxieYns45PPR5HiLIpteTufjzbLkTTaRhppNBqNpRnNytnJIQlw37A30HtVvflR1QAIAmwA7AZA4j5n+lR1bX3RNayn773vfe9TQDVfRBdwC7CXoJYzALxBVa+YPpe1lrhyufjqM1meIOdOLapJSCo57OIotWZeHC8o/+URn5NZ+PU7LW7vu5xkhEJbMoqKy58f/grfP/s4XfF2Prz93dzWUTtLMqpkDj5I25N/iRfPMPyG36DYc33N1PGJSJyOWAv2AqHZinJyrHaGgPWKYuFmNoHtNOT6nZkYmXhjrm1oHJedFVlEmoH/BbyPmZrO7JME8IEvA7+uqoubxWqNcLXIpUrFrzBRGqPk1e7YFq+EXRyp2XGbKwfp+l8bgl+9RXjbzuXnOxWE9nQUx7J4cfwQf3ro7zieP8Mtbdfy4R3vYWOio+Y1oiMDdD76xzjZ05Tat5Pdcx+5/jde8uFni0V7rJVk5OLAhorvc3ai8Ykxr2TUigUZlBuQ6VhE6G5e3ORzhrVD3VLui8g24BeBm5gZxzJEkH/s26p65DLLuipcbXKpUnDzZMvjuDUjfnzs4njNicdKrvLf/8Hn2dPw3n3BwMvlplS3xaItHSUigut7fOvUj/jCse/g+h7v7ft53rX5rTXncBG3ROrIIzS98g2iEydxE21M7v7HTO66Fz++cNLPjJOiLdp8QdlLrsf5yeVFma0n/GgTXrIxQ9jMTJZXHmY+lxpcrXKBYAzIVCXLVCWLX6uG6haCZjJdOFTY9ZU/f0J57Khy7y7hV24UrGUKJmJZtKdi0xONjZTG+auBr/Hj88+wMd7Bh3e8h1var619IfVJnHqOple+QeL0c/h2lNz2u8nueQeVlr55T4laDp3xVhwrqOnkyi6jOdO5vBjcZBcarR0Ushxijk1Pc9zMA3OFYORSg6tZLlU832WiPE7BrdGvoF4wffIlEhj6qnz+WeU7B5XbNsOHbrNIR5cvmJakg2PN/Fp9fuwgf3b4K5zIn+W29r18aPu7F9VUBuCMHafplW+QPvII4lco9NzAxOvfSbHnhouac0SEZidNk5MmV/IYL5iO5cXQqAnGqqRjEbqaao/JMqw+Ri41WA9yqVLyikyUx6jUCNENOvvHCLrSLkZV+fYB5Uv7leY4fOR2i33dyxOMILQkHeKRmfb2iu/yzVM/4otHv4OPTjeVRa3FdfpaxQkyB79L5sB3iBTGKLdsJnvNO8htvxuNXPhQtMXCd2PgOYuOuFvvqBUN+18a8321paK0JFdmamvD8jFyqcF6kkuVXGWKbHkc/1LjRdTFLowilwgMGBhVPvtTn1NZ+PnXCe+/XohFlieZprhDKnphHrTh0hh/eeRr/MPQs3THO/jwjvdyc/uix/SCVyE1+BhNrzxAbHQQL5Zhctfbmdz9j/GSM2l0xgtlyq6SjqRJ20nTLLMIfCeNl6o5h+Cy2dAUJxW7dF48w+pi5FKD9SgXWHzoslXOYpUmWChkuewqX3peefCg0p2Bj95hsaNjeQ/nZDRCU9y5aDaa/WMH+NNDX+FU4Ry3t+/j13a8h674EnKsqRI79zLNLz9A4sSTYNnk+t/I1Pa7cZt6OCdNlEPP2mKTiaRIGcnUxIu14ieWcB+WgCVCd0ucWMREkK1VjFxqsF7lUsX1XSYrE+QvNYWwXyFSGAF/4X6Jl88p//txn7EC3P964ZeuFSLW0h/O0YhNayLK3FMrvssDJx/hS8ceRFH+Sd89vGvzW6Y75RdLJHuaple/RfrQD7DCPGS+2JRSXZSSXRTTGyimNlBOd+M0bSXSvBWc2mlz1itevBV/KaJfAo5t0dOSwF7G/0eGxmPkUoP1LpcqFa9MtjJOccHEj4pVymKVFx4Zny8rf/OM8tigsrUVPnqnxabmpT8YIpZFayoIVZ7L+eIof3nkq/x0eD+dsVbu23Q3P7/xDpKRpc2sLeUcsZHDRCbPUho+RSx3lnjuHPHcOSJzplx246146V7cTC9uuidcduMlOlEnie8ka05tcDXTSMHEHZtuE0G2JjFyqYGRy4WUvCLZ8jjlBdKyiFcMQpYvMX7myRPKXzzpU6zA+64X7t219JBlS4TWZJToAuMenht9lS8f/x4vTxwmYce5p/tO3tH7pqU1lxEkrLxgjIsqkUqO2NSMbOK58yRy5wPxFEaQeZoI1XLwnSQaSeBHEtPS0ep6JNznJNFwu5fooNy+a1HpeNY6XrwNP774Ce2WQibu0JlpTHSaYflcEXIRkc3AZ4C3EYz4/wHwMVU9XuO8m4EPA/8I6AOGgceAT6hq7QnZMXJZiLybY7I8scAgzNoDL8cLyud+Fgy63LMB/s3tFh2ppf76FJoTEZLOwrWCQ5PHeODkIzx2/lkA7uy8nvs3vZldTf2L+oSy5zOyhDEuUd+nrZwjnR/BLowgbh6rUpi1LCCVPNb0Mo9M788j84wjctPdlNt2U27fTbltF+X2XaiTWnSZ1gqNFEx7OkZzwqSIWUusebmISBJ4HigBnyDoOf5DIAnsu9Q0yyLyKYJkmZ8HXgZ6gd8hyHl2vaqeqPX5Ri4Lo6rk3CkmyxPzRpaJV8IqZRFv/nExqsqPBpT//xlFBP7lTcLPbV36yP50zCFTI3JoqDjGt049yvfO/IScV+Capm28c9Pd3N5xHfYlQmaLFY+xZYxxccQh46RIWEtoslEFvxJIxs0TmTpLdOQA0dGDREdeJZI7O31opakvFM4uym27qbTtROdJXbPWaKRgupsTJKKmg3+tcCXI5TeATwO7VPVwuG0rcAj4j6r66Uuc26mqQ3O2bQEGgT9U1d+t9flGLrXx1Q9H+k/OH1nml7FLEwsOvjw3FXT2HxyCWzfDB2+xaIovTTAJx6Y5Eb0okmwuebfID84+wTdOPcK54ggb4u3c13s3b914+7x5xXJll2xx8an552KLTcpOkIokF0yMuVis4ngomgNERw/gjBwkUhgGQMWi0twf1m6CWk6lZXvDkkleDl68HT/eUvfr2pbQ3ZwgGjHjkdYCV4JcHgbiqvqGOdsfBVDVu5ZxzXPAt1T1g7WONXJZPJ7vMlnJkluoOewSkvH9YODl372gJKPw4Vstbtq0NMFEbZvW5MWRZPOWVX1+NvwCXz/5Q17NDpCyE9zT/QZ+sfcuOmf9ss6WXHKl5culiiAk7BgpO1UzL9pSsPLDREcPBMIZOUh09AB2KQiq8O04pa7rKHbfQqn7ZirNWxuSWHI5NEowjm3R25LAMhFkq86VIJezwAOq+pE52z8LvFdVl5QpT0SuAV4BflNVP1XreCOXpVPxK2TL4xQXShPjl7FLWWSedDPHxpTPPu5zfBze2C/88nWypL4YWyxaUxemjKnFwexRvn7yh/x06DlAeGPXDdy/6c3szGxhvFCmUKk99fJScMQhHUmStBP1j3JSxc4FzWmx888TP/s0TjbomvQS7RQ33kyx+xaKG2/CTyxlttL60yjBJKMRNjav/SbCq50rQS5l4NOq+ltztv8h8FuquugYTxGJAA8D1xA0s9VM/2/ksnwqfoWCmyPv5vD8eR7QfiWsyVwomYqnfPUl5duvBv/v3bNLeOceIR1b3IN4vpQxi+FccSTsl/kpBa/Irkw/t7bdxPVNe8lE6t+BbiGkIklSdpJIA0OV7dw54meeIn72aWJnn5mu2ZRbtgW1mo23UOratyp9Nl6ioyHRcM0Jh/a0iSBbTdabXP4U+CDwC6r60CWO+zBBlBl9fX03HTt2bMnlNnK5kJJXJO/mKLj5i/tl/EpYk7nw+xrOKX//ovLjASXhwH17grDlxaaQSYUd/UutG+TdAg+dfZzvn3mc4/kzWFhc27Sb29pu4Lrm1y86h9lSiFsx0pEkcbvBD3j1ccYOBbI58zSxoRcRv4JaDqXOvUGtpvtmKq07G5YbbC6NEkxrMkpryuQgWy2uBLmcA75+uc1iIvJfgP8IfEBV/3axn29qLvXFV5+Cmyfv5ih7c0J8F5DM8XHly/uDsOXWBLxnr3DXNlnUyOyIZdEUd4gto5NXVXnq3CA/G3uGJ8f2M16ZIG7FuLFlL7e23sDuzI66J7OMiE0qkiJpJy4ZxVYvxC0GzWdnniJ29mmi4wMAeLFmij23k9t2L6UNNzRcNG6iE40tPM/Ocok7Nl2ZmJkHZhW4EuTyQyCqqm+cs/1HBOWr2aEvIr9NEL78/6jq/1zK5xu5NA7Xd8PaTO7C8TK+i13OIpUcs3OWvXpe+eJzPodGoKcJ3nedxc2bWFS/RdKJkIk7i+rsny6GwrnJQrju89rUAD8bfZZnxl+k6BdpjjRxa+v13NZ2I5sTPXXtP6kGAMTtOHErtmIZma3CMPEzzxA/8xSJUz/FqkzhJjeQ23YPuW1vx8v0NuyzGyUY2xI60jGT6HKFuRLk8jHgU8DrVHUg3NZPEIr8W6r6X2uc/++APwF+W1X/aKmfb+SyMlSbzYpufmbiMvUQtxCM+wizL6sqT5+ELz3vczoLOzvg/ddb7O6q/WC3RGiKOyQWOV1uxfcZnrp4AGXZr/DixKs8MfYsL2UP4KlHd7yL21pv5NbWG+iI1TfViSDErCgJO07cjq9IjQaC2TzjJ/+B1MCDxM88haAUO/eR3/528n13ow3IqdYowUAwkr8jHTWpYlaIK0EuKYJBlAVmBlH+AZAhGEQ5FR63BTgCfFJVPxluex/wBeB7wH+ec+msqr5S6/ONXFYWX32KboG8m6M0O52/uoFkKgXEL+H5yqMDQZ/MWAFu7A1qMptbaj84YhGb5oSDXeMhU3Q9xvKXHkCZc/M8Pf48T44+x6FckPRhR2ort7XdwE0t+0g3IBAgZkWJ23ESVqyhgQCzsfPnSQ4+RGrguzjZ4/h2nELfXQ1pNmvkbJaObdHVFDPZlFeANS8XABHp48L0Lw8TpH85OuuYfoLBkf9ZVX8/3PY3wAcWuOyjqvqmWp9t5LJ6eL5L0StS9PKUvNJMIIDvYrk5rEqeUrnMdw8q33hFKbjwj7YK790rtNcIXxaETDxy0Rwxs8lXXCYKix/jMlwa5amx/Twx9ixniucQhG2pPvY2XcPept1sqnPTGQRhzQk7RsKOLzkD9LJQJTryCqkjD5I89sOg2Sy1kdy2e8ltvaduzWaNFIyI0JaKmnQxDeaKkMtqYuSyNvDVp+QVKXoFim5hJt2MX8aqFMjlpnjgxQrfe00RgvDl+/YImRrhy1HbpikRmXdczGTJZWoZAyhVlZOF0zw38RIvZg9wLH8SgBaniWubdrO3aTfXZHbWPTIsInbQdGbF6zpQcyHELZE4+RjJgQeJn3k6aDbruo78treT73vTZTebefE2/FhLwwZ+JqMROjMxk7K/QRi51MDIZW1S8ooU3QJFrzAdDCBekeGxHF99borHBnwsgWu64MZe4cZeYUNmoYeIkIpFLgpbnihUyFcWzu68WCYqWV7OHuSF7AFeyb5G0S9ii83O1Fb2Nge1mg2xzsuu1agqQzkYHA1eEwUhYtlELBtbgszTloAlwfPaQhABu/peLnwfj1h0JC06kzYdSZuOlE3Smb/5y86fJznwEKmBB3EmT+BHEuS33M3kNe/Dbe5f/t9kxfCSHQ0bgxOxLDozMZOTrAEYudTAyGXtU/ErFN08Ra8QTgWgnDw/yU8O5XjuRJlTE0Etp7dpRjQ7O7joF+vcsOXRfJmSW9/R+a56HJk6yovZV3kxe4AzxXMAdETb2Nd8Ddc27WZXenvNJi5VZTgHA6MwOKoMjCqDozAVdhHZFrQlgg5K1fCFXLDuq+KH731l5j3B+/lIOUJHKpBNZyic2fLpTFhszL9K0+B3SQ5+H8srUtj0BrJ73k+5c++yvzcv1hIkvGxQMENLMkpr0jGd/XXEyKUGRi5XFp56YY0mT9kr4atybqLC/uN5njuW58CZIp4P6Zhw3aYoN26KcF23RTLiIXjgeyQci6Z4lNFciYp/cbbnejJcGuWl7AFezB7gwORhKlrBEYdrMjvY07SLnamt9MQ3MFawZkQyogyMQTWQzRbY3AJb24RtbbCtXdjcDI698IPSFouoFSVqRYlb0Xlllq/4DOc8hvIew3kvXPcZznsM5YJtI3kPb84jwhJoS1jsyRT4YPT73DH+TaKVSUqd+8jueT/F3tuXJQmVCF6ysyFRagCxcEyMY8bE1AUjlxoYuVzZVPwKFa9MxS9T9stki0WeP55nf/iaKvrYFuzaGOeGLUlu2JJkQ5OFjYd6HnhlxCsjfhnmmWulnhTdMs+OHmH/+AEOFw4w5Y8GO7w4bmELXn4rfqGfnugmtrc7bG2DbW3C5haIXkIki8FCiFpRYnaMmOUQtRbXZ+P5ynhxRjhDoXCGch6HRiscGqmQoMi/iD7Kr0W+Q4c/RD7TT+Ha95Pvf+uyZuj0nTReogOs+jdlWSJ0ZGKkzZiYy8bIpQZGLlcfVeEU3CKvnMny9NEszx7NcWos6LzvaXG4fkuSnRtiJByLRNQi7ghxR0laHnG7gu27iF8BvwLzzDw5HyU3aMoaycNQLlzPwVA+WB/Nz26OUuzoGB3tgyQyRylHjjIZzh7hSIT+VB87U1vZmd7KttQWEnUODghk4+BYDo5EcCyHiESW3Gw0VvB45nSJp04Vee5UjtuKP+EjkW9yjXWCMbuDI33vJn39O0kl00u6roqNl2hvWERZOh6hIxUz2ZUvAyOXGhi5rA8qXpkTY1M8PjDMU4MTvHw6j3eJFrG4IyQci3hUiEeEhAOJCCQiPvGIknQU14PhfNDJPpKD7JzxmEHzEbSnoCMldKagPRmsd6SgKwXRWXnUJitTHM4NcmhqkEO5QU7kT+PjIwibEz3sTG9lR3orO1NbaXLq/9AVBMeK4IhzwXKx2QNUlWMTLk+fLFAefJw3T/wfbpEDjGuKB2Nv5+SW+9mzZQN7OqNEFvlQ9yMJvERnQ+atsS2hOeHQFHeMZJaBkUsNjFzWJ1PFCifGppgslcgWS0yVy+RKFfJlj0LZp1gJXoWyUpi1PrPdx7aE9rRFR8qaFkZHAjrTSkfCpzWhlxUGW/SKDOSOcygUzmDuGBUNots2xDrZnupne2oL21Jb6I53NSyFTETsoJZTlY7lLGpitIqnnDq0n/aDX2T31M8oqcPfeXfxeflFuro3c2tvnDs2x9mYuXQTlSL48daGzXBpJLM8jFxqYORimI3nu0Gzml8OlxVcf5mTiakXpLjxw2V13Q/Wg/0KLC6owPVdjhVOcmhqkMNTgxzJHSPnBdMZxK04W1Obp2WzNdlHKtKYjnEIAgaC5rRoKJ7IJbMJRCaOEX/pizQdewjU52G5gz8p/gIvaz/bWh3u7AtE8/rO6IJCViuGm+oEuzGp9o1kloaRSw2MXAy18NXHDUUTSKeM61dmcqRdNgrqg/pIuER9hOq6BlLCB99HUFAP9X3Ol85zJHeMgfB1qngWDfuINsa62JbasiK1G6gKJ+jHiYavuTUcOz9E+sBXSB/6Bpab51TyGr4s9/C50RsoqENzzOK2TYFobtsUJxO7uLxB2HJbwwZfGsksDiOXGhi5GJbL3Eg11y/XUThLIZQTSrGS59jUEQazhxiYPMzg5BFybjAtddxOsDXdz9b0Nralt7E11U86kgxrT+FLw+tBuJ1AZtP7Z++rXeNaSDhSniR15EHSh76GM3kKN9bKKx338AXvLTx4JsNEyccW2Lshyp19Ce7YHGdL80zAgVoOXrwNjS4tUGApGMlcGiOXGhi5GOrJ2hFOgKoyVDzLQPY1BrOHGMy+xqnc8enaTVeim/7MDrY27WRrZie9qT7sxYYPq4fl5pFyHvEvzi69EFXhxKwoMStC5vwLZF77GvFTjwNQ6LmTl7p+kW/lruEnJ8scGQ2j/DI2d25OcGdfnOs2xojagloOfrQJP9bUsAGYRjLzY+RSAyMXQ6NZa8IpugWOTR5hcPIQRycPczR7iGwlmBrZsRw2p7fSn9k5LZ22WEftEOXpZKMF8C+daXoutljErCipwjgdA98jc+Q72KVxKplNTL3ufo52vY3Hzjk8fqLIM6eLlD1IOMItPTFu35zgjk1x2lMOfqwJP9rUkMgyMJKZi5FLDYxcDKtBtf/GnRU0cMGEaiuIqjJaGp4WzdHJwxyfGqASBjI0Oc30NwWy6c/sYEtmO4lLBQv4FSw3j1XJh+OElkbUV7rOPEXHkQdJDL+Cb8fI97+Nqdfdz2TTDp45XeLxE0UeP1HkfC4Y+LqzzeH2zUFfze6eNiTR0rB8ZUYyAUYuNTByMawVqoEDru9OR6u5WsHzG5s5YD483+VU7nhQu8ke5ujkYc4VTgPBeJiNyd6wdrOd/qYd9CQ3z9+cFma1tipTy8qAkBo/Rs/g92k7/mMsr0Sp4/VMve6XgqzMlsPAmMsTJwo8fqLIS+fLeApNMYtbe2PcvqWJm7dvpKm55XK/jnkREVJRm2QsQipqr7u8ZUYuNTByMax1PPUuqOFUl9PTEqwQ+cpUULuZPMzg5GGOTR5mqjIJzG5OC2o2WzI76IxvuOCBK14RqRSw3PySRWOXc3Qef5SNAw+RmDqDG2smt+1ein13U27fDWIxWfJ56lSRx08W+dmJImNFHwH2dMW4rb+VW3d2s2NDBqsBErBESEZtUrEIyXUiGiOXGhi5GK5UZo/JKYdNbCvZtKaqjBSHOBqK5ujkkbA5LehzSUXS06Lpz2xnS2Y7TdGgFiFuEauSQ9w8i02vE3yoT/PQS2w88hAtZ5/FUo9yop1c752U+u6isuFGsCL4qhwcrvDEiSKPnyxwYKiCAm3JCLf2t3H79g5u3NLakBxjlgjJmE2gdnzyAAAUrUlEQVQ6FiHhXL2iMXKpgZGL4WrCV396LM5Mv47LSv0799TjTO4ERyePhMI5zOncienotLZYJ1sy26ZrOJvTW0ipYpVzSw8EKE/RevZZ2k4/Scu557G9Mq6TJttzK4XNP4fbcwdWmGF5rODxs5NBP82Tp4pMlRVLYEdXmn2bmtnX28Le3maak/UNBrAtIRmNBKK5yuaUMXKpgZGL4WpHVS+QTXV9pf7tl7wiJ6YGwya1IxzLHmakNDS9vyvRzZb0NvpSffTHN7Il2k58iaPwLbdEy7nnaTvzJK1nniVSyeHZMSY23sBk752Ue+/ESbRji43rKy+fL/PUqSIvnCvz8vky5XBegf72JPs2tbBvUzN7e5vpzNQvG8DVJhojlxoYuRjWK9Oy8WbEs1L9OJPlCY5NDXB8coDj4XK8HExBIAgbEhvYkuilL9FDf3ITmxO9ixaO+C5NQ6/QdvpJ2k4/RbQ0ji822c5rGe+9jdymNxBJbiBqRYlYEcqecmC4zPNnS+w/5/LS2SL5SvA99LTE2dvbPC2cnuZ4XZq5IpZFImoTdywSjk3kCpxjxsilBkYuBsMMnu9SnlPLWalotWx5nONTgxyfPBIuLxTOxngnfYlNbEkGr82JntrCUZ/06GHaTj9J++kniefOoQiT7a9jbONN5Dr3UG59HVEnPZ1FwFfhyGiF/WcrPH/e5fkzBbLFoC+rPR1lXyiba3ub2NKWrIsYHNsiFoom7thXxIRmRi41MHIxGC7NTLRamXJYy1l2Ms8lMlEe5/jkACemjnAse4gTU0cZDwd8ArRHW+mJb6QnvoGeRLDsjm8gOt800qoksyfCGs2TpCaOAeBbEaZatjHVvots+y6K7Xuwkp3ELAfHimKLzbFxj/1DfiCc0zlGckH/UMQStrQn2d6ZZntnKlymL7vvJmJZxB2LeNQmHrGJRtaebIxcamDkYjAsneqYnPL0QNCV68eZKJ7nxNirnJoa5HThDKeLZzlXGsINw5sFoSPaRk9iQyieQDob450XTPfsFMdJjx6iaeQg6ZGDpMcHsMJou0JqI5Ptu5hsfx259t14zVuJ2jGilkNEHM7lHV4a8Tk8WuHISImB4fy0cADaU1G2d6bYFspmW2eKvrbksqdgsC0hHtZq4o5FLLL6fTZGLjUwclke6XiEVDSCp4rvK56v4ToXbFvNNCeGlWd2qpuV6McRr4R4RfxKgaH8KU4Xz3C6cJbTxXOcLp7jXHEIP0ywKQhdsY7pWk53rIvOWDsdsXbSdhLLr5AeHyAz8hqZkYNkRl/DKWUBcJ0Uk207p4VTatuNHc2E0w5EiNhxJioOh8fgyGiFw6OBcI6N5HHD6UcdW+hvD2s3XSm2tqfY3JakPR1d8tgb2xJiERvHFqIRK3jZ1oqGPRu51MDIZWk4tkV7OkoyurjxAarziCeUT8VTKp6P6ymuv7IDAg0rx9x+nMaNx9FQNiWkUkTCzzpfGuZUMRROKJ6h0si0dADiVoyOaBsdsTY6o+10xNroiLaxyfPZmj1L2+gRMiMHSU6eDD5JLPJNmylkeiikeyimuyllenGb+rCiTYFwrDi+leDklMXhUY8joyWODBcYGJpiLD/TrBiPWPS2JuhtTbC5Ncmm1gS9LcF6U2JpU087tkUsYuHY1rR0GtV/Y+RSAyOXxdOUcGhLRhuST8n3lbLnU/F8Kp7ien74XlcsZNawclTH43i+h6uBbDx16z5PjrihbNziBZmbK77LUGmY4fIoQ6VRhssjFywrOvPwF4QWp4mOWDtdkSZ6XY8thUn6cyP0TQ3TM3me6KyBoOVYC8VMN4V0D4V0N+XMJtymzfiZXiKRNLaTJFuJMTimnMhWODVR5uR4gZNjBc5MFPH8mWtl4pELZLMplNCm1sSif+CJSCgZIWbb0+uXG4hg5FIDI5faOLZFZyZG3Fmddt5q7WZGPj4V19R2rlY89fB8F9d3cUPhBOJxL7OJzUcqJcQvYbnFBQdtqipZd5Lh0ihD5ZE5y9ELAgogkE+znaRDonSqxUa3QncpT29+nJ5Sjg2uS5fnEcWimNpAMd0dSCfdg5fsxEttxE/1oMlOsOKczwunssyIZywQz/nJC6c1aE06dKRjdKRjdGZidKSj4fsoHZlg+6VynnU3Jy5rvE0tudQ/94HhqkEkyP7amnRWNYWFY1s4NiS4+B+CN6tvZ6b5TfGVC7fPWjesbWyxsW2b6DxhxnPF46mL53vBUr0atVwLdRIoCfwYgI94FfBdxHcRvxwuKzQ7TTQ7TWyn/6KrVPwKw+UxhkuBaMYqE8GyPM6xygT7yVEQF+JpYGYys2YidPmwwR2mZ/wYXcMV2j2PNs+nzfNo1QjpaDPdiTb2JDqoJDvxUp14Xd246R6KyV5Ol5s5PSmcnHQ5PVFhOFfm3GSRl09PTIdLzybh2BfIZkZAMXZ3Z7ipr7VhY2zWhFxEZDPwGeBtgAA/AD6mqscXcW4c+APgnwMtwH7g46r648aV+OonGglqK2shKuVS2JYsOQJntmiml2F/kBHS2uZS4lHVQD6hcKryqTa5XTxex0LtGNixOdnNNBCO5yJaRrwK4rvh9AGKYzl0x7vojnctWM6iV2SskmWsPD4toLFysDxaGee58gQ5Lz/vuTGdpN2boG3iAO1jfiigQEKtarE1kmZftJl0rJVksplIWwt+tImy08wETYz7GYa8NOcrKU6Vk5zKOwzlXV44Oc7IVHk6wADg1U/eS6P+ia+6XEQkCfwQKAEfIMhi94fAIyKyT1VrtT39JfALwG8CA8C/Bb4nIneo6v7GlfzqRERoTTo0J1a3ttJIbEuwEZbSyjdbNrOF5KtOBymoMr1v9rphZRARIhIhQoR5KrkXyCcQjoev3vTSVz9cApaDWg5K4sKLhDUb8Svge4h6wRIffI/q1M9xO063Hb+kgCq+y5Q7RdadIluZYtKdIutOMulOMVmZIluZZLAywQvuFFmvgHeBAqeCl3+C6JSSVp8mzyfj+6TVJ+P5ZHylzffp833SEiGVjpJsjhO3kkRIEY+2E+NNQOryvvgFWHW5AL8GbAN2qephABF5ATgEfAT49EInish1wPuBf6Wqfx1uexR4GfgkcF9ji351EXNsOtOxNTlga7VZTg2pij8tJablVF1X/8JtyoyYfA3O1XC7r5jghstgtnxil/hhUZXQjHj8Oev+jJCYG3CioB7ie6B+sK4eqAa1H/WRcLtjQWu0hdZo7flmfPUpeMVQRIGAptwcBa9I3stTrExRdHPk3TzDXoHjXpG8XyanFcrM7qfymRYT53hS/bn6rBtrQS73AU9UxQKgqoMi8hPgnVxCLuG5FeDLs851ReRLwG+JSExVFz+59zrFEqE1Ga17RlhDgGUJFvWpBarOSGa2jHT2OkzXpJgjJmXm2PA/qs/G6vnB+sznzX52zj6W6WtdmdKbr2YuwQ5sLMBh7iEy6z6KhC8AUSwUFRBRBB8FRKoPdg3Wpfo9Bdt9P2iu830P3/fDpYf6ocB8D1Uf9X2afZ8u9UEV9b3wfoY3P1wG98OHWTXmslch7xVCERXIuQXyXoFMcyfx6EyfUL1ZC3J5PfDAPNtfBt67iHMHVXVu4+XLQBTYEa4bFiARtelIx66IXEaG4IFoB0+z1S7KvFQlM1c6C0kMmH6AVx/cM++r+2XO+9nnrs3voZ5UJTK9nP5hoLMkX/2eL3xf/eWhhFJSPzhKlUQ009Dvby3IpQ0Ym2f7KNB6GedW9zcEIRilfiWTcGwycVNbMdSPaRFMP7Ou/od/oxGRQLxX2Fd5ZT8dl4mIfBj4MEBfX9+yrmFZQlcmXs9iGQwGw1XDWmgLGWP+GspCtZLFngszNZgLUNU/V9WbVfXmzs7ORRfUYDAYDItjLcjlZYK+k7nsAV5ZxLlbw3DmueeWgcMXn2IwGAyGRrMW5PIN4HYR2VbdICL9wBvCfZfim4DDrI5/EYkAvww8ZCLFDAaDYXVYC3L5HHAUeEBE3iki9xFEj50A/qx6kIhsERFXRH63uk1VnyMIQ/5vIvIhEXkL8CVgK/B7K/g3GAwGg2EWqy6XcAT+m4HXgL8FPg8MAm9W1alZhwrBuNu5Zf5V4K8JRvV/G9gM3Kuqzza46AaDwWBYgDURLRbmEHt3jWOOMk8wnqoWgP8QvgwGg8GwBlj1movBYDAYrj6MXAwGg8FQd4xcDAaDwVB31v1MlCIyBBxb7XJcQXQAw6tdCEPDMff56udy7/EWVV1wFPq6l4thaYjI05ea2tRwdWDu89VPo++xaRYzGAwGQ90xcjEYDAZD3TFyMSyVP1/tAhhWBHOfr34aeo9Nn4vBYDAY6o6puRgMBoOh7hi5GBqKiPyOiLwmIr6I3L/a5TFcPuaerg9E5Msi8oKIPCciT4aJgRfNmsgtZriq+T5BMtK/Wu2CGOqGuafrg4+o6jiAiNwAPCwiHarqL+ZkU3NZR4jIJhH5HyLyuIjkRUTDuXPmO3aziPy9iEyISFZEvioiS54TWlWfUNWByy27YfnU+76be7o2acB9Hp/1tnmp5TFyWV/sAP4JwfTQjy10UDiz5w+B3cAHgF8BdgKPiEhqBcppqC/mvq8P6n6fReQzIjIA/B/g3YuttQCgqua1Tl6ANWv9Q4AC/fMc9xuAB+yYtW0r4AL/Yda2HxCkj5jv9YY51/wRcP9qfwfr8VXv+27u6dp8Neo+h/vvBZ4Coostj6m5rCN08b867gOeUNXDs84dBH4CvHPWtreqascCr5/Ut/SG5VLv+25YmzTyPqvqd4FWYO9iy2PkYpiP1wMvzbP9ZWDPCpfFsHKY+74+qHmfRSQhIlurO0TkDqAdWHRfm5GLYT7aCNpt5zJK8Otl0YjI74vISeAO4C9E5KSIbKpDGQ31Z1H33dzTK57F3OcE8AUReUlE9gOfIuhzme+8eTGhyIaGoqq/D/z+KhfDUEfMPb36UdVRgh8Py8bUXAzzMcb8NZSFfvEYrg7MfV8frMh9NnIxzMfLBO2yc9kDvLLCZTGsHOa+rw9W5D4buRjm4xvA7SKyrbohHIz1hnCf4erE3Pf1wYrcZ5MVeZ0hIu8JV98C/Gvgo8AQMKSqj4bHpIDngQLwCYJ4+T8AMsA+VZ1a6XIbLg9z39cHa+k+G7msM0RkoRv+qKq+adZxfcBngLcBAjwMfExVjza6jIb6Y+77+mAt3WcjF4PBYDDUHdPnYjAYDIa6Y+RiMBgMhrpj5GIwGAyGumPkYjAYDIa6Y+RiMBgMhrpj5GIwGAyGumPkYjAYDIa6Y+RiMBgMhrpj5GIwGAyGumPkYjDUGRH5ExFREblrtctiMKwWJv2LwVBnROQ4EAc2LmFec4PhqsLUXAyGOiIitwCbgQeMWAzrGSMXg6G+vCtcfm1VS2EwrDJGLoZ1SdgnouH6L4vI4yIyJSKTIvKwiLxxmZf+JSAL/GCR5dgQluW1efb962o5RWTHnH17w+0/W2Y5DYaGYuRiWNeIyCeBLwBl4NvASeDNwMMicscSr7UH2AV8R1XLizytOmd5Zs61bOD/nbWpbc55/z5c/vFSymgwrBRGLob1zr8FblXVu1T1lwnmFv8cEAU+ucRrVZvEvrrYE0IJ5ZkjF+DdwHbgp+H7abmISBfwfuAwpvnNsEYxcjGsd35PVZ+pvgk74X8nfPtzIuIs4VrvAorAg0sswyiQEpHZ/x4/DpwH/nv4fnbN5aNADPi0CRowrFWMXAzrnW/N3aCq5wiaq2JA+2IuIiL9wA3A95cxB/kFTWMi8jbgRgKxnA33tYX7YsC/IZgX/W+W+DkGw4ph5GJY7xxfYHs2XMYXeZ0lN4nNYm6/y8eBKeCzwES4rTVc/jOgC/ifqlpYxmcZDCuCkYthXVPHZqV3AS7wzWWcOxouMyJyM/AW4HOqOgaMh/uqzWIfI+ij+V9zLyIiZ0Tkd0XkEyJyIox8+5yI2CJyp4g8KiI5EXleRPbNOffdIvIDETknIgUReUlE3jtr/01hdNoHZm1rFpEXReQxEVmshA3rBCMXg+EyEZENwB3Aj1V1ZBmXqNZcmghqLRXgM+G2as2lTUTeCuwF/mru54Sd/BuBDwI94fJ/AB8Kl/8b+FPgnxLUgj49pwz7gL8H/jnwDuBHwBdEZBdA2C/1deC3Q1k54fEO8E5VLS7j7zZcxURWuwAGw1XA/QQ/1JbTJAYzcrmRoAb0eVU9EW7LAkpQc/n3gMfFYgC4Llz+rap+Ilx/SER+nUAW+8KaEGHt6IOzT1bV36uuh2HQPwqPuRU4GO76PWA/gaDeTCCk21V1FINhDkYuBsPl8y4CAXx9medXH87/CRBmjV1RVU9EpgjE0w18RVUH57nGPqAE/H/VDSISIQhK+GxVLCEZZprbqkECHwF+FdhGUIOqkp9VlhdE5CvAn4Wb3rRAWQwGIxeD4XIQkRbgbuBJVT21zMtUH/ybCAZgvjRn/0S4DxYeNHkd8JSqTs7atodgvM7Dc47dB7wEICICPEAgr/8GPAuMAG8KP+vAnHMPA0ngj1X1qVp/mGH9YvpcDIbL4x0E/Q6XM5hxdq1iPnlU+10emT0mZw7XETRZzd3mAS/Os/35cP0O4B7gn6nqH6nqd0NpbCcYszMtFxH5pwR9Qk8BvyIiiUv+VYZ1jZGLYV2iqqKqcon9/eExR2tc6nJCkKuf9cVqeVT10Xn2Xxvue/N854ed67u5WC7XAwdnhyyLSC/B2J3qsZvD5WyJ7AH+JfCSqnrhtrsIxtV8HHhPeI2PLvFPNawjjFwMhsvjceA3VfXQKpbhGoLmr/nk8tycbdWO/2rN5VnCIAEReUsYAPAAQa1lP4CIXEPQn/QXqvpfVfU48NfAx0UkVe8/xnB1YORiMFwGqvrHqvqpVS7GdQRjbOb21SzUVDZc7R8KpfhrBM1jDwC/QJDXLA7sF5GNBOls/gH4d7Ou80dAM/Abdf1LDFcNZiZKg8FgMNQdU3MxGAwGQ90xcjEYDAZD3TFyMRgMBkPdMXIxGAwGQ90xcjEYDAZD3TFyMRgMBkPdMXIxGAwGQ90xcjEYDAZD3TFyMRgMBkPd+b+mAs3+hPFqYAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "class FlassPlot:\n",
    "    from matplotlib import scale as mscale\n",
    "    \n",
    "    class ExpScale(mscale.ScaleBase):\n",
    "        from matplotlib import transforms as mtransforms\n",
    "\n",
    "        \"\"\"\n",
    "        Scales data in range -Infinity, 1 using log(1 - x)\n",
    "        \"\"\"\n",
    "\n",
    "        # The scale class must have a member ``name`` that defines the\n",
    "        # string used to select the scale.  For example,\n",
    "        # ``gca().set_yscale(\"mercator\")`` would be used to select this\n",
    "        # scale.\n",
    "        name = 'exp'\n",
    "\n",
    "        def __init__(self, axis, **kwargs):\n",
    "            \"\"\"\n",
    "            Any keyword arguments passed to ``set_xscale`` and\n",
    "            ``set_yscale`` will be passed along to the scale's\n",
    "            constructor.\n",
    "            \"\"\"\n",
    "            from matplotlib import scale as mscale\n",
    "            myscale = kwargs.pop(\"expscale\", 1.0)\n",
    "            mscale.ScaleBase.__init__(self, axis, **kwargs)\n",
    "            self.myscale = myscale\n",
    "\n",
    "        def get_transform(self):\n",
    "            \"\"\"\n",
    "            Override this method to return a new instance that does the\n",
    "            actual transformation of the data.\n",
    "            \"\"\"\n",
    "            return FlassPlot.ExpScale.ExpTransform(self.myscale)\n",
    "\n",
    "        def set_default_locators_and_formatters(self, axis):\n",
    "            \"\"\"\n",
    "            Override to set up the locators and formatters to use with the\n",
    "            scale.  This is only required if the scale requires custom\n",
    "            locators and formatters.  Writing custom locators and\n",
    "            formatters is rather outside the scope of this example, but\n",
    "            there are many helpful examples in ``ticker.py``.\n",
    "\n",
    "            In our case, the Mercator example uses a fixed locator from\n",
    "            -90 to 90 degrees and a custom formatter class to put convert\n",
    "            the radians to degrees and put a degree symbol after the\n",
    "            value::\n",
    "            \"\"\"\n",
    "\n",
    "            return axis\n",
    "\n",
    "        def limit_range_for_scale(self, vmin, vmax, minpos):\n",
    "            \"\"\"\n",
    "            Override to limit the bounds of the axis to the domain of the\n",
    "            transform.\n",
    "            \"\"\"\n",
    "            return vmin, vmax\n",
    "\n",
    "        class ExpTransform(mtransforms.Transform):\n",
    "            # There are two value members that must be defined.\n",
    "            # ``input_dims`` and ``output_dims`` specify number of input\n",
    "            # dimensions and output dimensions to the transformation.\n",
    "            # These are used by the transformation framework to do some\n",
    "            # error checking and prevent incompatible transformations from\n",
    "            # being connected together.  When defining transforms for a\n",
    "            # scale, which are, by definition, separable and have only one\n",
    "            # dimension, these members should always be set to 1.\n",
    "            input_dims = 1\n",
    "            output_dims = 1\n",
    "            is_separable = True\n",
    "\n",
    "            def __init__(self, scale):\n",
    "                from matplotlib import transforms as mtransforms\n",
    "                \n",
    "                mtransforms.Transform.__init__(self)\n",
    "                self.scale = scale\n",
    "\n",
    "            def transform_non_affine(self, a):\n",
    "                \"\"\"\n",
    "                This transform takes an Nx1 ``numpy`` array and returns a\n",
    "                transformed copy.  Since the range of the Mercator scale\n",
    "                is limited by the user-specified threshold, the input\n",
    "                array must be masked to contain only valid values.\n",
    "                ``matplotlib`` will handle masked arrays and remove the\n",
    "                out-of-range data from the plot.  Importantly, the\n",
    "                ``transform`` method *must* return an array that is the\n",
    "                same shape as the input array, since these values need to\n",
    "                remain synchronized with values in the other dimension.\n",
    "                \"\"\"\n",
    "                import numpy as np\n",
    "                \n",
    "                return np.exp(self.scale * a)\n",
    "\n",
    "            def inverted(self):\n",
    "                \"\"\"\n",
    "                Override this method so matplotlib knows how to get the\n",
    "                inverse transform for this transform.\n",
    "                \"\"\"\n",
    "                return FlassPlot.ExpScale.InvertedExpTransform(self.scale)\n",
    "\n",
    "        class InvertedExpTransform(mtransforms.Transform):\n",
    "            input_dims = 1\n",
    "            output_dims = 1\n",
    "            is_separable = True\n",
    "\n",
    "            def __init__(self, scale):\n",
    "                from matplotlib import transforms as mtransforms\n",
    "                \n",
    "                mtransforms.Transform.__init__(self)\n",
    "                self.scale = scale\n",
    "\n",
    "            def transform_non_affine(self, a):\n",
    "                import numpy as np\n",
    "                \n",
    "                return np.log(a / self.scale)\n",
    "\n",
    "            def inverted(self):\n",
    "                return FlassPlot.ExpScale.ExpTransform(self.scale)\n",
    "\n",
    "    @staticmethod\n",
    "    def yo():\n",
    "        import matplotlib.pyplot as plt\n",
    "        plt.figure(figsize=(4,2))\n",
    "        \n",
    "    @staticmethod\n",
    "    def forpaper():\n",
    "        import matplotlib.pyplot as plt\n",
    "        from matplotlib import scale as mscale\n",
    "\n",
    "        mscale.register_scale(FlassPlot.ExpScale)\n",
    "\n",
    "        SMALL_SIZE = 16\n",
    "        MEDIUM_SIZE = 22\n",
    "        BIGGER_SIZE = 24\n",
    "\n",
    "        plt.rc('font', size=SMALL_SIZE)          # controls default text sizes\n",
    "        plt.rc('axes', titlesize=SMALL_SIZE)     # fontsize of the axes title\n",
    "        plt.rc('axes', labelsize=MEDIUM_SIZE)    # fontsize of the x and y labels\n",
    "        plt.rc('xtick', labelsize=SMALL_SIZE)    # fontsize of the tick labels\n",
    "        plt.rc('ytick', labelsize=SMALL_SIZE)    # fontsize of the tick labels\n",
    "        plt.rc('legend', fontsize=SMALL_SIZE)    # legend fontsize\n",
    "        plt.rc('figure', titlesize=BIGGER_SIZE)  # fontsize of the figure title\n",
    "        \n",
    "    @staticmethod\n",
    "    def exppic(x, y, label, expscale, **kwargs):\n",
    "        import matplotlib.pyplot as plt\n",
    "        import matplotlib.ticker as ticker\n",
    "        plt.ylim(0.0, 1.01)\n",
    "        plt.plot(x, y, label=label, **kwargs)\n",
    "        plt.gca().set_xscale('log')\n",
    "        plt.gca().set_yscale('exp', expscale=expscale)\n",
    "        plt.ylim(0.0, 1.01)\n",
    "        plt.gca().yaxis.set_major_locator(ticker.FixedLocator([0.38,0.68,0.86,0.95,1]))\n",
    "\n",
    "        plt.legend()\n",
    "        \n",
    "    @staticmethod\n",
    "    def pic(x, y, label): \n",
    "        import matplotlib.pyplot as plt\n",
    "        plt.semilogx(x, y, label=label)\n",
    "\n",
    "        #plt.loglog(x, y, label=label)\n",
    "        plt.legend()\n",
    "        \n",
    "    @staticmethod\n",
    "    def axeslabel(xlabel, ylabel):  \n",
    "        import matplotlib.pyplot as plt\n",
    "        plt.xlabel(xlabel)\n",
    "        plt.ylabel(ylabel)\n",
    "        \n",
    "    @staticmethod\n",
    "    def title(title):\n",
    "        import matplotlib.pyplot as plt\n",
    "        plt.title(title)\n",
    "        \n",
    "    @staticmethod\n",
    "    def legend(**kwargs):\n",
    "        import matplotlib.pyplot as plt\n",
    "        plt.legend(**kwargs)\n",
    "        \n",
    "    @staticmethod\n",
    "    def savefig(filename):\n",
    "        import matplotlib.pyplot as plt\n",
    "        plt.savefig(filename, bbox_inches='tight')\n",
    "        \n",
    "    @staticmethod\n",
    "    def plt():\n",
    "        import matplotlib.pyplot as plt\n",
    "        return plt\n",
    "\n",
    "import pickle\n",
    "allres = pickle.load(open( \"epsilongreedy_ci_allres.euclidean.p\", \"rb\" ) )\n",
    "\n",
    "renameit = { \"Binomial CI\": \"Binomial\", \"Asymptotic Gaussian\": \"Gaussian\", \"MLE CI\": \"MLE\", \"Euclidean\": \"EuclidMLE\" }\n",
    "skip = { \"Binomial CI\", \"Euclidean\" }\n",
    "FlassPlot.forpaper()\n",
    "for name, res in allres:\n",
    "    if name in skip:\n",
    "        continue\n",
    "    x = [ x[0] for x in res ]\n",
    "    y = [ x[1]['width'] for x in res ]\n",
    "    ylo = [ x[1]['widthlo'] for x in res ]\n",
    "    yhi = [ x[1]['widthhi'] for x in res ]\n",
    "    FlassPlot.plt().semilogx(x, y, label=renameit.get(name, name))\n",
    "    FlassPlot.plt().fill_between(x, ylo, yhi, alpha=0.15)\n",
    "\n",
    "FlassPlot.legend(loc=\"center\", bbox_to_anchor=[0.8,0.825])\n",
    "FlassPlot.axeslabel('n / $w_{max}$', 'CI width')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZ8AAAEdCAYAAADTmmpHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOydeXxcVfn/32f2mWSyJ83WNG3TvUCBFqEILSBQKvAVF0CkbPIDURH5IiAqRcCvUBFU1CI7WkQUAZFNWqDsrdCWrXQJbZMu2fdk9uWe3x9nMtnbTDptlp7363Vfd+6559773GTmfu455znPI6SUaDQajUZzMDENtwEajUajOfTQ4qPRaDSag44WH41Go9EcdLT4aDQajeago8VHo9FoNAcdLT4ajUajOehYhtuA0UJOTo4sLS0dbjM0Go1mVLF+/fpGKWVu73ItPoOktLSUdevWDbcZGo1GM6oQQuzsr1x3u2k0Go3moKPFR6PRaDQHHS0+Go1GoznoDGnMRwgxCzgOyAU+k1L+O1ZuAixSylDyTNRoNBrNWCOhlo8QokQI8TrwCXA/8AvgK92qXA74hRCnJM9EjUaj0Yw1Bi0+Qogc4C1gIbARuA8Qvao9BRjA/yTJPo1Go9GMQRJp+dwElADLgDlSyu/3riClbEG1ir6YHPM0Go1GMxZJRHzOAiqAn8i9JwHaARQmYoQQolgI8XshxBohhE8IIYUQpYM81iSEuEkIUSmECAghPhZCfG2Auv9PCLFFCBEUQmwVQnwnETs1Go1GkxwSEZ/xwIZ9CA9ABMhM0I4y4FygBXg7wWNvB34O/AE4A1gLPCWEWNy9khDi/6HGqZ4GFqG6CJcLIa5K8HoajUaj2U8S8XbzAxmDqFcKtCZox1tSynEAQojLgdMGc5AQIg/4EXCnlPLXseLVQogy4E7gpVg9C/B/wAop5U+71SsEbhdCPCSlDCdos0aj0WiGSCItn43A0UKI9IEqCCGKgCOADYkYIaU0EqnfjdMBG/B4r/LHgcOEEBNj251u4b3rrQCy0WNUGo1Gc1BJRHyeQLV87hdC2HrvjM3xuRew0/chf6CYBQSBbb3KP4utZ3arB0pA91ZPo9FoNAeBRMTnIeBd1NjMZiHEvbHy2UKIZcBm4BzgTZRQHQyygNZ+xqGau+3vvm7ZRz2NRqPRHAQGPeYjpYzEBvEfRAlQp6v13NgC8C/g4kE4JYwKhBBXAFcAlJSUDLM1Gs1BIBpWSyQIHXXQvhv8zWB1gS0FrClqbXerMrMVhGngxRR7vzWiYES6lkAHeGrBUw/eevA2grcBfE2xpVld198KJgt84QqYfw3YXMP799EkjYTC60gpO4DzhRC3ojzLJgFmYDfwspTyw+SbuFdagAwhhOgleJ0tmeZu9UB54dXspV4PpJQPAA8AzJ07d2iC6m+FkEf9WG2pYDIP6TSaMYwRVQ/7SECtAy3QuB2at0HLTgi0qu+PPR2c6eDIAGcmODLBlQmOdLDYQZjVg9pkji0WVSZETFRCXWtfE7RVQXsVtFfHhKBOCYC3US3GIHxwzDawONT1LU61tjpiZQ71GZO6h0Cr+j0E2tS99oc1BZwZ4MyCzEkwPheaK+CNO2HdY3D8NTD3sth5NaOZIcV2k1JuRnWzDTefocaYJtNz3KdzDGdTt3qgxn5q9lIv+fz7+7DjDcibCeNmQeFRUHikenh0vkFqQRq7SKne9DuFpVNkgl5orYTmHdC6q0sE2quUMHTH4hj4YQ2AUC0CmzsmUqk9PwuTOmd3YYn4e53CBK5sSMmFnKkw8URIK4L0YnDlQNinXqJCPgh5IeyNrX0Q9netIwEIB1TLJRJU15GGEkhHBuQVdl0nJQ/ceZCaD2kFkFaoBNbUazRAStj0HLx2G7xyE6x7GI6/Fmafo35DmlHJaE8m9x8gDHwLuLVb+YXARillRWx7DdAYq/dqr3rNqLGsA0J01lfxBFpwNX6Odfd/gUeQZhsidxrkzVKiVHiU+hHaUlTryJYK5tH+rxkjdD5sjYh6iMqoaqlIo2sdL4uCYaiWRagDgh51bKC1S1g61x216thO7Gnq4VswB9KLIGuSEoHcaeqhHfIpAfE3d3VLBVq6WhLBDiUOwdh1PXWx63tiD/8M9cBPH69eflLHqetljIf0EsgoiXWrOVVX2lAxol1dbDK2hq4W0lBetISAWV+B6WfC2vvgvd/Bv7+nRGj+D2DyQvUypxlVDPoJJ4QY7KBHCGhONLK1EOLrsY9Hx9ZnCCEagAYp5ZuxOhHgz1LKbwNIKeuFEPcANwkhOlAu3ucBJwNnd55bShkWQtyMmlRahRKgk4HLgKsPZBTuHzet4UNngAkzvsBkSxozfV4Oa2ugoGUXzk//gZAGUghk5kTEuNmIcTFBSitUImRP1YJ0MIkEux7o/mao3wxN27q94ft7vumH/ertPuRT67BfiU9/mO3q/5o5CSac0FNk0gpi4ykuJQC9SQEyB/gJSqm604xw15iNEetei4SUGNhTVLeY1aHWB+q71NnlRx+H2P3HbIHjr4Y534S3fw0bVsA/L4XJJ8ExV6iXuNQ8JVaaEY8YrG+AEMIABjvuYaC6sh4Ffi+ljA7i/AOd+00p5cJudf4spbyk23FmVNy5/wfkA1uB26SU/+znGlcC1wETgF3Ab6SUywdzQ3PnzpVDSaP9VPlTvLjjRSraKmgOqKElgWCcM4eJ9hymGTDb6+GI1lpyW3Zhjqp+9kjqOIy8GZjGHYYl/zBwF6gHhy0lNvibGntQufSPbX+IRpTQdI5H+Jqh5mOiu9ciqj7A5O85X1qa7WBzIaxO9RC3uWIPdaf6X1h7f3ap7q+siarV0fkyYY3973p3MWkGh5TqxeDtu1WXHMCMs+DIC5WYuwv0y9oIQQixXko5t095AuJTiRKfCd2K21BC073NuwslArZY/deAMwYjQCOZoYoPwO6O3XSEOqj11FLRXkFleyWVbZVUtlfSGlQPN4GgwJFFmcnFzHCUwz2tzGnchTvkAyBiTyWYWUo4ayLRrEmI7CmYU3Kxmm1YbalYHRmxfv9Y193+dJ0kAUMa+CN+/BE/4Zigil4iKWJB0TvLhUS9pWMgpKEeMEjMwoTd4sJmdWGxOPZPbA0Dgm2x1k0rhLyE2qsJ716Dac867HWbMRlhohY7lXlTeT+rgBpnOvmuPPJTi3Db00CYsFkcOCxOHFYXDosLh9WJw+LCZnEizNauAX+TRXU56QfhgSEcgJ3vwpo/wvbX1ff/sHNh5v+oLsW0Iu2cMMwkQ3xMwN9Q0QBuB56QUrbH9rmBC4CbgfdQYylfQHmKTUV1bQ2qhTFS2R/x6cSQBt6wF2/YS0eoA0/YQ6O/MS5EnaLUHmoHwISgyJrGNKyUBYOM97ZQ0tFEQSRCTjRK1J6GN3M8voxifBklhLImYnJmYhMWrBYHNnsaFkcaNlsaVosDi9mGtbPfvdMTymTer4e5EY0QCHXgC3XgD3sIhLz4w16CER8YUaLREN6wByEFAolJym5rELG1CYmQfQWqNyZhwm52YLM4sFns2C1ObBanEieLA1vnmEXc68uqxjxiLZuwvwVv2EOoYQvmPR/gqv4EZ3sVBrDZncN/c4r50OFks+GlLtQ3SlSaLY2i1CKKUosoTC2kOLWYwtRCXFblAiwQOCwOHGaHWlscWE1WDGkQlVGklBjSUAsGhhFby4EXs8lMhj2DDHtG/DqaXngaYNur8MGDULVejW8duUR1yaUXq+5NzbCQDPG5DiU6R0gpPx+gzhRUSoWlUsq7hBAzYttrpZQnDNn6EUAyxKc/ApEAnrCHjlAH3rAXX9hHa7CVyvZKKtoqqGzdTmX7TjwRX4/jzECONJEfiVAY9FMQiZAfiZJtdpDuysOdVowpoxRf5nii3TyCBAKLyYxVWLB2rs02LCYrVosDqzm2NtmxWGzqIQ4Y0TCBsA9f2EMg4sMfVkswGoyf2x8NsjvQyG5/A7v8Dez2N1AVbCKcQKNXSDXzWQkSCCQuCQXSRKGwMc7kYJzFRa41nVx7Bmn2TKKOVKI2FxFrCtJix2ayxBYrdpMVAfgCbZhrPia19lMy6jYRDnn51G7n/cx8NrhS2EQAX8y12G1NpSyjjLLMKZRllJHpyKTWW0uVpyq+VHuqe9x7piOTopQiitxKlIpS1NphGfit25AGoWiIYDRIMBqMf+5dZrfYOSznMJwWJzazLS5Eaba0fYr1IUU0Ai0VUP4KrH9UjdVlTlSu2VMXQc4U3UU9DCRDfDYCu6SUi/dR7yVggpRyVmx7LTBVSjmqowgcKPHpTcSI9GgZecNeokYUX8RHs7+JZk8tzd5amv2NNAeaaQq10RJqpznsIdprSM5hGORHouRJQQ4msrCQLWxkmm1kmZ1kWFJIt6YgrC6iVqdaLA6iVgdRqxPD4sRsdSKEIBT2Ywl5sYS8mIMe2gLN7Ao2URlqpTLqZbsMUC26vLcyogbTQiGmh0IUhpXHkxQQRSABQ6j+WonAAKImExGTBcNkJirMam0yERVmPEJSQ5RqYVBrFhjdHiA2Q1IUiTA+EqE4HKHQMCiSVvJNNnLNLqy2VMxhH77WCj62WdngdLE+xc02U5ROa4ucuUxOn8yU7BmUZU4lz5WH1Wwl1ZqK2+bGbrbji/jwhX34Ij5C0RCGNGgONLOnYw/Vnuq4KNV4awh3mx+T48why5HVr7CEBzOPJoZFWJidM5t5+fOYkzcHp8WJWZhJs6WR4cgg3Z6O1TS8Xa0jBn8rNJZD+Ur48C/K8++478Ocb0HudD3OdpAZSHwS6YiehGrF7ItWVLbTTiqBIxO4ziGNxWQh3Z5Oul3Fb5VS4ov48Ef8BCKBvg+vaAhCfoxgOx3+Jpr8DTSH2mn1N9PmraHN30xTxMdOojQKg6gIAAGgXSW/iECmJ0puJEpuNEpeVH3Oi6rtnKiBWZiptgi22qxstdnYYrPSZu5ymS0JR5gZlZwlLUwSDiaZXaRb3UQzUpF2N8KWimG2Y1hsGBY7htlK1GLDMNswTDaiFjVLvlM6JTI23qMGDaPSUGVAREZpCbTR6Kujyd9EQ7CFhnA7uyNePjAC+OkeozZETrQJkwXqUwsAsAkLk1IKWJxWSlnWDCbnzCLF7sYszLhtbtw2N2m2NJwWZ49WRWa3Yc2IEYmL0bSsafjDfnyxlqkhDRp8DT1aSW3BNtw2NznmHGxmG3azHbvZPuDn7ts2s43WQCvr6tbxQd0HfNTwUR8hagmqOdSp1lQyHZmk29NxWvrxmDtUcGZA0Vzl+TZhPqz+P1jzBzWX6fBzlTepnls37CTS8qkDfEDZQM4DMc+zbYCrW4qEF4BjpJR5yTF5eDhYLZ9E6N5tExelsJ9goIVQoIVwoF25BMfmWhhS4on6aQ17aQ17aA17aAu20x5uozXkoTXioyXqp9UI0l+YcRuCCZZUJtgyKXHkUJRSQEFKEY5YyBOLMOMy20kxO0ixOEixOLFZU7rcb0XvtWlQ5VJKQmEvgZCHQNhDMOTDH/YQDPsIRvzKrdiIIiNhPBEv9aFWGoJtNITaaQi1EpFRJroKmJI5leLMqVicGZjNVlJtqaTZ0nDb3Lgsrv3qwjKkQSASiIuSN+zFH/ET3UuXo0BgNpnVWpgxmUxqLXquI0aElmALESPCjrYdfFD7Aevq1tESaOm3RQTgMDvIsGeQ7kjHbXUfut1zIS/UfAorfwJVG1SEhFnnwLjZ2gnkIJGMls8q4JvAfUKIa6WU3l4XcAH3oFJtdw8sOgUVfkeTZEzCFB/U7g9DGkqYgh2Ewj5CEX9sCRCO+AlGAhjRcNckydhESSMapj3cERMkL2EjQpEjh3GOTMxmNQ5kMttIsblJsaXisqWRYk/HYUvt5t1lTdrbpQDsVgd2Vza983l03mMgEiAYDeIP+SgMK5EKh/3q3jBhcmSQanfHxSbFmpLUB7JJmHBZXcohoFujIxAJEDJC/YpKItePGlFag61k2DMoyyjjvGnn9RCi/lpEgWiAWl8tTouT0rRSUm2pSbvfUYMtBYqPhlP/D1beBO/+To37SEMJkOUAzEfSDIpEWj6lwHpUWoUW4AVgJ13u12cSizINzJVSVgghpqPm+9wtpbw+2cYfTEZiyycZRIwIoWiIsBEmFA0RMkJqHQ0RjgQIRQIYMhoTGPXQTrGm4DA7RvzbdNSIEogGkFImXWyGk3A0TFOgiaZAE76wD0Ma7GjdwQd1e28R5bnyKEotwmI6BN/4I0HY/T6s/BnUfAxf/F+YvlgJkHbFPqDst8NB7CSzUQnYjogVdR7c+av+FFgipfwkVj8FNeenPhaUdNQyVsVHM7rxhX1KiPxNhI1wv0LktDj55vRvcnzh8djMNiakTSDTcQiGown5YM/7sPJmqNuoBGjq6UqAdLTsA0ZSxKfbyU4AFgBFsaJqVCrsN/fLyhGMFh/NSKc91E6jv5GWQEt8jtD21u08/fnTlLeUMyd3DhfPuph0ezqZ9kwmpE3AOsyTkQ86wQ7Y/QG8uhTqPoMTfgRlpygBsh+C3ZIHgaSKz6GIFh/NaMGQBi2BFpoCTbQF2zCkwaqdq3j686exm+0smbGEYwqOwSzMFLuLyXONal+gxPG3wp71SoDqN8GJN8CkhTBupoq+rUkqWnz2Ey0+mtFIOBqm1ldLrbeWGk8ND218iIq2Cublz+PCGRfitrlJtaZSml56aLlne5ugegOsWgoNW2DBjSqNRO50cI3qKYkjjqSLjxAiHUija7ynB1LKXUM68QhFi49mNNMWbKOirYJAJMDLlS/z3LbnSLGmcPGsizky70gVWzC1gIKUAkziEJmE2VEHNR/BqlvUpNQFN0LpF1Uai5Sc4bZuzJAsh4MsVIidrwG5e6kqpZRjyqVGi49mtBM2wlS2qWC2u9p38fDGh9ndsZv5hfO5YPoFuKwuHGYHpemluG3u4Tb34NC2B2o/U11wjZ/DwptgwnGQXQbu/OG2bkyQjPA6mcD7qEgHUVTeHhcqM2g+qgUkUVGtkVJOTIrlIwQtPpqxQr2vnl3tuwgbYf69/d+8VPES6bZ0Lp19KbNzZgOQ68yl2F18aLhlt1RC/RZYdbPKLLvwJhj/BRWMNL1on4dr9s5A4pNI+/pGVLrqR4F04J+oFk4R4AauRGUFfWesCY9GM5bIc+UxK2cWabY0vjrlq/z0Cz/FYXFwz/p7+Mtnf8Ef8dPgb2Bj48Z4DqoxTWapaumcepsKRPrGHbDnAyVELZXDbd2YJZGWz2dADlAipQwKIR4FLpJSmrvVmQusBX4w2lMo9Ea3fDRjDUMa7OnYQ52vjnA0zLPbnuWVylfIdmbz7dnfZlrWNACyHFlMSJswtltBUkLDVhUVe+XPlOic9DMonqvSM2SW6smoQyQZLZ9SYJ2UsjOOvIydOC4+Usp1wDvAt4duqkajORiYhImStBKmZk7FZXVx7rRzufGYGzFhYtkHy/jblr8RioZoDjSzqWkT3rB33ycdrQihMqCmj4dTb4eMCSogadV68DaodfMOlaJckxQSEZ8o0N5tu/Ob2NstpBoVz02j0YwC0u3pzMpW3XBTM6dy6/xbOXn8yazauYpb3ruF7a3bCUaDbGraRK23drjNPXCYTJA3A9IK4bRfqEyoq/8Pqj9UseDaqmDPOmjdrTLiavaLRMSnGhjfbbsytj66V70ZQBCNRjNqsJqtTMuaxnj3eBwWBxfOvJAfzf0RYSPMHe/fwX8q/oOUkt0duylvKU8oF9GowmRW0Q7c4+DUXygheu1WWPeIipBtRFSXXNU66KiNp/7QJE4i4rMBmN6tm+01lIfbnUKIGUIItxDiRlTct4+TbKdGozkI5KfkMzN7Jg6zg5nZM7lt/m0cmXsk/yj/B3/86I/4wj7agm181vhZPN37mMNsgXGHqXxAp/0SJi6Ez56FZ6+ArS+p6O+RoHLNrt6gJqxqEiYRh4MlwJ+Bs6SUL8bKngPOAnqf5CQp5VvJNHS40Q4HmkOJqBFld8duGvwNSClZuXMlT5U/RY4zh+8e8V1K0koAKEwtpDClcMxEDO9B2A81n6iEjU3b4IOHVEDSjBKY+20o6tbp40hTnnKOtOGzd4SSjHk+FmAc0Cal9MTKUoA7ga+j0ilsAW6TUj6dLMNHClp8NIciLYEWKtsriRgRylvKue/j+/CFfSyZuYQvFn0RALfNzaT0SdjMYzA3TtADtZ+q7jYpYdcaWP8odNQo8Zn7bSVGnbiylWecjpIdR8d220+0+GgOVYLRINtatuGLqC63+z+5ny3NWzix6EQumHEBNrMNi8nCpPRJ8fTvY4pwQLlgexvVdjQMW56Hj/8OET9MPQPmXNAVlFQISM1XoqST1SWl5XMP0CqlvC3Zxo0GtPhoDmWiRpTK9kqaA81EjSj/2vYvXqx4kRJ3Cd+d8914ZOx8Vz7F7uKx2Q3nb1Hu1iGf2g60wUdPQPnLYHHC4efBjLNUFl9QzgtphZBWfEin7E6G+ISBf0spv5Zs40YDWnw0Gqj11rK7YzcAH9V/xEOfPgTA5Yddzpy8OQCkWFOYnDEZu9k+bHYeMKSE9mpo3aW64kB9XveI8oBzF8DRl0LJcaoFBCq1vDsf0ooOyZZQMsRnN/CelPK8ZBs3GtDio9Eo2oJt7GjbQcSI0OBrYPnHy9nZvpPFExdzTtk5mE1mzMJMaXopWY4xmp4gEoLWncrdupOqDbDuISVG42bDvMtV2J5OhAlSx6l4cdZDJ31FMsTnUeBUoFRKGUmyfSMeLT4aTRfdx4HC0TBPbHmCN/e8yfTM6Vx5xJXxsZ9cZy4laSVjN01DoF11xQU71LYRhc9XwkePq32TT4Yjl/RM0SAEuHKUCNnHfvTwZIhPCfAh8CxwjZRyDMfa6IsWH42mJ93HgQDerXqXFZtW4LQ6ueqIq5iaORUAq8lKfko+uc5czCbz3k45eumoU5NPoyG1HfLCp/+ATc+BMMPU02DW1/rmCXJmQHoxODMPuskHi2SIz1JgKvBNoAl4FdgJ+PupLqWUtw/d3JGHFh+Npn+6jwPt7tjN8o+W0+Bv4OtTv87pE06POx+YhZlcVy75rnysnYPyY4loJNYVV9MV+aCjFj5+EnasVi2eySfD7G9AWkHPY+2pyjEhJadrrGiMkAzxMVCTSff2l+ncL7tHux4LaPHRaAam+ziQL+zj0Y2Psr5+PUflHcVlsy/DZe2a9yIQ5DhzyE/Jx2EZg5GiQ17VFedv7Srz1MHGZ1SXnIxC6Qlw2LmQOaHnsVanckxIHadizY0BkiE+P6dvJIMBkVLeOmjrRgFafDSavdN9HKh7VIQsRxZnTjqT+YXz+6RlyLRnUpBaQIo1ZZisPoB4G6G5AiKBrjJfM2z6lwrTEwnA+GOVi3ZOr1jMZpty03YXjHo3bT3JdD/R4qPR7BtDGlS2VdIUUPHOylvKeWLzE+zq2EWmI5NFpYs4sfjEPm7YabY08lPyx94kVcOA9j3KPbt7OoZAu5qouvnfqqVUeBQcfq7ykuuOyaycEyw2JUhmm3Ld7vw8CoRJi89+osVHoxk83ceBpJRsbNzIixUvUt5Sjtvm5rQJp3HS+JN6dMcBuKwu8l35ZDmyxtZEVSOqBKi9qqcIhXyqFbTpXxBohbyZqiVUeNTgxn6ESU1qNVvBZI0JUmw7/tk+rOF+kio+Qoh0YB6QC+yUUr63/yaObLT4aDSJ0R5qZ3vrdiJG18yM8pZyXtjxAhsbN+K0ODml5BROnXAqbltPl2O72U5+Sj45zpyx5aY9kAhFAvD5Ktj4NPga1fygw86FkmOVwOwvJoty644vaQet1ZQU8YmJzm+AbwGdlv9ZSnlZbP/lwG3AV6WUa/fb6hGEFh+NJnGC0SAVbRV0hDp6lFe2VfJixYtsqNuA1WxlQfECFpUuItPR0+XYarJS7C4mx9k7Z+Uox4gqr7i2qi73bFCCtGM1fPqU2p9RAjPOVu7Yjkzlmm11JccjzurqEiNHWvLO24tkOBykoFJkHwHUA+uAxcBj3cQnH6gC7pJS/jhJto8ItPhoNEOnPdROtae6jwhVe6p5qeIl1tasRSA4vuh4zig9g3Ep43rUK04tpiC1l3vyWMAwYiK0p6cIGVGofEfNFWrd2fMYk1WJkCNDzQ9yZnQJU+9ya8rgBcVkUS7fnS0ju7srTt1+kAzxuQW4BXgc+I6U0hdzv46LT6zep4BfSnnMfls9gtDio9HsPx2hDqo91X0S0TX6G3m54mXernqbqBHlCwVfYPHExRS7i+N18l35jE8b3/uUY4NOEWqvUonqOpGGCtfjb1Gu24HOdWu3slYV5FT2k9rbZFEOCzlTVIrw3BmQNVGVDwarS9V3DT1MUjLEZyOQAUyWUgZjZf2Jz9PAsVLKoiFbOwLR4qPRJA9PyEOVp6qPCLUGW1lZuZLVu1cTjAY5Mu9Izpx0JhPTJwKQ7cxmYtrEseWM0B3DAE+tagl1F6F9IQ0V4qc/keqog8at4G1Qdc32nmKUO33vSfCyy/pOik2AZIiPD3hFSnlOt7L+xOevwNeklGNq9pgWH40m+XjDXqo91bQGW3uUe0IeXtv1Gq/ufBVfxMclsy/hhKITAMiwZzA5Y/LYckTojWGoialte3rOE9ofvA3QsAXqN6uleYea8AoqukLedCVGeTPUGFPn33cEiE8bKqr1Gd3K+hOft4CZUsoxNUKoxUejOXD4wj6qPdW0BFt6lPsjfpZ/tJzPmj7j/Gnnc1rpaYDKnlqWUdZn0uqYQ0o1MTUSUGNC0XBsHftshLtC+SRKJACN26AhJkYNWyAYa4naUlSLKHcGlH0Jpi0aciTuZIjP+0AJMGGgbjchRCZQCWyQUp40JEtHKFp8NJoDjy/so8ZbEw9WChA2wjz4yYOsq1vHWZPO4itlX0EIgcviYmrm1LEZJ26wSNklSEa4rzhFQ6r7LtxfCM5+ztVerUSoU5BadwESrloD42YOycSBxCeR14Z/AncCy4AfDlDnl0Aq8I+ELdRoNIc8LquLyRmTKYwUUuOpoSnQhNVk5TtHfIc/f/Znnt/xPL6Ij29O/ya+iI/NzZuZljVtbCauGwxCqOgH+0pSFw2rqArBzsXT162beaYAACAASURBVEFBCJXmIb0Iyk5RZSGPGkvKnZ500xMRnz8AFwNXCyHmAs/EykuFEFcB3wAWAJ8CDyfVSo1Gc0jhtDiZlDGJwkghO9t30h5q55JZl+Cyunil8hV8ER+XzbqMIEE2N21maubUPtESNN0wWyElWy2gxpRCHTFB6lCC1H3Saye2VCiYc0CCnA5afGKu1acBTwHzgeNiuxbEFgGsB74ipQz1fxaNRqMZPA6Lg6mZU9nRtoPmQDPnTj0Xl8XFs9ueJRAJ8J3DvwPAluYtTMmc0idSgmYATCZwpKulk5BPiVCnIIV9B9SEoYbXWYSaYDoJMAO7gZeBf8kxGixOj/loNMOHlJLK9koa/Y0AvLbzNf665a/MyJrB94/8Pk6LE4GgLKOMDEfGMFs7RujsqrPY1eTTIaIDi+4nWnw0muFnV/su6nx1ALxX/R6PbHyECWkTuPaoa0m1qQfkxPSJYy8czyhmIPEZdEeeEGJorg4ajUaTJErSSihIUXNO5hfO53tzvsfujt0s+2AZLQHlpl3RVkGtt3Y4zdQMgkRGkT4VQvxXCHGVECKp7VohxHghxD+FEG1CiHYhxDNCiJJBHjsxdmyrEMIrhFgdc4joXa9SCCH7Wb6SzHvRaDQHlmJ3MePdKszOkXlHcu1R19Lob+TO9++k3lcPqHTeezr2DKeZmn2QiPjUo9Io/AGoEUL8XQhxhhD7N81YCOECXgemo7zplgBTgNWxYKZ7OzYbFex0NnAlcH5s12ohxIx+DnkF5SjRfXlzf+zXaDQHn/yUfCakqRTUM7JncP3c6/FFfNzx/h1UeaoAqPHWUNlWiR5aGJkkMsnUDJwOXAKcBdhRabVrUcFG/yyl3JSwAUJcA9wDTJNSbouVTQQ+B26QUt6zl2N/Bvw8duz2WFkKsAN4U0p5bre6lcA7UsoLE7UR9JiPRjMSafQ3UtFWAUCVp4pfr/s1ESPCtUddy6SMSQDYzDZsJhtmkxmLyYJVWOOf44vo+jxWwvZIKQlGg4SNMGZhxmqyYjFZDnpcvGQnk8sAvolqqXRGr5YoV+tHgb9JKVsHOLz3uV4DHFLK43uVvwkgpVywl2NfQAU6ndGr/J8ob7w0KWUkVlaJFh+NZszREmhhe+t2JJJ6Xz13r7ub9lA7PzjyB8zI7q8DZO8IhBIpkxIpq8mKw+LAaXHiMKv1SApsGowGCUQC8XUgGiAYCRKI9h8TrvPe+lv3V7a/HDBvNyHEVOBS4EKgCCVCQSnloGZ8CSFqgeeklFf2Kl8OfENKmbuXY58DSqWUR/Qq/ytwATBdSrk1VlYJZAJWlHv4h8CdUsp/DcZOLT4azcilLdjGttZtGNKgNdjK3evups5Xx1VHXMWReUcm/XqdItQpSp2fD1SrKRwN44/646LSXWwkB65bUSAoTS/dL+/BZITX6RcpZTlwkxBiKXAX8ANUl9xgyQJa+ilvRonF3tgKnCqEyJZSNgHExqA6W2Pdk1A8D3wAVADjgO8DzwohlkgpH0/A3oTZueSiPmXuMxaRdcEFGH4/u6+4ss/+9HPOIeOr5xBpaaHqB9f02Z/5zfNJW7yYcE0N1Tfc2Gd/1qWX4j75JII7Kqi95ZY++3Ou+g4p8+cT2LyZul/e0Wd/7rXX4jrqSHwbPqThN7/ps3/cT27CMWMG3vfeo/G+P/XZn3/rrdgnTaTj9dU0P/pon/2Fv1qGtaCA9pdeouVvT/bZX3Tv77BkZtL6zLO0Pftsn/3jH7gfk9NJ8xNP0PHyf/rsn7DiLwA0PfwInjfe6LFPOByUPPgAAA3Ll+Nb0zPprjkjg+Lf3wtA/d334P/oox77Lfn5FN31KwBqf/lLgpu39NhvKy2l4PbbAKi5eSmhysoe++0zppP/k58AUHX9DURqe3pmOefMIe+6/wVgz9U/INrasxPBddyx5H73uwDs+n9XIAM933BTFy4k+9sq1u+h9N3LkFH8ET/ify/hxnk38srjvyB7xW8JuPJwWVyYTWbMwkzwp99B5mVjeX0t1ude73N+/61XQ4Yby8tvY/3P2333L7uOgAMiT79AYPX7dP/vCGHCdt+dOCwO5BP/IvL2WtQjSar4n3YbWX+8h6iM0vHAI4TfX4+USj4kEtJSEb+8kaiMIu9bARu3qvKYvhi5WQR/pibW2n7/OI5tu3rYZozPJ/gj9b+3//oRTLt7freiZSWErladP/Zf/AlTQ3PP/bPKCF2hRiscS+9FtHkwHr2/z98gGey3+AghZqHGgb6FeqgDDCKKXVL4E0rs/iKE+AHgA34KTIztjwcvklJe3f1AIcSzwFrgDtSYVR+EEFcAVwCUlAzK+U6j0QwTFmHGZXESMZlJtaVyzpRzaHrnsfi8oE4eePsnhHLSmF8uOaHDi9lkxiSUMJlNZj6r/QBbOIfxvnpyjBAWMfhxIBlreREEq78ZS68oAdIUpqn1cwCswVYsvXL2SCNMIJbjyGZEMCfYMyWlpDXYSp23jlxPDVZ/AxEjilmYMJss+Dxm9tR/iNvmZnI0gF0amISJ4ehEHOqYTyZKbC4GjoK47e8BjwF/l1J29H90n3PVoSIjJNztFqv3NeCPdAnfBpT33I9QEbh37eXYG1CBUgullDV7u47udtNoRge+sI/ylnLCRpiIEWFb6zbaQ+14Q148YU986b7tDXnxRXwDdmFlOjIpTCmkMLWQwpRCilKLKEgtIMW6V4fcA4Y37KXWW0udr446bx11vrr4djDaJWgWk4UMewbesBd/pP82gdVkJc2Whtvm7rm2u0m3pVOWUcbC8QuHHDtvv7vdYt5ui1GC82XAhhKdPcAKVGqFz4dg22fArH7KZwL79J6TUj4thPgXMBUISSm3CyHuA3bvTXh6n2bQ1mo0mhGNy+pietZ0trZsBWB61uAiMhvSwBv29hGm1mArNZ4aqr3VvLn7TUJGV+jKdHt6XIwKU7vEqTPaQiJIKQkZIQKRAP6IP772hD1xkan11VLnrcMT9sSPEwhynbnkpeQxNXMq41LGke/KZ1zKOLIcWfFWW9gI0xHqoCPUQXuwnfZQu/rcbd0WamOPZw/twXYiylcLgOe+8hyT0iclfE97I5FutyogFyU4AeDvKM+2V/czntu/gV8LISZJKXcACCFKgeOBHw/mBFLKKLA5dmwhcB5q/GlAhBCWWL1dUko9HVqjGUM4LA5mZM1gS/OWHi2BvWESJtw2twpOOkCDxpAGTf4mqr3VVHu6lrer3u5xnTRbWlyIsp3ZcYeB7qLS/XMgEsAf9WP0TnPQjUx7JuNc4zh63NFdAuMaR64rd1BeaVaTlSxHFlmOrH3WlVISiAZoD7bjsrooSi3a5zGJksg8HwP4L0pwnpRStu/jkMGeNwX4GDVO9DNUK+R2wA0cLqX0xOpNALYDt0kpb4uVWYFfoSaKtqNaUDfF6p3SGV1bCPFN4H+Al1BBUMcB3wO+CHxTStl3xLsXuttNoxl9hKNhKtsrCRkhIkaEiBHZ6wN+qBjSoCXQQrWnmipvFdWe6nhrqbO7y262d3nHmZV3XG9vOafZ2ceLzmV1kevMxWFxJN3uwVCUMp5Cd/6Qj0+Gt9uMTrflZCKl9AohTgZ+g+q+E8BrwA87hSeGQLlIdx/5k6hoCBcAGaguwEeAX/ZK61AB5KFaQ1mAF1gHLJJSvpLse9JoNCMDq9nKlMwpPcoMacSFKCIj8c9RI0rEiBCW4R5lYSNMVEb3eh2TMJHtzCbbmc1huYfFyztbEJ2TXA8GnfOSHGYlblaTlahU9xGOqnvrHA8b6N6klLQHwjR6QrhEmMIDkKkikXw+SReebufeBXxtH3UqoadTRmwC6ZmDOP9a4OT9MFGj0YwRTMKkoh6Y95H9sxuhaCjeVda9u2xfoiSEwGlx7q/JfTALcw+BsZvtOC1O7GZ7wiInpYyLUSgaorbdQ1WbF28wiI1IQn+nREjY1TrW1fV1YCFqUimo8aA3gH9KKftJh6fRaDSjl06xSren9ygfqigNhEmYlMu3MGMymbAIS5fQxESmszWTLIQQmIWVJl+U6lZJKOLEZXLiimlmmu3AePQlJD5CiKNRmUwnQB/X8MuBXwghviGl3JAk+zQajWbEMpAohaNhfBEf/oifcDSMECIeN87cbU5Rj7UwH/SwPZGoQW17gNq2AOFo3/H/8roOVqyt5DfnzcFuSW63YSKu1sWoqNBZwC7gr6gAnqAymn4LNbnzFSHEHCllVVIt1Wg0mlGC1Wwl3ZzeR5RGCqGIQU2bn7r2IFGjp+hIKflkTxvPfVzF5poO0p1WttV7mFWY3HtJpOXzY5Tw3Atc37t7TQhxC2pA/5pY3av7nEGj0Wg0w0YgHKW61U9DR5BemoMhJR9UNvPcR9VUNHrJSrFx0XETuOz4iZTmJL/rLRHxWYRq6Vzb37weKWVECHEdKt3CYrT4aDQazYjAF4pQ3eqn0ROi99M7Yhi8u62Jf39cRXVrgPw0B1ecMIkvTsnBajbhtB0YL71ExKcIeHZvE0qllIYQ4n3gnP22TKPRaDT7RdSQ7Gjw0OgJ9dkXihis3lrPC59U0+gJMSHLxQ9OLuMLE7MxmQ782FMi4uOnZ5Togcji4AUW1RyitLe3U19fTzisnSs1XVitVvLy8khLSxtuU4adUMRga20HnmCkR7kvFGHVpjpe2lhLuz/M1HGpXHb8ROaMz+jj8OCwmnCNgJbPJ8BCIcR0KeWW/ioIIaahXLDX9rdfo0kG7e3t1NXVUVRUhNM5shJ7aYYPKSV+v5+qKuXrdCgLUCAcZXNNO4FwVzSHdn+YlzfWsnJTLb5QlMOL0/nKnCKm57v7/IZcNjOFGU5yUm0H7PeViPg8DJwIvB5LX/14t/A1VlQyudtRydoeTLahGk0n9fX1FBUV4XINLcquZmwihMDlclFUVER1dfUhKz4dgTBbazvirtNNniAvfFrD65vrCUcNjpmYxdlHFDIpt2/wU7fDQmGGk6yUAzOxtDuJRDhYIYRYhEqf/SBwvxCiBhXiphAV9kYAT0gp/3ogjNVoAMLhME5n8meNa8YGTqfzkO2ObfGG+LzeE3efXvlZLX9ZuxMkfHFKDmcdUUhRRt/fTrrTSlGGk3RX8iav7ouEJplKKb8lhHgXuA41p6e42+4dwD1SyuVJtE+j6Rfd1aYZiEP1u1HfHmBHoxcpldv03z/Yzb8/ruaokgwumT+RXHffBNNZKTYKMxy4HQdPdDpJOLxOTFyWCyGK6BZeR08q1Wg0muFhd7OPPS3KzysSNfjTWzt4d1sjX5oxjkvnl/bwXhMCclJtFGY4cdn2O5n1kBnylWNiowVHo9FohgkpJTsavdS3q1xCvlCEe1aV81l1O+fPG8/ZRxTGW4ImAbluO4UZThzWgxNhe28MLjE5IISwCyFKhBADBtcWQrhjdQ78aJVGMwZZs2YN559/PsXFxdhsNtLS0pg3bx4333wzNTV7zfQ+rPz85z8/ZLu7houoIdla1xEXnmZviFuf38SWmg6+u3Ay/zOnSAUNNQkKMxwcWZLJpNzUESE8kID4oMLmVABH76XO0bE6398fozSaQ5G7776b448/noaGBn7xi1/w6quv8uSTT3L66afzwAMPcNlllw23iQNy+eWXs2bNmuE245AhHDXYXNNOi1c5Vuxu9rH0uY00dAS5YdE0TpiSC0CK3cwR49OZkJ2CzZLI4/7Ak0i329nAbinlGwNVkFK+IYTYg8oaes9+2qbRHDKsXr2a66+/nmuuuYbf/OY3PfYtXryYm266iaeeemqYrNs3xcXFFBcX77uiZr/pPYdnU00796zcitViYulZMynNVnHYMlxWpo5zYz4I0QqGQiJSOBnYPIh6m4CyoZmj0RyaLFu2jJycHJYtW9bv/pSUFC655JL49i233MJRRx1FWloaOTk5nHzyyaxd23Nu92OPPYYQgsrKyh7l/XWR/e53v2PGjBk4nU4yMzOZO3cuzz77bHz/K6+8wvz580lPTyc1NZVp06Zx22237fWcf/jDHzjuuOPIysoiIyODY489lhdffLFHncrKSoQQ3H///SxdupSCggIyMjI466yz2LNnzz7/bocanmCEz6rb4sKzZnsTd7y0mXSXldvOnh0Xnrw0O9PzR67wQGItnyygeRD1moHsoZmj0Rx6RCIR3nzzTb761a9isw1uuLSqqoprr72W4uJivF4vjz/+OCeeeCLr16/nsMMO2/cJuvHXv/6V6667jqVLl3LCCSfg9/v55JNPaG5WP/cdO3Zw9tln8/Wvf52lS5dis9n4/PPP2bFjx17PW1lZyeWXX05paSmRSITnn3+eM888k5dffplFixb1qHvHHXcwf/58HnnkEerr67nuuuu48MILeeONNxK6l7FMqy9EeV3XHJ6XPq3h8bU7mTrOzY9Om0aqQz3OizOdjM8a+ROwExGfRgbXoikDWodmjkYzNG59/jM2VbcPqw0zC9O45axZCR/X1NREIBCgpKSkz75IpGdcLotF/WQfeuiheFk0GmXRokXMmjWLhx56iN/97ncJXX/NmjUcfvjhLF26NF62ePHi+OcNGzYQCoW477774lEDTj5531npf/3rX8c/G4bBKaecQnl5Offdd18f8SktLeWJJ56Ibzc0NHD99ddTXV1NYWFhQvczFqnvCLCjoWsOz1/X7uSljbUcMzGL7y0sw2YxIQRMykkhL80x3OYOikS63f4LzBVCzBuoQmzfXOD9/TVMoznUqa2txWq19lg6xejVV1/lpJNOIjs7G4vFgtVqpby8nK1btyZ8nXnz5vHRRx9x9dVX8+qrr+Lz+XrsnzNnDlarlfPPP59//vOf1NfXD+q869ev58wzz2TcuHFxG1etWtWvjd3FDoi33nbt2pXw/Yw19rT42F6vhCcUMfj965/z0sZaFs3K55qTp2CzmDCbBNPz3aNGeCCxls/9wFeAfwkhLpFSruq+UwhxKvBobPNPSbJPoxkUQ2lxjBSys7NxOBx9HrQ5OTl88MEHADzwwAM8+KAKmbhhwwYWL17M6aefzsMPP0xBQQFms5nLL7+cQCCQ8PUvuugiAoEADz/8MMuXL8dqtbJ48WLuueceSktLKSsr45VXXmHZsmUsWbKEYDDIMcccw7Jly1iwYEG/59y9ezennHIKM2fO5Pe//z0lJSVYLBZuvvlmNm/uO3ScldUzYL7drmbjD+V+xhLdJ496ghHuXrmVLbUdfOsLJXz5sAKEENgsgmn5aaTah2/C6FBIJLbbK0KI+4Ergf/EvNo6X2GmoULtCOBBKeVLSbdUoxmjWCwWTjzxRFatWkUoFIqP+1gsFubOnQvACy+8EK//9NNPY7FYeOaZZ7Bau8KitLS0kJGREd92ONRbcCjUM5dLU1NTj20hBFdeeSVXXnklLS0trFy5kuuuu47zzjuP//73vwCcdNJJnHTSSQSDQd59912WLl3Kl7/8ZSorK8nJyelzT//5z39oa2vjH//4Rw8vuN6tKs3AdBeeJk+QO/+zhZq2AN8/qYzjy9Tf3GkzMz3fPWLm7iRCQo7fUsqrgP9FORWMB74UW8YDTcB1Usork22kRjPWueGGG2hsbOTGG2/cZ12fz4fZbO7hXfb666/3aTlNmDABgI0bN8bLIpEIK1euHPDcmZmZnHfeeZx77rk9juvEbrdz8sknc8MNN+D1eqmoqBjQRqCHOJaXl/Puu+/u8/40sKupS3h2Nnm5+bmNNHtD3HTG9LjwuB0WZhemjUrhgaHFdvutEOL3qLGdCaio1ruA9VLKyF4P1mg0/XLKKadw55138uMf/5hPPvmEiy66iIkTJxIIBCgvL+fJJ58kJSUFIQSLFi3it7/9LZdccgmXXnop5eXl3H777RQVFfU457x585g8eTLXX389hmFgt9tZvnw5wWCwR70rrrgCt9vNcccdR15eHuXl5axYsYLTTjsNgD/96U+89dZbLF68mPHjx9PY2Mgdd9xBYWEhs2fP7vd+vvSlL2GxWLjooou47rrrqKmp4ZZbbqGkpATDMPo9RqPY2eSlulV1N26uaeeuV7bitJm55axZlMS82LJTbZTlph6UjKMHDCmlXgaxHH300VIzMti0adNwm3DAeOedd+Q3vvENWVhYKK1Wq3S73XLu3Lly6dKlsrq6Ol7v3nvvlaWlpdLhcMi5c+fKVatWyQULFsgFCxb0ON/GjRvlggULZEpKihw/fry8++675S233CLVT1/x2GOPyQULFsjc3Fxps9lkaWmp/OEPfyjb2tqklFK+99578uyzz5bFxcXSZrPJ/Px8+fWvf11u2bIlfo7e55RSyr///e9y2rRp0m63y5kzZ8q//e1v8uKLL5YTJkyI16moqJCAfPDBB3scu3r1agnI1atXD+nvOFq/IxUNHvnetkb53rZG+cCb2+WUn7wk59/xmnz+o6p4eUWDZ7jNTAhgneznmSrUPs2+mDt3rly3bt1wm6EBNm/ezIwZM4bbDM0IZjR+RyoavdS2qRbPR7tbuWfVVvLTnfx08QzSnVaEgAnZLgrSR1cuKyHEeinl3N7lo8s9QqPRaMYg3YVn/c4WfvtqOcWZTn6yeAZuhxWTgLK8VLJT++bkGa1o8dFoNJphZEeDh7pYZOr/VjTx+9e2UZrj4sdnzCDVbsFqFkzNd5M2DAnfDiRafDQajWaY2N7giadEeHdbI8vf2EZZXio3LpqOy2bBZjExsyANp210erTtDS0+Go1Gc5CRUrK9wUtDhxKet8ob+NNb25me7+aG06fjsJqxW5XwjFZX6n2hxUej0WgOIr2FZ/WWeh58ewezitL50WlTsVvMOKwmZhamYbeMTeEBLT4ajUZz0FDC46GhQ0WdWLmplkffreSI4nT+99Rp2CwmXDYzMwrSRlzyt2SjxUej0WgOAlJKttV7aPQo4Xnp0xpWrN3J0RMyueaUKVjNJlLsSnis5rEtPLAX8RFCnLg/J5ZSvrU/x2s0Gs1YQUrJ5/UemmLC8++PqvjbB7s5ZmIWV59UhsVswu2wMD3fjeUQEB7Ye8vnDVTonKEg93FujUajOSToLTzPbNjDU+v3MH9yNt9dWIbZJEhzWpienzaiM48mm70JxFsMXXw0Go1GA2yLCY+UkqfW7+HZD6s4cUoOV544GZNJkOGyMnXcyE55fSAYsH0npVwopTxpqMvBvAmNZqywZs0azj33XAoLC7HZbGRnZ3Pqqafy5z//mWg0yhtvvIEQIr5YLBZKSkr47ne/S0tLS49zVVZWxus98MADfa7l9Xpxu90IIfjZz34WL3/ssccQQlBZWXmgb3dAysvLueaaazj88MNJTU2loKCAs88+m48//njYbBoK9e0BGmPC88T7u3j2wypOmpbHlQuU8GSmWJl2CAoPJJhSQaPRHDh++9vfcvzxx9Pc3MyyZct49dVXeeSRR5g6dSpXXXVVj5w+9957L2vWrGHlypUsWbKEBx54gIsuuqjf87rdblasWNGn/Omnn+6RlqGTL3/5y6xZs4aCgoLk3VyCrFy5ktWrV3PxxRfz/PPPs3z5choaGjj22GNZv379sNmVCIFwlMomH1JK/rJ2Jy98UsOpM8dx+QkTMQlBdqqNaePcozsy9f7QX7RRveio1iOZ0RqxeG+8+eabUgghr7766n73b9u2TX788cfxaM+rVq3qsf/yyy+XgKypqYmXdUaMvvjii6UQQu7YsaPHMaeccoq85JJLJCB/+tOfJv+m9oOGhgZpGEaPstbWVpmRkSGXLFmyz+OH+ztiGIb8dE+rfOfzBvntx96XE258QV61Yp189/MG+d62Rvl5XUef+xurMEBU6yG3fIQQ6UKI8UKIkv6WJGmjRnNIsGzZMrKysvjVr37V7/7Jkydz+OGHD3j8UUcdBdAnoRzAF7/4RSZOnMjjjz8eL9uzZw+rV6/ut7XUX7dbaWkpF154IU8++SQzZswgJSWFuXPn8s477/Q4duHChSxcuLDPOUtLS7nkkkvi27W1tVx88cUUFhZit9spKCjgzDPPpL6+HlApxHu3ytLT05k6dSpVVVUD/h1GCnta/HQEIjyzoYpXN9dz9hGFXHjsBIQQjEuzU5aX2m+r81AiIfERQmQJIf4ohKhFZTOtBCr6WXYk2U6NZswSjUZZvXo1p512Wjz1daJUVlZiNpspLS3td/+SJUt6iM/jjz9OcXFxv0IxEG+//TZ33303t99+O3//+9+JRqOceeaZtLa2JmzvkiVLWLNmDXfddRerVq3i3nvvpbi4eK9ptpubm9m4ceOIT5XQHghT1epnS207z3y4hxOm5HD+vPEIIShIdzApN3W4TRwRDNodWgiRCfwXmAREAT/gAmqAfEDQldVUozm4vPxjqP10eG3IPwzOuDPhwxobG/H7/fG014PBMAwikQh+v5/XXnuN++67jx/+8Ifk5eX1W/+iiy7i1ltvZe3atRx77LGsWLGCCy+8MKG37/b2dj766CMyMzMByM/PZ968ebz00ktccMEFgz4PKMeKX/7yl3zrW9+Kl33jG9/Y6zFXX301Ukp++MMfJnStg0nUkGyv99ARiPDH1dvIc9u5dP5EhBAUZzoZH8tEqklsLs6NwGTgEeBq4D5giZSySAjhAr4F/BJ4R0q5JOmWajSaOKeffnqP7S9/+cvcddddA9afNGkSxx9/PCtWrMBisbBp0yaeeeaZhK553HHHxYUH4LDDDgP67+rbF/PmzeOuu+5CSsnJJ5/M7Nmz9yqEd9xxB0888QQPP/wwZWVlCV/vYFHR6MUfivLwOzto8Ya59X9m4bSZGZ/lpDhTC093EhGfs4AG4HtSyqAQIj4HSErpAx4UQnwIrBVCrJFSLk+yrRrNwAyhxTFSyM7Oxul0snPnzkEf88c//vH/t3ff8VWV9wPHP9/svclgJGGEsJcRQa0iS6uIVFqVUpFWrdZfbV2tW3FUq7VaV6ugLYqzD5uvQwAAIABJREFUWhGcKMhQFBkqyAwjREgge+/x/P44J3gJWZeMm/F9v155nXufM/K9Hsz3Ps95BuPHj6egoIBFixbx5ptv8sADD3DPPfc0es68efO44447qKmpYfz48SQmJjoVZ1hY2HHvvb2thc3Ky8udug7Am2++yX333cejjz7KDTfcQExMDNdeey133XUXbm7HPw147rnnuOOOO3jwwQf5zW9+4/Tv6ig5xRVkFVWwJjmLDQdymXNqPwb2CiA23I8+IV1r9dGO4Mwzn3isXgsV9nsDICLHpl01xmwGvgCubKsAleruPDw8mDRpEp9++ikVFRXNnwAMHjyYpKQkpkyZwuuvv86kSZN4+OGHOXToUKPnXHLJJZSUlLBo0aJGu2W3lo+PD5WVlSeU5+bmHvc+MjKSZ599lrS0NHbv3s38+fO59957ef755487bsmSJVx33XXcfPPN3Hnnne0Sc1uoqK4hJbuE9PwyXvryIMN7BzFjdG/CA7w08TTCmeRTAxQ6vC+xtxH1jksHEloTlFI9zW233UZOTg5//vOfG9yfkpLCtm3bGtwnIjzxxBNUVFTw1782XgMMCQnh9ttvZ+bMmVx22WVtEnd9cXFxJCcnH5eA1q1bR1FRUaPnJCYm8tBDDxEaGsr27duPlS9dupRf//rXXHXVVTz22GPtEm9b2Z9ZQmllDc+s3oenuxvXTRqEr5c7AyL8XR1ap+VMs1s60M/h/UF7ewrwoUP5UKBlX9+UUgCcddZZPP7449x0003s3LmT+fPnExsbS15eHqtWreKFF17gtddeIzg4uMHzx4wZw+zZs3nxxRe588476d27d4PHNdUs1xYuu+wyFi5cyG9+8xvmz59PSkoKjz/++HFxFxQUMHXqVObOncuQIUPw9PRk2bJl5OXlMX36dMBKWHPmzGH06NHMnz+fDRs2HDvf29ubsWPHtuvncMaRgjIKyqr47+ZDpGSXcPO0wYQHeJEQGdBjJgk9Gc4kn2+A6SLiboypAVZh9XD7q4ikAIeB64DRwGdtHqlS3dwNN9zA+PHjeeKJJ7jlllvIzs4mMDCQpKQknn/+eS688ELWrWt8svj777+fd955h0ceeYQnn3yyAyP/0TnnnMNzzz3HY489xv/+9z/Gjh3LK6+8wuzZs48d4+Pjw7hx41i0aBGpqam4ubmRmJjIq6++ykUXXQTAZ599RkVFBd988w1nnHHGcb8jLi7OpVP/OCqtrOaHnFK2Hc4/NoNBUnwY/cL8CPTxdHV4nZpYA1BbcKDI5cBLwIXGmA/ssmVYHRHqX+Qc082WVEhKSjKbN292dRgK2LVrV6cf66FcqyP+jdTWGranF5CeX86t/9tGoI8Hf5k1ksggb4bGBLXr7+5KRGSLMSapfrkzdcLXsZrd1jqU/RJ4FsgEqoHtwCXdLfEopVR9h/JKKS6v5vm1+ymtrOb6yQkE+LgzUAeRtkiLm92MMdVAWr2yEqwxP9e3cVxKKdVpFZRWkZ5fzoodGXx7KJ/5p8cTF+7HoF6B3X7567ai/5WUUsoJ1TW17MsqJjWnhNc2pjIuNoTpw6LoE+JLsJ8+52mpFicfEQkVkbNEpOFuNNYxfexjQtomPKWU6lxSsksoKq/i6c/24e/lwTVnDSTI15O+oTqexxnO1Hz+CKwGmlrkI9o+5vetCUoppTqjzCJrcbhXNqSSll/G7yYNtLpVR+ks1c5yJvlcAOwzxjS6kpO9bz8ww5kg7KUZ3haRAhEpFJF3Wrosg72Ew0si8oOIlIlIsog8KCInjO6ya2//sI+tEJHDIrLYmViVUj1TeVUNqTmlbDqYy8pdmcwYFcOoviEM6BWAt4d78xdQx3FmnE88sKG5g4A9wPiWXtSelPQzrIGpV2B1234QWC0io+xODY2d6w+sBDyBu7Fm1D4VuA9rloVLHY4NxZr6xwB3YQ2S7Q0cP4hAKaXqMcawL7OYzMIKFq47QP8Ify5N6kdMsA9h/l6uDq9Lcib5BAKNz5HxoyKg4WHYDbsaa5mGRGPMPgAR2QbsBa4BHm/i3DOwksy5xphP7LLVIhIG3CIifvakpwAPAwHASGOM4zRBbzgRq1KqB0rLL6OgtIpnV++jqqaW688ZRIifF7G6RMJJc6bZ7SgwogXHDQeynbjuTGBDXeIBMMakAOuBi5o5t+4rR2G98nyszyZwrIY0D3ihXuJRSqkmFVdUczivjPe2pbPzSCHzT4+nb5gfCVEBuLnpc56T5UzyWQ8MF5HzGztARH4KjMRq3mqp4ViDU+vbAQxr5tyVWDWkR0RkmIgEiMhkrM4Rzzk02Z0C+AIZ9rOlMhEpFpF3RaS/E7EqpXqQ2lqruW1vRjFvbT7MxAHhnD24F/0j/PHx1Oc8reFM8qmbLOp1EblaRLzrdoiIt4hcjTULggGecuK6YUBeA+W5QGgD5ccYY8qBM7E+xw6sJr9VwPsc3+Ournv4Y1izc88EfguMBdaISKAT8SrV5hYvXoyIICIkJyefsH/t2rXH9q9cuRKABQsWICJUV1c3et358+cfO6/+z6xZs9rt83QXafll5BZX8vRnewn19+TKM/sTGeRDr0Dv5k9WTXJmhoONInIX8BfgOeBpEalbwrAfVhOYAPcYY75s80gbICI+wJtAJHA5VoeD8cA9WNP9/M4+tC7JHgAuM/aEdiKyH6sTxa+wVmatf/3fYiUpYmNb1PlOqVYJDAxkyZIlPPDAA8eVv/TSSwQGBja5NEFjevXqxfLly08or784nDpeSUU1afll/Gd9ClnFFdw7YzgRgd7012US2oQzHQ4wxjwsIruBe4FRgON6ttuA+4wxS52MIY+GaziN1YgcXQlMAgYZY/bbZetEpABYKCLPGWO2Ajn2vlXGYSZVY8zXIlKIVQM6gTFmIbAQrIlFW/h5lDppF198Ma+88gr333//sXEjZWVlvP3228yePZvFixc7fU0vLy8mTJjQxpF2b8YYDmSVsH5fNp/vy2b2uL4MjQlkcFQA7vqcp004Pb2OMWapMWYM1mDTCcBpQIwxZsxJJB6wmsuGN1A+DNjZzLkjgTyHxFNno72tm9Z2RzPXqW1mv1Id4vLLLyc1NZUvvvjxsenSpUupra09blkC1b6OFJRzOK+U/6w/SEJkAD8b24f4CH/8vJz6vq6acNJzuxljMowxG40xm4wxGa2IYTkwQUQG1BWISDxWN+oT2wqOdxQIFZFB9cpPs7dpdqyHgc3ANHEYhiwiE4EgYFMr4leqzcTFxXHWWWexZMmSY2Uvv/wyP/vZzwgIOPnZkqurq0/4aelyKj1NeVUNh3JLefGLFCqqa7jm7IFEBnkTFeTj6tC6lc6QxhdhdQ5YZj9TMsADwCHg2ILuIhKHNXvC/caY++3ixcBNwIci8hesZz5JWANOt2D10KtzG7ACeFtEXgB6YT2/2g281l4fTnWMRzY+wu7c3S6NYUjYEG4df2urrzNv3jxuvvlmnnrqKfLy8li5ciUfffTRSV8vLS0NT88TJ7z829/+xi233NKaULulA1klfL43m82pecw9LZYBvfx1Oex20GjyEZF59sulxpgih/ctYox5uYXHldjdo58AlmB1WlgF3GCMKXYMCXDHobZmjDkoIhOABVizIkRgJa2FwF+MMbUOx64SkQuB+4GlQAnwAfAnY0yZM59Nqfb0i1/8gt///ve89957pKamEh0dzZQpU5pcxbQpkZGRfPDBByeU9+vXr7WhdjuZheWk5pSw+Eurue2CkTG6HHY7aarmsxirFrIBqwtz3fuWalHyATDG/AA02aBtjDmIPWi0XvlO4JIW/p6PgJP/Cqk6rbaocXQWgYGBzJo1iyVLlnDw4EHmzp2Lm9vJ//Hz9PQkKemEhSRVPZXVtRzMKTnW3Hbt2QN1Oex21FTyeRkr2RTUe6+Uamfz5s3jggsuoLa2ltdff93V4fQIB3NKWJv8Y3Pb4KhAXSahHTWafIwx85t6r5RqP9OmTeOSSy4hJCSE4cMb6gyq2lJOcQX7MotZ/GUKg6MCmDEyhkGRukxCe2pxhwMRGQXUGmMamgpHKdWG3N3dW1zjeeedd05olouJieGMM6wJ2ysrK9mw4cQJ6f38/Bg1alTrg+3iqmtqScku5sUvUqisruXaswYS38sfXy+dPqc9OdPb7Tvgc+DsdopFKXUSLr300hPKLrjgAt5//30AsrKymDhx4gnHDB8+nO3b9btkam4pn+3OYktqHr86LY4hMUHEBGtzW3uTlvb1F5Fc4ENjzK/aN6TOKSkpyWzevNnVYShg165dDB06tPkDVY/V0n8jBaVVrN+fzZ/e3krfED/uu2g4Y2NDdHG4NiQiW4wxJ/R4caYLzXfAwLYLSSmlXKem1rA/q4gXPrea2645awADdVXSDuNM8nkKOE1EzmuvYJRSqqMczitl5a5Mvvkhj8tOjWVE32CdrboDOfPM5xvgGayZCP6NNVAzFWhwgKY9dkcppTqd4opqdqQX8tKXB0mMCuTC0TE6W3UHcyb5pNhbwVpm4LdNHGucvLZSSnUIYwz7Mop44fMDVNUYrjl7AAmRgXjqLAYdypkEcQgdZKqU6uLS8stYsSODb37I5/IJcYzpF0Kov5erw+pxnFlMLr4d41DtqLqmlupaQ60x1NQaamuxXhur7Nh7+5haYz2MNfYxXu5u9A7x7VTLBhtjdACgalBTPXjLKmv4/nABL39lNbfNGtubuHBtbnMFbRrr5jILy0nJLqG2lXXWzKIKIgK86BPi5/LBd56enpSVleHn5+fSOFTnVFZW1uAs3sYY9mUWsXCd1dz2u0kDSYgK1MXhXESTTzdVW2tIySkhs7DihH3GGKpqDBXVNVRW11JZXUtFjb2trj1WXmHv8/F0Z8KAMLKKKskurnR5EoqMjCQtLY0+ffrg6+urNSAFWP+uy8rKSEtLIyoq6oT9GYUVfPD9Ub49lM+8iXGMiw0lSCcNdRlNPt1QRXUNyUeLKa6oZv2+bJZtTaessvpYMqmsrnX64d1rX6fy05ExTB8WhTGQXVxJuL8XfUJ9O3x1x6CgIADS09Opqqrq0N+tOjdPT0+ioqKO/RupU15Vw3eH8nj5q4MMiQ7k4nF96Bemsxi4UlPr+dRgdTAYZoxJtt+3lDHGaGJzgYLSKvZmFlFSUcPLXx1k1e5M4sL9GBYThJeHO94ebnh7uOF1bOt+7L2Xu+M+92PHpOWXsXxrOm9uOsTy79KZPjyKn46I+TEJBXjRt4OTUFBQ0Al/YJRqzIGsYp5be4DqGsPvzh7I4KhArTG7WFN/LYTj189x5k7pXXWBtPwyDuWWkp5fxpMr95KaW8qFo2K45NR+eLRiPZggX0+GxgSRkl3Csu/SWP5dOh9+f4RzEiOZMao3ADnFlYT5W0nI31u/d6jOI6uoguVb0/nuUD5XTIzj1P5hHV5bVydqakkFt6beq87DmiakmJziSr7an8Oizw/g7ib8aXoi4+JC8fd2x8PNDXc3wU1ARI69dhPBzX7tLnLcvrrXVdW1HCkop3+EPzdMHUx6fhnvbU1n1a5MVu3K5IxB4cwc3QeA3BIrCfUJ9SVAk5BysYrqGr5JzeXlr1IZEh3Iz5P6EhPs4+qwFPrMp8srq6xhT0YRhWVVvLIhlU92ZpAQGcAfpiQQGehNXLg/0a39n80bQv29KK6o5kh+GSJwzdkD+fkpfXn/+yN8tiuTz/dmc2p8GDPH9AYCyC2pJNTfkz4hvroSpHKZ/ZnF/GvtAWpqDf93ziASIrW5rbPQ5NOF5RRXsD+rxGpmW7WXlOwSLhgZw2Xj+xHk40lCVECbNi8EeHuQEBVIv6oajhaU4+4mXDExnp+N6cPHO46yYsdRNh7MZVSfYC4a24eh0YHklVQRE+xDvE5dojpYRmE5735b19wWz2kDwjrVWLWe7qT+MonIRGAS0NcuSgPWGGO+bKO4VBOMMRzKLSMtv4yNKbk8v24/Atw8bTBJ8WFEBXkTH+6PWzuNX/DxdCc+wp++ob5kFFVwtKCMS5L6MWNUDCt3ZfLh90d44P2dJEQGMGtMH8bGhlBdW8vAXroypOoY5VU1fLkvh5e+Osjw3kFcNr4fkYHa3NaZOJV8RCQBWAKcWldkb429fzMwzxizp80iVMepqqllb0YxOcUVvPr1D3y84ygDe/nzxykJ9A7xZUCvAMI6aKoQD3c3+oT4EhPkQ3ZxBekF5cwc3ZvzhkezJjmT97ce4W+f7GFQZAA3TRtMVY1hsA7qU+3MGMOuI4X8Y1Uynu5u3DA1gUGRAa4OS9XjzDLa/YB1QBRQCLwHHLR3xwMzsJLSWhEZr7Nat73iimr2HC0iLa+UJ1ftZX9WCeeNiGbu+FjCArwYFOmatUjc3ITIIB8ig3zIK6kkLb+M6cOimTwkki/2ZrP4y4Pcs2w7t543hJpaQ2K0TuKo2k9afhmL1h0gNaeUW6YnkhQXpv/eOiFnaj4PYCWeJcAfjDEFjjtFJAhrzZ95wP3A/DaKUfHjNDkbU3J5bu1+ag3cMDWBCQPC6RvqS5+QzjHSP9Tfi1B/L4rKqzhSUM45QyKJC/fn0Y93s2D5Dm6ankhNrWFITKAu2qXaXHFFNe9tTefD7UeZPiyKC0fH6KShnZQzy2gfBcqBQcaY6kaO8QD2A97GmOg2i7ITcNUy2sYYUrJLSMsv4/WNh/jw+yP0j7Ca2WLD/UiIDOjUvcnKq2pIyS5hb0Yxj3y8m4zCcq49eyCTh0YyNDrI5fPEqe6jttawNjmLP77xLcF+Xjx56RjGxoZ0ii9lPVljy2g7U/MJBj5rLPEAGGOqReRL4KKTiFHVU1VTS3JGEQeySnhq1V72ZhYzbVgUvzotjt4hPvSP8Mejkzcn+Hi6MyQ6EF9PdxbMHM7jn+7hmdX7yC6uYPa4PgztHazjgVSbOJhTwhOfJlNeVct9UxMY3idIE08n5uxicqEtOC4Ya4VT1QolFdXsyShiU0ouT3+2j5pawx8mD+LMhF7ER/h1qZ47IkJ8hD9+Xu7ccf5Qnluznzc2HSKrqIKrfjKAYTFBBPt13tqb6vwKSqt48YsUtqUVcNWZ/TlnSKQ263ZyziSfl4F7RSSxsd5sIjIEmIz1zEedpJziCvZmFrP0mzT+u/kQfUJ9uWnqYBKiAhkUGdBlm6oig3zw9nTnj1MTiAj0Ztl36eSUVHLj1ARG9AkmPMDb1SGqLqi6ppYVO4/yxqZDjI8PY97EOEL89DlPZ+dM8vkbkASsEZH7gVeNMYUAIhIIzAXuAd4H/trWgfYUh3JL2XO0iH+t3c+W1DxOHxjO1T8ZwIBe/vQL9Wu3sTsdJdjXk1F9Q/DycCciwJt/r09hwXs7ufW8RMbGhhIV1HVqdKpz2HWkkMc/SSbY15Obpw+mX5iu89QVOJN89trbKOAZ4BkRybfLQhyOOwXYV6+t1RhjBp50lD1A3fxs3/6QzxOfJpNVVMEVE+OYMSqGQZGB3arHjo+nOyN6B+Ht4UaYvxdPrdrLXe9aXbEnDAjXPx6qxXKKK3jsk2Qyisq5b+ZwxsaG6nOeLsKZ5BPv8Lru7jb0DCiugbJWrqPZvZVX1ZCcUcQnOzJYuO4Afl7u3DVjKKfGh5EQ5ZqxO+3Nw92NIdGB+Hi6E+LryaMr9hzrij15SCT9dToe1YzK6lqWfJXK2uQsZo/rw0Vj+uDl0bk74KgfOZN8+rdbFD1YQVkVO9MLWPxlKit2HGVIdCB/nJrAiN7B9A3tHGN32ouI0D/CH19PdwJ9PXjkoz08/OEu8koquWhMbwZF6nQ8qnFf7s/m+XUHSIwK5Popgwj21U4rXUmLk48xRnuwtbGMwnK2pObxj5XJJGcUc/6IaK44PY4h0cE9qvdXdLAPZydEEujjyaMf7+aZ1fvIKa7g8olxJEYH6XQ86gRpeaX85YNdiMDt5w8hPlynz+lqdICFC9QNHF29J4unVu2lvKqGP0xO4NwRUSREBvbIpoNgP08m9A/nvpnD+cfKvby+6RBZxRVcPzmBYb2DdHoUdUx5VQ2PfZLM3sxibp4+mDMGRbg6JHUSNPl0sKqaWvYcLeSNjYd5bWMq0UE+3D1jKKf1D+/2zWzN8fVyZ2xsKLefP4RF61JYvjWdnOJKbvvpEMbFhWoCUhhjeOebw7z7bRrnJPZi/unx+u+ii3LqronIIBFZJCL7RKRURGoa+Wl0FoSerLSymo0Hc3nwg1288nUqSXFhPPLzkUwbFk2/ML8enXjqeLq7MaJ3MDdMTeDKM/vz3eF8bnvne77an0Nlda2rw1MutvNIIX9bsYeYYB/unjGsU08tpZrmzKzWScBngD8/9nZr9PDWBNUd5ZZUsnZPFo99sof0gjJ+OT6WuafFkhDVM5vZmiIiDOgVwFU/6U+Yvxf/WJnMbf/bxn0XjeCswRHdsvefal5xeRX3LNtBUXk1D84ayYBe+pynK3Pmr96jQADwX2AcEGiMcWvsp12i7aIO55Xy8lcHuX3pNorKq7jz/KFcd85AhvcJ1sTThJhgX+aeFsvdM4aRV1rFbf/bxqc7MqiornF1aKqD1dYanv5sH1tS85g3MY5zh0e5OiTVSs488zkN2GWMmdNewXRH1TW1PPrxHpZvTWdQZAB/PjeRCQPDCdLmghYJ8fNi9ri++Hq688AHu7h96feUV9cwY1RvXRK5B1mTnMm/16cwpl8IN04b3Okn1FXNcyb5lAFb2yuQ7sgYw7WvbGHlrkymDYvi+smDGBqjPbec5e/twcwxvQnw9uDuZdu5+90dlFXW8IukfpqAeoCMwnLuWbYDPy8PHr54pD7n6SacST4bgQHtFUh3JCLMGtuHIdFB/PK0WHqH+Lo6pC7L28OdacOi8Pfy4Pal33P/+zspqahh/hnxmoC6scrqWu5Ztp3DeWU8fPEIhsYEuTok1Uac+Qr+F2CsiFzcXsF0RzNG9ebaSQM18bQBD3c3zkyI4Kk5Y4gN8+PRFbt5dvU+yir1GVB3VFNreOHzA6zYkcGsMb25JCnW1SGpNtTi5GOMWQ9cBiwUkSUi8isRmSQiZzX040wQItJPRN4WkQIRKRSRd0Skxf/SRGSoiLwlItkiUiYie0Tkj/WOCReRJ0XkgH1Miog8IyK9nIn1ZOhiaW3HzU04JS6Mf/5yHENjgnj6s308umI3pZXau787Mcbw1pZDPP5pMoOjAlgwc7jOdNHNOPtX0QsoBX5p/zTGtPTaIuKH1YW7ArjCPvdBYLWIjDLGlDRzfl0X8DXAVUABkIDVM6/uGAGWA4Oxln3YBQzDWncoSUQmmpauJ646hcSYIP71q3Hc/N+t/Gf9QYrLq7n3wmEE6POAbuHTnRnct3wn0cE+/HPuKbo+TzfkzDif2cCrWLWlHOAgUNwGMVyN9Swp0Rizz/5d27CWcLgGeLyJmNywFrlbZYz5mcOu1fUOTQBOB64xxiy0y9aISC3wL6yk1OACearzig3z559zx/Gnt7fx1pbDFFdU8+jsUQTqBJNd2saUHG55ayv+3u48O3ccgyJ1PE935EzN5w6swaPXAQuNMW013HwmsKEu8QAYY1JEZD1wEU0kH2ASMBQrSTWl7mtTYb3yuvWItPtZF9Ur0Ien54zlzqXfs3zrEYorqnlqzlhC9Ztyl7TnaCH/99q3GOCpOWMZ3Tek2XNU1+TMH90hwHpjzHNtmHgAhgPbGyjfgdU01pQz7a2PiGwQkSoRyRSRp0TE8Qn/DmAdcLeIJIlIgIiMx2qC+8gYs6u1H0K5TqCPJ4/MHsWvJsTx+d5srly8iayicleHpZyUnl/G1S9vobCsir//YjSnD9QJQ7szZ5JPAXC4HWIIA/IaKM+l4cXqHPW2t28CnwDTsGZiuAp4re4g+3nO+VhNa5uAIuBr4AAwuxWxq07C18uDey8cxnWTBvLtD/nMe3Ejafmlrg5LtVBBWSVXvrSJtPwyHvrZSKYPj3Z1SKqdOZN8PgFOlc41+2Vd/K8YY+4xxqwxxjwG3AfMEpGhDscuAiYA1wJn29sk4G372dEJROS3IrJZRDZnZWW136dQbcLT3Y1bpidy+/lDSM4oZu6irzmQ1RaPJVV7qqiq4eqXt7D7SBF3nT+U2af0dXVIqgM4k3zuBAKBx0SkLfsO59FwDaexGpGjHHv7ab3yT+ztWAARuQCYA1xujHneGLPOGPM8cDlWjejChi5ujFlojEkyxiT16tXuPbJVG3BzE67+yQAeungE6fnlzH3ha7Yfzm/+ROUStbW1XP/6t2xMyeX6yYP49Zm6YHJP4UwSuRL4CLgBq1axGqsZrqHnP8YY80ALr7sD67lPfcOAnS04tyl1sY20t5vq7d9ob4cCy5q5luoiRIRLT40l2M+Tm9/cytwXN3L95EHMmxiHl86I3anc+e52PtmZweUT4rhpeqKrw1EdyJnkswBrDI4A/e2f+ur2G6ClyWc5Vm1qgDHmAICIxANnALc1c+5HWOODzgXecyg/z95utrdH7e14YKXDcafZ27QWxqq6kPOGxxA634vb3vmeBz/YxQfbjrBg5nBG99MeVJ3B3z/Zw+sbDzFjVAz3zWyub5HqbqSlYytFZAFWUmkRY8x9LbyuP9aEpWXAXfyYuAKBUcaYYvu4OGA/cL8x5n6H8+8F7sbqaPAZ1nOce4E3jTHz7WOCsAaWin3t3Vi99+4FKoFhdb+nMUlJSWbz5s1NHaI6qeLyKp5ZvY/FXx6kthbmTYzjxmmD8deZJ1zm5S8Pcs/yHfwkIYL/zD9VZ6nuxkRkizEm6YTyzjCw355K5wms3moCrAJuMMYcdDgmHkgB7jPGLHAoF+BGrPFHscAR4CXgAWNMlcNx/bBqb5OBGPu4lcACY0yzNR9NPl3f7iOF3Lt8B1+n5BIb5seCmcOYPETXheloH2xL5w+vf8fIvkG88dsJ+Hjql4DurFMnn65Ak0/3YIzhjU2HeGzFHnJLKjl/ZAzTOZfyAAAPMElEQVT3zRxORKC3q0PrEb7an80V/9lEbJgf/7t2IsE6GLjbayz5nFRdV0SCRWSqiMwRkdNbH55SHUNEmDM+lk9vOpuLx/Xhw+1HmPr4Wl79OhX9Ita+dqYXcPXLW4jw9+K1q07TxNPDOZV87KTzbyATWAG8gjWgs27/VSKSLiIT2jZMpdpWmL8Xf79kDG9cPYGIAG/uXLqdnz/3FfsydVxQezicV8rlL27E01147eoJRAb5uDok5WItTj52x4A1wHys8TcfYT2fcfQ+EAXMapvwlGpfpw0IZ8WNZ3HztMHsSC/gp0+u428r9lBV05YzSPVsuSWVzFm4gfKqGpZcOZ74CH9Xh6Q6AWdqPrcAo7FqOwOMMTPqH2CMOYo1Nmdy24SnVPtzdxOun5LApzeexcQB4Ty7eh/THl/LhgM5zZ+smlRQWsncFzaQUVjBoiuSGNFHu7krizPJ5xdAOnC1MaapSbOSgT6tikopF+gX5s/LV57Gs78cS0llDZct3MCNb35HQWlV8yerY2pqDSt3ZXD1S5s59aFV7DlaxFNzxuhEoeo4zvRxHACsMMZUNHNcORB+8iEp5VoXjOrNpMRI/vrRbl79OpWPtx8hKT6MKUMiOXdENDHBuiR6Q/ZmFPHq1z/w3rZ0coor8fd2Z8bIGOadHseYfs3NEax6GmeSTxXQkqeE/XBykTl7DI7jOJ+VWON8fmjmvDjgKWAMEAmUYE2584gx5sN6x8ZiDTA9B+gFHAL+Czzc3Gqpqufx9/bggVkjuDSpHy9+cYAv9ufw+d5sFry3kwER/pw1uBfTh0Vxav8wPHvwAMn80kre+SaN/24+xO6jRbgJTBgQziUX9OO8EdH4eOp0RqphziSfPcBYEfFurPYjIqFYz4W+aelFW7mMdgCQjTUzwmEgCGtl1A9EZLYx5h37d/hjJTRPrNkQfgBOxZr9OgG4tKXxqp5lRN9gnrhsLMYYkjOKWLEjgzV7MnllQyqLvzyIn5c7p/UPY+rQKKYMjSI6uPv34qqqqWXNnkze2HiItclZVNcaBvby58/nJvLzpL5EBnb//waq9ZxJPm8DfwUewZpctCEPYSWE/zpx3ZNeRtsYswNrwtNjROQDrJkQfg28YxefgZVkzjXG1M14vVpEwoBbRMSvmedYqocTERKjg0iMDuIPUxIorqjmi71ZfLozg3V7s1m9J4s7393OoMgAzknsxZShUZwSF9qtakU70wt5c9MPLN+aTl5pFSG+nlw2vh9zxscyvHewq8NTXYwzyecZrJrJ9SKSxI9/2ONF5HdYHRLOBr4HXnTiuq1ZRvsExphqESkAqh2Km1pG240Tu4wr1aQAbw/OGxHDeSNi7FpRMSt3ZfDZ7kz+vf4giz5Pwd/LndMHhnPGoAiig33oFehDZKA3vQK9O3VzlDGGwrJqsksqyCmuZOuhfN7acojkjGI83ISzE3txaVI/zhkS2a2Sq+pYTk2vIyJ9gLewFmVznMEa+/UWYFZL5kpzuOZRYJkx5pp65f8EfmGMaXYhHXsxODcgAvgtVjPcT40xq+z9PsA2rPncfofV7DYeeBVYaoy5rrnfodPrqJYqKq9i/b4cVu3KYE1yFllFJ7ZSB3h7EB7gRa9AbyIDvYkO8iEyyIdeAd5EBnnb5T6E+Hri5ta670bGGEora8gpriTHTig5JRVkF1eSXVxBdpH1OqekgtySSvJKq6ipPf7vwvDeQVyS1JeZo/sQ6q8zE6iWa2x6Hadm9LOTyukich7WImwDAHesh/cfAe8a5+coac0y2nUeBW62XxcDl9UlHjvuchE5E/gfx68B9ALweyfjVapJgT6enDcimvNGRGOMIau4gqwi6yfT3mYUlpNZWEFGUTnfHy5gTUkWpZU1J1zL3U0I9PE47lsegOP/ZaZuz3FlP6qsrqWiuuFBs35e7oT6eRHm70XvEF9G9Q0hIsCLiABvIgK8CQ/wIjbMj7hwHRiq2tZJTSdrjPkY+LiNY2mNfwBvANHAPOA1Efm5MeZ9OFbzeROrR9zl/FjzuQeree53DV1URH6LVZMiNja2nT+C6o5EhMhAnxY9hC+pqD4uQWUVlZNZVEFBWdWP7cIOlSBxeCNi7WpolXtPdyE8wJtwfyuphPl7ER7gRbi/N75enbf5T3VvnWEu89Ysow2AMeYwVm83gPdFZA3wGNZ0P2B1SpgEDDLG7LfL1tnPhhaKyHPGmK0NXHchsBCsZrcWfRqlTpK/twf+3h46/YzqETrD08LWLKPdmM3AIIf3I4E8h8RTx3EZbaWUUh2kM9R8WrOM9gnszgdnYq16WucoECoigxx71eHEMtpbtmzJFpFUZ+PpwSKwxmCp7k3vc/fX2nsc11ChyxeTa80y2vbS3mHAeqwEE43VxDYV+KUx5g37uHis3m5Hgb9gPfNJwhpwmgyMN8boNMZtSEQ2N9TDRXUvep+7v/a6xy6v+RhjSkRkMtb0Oks4fhltx2l6BKtnnWNT4TdYA14vA4KxkstW4CfGmPUOv+OgvcbQAqzZEyKweugtBP6iiUcppTqWy2s+qnvSb8Q9g97n7q+97nFn6HCguqeFrg5AdQi9z91fu9xjrfkopZTqcFrzUUop1eE0+ahOQUTuFpFkEakVkVmujke1nt7TnkFE3hSRbSLyrYhsFJEpLTnP5b3dlLJ9ijXR679dHYhqM3pPe4ZrjDH5ACIyFlglIhHN9SLWmo86gYj0FZGnReQrESkVEWOPlWro2H4i8raIFIhIoYi8Y68a6xRjzIa6QcbKNdr6vus97Zza4T7nO7xt8cJOmnxUQwYBl2DNrfd5Ywc5rEI7BGutp8uxFu1bbQ8eVl2L3veeoc3vs4g8ISIHsFYOmN2isZPGGP3Rn+N+ADeH11dhzToR38BxfwRqsCZsrSvrjzVT+E0OZSuxpudo6OeMetdcg7UmlMv/O/S0n7a+73pPO+dPe91ne/95wCbAq7k4tOajTmBaPuNDg6vQYk13dJFD2VRjTEQjP+sbuK5ygba+76pzas/7bKzldkKxJnNukiYf1RrDge0NlO/AmpVcdU9633uGZu+ziPiKSP+6HSIyEQgHmn3Wp8lHtUZbrEILWJPEishhYCLwgogcFpG+bRCjanstuu96T7u8ltxnX6zFO7eLyHdY66jNNsY0uxabdrVWnYIxZgHWxK+qm9B72v0ZY3Kxvlw4TWs+qjVavQqt6pL0vvcM7XqfNfmo1miPVWhV56f3vWdo1/usyUe1xnJggogMqCtwWIV2uYtiUu1P73vP0K73WWe1Vg0SkZ/bL6cA1wLXAVlAljFmrX1Mi1ahVV2H3veeoTPcZ00+qkEi0tg/jLXGmEkOx8VirUI7jeNXoT3Y3jGqtqf3vWfoDPdZk49SSqkOp898lFJKdThNPkoppTqcJh+llFIdTpOPUkqpDqfJRymlVIfT5KOUUqrDafJRSinV4TT5KKWU6nCafJRSSnU4TT5KuYiIPCkiRkTOdnUsSnU0nV5HKRcRkR8AHyDaGFPr6niU6kha81HKBUTkVKAfsEwTj+qJNPko5RoX29ulLo1CKRfR5KNUE+xnMsZ+famIfCUixSJSJCKrROTMk7z0z4BCYGUL44iyY0luYN+1dXGKyKB6+0ba5V+fZJxKtQtNPkq1gIjcD7wGVAIfAIeBycAqEZno5LWGAYnAh8aYyhaelmdvA+tdyx24xaEorN55N9rbR52JUan2pslHqZb5P2C8MeZsY8ylWGvbLwK8gPudvFZdk9s7LT3BTlKl1Es+wGxgIPCl/f5Y8hGRSOCXwD60eU91Mpp8lGqZe40xW+re2J0E7rbf/kREPJ241sVAOfCRkzHkAv4i4vj/7a1AJvCU/d6x5nMd4A08rp0aVGejyUeplnm/foExJgOrOcwbCG/JRUQkHhgLfGqMKXYyhuOa3kRkGjAOK/EctfeF2fu8gd8BWcBiJ3+PUu1Ok49SLfNDI+WF9tanhddxusnNQf3nPrcCxcA/gQK7LNTezgUigWeMMWUn8buUaleafJRqgTZstroYqAbeO4lzc+1toIgkAVOARcaYPCDf3lfX7HYD1jOiZ+tfRESOiMg9InKXiByye+4tEhF3ETldRNaKSImIbBWRUfXOnS0iK0UkQ0TKRGS7iPzCYf8pdu+6KxzKgkXkexH5XERamqRVN6fJR6kOIiJRwERgnTEm5yQuUVfzCcKq9VQBT9hldTWfMBGZCowE/l3/99idEKKBK4He9vZp4Cp7+y/gOWAOVi3q8XoxjALeBn4FXAisAV4TkUQA+7nYu8CddjLztI/3BC4yxpSfxOdW3ZCHqwNQqgeZhfWF72Sa3ODH5DMOqwb1qjHmkF1WCBisms+NQA0nJg6A0fZ2iTHmLvv1JyLye6xkMsquSWHXrq50PNkYc2/da7ub9xr7mPHAHnvXvcB3WAlsMlbCmmCMyUUpmyYfpTrOxVgJ4t2TPL/uj/cdgOAwdscYUyMixViJKQZ4yxiT0sA1RgEVwCN1BSLigdVp4p91iccWyI/NeXWdGK4Bfg0MwKqB1Sl1iGWbiLwFPG8XTWokFtWDafJRqgOISAhwDrDRGJN2kpepSwx9sQaobq+3v8DeB40PKh0NbDLGFDmUDcMar7Sq3rGjgO0AIiLAMqzk9g/gGyAHmGT/rt31zt0H+AGPGmM2NffBVM+jz3yU6hgXYj33aM1gT8daSUPJpe65z2rHMUn1jMZqEqtfVgN830D5Vvv1ROBcYK4x5iFjzMd2UhmINWbpWPIRkTlYz6Q2AZeLiG+Tn0r1SJp8lGqCMUaMMdLE/nj7mIPNXKo1XazrftfrdfEYY9Y2sH+EvW9yQ+fbD/+HcGLyGQPsceySLSJ9sMYu1R3bz946JplhwHxguzGmxi47G2tc0a3Az+1rXOfkR1U9gCYfpTrGV8CfjDF7XRjDUKzmtYaSz7f1yuo6JtTVfL7B7sQgIlPsDgrLsGo93wGIyFCs51kvGGP+boz5AfgPcKuI+Lf1h1FdmyYfpTqAMeZRY8xjLg5jNNYYo/rPihprisuuez5lJ82rsZrflgEXYM0r5wN8JyLRWNMFfQH8weE6DwHBwB/b9JOoLk9XMlVKKdXhtOajlFKqw2nyUUop1eE0+SillOpwmnyUUkp1OE0+SimlOpwmH6WUUh1Ok49SSqkOp8lHKaVUh9Pko5RSqsP9PxApU4p24X4bAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "import pickle\n",
    "del allres\n",
    "allres = pickle.load(open( \"epsilongreedy_ci_allres.euclidean.p\", \"rb\" ) )\n",
    "\n",
    "renameit = { \"Binomial CI\": \"Binomial\", \"Asymptotic Gaussian\": \"Gaussian\", \"MLE CI\": \"MLE\", \"Euclidean\": \"EuclidMLE\" }\n",
    "skip = { \"Binomial CI\", \"Euclidean\" }\n",
    "FlassPlot.forpaper()\n",
    "for name, res in allres:\n",
    "    if name in skip:\n",
    "        continue\n",
    "    x = [ x[0] for x in res ]\n",
    "    y = [ x[1]['cov'] for x in res ]\n",
    "    ylo = [ x[1]['cov'] - 1.96 * x[1]['covstd'] for x in res ]\n",
    "    yhi = [ x[1]['cov'] + 1.96 * x[1]['covstd'] for x in res ]\n",
    "    FlassPlot.exppic(x, y, label=renameit.get(name, name), expscale=7.0)\n",
    "    FlassPlot.plt().fill_between(x, ylo, yhi, alpha=0.25)\n",
    "FlassPlot.exppic([ x[0] for x in res ], [ 0.95 for x in res ], label='_nolegend_', expscale=7.0, ls='dashed')\n",
    "FlassPlot.legend(loc=\"center\", bbox_to_anchor=[0.5,0.425])\n",
    "FlassPlot.axeslabel('n / $w_{max}$', 'empirical coverage')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}