demo_conv_batchnorm_fusing.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Fusing convolutional and batchnorm layers in TensorFlow\n",
    "\n",
    "- Batchnorm layers can be fused with preceding 2D convolutional layers, into a single convolutional layer.\n",
    "- Batchnorm layers *cannot* be fused with subseqeunt 2D convolutional layers, into a single convolutional layer, *if* that convolutional layer adds padding to its input.\n",
    "- This script demos both types of fusion, and highlights how the fusion can fail in the latter case when padding is used.\n",
    "- Thus, in some models implemented on specialized hardware (e.g., Edge TPU), it is desirable (necessary!) to have an API that implements a batchnorm layer, or at least the broadcasted scaling and shifting that can mimic a static batchnorm layer after training. Moreover, in the case of a sequence of (Conv2D->RELU->BN) layers, the batchnorm layer clearly cannot be fused with the convolutional layer and implementation as broadcast multiplies and additions would be beneficial.\n",
    "\n",
    "Batchnorm layers scale and shift the input data, providing normalized data to the subsequent layer. In the original batchnorm [publication](https://arxiv.org/abs/1502.03167), bathnorm layers follow convolutional layers, and precede RELU layers (Conv2D->BN->RELU). Since that publication, many researchers have used batchnorm layers that follow the RELU layer (Conv2D->RELU->BN), claiming superior results.\n",
    "\n",
    "After training, if the moving mean and moving standard deviation of the batchnorm layer are kept fixed, then the scaling and shifting parameters *might* be \"fused\" with a preceding or subsequent convolutional layer (by scaling and shifting the parameters of the convolutional layer), thereby allowing the explicit batchnorm layer to be removed (as it will be implicitly implemented by the convolutional layer).\n",
    "\n",
    "For a nice summary of how this fusing can be accomplished when the batchnorm layer follows the convolutional layer, see here: [https://tehnokv.com/posts/fusing-batchnorm-and-conv/](https://tehnokv.com/posts/fusing-batchnorm-and-conv/).\n",
    "\n",
    "*However*, if the batchnorm precedes a convolutional layer, the fusion can only be accomplished if the convolutional layer does not add any padding to the input. If padding is added, attempts to fuse the batchnorm layer will results in errors at the edges of the output tensors, compared to outputs of the unfused pair of layers. This discrepency will be demo'd in this script."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "from tensorflow.keras.layers import Layer, Input, Conv2D, BatchNormalization\n",
    "from tensorflow.keras import Model\n",
    "\n",
    "# %matplotlib notebook\n",
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Define the architectures that will be used"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "def build_conv_bn(input_shape, n_chan_out, padding='same'):\n",
    "    # Build architecture with batchnorm after convolution\n",
    "    x = Input(shape=input_shape)\n",
    "    y = Conv2D(n_chan_out, (3,3), padding=padding, name='paired_conv_1')(x)\n",
    "    y = BatchNormalization(axis=-1, name='paired_bn_1', fused=False,\n",
    "                           beta_initializer=tf.initializers.RandomUniform(0, 1),\n",
    "                           gamma_initializer=tf.initializers.RandomNormal(0, 0.5),\n",
    "                           moving_mean_initializer=tf.initializers.RandomNormal(0, 0.5),\n",
    "                           moving_variance_initializer=tf.initializers.RandomUniform(0, 1))(y)\n",
    "    model = Model(x, y)\n",
    "    return model\n",
    "\n",
    "def build_bn_conv(input_shape, n_chan_out, padding='same'):\n",
    "    # Build architecture with batchnorm before convolution\n",
    "    x = Input(shape=input_shape)\n",
    "    y = BatchNormalization(axis=-1, name='paired_bn_1', fused=False,\n",
    "                           beta_initializer=tf.initializers.RandomUniform(0, 1),\n",
    "                           gamma_initializer=tf.initializers.RandomNormal(0, 0.5),\n",
    "                           moving_mean_initializer=tf.initializers.RandomNormal(0, 0.5),\n",
    "                           moving_variance_initializer=tf.initializers.RandomUniform(0, 1))(x)\n",
    "    y = Conv2D(n_chan_out, (3,3), padding=padding, name='paired_conv_1')(y)\n",
    "    model = Model(x, y)\n",
    "    return model\n",
    "\n",
    "def build_fused(input_shape, n_chan_out, padding='same'):\n",
    "    # Build fused architecture (one without the explicit batchnorm layer)\n",
    "    x = Input(shape=input_shape)\n",
    "    y = Conv2D(n_chan_out, (3,3), padding=padding, name='fused_conv_1')(x)\n",
    "    model = Model(x, y)\n",
    "    return model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Define methods for setting parameters of the fused models\n",
    "\n",
    "#### Fusion of convolution followed by batchnorm\n",
    "In this case, the fusion is fairly straightforward. We compute the new parameters\n",
    "of the resulting convolutional layer as follows. (Abusing notation by aggregating\n",
    "the convolutional parameters into a single ```w``` variable, and indicting the\n",
    "convolution operater as ```**``` and the multiply operator as ```*```.)\n",
    "\n",
    " - For convolution we have:\n",
    "     - ```y = w_conv**x + b_conv```\n",
    " - For batchnorm we have:\n",
    "     - ```z = m_bn*y + b_bn```\n",
    " - Combining we get:\n",
    "     - ```z = m_bn*(w_conv**x + b_conv) + b_bn```\n",
    "     - ```z = (m_bn*w_conv)**x + (m_bn*b_conv + bn)```\n",
    " - So:\n",
    "     - ```w_fuse = m_bn * w_conv```\n",
    "     - ```b_fuse = m_bn * b_conv + b_bn```\n",
    "     - ```z = w_fuse*x + b_fuse```\n",
    "     \n",
    "     \n",
    "#### Fusion of batchnorm followed by convolution\n",
    "In this case, the inclusion of the offset (shift) parameter in the batchnorm\n",
    "can cause problems if padding is used.\n",
    "\n",
    " - For batchnorm we have:\n",
    "     - ```y = m_bn*x + b_bn```\n",
    " - For convolution we have:\n",
    "     - ```z = w_conv**y + b_conv```\n",
    " - Combining we get:\n",
    "     - ```z = w_conv**(m_bn*x + b_bn) + b_conv```\n",
    "     - ```z = (w_conv*m_bn)**x + w_conv**b_bn + b_conv```\n",
    "\n",
    "But the above \"combined\" equations are not as simple as they appear. The new convolution kernel, ```w_conv*m_bn``` is fine as it is. But the ```w_conv**b_bn``` portion of the new bias term needs further\n",
    "explanation. Because the batchnorm shift parameter is added before the convolution is applied, we must sum all the weights and multiply them by the shift parameter, which is not be clear from the notation.\n",
    "\n",
    " - The new bias term is really this:\n",
    "     - ```SUM_ij[w_conv_ij]*b_bn + b_conv```\n",
    "\n",
    "The problem occurs when zero padding is added to the input to the convolution layer. In that case, the new zero values have not had the shift of the batchnorm layer added to them (the scaling parameter doesn't matter, since it would have been multipled by zero, had the zero-padding been applied before the batchnorm layer rather than after it). But we'll plow ahead anyway...\n",
    "\n",
    " - So:\n",
    "     - ```w_fuse = w_conv * m_bn```\n",
    "     - ```b_fuse = SUM_ij[w_conv_ij]*b_bn + b_conv```\n",
    "     - ```z = w_fuse*x + b_fuse```\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "def fuse_conv_bn_params(model_source, model_target):\n",
    "    names = [layer.name for layer in model_source.layers]\n",
    "    for name in names:\n",
    "        # Assume names have fixed prefix, but model-specific\n",
    "        # suffixes for multiple conv-bn layer pairs.\n",
    "        if name.startswith('paired_bn'):\n",
    "            name_bn = name\n",
    "            name_conv = 'paired_conv' + name[len('paired_bn'):]\n",
    "            name_fuse = 'fused_conv' + name[len('paired_bn'):]\n",
    "\n",
    "            ## Get effective scaling and shifting parameters of the batchnorm\n",
    "            layer_bn = model_source.get_layer(name=name_bn)\n",
    "            params_bn = layer_bn.get_weights()\n",
    "            gamma = params_bn[0]\n",
    "            beta = params_bn[1]\n",
    "            moving_mean = params_bn[2]\n",
    "            moving_variance = params_bn[3]\n",
    "            epsilon = layer_bn.epsilon\n",
    "\n",
    "            m_bn = gamma / np.sqrt(moving_variance + epsilon)\n",
    "            b_bn = beta - m_bn * moving_mean\n",
    "\n",
    "            ## Get weight and bias parameters of the convolution\n",
    "            layer_conv = model_source.get_layer(name=name_conv)\n",
    "            params_conv = layer_conv.get_weights()\n",
    "            w_conv = params_conv[0]\n",
    "            b_conv = params_conv[1]\n",
    "\n",
    "            ## Compute new convolution kernel and bias of composite layer/model            \n",
    "            # Need to be sure we broadcast across correct dimensions\n",
    "            b_fuse = m_bn * b_conv + b_bn\n",
    "            m_bn = np.reshape(m_bn, (1, 1, 1,  m_bn.size))\n",
    "            w_fuse = m_bn * w_conv\n",
    "\n",
    "            ## Set parameters for composite layer/model\n",
    "            layer_fuse = model_target.get_layer(name=name_fuse)\n",
    "            layer_fuse.set_weights([w_fuse, b_fuse])\n",
    "\n",
    "def fuse_bn_conv_params(model_source, model_target):\n",
    "    names = [layer.name for layer in model_source.layers]\n",
    "    for name in names:\n",
    "        # Assume names have fixed prefix, but model-specific\n",
    "        # suffixes for multiple conv-bn layer pairs.\n",
    "        if name.startswith('paired_bn'):\n",
    "            name_bn = name\n",
    "            name_conv = 'paired_conv' + name[len('paired_bn'):]\n",
    "            name_fuse = 'fused_conv' + name[len('paired_bn'):]\n",
    "\n",
    "            ## Get effective scaling and shifting parameters of the batchnorm\n",
    "            layer_bn = model_source.get_layer(name=name_bn)\n",
    "            params_bn = layer_bn.get_weights()\n",
    "            gamma = params_bn[0]\n",
    "            beta = params_bn[1]\n",
    "            moving_mean = params_bn[2]\n",
    "            moving_variance = params_bn[3]\n",
    "            epsilon = layer_bn.epsilon\n",
    "\n",
    "            m_bn = gamma / np.sqrt(moving_variance + epsilon)\n",
    "            b_bn = beta - m_bn * moving_mean\n",
    "\n",
    "            ## Get weight and bias parameters of the convolution\n",
    "            ## Compute new convolution kernel of composite layer/model\n",
    "            layer_conv = model_source.get_layer(name=name_conv)\n",
    "            params_conv = layer_conv.get_weights()\n",
    "            w_conv = params_conv[0]\n",
    "            b_conv = params_conv[1]\n",
    "\n",
    "            ## Compute new convolution kernel and bias of composite layer/model\n",
    "\n",
    "            # New convolutional kernel...\n",
    "            m_bn = np.reshape(m_bn, (1, 1, m_bn.size, 1))\n",
    "            w_fuse = w_conv * m_bn\n",
    "            \n",
    "            # New bias...\n",
    "            # Mimic impact of convolution on the shift/offset term of the batchnorm.\n",
    "            # This is slightly incorrect at the x/y spatial edges of the tensors if\n",
    "            # conv padding is used.\n",
    "            \n",
    "            # b_bn has no x/y spatial dependence, so sum conv weights in x and y...\n",
    "            w_conv_sum = np.sum(w_conv, axis=(0,1))\n",
    "            # mimic the convolution, applied to the shifting parameter of the batchnorm\n",
    "            b_bnconv = np.sum(w_conv_sum * np.reshape(b_bn, (b_bn.size, 1)), axis=0)\n",
    "            b_fuse = b_conv + b_bnconv\n",
    "           \n",
    "            # Set parameters for composite layer/model\n",
    "            layer_fuse = model_target.get_layer(name=name_fuse)\n",
    "            layer_fuse.set_weights([w_fuse, b_fuse])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Set model input/output sizes, and create random input sample for testing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set the seed, for repeatability\n",
    "tf.random.set_seed(0)\n",
    "\n",
    "input_shape = (20, 20, 3)\n",
    "n_chan_out = 16 # Unimportant. Will sum over all output channels for plotting\n",
    "\n",
    "# Create random data for model testing through models and measure the output delta\n",
    "n_samples = 1\n",
    "x = tf.random.normal([n_samples] + list(input_shape))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Build and test fused models: Convolution (with padding) followed by batchnorm\n",
    "Note that the output errors (difference) between the two models is just numerical accuracy error, which is very small."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Mean Absolute Error: 5.26e-08\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgEAAAHBCAYAAADw5mhIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nOzdebzcVX3/8dc7NxuELCRhCUlYFAQRBTWyWK0oCIGq2FYtVAXX1Kqttvpzr7jRurSlWq0WAQEti9oisUYBN1Br0IDsiwkhmI2E7AkJJPfm8/vjnEsmw8zd5tx8bzLv5+MxjzvznfP9fM98Z+bO+X7O+X6PIgIzMzNrP8OqroCZmZlVw40AMzOzNuVGgJmZWZtyI8DMzKxNuRFgZmbWptwIMDMza1PDq66AmZm1p9NfOiZWr+kqHvfWO5+4PiJmFg+8B3IjwMzMKrF6TRe/uf7g4nE7psyfXDzoHsqNADMzq0QA29ledTXamscEmJmZtSlnAszMrCJBVzgTUCVnAszMzNqUMwFmZlaJNCbAk9hVyY0AMzOrjAcGVsvdAW1CUkg6PN//mqR/qHnuryWtkLRJ0iRJfyRpfn786upqPXCSLpP0marrYUObpEPzd8MHRNaW3AjoA0lvknSXpM2SHpH0VUkT+rH+IkmnFqxPS/Ei4h0R8ekcawTwr8BpEbFPRKwGPgV8OT/+Xpla7z52lwaEpNMl3Sxpo6RHJd0k6VWDvM0/kfRLSevyd+FiSWNrnv+5pMdznTZIulXShySNKrDt7h/sTfm2QtJ/5M9wd5lFklZKGlOz7G2Sft7q9q28IOiK8jfrOzcCeiHpfcDngP8HjAdOBA4BbpQ0ssq6FXIAMBq4p2bZIXWP+8xHVLuGpNcA3wGuAKaR3sePA68c5E2PBz4DHAQ8E5gKfKGuzLsjYiwwBXgfcDYwR5KaBZV0QD/qMCEi9gGeDZwEvKvu+Q7gPf2It0sp8f9eGxL8QeyBpHHAJ4G/iYgfRcS2iFgEvA44FHhDLrfTkaOkkyUtyfe/CRwMfD8fvXyg5ohmlqRlkpZLen/N+v2K16Tu/y/HXSbpLXXPXSbpM5KeATyQF6+T9FNJDwJPq4k/StJ4SZfkeEvzuh051psk/UrShZJWA5/Iy98i6T5JayVdL+mQmu2HpHfkLod1kr5S+wMh6e153Y2S7pX0vLz8IEn/nY96H5L0t728hZMl3Zjj3FRXh6Pyc2skPSDpdXn5LOD1wAfy6/++pDdL+n7NuvMlfafm8WJJx/UUNz83StI/S/pDPor9mqS9at9jSe/LR7LLJb25yXsrUvbm0xFxcUSsj4jtEXFTRLw9lxkm6WOSHs7xrpA0Pj/X/fk7L9dllaSP1uzjLZIm1mzvubnMiIi4Mn8XNkfEWuDrwB81qmdEPBYRPwdeRfqx/pMe3qsFkq6T9GrVHNn3JCJWAjcCR9c99QXg/epHtq5bfq+7P3sLJf1VzXN3S3plzeMReb88Nz8+UdL/5c/0HZJOrin7c0kXSPoVsBl4Wv7uLMzbekjS6/tb3z3BdqL4zfrOjYCevZB0lPw/tQsjYhMwB3h5bwEi4o3AH4BX5vT652uefilwBHAa8EH1IcXfSzwAJM0E3p/rdwTQMG5E/B54Vn44ISJeFhFPr4v/BHAZ0AkcDjw31/dtNaFOABaSjkYvkHQW8BHgz4D9gF8AV9Vt/hXAC4DnkBpVp+e6v5bUkDgXGEf6AVmtdOT0feAO0tHnKcB7JZ3ew+56PfBpYDJwO/BfeRtjSD8eVwL7k45U/0PS0RFxUS73+fz6XwncBLw4/7AeBIwk/agh6WnAPsCdPcXN9fks8AzguLwvp5KO3rsdSDrSngq8FfiKpH0bvK4jgenAd3t47W/Kt5eSGnX7AF+uK/OiHOsU4OOSnhkRy4BfA39eU+4vge9GxLYG2/ljeskaRcQfgHnAi3soNh34IfBBYImkf5X07J7i5vfidGBu3VPzgJ+TvgP9tZL02RwHvBm4sLsRSsq6vKGm7JnA8oj4naSpwA9IWZKJedv/LWm/mvJvBGYBY4FHgS8BZ+SsyQtJn9G2EkAXUfxmfedGQM8mA6siorPBc8vz8634ZD5augv4BnBOi/G6vQ74RkTcHRGPkY/OB0IpTXsm8N5c15XAhaQfuG7LIuLfI6IzIrYA7wD+KSLuy/vuH4Hjao/Egc9GxLr8A/Ez0g8jpMbF5yPit5EsiIiHSQ2G/SLiUxGxNSIWko5Ca+tR7wcRcXNuyHwUOEnSdNI/+UUR8Y1c598B/w28tlGQvK2NuY5/DFwPLJN0FPAS4BcRsb2nuPnofRbwdxGxJiI25v1SW/9twKdyxmkOsIn0I11vUv67vIfX/nrgXyNiYW60fhg4Wzt313wyIrZExB2kxtWxefmV5M9irvfZedlOJL0cOI+dGzLNLCP9ODaUPwtfi4iTSPv4cVIXwjxJL6srvkrSOmAp8BiNG0MfB/6m7ke4VxHxg4h4MH/2bgJuYEfj5VvAmUoZQkg/6t/M998AzImIOTkrcyOpMXJmTfjLIuKe/J3oBLYDx0jaKyKWR8SAuuDMWuFGQM9WkVLKjfq5p+TnW7G45v7DpH7WEg5qEHugDgFGAMtzmnMd8J+kI91uixus88Wa8msAkY5wuz1Sc38z6UgV0hHhg03qcVB3zBz3I6TsQzNP1iv/EK4h7ZtDgBPqYr2edCTezE3AyaQfqJtIR5ovybebaurYLO5+wN7ArTXP/Sgv77a6rsFZu19qrc5/p/RQ34PY+X1/mHRKcO3+avYe/DepwTQlv97tpGzOkySdSGoYvCZnlHozlbT/0Y6BfZskNZo95mFSo+RuUsZk/7rnJ0fEBNL+/BWpUbaTiLgb+F/gQ32o25MknSFpbu7OWUf6EZ+cYy7L2/vz3NVwBjm7RHrvX1v33r+Ind+j2s/jY8BfkBrMyyX9IDcq2467A6rlQVw9+zXwBCmt/e3uhZL2If0D+Ehe9BjpH1K3+h+TZp/K6cD9+f7BpKOlVuJ1W55jd2tlmq7FpH0wuUlGpFF9FgMXRMR/NSrch+09vcnyhyLiiH7EenIf5PdsImkfLwZuiohm3TmN9u9NpEF3h5GO4Lt/4E9iR5q9adzcnbEFeFZELO3Ha2jkgbytPwf+uUmZZaQfpm4Hk44+V5AGEjYVEWsl3UD6kXomcHXEjiHXuQ98NvCWiPhJb5XN2ZfnkwbYkgf11ZcR6Ufz3Py65pGyY38aEY83qecWSZeR+v8nR0R9o/x84DbgX3qrY67DKFID6FzguojYJul7pAZst8tJ2arhwK9r3svFwDe7x2Q0sdPnKiKuB65XGhfyGVJmq6cuE7PinAnoQUSsJw0M/HdJM/NAoENJDYIl7EgF3k5KE06UdCDw3rpQK0j9svX+QdLekp5F6n+8psV43b4NvEnS0ZL2Jv0zHJCIWE5Kif6LpHG5X/zpkl7Sw2pfAz6cXxdKAwsbptobuJj0T/35Sg7P3Qi/ATZK+qCkvSR1SDpG0gt6iHWmpBcpncXxaWBuRCwmHSE+Q9Ib83s6QtILJD0zr9do/95E6l/fKyKWkI6MZ5JS87/LZZrGzd0FXyf1Me+f98vUXsY0NJR/kP+e9Pl5c8378iJJF+ViVwF/J+mw3AD6R+CaHhpy9a4k/Ri+hpquAEnHkDIYfxMR32+ybnfZvfPn5DrS+zenh+IPApcAi4DnRMRpEXFVswZAjj+KlJJ/hB3ZkSdFxALSd6q3AaTdRgKjSP31nZLOII1/qfU94Hmksw+uqFn+LeCVSqdtdkgarTTYs2GDS9IBks5SGkfyBKnrZ3t+rnvg5qF9rPduK8CnCFbMjYBe5IF3HyEdcW0AbiG1+k/Jfc2QGgN3kP6B3cCOH/Nu/wR8LKcJawcr3QQsAH4C/HNE3NBivO46/xD4N+CnOf5P+/eqn+Jc0j/Ie4G1pD7YpqnoiLiWdNR3taQNpLTuGX3ZUER8B7iA9MOzkfRPd2JEdJH63I8DHiJ1xVxMGkjXzJWkBtAa0pHoG/I2NpL+uZ9NOmJ+JNe3+1z2S4Cj8/79Xl7n96R/1L/IjzeQBkP+KtetL3E/SHo/5ub98mMa9/n3KiK+SzpSf0ve1grS0eR1ucilpM/RzaT99TjwN/3YxGzSoNJH8piBbu8jdWFcUpPSr+/L/rKkjblO/0Y6up6ZG0LNnBsRz4iIC3IjqyfrJG3K8U8CXlWbqajzKWBMk+d2kt+/vyU1oteSBkTOriuzhfR6DqNmwHBuXHYPiH2U9D/i/9H8f+wwUkNuGenz+RLgr/Nz00ldIq1mjHYL2wfhZn2n5t8dGyy5hf8QMKIfR2ZmNgRI+jjwjIh4Q6+FBxb/Y8CjEfGfgxF/KDn22JHxwzmtjq9+qqnTlt8aETOKB94DeUyAmVkfKV0/4a2kbohBERFD/mqVpYRP6aucuwPMzPpA0ttJaf4fRsTNVdfHrARnAioQ6aqDTS+hamZDT0R8nTS400oJ6HIioFLOBJiZmbUpNwJsj6A0n8EqSY/0XrppjIPzaPeOknXLsTcpXWK4MpJ+KOm8AnE8/S5Pzpvxy6rrsTsLfHZA1dwIsF1qMP5xKl117n3A0RHR01X/ehQRf4g0X0BXudo9GXuffPnhXuUf2MMHoQ5nRMTlpeP2l6RnSPpObrStl3SnpL8fjMZX3TavU5p8ao3SpFZH1jz/CUnblCbz2Sjp95K+rHTVRBs0omsQbtZ3bgQUtrsdHe0h9T2YdMndlbu6PtY/kp7OjmttPDsixpPmbJhBmlhnsEwgnfN/JOnSyb9hxzUVul0TaTKficCfkq7UeetQagjsbt9XG/rcCADyVei+W7fsi5K+lO/3ayrdfJW7m/JRzipJ1+SyT0mjKk0x+rZ8v+F6Derb21TEx0v6db7YzfJ8RDOy5vmQ9C5J84H5Na93saQNkm6V9OKa8p/IR27fykdJd+Ujqw8rTVO7WNJpNeUb7i+lK/J9jXRd+k1K11fv6xS7H8yp/m/U7YtTSTP3HZRjXqaaqZdryi3KZbv3z7z8WldI+tdG74/StLqz85HjAqXR4bX75NtKU/RulHSPpKbnJavm6D7X8StK14vfKOmW/OOIpO5R53fk1/MXefnbcx3W5Do1nGdC6Up135K0Or//v1WaBKr+s/YmSb/M+32t0lS2Z9TEOUzSzbl+P871/VaTbTb9fjTwSeD/IuLv89UoiYgHIuIvI6L78/CqvD/X5To/s2ZbiyS9Xyl7sF7SNZJG5+fuk/SKmrLDlY78nxcRv4mIS/LkTdtIk2AdKWkSdfIETveQLsb0KCnL1Ktm3yFJB0raXLstSc/LdRuRH/c29fZO39c9RQDbo/zN+s6NgORq0iVmxwLkf2CvY8flUi+jH1Ppki5RewOwL+k67f/ex3r0d71mUxF3AX9HmvjkJNJUse+sW/fVud7d09z+lnQ1vomk1/2d7n+u2StJV6Dbl3SZ3OtJn5+ppKuy1V7Y5DIa7K+IuI80Ycqvc3q8e773vkyxO5F0LfxZtS8iIn5MuhrhshzzTc12Vo0vAl+MiHGkeQq+3aTc1aTLQx9EunzuP2rnGe1elct0H2XWT9Xbk7NJP4j7kq4ieEF+PX+cnz82v55r8jb/ifSZnEK6mtzVTeKeR7qK4nTSJY3fQZqzoJETSPMQTAY+T7oKYHcu9UrS0fIk0iyUPZ0Xfxk9fz9qnUoPUyBLegbpksfvJV2ZcA7wfdU0Ykn7YSbpqn3PIU2ZTF6vdibO00mzgN7WYFN/TLoa4lMuN9wtdwtdR9+v59/wOxQRj5AmnHpdTdk3kuZk2Ka+Tb1d/301K8KNACDSVLW3kVKAAC8DNkfEXA1sKt1t5FnvIuLxiOhrH3h/12s4FXFE3BoRc3N9FpF+oOuv9f9P+ahoS17nWxGxOq/zL6RL3dZe0vYXEXF9vsLhd0j/rD6bj6quBg6VNKGP++tJ+Uentyl2twPnR8QT3fVt0TbgcKVJZzZFRP189N2T3vwR8MH8XtxOukzxuTXFfhlp6tguUgPp2Po4Pbg2H512kmaiO66Hsq8HLo2I2/Klqj9MyqYc2uS1TQIOj4iu/FnY0CTuwxHx9Vz/y0kNjAOUxli8APh4pGmbf0nd5XO79ff9znXraQrkvyBNAX1j/mz9M7AX8MKaMl+KiGURsQb4Pjv23ZXAq5Tmy4B02d/6H1OUruf/FdJle3vT4xTItXr5Dl1Ovmx1Psg4hx1zj/Rl6u2dvq97Eo8JqJYbATs8OYc66Z9HdxZgIFPpfoB0HYDf5LTmW/pYh/6u13AqYqVU/f9KekTpGvX/SJ4OtVmdc4r1vpxiXUc6mqxdZ0XN/S2kI6yumseQpqLty/6q1Zcpdh+NHiaSGYC3kjIP9+d0+SsalDkI6G6UdHuYnqdDHq2+99k2m8a3kZ2mBY40LfLqurp0+yYpS3O1UlfR57tTzj3VISI257v7sOO1b64pW/8Z79bf93s1/ZgCOc83sJg+TEMdacKg+0gT+exNytRcWVMWSfuRsm3/ERFPaSA08OQUyL3p5Tt0HWk+isOAlwPrI+I3+bm+TL3dbP/v1gI3AqrmQSY7fIc0U940UkbgpLy831Pp5vTf2wEkvQj4sVJf7/pcZG/SZERQM01ws/XyP7dGmk1F/FVSyv6ciNgo6b2kdHbDOue+yw+Qug3uiYjtktYysAsa9ba/6nvsVtH7FLv97eXbaSrmfOT1ZKMiIuYD5yhN7/tnwHcb9A0vAyZKGlvTEDiYaiZ12WlaYKWZ5yY1qks+ev4k8MmcKZhDSvlf0o/tLSe99r1rGgLTm5Tty/ej1o9JUwV/o8nzy4Bndz/ImaLp9H2/d3cJDAPurf3uSNqX1ACYHREX9BYofz5emevcW9kev0MR8bikb5OyAUexIwsAfZt62z3dNiicCcgi4lFSv903SPPW35eX93sqXUmv1Y4pRNfSPf4lbWMp8AalgXJvIfVJ97heD9VuNhXxWFIjY5Oko9gxO1kzY0l9uo8Cw5UmSBnXyzoN9WF/rQCmdffxRsEpdmv8nnRU/if5KPhj7JjJD0lvkLRf3va6vHin/RxpVrj/A/5JabDdc0gZhIaD4wqrn8r4KuDNko5Tmj73H4FbclfPTiS9VNKzc8NnA6l7oF+nTufusXmkQa4jJZ1E+jFsVLa/34/zgRdK+oLSNNndA2K/JWkCaXzGn0g6Jb937yM1Mv6vj9W/mjQm4a/ZeQrkcaQMya8i4kM9BVAaUPhM0n4/EPjXPmy3L9+hK0jjF17Fzo2AVqbe3u1tDxW/Wd+5EbCzK0kDl66sW96vqXRJ/am3KE13Oht4T+w4R/ztpClGVwPPYud/bj2t10izqYjfT+rS2Ej6gW14lkGN60kp+N+TUrGP01r6saf99VPgHuARSavysmJT7AJExHrSQMiLSY2ux0gD/LrNBO7J+/mLwNlN+lrPAQ4lHZ1eSxqX0OtRYQGfAC7P6eHX5W3+A2kK2+WkhmOzPvcDSft7Ayk1fhM7/+D01etJ2bDVpCmKryH9GDfS5+9HRDyY4x5Keg/Wk17XPGBjRDxAOlr+d1KW6JXAKyNia18qnRslvyaNIaj93P8p6fv1Zu2YAnmT0viHbn+RPxPrSd+/1cDzI2IZvev1OxQRvyI1yG7LDa3u5QOeetusVZ5KeDckT0Vsu5jS6ar3R8T5Vddldybpp8CVEXFx1XUZCo5+zsj41v8O+PpeTT3/kMWeSriPPCbAzJ5C0gtIA9QeIqXXzyKdymkDlPfp80j70oBAdDkhXSk3AsyskQOB/yENQFwC/HVE/K7aKu2+JF1OOtf/PXVnnJhVyo2A3VB4KmIbZBHxfdI5+FZARLQ8cdOeygP5quU8jJmZWZtyJsDMzCrRfbEgq86QbAQMH793jNp/fMtxuraUeXnRUegMilJxhhAV+v5KZfZNFEotRmehF7a9TJyO0WVmN95eqD6xtUwSUSVeVqF8Zgwv9P0ctud9zzsfXUfXxscG4ddadIUT0lUako2AUfuP56gv9vVKu81tuOMpE4QNyNZJZf4Bd4zr06nOvSrxTSz1Yzl8RJkzFEeMKLOPt24t85HeunZ074X6oOOxMv/gxh6+rvdCffDY5lG9F+qDruV7FYkzam3r+6drdJkf3a2Ty3yWO8ZuKxKHUg3afl0qqrFl//CV1oPYkDQkGwFmZrbnS5dEdSagSt77ZmZmbcqZADMzq4wHBlarpUyApJmSHpC0QNJTJuWQNErSNfn5W9R4/nMzMzOrwIAbAXmWsq+QJro4mjQ169F1xd4KrI2Iw4ELSZNkmJmZEZHODih9600rB7CSPpyXP1A722mzmJIOyzEW5Jgje9qGpEmSfpYnuPpyXb2eL+muvM6XpNbPz2olE3A8sCAiFuYZvq7mqdfEPgu4PN//LnBKiUqbmdmeYTsqfutJKwewudzZpBlgZwL/kaeF7ynm54ALc6y1OXbTbZBmoPwH0myw9b5Kmon2iHyb2eOL7YNWGgFT2XmqzCV5WcMyeba79aRrkZuZmVWhlQPYs4CrI+KJiHiINAX68c1i5nVelmOQY766p21ExGMR8UtSY+BJkqYA4yJibqTpf6+oiTVgQ2ZgoKRZwCyAkfuNq7g2ZmY22NIVA3f5SWqNDmBPaFYmIjoldR/ATgXm1q3bffDbKOYkYF3NlO+15ZttY1UP9V7SZNsD1sreXwpMr3k8LS9rWEbScGA8sLpRsIi4KCJmRMSM4eP3bqFaZmbW5iZLmldzm1V1hYaqVjIBvwWOkHQY6cf+bOAv68rMBs4Dfg28BvhpTmOYmVnbG7TLBq+KiBlNnuvPAeySugPYntZttHw1MEHS8JwNqC3fbBvNLM3r91Tvfhvw3s8v6N3A9cB9wLcj4h5Jn5L0qlzsEmCSpAXA3wNPGYVpZmbtqfuKgaVvvXjyADaP1D+bdMBaq/sAFnY+gJ0NnJ1H9h9GGpz3m2Yx8zo/yzHIMa/rZRuN91XEcmCDpBPzWINza2INWEtjAiJiDjCnbtnHa+4/Dry2lW2YmZmVkvvfuw9gO4BLuw9ggXkRMZt0APvNfAC7hvSjTi73beBeoBN4V0R0ATSKmTf5QeBqSZ8Bfpdj02wbOdYiYBwwUtKrgdMi4l7gncBlwF7AD/OtJUNmYKCZmbWfrkITJfVHKwewEXEBcEFfYublC0lnD9Qv72kbhzZZPg84ptFzA+W5A8zMzNqUMwFmZlaJQFWcImg1hmQjoLOrg9Vr9mk5TkwqM0f9s45a3HuhPliwcnKROFtXtn4KpcZvLVAT2LplRJk4j5Q5LbRjS6F/KPuVmRP+dSfM7b1QH9y/8YAicU542qIicS5ZdGqROFumdvZeqBcdY8u8V8+etrxInHvnHVokzgtfeG+ROLctn957oV4MGzZ4J3VtH5yzA6yPvPfNzMza1JDMBJiZ2Z6voisGWg3vfTMzszblTICZmVUiUCWnCNoOzgSYmZm1KWcCzMysMn24zK8NIjcCzMysEhEM1gRC1kfe+2ZmZm3KmQAzM6uI2I4HBlbJmQAzM7M25UyAmZlVIvCYgKq5EWBmZpXxFQOr5b1vZmbWppwJMDOzSgRiu68YWClnAszMzNqUMwFmZlYZjwmolhsBZmZWiQC2++yASg3JRsCwLWLM3aNbjvP4xChQG1i1eUyROFP23VAkzuqRnS3HGD2i9RgAq9fuUyTO9rFl6sOWkUXC7HfA+iJxbn1umX9w0+ZuKRLnP29+WZE4HYeUqU9HgRjvOfZnBaLAF394RpE4pcxft1+ROF1drX8Gw/32e6wh2QgwM7N2ILp8xcBKOQ9jZmbWppwJMDOzSnhMQPW8983MzNqUMwFmZlYZjwmolhsBZmZWiQi5O6Bi3vtmZmZtypkAMzOrjKcSrpb3vpmZWZtyJsDMzCoRwHYPDKyUGwFmZlYRuTugYt77ZmZmbcqZADMzq0S6YqC7A6rkTICZmVmbGnAjQNJ0ST+TdK+keyS9p0GZkyWtl3R7vn28teqamdmepIthxW/Wd610B3QC74uI2ySNBW6VdGNE3FtX7hcR8YoWtmNmZnugQO4OqNiAm0wRsTwibsv3NwL3AVNLVczMzMwGV5GBgZIOBZ4L3NLg6ZMk3QEsA94fEfc0iTELmAUwav9x7HvK8pbrtfjhyS3HAFj50KQicQ47svXXBDCio6vlGI8+vG+BmsArj/9dkTg/vu4FReJsPWpLkTir55d5zzd9dL8icRbML/O6NG5rkTivfeZtReIcv8/ClmP83c/PKVAT+JOXFPosP/SMInHWbdq7SJxpE9e1HGP58M4CNWlsu9P3lWp570vaB/hv4L0RsaHu6duAQyLiWODfge81ixMRF0XEjIiYMXz8Xq1Wy8zMzHrRUiNA0ghSA+C/IuJ/6p+PiA0RsSnfnwOMkFTm8NzMzHZrEdAVKn6zvmvl7AABlwD3RcS/NilzYC6HpOPz9lYPdJtmZmZWTitjAv4IeCNwl6Tb87KPAAcDRMTXgNcAfy2pE9gCnB0R0cI2zcxsD+KzA6o14EZARPwSep75ISK+DHx5oNswM7M9VzpF0AMDq+S9b2Zm1qY8d4CZmVWmy1MJV8qZADMzszblTICZmVXCswhWz5kAMzOrSBoYWPrW61almZIekLRA0ocaPD9K0jX5+VvyVXG7n/twXv6ApNN7iynpsBxjQY45soVtLJJ0V56Qb16/d3cDbgSYmVnbkNQBfAU4AzgaOEfS0XXF3gqsjYjDgQuBz+V1jwbOBp4FzAT+Q1JHLzE/B1yYY63Nsfu9jZq6vTQijouIGSX2hxsBZmZWme2o+K0XxwMLImJhRGwFrgbOqitzFnB5vv9d4JR84buzgKsj4omIeAhYkOM1jJnXeVmOQY756gFuY1C4EWBmZu1kKrC45vESnjoD7pNlIqITWA9M6mHdZssnAetyjPpt9XcbkIZR3CDp1jzpXss8MNDMzCrRPXfAIJhc12d+UURcNBgb2sVeFBFLJe0P3Cjp/oi4uZWAbgSYmVllBumKgat66DNfCkyveTwtL2tUZomk4cB40rw3Pa3baPlqYIKk4flov7Z8v7cREd1/V0q6ltRN0FIjwN0BZmbWTn4LHB7whaoAACAASURBVJFH7Y8kDcKbXVdmNnBevv8a4Kd53pvZwNl5ZP9hwBHAb5rFzOv8LMcgx7xuINuQNEbSWABJY4DTgLtb3RnOBJiZWSXS3AG79joBEdEp6d3A9UAHcGlE3CPpU8C8iJhNmiH3m5IWAGtIP+rkct8G7gU6gXdFRBdAo5h5kx8Erpb0GeB3OTb93YakA4Br88S8w4ErI+JHre4PDcVJ/fY+Ykoc+W9vaTnOxL23FKgNLP359N4L9cFxM+8rEmdz58iWYyxcO7FATWDK2I1F4nQM214kzgO/O7hInCOf+4cicR785SFF4mw9sLP3Qn0wYp+tReKgMv83hg9v/X3vnD+2QE1g9DPXFYnTdcu+ReJsPqTMe/6xl9Qf5PbfBX9+O4vu3lj813rSM/eLMy+rH5jfum+deMmtpU6h29M5E2BmZpXpwyl9Nog8JsDMzKxNORNgZmaV8NwB1XMjwMzMKjNIpwhaH3nvm5mZtSlnAszMrBqx608RtJ05E2BmZtamnAkwM7NKBD5FsGpuBJiZWWXcHVAtdweYmZm1KWcCzMysEr5OQPWcCTAzM2tTzgSYmVllnAmolhsBZmZWiSqmEraduTvAzMysTTkTYGZmlfF1Aqo1JBsBXVs7WLN0Qstx1owYV6A2MOa564vEmfvgYUXinP3seS3HGDdyS4GawOJN+xaJs2zu9CJxhh1W5nUt+d9Di8TZ+oxtReKcdPSCInGuPOxnReJ8ZMVzisS56vYXtByj4+DHC9QEpozdWCTO0//soSJxfvKj5xaJ84W7Xt5yjEe2LCxQExuKhmQjwMzM2kB4YGDVPCbAzMysTTkTYGZmlfDFgqrnRoCZmVXGjYBquTvAzMysTbWcCZC0CNgIdAGdETGj7nkBXwTOBDYDb4qI21rdrpmZ7d58saDqleoOeGlErGry3BnAEfl2AvDV/NfMzMwqtCvGBJwFXBERAcyVNEHSlIhYvgu2bWZmQ1g4E1CpEmMCArhB0q2SZjV4fiqwuObxkrzMzMza3HZU/GZ9VyIT8KKIWCppf+BGSfdHxM39DZIbELMAOia2frVAMzMz61nLmYCIWJr/rgSuBY6vK7IUqL0m7LS8rD7ORRExIyJmdOwzptVqmZnZEBf5ioGlb9Z3LTUCJI2RNLb7PnAacHddsdnAuUpOBNZ7PICZmVn1Wu0OOAC4Np0FyHDgyoj4kaR3AETE14A5pNMDF5BOEXxzi9s0M7M9hAcGVqulRkBELASObbD8azX3A3hXK9sxM7M9kdP3VfMVA83MzNqU5w4wM7PKuDugWs4EmJmZtamhmQnoEsPXdbQc5uAZjxSoDAzX9iJx/nDnwUXifGf481qOcd4xcwvUBB7vGlEkzqIDtxWJc+C+G4vE4dQycTbfs1+ROHPnP61InMPuf3qROHtP2FIkDttbPwrs2lzm39jCFZOLxHnwd9OKxOlog0M0TyVcvTb4mJmZmVkjQzMTYGZme75IFwyy6rgRYGZmlfG1/qvl7gAzM7M25UyAmZlVIvApglVzJsDMzKxNORNgZmYV8WWDq+ZGgJmZVcZnB1TL3QFmZmZtypkAMzOrjAcGVsuZADMzszblTICZmVUiwpmAqrkRYGZmlfHZAdVyd4CZmVmbciPAzMwqE1H+1htJMyU9IGmBpA81eH6UpGvy87dIOrTmuQ/n5Q9IOr23mJIOyzEW5JgjS2+jFW4EmJlZ25DUAXwFOAM4GjhH0tF1xd4KrI2Iw4ELgc/ldY8GzgaeBcwE/kNSRy8xPwdcmGOtzbFLb2PA3AgwM7PKRKj4rRfHAwsiYmFEbAWuBs6qK3MWcHm+/13gFEnKy6+OiCci4iFgQY7XMGZe52U5Bjnmq0tuo887uomhOTCwI+gc39VymEV3HlSgMhCTtxaJM/qYDUXiaMHYlmN8Y/tJBWoCWjGqSBzGbC8SZuWqcUXiDFs6ukic7XuVuRxabC8zeEqbO4rEiYXji8ThkG0th9hnv8cKVAR+/YJLi8R5z5JTi8R5xpgVReJ8e9FzW44xbFiZ72e9oE8/2gMxWdK8mscXRcRF+f5UYHHNc0uAE+rWf7JMRHRKWg9Mysvn1q07Nd9vFHMSsC4iOhuUL7WNlgzNRoCZmdnArYqIGVVXYnfgRoCZmVWmgqkDlgLTax5Py8salVkiaTgwHljdy7qNlq8GJkganrMBteVLbaMlHhNgZmbt5LfAEXnU/kjSILzZdWVmA+fl+68BfhoRkZefnUf2HwYcAfymWcy8zs9yDHLM60puo9Wd4UyAmZlVo4IrBub+93cD1wMdwKURcY+kTwHzImI2cAnwTUkLgDWkH1xyuW8D9wKdwLsiogugUcy8yQ8CV0v6DPC7HJvC2xgwNwLMzKytRMQcYE7dso/X3H8ceG2TdS8ALuhLzLx8IWlkf/3yYttohRsBZmZWnQoGBdgObgSYmVllPIFQtTww0MzMrE05E2BmZpXpy7X+bfA4E2BmZtamnAkwM7NKBB4TUDU3AszMrBoBuBFQKXcHmJmZtSlnAszMrDIeGFitAWcCJB0p6faa2wZJ760rc7Kk9TVlPt4snpmZme1aA84ERMQDwHEAkjpIsxld26DoLyLiFQPdjpmZ7cGcCahUqe6AU4AHI+LhQvHMzGyPJ58dULFSjYCzgauaPHeSpDuAZcD7+zLrUcfILiZOW9dypdatH9NyDICOZaOLxHn8wCJhYMoTLYco9Zo6J20rEmf0uNZfE8DEsY8VifPo0jL757JXfq1InDf/6s1F4gzff2uRONsmFRpT/NiIlkPsM7rMZ+fY77y390J9MHJamc/g/3FYkTiPbxjVcoyubR0FamJDUcuNgDyv8auADzd4+jbgkIjYJOlM4HukuZEbxZkFzAIYsd+4VqtlZma7A3cHVKpEc/4M4LaIWFH/RERsiIhN+f4cYISkyY2CRMRFETEjImYMH793gWqZmZlZT0p0B5xDk64ASQcCKyIiJB1PanSsLrBNMzPb3YWvGFi1lhoBksYALwf+qmbZOwAi4mvAa4C/ltQJbAHOjvBZoWZmZkNBS42AiHgMmFS37Gs1978MfLmVbZiZ2R7Mh4WV8hUDzcysQu4OqJLnDjAzM2tTzgSYmVl13B1QKWcCzMzM2pQzAWZmVh1nAirlRoCZmVUjAF8noFLuDjAzM2tTzgSYmVllfPm4ajkTYGZm1qacCTAzs+o4E1ApNwLMzKw6HhhYKXcHmJmZtakhmQmIEE9sG9FynDOOvLdAbWDO+uOKxDlq+iNF4mx4YnTLMU457oECNYEr73lBkTiPP7pXkThrVSa3uG3ytiJxZs17Y5E4R0xdWSTO7++dViROjOksEmfEPltbjrFy1bgCNYEJh68pEufFBy0sEud/HzimSBxt6Wg9yCAerRf6ytoAORNgZmbWpoZkJsDMzNpA4IGBFXMmwMzMrE05E2BmZhWRzw6omBsBZmZWHXcHVMrdAWZmZm3KmQAzM6uOMwGVcibAzMysTTkTYGZm1XEmoFJuBJiZWTUCnx1QMXcHmJmZtSlnAszMrDKeO6BazgSYmZm1KWcCzMysOs4EVMqZADMzszblRoCZmVmbcneAmZlVxgMDqzUkGwERorOz9STFoscmFqgNTHn6o0XiLHhkvyJxRu+1teUYV907o0BNoGN4V5E4TGj9NQEMmzeuSJxDXrasSJyHHyrznv9h2L5F4jB2W5Ew5x53S5E4d60/qOUYi648vEBN4PGXl/kM/n7D/kXiHDhxQ5E4yx4tUJ82+qGWNBG4BjgUWAS8LiLWNih3HvCx/PAzEXF5Xv584DJgL2AO8J6IiGZxJQn4InAmsBl4U0TcNsBtfAJ4O9D9o/WRiJjT0+t1d4CZmVUnVP7Wmg8BP4mII4Cf5Mc7yT/o5wMnAMcD50vqbql/lfRDfES+zewl7hk1ZWfl9Qe6DYALI+K4fOuxAQBuBJiZmdU6C7g8378ceHWDMqcDN0bEmpwluBGYKWkKMC4i5kZEAFfUrN8s7lnAFZHMBSbkOAPZRr+5EWBmZtWIQbrBZEnzam6z+lGrAyJieb7/CHBAgzJTgcU1j5fkZVPz/frlPcXtKVZ/twHwbkl3Srq0JnPQ1JAcE2BmZm1icMYbrIqIpgOfJP0YOLDBUx+tfZD72YvXcLDikroJPk3aq58G/gV4S08ruBFgZmZtJSJObfacpBWSpkTE8px6X9mg2FLg5JrH04Cf5+XT6pYvzfebxV0KTG+wTr+3EREral7H14H/bfY6u/WpOyCnFVZKurtm2URJN0qan/82TDtIOi+XmZ9HOpqZmQHpFMHStxbNBrp/q84DrmtQ5nrgNEn75t++04Drc7p/g6QT86j/c2vWbxZ3NnCukhOB9TlOv7eRGxfd/hS4m170dUzAZew8+hBaH0FpZmY21HwWeLmk+cCp+TGSZki6GCAi1pDS7b/Nt0/lZQDvBC4GFgAPAj/sKS7pFL+FufzX8/oD3cbnJd0l6U7gpcDf9fZi+9QdEBE3Szq0bvFZ7EhVXE5KU3ywrsyToxsBJN1Iakxc1ZftmpnZHm6IXYMgIlYDpzRYPg94W83jS4FLm5Q7ph9xA3hXk7r0dxtvbBSnJ62MCWhlBOVT5NGbswCGTx7fQrXMzGy3McQaAe2myCmCuSXT0lsZERdFxIyImNExbkyJapmZmVkPWmkErOgehNDLCMpGox7NzKzNDcagQM9F0D+tNAIGPIKyhW2amZlZIX09RfAq4NfAkZKWSHorrY+gNDOzdjf05g5oK309O+CcJk8NeASlmZmZBwZWy3MHmJmZtSlfNtjMzCrjgXzVcibAzMysTQ3JTMDeI7fyvKlLei/Yi1/ffXiB2sBNZ1xYJM5r7npzkTir509qOcb0ox8pUBN4ZN3YInE6hncVifPExDKHFY9uLHStiiE2SEnDyuyf/7r7+CJxhv1hdMsxRp2+vkBNYPhNE4rE+f20Mt+JsQsLHaM9s8x3a9A4E1ApZwLMzMza1JDMBJiZWRvwxX0q50aAmZlVx42ASrk7wMzMrE05E2BmZtVxJqBSzgSYmZm1KWcCzMysMh4YWC1nAszMzNqUGwFmZmZtyt0BZmZWHXcHVMqZADMzszblTICZmVXDVwysnBsBZmZWHTcCKuXuADMzszblTICZmVXHmYBKORNgZmbWppwJMDOzSggPDKzakGwEbNk2grsfPbDlOMPHbitQG3jnwtcWifPEtjK7e/hGtRxj1U8PKlATeNrpi4rEeehnhxaJE0dtLhJny6ZRReKMeajMe/7EuDJxtneWSf597MQfFImz/ti9W47x/eXPLlAT+MPzy7znFxz/vSJxPnLD64rEOeLopS3HWLPX1gI1saFoSDYCzMysTTgTUCk3AszMrBq+TkDlPDDQzMysTTkTYGZm1XEmoFLOBJiZmbUpZwLMzKw6zgRUyo0AMzOrjAcGVsvdAWZmZm3KmQAzM6uOMwGVcibAzMysTTkTYGZm1QicCaiYGwFmZlYZDwyslrsDzMzM2lSvjQBJl0paKenummVfkHS/pDslXStpQpN1F0m6S9LtkuaVrLiZme0BYhBu1md9yQRcBsysW3YjcExEPAf4PfDhHtZ/aUQcFxEzBlZFMzMzGwy9NgIi4mZgTd2yGyKiMz+cC0wbhLqZmdkeTlH+Zn1XYkzAW4AfNnkugBsk3SppVoFtmZmZWSEtnR0g6aNAJ/BfTYq8KCKWStofuFHS/Tmz0CjWLGAWQMfk8Ty2aXQrVQNg4oTHWo4B8OJJC4rE2Taxo0icOydPbTnGgjWTC9QE5t9ySJE40168tEicdZv3KhKnlHVPK/Oej3i4zOsaoSJhuHbFc4vEeWj1xJZjbHtgXIGawPaJXUXifOLqc4rEGX7kpiJxHvxd64naJzaPLFCTJnzkXqkBZwIkvQl4BfD6iGj4NkbE0vx3JXAtcHyzeBFxUUTMiIgZHWPHDLRaZma2uxiMQYFuVPTLgBoBkmYCHwBeFRGbm5QZI2ls933gNODuRmXNzMxs1+vLKYJXAb8GjpS0RNJbgS8DY0kp/tslfS2XPUjSnLzqAcAvJd0B/Ab4QUT8aFBehZmZ7XY0SLeW6iRNlHSjpPn5775Nyp2Xy8yXdF7N8ufnU+MXSPqSJPUUV8mXcvk7JT2vD9u4QNJiSZvq6jRK0jU51i2SDu3t9fbl7IBzImJKRIyIiGkRcUlEHB4R0/Opf8dFxDty2WURcWa+vzAijs23Z0XEBb1ty8zMrGIfAn4SEUcAP8mPdyJpInA+cAKpm/v8msbCV4G3A0fkW/cp9s3inlFTdlZev7dtfJ/G3etvBdZGxOHAhcDnenuxvmKgmZlVZ+iNCTgLuDzfvxx4dYMypwM3RsSaiFhLunbOTElTgHERMTePlbuiZv1mcc8CrohkLjAhx2m4DYAcf3kvdf8ucEp3JqIZzx1gZmaVGYLn9R9Q8wP7CKlru95UYHHN4yV52dR8v355T3F7itVoeU+eXCciOiWtByYBq5qt4EaAmZntaSbXXar+ooi4qPuBpB8DBzZY76O1DyIipPLNlMGKOxBuBJiZWXUG56dwVU+Xqo+IU5s9J2mFpCkRsTyn5Vc2KLYUOLnm8TTg53n5tLrl3RdBaRZ3KTC9wTrNttGT7lhLJA0HxgOre1rBYwLMzMx2mA10j8Q/D7iuQZnrgdMk7ZsH650GXJ/T/RsknZj74s+tWb9Z3NnAufksgROB9TlOw230o+6vAX7a7Do+3dwIMDOz6gy9gYGfBV4uaT5wan6MpBmSLgaIiDXAp4Hf5tun8jKAdwIXAwuAB9lxWf2GcYE5wMJc/ut5/R63IenzkpYAe+dT9z+RY10CTJK0APh7GpzZUM/dAWZmVo0hOOFPRKwGTmmwfB7wtprHlwKXNil3TD/iBvCuJnVpto0PkC7YV7/8ceC1jWI140yAmZlZm3ImwMzMqjPEMgHtxpkAMzOzNuVMgJmZVWaojQloN84EmJmZtSlnAszMrDrOBFRqSDYChg0LxuzzeMtxxozcWqA2cMn3m15cql8OmtFovof+W7VpTMsx9h5VZt90TuosEmfD46OKxHns3oazflZGY7cXiROHbCkS50VPe7BInBVbxhaJU0LnQWU+y/vtt6FInDFPL1OfdZv3KhJn+FGt/y9dMbrM97wRdwdUy90BZmZmbWpIZgLMzKwNlLnCn7XAmQAzM7M25UyAmZlVx5mASrkRYGZmlRAeGFg1dweYmZm1KWcCzMysOs4EVMqZADMzszblTICZmVVG4VRAldwIMDOzavg6AZVzd4CZmVmbcibAzMwq41MEq+VMgJmZWZtyJsDMzKrjTECl3AgwM7PKuDugWu4OMDMza1NDMhOwfWsHmxaPaznOprF7F6gNDDvk8SJxDh67pkicP/z+gJZjbNtvS4GawIjVZT5C60aPKRInpj5RJM6U/dcVifPI/fsXidO1taNInHn/8+wicbbsv71InPFHtv6d2D6mzLHMhlsnF4nz6OSuInGGjdtWJM6IkZ0tx+jaPojHi84EVMqZADMzszY1JDMBZmbWBsJjAqrmTICZmVmbcibAzMyq40xApdwIMDOzSgh3B1St1+4ASZdKWinp7ppln5C0VNLt+XZmk3VnSnpA0gJJHypZcTMzM2tNX8YEXAbMbLD8wog4Lt/m1D8pqQP4CnAGcDRwjqSjW6msmZntYSLK36zPem0ERMTNwEBO5j0eWBARCyNiK3A1cNYA4piZmdkgaOXsgHdLujN3F+zb4PmpwOKax0vyMjMzMyCNCSh9s74baCPgq8DTgeOA5cC/tFoRSbMkzZM0r2vTplbDmZnZUBeDdLM+G1AjICJWRERXRGwHvk5K/ddbCkyveTwtL2sW86KImBERMzr22Wcg1TIzM7N+GNApgpKmRMTy/PBPgbsbFPstcISkw0g//mcDfzmgWpqZ2R5JZaahsAHqtREg6SrgZGCypCXA+cDJko4jJV4WAX+Vyx4EXBwRZ0ZEp6R3A9cDHcClEXHPoLwKMzMz67deGwERcU6DxZc0KbsMOLPm8RzgKacPmpmZAe7Dr5ivGGhmZpXxaP5qeQIhMzOzNuVMgJmZVSPwFf4qNiQbAcNHdTLpsLUtx9n0m8kFagPbjtpcJM7chw8tEmefaRtajjFu9BMFagJPe8miInHuWFHmOlIbF48rEmfZlklF4px60l1F4sz75rFF4ox5xSNF4hw4YluROAuXtf4d3WvM1gI1gc6jylyfZFSRKLC9y4laG3xDshFgZmbtwWMCquWmppmZWZtyJsDMzKrjTECl3AgwM7NKCHcHVM3dAWZmZm3KmQAzM6tGhE8RrJgzAWZmZpmkiZJulDQ//923Sbnzcpn5ks6rWf58SXdJWiDpS5LUU1wlX8rl75T0vD5s4wJJiyVtqqvTmyQ9Kun2fHtbb6/XjQAzM6uMovytRR8CfhIRRwA/yY93rrM0kTSZ3gnA8cD5NY2FrwJvB47It5m9xD2jpuysvH5v2/h+XtbINRFxXL5d3NuLdSPAzMyqE4Nwa81ZwOX5/uXAqxuUOR24MSLWRMRa4EZgpqQpwLiImBsRAVxRs36zuGcBV0QyF5iQ4zTcBkCOv7zlV4obAWZmZrUOqPmBfQQ4oEGZqcDimsdL8rKp+X798p7i9hSr0fLe/HnuVviupOm9FfbAQDMzq8wgnSI4WdK8mscXRcRFT25T+jFwYIP1Plr7ICJCKl/DwYpL6ia4KiKekPRXpIzDy3pawY0AMzPb06yKiBnNnoyIU5s9J2mFpCkRsTyn5Vc2KLYUOLnm8TTg53n5tLrlS/P9ZnGXAtMbrNNsG01FxOqahxcDn++pPLg7wMzMqhLA9ih/a81soHsk/nnAdQ3KXA+cJmnfPFjvNOD6nO7fIOnEfFbAuTXrN4s7Gzg3nyVwIrA+x2m4jZ4qnhsX3V4F3Nfbi3UjwMzMqjP0BgZ+Fni5pPnAqfkxkmZIuhggItYAnwZ+m2+fyssA3kk6Cl8APAj8sKe4wBxgYS7/9bx+j9uQ9HlJS4C9JS2R9Ikc628l3SPpDuBvgTf19mLdHWBmZpbllPopDZbPA95W8/hS4NIm5Y7pR9wA3tWkLs228QHgAw2Wfxj4cKNYzbgRYGZmlfHcAdVyd4CZmVmbGpKZgM5tHaxaPr7lOM94yR8K1AZ+v3BK74X6oGNlmd392CGPtxzjibsnFKgJdD6vTDtyw6oxReKM2G9LkThdj+xdJM6Pbz+6SJy9Tt5QJM6GR1v/XgFM2W99kTgdS0a3HGPYM1v/PgCMG7O1SJwntpX5nm/r7CgSp3NbmTiDxnMHVMqZADMzszY1JDMBZmbWHjwmoFpuBJiZWTXKnNJnLXB3gJmZWZtyJsDMzCohQB4YWClnAszMzNqUMwFmZlad7VVXoL25EWBmZpVxd0C13B1gZmbWppwJMDOzavgUwco5E2BmZtamnAkwM7OKhOcOqJgbAWZmVhlfNrhavTYCJF0KvAJYGRHH5GXXAEfmIhOAdRFxXIN1FwEbgS6gMyJmFKq3mZmZtagvmYDLgC8DV3QviIi/6L4v6V+AnuYVfWlErBpoBc3MbA/m7oBK9doIiIibJR3a6DlJAl4HvKxstczMzGywtTom4MXAioiY3+T5AG6QFMB/RsRFLW7PzMz2FAHyFQMr1Woj4Bzgqh6ef1FELJW0P3CjpPsj4uZGBSXNAmYBdEwaz7DRnS1WDebfO7XlGADPPGZxkTiLJ08oEifuH99yjI6jNhaoCajQqJ69JjxeJM7jj40sEqfUPyZtLXMWbldXmTij9tpWJM7K2w8oEmfbfq3XZ3hnR4GawJrHRheJ07W5zHjrEWOfKBJnzN6txxk2zCn7PdWA/7NIGg78GXBNszIRsTT/XQlcCxzfQ9mLImJGRMzoGDdmoNUyM7PdSUT5m/VZK4cXpwL3R8SSRk9KGiNpbPd94DTg7ha2Z2Zme5oYhJv1Wa+NAElXAb8GjpS0RNJb81NnU9cVIOkgSXPywwOAX0q6A/gN8IOI+FG5qpuZmVkr+nJ2wDlNlr+pwbJlwJn5/kLg2BbrZ2ZmezDPIlgtzx1gZmbWpnzZYDMzq44zAZVyI8DMzKoRgK8TUCl3B5iZmbUpZwLMzKwSIjwwsGLOBJiZmbUpZwLMzKw6zgRUyo0AMzOrjhsBlXJ3gJmZWZtyJsDMzKrhUwQr50yAmZlZm3ImwMzMKuNTBKvlTICZmVmbGpKZAAmGj+xqOc4JM+4rUBu45SfPKhJnyvHLi8RZvNe41oN0lmn/rVxVoC7AuHFbisR5nJFF4jzr+YuKxLlv6YFF4nR1lXm/9h27uUiczvlji8QZfdyalmMsf3R8gZqUM2ZSmX382PrRReKse6z170TXtkE8XnQmoFJDshFgZmbtINwIqJi7A8zMzNqUMwFmZlaNwJmAijkTYGZm1qacCTAzs+r4YkGVciPAzMwq4+sEVMvdAWZmZm3KmQAzM6uOMwGVcibAzMysTbkRYGZm1Qhge5S/tUDSREk3Spqf/+7bpNx5ucx8SefVLH++pLskLZD0JUnqKa6SL+Xyd0p6Xk/bkLS3pB9Iul/SPZI+W1N+lKRrcqxbJB3a2+t1I8DMzCqSrxhY+taaDwE/iYgjgJ/kxzuRNBE4HzgBOB44v6ax8FXg7cAR+Tazl7hn1JSdldfvbRv/HBFHAc8F/kjSGXn5W4G1EXE4cCHwud5erBsBZmZmO5wFXJ7vXw68ukGZ04EbI2JNRKwFbgRmSpoCjIuIuRERwBU16zeLexZwRSRzgQk5TsNtRMTmiPgZQERsBW4DpjXYxneBU7ozEc24EWBmZtUZepmAAyKie7a3R4ADGpSZCiyuebwkL5ua79cv7yluT7EaLX+SpAnAK0mZhZ1iRUQnsB6Y1OR1Aj47wMzM9jyTJc2reXxRRFzU/UDSj4FGU3x+tPZBvoD5bQAABztJREFURISk4qcvlIgraThwFfCliFg40DhuBJiZWXUG5xTBVRExo/km49Rmz0laIWlKRCzPafmVDYotBU6ueTwN+HlePq1u+dJ8v1ncpcD0Bus020a3i4D5EfFvdfWaDizJjYTxwOpmrxXcHWBmZlZrNtA92v884LoGZa4HTpO0bx6sdxpwfU73b5B0Yu6LP7dm/WZxZwPn5rMETgTW5zgNtwEg6TOkH/j39lD31wA/zWMTmnImwMzMqtF9iuDQ8lng25LeCjwMvA5A0gzgHRHxtohYI+nTwG/zOp+KiDX5/juBy4C9gB/mW9O4wBzgTGABsBl4M0CzbUiaRuq2uB+4LY/7+3JEXAxcAnxT0gJgDXB2by9WvTQSKiHpUdJO6slkYNUuqE4pu1t9Yfer8+5WX3Cdd4Xdrb4w9Op8SETsVzro+FEHxAsPen3psPxo0YW39tQdYDsMyUxAXz5skubtTm/y7lZf2P3qvLvVF1znXWF3qy/snnW23dOQbASYmVmbGILZ6HbigYFmZmZtanfOBFzUe5EhZXerL+x+dd7d6guu866wu9UXds8699/QHBjYVobkwEAzM9vzjR95QLzwgF4HsPfbj5Z8yQMD+8jdAWZmZm1qSDcCJM2U9ECeFrHRTE79njZxMEmaLulnku7NUzy+p0GZkyWtl3R7vn28irrW1WlRnvry9rpLbXY/33Sqy11N0pE1++52SRskvbeuTOX7WNKlklZKurtmWUtTlFZU5y/kKUvvlHRtvlZ5o3V7/Aztwvp+QtLSmvf+zCbr9vi/ZRfX+Zqa+i6SdHuTdXf5Pt4lht7cAW1lyDYCJHUAXyFNs3g0cI6ko+uK9XvaxEHWCbwvIo4GTgTe1aDOAL+IiOPy7VO7topNvTTXp1EKreFUl1WIiAe69x3wfNLFNa5tULTqfXwZO6YQ7dbqFKWD7TKeWucbgWMi4jnA74EP97B+T5+hwXAZT60vwIU17/2c+if7+L9lsFxGXZ0j4i9qPtP/DfxPD+vv6n1se7gh2wgg/QNcEBEL83SJV5OmSazV72kTB1NELI+I2/L9jcB91M36tJtqNtVl1U4BHoyI3i4stctFxM2kK3bVGvAUpYNW0RqN6hwRN+TZyADmsvN10SvVZB/3RV/+twyKnuqc/3e9jjQpTJsYhCyAMwH9MpQbAb1Oo8gApk3cVXLXxHOBWxo8fZKkOyT9UNKzdmnFGgvgBkm3SprV4Pm+vBdVOJvm/zCH2j6G1qYoHQrewo5LoNbr7TO0K707d19c2iSLMlT38YuBFRExv8nzQ2kflxHA9u3lb9ZnQ7kRsNuStA8prffeiNhQ9/RtpEtwHgv8O/C9XV2/Bl4UEc8jpUffJemPq65QbySN5P+3d8euUURBHMd/gwkWIoIEEQtBxD5VCm1SKESRgNpERLAzgrWgdvbaGtDYRRELwUIC/gOCYqOipUVEEqxEtIn+LHYj5+U2nDF7s95+P02O28fxeLdsZmfe7UjTkh71ONzENf5D2dTjv7lliYjrKspdCxVDmnIO3ZZ0UNK4pE+SbibNYzPOauMsQFPWGEOkyUFAVXvFnmOiz7aJdYuIURUBwILtdbU9219sfy1fP5U0GhFjA55m95w+ln9XVNTXJ7qG9PNdDNpxSa9sL3cfaOIal5bXyiixcYvSRq11RFyQdFLSuaqOZH2cQwNhe9n2D9s/Jd2pmEcT13hE0mlJD6vGNGWNtxzlgFRNDgJeSDoUEQfKu74ZFW0SO/1128Q6lTW9eUnvbN+qGLN3bd9CREyo+A7SApeI2BERO9deq2hX+aZrWFWry0yVd01NW+MOm25ROqD5rRMRU5KuSJq2/a1iTD/n0EB07VU5VTGPfq4tg3ZU0nvbS70ONmmNMVwa+8RA26sRcVnFBXCbpHu230bEDUkvbT/RJtom1uyIpPOSXnf8zOeapP2SZHtORbByKSJWJX2XNJMZuKioSz8u/2eOSLpvezEiZqXfc+7Z6jJLeRE8Julix3ud801f44h4IGlS0lhELKnY8f+vLUoz5nxV0nZJz8pz5Lnt2YjYJ+mu7ROqOIeS5jsZEeMqSi0fVJ4jnfOturbUPd+qOdueV4/9LU1Y44Hgzj0VTwwEAKTYNbrHh3ef2fLPXVyZ44mBfWpsJgAAMOxM74BkBAEAgByWij2cyNLkjYEAAKBGZAIAAHkoB6QiEwAAQEuRCQAA5OEXaqkIAgAAOWye9Z+McgAAAC1FJgAAkIdyQCoyAQAAtBSZAABAGrMnIBVBAAAgCa1/s1EOAACgpcgEAAByWDwxMBmZAAAAWopMAAAgD10EU5EJAACgpcgEAABSWJLZE5CKIAAAkMOmHJCMcgAAAC1FJgAAkIZyQC4yAQAAtBSZAABAHvYEpArz3GYAQIKIWJQ0VsNHf7Y9VcPnDh2CAAAAWoo9AQAAtBRBAAAALUUQAABASxEEAADQUgQBAAC01C/r/tWh3LIMYAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 576x576 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Build models with random parameters\n",
    "padding = 'same' # same or valid\n",
    "model_conv_bn = build_conv_bn(input_shape, n_chan_out, padding=padding)\n",
    "model_fused = build_fused(input_shape, n_chan_out, padding=padding)\n",
    "\n",
    "# Fused conv+batchnorm parameters into new conv parameters\n",
    "fuse_conv_bn_params(model_conv_bn, model_fused)\n",
    "\n",
    "# Put data through both models and measure the output delta\n",
    "y_conv_bn = model_conv_bn.predict(x)\n",
    "y_fused = model_fused.predict(x)\n",
    "\n",
    "delta = y_conv_bn - y_fused\n",
    "mae = np.mean(np.absolute(delta))\n",
    "print('Mean Absolute Error: %0.2e' % (mae))\n",
    "\n",
    "# Show delta\n",
    "delta = delta[0]\n",
    "delta = np.sum(delta, axis=2) # Sum over all output channels for plotting\n",
    "\n",
    "plt.figure(figsize=(8, 8))\n",
    "plt.subplot()\n",
    "plt.imshow(delta, aspect='equal')\n",
    "plt.colorbar()\n",
    "_ = plt.title('Output difference between Conv2D->BN layers,\\nversus parameter fusion into single Conv2D layer')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Build and test fused models: Batchnorm followed by convolution (*with* padding)\n",
    "Note that edge errors occur when padding is added to the tensor after the batchnorm, but before the convolution."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Mean Absolute Error: 3.85e-02\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeIAAAHBCAYAAAC10bg7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3dd7xcVbn/8c+XEqIBDRCkhFBULCiCGlGuDaUFfhT1KgQLRTDilXv1WkG9oNiwXxUUI8SAUi1ovAZCFAVUUALSayiaBjEJVWqS5/fHWpPsTGbOmXNmzllnzvm+X695nZld1+yZM89+1l57LUUEZmZmVsY6pQtgZmY2kjkQm5mZFeRAbGZmVpADsZmZWUEOxGZmZgU5EJuZmRW0XukCmJnZyLTPm8bE0mUrOr7da254clZETOr4hgeIA7GZmRWxdNkK/jprm45vd90t7xzX8Y0OIAdiMzMrIoCVrCxdjOJ8jdjMzKwgZ8RmZlZIsCKcETsjNjMzK8iB2MzMikjXiKPjj95ImiZpsaSbmszfXdJDkq7LjxMq8yZJul3SXEnHdeI4uGrazMyKKdRYazpwCnBWD8tcERH7VydIWhc4FdgLmA9cLWlGRNzSTmGcEY8QkkLS8/Pz0yT9T2XeByTdL+lRSZtKeq2kO/Prt5Qrdf9Jmi7pC6XLYUObpO3y/4aTkhEkIi4HlvVj1V2BuRFxd0Q8BZwHHNRueRyIWyDpCEk3SnpM0n2Svi9pbB/Wv1fSnh0sT1vbi4hjIuLzeVvrA98E9o6IDSNiKXAScEp+/cvOlLp7dEMQl/QHSU/kk6WHJF0uaafK/M/mAHNwZdp6edp2Hdj/syT9r6R/5DLclV8P2P2bkjaQdIakv0t6JFcZ7luZv7uklbk8j0qaL+kCSa8aqDJZe4JgRXT+AYyTNKfymNKP4u0m6XpJF0l6SZ42HphXWWZ+ntYWB+JeSPoo8BXg48CzgdcA2wKzJY0qWbYO2RwYDdxcmbZt3euWObMYVMdGxIbAJsAfgB/XzV8GfC5Xp/UqB7pnt7DcKOB3wEuAScCzgN2ApaSMYaCsR/oRfCPpf/EzwAV1JxYL8zHZiPS/ehtwhaQ9BrBcfaLEv70Da0lETKw8pvZx/WuBbSNiZ+C7wIAmJP4y9EDSs4DPAf8ZERdHxNMRcS9wMLAd8O683BoZVD4zn5+f/xjYBvh1Pkv/RKU6bIqkhZIWSfpYZf0+ba9J2T+et7tQ0nvr5k2X9AVJLwBuz5MflHSppLuA51a2v4GkZ+dMZJGkBXnddfO2jpD0J0nfkrQU+Gye/l5Jt0p6QNIsSdtW9h+SjsnV3w9KOlWSKvPfl9d9RNItkl6Rp28l6eeS/inpHkn/1ctHOE7S7Lydy+rK8KI8b5lSw4uD8/QpwLuAT+T3/2tJR0r6dWXdOyX9tPJ6nqRdetpunreBpK8rZZH3K10ieEb1M5b0UaVGJIskHdnL+wMgIlaQqsh2rJt1MfAU+XvagnHAPElnS9qzh2BxGOk7+NaIuCUiVkbE4oj4fETMzO/nxUpZ+4OSbpZ0YOU4TM+f+W/yZ/MXSc/L874v6evVnUn6laSPRMS/IuKzEXFv3uf/AfcAr2xwTCIi5kfECcDppJPpXuXPuvbdu1vS+yvzbpJ0QOX1+pKWSHp5fv0aSX/O7/l6SbtXlv2DpC9K+hPwGPDc/L9zd97XPZLe1UoZh5sSjbV6ExEPR8Sj+flMYH2l2p4FwITKolvnaW1xIO7Zv5GyxV9UJ+YPaCbpgn2PIuI9wD+AA3JV71crs98E7ADsDXxSLVQ397I9ILXqAz6Wy7cD0HC7EXEHKasBGBsRb46I59Vt/0lSw4blwPOBl+fyHl3Z1KuBu0nZ9RclHQR8CngbsBlwBXBu3e73B14FvIx0YrNPLvs7SMH8MFKmdSCwNAeFXwPXk6qC9gA+LGmfHg7Xu4DPkwLMdcDZeR9jgNnAOcBzgMnA9yTtmM+czwa+mt//AcBlwOslrSNpK2AUKQNE0nOBDYEbetpuLs/JwAuAXfKxHA+sao0JbEHK9MYDRwGnStq4h/dHLsOo/F6vqpsVwP8AJypdguhRRCzI5fsb8C3gHkkn5fdYtSdwce2HqkF51id9VpeQjsN/AmdLemFlscmkk9yNgbnAF/P0c4FDaidm+f3vTTrRqN/P5rm8vdXe/AJ4Rf58erOY9N18FnAk8K3aiSCpYU/1pGY/YFFE/E3SeOA3wBdINRQfA34uabPK8u8BppCy9X8C3wH2jYiNSL8117VQvmElgBVExx/tkrRF5Tu4KylWLgWuBnaQtH3+v5sMzGh3fw7EPRtHquJY3mDeojy/HZ/LZ/k3Aj8CDm1zezUHAz+KiJsi4l/kLLU/8o/dfsCHc1kXk36kJ1cWWxgR342I5RHxOHAM8OWIuDUfuy8Bu1QzUuDkiHgwIv4B/J4UnCAF+K9GxNU5q5kbEX8nBe3NIuKkiHgqIu4GflhXjnq/iYjL88nEp0nXfCaQfmjvjYgf5TL/Dfg58I5GG8n7eiSX8Q3ALGChpBeRqkmviIiVPW03/1NPAf47IpZFxCP5uFTL/zRwUq55mQk8ClSDV73vSHowl+1YUmCrL/sM0o/+0fXzmrzX+yLi6xGxE+lEaixwVc7ods6LbUr6/jfzGtLJycn5s7oU+D/W/H5fGBF/zd+Ps1n9+V9B+n1+fX79duDKiFhY3UEO9mcDZ0bEbb28rYWA8nvpUUT8JiLuyt+9y0gnE7Wy/ATYT6mmDFJgrV0OeDcwMyJm5mx9NjCH9L9TMz0ibs7veTmwEnippGdExKKI6NflIOs7SecCVwIvzDVRRynV0h2TF3k7cJOk60knTJPzd2I56X9tFnArcEEnPjdfz+vZElL15noNgvGWeX47qhf9/w7s1GzBPtoKuKZu2/21LbA+sEira4/XYc2yz2uwzrclfaMyTaRMr1aW+yrzHiP9cEOq9rmrSTm2yoGnZl3SD3czq8oVEY9KWkY6NtsCr67b1nqsfY216jJgd1ImexnwICkI75Zf18rYbLubAc8ErqkcR+X3ULO07ntWPS6N/FdEnJ5rC14LzJD0xoi4oW65z5BO9Fa9P0nbAKtuucjXVevdSaqBmAi8iNWBbCnp+9/MVsC8fHJS83fWbNTS8POPiJB0HiloXw68kxQAV8nv98ekavdjeyhHzXhScH+wtwWVGn+dSMq01yF9Zjfmsi3MVcv/LulCYF/gQ3nVbUknXAdUNrc+6SSzpvp9/JekQ0iZ8xl5ux9t4aRi2OlEVXJfRUSPSU9EnEK6vanRvJmkGtGOcSDu2ZXAk6TM4ILaREkbkv4JP5Un/Yv0D1uzRd12mn3TJpAak0C65lY76+/v9moWseZ1jHaGN5lHOgbjmtQMNCrPPOCLEXF2P/f3vCbT74mIHfqwrVXHIH9mm5CO8Tzgsohodmmh0fG9DDgA2J6UyT5Iqg7ejdX/sE23m4PH48BLchVwx+SAd4WkuaRq3Bvq5s/O8/6jMu0fNAjyStf+9yZdGtgXuBT4MnBR5fP/LfAFSWNyjUu9hcAESetUgvE2wB0tvqVzgUsknUy67PHWSvkEnEG6DLJfRDzdwvbeClzbpKyrSNqAVINxGPCriHha0i9JJ0w1Z5JqF9YjZeq1z3Ie8OOIeF8Pu1jjexURs4BZSu0EvkCq4Xl9oxVteHPVdA8i4iFSdd93lXpTWV+pheYFpGbrtQzjOlKV1SaStgA+XLep+0kNoOr9j6RnKjWNPxI4v83t1VwAHCFpR0nPJJ3h90tELCJVz31D6ZaVdSQ9T9Ibe1jtNOD4/L5QauzVsNq3gdOBj0l6pZLn5yrtvwKPSPqkpGdIWlfSS9XzrSn7SXpdvpbzeeCqiJhHqiZ9gaT35M90fUmvkvTivF6j43sZ6Zr+MyJiPikTn0Sqpv1bXqbpdnNA+iHpmuNz8nEZ38s17pZJ2o3UWKtZNdmngYYN+yrbeA7pe/0l0vXm50fE2yLi13UnYT8mBZ6fKzVOW0fp/vNPSdoP+Aspy/1EPga7k05i1rrO20iu0l9C+i7MiohqJvt94MWkNgyP9/BelI/viaTA+almy1aMAjYgVeUvz9nx3nXL/BJ4BSkTrnYG8RPgAEn75O/maKUGeFs3Kd/mkg5Sum79JOkyxMo8r9aYc7sWytzVAgbq9qWu4kDci0iNoT4FfB14mPQjMw/YI197hPTDdD1wLylonV+3mS8Dn1FqTfmxyvTLSA1Vfgd8PSIuaXN7tTJfBPwvKZuZm/+24zDSj9QtwAPAz+ihajIiLiS1Uj1P0sPATaTsqlcR8VNSw51zSNc+fwlsEqll8P6ka4n3sPqHuqfbbc4hnYQsI7WsfXfexyOkH9jJpOztvlzeDfJ6ZwA75uP7y7zOHaQfyyvy64dJDdT+lMvWynY/Sfo8rsrH5bf0fA24N6co3zNL+s58Jn/2a4mIP5FOZnryGDApIl4eEd+OiIaXXvL3fk9Sbc5s0v/FX0ltJv4SqaODA0if+RLge8Bhfax2PSfv45zahHxC9n7Sd+A+rb5fuNraeKt8PB4lNazZCdi98r/VVP78/ot0IvsAqVp8Rt0yj5Oy5u2pNOLMJ3i1Ror/JP1GfJzmv7HrAB8hfU+WkS5zfCDPm0Cqyu9ozclQtXIAHt1G0YVnD90un+neA6zfQ3WvmQ1BSv0OvyAiWr0trK/b/wzwz4j4wUBsfyjZeedRcdHMzvcBM37rRddExMSOb3iA+BqxmVmLJG1CurXsPQO1j4gY0r26dVJ06HajbueqaTOzFkh6H6nK+aJIfRWbdYQz4gIi9c6l3pYzs6EjIn5IanBnnRKwwgmxM2IzM7OSHIhtWFDq/3qJpPt6X7rpNrbJrXBbGiShj9t+VGt3FTmolEaRObwD2/HQgazqZ/2PpcvRzQK3mgYHYhtkA/HjpdRL1EeBHSOivvOTlkXEPyL1L72ic6Vbte0Nc1eZvVJl7OgOl2HfiDiz09vtK0kvkPTTfOL0kKQbJH1kIE6A6vb5K6UBQ5YpDUTywsr8z0p6WmkAhkck3SHpFEk99SBmbRMrBuDRbRyIO6zbsoRhUt5tSN1DLh7s8ljfKI2yVLsXf6eIeDapj++JpMEQBspY0j3BLyT1yvVX4Fd1y5wfaQCGTUi9cW1B6pJ0yATjbvt/tdY4EAO5t6af1U37tqTv5Od9GgZQqTeoy/LZ/hJJ5+dl16rSU+pM/+j8vOF6Dcrb2zCKu0q6MndIsSif2Y+qzA9JH5R0J6k/4dr7nSfpYUnXSHp9ZfnP5gzmJzlbuDFnGMcrDdk3T9LeleUbHi+lnqtOIw2+8Khyn8xqbXjAT+Zq5x/VHYs9SZ1KbJW3OV2VYSMry92bl60dnzn5vd4v6ZuNPh+lYRdn5AxqrlKr2eoxuUDSWfmY3Cyp6X2LqmS56nkYwFpr3Ovz+zkkT39fLsOyXKatmuxndP6clubP/2qlgTvqv2tHSPpjPu4PKA3Dt29lO9tLujyX77e5vD9pss+m/x8NfA74c0R8JPfaRkTcHhHvrPWgJenAfDwfzGV+cWVf90r6mFIW/ZCk8yWNzvNulbR/Zdn1lDLgV+QBJs6INODG06SBS14oadP6AkYadONm4BBS5xwfbfJe6o9Dw/8hpZF8HqvuS9IrctnWz697GzZ0jf/X4SKAldH5R7dxIE7OI3WHuBGs6m/3YFb36jOdPgwDSOpO8RLSEG9bkwaWbkVf12s2jOIK4L9JPR3tRhoy8D/q1n1LLndtiL6rST0WbUJ63z+t/cBlB5B6b9qY1KXjLNL3ZzxwElDtfGA6DY5XRNxKGpnpylxVWxtEoJXhATchdaw/pfomIuK3pB6cFuZtHtHsYFV8G/h2RDyL1K/1BU2WO4/U5eNWpNFYviTpzZX5B+ZlatlWw07im2g4DGBEvCHP3zm/n/PzPr9M+k5uSep1qVl3kYeTehubQOp+8xhSH9eNvJo0HvU44KukwQdq9XrnkLLGTUmjd/V03+x0ev7/qNqT1DNbQ0pjZJ9L6tZ1M1Ln+r9W5USSdBwmkXq3ehlwRJ5+LmuO8LQPafS0axvs6g3AfRGxtFlZ8iWKX9F6/88N/4ci4j7gD7ncNe8Bzsv9WbcybGj9/6sNIw7EQKRh9q5ldefybwYei4ir1L9hAJ8mjxYUEU9ERKvXRPu6XsNhFCPimoi4KpfnXlKQrO8b+ss5O3g8r/OTiFia1/kGqVvGaveLV0TErNwT2E9JPxgn5+ziPGA7SWNbPF6r5B/+3oYHXAmcGBFP9tS/cB88DTxf0riIeDQi6sfxRWm4xNcCn8yfxXWkLjUPqyz2x0jD3q0gnaTsXL+dHjQbBrCRdwHTIuLa3L3k8aRahe2avLdNSf1Er8jfhYebbPfvEfHDXP4zSUF+c6Vr7q8CTog0jOEfaTLmal8/b3ofQvEQ0vCVs/N36+vAM0jj9dZ8JyIWRsQy0rjHtWN3DnCgUv/qkLqorA9oKPX/fCqpi8neLCQF1l718j90JrmL1Xyifyir+6pvZdjQNf5fhxNfI3YgrjqH1WfT72R1NlwdBvBBperUH5AGPK+pHwbwE6T7hP+aq9je22IZ+rpe/TCKW8Gqhin/J+k+pT6Nv8TaYyevUeZc3Xdrru57kJRVVde5v/L8cVKmsaLyGtJoPq0cr6rq8IC15S/O02v+GRFPNFm/P44iZeC35arb/RsssxVQOzGo6W0ov9Fq/Rpes2EgG9mKylCWEfEoaSjC8Q2W/TGptuI8pcsWX61Vf/ZUhoh4LD/dkNXv/bHKsvXf8Zq+ft6tDKFYfa8r875bGUJxLmmM2ANyMD6QSl/VAJI2I9U6fS8i1grSDYwn9QXdq17+h35F6r98e2Av4KGIqPX9XRs2tHb8lrF62NCaZse/qwUOxOAOPap+ShphaGtSZrxbnt7nYQBzVdT7ACS9Dvit0rW/h/IizyR1lA+VIQ6brZd/YBppNozi90nVx4dGxCOSPkyqWm1Y5nwt6xOkKuybI2KlpAfoX6cjvR2v+is4S+h9eMC+XvVZYxjJnIGsCuwRcSdwqNLQhG8DftbgWuFCYBNJG1WC8TaU6Yh/IenHGgClEXs2bVSWnEV+Dvhczphnkqqfz+jD/haR3vszK8F4QpNlW/n/qPot8O/UXeuvWEhlXO5cYzKB1o97rXp6HeCW6v+OpI1JQXhGRHyxtw3l78cBucy9Ldvj/1BEPCHpAlJW/CLWHPu6lWFDu/DKp7XKGXEWEf8kXcf5EWnc21vz9D4PAyjpHVo9/NkD1NokpH0sAN6t1HjpvVTG3m22Xg/FbjaM4kakQP+opBexelSXZjYiXeP7J7CeUqf2z+plnYZaOF73A1vXrvnFwAwPeAcpO/1/ORv8DKtHQELSuyVtlvddG2JvjeMcaTSdPwNfVmoA9TJSJt2wwVKH1Q/DeC5wpKRdlMbM/RJplKN761eU9CZJO+WTj4dJVdV9urUyX6qZQ2p4OEppiMUDmizb1/+PE4F/k/Q1pSE+a40UfyJpLOl6/f+TtEf+7D5KCvR/brH455GuUX+ANUduehappuBPEXFcTxtQauT1YtJx3wL4Zgv7beV/6CzS9ewDWTMQtzNsaNdbGer4o9s4EK9praHXsj4NA0i6vvYXpeHYZgAfitX3kL6PNDzaUuAlrPkD09N6jTQbRvFjpOr1R0hBrmHr64pZpOrgO0jVgk/QXlVYT8frUtKYufdJqg2z19HhASONI/0fpGu6C0gZcrUV9STg5nycvw1MbnLt7VBgO1KWdiHpOnWv2VEHfBY4M1dVHpz3+T+k4fcWkU7eml2D3YJ0vB8mVdNexpo/+q16F6lWaClp0PrzSQGxkZb/PyLirrzd7UifwUOk9zUHeCQibidljd8l1ZYcQBp7+KlWCp1PDK4kXVOufu/fSvr/OlKrh098VOl6eM0h+TvxEOn/bynwyohYSO96/R+KNBTlSuDafLJTm97vYUNtePAwiF1IHkbRBpnSrXS3RcSJpcvSzSRdCpwTEaeXLstQsOPLRsVP/q/fffA09cpt53kYRDPrbpJeRWo0dA+pqvcg0m1m1k/5mL6CdCwNCMQKV8w6EJtZQ1sAvyA1CpsPfCAi/la2SN1L0pmke4E/VNcS38yBuBuFh1G0ARYRvybdo2sdEBFtD7YxXHVj46pOc52AmZlZQc6IzcysiFqHHiPdkAzE4zZZN7ab0KwzoNbd/vf6zqTMzEaWdR57uu1tPL7iYZ5a8fgAREyxIlwxOyQD8XYT1uevs5p15NO63Y9+X+8LmZkNY2NuaOU26J79+b5WegO1/hqSgdjMzIa/1HWgM2IfATMzs4KcEZuZWTFurNVmRixpkqTbJc2VtFZH6pI2kHR+nv8XNR4/1czMbMTqd0acR3c5lTS25nzgakkzIuKWymJHAQ9ExPMlTSZ1bH5IOwU2M7PhIcKtpqG9jHhXYG5E3J1HRjmPtftQPQg4Mz//GbBHHl/UzMyMlajjj27TTiAez5rDfM3P0xouk0cJeojUd62ZmZkxhBprSZoCTAHYZvyQKZaZmQ2Q1LOWq6bbOQILgGqvG1vnaQ2XkbQe8GzSYNtriYipETExIiZutum6bRTLzMyse7STel4N7CBpe1LAnQy8s26ZGcDhwJXA24FLIyLa2KeZmQ0bbqwFbQTiiFgu6VhgFrAuMC0ibpZ0EjAnImYAZwA/ljSXNMj45E4U2szMup971krauhgbETOBmXXTTqg8fwJ4Rzv7MDMzG87cKsrMzIpZEd13u1GnuU7AzMysIGfEZmZWRCDfvsQQDcT3PrUhR/7j9W1v54mNfRuUmY1sG8yvv6u07yKe7kBJGltZoNW0pGnA/sDiiHhpg/nvAj4JCHgE+EBEXJ/n3ZunrQCWR8TEdsvjUxEzMxtppgOTeph/D/DGiNgJ+DwwtW7+myJil04EYRiiGbGZmQ1/pXrWiojLexoNMCL+XHl5FanDqgHjjNjMzIabcZLmVB5T2tjWUcBFldcBXCLpmja3u4ozYjMzKyLQQN2+tKQT1caS3kQKxK+rTH5dRCyQ9BxgtqTbIuLydvbjjNjMzKyOpJcBpwMHRcSqMRIiYkH+uxi4kDQkcFsciM3MrJiVrNPxR7skbQP8AnhPRNxRmT5G0ka158DewE3t7s9V02ZmVkQERQZ9kHQusDvpWvJ84ERg/VSmOA04AdgU+J4kWH2b0ubAhXnaesA5EXFxu+VxIDYzsxElIg7tZf7RwNENpt8N7Nzp8jgQm5lZIWIl7mva14jNzMwKckZsZmZFBGWuEQ81DsRmZlaMB31w1bSZmVlRzojNzKyIQKwcmJ61uoozYjMzs4KcEZuZWTG+RuxAbGZmhQSw0q2mfSpiZmZWkjNiMzMrRKxwz1rOiM3MzEpyRmxmZkX4GnHiI2BmZlaQM2IzMyvG14gdiM3MrJAIuWoaV02bmZkV5YzYzMyK8TCIzojNzMyKckZsZmZFBLDSjbUciM3MrBS5ahpXTZuZmRXljNjMzIpIPWu5atoZsZmZWUH9DsSSJkj6vaRbJN0s6UMNltld0kOSrsuPE9orrpmZDScrWKfjj27TTtX0cuCjEXGtpI2AayTNjohb6pa7IiL2b2M/ZmY2DAVy1TRtZMQRsSgirs3PHwFuBcZ3qmBmZmYjQUcaa0naDng58JcGs3eTdD2wEPhYRNzcZBtTgCkAY7YY04limZnZELeyC6uSO63tIyBpQ+DnwIcj4uG62dcC20bEzsB3gV82205ETI2IiRExcfTY0e0Wy8zMrCu0FYglrU8KwmdHxC/q50fEwxHxaH4+E1hf0rh29mlmZsNDBKwIdfzRbdppNS3gDODWiPhmk2W2yMshade8v6X93aeZmdlw08414tcC7wFulHRdnvYpYBuAiDgNeDvwAUnLgceByRERbezTzMyGEbeabiMQR8QfoefeuiPiFOCU/u7DzMyGr3T7khtr+QiYmZkV5L6mzcysmBUeBtEZsZmZWUnOiM3MrAiPvpQ4EJuZWSFurAWumjYzMyvKGbGZmRWz0o21nBGbmZmV5EBsZmZFlOprWtI0SYsl3dRkviR9R9JcSTdIekVl3uGS7syPwztxHFw1bWZmxRRqrDWd1OvjWU3m7wvskB+vBr4PvFrSJsCJwERSo+9rJM2IiAfaKYwzYjMzG1Ei4nJgWQ+LHAScFclVwFhJWwL7ALMjYlkOvrOBSe2WxxmxmZkVkfqaHpDGWuMkzam8nhoRU/uw/nhgXuX1/Dyt2fS2OBCbmdlwsyQiJpYuRKtcNW1mZsWsRB1/dMACYELl9dZ5WrPpbXEgNjMzW9MM4LDcevo1wEMRsQiYBewtaWNJGwN752ltcdW0mZkVUaqvaUnnAruTriXPJ7WEXh8gIk4DZgL7AXOBx4Aj87xlkj4PXJ03dVJE9NToqyUOxGZmVkyJ25ci4tBe5gfwwSbzpgHTOlkeV02bmZkV5IzYzMzKiAG7famrOCM2MzMryBmxmZkVEXj0JXAgNjOzglw17appMzOzopwRm5lZEaXuIx5qnBGbmZkV5IzYzMyKcUbsQGxmZoUM4DCIXcVV02ZmZgU5IzYzs2J8H7EzYjMzs6KcEZuZWRnhxlrgjNjMzKwoZ8RmZlaEO/RIHIjNzKwYB2JXTZuZmRXVdkYs6V7gEWAFsDwiJtbNF/BtYD/gMeCIiLi23f2amVl3c4ceSaeqpt8UEUuazNsX2CE/Xg18P/81MzMb8QbjGvFBwFkREcBVksZK2jIiFg3Cvs3MbAgLZ8QduUYcwCWSrpE0pcH88cC8yuv5eZqZmY1wK1HHH92mExnx6yJigaTnALMl3RYRl/d1IzmITwEYs8WYDhTLzMxs6Gs7I46IBfnvYuBCYNe6RRYAEyqvt87T6rczNSImRsTE0WNHt1ssMzMb4iL3rNXpR7dpKxBLGiNpo9pzYG/gprrFZgCHKXkN8JCvD5uZmSXtVk1vDlyY7lBiPeCciLhY0jEAEXEaMJN069Jc0u1LR7a5TzMzGybcWKvNQBwRdwM7N5h+WuV5AB9sZz9mZjYcdWdVcqe5Zy0zM7OC3Ne0mZkV46ppZ8RmZmZFOSM2M7MiPAxi4ozYzNm18vIAABpCSURBVMysIGfEZmZWRqROPUY6B2IzMyumG/uG7jRXTZuZmRXkjNjMzIoIfPsSOCM2MzMryhmxmZkV4i4uwYHYzMwKcqtpV02bmZkV5UBsZmbFRKjjj95ImiTpdklzJR3XYP63JF2XH3dIerAyb0Vl3oxOHANXTZuZ2YghaV3gVGAvYD5wtaQZEXFLbZmI+O/K8v8JvLyyiccjYpdOlskZsZmZFRFRJCPeFZgbEXdHxFPAecBBPSx/KHBuh95yQw7EZmZWzMpQxx+9GA/Mq7yen6etRdK2wPbApZXJoyXNkXSVpLe0895rXDVtZmbDzThJcyqvp0bE1H5sZzLws4hYUZm2bUQskPRc4FJJN0bEXe0U1oHYzMyKGaDbl5ZExMQm8xYAEyqvt87TGpkMfLA6ISIW5L93S/oD6fpxW4HYVdNmZjaSXA3sIGl7SaNIwXat1s+SXgRsDFxZmbaxpA3y83HAa4Fb6tftK2fEZmZWzGD3NR0RyyUdC8wC1gWmRcTNkk4C5kRELShPBs6LWCNnfzHwA0krSYnsydXW1v3lQGxmZkUErd332/H9RswEZtZNO6Hu9WcbrPdnYKdOl8dV02ZmZgU5IzYzs2Lc1bQzYjMzs6KcEZuZWRkx+I21hiJnxGZmZgU5IzYzs3J8kdiB2MzMynHVtKumzczMinJGbGZmxQxQX9NdxRmxmZlZQc6IzcysiMDXiMGB2MzMSgnAgdhV02ZmZiU5IzYzs2LcWKuNjFjSCyVdV3k8LOnDdcvsLumhyjInNNuemZnZSNTvjDgibgd2AZC0LrAAuLDBoldExP793Y+ZmQ1jzog7VjW9B3BXRPy9Q9szM7NhT241Tecaa00Gzm0ybzdJ10u6SNJLOrQ/MzOzYaHtjFjSKOBA4PgGs68Fto2IRyXtB/wS2KHJdqYAUwDGbDGm3WKZmVk3cNV0RzLifYFrI+L++hkR8XBEPJqfzwTWlzSu0UYiYmpETIyIiaPHju5AsczMzIa+TlwjPpQm1dKStgDuj4iQtCsp8C/twD7NzKzbhXvWgjYDsaQxwF7A+yvTjgGIiNOAtwMfkLQceByYHOG7xszMzGraCsQR8S9g07ppp1WenwKc0s4+zMxsGHNq5p61zMysJFdNu69pMzOzgpwRm5lZOa6adkZsZmZWkjNiMzMrxxmxA7GZmRUSgO8jdtW0mZlZSc6IzcysGHfx5IzYzMysKGfEZmZWjjNiB2IzMyvIjbVcNW1mZlaSM2IzMytGrpp2RmxmZlaSM2IzMysjcGMtnBGbmZkV5YzYzMwKkVtN40BsZmYluWraVdNmZjaySJok6XZJcyUd12D+EZL+Kem6/Di6Mu9wSXfmx+GdKI8zYjMzK2eQM2JJ6wKnAnsB84GrJc2IiFvqFj0/Io6tW3cT4ERgIqnk1+R1H2inTM6IzcxsJNkVmBsRd0fEU8B5wEEtrrsPMDsiluXgOxuY1G6BHIjNzKycGIBHz8YD8yqv5+dp9f5d0g2SfiZpQh/X7RMHYjMzKyNIraY7/YBxkuZUHlP6WLJfA9tFxMtIWe+ZHX7na/A1YjMzG26WRMTEJvMWABMqr7fO01aJiKWVl6cDX62su3vdun9op6DgjNjMzApSdP7Ri6uBHSRtL2kUMBmYsUaZpC0rLw8Ebs3PZwF7S9pY0sbA3nlaW5wRm5nZiBERyyUdSwqg6wLTIuJmSScBcyJiBvBfkg4ElgPLgCPyusskfZ4UzAFOiohl7ZbJgdjMzMop0KFHRMwEZtZNO6Hy/Hjg+CbrTgOmdbI8rpo2MzMryIHYzMysIFdNm5lZMS00rhr2nBGbmZkV5IzYzMzK8TCIzojNzMxKckZsZmZltNY39LDnQGxmZuU4ELtq2szMrKSWArGkaZIWS7qpMm0TSbMl3Zn/btxk3cPzMndKOrxTBTczs+5XoK/pIafVjHg6aw9+fBzwu4jYAfhdfr0GSZsAJwKvJg3GfGKzgG1mZjYStRSII+JyUsfXVQexeozGM4G3NFh1H2B2RCyLiAdI4zrWB3QzMxupYgAeXaadxlqbR8Si/Pw+YPMGy4wH5lVez8/T1pIHbp4CMGaLMW0Uy8zMukYXBs5O60hjrYho+zwkIqZGxMSImDh67OhOFMvMzGzIaycQ318bPDn/XdxgmQXAhMrrrfM0MzMb4QaiodZwbqzVyAyg1gr6cOBXDZaZBewtaePcSGvvPM3MzMxo/falc4ErgRdKmi/pKOBkYC9JdwJ75tdImijpdICIWAZ8Hrg6P07K08zMzFJf051+dJmWGmtFxKFNZu3RYNk5wNGV19OAaf0qnZmZDW9dWJXcae5Zy8zMrCD3NW1mZsV0Y+OqTnNGbGZmVpAzYjMzK8cZsTNiMzOzkpwRm5lZGV3aAUenORCbmVk5DsSumjYzMyvJGbGZmZXjjNgZsZmZWUnOiM3MrBg31nJGbGZmVpQDsZmZWUGumjYzs3JcNe2M2MzMrCRnxGZmVoZ71gIciM3MrCQHYldNm5mZleSM2MzMynFG7IzYzMysJGfEZmZWhHBjLXBGbGZmVpQzYjMzK8cZsTNiMzMrJN9H3OlHbyRNknS7pLmSjmsw/yOSbpF0g6TfSdq2Mm+FpOvyY0YnDoMzYjMzGzEkrQucCuwFzAeuljQjIm6pLPY3YGJEPCbpA8BXgUPyvMcjYpdOlskZsZmZlRMD8OjZrsDciLg7Ip4CzgMOWqNIEb+PiMfyy6uArdt5i71xIDYzs5FkPDCv8np+ntbMUcBFldejJc2RdJWkt3SiQK6aNjOzcgamsdY4SXMqr6dGxNS+bkTSu4GJwBsrk7eNiAWSngtcKunGiLirncI6EJuZWTEDdB/xkoiY2GTeAmBC5fXWedoaJO0JfBp4Y0Q8WZseEQvy37sl/QF4OdBWIHbVtJmZjSRXAztI2l7SKGAysEbrZ0kvB34AHBgRiyvTN5a0QX4+DngtUG3k1S/OiM3MrJxBvo84IpZLOhaYBawLTIuImyWdBMyJiBnA14ANgZ9KAvhHRBwIvBj4gaSVpET25LrW1v3iQGxmZiNKRMwEZtZNO6HyfM8m6/0Z2KnT5XEgNjOzMlq73WjYcyA2M7NiPOiDG2uZmZkV1WsgljRN0mJJN1WmfU3SbbkfzgsljW2y7r2Sbsx9cs5ptIyZmY1gg9+z1pDTSkY8HZhUN2028NKIeBlwB3B8D+u/KSJ26eGeLjMzsxGr10AcEZcDy+qmXRIRy/PLAe+H08zMhqcSoy8NNZ24Rvxe1uyHsyqASyRdI2lKB/ZlZmY2rLTValrSp4HlwNlNFnld7pPzOcBsSbflDLvRtqYAUwDGbDGmnWKZmVm36MIMttP6nRFLOgLYH3hXRDQ8lJU+ORcDF5KGn2ooIqZGxMSImDh67Oj+FsvMzLrFQDTU6sLA3q9ALGkS8AlSP5yPNVlmjKSNas+BvYGbGi1rZmY2UrVy+9K5wJXACyXNl3QUcAqwEam6+TpJp+Vlt5JU6zZsc+CPkq4H/gr8JiIuHpB3YWZmXUcD9Og2vV4jjohDG0w+o8myC4H98vO7gZ3bKp2Zmdkw5y4uzcysnC68pttpDsRmZlZMN97322nua9rMzKwgZ8RmZlaOM2JnxGZmZiU5IzYzs3KcETsQm5lZIV06SEOnuWrazMysIGfEZmZWjjNiZ8RmZmYlOSM2M7NifI3YGbGZmVlRzojNzKwcZ8QOxGZmVo6rpl01bWZmVpQzYjMzKyNw1TTOiM3MzIpyRmxmZuU4I3YgNjOzMoQba4Grps3MzIpyRmxmZuU4I3ZGbGZmVpIzYjMzK0bhlNiB2MzMyvB9xICrps3MzIpyRmxmZsX49iVnxGZmZkU5IzYzs3KcETsQm5lZOa6adtW0mZmNMJImSbpd0lxJxzWYv4Gk8/P8v0jarjLv+Dz9dkn7dKI8DsRmZlZODMCjB5LWBU4F9gV2BA6VtGPdYkcBD0TE84FvAV/J6+4ITAZeAkwCvpe31xYHYjMzG0l2BeZGxN0R8RRwHnBQ3TIHAWfm5z8D9pCkPP28iHgyIu4B5ubttcWB2MzMyoh0jbjTj16MB+ZVXs/P0xouExHLgYeATVtct8/cWMvMzIabcZLmVF5PjYipxUrTCwdiMzMrZ2BaTS+JiIlN5i0AJlReb52nNVpmvqT1gGcDS1tct89cNW1mZkWIIlXTVwM7SNpe0ihS46sZdcvMAA7Pz98OXBoRkadPzq2qtwd2AP7a7nHoNRBLmiZpsaSbKtM+K2mBpOvyY78m6/bYRNzMzGww5Wu+xwKzgFuBCyLiZkknSTowL3YGsKmkucBHgOPyujcDFwC3ABcDH4yIFe2WqZWq6enAKcBZddO/FRFfb7ZSpYn4XqQL2ldLmhERt/SzrGZmNtwUGAYxImYCM+umnVB5/gTwjibrfhH4YifL02tGHBGXA8v6se1WmoibmZmNaO1cIz5W0g256nrjBvMHpJm3mZkNHwWuEQ85/Q3E3weeB+wCLAK+0W5BJE2RNEfSnCcefKLdzZmZ2VA3EL1qjZRAHBH3R8SKiFgJ/JDGPYv0qZl3REyNiIkRMXH02NH9KZaZmVnX6dd9xJK2jIhF+eVbgZsaLLaqiTgpAE8G3tmvUpqZ2bCklaVLUF6vgVjSucDupJ5K5gMnArtL2oVUCXAv8P687FbA6RGxX0Qsl1RrIr4uMC03/TYzM7Os10AcEYc2mHxGk2UXAvtVXq/VRNzMzGyVLrym22nu4tLMzIrpxlbOneYuLs3MzApyRmxmZmUERXrWGmqcEZuZmRXkjNjMzIrxNWJnxGZmZkU5IzYzs3KcETsQm5lZGcJV0+CqaTMzs6KcEZuZWRkRvn0JZ8RmZmZFOSM2M7NifI3YgdjMzEpyIHbVtJmZWUnOiM3MrBhXTTsjNjMzK8oZsZmZlRHASqfEDsRmZlaO47Crps3MzEpyRmxmZsW4sZYzYjMzs6KcEZuZWTnua9oZsZmZWUnOiM3MrBhfI3YgNjOzUgLfvoSrps3MzIpyRmxmZkUIkBtrOSM2MzMryRmxmZmVs7J0AcpzIDYzs2JcNe2qaTMzs6KcEZuZWRm+fQlwRmxmZlaUM2IzMysk3Nc0zojNzKwgRecfbZVH2kTSbEl35r8bN1hmF0lXSrpZ0g2SDqnMmy7pHknX5ccuve2z10AsaZqkxZJuqkw7v7KTeyVd12TdeyXdmJeb09u+zMzMCjsO+F1E7AD8Lr+u9xhwWES8BJgE/K+ksZX5H4+IXfKjYXysaqVqejpwCnBWbUJEVKP/N4CHelj/TRGxpIX9mJnZSDP0qqYPAnbPz88E/gB8srpARNxReb5Q0mJgM+DB/uyw14w4Ii4HljWaJ0nAwcC5/dm5mZnZELN5RCzKz+8DNu9pYUm7AqOAuyqTv5irrL8laYPedthuY63XA/dHxJ1N5gdwiaQAfhARU9vcn5mZDRcBGpietcbVXQ6dWo0/kn4LbNFgvU+vUbyIyPGrIUlbAj8GDo+I2js5nhTARwFTSdn0ST0Vtt1AfCg9Z8Ovi4gFkp4DzJZ0W86w1yJpCjAFYMwWY9oslpmZjWBLImJis5kRsWezeZLul7RlRCzKgXZxk+WeBfwG+HREXFXZdi2bflLSj4CP9VbYfrealrQe8Dbg/GbLRMSC/HcxcCGwaw/LTo2IiRExcfTY0f0tlpmZdZOIzj/aMwM4PD8/HPhV/QKSRpFi2lkR8bO6eVvmvwLeAtxUv369dm5f2hO4LSLmN5opaYykjWrPgb1bKZCZmY0gMQCP9pwM7CXpTlKcOxlA0kRJp+dlDgbeABzR4DalsyXdCNwIjAO+0NsOe62alnQuqQXZOEnzgRMj4gxgMnXV0pK2Ak6PiP1IF7gvTCcFrAecExEX97Y/MzOzUiJiKbBHg+lzgKPz858AP2my/pv7us9eA3FEHNpk+hENpi0E9svP7wZ27muBzMxs5PDoS+5Zy8zMrCj3NW1mZuU4I3YgNjOzQgIYmPuIu4qrps3MzApyRmxmZkWIcGMtnBGbmZkV5YzYzMzKcUbsQGxmZgU5ELtq2szMrCRnxGZmVoZvXwKcEZuZmRXljNjMzIrx7UvOiM3MzIoakhnxI489g8vm7Nj2drZZtqIDpTEz617aYIP2N/Kk2t9GM86Ih2YgNjOzkSAciHHVtJmZWVHOiM3MrIzAGTHOiM3MzIpyRmxmZuW4Qw8HYjMzK8f3Ebtq2szMrChnxGZmVo4zYmfEZmZmJTkjNjOzMgJY6YzYgdjMzApxz1rgqmkzM7OinBGbmVk5zoidEZuZmZXkjNjMzMpxRuyM2MzMrCRnxGZmVoZvXwKGaCB+6h/zl/z9gx//ey+LjQOW9LRAbxsYZL2WdwjqtjJ3W3nBZR4M3VZeGHpl3nZgNhsQHvVhSAbiiNist2UkzYmIiYNRnk7otvJC95W528oLLvNg6LbyQneW2fpvSAZiMzMbIdxYy421zMzMSurmjHhq6QL0UbeVF7qvzN1WXnCZB0O3lRe6s8x958ZaAChcLWBmZgU8e9Tm8W+bT+74di+e/51ruukau6umzczMChrSgVjSJEm3S5or6bgG8zeQdH6e/xdJ2w1+KdcozwRJv5d0i6SbJX2owTK7S3pI0nX5cUKJstaV6V5JN+byzGkwX5K+k4/zDZJeUaKcuSwvrBy76yQ9LOnDdcsUP8aSpklaLOmmyrRNJM2WdGf+u3GTdQ/Py9wp6fDCZf6apNvy536hpLFN1u3xOzSI5f2spAWVz36/Juv2+NsyyGU+v1LeeyVd12TdQT/GgyKi848uM2QDsaR1gVOBfYEdgUMl7Vi32FHAAxHxfOBbwFcGt5RrWQ58NCJ2BF4DfLBBmQGuiIhd8uOkwS1iU2/K5WlUnbMvsEN+TAG+P6glq4iI22vHDngl8BhwYYNFSx/j6cCkumnHAb+LiB2A3+XXa5C0CXAi8GpgV+DEZgF7AExn7TLPBl4aES8D7gCO72H9nr5DA2E6a5cX4FuVz35m/cwWf1sGynTqyhwRh1S+0z8HftHD+oN9jG0QDNlATPoRmhsRd0fEU8B5wEF1yxwEnJmf/wzYQ5IGsYxriIhFEXFtfv4IcCswvlR5Ougg4KxIrgLGStqydKGAPYC7ImKI9d0CEXE5sKxucvX7eibwlgar7gPMjohlEfEAKRA2CjYd16jMEXFJRCzPL68Cth6MsrSiyTFuRSu/LQOipzLn366DgXMHoyxDwwBkw21mxH2ouVpRqcmYUZm+fa6hnZtrO0b1ts+hHIjHA/Mqr+ezdlBbtUz+sXgI2HRQSteLXE3+cuAvDWbvJul6SRdJesmgFqyxAC6RdI2kKQ3mt/JZlDCZ5j9aQ+0YA2weEYvy8/uAzRssM1SPNcB7gYuazOvtOzSYjs1V6dOa/IgO1WP8euD+iLizyfyhdIw7I4CVKzv/aE+vNVfZ45WalwMr079CqpV5PvAAqea2R0M5EHctSRuSqpg+HBEP182+Ftg2InYGvgv8crDL18DrIuIVpKq6D0p6Q+kC9SafZR4I/LTB7KF4jNcQ6XaFrrmYJenTpEsvZzdZZKh8h74PPA/YBVgEfKNQOfrjUHrOhofKMR7uWqm5aijXaryZVEPb8vpDORAvACZUXm+dpzVcRtJ6wLOBpYNSuiYkrU8KwmdHxFrXeiLi4Yh4ND+fCawvadwgF7O+TAvy38Wk66271i3Symcx2PYFro2I++tnDMVjnN1fq9LPfxc3WGbIHWtJRwD7A++KJvc7tvAdGhQRcX9ErIiIlcAPm5RjKB7j9YC3Aec3W2aoHOOOG2JV07RWcwUwWtIcSVdJqgXbTYEHK5dzWqptGcqB+Gpgh1zfPopUDTmjbpkZQK1V6duBS5v9UAyGfDZ0BnBrRHyzyTJb1K5jS9qV9BkUO3mQNEbSRrXnwN7ATXWLzQAOU/Ia4KHKF7WUptnDUDvGFdXv6+HArxosMwvYW9LGuVp17zytCEmTgE8AB0bEY02WaeU7NCjq2i68tUk5WvltGWx7ArdFxPxGM4fSMe4S43KQrD3WqMqX9FtJNzV4rNFWoJeaq21zo7l3Av8r6Xn9LeyQ7VkrIpZLOpb0I7QuMC0ibpZ0EjAnImaQgt6PJc0lNYDo/J3hffNa4D3AjZVbED4FbAMQEaeRThg+IGk58DgwueTJA+ls78Ict9YDzomIiyUdA6vKPBPYD5hLaqV8ZKGyAqt+iPYC3l+ZVi1v8WMs6Vxgd9IPwnxSS+iTgQskHUUaHOzgvOxE4JiIODoilkn6PClYAJwUEf1pkNSpMh8PbADMzt+RqyLiGElbAadHxH40+Q4VKu/uknYh/XjeS/6OVMvb7LdloMvbrMwRcQYN2jsMhWM8KAbmX3NJTy3LI2LPZvMk3S9py4hY1EPNVbWG4m5JfyC1Cfo5qTHrejkrbqm2xT1rmZlZEc9e/znxb5v8e8e3e/Hi0/rds5akrwFLI+JkpXvMN4mIT9QtszHwWEQ8mS97XQkcFBG3SPop8POIOE/SacANEfG9nvY5lKumzcxsWIvU13SnH+05GdhL0p2kSwYnQ6q5knR6XubFwBxJ1wO/B06OiFvyvE8CH8k1tZuSam57NGSrps3MbJgLSO3qho6IWErqo6B++hzg6Pz8z8BOTda/mz42pHNGbGZmVpAzYjMzK8fDIDojNjMzK8kZsZmZleM7dxyIzcyskIhO9A3d9Vw1bWZmVpAzYjMzK8dV086IzczMSnJGbGZmxYSvETsQm5lZKR0ZtrDruWrazMysIGfEZmZWRuCetXBGbGZmVpQzYjMzK2eIjb5UgjNiMzOzgpwRm5lZEQGErxE7EJuZWSERrprGVdNmZmZFOSM2M7NiXDXtjNjMzKwoZ8RmZlaOrxGjcD+fZmZWgKSLgXEDsOklETFpALY7IByIzczMCvI1YjMzs4IciM3MzApyIDYzMyvIgdjMzKwgB2IzM7OC/j/rz8FIZiBSvgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 576x576 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Build models with random parameters\n",
    "padding = 'same' # same or valid\n",
    "model_bn_conv = build_bn_conv(input_shape, n_chan_out, padding=padding)\n",
    "model_fused = build_fused(input_shape, n_chan_out, padding=padding)\n",
    "\n",
    "# Fused conv+batchnorm parameters into new conv parameters\n",
    "fuse_bn_conv_params(model_bn_conv, model_fused)\n",
    "\n",
    "# Put data through both models and measure the output delta\n",
    "y_bn_conv = model_bn_conv.predict(x)\n",
    "y_fused = model_fused.predict(x)\n",
    "\n",
    "delta = y_bn_conv - y_fused\n",
    "mae = np.mean(np.absolute(delta))\n",
    "print('Mean Absolute Error: %0.2e' % (mae))\n",
    "\n",
    "# Show delta\n",
    "delta = delta[0]\n",
    "delta = np.sum(delta, axis=2) # Sum over all output channels for plotting\n",
    "\n",
    "plt.figure(figsize=(8, 8))\n",
    "plt.subplot()\n",
    "plt.imshow(delta, aspect='equal')\n",
    "plt.colorbar()\n",
    "_ = plt.title('Output difference between BN->Conv2D layers,\\nversus parameter fusion into single Conv2D layer')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Build and test fused models: Batchnorm followed by convolution (*without* padding)\n",
    "In this case there is no edge effect error, since padding is not used."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Mean Absolute Error: 4.14e-08\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAckAAAHMCAYAAAC+8lPCAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3deZxcVZn/8c83O0mAEMKWBcIaQATBoIi4IAxGZHEcFxhRUUdG5+eug4qO+8Ioo+IyOlEQlF1EQUQgiAIqiwHZwxLWhCQkIQSyL93P749zCypNV3d11+mu6tvf9+tVr6RreeqpW8u5z7nnnqOIwMzMzF5oSLMTMDMza1VuJM3MzGpwI2lmZlaDG0kzM7Ma3EiamZnV4EbSzMysBjeSZmbWEElnSlos6e4MsQ6VdHvVZa2kN+XIs1f5+DxJMzNrhKRXAyuBX0TEPhnjjgfmApMjYnWuuD3hStLMzBoSEdcDy6qvk7SrpCsl3SrpBkl79iL0W4A/NKuBBDeSZmbWN2YCH46IlwKfAv63FzGOA87PmlUPDWvmk5uZWflIGgscDPxKUuXqkcVtbwa+0snDnoiI11fF2AF4MXBV32bbNTeSZmaW2xBgeUS8pOMNEXEJcEkdMd4G/CYiNuROrifc3WpmZllFxLPAI5LeCqBkvx6GOZ4md7WCG0kzM2uQpPOBG4FpkuZLeh/wDuB9ku4A7gGO7UG8qcAU4Lr82faMTwExMzOrwZWkmZlZDW4kzczMavDoVjMzy+r1h46Jp5a1ZY97653rroqIGdkDd8GNpJmZZfXUsjZuuWrH7HGH7vDghOxBu+FG0szMsgqgnfZmp5GFj0mamZnV4ErSzMwyC9rClaSZmVlLkTRO0sWS7pM0R9IrGonnStLMzLJKxySbNlHN6cCVEfEWSSOA0Y0EcyNpZmbZNWPgjqQtgVcDJwJExHpgfSMx3d06wEkKSbsV//+JpP+quu2Dkp6UtFLS1pJeKenB4u83NS/r3pN0lqSvNTsPa22SphbfDRcC5TJB0uyqy0kdbt8ZWAL8XNI/JP1M0phGnnBQN5KSTpR0l6TVkhZJ+rGkcT14/KOSDs+YT0PxIuIDEfHVItZw4DvAERExNiKeIq3h9sPi79/myXrgGAgNrKQ/S1pb7Mg8I+l6SS+uuv1LxY//26quG1ZcNzXD828h6XuSHi9yeKj4u8/OT5M0UtIZkh6TtELS7ZLeUHX7ayW1F/msLCbQvkjSgX2VkzUmCNoi/wVYGhHTqy4zOzz1MOAA4McRsT+wCvhMI69l0DaSkj4J/Dfwn8CWwEHATsCsoh97oNsOGEWafb9ipw5/18175P3qQxExFhgP/Bn4ZYfblwFfljS0nmBFI7RlHfcbAfwReBEwA9gCeAXwFPCyurPvuWHAPOA1pO/i54GLOjT6C4ptsjnpu3ofcIOkw/owrx4ploMatL+pLWI+MD8ibi7+vpjUaPbaoHxDJW0BfBn4cERcGREbIuJR0iKfU4ETivttUnkUe7Tzi///EtgR+F2xd3tyVRfPSZIWSFoo6VNVj+9RvBq5/2cRd4Gk93a47SxJX5O0B3B/cfVySddKegjYpSr+SElbFnvwCyU9UTx2aBHrREl/lfRdSU8BXyquf6/SiLGnJV0laaeq5w9JHyi6dJdL+pH0/LLkkt5fPHaFpHslHVBcP1HSryUtkfSIpI908xZOkDSriHNdhxz2LG5bJun+SsWl1C3zDuDk4vX/TtJ7JP2u6rEPSvpV1d/zJL2kq7jFbSMlnVZUX08qdXtvVv0eS/qkpMXFtn5PN68PgIhoAy4A9u5w05Wk4ywn1BMHmADMk3SupMO7+CF/F+kz+M8RcW9EtEfE4oj4akRcUbyevZSq3eWS7pF0TNV2OKt4z39fvDc3S9q1uO3Hkk6rfjJJl0r6RESsiogvRcSjxXNeDjwCvLSTbRIRMT8ivgD8jLSj263iva589h6W9O9Vt90t6eiqv4dLWipp/+LvgyT9rXjNd0h6bdV9/yzp65L+CqwGdim+Ow8Xz/WIpHfUk2PZtBPZL92JiEWkz/q04qrDgHsbeR2DspEEDiZVWZusjh0RK4ErgH/qLkBEvBN4HDi66L78VtXNhwK7A0cAn1YdXajdxANA0gzgU0V+uwOdxo2IB0jVAMC4iHhdROzaIf464CxgI7AbsH+R779VhXo58DCpKv26pGOBU4A3A9sAN/DCRVGPAg4E9iXtdLy+yP2tpIb2XaQK5RjgqeIH+3fAHcAk0of6Y5Je38XmegfwVdKP/+3AucVzjAFmAecB2wLHAf8rae+iW+Zc4FvF6z+atFbdqyQNkTQRGEGqnJC0CzAWuLOruEU+pwJ7AC8ptuUk4AtV+W5PqpAmAe8DfiRpqy5eH0UOI4rXelOHmwL4L+CLSt3qXYqIJ4r8/gF8l7QY7leK11jtcNKowJU18hlOeq+uJm2HDwPnVv0gQdo2Xwa2AuYCXy+uPx94e2WnqXj9R5B2Ajo+z3ZFvt31elwCHKD6jjktJn02twDeA3y3spMG/IJNdziOBBZGxD8kTQJ+D3yNVNl/Cvi1pG2q7v9O4CRSlbsE+D7whojYnPRbc3sd+ZVKAG1E9kudKp/LO0nfyW808loGayM5gdS3vbGT2xYWtzfiy8Xe8V3Az0krbOfwNuDnEXF3RKyiqO56o/ghOhL4WJHrYtIP6HFVd1sQET+IiI0RsQb4APDNiJhTbLtvAC+pruSAUyNieUQ8DvyJ9CGF1Ph+KyL+XlQDcyPiMVKDuk1EfCUi1kfEw8BPO+TR0e8j4vqiof8c8ApJU0g/go9GxM+LnP8B/Bp4a2dBiudaUeT4auAqYIGkPUldfzdERHtXcYsf/ZOAj0fEsohYUWyX6vw3AF8peiyuAFYC1Q1LR9+XtLzI7UOkRqdj7peRfpD/reNtNV7roog4LSJeTNrJGQfcVFRClRXjtyZ9/ms5iLTjcGrxXl0LXM6mn+/fRMQtxefjXJ5//28g/Xa+qvj7LcCNEbGg+gmKhvhc4OyIuK+bl7UAUPFauhQRv4+Ih4rP3nWkhr6SyznAkUo9TJAavUoX9wnAFRFxRVHlzgJmk747FWdFxD3Fa94ItAP7SNosIhZGRK8OcVjvRMTtxfHKfSPiTRHxdCPxBmsjuZTUZdfZcbYditsbMa/q/48BExuMVzGxk9i9tRMwHFhYdCMtB/6PVCFUzOvkMadX3X8Z6UdqUtV9FlX9fzXpRxXSKuMP1chjYiVmEfcUUvVay3N5FVXPMtK22Ql4eYdY7yBVcrVcB7yW1EheRzoG+JriUlkVvau425DOw7q16rYri+srnuqwQ1a9XTrzkYgYB2xGaqAvlrRvJ/f7PGknYVTlCkk76vkBLp1WhMCDpMp9LrAnzzcyT5E+/7VMBOYVOw4Vj1HH+x8RQaoaKw3qv1L0AFTlPoTUOK0n7Rx0ZxKp4V3e3R0lvUHSTUV3+XJSIzehyG0B8FfgX5QG7r2hKredSDtD1e/9IWy6nao/j6uAt5N2KBcWXc971vFaSqcZ3a19YbAOxrgRWEfao76ocqWksaQvyCnFVavY9ETUjj+2td61KaSBBZCO8VT2lnsbr2JhEbuikWn255G2wYQaFXVn+cwDvh4R53Z25zqeb9ca1z8SEbv3INZz26B4z8aTtvE84LqIqNVd3tn2vQ44mjR0/BukH9x3kLpdf1iVY6dxix/2NcCLim7NbIrG6AZJc0ldk3d2uH1Wcdt/VF33OJ00wErHmo8gdXe/AbgW+Cbwh6r3/xrga5LGFD/2HS0ApkgaUtVQ7gg8UOdLOh+4WtKppK78f67KT8AZpJ2jIyNiQx3x/hm4rUauz5E0klT5vwu4NCI2SPotaQev4mxSVT6MVOFW3st5wC8j4v1dPMUmn6uIuAq4Sum49NdIPSOv6uyB1voGZSUZEc+QurB+IGlGcaB+KqnBnM/zXS23k7phxkvaHvhYh1BPkgbDdPRfkkZLehHp+MeFDcaruAg4UdLekkYDX6zj5XYqIhaSupz+R2nY/xBJu0p6TRcP+wnw2eJ1oTTwp9OuzE78DPiUpJcq2a3opr0FWCHp05I2kzRU0j7qenj/kZIOKY7ZfRW4KSLmkbr+9pD0zuI9HS7pQEl7FY/rbPteRzqGvFlEzCd1C84gdT3+o7hPzbhFY/FT0jGubYvtMqmbY6p1U5pSa29qH5/7HNDpIK+qGNuSPtffIB3f3C0i3hwRv+uwg/RLUqPwa6WBSkOUzq89RdKRwM2k6vDkYhu8lrSD8YLjip0puqmXkj4LV0VEdQX4Y2Av0jHzNV28FhXb94ukRu2UWvetMgIYSeqe3qh0eskRHe7zW9IoyI+SjlFWnAMcLen1xWdzlNJgrMk18ttO0rFKx0nXkbrW24vbKgP7ptaR84AW0FengPS7QdlIAhQDY04BTgOeJf0AzAMOK451QfrRuAN4lNSgXNghzDeBzxfdMJ+quv46UlfWH4HTIuLqBuNVcv4D8D1SFTC3+LcR7yL9gNwLPE0aLl2zuy0ifkMaTXiBpGeBu0lVSbci4lekQRznkY61/RYYH2kE51GkY1eP8PyPaFenLJxH2kFYRhoBeULxHCtIP37HkaqeRUW+I4vHnQHsXWzf3xaPeYD0Q3ZD8fezpMFKfy1yqyfup0nvx03FdrmGro85dueHVd2lvwQ+X7z3LxARfyXtaHRlNTAjIvaPiNMjotPDCcXn/nBSL8gs0vfiFlK35M2RZi85mvSeLwX+F3hXHccOq51XPMd5lSuKnaV/J30GFlV1F1ePCp1YbI+VwN+BFwOvrfpu1VS8fx8h7WQ+TerqvazDfdaQqs2dqRrQV+x8VQasLSH9RvwntX87hwCfIH1OlpG67T9Y3DaF1D2dtcehVbX3waUZFE1qncuo2EN8BBjeRRemmbUgSV8A9oiIek+t6Wn8zwNLIuL/+iJ+K9lvvxHxhyvyzz8xafLCWyNievbAXRisxyTNzJ4jaTzp9Jx39tVzRERLz/aUU/TslI2WNmi7W83MIE1yQepG/UNEXN/sfKy1uJLMKNKsPerufmbWOiLip6TBV5ZLQFs5CklXkmZmZrW4kbSWpzSn7FJJi7q/d80YlZPs65oUvIexV+qFU7z1K0l/kPTuDHG8xBTPzV38l2bnMVClRZfLMbrVjaRl0xc/LJJ2BD4J7B0RXc2c06WIeDzSnK1t+bJ7LvbYYoq7bqlq/c/MObwhIs7OHbenJO0h6VfFTs0zku6U9Im+2Dnp8JyXKk2Qv0xp4v1pVbd/SdIGpQnHV0h6QNIPJXU1u5A1RLT1waUZ3Ej2wEDbuy5JvjuSpnVb3N/5WM8orfhROd/4xRGxJWne3Omkyb/7yjjSeY/TSDP23AJc2uE+F0aacHw8aaae7UlTCbZMQznQvq+DRekbyWIml4s7XHe6pO8X/+/RclFKM8VcV+wlL5V0YXHfF3RTKU0e/W/F/zt9XCf5drfc1ssk3VicEL+w2CMeUXV7SPp/kh4kzdFZeb3zJD0r6VZJr6q6/5eKPf9zir3su4o9888qLe00T9IRVffvdHspzWrzE9Jk4yuV5risdxmpTxddqT/vsC0OJ53UPrGIeZaqlherut+jxX0r22d28VqflPSdzt4fpeW5Lisqj7lKIxyrt8lFkn5RbJN7JNU8N0tV1aG6Xi6qMnLyjuL1vL24/v1FDsuKnDqd61dptpdzJD1VvP9/V5qovuNn7URJfym2+9NKyzVVL2K8s9JiziskXVPke06N56z5/ejEl4G/RcQnihmdiIj7I+JfK7PrSDqm2J7Li5z3qnquRyV9Sqn6fEbShZJGFbfNkXRU1X2HKVWOBxQTqp9RTDC/gTRR/zRJW3dMsJhk/h7S/KpLSL0U3ar1HZK0vdKi7VtX3feAIrfhxd/dLS+3yfe1DAJoj/yXZih9I0maMutISZvDc3NYvo3nZ/w4ix4sF0WaBu1q0lJAk4Ef1JlHTx9Xa7mtNuDjpFlQXkFaWuo/Ojz2TUXelaWc/k6azWQ86XX/qvLjUziaNLPLVqSp2K4ifTYmAV8hTXxecRadbK+ImEOa1PnGovuxMml2PctIjSdNJH1S9YuIiGtIs7ssKGKeWGtjVTkdOD0itiDNFXtRjftdQJqqbSJpRYpvSHpd1e3HFPepVCk/fEGE2jpdLioiXl3cvl/xei4snvObpM/kDqQZWWpN8/Zu0kxEU0jT5n2ANG9sZ15OWlN0AvAt4AzpubU9zyNVW1uTVpLp6tzAs+j6+1HtcNKsTZ1SWuf0fNJ0jNuQlqX7nTZd5PxtpGkBdyYtt3Zicf35bLrayOtJK/nc1slTvRpYFBFP1cql6Ha/lPrnVO30OxRp/cI/F3lXvBO4oJgjtp7l5Tp+X62FlL6RjLQc0208P5ny64DVEXGTerdc1AaKlSsiYm1E1HsMrqeP63S5rYi4NSJuKvJ5lNSAdZxv9ZvFXvWa4jHnRMRTxWP+hzSdWvW0aTdExFXFLEG/In2ZTy32yi8ApkoaV+f2ek7xo9zdMlLtwBcjYl1Xc3b2wAZgN0kTImJlRHRcixGlZbVeCXy6eC9uJ02F966qu/0l0vJIbaQdiP06xulCreWiOvMO4MyIuK2YFu6zpGp8ao3XtjVp7tW24rPwbI24j0XET4v8zyY1wNspHeM9EPhCpOWu/kKHKdoqevp+0/1SW28nLXM2q/hsnUZa6eTgqvt8PyIWRMQy0tqVlW13HnCM0pzFkKaW69jYoDSn6o9IU8N1ZwGp0etWN9+hs3l+ofahpO9qZf7nepaX2+T7WhY+JjmwnMemS/RUqsjeLBd1MulcyFuKbqP31plDTx/X6XJbSl2hl0tapDRP6Dd44fqXm+RcdGHNKbqwlpOqkerHPFn1/zWkPfS2qr8hrSxRz/aqVs8yUksiYm2Nx/fG+0iV631Fd+RRndxnIlBptCu6W/JplOo/ZlRrubDOTKRqybNIS3891SGXil+SqvwLlLriv6Xaiy4/l0NErC7+O5bnX/vqqvt2/IxX9PT9rmeprerX2l48dz1Lbc0F5pAmGx9NqvTPq7ovSgshXw38b0S8oAHtxCTS/Krd6uY7dClpTuCdSQuiPxMRlfl061lertb2H7DSosvlaCQHy4HiX5FWu5hMqihfUVzf4+Wiiu6V9wNIOgS4RulY0zPFXUaTJoaGqqWwaj2u+PJ3ptZyWz8mdYkeHxErJH2M1F3Yac7FsZOTSd2y90REu6Sn6d2kB91tr45HDZbS/TJSPT3SsMlyY8We+3ONbkQ8CByvtITVm0lrMXY8NrUAGC9p86qGckeaM/H0AtIPKQBKq0ds3VkuRfX1ZeDLRaV5BalL9YwePN9C0msfXdVQTqlx33q+H9WuAf6FDseWqywgTUwOPNfTMIX6t3uly3UIcG/1d0fSVqQG8rKI+Hp3gYrPx9FFzt3dt8vvUESslXQRqZrck+erSKhvebmSnHZfToOikoyIJaTjBj8nrV04p7i+x8tFSXqrnl8m52kqx6jTczwBnKA0kOW9VK2fWOtxXaRda7mtzUmN8EqlxVw/WCtA1f03kgYpDFOaxHmLrh/SuTq215PA5MoxpuibZaQeIFV1byyqqM/z/GocSDpB0jbFc1eWYtpkO0da2eFvwDeVBsPsS6pAOx28klnH5brOB94j6SVK6x5+g7TixqMdHyjpUEkvLnYMniV1v/bo9LHi8MNs0iC0EUpLcR1d4749/X58EThY0reVloKrDFg7R2kx44uAN0o6rHjvPklqhP9WZ/oXkI6JfpBNVxHZglRh/zUiPtNVAKUBP3uRtvv2wHfqeN56vkO/IB0/PYZNG8lGlpcb0NpD2S/NMCgaycILlugp9Gi5KNLxnJuVlu25DPhoPH+O3PtJy+g8BbyITb/8XT2uM7WW2/oUqct4BakB6nSUbJWrSF2cD5C6utbSWPdOV9vrWtK6h4skVZZjyrqMVKS1QP+DdAzxCVJlWT3adQZwT7GdTweOq3Gs53hgKqm6+Q3puGi3VUUGXwLOLrrf3lY853+RlmlaSNqxqnXMb3vS9n6W1PV4HZv+INersqj0U6RFgS8kNVadqfv7EREPFXGnkt6DZ0ivazawIiLuJ1VbPyD1MhxNWj9yfT1JF432jaRjmNWf+38mfb/eo+eX2VqpdPy14u3FZ+IZ0vfvKeClEbGA7nX7HYq0ZFk7aRHo6i7lXi8vZ63BS2W1GHm5LetnSqcj3RcRvV7E20DStcB5EfGzZufSbHvvOyLOubzXc3/U9NKd5nmpLDPrW5IOJA0geYTUfXks6VQd66Vimx5A2paDXiDaStJR6UbSbPDZHriENEBoPvDBiPhHc1MauCSdTTrX8aMdRkxbCbiRbDHh5basj0XE70jnIFoGEdHwxPJl1KyBNrmVox42MzPrA64kzcwsq8pkAmXQr43k0DFjYvi4umaBqktkXnxn2Kq88TaOyRuPIflGIg9Zm/cD3D4672pvI5dm/oLtsCFruHVrRnR/px7Ycuzq7u/UA7m7ulYt3yxrvPZa8wS1gCF1nZBSP2VeCDFnvHUrl7Fx7ao+aM1EW5Sjo7JfG8nh48Yz5T8+ni3ehi3zfvq2vSXvZ2Xxy/OeXhMj8r3ezR/I+yu16iV5p53c+Yy878WwLzzZ/Z164IG7ak1S0ztHHpx33Myatrzv702X7ps13pqJGZf1bM/7WRkzP++P+4hn8/4ODM04ieOc3303X7CScnermZlllaYTK0clWY5XYWZm1gdcSZqZWXZlGbjjStLMzKyGhhpJSTMk3S9prqQuZ983M7PBISKNbs19aYZed7cWy/X8iLTI6Hzg75Iui4h7cyVnZmYDU7u7W3kZMDciHi6WurkAT+5rZmYl0sjAnUlsuqbafODljaVjZmYDXZpxpxxDXvr8VUg6SdJsSbPbVmWe0sbMzKwPNVJJPgFUTzsyubhuExExE5gJMGrSFK/wbGZWep6WDuDvwO6SdiY1jscB/5olKzMzG7DKNONOrxvJiNgo6UPAVcBQ4MyIuCdbZmZmZk3W0Iw7EXEFcEWmXMzMrCTavOiymZlZuXnuVjMzyypQaU4BcSNpZmbZtZdkdGs5XoWZmVkf6NdKctgamHBXvlMln9onbxu/dnzeA83DVuQ9LTSG5Hu9w1fmzW3a5Cezxhv91fVZ4912x65Z4x1zyOys8W74vwOzxnv2tWuyxpt054as8ebvszFbrOEj8sUCWLnZqKzxXrnPg1nj3bZgcrZYbTe0ZYtVzTPumJmZDQI+JmlmZlkF8ikgZmZmZedK0szMshv009KZmZl1JoLSTHBejldhZmbWB1xJmplZZqIdD9wxMzMrNVeSZmaWVVCeY5JuJM3MLDvPuGNmZtaCJA2V9A9Jlzcay5WkmZllFYj25s6481FgDrBFo4FcSZqZWWlImgy8EfhZjniuJM3MLLsmHpP8HnAysHmOYG4kzcwsq6DPFl2eIKl6nbqZETGz8oeko4DFEXGrpNfmeEI3kmZmNlAsjYjpXdz+SuAYSUcCo4AtJJ0TESf09gl9TNLMzDITbX1w6U5EfDYiJkfEVOA44NpGGkhwI2lmZlaTu1vNzCyrPjwmWX8OEX8G/txonH5tJDeMhYWHRLZ4B0+/L1ssgId+sGfWeGMX5HutAAsPyXfekTbmPYdp/uVTs8ZbfcCarPFiZHvWeL+/5sCs8WKPvJ8VMp+jNvGUuVnjrZ6Z77u29MC8760yn95369V7Z40Xe6/IGKwck5D3JVeSZmaWXT3HEAcCN5JmZpZVhJre3ZpLOV6FmZlZH3AlaWZm2ZVlqaxyvAozM7M+4ErSzMyyCqB9sA/ckTQF+AWwHWmbzIyI03MlZmZmA5VK093aSCW5EfhkRNwmaXPgVkmzIuLeTLmZmZk1Va8byYhYCCws/r9C0hxgEuBG0sxsEEsz7pSjuzVLPSxpKrA/cHMnt50kabak2W0rV+Z4OjMzs37R8MAdSWOBXwMfi4hnO95erPU1E2DkjlMyz71lZmatqImLLmfVUCMpaTipgTw3Ii7Jk5KZmQ1kgdzdKknAGcCciPhOvpTMzMxaQyOV5CuBdwJ3Sbq9uO6UiLii8bTMzGwgax/s3a0R8RcoydmiZmZmnfCMO2ZmllUEtA32Y5JmZmZl50rSzMyyK8voVjeSZmaWVToFpBwdlf3aSA4ftYHJ0xZni3fv0u2yxQLg+Kfzxsts81lbZ4u123vvyxYL4Oa7dssab+iCUVnjvejAR7PGe/AvU7PGG/tw3r3udbuszxrvjkUTs8bbYkPGeUUyT1EyatHQrPE2jsmb4NTv5mt8Fj2ZLVRpuZI0M7Ps2kpy8kM56mEzM7M+4ErSzMyyKtMqIG4kzcwss/IM3CnHqzAzM+sDriTNzCy7dg/cMTMzKzdXkmZmllWZ5m51I2lmZtl54I6ZmVnJuZI0M7Os0tyt5ehudSVpZmZWgytJMzPLzqeAmJmZlZwrSTMzy8pzt5qZmXXBp4CYmZmVnCtJMzPLK3wKiJmZWen1ayUZTw1n/S+3yxbv6Ve2Z4sFsMuv2rLGW7rvyKzxxj26MVus226Yli0WwNhlefcaj3j7TVnj/fXJXbLGG7bXs1nj7XDIsqzx7luQ73sG0L5kVNZ4qw/J990dtSjvz9jQly7PGm/Htz6QNd4Tn3hZtlgbHu6bOikozykg7m41M7Ps3N1qZmZWcq4kzcwsqzKdJ+lK0szMrAZXkmZmll1ZKsmGG0lJQ4HZwBMRcVTjKZmZ2UDmpbI29VFgToY4ZmZmLaWhRlLSZOCNwM/ypGNmZmXQjrJfmqHRSvJ7wMlA3rP6zczMWkCvj0lKOgpYHBG3SnptF/c7CTgJYMSYrXr7dGZmNlBEeQbuNFJJvhI4RtKjwAXA6ySd0/FOETEzIqZHxPRhI8c08HRmZmb9q9eNZER8NiImR8RU4Djg2og4IVtmZmY2IFUmE8h9aQafJ2lmZtmVpbs1SyMZEX8G/pwjlpmZWatwJWlmZlk1azIBSVOAXwDbkXp9Z0bE6Y3EdCNpZmZlsRH4ZETcJmlz4FZJsyLi3t4GdCNpZmbZRRMqyYhYCCws/r9C0hxgEuBG0szMWkezZu8Q5LMAABkLSURBVMipkDQV2B+4uZE4/dpIDt9mHdu+/9Fs8cb/2+hssQCWvXzbrPHGv/GJrPEWXzspW6zt91+YLRbA/EV5J4r4/e9fnjXehrGRNd6opXlXmbt/3BZZ48WIvK93hz0XZ423cu3IbLHaH8n72Ztyyoas8dpfsmfWeFvfnS+/x9fk/Zz0gwmSZlf9PTMiZna8k6SxwK+Bj0XEs408oStJMzPLKvpuxp2lETG9qztIGk5qIM+NiEsafUIvumxmZqUgScAZwJyI+E6OmK4kzcwsu2YM3CFNl/pO4C5JtxfXnRIRV/Q2oBtJMzPLrDnnSUbEXyDviCF3t5qZmdXgStLMzLJrUndrdq4kzczManAlaWZmWVWWyioDV5JmZmY1uJI0M7O8Ik0oUAZuJM3MLLtmz92ai7tbzczManAlaWZmWQU+BcTMzKz0XEmamVlmzZmWri+4kTQzs+zKMrrV3a1mZmY1uJI0M7PsPHDHzMys5Pq1kly7YiQPXLdztngjjsm7pzJ+zoas8Z64dWLWeAccdV+2WE9+dddssQCGnbguazwYmTXamF2eyRpv1bajssabdPHwrPEWHJJ3/3fEd8dnjTfp9sezxZp7et5tN2fXrbLGGzl+TdZ4L5n0ULZYw9+b+3ubRJSnknR3q5mZZVeW0a3ubjUzM6vBlaSZmWXnU0DMzMxKzpWkmZll54E7ZmZmnQhUmkayoe5WSeMkXSzpPklzJL0iV2JmZmbN1mgleTpwZUS8RdIIYHSGnMzMbIArybid3jeSkrYEXg2cCBAR64H1edIyMzNrvkYqyZ2BJcDPJe0H3Ap8NCJWZcnMzMwGphLNuNPIMclhwAHAjyNif2AV8JmOd5J0kqTZkma3rXL7aWZmA0cjjeR8YH5E3Fz8fTGp0dxERMyMiOkRMX3omDENPJ2ZmQ0Y0QeXJuh1d2tELJI0T9K0iLgfOAy4N19qZmY2UJWlu7XR0a0fBs4tRrY+DLyn8ZTMzMxaQ0ONZETcDkzPlIuZmZWE5241MzMrOU9LZ2ZmWQU+JmlmZta5AErSSLq71czMrIZ+rSSHroPNH8kYb0N7vmDA468fmjVejN6YNd7dT+6QLdaBX7ovWyyAJx7dOWu8re7Le9T/48ddkTXeF+84Jmu8JfttljXeFnPzbr95hw/PGm/E9N2yxdrm123ZYgG0D89bAa1+W97fgYWn5tt2GxaOzBarIw/cMTMzKzkfkzQzs/xKUkm6kTQzs8y86LKZmVnpuZI0M7P8StLd6krSzMysBleSZmaWlxddNjMzKz9XkmZmll9Jjkm6kTQzsz7g7lYzM7NScyVpZmb5laS71ZWkmZlZDa4kzcwsv5JUkm4kzcwsLy+6bGZmVn6uJM3MLDsvumxmZlZyriTNzCy/klSS/dpIto9rY/2xy7PFa7tpq2yxAEYuyxqOHX63IWu8ZXttmS3WU28Zky0WwE7bPJ013tq1m2WN95krj8sab6u783bCnPjR32eNl9t5j0/PGm/Fn7fLFmvB6/L+Go9ckve93fjguKzxVh6a7/Wuv60PB9d44I6ZmVlrkTRD0v2S5kr6TKPx3N1qZmbZqQndrZKGAj8C/gmYD/xd0mURcW9vY7qSNDOzsngZMDciHo6I9cAFwLGNBHQjaWZmeUUfXWCCpNlVl5M6PPMkYF7V3/OL63rN3a1mZjZQLI2IvKPIuuFG0szMMlOzRrc+AUyp+ntycV2vNdTdKunjku6RdLek8yWNaiSemZmVRN90t3bn78DuknaWNAI4DriskZfR60ZS0iTgI8D0iNgHGFokZGZm1u8iYiPwIeAqYA5wUUTc00jMRrtbhwGbSdoAjAYWNBjPzMzKoEkz7kTEFcAVueL1upKMiCeA04DHgYXAMxFxdcf7STqpMhKp7dnVvc/UzMysnzXS3boV6fyTnYGJwBhJJ3S8X0TMjIjpETF96Baje5+pmZkNHM05JpldIwN3DgceiYglEbEBuAQ4OE9aZmY2YFUWXc59aYJGGsnHgYMkjZYk4DDSgVIzM7NS6PXAnYi4WdLFwG3ARuAfwMxciZmZ2cDVjLlb+0JDo1sj4ovAFzPlYmZm1lI8446ZmeVXkkrSE5ybmZnV4EbSzMysBne3mplZdh640wtDnh7K6Au3zBZv5eRsoQDYsEV71niPvivzp6R9XbZQD121S7ZYfWHUhLzbbviKrOGIzH0w//fLN2aN1z40azjW7bkma7zhY/K9v8OX532x67fM/L0dkjeexq3PF2x43t+8MnIlaWZm+TXp5P/cfEzSzMysBleSZmaWVxPnWs3NjaSZmeVXkkbS3a1mZmY1uJI0M7PsynIKiCtJMzOzGlxJmplZfiWpJN1ImplZfiVpJN3damZmVoMrSTMzy0rhgTtmZmal50rSzMzyK8ncrW4kzcwsP3e3mpmZlZsrSTMzy84Dd8zMzErOlaSZmeXnStLMzKzcBnQlue6AVVnj7fGFFVnjrZ88Lmu8fU67I1us7x0xO1ssgGk3vCtrvI1/G5s33k5rssZ7dtTIrPHaxm3IGm/rvw3PGm+bO/LG2+Xzd2eL9dc/7ZMtFsDYx/LWDmu3yVtSxapR+YKt76M6qUSTCQzoRtLMzFpUSRpJd7eamZnV4ErSzMzycyVpZmZWbq4kzcwsu7IM3Om2kpR0pqTFku6uum68pFmSHiz+3apv0zQzM+t/9XS3ngXM6HDdZ4A/RsTuwB+Lv83MzEql20YyIq4HlnW4+ljg7OL/ZwNvypyXmZkNZNEHlybo7cCd7SJiYfH/RcB2mfIxMzNrGQ0P3ImIkGofopV0EnASwIjRPnRpZlZ6JZpxp7eV5JOSdgAo/l1c644RMTMipkfE9GGjxvTy6czMbEAZ5N2tlwHvLv7/buDSPOmYmZm1jnpOATkfuBGYJmm+pPcBpwL/JOlB4PDibzMzs6QklWS3xyQj4vgaNx2WORczM7OW4hl3zMwsK+GBO2ZmZqXnStLMzPIrSSXpRtLMzPLyeZJmZmbl16+VpNph2Lp8uxcxb3S2WAB7XXBX1ng3fvtlWeNd9reXZot1/U67ZosFMHnm8KzxFrwqazjeuOfd3d+pBy5vf3HWeMMWjMwa7+lD12aNt/buzbLGe/z2vbLFGjIl72tdsVXez/Luv1iXNd6TJ6/PFmvIqLZssV7AlaSZmVm5+ZikmZnlV5JK0o2kmZll54E7ZmZmJedG0szM8muxuVslfVvSfZLulPQbSePqeZwbSTMzGwxmAftExL7AA8Bn63mQG0kzM8urL6rIBivJiLg6IjYWf94ETK7ncR64Y2Zm2bX4wJ33AhfWc0c3kmZmNlBMkDS76u+ZETGz8oeka4DtO3nc5yLi0uI+nwM2AufW84RuJM3MLL++qSSXRsT0mk8ZcXhXD5Z0InAUcFhE1JWhG0kzMys9STOAk4HXRMTqeh/nRtLMzLJrwWOSPwRGArMkAdwUER/o7kFuJM3MrPQiYrfePM6NpJmZ5dd6lWSvuJE0M7O8MpzX2Co8mYCZmVkNriTNzCwrFZcycCVpZmZWgytJMzPLryTHJPu1kdywRTD/8HxbbsRT2UIBcO38PbLG27hj3kJ94nVt2WJt9uTobLEAHnp73o/SZjs8kzXeny48MGu80a/Im9+6ISOzxstt88fas8ZbNS3fZ3nHXw7NFgtgwSF54z34rrzv7fA5+b67sTbva63WgudJ9oq7W83MzGpwd6uZmeXnStLMzKzcXEmamVl+Jakk3UiamVleMYgG7kg6U9JiSXdXXfdtSfdJulPSbySN69s0zczM+l89xyTPAmZ0uG4WsE9E7As8AHw2c15mZjaQRR9cmqDbRjIirgeWdbju6ojYWPx5EzC5D3IzMzNrqhzHJN8LXJghjpmZlcSgOSbZFUmfAzYC53Zxn5MkzZY0u23lqkaezszMrF/1upKUdCJwFHBYRNTcZ4iImcBMgJE7TS7JvoWZmXWpJL/2vWokJc0ATgZeExGr86ZkZmYD3aDpbpV0PnAjME3SfEnvA34IbA7MknS7pJ/0cZ5mZmb9rttKMiKO7+TqM/ogFzMzK4MmnrKRm+duNTMzq8HT0pmZWX4lqSTdSJqZWVZiEA3cMTMzG6xcSZqZWX4lqST7tZEctdkGXrT3vGzxFlwwNVssgGd3HZU1XtsuG7u/Uw+snNaeLdawJXlfK+35cgNYvWRM1njT3vBY1nhLLtgxa7wNecOx2W2bZY335CFtWeONu3VktlhL980WCoCha/PGG/1Y3p/Z1btsyBYrhpWkJetDriTNzCw71Z6IbUBxI2lmZnn5PEkzM7PycyVpZmbZ+RQQMzOzknMlaWZm+ZWkknQjaWZm2bm71czMrORcSZqZWX6uJM3MzMrNlaSZmeUVPiZpZmZWeq4kzcwsv5JUkm4kzcwsKy+6bGZmNgi4kjQzs/xKslSWK0kzM7MaXEmamVl2ZTkm2a+N5Ma2ISxeNTZbvBEr874LbeuGZo231R154w1Zny/esgPas8UCGP503k6J9dtuzBpvt82XZo23cMxOWeOtH5/39e50xbqs8eZOHZU13oqDV2eLtd1vRmaLBTBm/tqs8dZsn3fbbdg838+2NipbrE140WUzM7Pyc3ermZllp7ydVU3jStLMzKwGV5JmZpZfSY5JupE0M7PsyjK61d2tZmZmNXTbSEo6U9JiSXd3ctsnJYWkCX2TnpmZDThBmnEn96UJ6qkkzwJmdLxS0hTgCODxzDmZmZm1hG4byYi4HljWyU3fBU6mNIdnzcwsF0X+SzP06pikpGOBJyLijjrue5Kk2ZJmb3w23ywbZmZmfa3HjaSk0cApwBfquX9EzIyI6RExfdgWo3v6dGZmNhBFH1wy6OlYmt6cArIrsDNwhySAycBtkl4WEYt6Ec/MzEqkVRdd7s1Ymh5XkhFxV0RsGxFTI2IqMB84wA2kmZm1uB6PpannFJDzgRuBaZLmS3pf7/MzM7PS64vTP9IpIBMqY1yKy0n1ptSTsTTVuu1ujYjju7l9ak+e0MzMrJeWRsT0WjdKugbYvpObPkcaS3NET5/Q09KZmVl2zTgmGRGHd3a9pBfTy7E0biTNzCy/Fhq4ExF3AdtW/pb0KDA9Irpdjd1zt5qZmdXgStLMzLJrxVNAKnoylqZfG8l4Zhhtl+WbC33JazZmiwUw7taRWeM9My3v0tyT9noyW6wVt3R2bLsBmfskhi3P+9G87sKXZo03fE3eXwCNyftZnvv2zbLGO2D/uVnjrXhVt71cdVt2+R7ZYgGsujrveg0jl+f9rGzYYX22WDG8hVuyFuFK0szM8gqgvRwNsBtJMzPLrxxtpAfumJmZ1eJK0szMsmvlgTs94UrSzMysBleSZmaWX5SjlHQlaWZmVoMrSTMzy64sxyTdSJqZWV6BTwExMzMrO1eSZmaWlQB54I6ZmVm5uZI0M7P88q7v0DRuJM3MLDt3t5qZmZWcK0kzM8vLp4CYmZmVnytJMzPLLEozd6sbSTMzy87T0vVC20h4Zrd88UbNG54vWB8YOXll1njtP902X6w3r8kWC6Bt/dCs8fR03vd2+KuXZY23fNnYrPG2mD0qa7xVk/P+Qt2/NN9nD2DNafl+CMZfrGyxANbsmnfbrXtV3t+BUfdsni2W1ufddmXkStLMzPIrSXerB+6YmZnV4ErSzMzyClBJZtxxJWlmZlaDK0kzM8tvsByTlHSmpMWS7u5w/Ycl3SfpHknf6rsUzcxswIk+uDRBPd2tZwEzqq+QdChwLLBfRLwIOC1/amZmZs3VbXdrRFwvaWqHqz8InBoR64r7LM6fmpmZDVSDfRWQPYBXSbpZ0nWSDsyZlJmZWSvo7cCdYcB44CDgQOAiSbtEvHDXQdJJwEkAQ7faqrd5mpnZQDLIK8n5wCWR3EJag3pCZ3eMiJkRMT0ipg8dM6a3eZqZ2UARpFYh96UJettI/hY4FEDSHsAIYGmupMzMzFpBt92tks4HXgtMkDQf+CJwJnBmcVrIeuDdnXW1mpnZ4COiNAN36hndenyNm07InIuZmVlL8Yw7ZmaW32CpJM3MzHqsJI2kJzg3MzOrwZWkmZnlVTkFpARcSZqZmdXQv5WkoH1kvt2LFx30ULZYAHfesHvWeJvP2jxrvKd3V7ZYY/82OlssgPVbZA1H+4i88VauHpk13shH8sZbs23e4zfb7vdk1njrL9wub7zd8sVa8rK2fMGAN798dtZ4v7/8oKzxtnwo32dl6LpsoV6gLKeAuJI0MzOrwcckzcwsv5JUkm4kzcwssyhNI+nuVjMzsxpcSZqZWV6BK0kzM7OycyVpZmb5lWQyATeSZmaWnc+TNDMzKzlXkmZmlp8rSTMzs3JzJWlmZnkF0F6OStKNpJmZZeYZd8zMzErPjaSZmeUXkf/SIEkflnSfpHskfauex7i71czMSk/SocCxwH4RsU7StvU8zo2kmZnl13rHJD8InBoR6wAiYnE9D3J3q5mZDQZ7AK+SdLOk6yQdWM+DXEmamVlefXcKyARJs6v+nhkRMyt/SLoG2L6Tx32O1N6NBw4CDgQukrRLRNclr7q5PStJS4DH6rjrBGBpH6fTiFbOr5VzA+fXiFbODZxfI5qV204RsU3uoFuO3C4OnviO3GG58tHv3hoR03vzWElXAv8dEX8q/n4IOCgilnT1uH6tJOt9MyTN7u2G6A+tnF8r5wbOrxGtnBs4v0a0cm4l8lvgUOBPkvYARlDHjom7W83MLL/WG7hzJnCmpLuB9cC7u+tqBTeSZmY2CETEeuCEnj6uVRvJmd3fpalaOb9Wzg2cXyNaOTdwfo1o5dx6rkRzt/brwB0zMyu/LUdsFwdvd1z2uFfO/36vB+70ls+TNDMzq6GlGklJMyTdL2mupM80O59qkqZI+pOke4t5/z7a7Jw6I2mopH9IurzZuVSTNE7SxcW8iXMkvaLZOVWT9PHifb1b0vmSRjU5nzMlLS4GGVSuGy9plqQHi3+3arH8vl28v3dK+o2kca2SW9Vtn5QUkiY0I7cih07z6828oi2tBedu7Y2WaSQlDQV+BLwB2Bs4XtLezc1qExuBT0bE3qSTUf9fi+VX8VFgTrOT6MTpwJURsSewHy2Uo6RJwEeA6RGxDzAUyN9X1DNnATM6XPcZ4I8RsTvwx+LvZjmLF+Y3C9gnIvYFHgA+299JFc7ihbkhaQpwBPB4fyfUwVl0yK/DvKIvAk5rQl7WiZZpJIGXAXMj4uFiFNIFpA9NS4iIhRFxW/H/FaQf+UnNzWpTkiYDbwR+1uxcqknaEng1cAakUWYRsby5Wb3AMGAzScOA0cCCZiYTEdcDyzpcfSxwdvH/s4E39WtSVTrLLyKujoiNxZ83AZP7PTFqbjuA7wInk4aVNE2N/Ho1r2jr6oMqcrBXkqQGZ17V3/NpsUaoQtJUYH/g5uZm8gLfI/0ItDc7kQ52BpYAPy+6gn8maUyzk6qIiCdIe+6PAwuBZyLi6uZm1antImJh8f9FwHbNTKYb7wX+0OwkKiQdCzwREXc0O5caejWvaMsKoL09/6UJWqmRHBAkjQV+DXwsIp5tdj4Vko4CFkfErc3OpRPDgAOAH0fE/sAqmttVuIni2N6xpMZ8IjBGUo/Pp+pPxUnQLTk0XdLnSIcnzm12LgCSRgOnAF9odi5dqJ5X9D9J84qquSkZtFYj+QQwpervycV1LUPScFIDeW5EXNLsfDp4JXCMpEdJXdWvk3ROc1N6znxgfkRUKu+LSY1mqzgceCQilkTEBuAS4OAm59SZJyXtAFD823JdcpJOBI4C3lHPbCb9ZFfSDtAdxfdjMnCbpM4mwm6W+cAlkdxC6g1q2uCiLNzdmt3fgd0l7SxpBGngxGVNzuk5xV7dGcCciPhOs/PpKCI+GxGTI2IqadtdGxEtUQ1FxCJgnqRpxVWHAfc2MaWOHgcOkjS6eJ8Po4UGFlW5DHh38f93A5c2MZcXkDSD1N1/TESsbnY+FRFxV0RsGxFTi+/HfOCA4nPZKirzitKTeUWt77VMI1kc8P8QcBXpB+qiiLinuVlt4pXAO0kV2u3F5chmJzWAfBg4V9KdwEuAbzQ5n+cUFe7FwG3AXaTvRVNnQJF0PnAjME3SfEnvA04F/knSg6Tq99QWy++HwObArOL78ZMWyq1l1MjvTGCX4rSQC6hzXtGWVpJK0jPumJlZVlsO3zYOHv8v2eNeufgn/T7jTqvO3WpmZgNWlGbuVjeSZmaWV0BEq52J1jstc0zSzMys1biSNDOz/ErS3epK0szMrAZXkmZmll9JzpxwI2lmZnlFNG2u1dzc3WpmZlaDK0kzM8uvJN2triTNzMxqcCVpZmbZRUmOSbqRNDOzzJo3IXlu7m41MzOrwZWkmZnlFXjGHTMzs7JzJWlmZvl5FRAzM7NycyVpZmZZBRAlOSbpRtLMzPKKcHermZlZ2bmSNDOz7MrS3epK0szMrAZXkmZmll9JjkkqSjK/npmZtQZJVwIT+iD00oiY0Qdxa3IjaWZmVoOPSZqZmdXgRtLMzKwGN5JmZmY1uJE0MzOrwY2kmZlZDf8fnBhpzbjD6RIAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 576x576 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Build models with random parameters\n",
    "padding = 'valid' # same or valid\n",
    "model_bn_conv = build_bn_conv(input_shape, n_chan_out, padding=padding)\n",
    "model_fused = build_fused(input_shape, n_chan_out, padding=padding)\n",
    "\n",
    "# Fused conv+batchnorm parameters into new conv parameters\n",
    "fuse_bn_conv_params(model_bn_conv, model_fused)\n",
    "\n",
    "# Put data through both models and measure the output delta\n",
    "y_bn_conv = model_bn_conv.predict(x)\n",
    "y_fused = model_fused.predict(x)\n",
    "\n",
    "delta = y_bn_conv - y_fused\n",
    "mae = np.mean(np.absolute(delta))\n",
    "print('Mean Absolute Error: %0.2e' % (mae))\n",
    "\n",
    "# Show delta\n",
    "delta = delta[0]\n",
    "delta = np.sum(delta, axis=2) # Sum over all output channels for plotting\n",
    "\n",
    "plt.figure(figsize=(8, 8))\n",
    "plt.subplot()\n",
    "plt.imshow(delta, aspect='equal')\n",
    "plt.colorbar()\n",
    "_ = plt.title('Output difference between BN->Conv2D layers,\\nversus parameter fusion into single Conv2D layer')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "env_tensorflow2",
   "language": "python",
   "name": "env_tensorflow2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}