tutorial/20.mnist.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Load and minibatch MNIST data\n",
    "(c) Deniz Yuret, 2019\n",
    "* Objective: Load the [MNIST](http://yann.lecun.com/exdb/mnist) dataset, convert into Julia arrays, split into minibatches using Knet's [minibatch](http://denizyuret.github.io/Knet.jl/latest/reference/#Knet.minibatch) function and  [Data](https://github.com/denizyuret/Knet.jl/blob/master/src/data.jl) iterator type.\n",
    "* Prerequisites: [Julia arrays](https://docs.julialang.org/en/v1/manual/arrays)\n",
    "* New functions: [dir](http://denizyuret.github.io/Knet.jl/latest/reference/#Knet.dir), [minibatch, Data](http://denizyuret.github.io/Knet.jl/latest/reference/#Knet.minibatch)\n",
    "\n",
    "In the next few notebooks, we build classification models for the MNIST handwritten digit recognition dataset. MNIST has 60000 training and 10000 test examples. Each input x consists of 784 pixels representing a 28x28 image. The corresponding output indicates the identity of the digit 0..9.\n",
    "\n",
    "![](http://yann.lecun.com/exdb/lenet/gifs/asamples.gif \"MNIST\")\n",
    "\n",
    "[image source](http://yann.lecun.com/exdb/lenet)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "# Load packages, import symbols\n",
    "using Knet: minibatch\n",
    "using MLDatasets: MNIST\n",
    "using ImageShow"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "28×28×60000 Array{Float32,3}\n",
      "60000-element Array{Int64,1}\n",
      "28×28×10000 Array{Float32,3}\n",
      "10000-element Array{Int64,1}\n"
     ]
    }
   ],
   "source": [
    "# This loads the MNIST handwritten digit recognition dataset:\n",
    "xtrn,ytrn = MNIST.traindata(Float32)\n",
    "xtst,ytst = MNIST.testdata(Float32)\n",
    "println.(summary.((xtrn,ytrn,xtst,ytst)));"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><tbody><tr><td style='text-align:center;vertical-align:middle; margin: 0.5em;border:1px #90999f solid;border-collapse:collapse'><img style='max-width: 100px; max-height:100px;display:inline' src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAHAAAABwCAAAAADji6uXAAAABGdBTUEAALGPC/xhBQAAACBjSFJNAAB6JgAAgIQAAPoAAACA6AAAdTAAAOpgAAA6mAAAF3CculE8AAAAAmJLR0QA/4ePzL8AAAGrSURBVGje7dk/a9VQHIfxT711EFysKDhYOzl0KSKCoILiYtuhg30L10U7dnZ3dPAddBEEQRERKuigDl1E7T+8HVREEOqghaKFOiRDKVy9aUp78uN8l5z8IQ8PX06SQ8jJycnJycnJycnJyamfvm4nJtHGV6xjBt/wsSbwwF4bxgd27XAFQ9uO/cSH/9zwC+5gLhXD+MD+bifaGME8hnEGl3Een3Fyy7Ub+I4T5f4nucM9TF+Vi48oupzDuS3H17GMBQzgFu6lYhgfWKnDf+U67uM9rmA1FcP4wF3p8DjeldtJPEjJMD6wv/4tuIlj+IGl1AzjA2vPwwt4joOKb56XqRnGB9aeh2OK/mbxOkXD+MBaHR7CNfzGbfxJ0TA+sFaH04q1xlO8StUwPnDH78NxPMQaRvX2HN0Xw/jAHc3Do7iLFp7ovb99MYwPrDwPW3iDs+go3oedlA3jAyt3eBqL5XgCj1I3jA+s9Cw9hWfleBqPm2AYH1ipwxsYLMcvsNkEw/jAnju8hKkmGsYH9tzhRRwuxx38aophfGDltcVbXNX931JyhvGBOc3PX/q9Oc17OzXKAAAAAElFTkSuQmCC\"></td><td style='text-align:center;vertical-align:middle; margin: 0.5em;border:1px #90999f solid;border-collapse:collapse'><img style='max-width: 100px; max-height:100px;display:inline' src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAHAAAABwCAAAAADji6uXAAAABGdBTUEAALGPC/xhBQAAACBjSFJNAAB6JgAAgIQAAPoAAACA6AAAdTAAAOpgAAA6mAAAF3CculE8AAAAAmJLR0QA/4ePzL8AAAHhSURBVGje7dnPi01hHMfxF80oK5lZaMpKsqAojY212DBSin9DBiWy4l+ws8BGk2kiCztTCmWBmtgobEhSIik/snge3bm3cc85ppxzvz2fuvXt+5zuu8/99D33PM+hqKioqGi1WlP3wlmsx04czb3LeIBrDYBr/7fD+MBaGd7Qy21QL7EPb7rqMD6wMsPB/F7gLrbgUO6dx6WuOowPHBu2OI0juV7CDD7gC9bhIXZhossO4wOHZjglDeoSDuDtsrVZbM/1nS47jA8cmuFtbMVnfBxYO4bxUXAYHzhWdcHrFXqnsC3Xj/Knsw7jA2vvLf7oIOak/8P3OI7FLjuMD6ycw0FNS/mRnlmb5NeKw/jARhkuYH+ur+LcKDiMD6x9L53CU0xK+4u90v6+8w7jA2vP4U0pP7ju3/JrxWF8YK0MZ7A71/dwYZQcxgdWZjiJs3p7wSfSOc3IOIwPrMzwJPbkesHqZrAVh/GBlc803/RmcLP+87aRcBgf2GhvMYHvA71PuTeODbm3ESeWXfMTZ/C1DYfxgY0yfLZCb06azU3SGerf9A4X23AYH1h5L53H4QZf+AO/cn0Lj3N9X3rnH/8n7V6GcFr/O4od+mfuCl7leh7Pu+QwPrCoqKioqKioKIJ+AzhFQPcCumLaAAAAAElFTkSuQmCC\"></td><td style='text-align:center;vertical-align:middle; margin: 0.5em;border:1px #90999f solid;border-collapse:collapse'><img style='max-width: 100px; max-height:100px;display:inline' src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAHAAAABwCAAAAADji6uXAAAABGdBTUEAALGPC/xhBQAAACBjSFJNAAB6JgAAgIQAAPoAAACA6AAAdTAAAOpgAAA6mAAAF3CculE8AAAAAmJLR0QA/4ePzL8AAAFHSURBVGje7detSgRRGMbxn19lo+BH06QILipYRMRgEwSzyRsQ78PqHdiExbSg1WrRKCaLRcMmxeJHcC5gZwbcc17OU94Jw/z58/CemaGkpKSkpKRkrO0DlvCIU5wPcf/4fxvGB062fcAGvvGSqmF8YOsO1/GOq1QN4wNbddjFCS5SNowPbNXhMjq4TNkwPrDVN80dZrDq7zxN0jA+sPEeLmITT4bvbySG8YGNO9yt5lvqhvGBjTvsVvMsdcP4wEbvwy308YxtfKZsGB/YaA/3MI0b9fobiWF8YKMO1/CDXg6G8YG1z9J5PGCAlRwM4wNr7+ExZnGdi2F8YO0OF6o5yMUwPrB2hwfV7OdiGB9Yq8MdzOVmGB9Yq8NDTOAet7kYxgcO3WEH+9V1D1+5GMYHDv1vMeVv915xhI9cDOMDS0pKSkpKSvgFnPUiCkwgYfcAAAAASUVORK5C\"></td><td style='text-align:center;vertical-align:middle; margin: 0.5em;border:1px #90999f solid;border-collapse:collapse'><img style='max-width: 100px; max-height:100px;display:inline' src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAHAAAABwCAAAAADji6uXAAAABGdBTUEAALGPC/xhBQAAACBjSFJNAAB6JgAAgIQAAPoAAACA6AAAdTAAAOpgAAA6mAAAF3CculE8AAAAAmJLR0QA/4ePzL8AAAH0SURBVGje7dmxaxRBFAbwX6KdrVdYRRBR0lmoXQQrBQ9sRMEUYmEQMfYGlLQKVkpEG/8BG5EUNkIs0sTCJoIgeDYiaiFICIhisXvscNns7SKYcZivub3Hm/3m4+PNzrwhIyMjIyMjY6LrgD24izm8xjkMOoyf/NcK0yfs7OFBrAeznceDmBWmT7i7S3IPT/43hekTtvZwHmdxbCQ+U876DVZiVJg+Yeu19Bd+18x2GBvgvOIbGZXC9AlbebiM07Z6+A0/MBXEdsWmMH3CsWvpCRxS+Bd6+BAv8B0nsVDGr2IpJoXpEzbW4X6sYq9q3RzgKRaxUeZNlXk9bOIW7uNnDArTJ2z0cPQc8RIX8LUm9zruqbw+jPcxKEyfsPW+dA2X1fsHz3ARR2NTmD7hWA+HMzo+Jm+izB3mL2I2BoXpEzZ6OGfrXnQ79HFEtfe5HYvC9AkbPey3eEEP07gZxL6o38/siML0CTv12uqwgGvB/w+4hI+xKEyf8K88XFacHUO8xauYFKZP2Hi2eIcD5fOZ8vcx9gWzHf1e5j5NXHW4hDvl83OVX6Fvo72b6BSmT9hYh2H/pa7mJvFZsX5ewSdV7yYahekTju15zyjunG6o9zDfAe84Yeu7p1OKWusrejKPysHrtt+DRqEwfcKMjIyMjIwM/gC5j04wkka+LwAAAABJRU5ErkJg\"></td><td style='text-align:center;vertical-align:middle; margin: 0.5em;border:1px #90999f solid;border-collapse:collapse'><img style='max-width: 100px; max-height:100px;display:inline' src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAHAAAABwCAAAAADji6uXAAAABGdBTUEAALGPC/xhBQAAACBjSFJNAAB6JgAAgIQAAPoAAACA6AAAdTAAAOpgAAA6mAAAF3CculE8AAAAAmJLR0QA/4ePzL8AAAHWSURBVGje7dnPi05RHMfx12hqRFJPk5KFKSsyC2woJRtC2aCUFQt/gO2sJBsLdqztLGRnyyzsRlGUDTaWfqQQJR6LczVPuPfcujXP8e286+ncnufb99Pnfjr3nHseKpVKpVKpVP5/ZvoW7sU9LHTUHMELvOmoWbfWDuMLzvYtPIq5TM1JXMDZkhzGF+yV4SyO96h7jEvYiC+lOIwv2CvDwziAa5m6EXZhg5rhGpJdDxfxEO+xD587apdxEFvxthSH8QWz83BJejYe053fCIfwszSH8QU7MzwtrYMvsZJptCTlt4yPJTmML9iZ4RlpbbuVabKAc/iBq/heksP4gq0Zbsb+5vpmpslFzEvvhg9KcxhfsDXDOWzDnR5NdjTj8xIdxhdszfATnkr70hE+tNRtkdZNeFSiw/iCrRl+xSucwn1c/+P33dL8245x891Ynvi3tKz3w524jBP+Pmd7J2U2P9Fkk5R9UQ7jC/Y6895jdc37zd1mvC3tSel36BP/lpbzLJ3kSfP5F68nrhfxrDSH8QV7/2/RxozVyZzLbyoO4wsOznCs315mag7jCw7OcH0zfivVYXzBwRmel85Hr5TqML7g4AxXcEP+jG1qDuMLViqVSmU4vwBr3jsoVUdw/wAAAABJRU5ErkJg\"></td></tr></tbody></table><div><small>(a vector displayed as a row to save space)</small></div>"
      ],
      "text/plain": [
       "5-element Array{Base.ReinterpretArray{Gray{Float32},2,Float32,Array{Float32,2}},1}:\n",
       " [Gray{Float32}(0.0f0) Gray{Float32}(0.0f0) … Gray{Float32}(0.0f0) Gray{Float32}(0.0f0); Gray{Float32}(0.0f0) Gray{Float32}(0.0f0) … Gray{Float32}(0.0f0) Gray{Float32}(0.0f0); … ; Gray{Float32}(0.0f0) Gray{Float32}(0.0f0) … Gray{Float32}(0.0f0) Gray{Float32}(0.0f0); Gray{Float32}(0.0f0) Gray{Float32}(0.0f0) … Gray{Float32}(0.0f0) Gray{Float32}(0.0f0)]\n",
       " [Gray{Float32}(0.0f0) Gray{Float32}(0.0f0) … Gray{Float32}(0.0f0) Gray{Float32}(0.0f0); Gray{Float32}(0.0f0) Gray{Float32}(0.0f0) … Gray{Float32}(0.0f0) Gray{Float32}(0.0f0); … ; Gray{Float32}(0.0f0) Gray{Float32}(0.0f0) … Gray{Float32}(0.0f0) Gray{Float32}(0.0f0); Gray{Float32}(0.0f0) Gray{Float32}(0.0f0) … Gray{Float32}(0.0f0) Gray{Float32}(0.0f0)]\n",
       " [Gray{Float32}(0.0f0) Gray{Float32}(0.0f0) … Gray{Float32}(0.0f0) Gray{Float32}(0.0f0); Gray{Float32}(0.0f0) Gray{Float32}(0.0f0) … Gray{Float32}(0.0f0) Gray{Float32}(0.0f0); … ; Gray{Float32}(0.0f0) Gray{Float32}(0.0f0) … Gray{Float32}(0.0f0) Gray{Float32}(0.0f0); Gray{Float32}(0.0f0) Gray{Float32}(0.0f0) … Gray{Float32}(0.0f0) Gray{Float32}(0.0f0)]\n",
       " [Gray{Float32}(0.0f0) Gray{Float32}(0.0f0) … Gray{Float32}(0.0f0) Gray{Float32}(0.0f0); Gray{Float32}(0.0f0) Gray{Float32}(0.0f0) … Gray{Float32}(0.0f0) Gray{Float32}(0.0f0); … ; Gray{Float32}(0.0f0) Gray{Float32}(0.0f0) … Gray{Float32}(0.0f0) Gray{Float32}(0.0f0); Gray{Float32}(0.0f0) Gray{Float32}(0.0f0) … Gray{Float32}(0.0f0) Gray{Float32}(0.0f0)]\n",
       " [Gray{Float32}(0.0f0) Gray{Float32}(0.0f0) … Gray{Float32}(0.0f0) Gray{Float32}(0.0f0); Gray{Float32}(0.0f0) Gray{Float32}(0.0f0) … Gray{Float32}(0.0f0) Gray{Float32}(0.0f0); … ; Gray{Float32}(0.0f0) Gray{Float32}(0.0f0) … Gray{Float32}(0.0f0) Gray{Float32}(0.0f0); Gray{Float32}(0.0f0) Gray{Float32}(0.0f0) … Gray{Float32}(0.0f0) Gray{Float32}(0.0f0)]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Here is the first five images from the test set:\n",
    "[MNIST.convert2image(xtst[:,:,i]) for i=1:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[7, 2, 1, 0, 4]\n"
     ]
    }
   ],
   "source": [
    "# Here are their labels\n",
    "println(ytst[1:5]);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "100-element Knet.Train20.Data{Tuple{Array{Float32,3},Array{Int64,1}}}"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# `minibatch` splits the data tensors to small chunks called minibatches.\n",
    "# It returns an iterator of (x,y) pairs.\n",
    "dtrn = minibatch(xtrn,ytrn,100)\n",
    "dtst = minibatch(xtst,ytst,100)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "28×28×100 Array{Float32,3}\n",
      "100-element Array{Int64,1}\n"
     ]
    }
   ],
   "source": [
    "# Each minibatch is an (x,y) pair where x is 100 (28x28) images and y are the corresponding 100 labels.\n",
    "# Here is the first minibatch in the test set:\n",
    "(x,y) = first(dtst)\n",
    "println.(summary.((x,y)));"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "600"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Iterators can be used in for loops, e.g. `for (x,y) in dtrn`\n",
    "# dtrn generates 600 minibatches of 100 images (total 60000)\n",
    "# dtst generates 100 minibatches of 100 images (total 10000)\n",
    "n = 0\n",
    "for (x,y) in dtrn\n",
    "    global n += 1\n",
    "end\n",
    "n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "@webio": {
   "lastCommId": null,
   "lastKernelId": null
  },
  "accelerator": "GPU",
  "kernelspec": {
   "display_name": "Julia 1.7",
   "language": "julia",
   "name": "julia-1.7"
  },
  "language_info": {
   "file_extension": ".jl",
   "mimetype": "application/julia",
   "name": "julia",
   "version": "1.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}