notebooks/14 - Generators and Named Tuples.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Generators\n",
    "\n",
    "In Python there are often many different ways to accomplish a single task.  So far you are familiar with many of Python's primitive data types - specifically for this discussion we will focus on data types that can be iterated over.  \n",
    "\n",
    "A list is a perfect example of a type that can be iterated over.  When Python iterated through the list there are a few major things to note about how it does it:\n",
    "\n",
    " - The list is fully loaded into memory\n",
    " - Each individual item in the list is exposed to the scope as the variable name you provided.\n",
    " \n",
    "List iteration for review:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for name in [\"Angel\", \"Adrian\", \"Ari\"]:\n",
    "    print(name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A generator is different from an iterator in that it looks more like a function with a return value.  Do not be fooled though, this is not a typical function that will return a value and exit forever.  Instead the generator function will return a generator object (from the `yield` statement)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def generator_function(values):\n",
    "    for val in values:\n",
    "        yield val ** val\n",
    "        \n",
    "generator_object = generator_function([1, 2, 3, 4, 5, 6, 7])\n",
    "\n",
    "for square in generator_object:\n",
    "    print(square)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "How this works may still be unclear - that's because it's iterating through the sequence `lazily`.  This means instead of moving through a sequence all at once the generator will iterate and calculate the value only when it's asked to - waiting until the very last minute before it has to do any work.  Hence the `lazy` term.  This is more clearly demonstrated below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def generator_func():\n",
    "    yield \"Angel\"\n",
    "    yield \"Adrian\"\n",
    "    yield \"Ari\"\n",
    "    \n",
    "generator_object = generator_func()\n",
    "\n",
    "# the next() function will force iteration through another step.\n",
    "print(next(generator_object))\n",
    "\n",
    "# print a few more next() calls here of your own and see what your results are.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The above code snippet more clearly demonstrates what is going on inside of a generator object.  The first time you call the function nothing really happens.  It's only after you need to iterate over the generator that you actually see something happening. Your own personal useage of `next` may vary in projects but it serves a good purpose to demonstrate what's going on here.\n",
    "\n",
    "A smart use for a generator is for file I/O. The file you read in can be any size and if all you needed was access to the file contents \"down the line\" you would have to read the whole file in and pass all of that memory around.  With a generator you can create a reference to the file that doesn't get read until it's absolutely neccesary."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def file_contents(file_path):\n",
    "    for line in open(file_path):\n",
    "        yield line\n",
    "\n",
    "def do_something_two(file_generator):\n",
    "    # now that we are here lets start reading the file\n",
    "    # As this iterates it will read a line from the file into memory and then\n",
    "    # dump it as it's no longer needed\n",
    "    for line in file_generator:\n",
    "        print(line)\n",
    "\n",
    "def do_something_one(file_generator):\n",
    "    # other stuff happens here\n",
    "    do_something_two(file_generator)  # Just hand the file reference to it's final destination\n",
    "        \n",
    "def main_function():    \n",
    "    file_generator = file_contents(\"README.md\")  # haven't read the file yet\n",
    "    do_something_one(file_generator)  # Invoke the function above, pass it file reference\n",
    "\n",
    "\n",
    "main_function()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Python makes an extra effort for us as programmers so that we can treat an iterator the same way we would treat a generator.  This can make for refactoring memory intensive code to use a more lighter weight generator pretty painless. Just make sure you've got adequate test coverage for such a refactor."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Generators instead of comprehensions\n",
    "\n",
    "We can turn any list comprehension into a generator by substituting parentheses for brackets, like so:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "[name for name in [\"Angel\", \"Adrian\", \"Ari\"]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gen = (name for name in [\"Angel\", \"Adrian\", \"Ari\"])\n",
    "next(gen)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "next(gen)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}