012_dbs_4_neo4j.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "<a href=\"https://colab.research.google.com/github/rzl-ds/gu511/blob/master/012_dbs_4_neo4j.ipynb\" target=\"_parent\">\n",
    "    <img src=\"https://colab.research.google.com/assets/colab-badge.svg\"/>\n",
    "</a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# `neo4j`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## graph databases\n",
    "\n",
    "in the previous database lectures we covered relational databases (specifically, the `aws` `rds` service and `postgres` as a `rdbm`), and a particular type of `nosql` database (`aws`'s proprietary cloud-based key-value / document store `dynamodb`). I also mentioned that there were several other types of `nosql` databases. among those other types, there is one that is gaining a lot of prominence:  **graph** databases."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "to satisfy my own curiosity: by show of hands, who here has previous experience with:\n",
    "\n",
    "1. graphs (as a general concept, *e.g.* in MATH 442)\n",
    "2. graph calculations (*e.g.* betweenness, page rank)\n",
    "2. graph databases (`neo4j`, `titan`, `cosmos db`)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "graph databases are having a bit of a moment. it's a hype moment, in part, but a moment nevertheless.\n",
    "\n",
    "as a data scientist, I think this is *awesome*. graph databases have immediate implications in a handful of interesting data science problems. before we get into that, though..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### what is the problem?\n",
    "\n",
    "why did anyone ever go about creating graph databases in the first place?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "I have previously been framing the difference between traditional `rdbm`s and more modern `nosql` as being \"relational vs. non-relational\". it's important to remember here that \n",
    "\n",
    "*<div align=\"center\">relational `!=` relationship</div>*\n",
    "\n",
    "remember: \"relational\" effectively means normalized tables"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "as it so happens, both relational databases and most `nosql` (*e.g.*, document stores, key-value stores, columnar) databases are fundamentally ill suited for analyzing the \"interconnected-ness\" of data -- a concept that is often the primary driver in questions data scientists are interested in asking (*e.g.* fraud rings in insurance claims)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### a motivating example: relational databases\n",
    "\n",
    "let's think about the way we would choose to model the following data:\n",
    "\n",
    "+ movies\n",
    "+ actors, writers, producers, and directors\n",
    "+ reviewers"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "suppose the information is basically already collected for you. how might you organize these concepts in a *relational database*?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "    \n",
    "base = 'https://s3.amazonaws.com/shared.rzl.gu511.com/neo4j'                                                                                 \n",
    "                                                                                                                                            \n",
    "actedin = pd.read_csv('{}/{}'.format(base, 'acted_in.csv'))                                                                                \n",
    "directed = pd.read_csv('{}/{}'.format(base, 'directed.csv'))                                                                                \n",
    "movie = pd.read_csv('{}/{}'.format(base, 'movie.csv'))                                                                                      \n",
    "person = pd.read_csv('{}/{}'.format(base, 'person.csv'))                                                                                    \n",
    "produced = pd.read_csv('{}/{}'.format(base, 'produced.csv'))                                                                                \n",
    "reviewed = pd.read_csv('{}/{}'.format(base, 'reviewed.csv'))                                                                                \n",
    "wrote = pd.read_csv('{}/{}'.format(base, 'wrote.csv'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "person.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "movie.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "actedin.head()                                                                                                                              "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "directed.head()                                                                                                                             "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "produced.head()                                                                                                                             "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "reviewed.head()                                                                                                                             "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "wrote.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "as it just so happens, I already put those very dataframes into tables with the same names on our shared `rds postgres` service -- this should be ***easy***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "so now suppose our goal is to play the [six degrees of Kevin Bacon](https://en.wikipedia.org/wiki/Six_Degrees_of_Kevin_Bacon) game.\n",
    "\n",
    "let's start with something simple: just find every pair of people who have acted together in the same movie and the name of that movie"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "no problem in `sql`, right?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "qry = \"\"\"\n",
    "SELECT\n",
    "    p1.name AS p1_name\n",
    "    , m.title as m_title\n",
    "    , p2.name AS p2_name\n",
    "FROM\n",
    "    person AS p1\n",
    "    JOIN acted_in AS a1\n",
    "    ON\n",
    "        p1.id = a1.person\n",
    "    JOIN movie AS m\n",
    "    ON\n",
    "        a1.movie = m.id\n",
    "    JOIN acted_in AS a2\n",
    "    ON\n",
    "        m.id = a2.movie\n",
    "        AND a1.person != a2.person\n",
    "    JOIN person AS p2\n",
    "    ON \n",
    "        a2.person = p2.id\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "import psycopg2\n",
    "import getpass\n",
    "\n",
    "host, port, user, dbname = ('rzl-gu511-shared.cdmknaubrmaw.us-east-1.rds.amazonaws.com', 5432, 'gu511', 'gu511')\n",
    "password = getpass.getpass('password: ')\n",
    "\n",
    "with psycopg2.connect(host=host, port=port, user=user, dbname=dbname, password=password) as conn:\n",
    "    partners = pd.read_sql(qry, conn)\n",
    "partners.tail(15)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "simple...\n",
    "\n",
    "now how about people who have worked in a movie with someone who worked in a movie with someone else?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "qry = \"\"\"\n",
    "SELECT\n",
    "    p1.name AS p1_name\n",
    "    , m1.title AS m1_title\n",
    "    , p2.name AS p2_name\n",
    "    , m2.title AS m2_title\n",
    "    , p3.name AS p3_name\n",
    "FROM\n",
    "    person AS p1\n",
    "    JOIN acted_in AS a1\n",
    "    ON\n",
    "        p1.id = a1.person\n",
    "    JOIN movie AS m1\n",
    "    ON\n",
    "        a1.movie = m1.id\n",
    "    JOIN acted_in AS a2\n",
    "    ON\n",
    "        m1.id = a2.movie\n",
    "        AND a1.person != a2.person\n",
    "    JOIN person AS p2\n",
    "    ON \n",
    "        a2.person = p2.id\n",
    "    JOIN acted_in AS a3\n",
    "    ON\n",
    "        p2.id = a3.person\n",
    "    JOIN movie AS m2\n",
    "    ON\n",
    "        a3.movie = m2.id\n",
    "    JOIN acted_in AS a4\n",
    "    ON\n",
    "        m2.id = a4.movie\n",
    "        AND a3.person != a4.person\n",
    "        AND a1.person != a4.person\n",
    "    JOIN person AS p3\n",
    "    ON \n",
    "        a4.person = p3.id\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "with psycopg2.connect(host=host, port=port, user=user, dbname=dbname, password=password) as conn:\n",
    "    partners = pd.read_sql(qry, conn)\n",
    "partners.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "wanna keep going?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "how about we add in non-actors (i.e. join on `acted_in` and `directed` and `writed` and `produced`)?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "it may seem bad, but in reality, it's actually worse than it seems.\n",
    "\n",
    "those self joins are often *exponentially* expensive (think about friends of friends of friends on facebook). every extra node makes our query more difficult to process"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "also, as confusing as constructing these queries was (and we could have changed our database designe to facilitate them better, for sure), we still end up performing another self join every time we want to hop.\n",
    "\n",
    "`sql` is great for one-to-one or one-to-many sorts of relationships, but less so for one-to-one-to-many-to-one-to-many-to-many-to-one-to..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "in summary: the problem here isn't so much the relational database as it is the problem we are trying to solve with the relational database."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### a motivating example: `nosql` databases\n",
    "\n",
    "the situation isn't much better in `dynamodb`-world. remember, the price of distributed servers and super-fast lookups in our key-value / document stores was that we could basically *only* perform super-fast lookups on primary keys."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "suppose that in the above movie example we had encoded movies as:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "movie = {\n",
    "    'released': 1999,       # partition / hash key\n",
    "    'title': 'The Matrix',  # sort key\n",
    "    'tagline': 'Welcome to the Real World',\n",
    "    'actors': [\n",
    "        {'roles': 'Neo', 'name': 'Keanu Reeves'},\n",
    "        {'roles': 'Trinity', 'name': 'Carrie-Ann Moss'},\n",
    "        {'roles': 'Morpheus', 'name': 'Laurence Fishburne'},\n",
    "        {'roles': 'Agent Smith', 'name': 'Hugo Weaving'},\n",
    "        {'roles': 'Emil', 'name': 'Emi Eifrem'},\n",
    "    ],\n",
    "    'producers': ['Joel Silver'],\n",
    "    # no writers attribute for this node\n",
    "    'directors': ['Lilly Wachowski', 'Lana Wachowski']\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "if we encoded individual movies in this way, we could quickly look up all movies released in a given year, or a specific movie. but if we wanted to do anything with the actors, we are already out of luck: we can't index on the compound attributes, but even if we could, we're still stuck performing \"a lookup, then a scan, then a lookup, then a scan, etc.\" for all of the hops in our query"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### what to do?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "these scenarios may feel contrived, but they are actually an incredibly common scenario to care about and want to model:\n",
    "\n",
    "+ what is the shortest path between two people in a social network? \n",
    "    + can we determine how they met? or who connected them?\n",
    "+ are there secret, back-channel conversation pathways or financial flows?\n",
    "+ are certain actors in a network of people exerting more influence, or suddently exerting influence where they weren't before?\n",
    "+ are individual group leaders critical to the communication chain and survival of a hostile movement?\n",
    "+ do certain words always appear within some number of eachother, or in certain orders, in texts indicating securities fraud?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "relational databases aren't good at handling complicated relationships and networks, and our other `nosql` options often struggle to handle relationships between records at all (by design).\n",
    "\n",
    "because of the ubiquity of these enherently graph-like problems, a special class of databases was created to specifically model graphs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### what is a graph database?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "a graph database is a `nosql` (\"non-relational\", \"no tables\") database which approaches all data storage as being of basically two types: *nodes* and *relationships*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### nodes\n",
    "\n",
    "a node is, extremely generically speaking, \"a thing.\" nodes usually represent the high-level concepts you first think about when you're talking about a problem domain. they are concepts which are connected in some yet-defined way to otehr concepts.\n",
    "\n",
    "they are usually the things which we describe in our language with nouns, but they don't have to be."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "a graph model distinguishes between nodes in two basic ways: *labels* and *properties*."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "first, a single graph can have nodes of many different \"types\" of things, in much the same way our understanding of a problem involves multiple concepts.\n",
    "\n",
    "the technical term we will use for the \"type\" of node is the node's *label*. a single node can represent zero, one, or more \"types\" of things, and can have zero, one, or more *labels*."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "for example, in the model above we could say that we have `Movie` and `Person` as two types of entites, and we could have nodes labelled `Movie` (a thing that is a movie) and nodes labelled `Person` (a thing that is a person).\n",
    "\n",
    "for example, we could have another concept, such as [SAG](https://en.wikipedia.org/wiki/Screen_Actors_Guild) member, and `SAG` could be another label on a node with a `Person` label."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "what we said above applies to a single, generic node. a graph is composed of possibly many nodes, and possibly many with the same *label*. \n",
    "\n",
    "individual nodes have *properties*: properties that describe them in greater detail. as was the case with `dynamodb`, there is no schema to the *properties* of a node -- anything goes. there is, generically, a demand that there be one unique identifier, but that is usually handled by the database itself."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "one node *labelled* `Person` may have the properties\n",
    "\n",
    "```json\n",
    "{\n",
    "    'name': 'Keanu Reeves',\n",
    "    'born': 1964\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "in summary,\n",
    "\n",
    "1. nodes are categorized into broad groups by their *labels*\n",
    "2. individual nodes are describe by their *properties*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "schematically, nodes are often written in a sort of pictoral way (more on this later):\n",
    "\n",
    "```cypher\n",
    "(p:Person {name: 'Keanu Reeves', born: 1954})\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### edges \n",
    "\n",
    "relationships between nodes in our graph are called *edges*.\n",
    "\n",
    "because nodes are representing relationships and connections between nodes (generally nouns), it is common for them to be representing actions (generally verbs). those verbs themselves are active or passive, which results in *direction* of the edges -- all edges in these graphs are *directed*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "take our movie example. Keanu Reeves was an actor in The Matrix, playing a character named Neo. we could create an edge between the `Person` node representing Keanu Reeves and the `Movie` node representing \"The Matrix\".\n",
    "\n",
    "as with nodes, edges have two sorts of distinctions: the *type* and *properties*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "these two distinctions are analogous to the distinction types for nodes:\n",
    "\n",
    "1. edges are categorized into broad groups by their *types*\n",
    "    + while nodes can have many labels, edges can have **one and only one** *type*\n",
    "2. individual edges are describe by their *properties*\n",
    "    + usually quantitative things (weights, time lengths)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "there is a similarly pictoral `ascii` reprentatino of edges:\n",
    "\n",
    "```cypher\n",
    "(p:Person)-[a:ACTED_IN {weight: 2.5}]->(m:Movie)\n",
    "```\n",
    "\n",
    "note, in particular, that *edge types* are all-caps, and the arrows are written as directed (as if you were reading the relationship out loud)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "##### to label, relate, or \"propertize\"...\n",
    "\n",
    "up above, we discussed another concept which might seem like a good candidate for a *node label*: the role a person had in a film (actor, director, producer, writer)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "for example, Clint Eastwood has been both an actor and a director. we could have created *node labels* \"Actor\" and \"Director\" and treated the Clint Eastwood node as having *labels* `Person, Actor, Director`. that would be a totally valid graph model.\n",
    "\n",
    "```cypher\n",
    "(p:Person:Actor:Director {name: 'Clint Eastwood'})\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "however, consider: Clint Eastwood is *always* a person, but he is not *always* an Actor, nor is he *always* a Director: he takes on different combinations of roles for different movies.\n",
    "\n",
    "in this sense, it could be that the role is something we treat as a relationship between Clint Eastwood the Person node and the Movie nodes of which he has been a part (e.g \"Unforgiven\")."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "```cypher\n",
    "(p:Person {name: 'Clint Eastwood'})-[:ACTED_IN {role: 'William Munny'}]->(m:Movie {name: 'Unforgiven'})\n",
    "\n",
    "(p:Person {name: 'Clint Eastwood'})-[:DIRECTED]->(m:Movie {name: 'Unforgiven'})\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### the cost of putting your relationships first\n",
    "\n",
    "graph databases elevate the relationships -- basically, those `JOIN` conditions in our complicated `sql` statement above -- into first-class properties of every node (record). this means that accessing those related neighbor nodes and traversing those relationships is *very* fast, and easy to visualize / write."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "that being said, all of this is predicated on looking at relationships *from some starting point*. generally speaking, graph database calculations are very fast when *anchored* on a starting node, and if you can formulate your problem such that it is anchored this way, that will be better."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "there are *plenty* of network calculations that can't be isolated to single nodes, and require the entire graph (e.g. graph diameter, page rank)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### what graph database options are out there?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### semantics\n",
    "\n",
    "it's common for people to discuss different concepts in the graph database world with the same language, and to often be a little careless when doing so."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "there are three different types of \"technologies\" commonly discussed in the context of graph databases:\n",
    "\n",
    "1. **graph databases**: the software that store, model, and represent data as nodes and edges in a graph\n",
    "2. **query languages and graph apis**: domain-specific languages (both theoretical and implemented) that can be used to interact with graph database software\n",
    "3. **graph and network calculation libraries**: code implementations of algorithms and metrics which leverage the above"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "some of these semantic distinctions exist in the relational database world as well: `postgres` is a database, `sql` is a query language. the calculation libraries are specific to the data type, so the analogy is not as well defined."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "##### databases\n",
    "\n",
    "there are [a ton of options](https://en.wikipedia.org/wiki/Graph_database#List_of_graph_databases) out there, and the number will likely grow with the hype."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "among them, `neo4j` is [the clear leader](https://db-engines.com/en/ranking/graph+dbms), with Microsoft's Azure Cosmos DB in a rising second. we will focus on `neo4j`.\n",
    "\n",
    "for perspective, it's also useful to [compare `neo4j` to relational and other `nosql` databases](https://db-engines.com/en/ranking/) (22nd, between `dynamodb` and `couchbase`)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "##### query languages and `api`s\n",
    "\n",
    "`sql` is an abstract language designed for working with relations (tables), and it gets implemented in individual `rdbm`s. \n",
    "\n",
    "in a similar way, people are actively developing the `sql` of the graph database world. no one language has been selected, but [`cypher`](https://neo4j.com/developer/cypher-query-language/) (the query language of `neo4j`) is an early candidate.\n",
    "\n",
    "[other alternatives exist](https://en.wikipedia.org/wiki/Graph_database#APIs_and_graph_query-programming_languages), and of them `graphql` (facebook) and `gremlin` are the most common"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "##### graph calculation and visualization libraries\n",
    "\n",
    "finally, calculation libraries have cropped up as graph and network analysis has become more important. some require loading an entire graph into memory where all calculations are done, and some are written utilizing the above items to perform *ad hoc* queries and limit memory constraints.\n",
    "\n",
    "a large segment of this space focuses primarily on *visualization* of large graphs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "popular libraries include:\n",
    "\n",
    "+ [`networkx`](https://networkx.github.io/): for calculation and limited vis\n",
    "+ [`igraph`](http://igraph.org/redirect.html): same\n",
    "+ [`stinger`](http://www.stingergraph.com/): an impressively fast graph algorithm library (implements its own properietary graph storage structure)\n",
    "+ [`gephi`](https://gephi.org/): excellent visualization studio with some surprising calculation abilities\n",
    "+ [`SNAP` (stanford network analysis platform)](https://snap.stanford.edu/snap/): bills itself as being the best network analytics package for very large (millions of nodes, billions of edges, which *is* quite large) networks. I haven't tried it\n",
    "+ [`sigma.js`](http://sigmajs.org/): a very prominent `javascript` graph rendering and visualization library (no computation to speak of)\n",
    "+ [`cytoscape`](http://www.cytoscape.org/) and `cytoscape.js`: *the* network library for the molecular biology community, mostly vis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## modelling"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "the concept of a graph is so generalizable that just about any data *could* be cast as a graph, so go nuts.\n",
    "\n",
    "that being said, there are some scenarios that immediately lend themselves to graph modelling"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "first, basically any time you have people interactings with eachother (and you care more about the interactions than the people). this includes\n",
    "\n",
    "+ [social networks](https://neo4j.com/use-cases/social-network/)\n",
    "+ [recommender systems](https://neo4j.com/use-cases/real-time-recommendation-engine/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "another huge area of interest has been financial \n",
    "\n",
    "+ fraud ([in banks](https://neo4j.com/graphgist/9d627127-003b-411a-b3ce-f8d3970c2afa?ref=solutions), [in insurance](https://neo4j.com/blog/insurance-fraud-detection-graph-database/))\n",
    "+ financial networks and money laundering\n",
    "    + [the panama papers](https://panamapapers.icij.org/) (available as a `neo4j` sandbox)\n",
    "    + [the paradise papers](https://www.icij.org/investigations/paradise-papers/explore-politicians-paradise-papers/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "they are also gaining a lot of popularity in areas which focus on resource management and connectedness, such as\n",
    "\n",
    "+ [identity and access management systems](https://neo4j.com/use-cases/identity-and-access-management/)\n",
    "+ large-organization data management ([knowledge graphs](https://neo4j.com/use-cases/knowledge-graph/), [data provenance systems](https://neo4j.com/use-cases/master-data-management/))\n",
    "+ [IT architecture and critical asset identification](https://neo4j.com/use-cases/network-and-it-operations/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "finally, they can be useful for sequential data where items are reused or nested:\n",
    "\n",
    "+ [text analytics](https://graphaware.com/neo4j/2016/07/07/mining-and-searching-text-with-graph-databases.html)\n",
    "+ [time series analysis](https://github.com/SocioPatterns/neo4j-dynagraph/wiki/Representing-time-dependent-graphs-in-Neo4j)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### example model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "**<div align=\"center\">walkthrough: an example model in the `neo4j` sandbox</div>**\n",
    "\n",
    "let's look at how that movie data model we discussed above can be represented in a graph. start by heading to [the `neo4j` sandbox site](https://neo4j.com/sandbox-v2/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "source": [
    "the walkthrough goes through the following steps:\n",
    "\n",
    "1. create a `neo4j` sandbox\n",
    "    1. open [the `neo4j` sandbox site](https://neo4j.com/sandbox-v2/)\n",
    "    2. log in\n",
    "    3. under the \"Launch a New Sandbox\" header, find the \"Neo4j 3.3\" (*i.e.* \"Blank Sandbox\") option and click \"Launch Sandbox\"\n",
    "    4. that will take some time to launch, so look at the other options as example models\n",
    "    5. click on the link to the `neo4j` browser\n",
    "2. run the \"Movies\" gist\n",
    "    0. you could just type `:play movie-graph` and run it, **or**\n",
    "    1. click on the Star icon in the left menu bar\n",
    "    2. click on the \"Example Graphs\" menu item\n",
    "    3. click on the \"Movie Graph\" element\n",
    "    4. run (play button) the graph line that was brought up\n",
    "    5. click right and run the first major code block\n",
    "3. check out the movie graph relationship schema\n",
    "    0. you could just type `CALL db.schema()` and run it, **or**\n",
    "    1. click on the Star icon in the left menu bar\n",
    "    2. click on the \"What is related, and how\" option"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## cypher"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "in the previous walkthrough, we were able to use the `neo4j` web browser to auto-generate some query strings like the following:\n",
    "\n",
    "```cypher\n",
    "MATCH (tom:Person {name:\"Tom Hanks\"})-[:ACTED_IN]->(m)<-[:ACTED_IN]-(coActors),\n",
    "      (coActors)-[:ACTED_IN]->(m2)<-[:ACTED_IN]-(cruise:Person {name:\"Tom Cruise\"})\n",
    "RETURN tom, m, coActors, m2, cruise\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "this declarative query language was created by `neo4j` and is called `cypher`. it is one of several domain-specific query languages out there (discussed above), but the only one we will focus on at this time.\n",
    "\n",
    "it has a lot of similarities to `sql`, as well as some obvious differences. let's discuss the basics. refer to [the documentation](https://neo4j.com/docs/developer-manual/current/cypher/) for total coverage of the query language"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### patterns\n",
    "\n",
    "`cypher` allows you to traverse the graph by matching text patterns against graph structures: if you describe a graph structure using `cypher` shorthand *patterns*, `neo4j` will look for all segments of the graph which match that *pattern*."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### `MATCH`ing patterns\n",
    "matching patterns is so important in `cypher` that is how almost every query starts. think of `MATCH` as the `SELECT` of `cypher`.\n",
    "\n",
    "the most basic query syntax is\n",
    "\n",
    "```cypher\n",
    "MATCH ...  // ** some pattern here **\n",
    "RETURN ... // ** some properties, or calculations on the elements matched by that pattern **\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### pattern pieces: nodes, edges, and paths\n",
    "\n",
    "to start with, the main objects in the graph database (nodes and edges) have plain-text symbolic representations (like we discussed above):\n",
    "\n",
    "```cypher\n",
    "// nodes\n",
    "(nodevariable:NodeLabel {nodeProperty: 'nodePropertyValue'})\n",
    "\n",
    "// edges\n",
    "()-[edgevariables:EdgeLabel {edgeProperty: 'edgePropertyValue'}]->()\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "the first word inside these symbolic representations is a node or edge *variable*. this is analogous to aliasing in `sql`:\n",
    "\n",
    "```sql\n",
    "SELECT\n",
    "    t1.firstname AS name\n",
    "...\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "you can use variables in the parts of the query that follow (*e.g.* the `RETURN` statement) to reference the items matching that symbolic pattern of that node or that edge"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "*paths* -- collections of connected nodes and the edges which connect them -- can be expressed by chaining the above elements together, as was done in the Tom Hanks meets Tom Cruise query:\n",
    "\n",
    "```cypher\n",
    "MATCH (hanks:Person {name:\"Tom Hanks\"})-[:ACTED_IN]->(m)<-[:ACTED_IN]-(coActors),\n",
    "      (coActors)-[:ACTED_IN]->(m2)<-[:ACTED_IN]-(cruise:Person {name:\"Tom Cruise\"})\n",
    "RETURN tom, m, coActors, m2, cruise\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "```cypher\n",
    "MATCH (hanks:Person {name:\"Tom Hanks\"})-[:ACTED_IN]->(m)<-[:ACTED_IN]-(coActors),\n",
    "      (coActors)-[:ACTED_IN]->(m2)<-[:ACTED_IN]-(cruise:Person {name:\"Tom Cruise\"})\n",
    "RETURN tom, m, coActors, m2, cruise\n",
    "```\n",
    "\n",
    "this path can be described by simply reading it:\n",
    "\n",
    "+ a person named \"Tom Hanks\"... acted in... a movie\n",
    "+ that movie was acted in by another person\n",
    "+ that person acted in a movie\n",
    "+ that movie was acted in by a person named \"Tom Cruise\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "generally speaking, you can leave out as much of the pattern as you want.\n",
    "\n",
    "the only *truly* required things are the `()` and (if more than one node is involved), the `--` elements (I have intentionally left out the direction on that edge as well as the label; neither are *necessary* for a path represenation, though either are often good to include)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "the simplest example is the pattern which matches any node:\n",
    "\n",
    "```cypher\n",
    "MATCH (n)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "an only slightly more complicated example: the following would find all pairs of nodes of any type that are 4 edges away from eachother (a harder thing than the one we didn't even try in `sql` earlier), and assign them to variables `n0` and `n1`:\n",
    "\n",
    "```cypher\n",
    "MATCH (n0)--()--()--()--(n1)\n",
    "```\n",
    "\n",
    "this could (will) re-use any one path any number of times."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "writing all of those edges can be tedious, so `cypher` has a shorthand for representing paths with several consecutive edges of the same type (requires you don't care about the nodes between):\n",
    "\n",
    "```cypher\n",
    "// any length\n",
    "MATCH (n0)-[*]-(n1)\n",
    "          \n",
    "// exactly two edges\n",
    "MATCH (n0)-[*2]-(n1)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "```cypher\n",
    "// 1, 2, or 3 edges\n",
    "MATCH (n0)-[*1..3]-(n1)\n",
    "              \n",
    "// up to 5 edges\n",
    "MATCH (n0)-[*..5]-(n1)\n",
    "\n",
    "// 5 or more edges\n",
    "MATCH (n0)-[*5..]-(n1)          \n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### `RETURN`ing results\n",
    "\n",
    "once you have matched a pattern and created variables from portions of it, you can `RETURN` those variables.\n",
    "\n",
    "under the hood, this is (yet again) an `http` request. the response body is a `json` object describing the query result (check out the \"`</>` Code\" button in your query result box)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "if what was returned is limited to node and edge objects, `neo4j`'s web browser will display them in the graphs we are familiar with. otherwise, they will be displayed as a table"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "for example, it's easy to get a visual graph of a fixed number (`LIMIT`) of nodes:\n",
    "\n",
    "```cypher\n",
    "MATCH (n)\n",
    "RETURN n\n",
    "LIMIT 10\n",
    "```\n",
    "\n",
    "be sure to check out the other tabs in your browser window"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "it's also easy to pack pattern objects into paths and return the *paths*:\n",
    "\n",
    "```cypher\n",
    "MATCH p=(n0)--(n1)\n",
    "RETURN p\n",
    "LIMIT 10\n",
    "```\n",
    "\n",
    "again, check out the other tabs in the browser window"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "as I mentioned above, you don't *have* to return variables (`n`, `p`) -- you can return properties as well. for example:\n",
    "\n",
    "```cypher\n",
    "MATCH (p:Person)-[a:ACTED_IN]->(m:Movie)\n",
    "RETURN p.name, a.roles, m.title\n",
    "```\n",
    "\n",
    "*note*: the browser no longer renders these as graph objects *because they aren't objects*, they're *properties*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### operators\n",
    "\n",
    "just like `sql`, `cypher` has several built-in [operators](http://neo4j.com/docs/developer-manual/current/cypher/syntax/operators/) -- reserved keywords that accomplish certain basic calculations, logical comparison, or control flow.\n",
    "\n",
    "[this table](http://neo4j.com/docs/developer-manual/current/cypher/syntax/operators/#query-operators-summary) covers the list, and if you are familiar with `sql` you should be familiar with what all of these are doing."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### clauses\n",
    "\n",
    "both `MATCH` and `RETURN` are important `cypher` keywords that designate that the words following them (in this case: patterns) should be interepreted in a certain way and `neo4j` should take a certain action. they are part of a broader class of special action or control flow keywords called *clauses*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "allow me divide clauses up into two types:\n",
    "\n",
    "1. those used to build queries and read data\n",
    "2. those used to create and update data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### query operations (read)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "##### `MATCH`\n",
    "\n",
    "the `MATCH` statement indicates that what follows should be treated as a pattern and `neo4j` should identify graph pieces that match that pattern."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "##### `RETURN`\n",
    "\n",
    "the `RETURN` statement instructs the query engine to take whatever variables or values are defined at that point in the query and limit them down to a certain structured subset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "there are several other important *clauses* which we will cover at a high level -- read about the rest [here](http://neo4j.com/docs/developer-manual/current/cypher/clauses/)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "##### `WHERE`\n",
    "\n",
    "the `WHERE` clause only applies following a `MATCH` clause, and it does what you expect: it takes the list of sub-graphs matching a given pattern and filters them to those which satisfy some requirement\n",
    "\n",
    "```cypher\n",
    "MATCH (n:Person)-[a:ACTED_IN]->(m:Movie)\n",
    "WHERE size(a.roles) > 1\n",
    "RETURN n, a, m\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "##### `ORDER BY`, `SKIP`, and `LIMIT`\n",
    "\n",
    "these three clauses all affect how the results of a pattern match query are returned, either by sorting them, skipping some number of them, or only returning the first number of them"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "##### `WITH`\n",
    "\n",
    "this clause follows a `MATCH` query and uses the results of that query to \"set the stage\" for the next. basically, it allows you to chain several queries together. for example:\n",
    "\n",
    "```cypher\n",
    "MATCH (actor:Person)-[:ACTED_IN]->(m1:Movie)\n",
    "WITH actor\n",
    "MATCH (actor)-[:DIRECTED]->(m2:Movie)\n",
    "RETURN actor, m2\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "##### `UNWIND`\n",
    "\n",
    "this will take a list of items and *unwind* them into a sequence of rows. this is used to explode collections of things into separate records. For example:\n",
    "\n",
    "```cypher\n",
    "UNWIND [1, 2, 3, 4, 5, 6, 7] AS x\n",
    "RETURN x\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "##### `UNION` and `UNION ALL`\n",
    "\n",
    "just as in `sql`, this takes the results of multiple queries and combines them. the former removes duplicates and the latter keeps them"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### create and update clauses (write)\n",
    "\n",
    "the following clauses are used to create new nodes and edges or to update existing ones"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "##### `CREATE`\n",
    "\n",
    "create a node or relationship:\n",
    "\n",
    "```cypher\n",
    "// one node at a time\n",
    "CREATE (n:TestLabel {x: 1})\n",
    "\n",
    "// many elements in a path\n",
    "CREATE p = (m1:MyLabel {msg: 'hello world'})-[e:MY_TYPE {x: 1}]->(m2:MyLabel {msg: 'goodbye world'})\n",
    "RETURN p\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "##### `DELETE`, `DETACH DELETE`\n",
    "\n",
    "you can `DELETE` and isolated node or any edge, or you can `DETACH DELETE` a connected node\n",
    "\n",
    "```cypher\n",
    "// isolated node\n",
    "MATCH (n:TestLabel {x: 1})\n",
    "DELETE n\n",
    "                      \n",
    "// will fail because node isn't detached\n",
    "MATCH (m1:MyLabel {msg: 'hello world'})\n",
    "DELETE m1\n",
    "                     \n",
    "// will detach node then delete it\n",
    "MATCH (m1:MyLabel {msg: 'hello world'})\n",
    "DETACH DELETE m1\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "##### `SET` and `REMOVE`\n",
    "\n",
    "the `SET` clause allows you to set labels on nodes or properties on nodes and edges, and `REMOVE` lets you undo that operation\n",
    "\n",
    "```cypher\n",
    "MATCH (e:Person {name: \"Emil Eifrem\"})\n",
    "SET e.l33t = true, e:NeoFounder, e:BigHollywoodActor\n",
    "RETURN e\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "```cypher\n",
    "MATCH (e:Person {name: \"Emil Eifrem\"})\n",
    "REMOVE e:BigHollywoodActor\n",
    "RETURN e\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "##### `MERGE` [`ON CREATE`, `ON MATCH`]\n",
    "\n",
    "if you are familiar with the `sql` concept of an `UPSERT` (an update or an insert), this is the `cypher` equivalent. basically, this statement looks to match a given pattern, and if it doesn't find that pattern to create some nodes. `MERGE` on its own will do only this.\n",
    "\n",
    "```cypher\n",
    "MERGE (z:Teacher {name: \"Zach Lamberty\"})\n",
    "RETURN z\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "there are times, however, where you want to *do* something when you create a node, and then you want to *do something different* when you are matching / updating. this is facilitated through the trailing clauses `ON CREATE` and `ON MATCH`:\n",
    "\n",
    "```cypher\n",
    "MERGE (c:TA {name: \"Carlos Blancarte\"})\n",
    "ON CREATE SET c.created = timestamp()\n",
    "ON MATCH SET c.lastMatched = timestamp()\n",
    "RETURN c\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "##### `LOAD CSV`\n",
    "\n",
    "no database would be worth a single durn thing if it couldn't perform bulk inserts of `csv`s, so let's cover that too.\n",
    "\n",
    "every invocation of `LOAD CSV` needs to provide the file path or url of the `csv` file. `neo4j` then loads every record in that `csv` into:\n",
    "\n",
    "1. a list (items accessed as `record[1]`, `record[2]`, *etc.*) if invoked simply as `LOAD CSV`\n",
    "2. an object (items accessed as `record.col1`, `record.col2`, *etc.*) if invoked with `LOAD CSV WITH HEADERS`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "```cypher\n",
    "LOAD CSV FROM 'https://neo4j.com/docs/developer-manual/3.3/csv/artists.csv' AS line\n",
    "CREATE (:ArtistNoHeaders { name: line[1], year: toInteger(line[2])})\n",
    "                                                      \n",
    "LOAD CSV WITH HEADERS FROM 'https://neo4j.com/docs/developer-manual/3.3/csv/artists-with-headers.csv' AS line\n",
    "CREATE (:ArtistHeaders { name: line.Name, year: toInteger(line.Year)})\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "every single discussion I have ever seen about `LOAD CSV` has stressed at least 1 million times: when you load files of around 100k or more rows, preface your command with \n",
    "\n",
    "```cypher\n",
    "USING PERIODIC COMMIT\n",
    "LOAD CSV FROM ...\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### functions\n",
    "\n",
    "there are upwards of 140 [built-in functions](http://neo4j.com/docs/developer-manual/current/cypher/functions/) you can use to munge data within your queries, perform aggregate calculations, map or reduce calculations across lists, and much more."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "in the interest of time we won't go into any in particular right now, but know that these are mostly operating in a *local* scope; that is, they will take a single subgraph matching a pattern and operate on it.\n",
    "\n",
    "the exception to that (as with `sql`) is *aggregation* functions. there is no explicit aggregation clause (like `GROUP BY`) in `cypher`. instead, aggregation is implied based on the properties in the neighboring `RETURN` or `WITH` clause"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### procedures\n",
    "\n",
    "there is one clause we didn't discuss: `CALL`. we've actually already used this procedure, too, wayyyyyy back when we first opened the sandbox:\n",
    "\n",
    "```cypher\n",
    "CALL db.schema()\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "the `CALL` clause will invoke a *procedure*, which is a piece of built-in or user defined `java` code that extends the capabilities of the base `cypher` language to do something different: connect to some other system, or perform a complicated calculation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "several *procedures* are built-in and we can investigate what they are using a *procedure* (if you don't know what a dictionary is, me telling you to \"look it up in the dictionary\" won't be of much help will it?)\n",
    "\n",
    "```cypher\n",
    "CALL dbms.procedures()\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "the [`apoc`](https://neo4j-contrib.github.io/neo4j-apoc-procedures/) procedures (`apoc` stands for \"a package of components\" or \"awesome procedures on cypher\") are of particular interest, as that is where `neo4j` and the community have started to collect their advanced procedures, including optimized graph algorithm implementations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "with `apoc`, calculating betweenness centrality and page rank directly from a cypher query is no problem:\n",
    "\n",
    "```cypher\n",
    "MATCH (node:Person)\n",
    "WITH collect(node) AS nodes\n",
    "CALL apoc.algo.betweenness(['ACTED_IN', 'DIRECTED', 'PRODUCED', 'WROTE'], nodes, 'BOTH') YIELD node, score\n",
    "RETURN node, score\n",
    "ORDER BY score DESC\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "```cypher\n",
    "MATCH (node:Person)\n",
    "WITH collect(node) AS nodes\n",
    "CALL apoc.algo.pageRank(nodes) YIELD node, score\n",
    "RETURN node, score\n",
    "ORDER BY score DESC\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "additionally, `apoc` has implemented a procedure for reading and efficiently navigating `json` objects: \n",
    "\n",
    "```cypher\n",
    "WITH \"https://api.stackexchange.com/2.2/questions?pagesize=100&order=desc&sort=creation&tagged=neo4j&site=stackoverflow&filter=!5-i6Zw8Y)4W7vpy91PMYsKM-k9yzEsSC1_Uxlf\" AS url\n",
    "CALL apoc.load.json(url) YIELD value\n",
    "UNWIND value.items AS item\n",
    "RETURN item.title, item.owner, item.creation_date, keys(item)\n",
    "```\n",
    "\n",
    "I wonder if we'll find a way to use that..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "there is *a lot* more than can be done using `cypher` -- go ahead and bookmark the [`cypher` reference cheatsheet](http://neo4j.com/docs/cypher-refcard/current/) now"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## `cypher` in `python`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "you're never gonna believe this.\n",
    "\n",
    "we just spent all that time doing all that cool stuff via a web interface and now... we're just going to do it in `python` instead, because doign things from the terminal makes us feel warm and fuzzy inside"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### library options\n",
    "\n",
    "basically, there are two choices, and then others.\n",
    "\n",
    "a developer named Nigel Small wrote an excellent `python` adapter for `neo4j` called [`py2neo`](http://py2neo.org/), so naturally `neo4j` hired him to have him write their other library, [`neo4j-driver`](https://neo4j.com/docs/api/python-driver/1.5/)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "the community had collected pretty hard around `py2neo`, but I think the future is in `neo4j-driver` (if for no other reason that that the library is being developed in tandem with drivers for several other languages, so if you're switching languages or they ever make an `R` implementation the learning curve will be much less steep).\n",
    "\n",
    "I'm going to recommend using `neo4j-driver`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### `neo4j-driver`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "first things first: [read the manual](https://neo4j.com/docs/api/python-driver/1.5/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "good, now that's out of the way. let's install the driver package. from a command line (your laptop or the `ec2` server, whichever you'd like to use\n",
    "\n",
    "```bash\n",
    "pip install neo4j-driver\n",
    "```\n",
    "\n",
    "if you experience permission errors, make sure you are using your *`conda` environment's* pip (try `which pip` to find out). you shoudl be using something like `~/miniconda3/bin/pip` or `~/miniconda3/envs/myenv/bin/pip`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "if you started a `neo4j` sandbox with me earlier, there is a chance that some code using this library has already been generated for you.\n",
    "\n",
    "+ head to https://neo4j.com/sandbox-v2/#\n",
    "+ in the box for your sandbox, click the \"Code\" tab\n",
    "+ select the `py` tab on that panel\n",
    "+ copy and paste!\n",
    "+ if there is nothing, launch the \"Recommender\" sandbox and repeat (the blank sandbox code snippers were having (no joke) `s3` permission issues at the time of writing this)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "the code for your sandbox should look something like this:\n",
    "\n",
    "```python\n",
    "# pip install neo4j-driver\n",
    "\n",
    "from neo4j.v1 import GraphDatabase, basic_auth\n",
    "\n",
    "driver = GraphDatabase.driver(\n",
    "    \"bolt://YOUR_SERVER_IP:YOUR_PORT\", \n",
    "    auth=basic_auth(\"neo4j\", \"YOUR_PASSWORD\")\n",
    ")\n",
    "session = driver.session()\n",
    "\n",
    "cypher_query = \"\"\"\n",
    "MATCH (n)\n",
    "RETURN id(n) AS id\n",
    "LIMIT $limit\n",
    "\"\"\"\n",
    "\n",
    "results = session.run(\n",
    "    cypher_query,\n",
    "    parameters={\"limit\": 10}\n",
    ")\n",
    "\n",
    "for record in results:\n",
    "    print(record['id'])\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "let's walk through this code one block at a time.\n",
    "\n",
    "after importing the necessary libraries, we create a *driver* object. the *driver* manages our *sessions* (individual statefull communications with the database) and creates *connections* (open channels through which *sessions* can talk to the database)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true,
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "from neo4j.v1 import GraphDatabase, basic_auth\n",
    "\n",
    "driver = GraphDatabase.driver(\n",
    "    \"bolt://52.207.74.86:34077\", \n",
    "    auth=basic_auth(\"neo4j\", \"smokes-cards-pails\")\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "after this, we create a *session*. a *session* is a stateful object which can make queries to the database (as needed) to execute queries and make transactions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "session = driver.session()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "next, we define our query. I've made a slight tweak to the query to include the node object itself (not just its `id`)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cypher_query = \"\"\"\n",
    "MATCH (n)\n",
    "RETURN id(n) AS id, n\n",
    "LIMIT $limit\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "note that we have yet another way of parameterizing our query strings here (for validation and sanitization): the `$` character."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "the next stage is to simply execute the `cypher` code:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "results = session.run(\n",
    "    cypher_query,\n",
    "    parameters={\"limit\": 10}\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "the object returned by this `session.run` method is not a native `python` collection but a special object:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "the `results` object itself is natively iterable:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "for record in results:\n",
    "    print(record['id'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "however, the `results.data()` method will often be more interesting to us. we need to re-query (the `results` item has been iterated so there would be no `data` to return)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "results = session.run(\n",
    "    cypher_query,\n",
    "    parameters={\"limit\": 10}\n",
    ")\n",
    "results.data()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "q = \"\"\"\n",
    "MATCH (p1:Person)-->(m1:Movie)<--(p2:Person)-->(m2:Movie)<--(p3:Person)\n",
    "RETURN p1.name, m1.title, p2.name, m2.title, p3.name\n",
    "\"\"\"\n",
    "results = session.run(q)\n",
    "pd.DataFrame(results.data()).head(20)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "now compare those eight lines to the code we wrote before to do the same thing for *only* actors in the relational setting:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "```python\n",
    "qry = \"\"\"\n",
    "SELECT\n",
    "    p1.name AS p1_name\n",
    "    , m1.title AS m1_title\n",
    "    , p2.name AS p2_name\n",
    "    , m2.title AS m2_title\n",
    "    , p3.name AS p3_name\n",
    "FROM\n",
    "    person AS p1\n",
    "    JOIN acted_in AS a1\n",
    "    ON\n",
    "        p1.id = a1.person\n",
    "    JOIN movie AS m1\n",
    "    ON\n",
    "        a1.movie = m1.id\n",
    "    JOIN acted_in AS a2\n",
    "    ON\n",
    "        m1.id = a2.movie\n",
    "        AND a1.person != a2.person\n",
    "    JOIN person AS p2\n",
    "    ON \n",
    "        a2.person = p2.id\n",
    "    JOIN acted_in AS a3\n",
    "    ON\n",
    "        p2.id = a3.person\n",
    "    JOIN movie AS m2\n",
    "    ON\n",
    "        a3.movie = m2.id\n",
    "    JOIN acted_in AS a4\n",
    "    ON\n",
    "        m2.id = a4.movie\n",
    "        AND a3.person != a4.person\n",
    "        AND a1.person != a4.person\n",
    "    JOIN person AS p3\n",
    "    ON \n",
    "        a4.person = p3.id\n",
    "\"\"\"\n",
    "\n",
    "with psycopg2.connect(host=host, port=port, user=user, dbname=dbname, password=password) as conn:\n",
    "    partners = pd.read_sql(qry, conn)\n",
    "partners.head(10)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## standing up our own `neo4j` service"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "the sandbox environment is great for learning about `neo4j` for the first time, or for demoing its awesomeness to prospective clients, but it's not a viable long-term solution.\n",
    "\n",
    "at the moment, the primary means of deploying `neo4j` in `aws` is through standing up an independent `aws` `ec2` instance and installing `neo4j` ourselves."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "up until literally this week (at the `aws` re:Invent conference, when [`aws` `neptune`](https://aws.amazon.com/neptune/) was announced), there was no natively-hosted `neo4j` graph database instance.\n",
    "\n",
    "now that they are all-in on `neptune` (which only touts as query languages `gremlin` and `sparkql`, so is likely *not* a fork of `neo4j`), it seems unlikely we'll see a native solution any time soon"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "**<div align=\"center\">walkthrough: standing up our own `neo4j` server</div>**\n",
    "\n",
    "+ create a new `aws` `ec2` server (free tier, ubuntu ami, all defaults)\n",
    "+ `ssh` into that new `ec2` server (`ubuntu@ip.ip.ip.ip`)\n",
    "+ run the contents of the next cell\n",
    "+ `sudo nano /etc/neo4j/neo4j.conf`\n",
    "    + delete the leading `#` from `#dbms.connectors.default_listen_address=0.0.0.0`\n",
    "+ start the `neo4j` service: `sudo service neo4j start`\n",
    "+ update the `aws` `ec2` security group to allow custom `TCP` traffic on port 7474 and another for 7687\n",
    "+ go to your `ip.ip.ip.ip:7474` and use `neo4j: neo4j` as username and password"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "```\n",
    "wget -O - https://debian.neo4j.org/neotechnology.gpg.key | sudo apt-key add -\n",
    "echo 'deb http://debian.neo4j.org/repo stable/' | sudo tee -a /etc/apt/sources.list.d/neo4j.list\n",
    "sudo apt-get update\n",
    "\n",
    "sudo apt-get install neo4j=3.3.0\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "open links:\n",
    "\n",
    "+ neo sandbox: https://neo4j.com/sandbox-v2/#\n",
    "+ movie datasets\n",
    "    + http://www.omdbapi.com/\n",
    "    + https://grouplens.org/datasets/movielens/\n",
    "+ csv bulk load example: https://neo4j.com/developer/guide-import-csv/\n",
    "    + could load northwind example dataset: https://neo4j.com/developer/guide-importing-data-and-etl/\n",
    "    + also northwind: https://code.google.com/archive/p/northwindextended/downloads\n",
    "    + another alternative: consumer complaints: https://catalog.data.gov/dataset/consumer-complaint-database\n",
    "+ paradise papers: https://www.icij.org/investigations/paradise-papers/explore-politicians-paradise-papers/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "***<div align=\"center\">Charlie could have delivered that letter with `neo4j`</div>***\n",
    "<div align=\"center\"><img src=\"https://i.ytimg.com/vi/ghrdSTC66MA/hqdefault.jpg\" width=\"600px\"></div>\n",
    "\n",
    "# END OF LECTURE\n",
    "\n",
    "next lecture: [`redshift`](012_dbs_5_redshift.ipynb)"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}