05_Answering_Business_Questions_using_SQL.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.8"
    },
    "colab": {
      "name": "05-Answering-Business-Questions-using-SQL.ipynb",
      "provenance": [],
      "include_colab_link": true
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/lis-r-barreto/Data-Engineer-Career-Path/blob/main/05_Answering_Business_Questions_using_SQL.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UAswxw-5y372"
      },
      "source": [
        "## Introduction and Schema Diagram"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "S0-v2Owly378"
      },
      "source": [
        "%%capture\n",
        "%load_ext sql\n",
        "%sql sqlite:///chinook.db"
      ],
      "execution_count": 1,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RipHESFWy38A"
      },
      "source": [
        "## Overview of the Data"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "XNBT_v1Xy38F",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 77
        },
        "outputId": "df7848ac-3248-459c-fdb4-4fdc90fbd956"
      },
      "source": [
        "%%sql\n",
        "SELECT\n",
        "    name,\n",
        "    type\n",
        "FROM sqlite_master\n",
        "WHERE type IN (\"table\",\"view\");"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            " * sqlite:///chinook.db\n",
            "Done.\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<table>\n",
              "    <thead>\n",
              "        <tr>\n",
              "            <th>name</th>\n",
              "            <th>type</th>\n",
              "        </tr>\n",
              "    </thead>\n",
              "    <tbody>\n",
              "    </tbody>\n",
              "</table>"
            ],
            "text/plain": [
              "[]"
            ]
          },
          "metadata": {},
          "execution_count": 2
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IqBlL9idy38H"
      },
      "source": [
        "## Selecting New Albums to Purchase"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ugmbvr6ly38J",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "5ed9d92f-5218-4f4a-846b-662a3b9d7daa"
      },
      "source": [
        "%%sql\n",
        "\n",
        "WITH usa_tracks_sold AS\n",
        "   (\n",
        "    SELECT il.* FROM invoice_line il\n",
        "    INNER JOIN invoice i on il.invoice_id = i.invoice_id\n",
        "    INNER JOIN customer c on i.customer_id = c.customer_id\n",
        "    WHERE c.country = \"USA\"\n",
        "   )\n",
        "\n",
        "SELECT\n",
        "    g.name genre,\n",
        "    count(uts.invoice_line_id) tracks_sold,\n",
        "    cast(count(uts.invoice_line_id) AS FLOAT) / (\n",
        "        SELECT COUNT(*) from usa_tracks_sold\n",
        "    ) percentage_sold\n",
        "FROM usa_tracks_sold uts\n",
        "INNER JOIN track t on t.track_id = uts.track_id\n",
        "INNER JOIN genre g on g.genre_id = t.genre_id\n",
        "GROUP BY 1\n",
        "ORDER BY 2 DESC\n",
        "LIMIT 10;"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            " * sqlite:///chinook.db\n",
            "(sqlite3.OperationalError) no such table: invoice_line\n",
            "[SQL: WITH usa_tracks_sold AS\n",
            "   (\n",
            "    SELECT il.* FROM invoice_line il\n",
            "    INNER JOIN invoice i on il.invoice_id = i.invoice_id\n",
            "    INNER JOIN customer c on i.customer_id = c.customer_id\n",
            "    WHERE c.country = \"USA\"\n",
            "   )\n",
            "\n",
            "SELECT\n",
            "    g.name genre,\n",
            "    count(uts.invoice_line_id) tracks_sold,\n",
            "    cast(count(uts.invoice_line_id) AS FLOAT) / (\n",
            "        SELECT COUNT(*) from usa_tracks_sold\n",
            "    ) percentage_sold\n",
            "FROM usa_tracks_sold uts\n",
            "INNER JOIN track t on t.track_id = uts.track_id\n",
            "INNER JOIN genre g on g.genre_id = t.genre_id\n",
            "GROUP BY 1\n",
            "ORDER BY 2 DESC\n",
            "LIMIT 10;]\n",
            "(Background on this error at: https://sqlalche.me/e/14/e3q8)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cej5mqPKy38M"
      },
      "source": [
        "Based on the sales of tracks across different genres in the USA, we should purchase the new albums by the following artists:\n",
        "\n",
        "- Red Tone (Punk)\n",
        "- Slim Jim Bites (Blues)\n",
        "- Meteor and the Girls (Pop)\n",
        "\n",
        "It's worth keeping in mind that combined, these three genres only make up only 17% of total sales, so we should be on the lookout for artists and albums from the 'rock' genre, which accounts for 53% of sales."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "K_AArk7Yy38O"
      },
      "source": [
        "## Analyzing Employee Sales Performance"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_Q1CzFOHy38P",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "85419b29-6b0c-4e99-b774-5b902131d23c"
      },
      "source": [
        "%%sql\n",
        "\n",
        "WITH customer_support_rep_sales AS\n",
        "    (\n",
        "     SELECT\n",
        "         i.customer_id,\n",
        "         c.support_rep_id,\n",
        "         SUM(i.total) total\n",
        "     FROM invoice i\n",
        "     INNER JOIN customer c ON i.customer_id = c.customer_id\n",
        "     GROUP BY 1,2\n",
        "    )\n",
        "\n",
        "SELECT\n",
        "    e.first_name || \" \" || e.last_name employee,\n",
        "    e.hire_date,\n",
        "    SUM(csrs.total) total_sales\n",
        "FROM customer_support_rep_sales csrs\n",
        "INNER JOIN employee e ON e.employee_id = csrs.support_rep_id\n",
        "GROUP BY 1;"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            " * sqlite:///chinook.db\n",
            "(sqlite3.OperationalError) no such table: invoice\n",
            "[SQL: WITH customer_support_rep_sales AS\n",
            "    (\n",
            "     SELECT\n",
            "         i.customer_id,\n",
            "         c.support_rep_id,\n",
            "         SUM(i.total) total\n",
            "     FROM invoice i\n",
            "     INNER JOIN customer c ON i.customer_id = c.customer_id\n",
            "     GROUP BY 1,2\n",
            "    )\n",
            "\n",
            "SELECT\n",
            "    e.first_name || \" \" || e.last_name employee,\n",
            "    e.hire_date,\n",
            "    SUM(csrs.total) total_sales\n",
            "FROM customer_support_rep_sales csrs\n",
            "INNER JOIN employee e ON e.employee_id = csrs.support_rep_id\n",
            "GROUP BY 1;]\n",
            "(Background on this error at: https://sqlalche.me/e/14/e3q8)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2LqzlJFyy38T"
      },
      "source": [
        "While there is a 20% difference in sales between Jane (the top employee) and Steve (the bottom employee), the difference roughly corresponds with the differences in their hiring dates."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TsbVXPDZy38V"
      },
      "source": [
        "## Analyzing Sales by Country"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Gz3HDp1Qy38W",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "62da2298-4014-4355-88fe-80f0109348ec"
      },
      "source": [
        "%%sql\n",
        "\n",
        "WITH country_or_other AS\n",
        "    (\n",
        "     SELECT\n",
        "       CASE\n",
        "           WHEN (\n",
        "                 SELECT count(*)\n",
        "                 FROM customer\n",
        "                 where country = c.country\n",
        "                ) = 1 THEN \"Other\"\n",
        "           ELSE c.country\n",
        "       END AS country,\n",
        "       c.customer_id,\n",
        "       il.*\n",
        "     FROM invoice_line il\n",
        "     INNER JOIN invoice i ON i.invoice_id = il.invoice_id\n",
        "     INNER JOIN customer c ON c.customer_id = i.customer_id\n",
        "    )\n",
        "\n",
        "SELECT\n",
        "    country,\n",
        "    customers,\n",
        "    total_sales,\n",
        "    average_order,\n",
        "    customer_lifetime_value\n",
        "FROM\n",
        "    (\n",
        "    SELECT\n",
        "        country,\n",
        "        count(distinct customer_id) customers,\n",
        "        SUM(unit_price) total_sales,\n",
        "        SUM(unit_price) / count(distinct customer_id) customer_lifetime_value,\n",
        "        SUM(unit_price) / count(distinct invoice_id) average_order,\n",
        "        CASE\n",
        "            WHEN country = \"Other\" THEN 1\n",
        "            ELSE 0\n",
        "        END AS sort\n",
        "    FROM country_or_other\n",
        "    GROUP BY country\n",
        "    ORDER BY sort ASC, total_sales DESC\n",
        "    );"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            " * sqlite:///chinook.db\n",
            "(sqlite3.OperationalError) no such table: invoice_line\n",
            "[SQL: WITH country_or_other AS\n",
            "    (\n",
            "     SELECT\n",
            "       CASE\n",
            "           WHEN (\n",
            "                 SELECT count(*)\n",
            "                 FROM customer\n",
            "                 where country = c.country\n",
            "                ) = 1 THEN \"Other\"\n",
            "           ELSE c.country\n",
            "       END AS country,\n",
            "       c.customer_id,\n",
            "       il.*\n",
            "     FROM invoice_line il\n",
            "     INNER JOIN invoice i ON i.invoice_id = il.invoice_id\n",
            "     INNER JOIN customer c ON c.customer_id = i.customer_id\n",
            "    )\n",
            "\n",
            "SELECT\n",
            "    country,\n",
            "    customers,\n",
            "    total_sales,\n",
            "    average_order,\n",
            "    customer_lifetime_value\n",
            "FROM\n",
            "    (\n",
            "    SELECT\n",
            "        country,\n",
            "        count(distinct customer_id) customers,\n",
            "        SUM(unit_price) total_sales,\n",
            "        SUM(unit_price) / count(distinct customer_id) customer_lifetime_value,\n",
            "        SUM(unit_price) / count(distinct invoice_id) average_order,\n",
            "        CASE\n",
            "            WHEN country = \"Other\" THEN 1\n",
            "            ELSE 0\n",
            "        END AS sort\n",
            "    FROM country_or_other\n",
            "    GROUP BY country\n",
            "    ORDER BY sort ASC, total_sales DESC\n",
            "    );]\n",
            "(Background on this error at: https://sqlalche.me/e/14/e3q8)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Q5K7fzQey38Y"
      },
      "source": [
        "Based on the data, there may be opportunity in the following countries:\n",
        "\n",
        "- Czech Republic\n",
        "- United Kingdom\n",
        "- India\n",
        "\n",
        "It's worth keeping in mind that because the amount of data from each of these countries is relatively low.  Because of this, we should be cautious spending too much money on new marketing campaigns, as the sample size is not large enough to give us high confidence.  A better approach would be to run small campaigns in these countries, collecting and analyzing the new customers to make sure that these trends hold with new customers."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PSdXHZLny38Z"
      },
      "source": [
        "## Albums vs Individual Tracks"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "i1LW-mr4y38a",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "c1523e73-1ecd-469a-9818-2157f5bc3f3c"
      },
      "source": [
        "%%sql\n",
        "\n",
        "WITH invoice_first_track AS\n",
        "    (\n",
        "     SELECT\n",
        "         il.invoice_id invoice_id,\n",
        "         MIN(il.track_id) first_track_id\n",
        "     FROM invoice_line il\n",
        "     GROUP BY 1\n",
        "    )\n",
        "\n",
        "SELECT\n",
        "    album_purchase,\n",
        "    COUNT(invoice_id) number_of_invoices,\n",
        "    CAST(count(invoice_id) AS FLOAT) / (\n",
        "                                         SELECT COUNT(*) FROM invoice\n",
        "                                      ) percent\n",
        "FROM\n",
        "    (\n",
        "    SELECT\n",
        "        ifs.*,\n",
        "        CASE\n",
        "            WHEN\n",
        "                 (\n",
        "                  SELECT t.track_id FROM track t\n",
        "                  WHERE t.album_id = (\n",
        "                                      SELECT t2.album_id FROM track t2\n",
        "                                      WHERE t2.track_id = ifs.first_track_id\n",
        "                                     ) \n",
        "\n",
        "                  EXCEPT \n",
        "\n",
        "                  SELECT il2.track_id FROM invoice_line il2\n",
        "                  WHERE il2.invoice_id = ifs.invoice_id\n",
        "                 ) IS NULL\n",
        "             AND\n",
        "                 (\n",
        "                  SELECT il2.track_id FROM invoice_line il2\n",
        "                  WHERE il2.invoice_id = ifs.invoice_id\n",
        "\n",
        "                  EXCEPT \n",
        "\n",
        "                  SELECT t.track_id FROM track t\n",
        "                  WHERE t.album_id = (\n",
        "                                      SELECT t2.album_id FROM track t2\n",
        "                                      WHERE t2.track_id = ifs.first_track_id\n",
        "                                     ) \n",
        "                 ) IS NULL\n",
        "             THEN \"yes\"\n",
        "             ELSE \"no\"\n",
        "         END AS \"album_purchase\"\n",
        "     FROM invoice_first_track ifs\n",
        "    )\n",
        "GROUP BY album_purchase;"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            " * sqlite:///chinook.db\n",
            "(sqlite3.OperationalError) no such table: invoice_line\n",
            "[SQL: WITH invoice_first_track AS\n",
            "    (\n",
            "     SELECT\n",
            "         il.invoice_id invoice_id,\n",
            "         MIN(il.track_id) first_track_id\n",
            "     FROM invoice_line il\n",
            "     GROUP BY 1\n",
            "    )\n",
            "\n",
            "SELECT\n",
            "    album_purchase,\n",
            "    COUNT(invoice_id) number_of_invoices,\n",
            "    CAST(count(invoice_id) AS FLOAT) / (\n",
            "                                         SELECT COUNT(*) FROM invoice\n",
            "                                      ) percent\n",
            "FROM\n",
            "    (\n",
            "    SELECT\n",
            "        ifs.*,\n",
            "        CASE\n",
            "            WHEN\n",
            "                 (\n",
            "                  SELECT t.track_id FROM track t\n",
            "                  WHERE t.album_id = (\n",
            "                                      SELECT t2.album_id FROM track t2\n",
            "                                      WHERE t2.track_id = ifs.first_track_id\n",
            "                                     ) \n",
            "\n",
            "                  EXCEPT \n",
            "\n",
            "                  SELECT il2.track_id FROM invoice_line il2\n",
            "                  WHERE il2.invoice_id = ifs.invoice_id\n",
            "                 ) IS NULL\n",
            "             AND\n",
            "                 (\n",
            "                  SELECT il2.track_id FROM invoice_line il2\n",
            "                  WHERE il2.invoice_id = ifs.invoice_id\n",
            "\n",
            "                  EXCEPT \n",
            "\n",
            "                  SELECT t.track_id FROM track t\n",
            "                  WHERE t.album_id = (\n",
            "                                      SELECT t2.album_id FROM track t2\n",
            "                                      WHERE t2.track_id = ifs.first_track_id\n",
            "                                     ) \n",
            "                 ) IS NULL\n",
            "             THEN \"yes\"\n",
            "             ELSE \"no\"\n",
            "         END AS \"album_purchase\"\n",
            "     FROM invoice_first_track ifs\n",
            "    )\n",
            "GROUP BY album_purchase;]\n",
            "(Background on this error at: https://sqlalche.me/e/14/e3q8)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "URmilfRay38b"
      },
      "source": [
        "Album purchases account for 18.6% of purchases.  Based on this data, I would recommend against purchasing only select tracks from albums from record companies, since there is potential to lose one fifth of revenue."
      ]
    }
  ]
}