analysis.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "fc43a51c",
   "metadata": {},
   "source": [
    "# Analyzing FEMA's National Flood Insurance Program (NFIP) Data With DuckDB\n",
    "Author: Mark Bauer"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "61d43be8",
   "metadata": {},
   "source": [
    "# Introduction\n",
    "\n",
    "Refer to the [Introduction section](https://github.com/mebauer/duckdb-fema-nfip) on the project's main page."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ad12bbb4",
   "metadata": {},
   "source": [
    "# OpenFEMA Dataset: FIMA NFIP Redacted Claims - v2\n",
    "Federal Emergency Management Agency (FEMA), OpenFEMA Dataset: FIMA NFIP Redacted Claims - v2. Retrieved from https://www.fema.gov/openfema-data-page/fima-nfip-redacted-claims-v2. This product uses the FEMA OpenFEMA API, but is not endorsed by FEMA. The Federal Government or FEMA cannot vouch for the data or analyses derived from these data after the data have been retrieved from the Agency's website(s).\n",
    "\n",
    "Read more about OpenFEMA's  [Terms and Conditions](https://www.fema.gov/about/openfema/terms-conditions).\n",
    "\n",
    "**Dataset Description**:\n",
    ">This dataset provides details on NFIP claims transactions. It is derived from the NFIP system of record, staged in the NFIP reporting platform and redacted to protect policy holder personally identifiable information.\n",
    ">\n",
    ">This dataset is not intended to be an official federal report, and should not be considered an official federal report.\n",
    "\n",
    "**About the National Flood Insurance Program**:   \n",
    ">Congress passed the National Flood Insurance Act (NFIA), 42 U.S.C. 4001 in 1968, creating the National Flood Insurance Program (NFIP) in order to reduce future flood losses through flood hazard identification, floodplain management, and providing insurance protection. The Department of Housing and Urban Development (HUD) originally administered the NFIP, and Congress subsequently transferred the NFIP to FEMA upon its creation in 1979. FEMA and insurance companies participating in FEMA's Write Your Own (WYO) program offer NFIP insurance coverage for building structures as well as for contents and personal property within the building structures to eligible and insurable properties. The WYO program began in 1983 with NFIP operating under Part B of the NFIA and allows FEMA to authorize private insurance companies to issue the Standard Flood Insurance Policy (SFIP) as FEMA's fiduciary and fiscal agent. FEMA administers NFIP by ensuring insurance applications are processed properly; determining correct premiums; renewing, reforming, and cancelling insurance policies; transferring policies from the seller of the property to the purchaser of the property in certain circumstances; and processing insurance claims.\n",
    ">\n",
    ">The paid premiums of SFIPs and claims payments for damaged property are processed through the National Flood Insurance Fund (NFIF). NFIF was established by the National Flood Insurance Act of 1968 (42 U.S.C. 4001, et seq.), and is a centralized premium revenue and fee-generated fund that supports NFIP, which holds these U.S. Treasury funds.\n",
    ">\n",
    ">The Flood Insurance Claims Manual (https://nfipservices.floodsmart.gov/insurance-manuals) provides claims guidance to WYOs, vendors, adjusters, and examiners so that policyholders experience consistent and reliable service. The Manual provides processes for handling claims from the notice of loss to final payment. The NFIP has provided answers to Frequently Asked Questions (FAQs) to assist the public in understanding and navigating the data our program makes available: https://www.fema.gov/sites/default/files/documents/fema_nfip-data-faqs.pdf.\n",
    "\n",
    "**Data Dictionary**:  \n",
    "View the data dictionary on OpenFEMA under the [Data Fields](https://www.fema.gov/openfema-data-page/fima-nfip-redacted-claims-v2) section."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "92bb3fc0",
   "metadata": {},
   "outputs": [],
   "source": [
    "# import libraries\n",
    "import duckdb\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "2abfd2f0",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Python implementation: CPython\n",
      "Python version       : 3.11.0\n",
      "IPython version      : 8.6.0\n",
      "\n",
      "duckdb: 1.0.0\n",
      "pandas: 1.5.1\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# reproducibility\n",
    "%reload_ext watermark\n",
    "%watermark -v -p duckdb,pandas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "3d2634ce",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "FimaNfipClaims.csv\r\n"
     ]
    }
   ],
   "source": [
    "# examine datasets\n",
    "%ls data/"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e99b3d8e",
   "metadata": {},
   "source": [
    "# Redacted Claims: Before Getting Started\n",
    "Please note that this dataset provides details on NFIP claims transactions and is ***redacted*** to protect policy holder personally identifiable information. The claim's `latitude` and `longitude` field **should not be used to represent the precise location of the insured building**. From the data dictionary:\n",
    "\n",
    ">Latitude: **Approximate latitude of the insured building (to 1 decimal place)**. This represents the approximate location of the insured property. The precision has been lessened to ensure individual privacy. This may result in a point location that exists in an incorrect county or state. Use the state and county fields for record aggregation for these dimensions.\n",
    ">\n",
    ">Longitude: **Approximate longitude of the insured building (to 1 decimal place)**. This represents the approximate location of the insured property. The precision has been lessened to ensure individual privacy. This may result in a point location that exists in an incorrect county or state. Use the state and county fields for record aggregation for these dimensions.\n",
    "\n",
    "For more information, visit the [Data Dictionary](https://www.fema.gov/openfema-data-page/fima-nfip-redacted-claims-v2) and review [Frequently Asked Questions about NFIP Policies and Claims Data](https://nfipservices.floodsmart.gov/frequently-asked-questions-about-nfip-policies-and-claims-data)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "33fdda62",
   "metadata": {},
   "source": [
    "This analysis presents all financial figures in nominal dollars. No adjustments for inflation have been made, meaning that the values reflect the actual dollar amounts at the time of measurement. Additionally, the date and time of data access can be found in the download-data notebook."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a60a95b9",
   "metadata": {},
   "source": [
    "# Create a DuckDB database instance using the Python client"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "683565a1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "┌──────────────────────┬──────────────────────┬──────────────────────┬───┬──────────┬───────────┬──────────────────────┐\n",
      "│ agricultureStructu…  │       asOfDate       │ basementEnclosureC…  │ … │ latitude │ longitude │          id          │\n",
      "│        int64         │      timestamp       │        int64         │   │  double  │  double   │       varchar        │\n",
      "├──────────────────────┼──────────────────────┼──────────────────────┼───┼──────────┼───────────┼──────────────────────┤\n",
      "│                    0 │ 2020-11-13 14:50:3…  │                    2 │ … │     30.3 │     -87.7 │ e7af3d9f-b605-4653…  │\n",
      "│                    0 │ 2020-12-11 16:25:4…  │                    0 │ … │     30.3 │     -87.7 │ bbaeaf64-c162-41bf…  │\n",
      "│                    0 │ 2020-03-27 12:15:4…  │                    0 │ … │     33.5 │    -112.1 │ 256da746-b30b-4129…  │\n",
      "│                    0 │ 2020-03-27 12:15:4…  │                    0 │ … │     33.5 │    -112.1 │ e3dcbb27-a2d0-4a9e…  │\n",
      "│                    0 │ 2020-03-26 12:56:2…  │                    0 │ … │     34.4 │    -119.6 │ f77efb94-0188-4fa0…  │\n",
      "├──────────────────────┴──────────────────────┴──────────────────────┴───┴──────────┴───────────┴──────────────────────┤\n",
      "│ 5 rows                                                                                          73 columns (6 shown) │\n",
      "└──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# create a DuckDB database instance\n",
    "con = duckdb.connect()\n",
    "\n",
    "# create table claims of dataset\n",
    "con.execute(\"\"\"\n",
    "    CREATE TABLE claims AS\n",
    "        FROM read_csv('data/FimaNfipClaims.csv')\n",
    "\"\"\")\n",
    "\n",
    "# sanity check\n",
    "con.sql(\"\"\"\n",
    "    SELECT *\n",
    "    FROM claims\n",
    "    LIMIT 5\n",
    "\"\"\").show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "6d8efba5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>database</th>\n",
       "      <th>schema</th>\n",
       "      <th>name</th>\n",
       "      <th>column_names</th>\n",
       "      <th>column_types</th>\n",
       "      <th>temporary</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>memory</td>\n",
       "      <td>main</td>\n",
       "      <td>claims</td>\n",
       "      <td>[agricultureStructureIndicator, asOfDate, base...</td>\n",
       "      <td>[BIGINT, TIMESTAMP, BIGINT, BIGINT, BIGINT, TI...</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  database schema    name                                       column_names  \\\n",
       "0   memory   main  claims  [agricultureStructureIndicator, asOfDate, base...   \n",
       "\n",
       "                                        column_types  temporary  \n",
       "0  [BIGINT, TIMESTAMP, BIGINT, BIGINT, BIGINT, TI...      False  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# list tables and schemas\n",
    "con.sql(\"SHOW ALL TABLES\").df()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "1684eb6f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "┌────────────┐\n",
       "│ count_rows │\n",
       "│   int64    │\n",
       "├────────────┤\n",
       "│    2671486 │\n",
       "└────────────┘"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# count of rows\n",
    "con.sql(\"\"\"\n",
    "    SELECT\n",
    "        COUNT(*) AS count_rows\n",
    "    FROM\n",
    "        claims\n",
    "\"\"\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "23137a78",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "┌───────────────┐\n",
       "│ count_columns │\n",
       "│     int64     │\n",
       "├───────────────┤\n",
       "│            73 │\n",
       "└───────────────┘"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# count of columns\n",
    "con.sql(\"\"\"\n",
    "    SELECT\n",
    "        COUNT(column_name) AS count_columns\n",
    "    FROM\n",
    "        (DESCRIBE FROM claims)\n",
    "\"\"\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "8f6fe0c9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "┌─────────────────────────┐\n",
       "│        asOfDate         │\n",
       "│        timestamp        │\n",
       "├─────────────────────────┤\n",
       "│ 2024-10-04 00:59:55.552 │\n",
       "└─────────────────────────┘"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# last refreshed\n",
    "con.sql(\"\"\"\n",
    "    SELECT\n",
    "        asOfDate\n",
    "    FROM\n",
    "        claims\n",
    "    ORDER BY\n",
    "        asOfDate DESC\n",
    "    LIMIT 1\n",
    "\"\"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "59aadc50",
   "metadata": {},
   "source": [
    "# Examine Dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6ea658f8",
   "metadata": {},
   "source": [
    "## Column Info"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "946b22cb",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "┌────────────────────────────────────────────┬─────────────┐\n",
      "│                column_name                 │ column_type │\n",
      "│                  varchar                   │   varchar   │\n",
      "├────────────────────────────────────────────┼─────────────┤\n",
      "│ agricultureStructureIndicator              │ BIGINT      │\n",
      "│ asOfDate                                   │ TIMESTAMP   │\n",
      "│ basementEnclosureCrawlspaceType            │ BIGINT      │\n",
      "│ policyCount                                │ BIGINT      │\n",
      "│ crsClassificationCode                      │ BIGINT      │\n",
      "│ dateOfLoss                                 │ TIMESTAMP   │\n",
      "│ elevatedBuildingIndicator                  │ BIGINT      │\n",
      "│ elevationCertificateIndicator              │ VARCHAR     │\n",
      "│ elevationDifference                        │ DOUBLE      │\n",
      "│ baseFloodElevation                         │ DOUBLE      │\n",
      "│ ratedFloodZone                             │ VARCHAR     │\n",
      "│ houseWorship                               │ BIGINT      │\n",
      "│ locationOfContents                         │ BIGINT      │\n",
      "│ lowestAdjacentGrade                        │ DOUBLE      │\n",
      "│ lowestFloorElevation                       │ DOUBLE      │\n",
      "│ numberOfFloorsInTheInsuredBuilding         │ BIGINT      │\n",
      "│ nonProfitIndicator                         │ BIGINT      │\n",
      "│ obstructionType                            │ BIGINT      │\n",
      "│ occupancyType                              │ BIGINT      │\n",
      "│ originalConstructionDate                   │ TIMESTAMP   │\n",
      "│ originalNBDate                             │ TIMESTAMP   │\n",
      "│ amountPaidOnBuildingClaim                  │ DOUBLE      │\n",
      "│ amountPaidOnContentsClaim                  │ DOUBLE      │\n",
      "│ amountPaidOnIncreasedCostOfComplianceClaim │ DOUBLE      │\n",
      "│ postFIRMConstructionIndicator              │ BIGINT      │\n",
      "│ rateMethod                                 │ VARCHAR     │\n",
      "│ smallBusinessIndicatorBuilding             │ BIGINT      │\n",
      "│ totalBuildingInsuranceCoverage             │ BIGINT      │\n",
      "│ totalContentsInsuranceCoverage             │ BIGINT      │\n",
      "│ yearOfLoss                                 │ BIGINT      │\n",
      "│ primaryResidenceIndicator                  │ BIGINT      │\n",
      "│ buildingDamageAmount                       │ BIGINT      │\n",
      "│ buildingDeductibleCode                     │ VARCHAR     │\n",
      "│ netBuildingPaymentAmount                   │ DOUBLE      │\n",
      "│ buildingPropertyValue                      │ BIGINT      │\n",
      "│ causeOfDamage                              │ VARCHAR     │\n",
      "│ condominiumCoverageTypeCode                │ VARCHAR     │\n",
      "│ contentsDamageAmount                       │ BIGINT      │\n",
      "│ contentsDeductibleCode                     │ VARCHAR     │\n",
      "│ netContentsPaymentAmount                   │ DOUBLE      │\n",
      "│ contentsPropertyValue                      │ BIGINT      │\n",
      "│ disasterAssistanceCoverageRequired         │ BIGINT      │\n",
      "│ eventDesignationNumber                     │ VARCHAR     │\n",
      "│ ficoNumber                                 │ BIGINT      │\n",
      "│ floodCharacteristicsIndicator              │ BIGINT      │\n",
      "│ floodWaterDuration                         │ BIGINT      │\n",
      "│ floodproofedIndicator                      │ BIGINT      │\n",
      "│ floodEvent                                 │ VARCHAR     │\n",
      "│ iccCoverage                                │ BIGINT      │\n",
      "│ netIccPaymentAmount                        │ DOUBLE      │\n",
      "│ nfipRatedCommunityNumber                   │ VARCHAR     │\n",
      "│ nfipCommunityNumberCurrent                 │ VARCHAR     │\n",
      "│ nfipCommunityName                          │ VARCHAR     │\n",
      "│ nonPaymentReasonContents                   │ VARCHAR     │\n",
      "│ nonPaymentReasonBuilding                   │ VARCHAR     │\n",
      "│ numberOfUnits                              │ BIGINT      │\n",
      "│ buildingReplacementCost                    │ BIGINT      │\n",
      "│ contentsReplacementCost                    │ BIGINT      │\n",
      "│ replacementCostBasis                       │ VARCHAR     │\n",
      "│ stateOwnedIndicator                        │ BIGINT      │\n",
      "│ waterDepth                                 │ BIGINT      │\n",
      "│ floodZoneCurrent                           │ VARCHAR     │\n",
      "│ buildingDescriptionCode                    │ BIGINT      │\n",
      "│ rentalPropertyIndicator                    │ BIGINT      │\n",
      "│ state                                      │ VARCHAR     │\n",
      "│ reportedCity                               │ VARCHAR     │\n",
      "│ reportedZipCode                            │ VARCHAR     │\n",
      "│ countyCode                                 │ VARCHAR     │\n",
      "│ censusTract                                │ VARCHAR     │\n",
      "│ censusBlockGroupFips                       │ VARCHAR     │\n",
      "│ latitude                                   │ DOUBLE      │\n",
      "│ longitude                                  │ DOUBLE      │\n",
      "│ id                                         │ VARCHAR     │\n",
      "├────────────────────────────────────────────┴─────────────┤\n",
      "│ 73 rows                                        2 columns │\n",
      "└──────────────────────────────────────────────────────────┘\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# examine column datatypes\n",
    "con.sql(\"\"\"\n",
    "    SELECT\n",
    "        column_name,\n",
    "        column_type\n",
    "    FROM\n",
    "        (DESCRIBE claims)\n",
    "\"\"\").show(max_rows=80)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "83987c2d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "┌────────────────────────────────────────────┬─────────────────┐\n",
      "│                column_name                 │ null_percentage │\n",
      "│                  varchar                   │  decimal(9,2)   │\n",
      "├────────────────────────────────────────────┼─────────────────┤\n",
      "│ floodCharacteristicsIndicator              │           98.51 │\n",
      "│ eventDesignationNumber                     │           94.34 │\n",
      "│ lowestAdjacentGrade                        │           81.26 │\n",
      "│ crsClassificationCode                      │           80.65 │\n",
      "│ nonPaymentReasonBuilding                   │           78.02 │\n",
      "│ elevationCertificateIndicator              │           77.72 │\n",
      "│ lowestFloorElevation                       │           76.38 │\n",
      "│ baseFloodElevation                         │           75.70 │\n",
      "│ elevationDifference                        │           72.95 │\n",
      "│ floodZoneCurrent                           │           72.15 │\n",
      "│ nfipCommunityNumberCurrent                 │           72.10 │\n",
      "│ nfipCommunityName                          │           70.76 │\n",
      "│ basementEnclosureCrawlspaceType            │           69.92 │\n",
      "│ nonPaymentReasonContents                   │           68.95 │\n",
      "│ buildingDescriptionCode                    │           64.28 │\n",
      "│ contentsReplacementCost                    │           59.75 │\n",
      "│ contentsDamageAmount                       │           59.72 │\n",
      "│ contentsPropertyValue                      │           59.72 │\n",
      "│ obstructionType                            │           43.74 │\n",
      "│ iccCoverage                                │           36.32 │\n",
      "│ ficoNumber                                 │           35.27 │\n",
      "│ locationOfContents                         │           33.42 │\n",
      "│ floodEvent                                 │           28.48 │\n",
      "│ buildingDamageAmount                       │           23.76 │\n",
      "│ buildingPropertyValue                      │           23.76 │\n",
      "│ buildingReplacementCost                    │           23.76 │\n",
      "│ amountPaidOnBuildingClaim                  │           22.78 │\n",
      "│ amountPaidOnContentsClaim                  │           22.78 │\n",
      "│ amountPaidOnIncreasedCostOfComplianceClaim │           22.78 │\n",
      "│ contentsDeductibleCode                     │           20.56 │\n",
      "│ disasterAssistanceCoverageRequired         │           13.45 │\n",
      "│ buildingDeductibleCode                     │           11.77 │\n",
      "│ waterDepth                                 │            9.40 │\n",
      "│ floodWaterDuration                         │            9.06 │\n",
      "│ replacementCostBasis                       │            8.56 │\n",
      "│ ratedFloodZone                             │            5.20 │\n",
      "│ censusTract                                │            5.16 │\n",
      "│ censusBlockGroupFips                       │            5.16 │\n",
      "│ countyCode                                 │            2.32 │\n",
      "│ causeOfDamage                              │            2.26 │\n",
      "│ reportedZipCode                            │            2.09 │\n",
      "│ rateMethod                                 │            1.85 │\n",
      "│ latitude                                   │            1.51 │\n",
      "│ longitude                                  │            1.51 │\n",
      "│ condominiumCoverageTypeCode                │            1.40 │\n",
      "│ numberOfFloorsInTheInsuredBuilding         │            0.65 │\n",
      "│ numberOfUnits                              │            0.11 │\n",
      "│ originalConstructionDate                   │            0.03 │\n",
      "│ occupancyType                              │            0.02 │\n",
      "├────────────────────────────────────────────┴─────────────────┤\n",
      "│ 49 rows                                            2 columns │\n",
      "└──────────────────────────────────────────────────────────────┘\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# approximate column null percentage\n",
    "con.sql(\"\"\"\n",
    "    SELECT\n",
    "        column_name,\n",
    "        null_percentage\n",
    "    FROM\n",
    "        (SUMMARIZE FROM claims)\n",
    "    WHERE\n",
    "        null_percentage > 0\n",
    "    ORDER BY\n",
    "        null_percentage DESC\n",
    "\"\"\").show(max_rows=80)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "310f9ef4",
   "metadata": {},
   "source": [
    "## Preview Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "f4b54f07",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "┌──────────────────────┬──────────────────────┬──────────────────────┬───┬──────────┬───────────┬──────────────────────┐\n",
      "│ agricultureStructu…  │       asOfDate       │ basementEnclosureC…  │ … │ latitude │ longitude │          id          │\n",
      "│        int64         │      timestamp       │        int64         │   │  double  │  double   │       varchar        │\n",
      "├──────────────────────┼──────────────────────┼──────────────────────┼───┼──────────┼───────────┼──────────────────────┤\n",
      "│                    0 │ 2020-11-13 14:50:3…  │                    2 │ … │     30.3 │     -87.7 │ e7af3d9f-b605-4653…  │\n",
      "│                    0 │ 2020-12-11 16:25:4…  │                    0 │ … │     30.3 │     -87.7 │ bbaeaf64-c162-41bf…  │\n",
      "│                    0 │ 2020-03-27 12:15:4…  │                    0 │ … │     33.5 │    -112.1 │ 256da746-b30b-4129…  │\n",
      "│                    0 │ 2020-03-27 12:15:4…  │                    0 │ … │     33.5 │    -112.1 │ e3dcbb27-a2d0-4a9e…  │\n",
      "│                    0 │ 2020-03-26 12:56:2…  │                    0 │ … │     34.4 │    -119.6 │ f77efb94-0188-4fa0…  │\n",
      "├──────────────────────┴──────────────────────┴──────────────────────┴───┴──────────┴───────────┴──────────────────────┤\n",
      "│ 5 rows                                                                                          73 columns (6 shown) │\n",
      "└──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# preview data\n",
    "con.sql(\"\"\"\n",
    "    SELECT *\n",
    "    FROM claims\n",
    "    LIMIT 5\n",
    "\"\"\").show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "cb2030c2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>agricultureStructureIndicator</th>\n",
       "      <th>asOfDate</th>\n",
       "      <th>basementEnclosureCrawlspaceType</th>\n",
       "      <th>policyCount</th>\n",
       "      <th>crsClassificationCode</th>\n",
       "      <th>dateOfLoss</th>\n",
       "      <th>elevatedBuildingIndicator</th>\n",
       "      <th>elevationCertificateIndicator</th>\n",
       "      <th>elevationDifference</th>\n",
       "      <th>baseFloodElevation</th>\n",
       "      <th>ratedFloodZone</th>\n",
       "      <th>houseWorship</th>\n",
       "      <th>locationOfContents</th>\n",
       "      <th>lowestAdjacentGrade</th>\n",
       "      <th>lowestFloorElevation</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>2020-11-13 14:50:38.288</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2020-09-16</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>6.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>AE</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>16.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>2020-12-11 16:25:40.587</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2020-09-16</td>\n",
       "      <td>1</td>\n",
       "      <td>None</td>\n",
       "      <td>4.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>AE</td>\n",
       "      <td>0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>4.4</td>\n",
       "      <td>14.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>2020-03-27 12:15:45.887</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2016-08-02</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>AH</td>\n",
       "      <td>0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>2020-03-27 12:15:45.887</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2014-09-08</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>AH</td>\n",
       "      <td>0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>2020-03-26 12:56:27.476</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2018-01-09</td>\n",
       "      <td>1</td>\n",
       "      <td>None</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>AE</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   agricultureStructureIndicator                asOfDate  \\\n",
       "0                              0 2020-11-13 14:50:38.288   \n",
       "1                              0 2020-12-11 16:25:40.587   \n",
       "2                              0 2020-03-27 12:15:45.887   \n",
       "3                              0 2020-03-27 12:15:45.887   \n",
       "4                              0 2020-03-26 12:56:27.476   \n",
       "\n",
       "   basementEnclosureCrawlspaceType  policyCount  crsClassificationCode  \\\n",
       "0                                2            1                    NaN   \n",
       "1                                0            1                    NaN   \n",
       "2                                0            1                    NaN   \n",
       "3                                0            1                    NaN   \n",
       "4                                0            1                    NaN   \n",
       "\n",
       "  dateOfLoss  elevatedBuildingIndicator elevationCertificateIndicator  \\\n",
       "0 2020-09-16                          0                          None   \n",
       "1 2020-09-16                          1                          None   \n",
       "2 2016-08-02                          0                             2   \n",
       "3 2014-09-08                          0                             2   \n",
       "4 2018-01-09                          1                          None   \n",
       "\n",
       "   elevationDifference  baseFloodElevation ratedFloodZone  houseWorship  \\\n",
       "0                  6.0                10.0             AE             0   \n",
       "1                  4.0                10.0             AE             0   \n",
       "2                  NaN                 NaN             AH             0   \n",
       "3                  NaN                 NaN             AH             0   \n",
       "4                  NaN                 NaN             AE             0   \n",
       "\n",
       "   locationOfContents  lowestAdjacentGrade  lowestFloorElevation  \n",
       "0                 NaN                  0.0                  16.0  \n",
       "1                 7.0                  4.4                  14.4  \n",
       "2                 3.0                  NaN                   NaN  \n",
       "3                 3.0                  NaN                   NaN  \n",
       "4                 NaN                  NaN                   NaN  "
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# preview data as pandas dataframe for readability\n",
    "sql = \"\"\"\n",
    "    SELECT *\n",
    "    FROM claims\n",
    "    LIMIT 5\n",
    "\"\"\"\n",
    "\n",
    "# examine each column in sections because of large number of columns\n",
    "con.sql(sql).df().iloc[:, :15]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "5403e1f1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>numberOfFloorsInTheInsuredBuilding</th>\n",
       "      <th>nonProfitIndicator</th>\n",
       "      <th>obstructionType</th>\n",
       "      <th>occupancyType</th>\n",
       "      <th>originalConstructionDate</th>\n",
       "      <th>originalNBDate</th>\n",
       "      <th>amountPaidOnBuildingClaim</th>\n",
       "      <th>amountPaidOnContentsClaim</th>\n",
       "      <th>amountPaidOnIncreasedCostOfComplianceClaim</th>\n",
       "      <th>postFIRMConstructionIndicator</th>\n",
       "      <th>rateMethod</th>\n",
       "      <th>smallBusinessIndicatorBuilding</th>\n",
       "      <th>totalBuildingInsuranceCoverage</th>\n",
       "      <th>totalContentsInsuranceCoverage</th>\n",
       "      <th>yearOfLoss</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2</td>\n",
       "      <td>1975-01-01</td>\n",
       "      <td>2008-10-20</td>\n",
       "      <td>2695.44</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>8</td>\n",
       "      <td>0</td>\n",
       "      <td>7400</td>\n",
       "      <td>0</td>\n",
       "      <td>2020</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>54.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1991-09-17</td>\n",
       "      <td>2017-10-05</td>\n",
       "      <td>2750.18</td>\n",
       "      <td>121.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>8</td>\n",
       "      <td>0</td>\n",
       "      <td>10800</td>\n",
       "      <td>700</td>\n",
       "      <td>2020</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "      <td>1949-01-01</td>\n",
       "      <td>2014-09-02</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>B</td>\n",
       "      <td>0</td>\n",
       "      <td>153500</td>\n",
       "      <td>15800</td>\n",
       "      <td>2016</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "      <td>1949-01-01</td>\n",
       "      <td>2014-09-02</td>\n",
       "      <td>25156.27</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>B</td>\n",
       "      <td>0</td>\n",
       "      <td>139500</td>\n",
       "      <td>15000</td>\n",
       "      <td>2014</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1960-01-01</td>\n",
       "      <td>2001-08-05</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>250000</td>\n",
       "      <td>0</td>\n",
       "      <td>2018</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   numberOfFloorsInTheInsuredBuilding  nonProfitIndicator  obstructionType  \\\n",
       "0                                   2                   0              NaN   \n",
       "1                                   3                   0             54.0   \n",
       "2                                   1                   0              NaN   \n",
       "3                                   1                   0              NaN   \n",
       "4                                   1                   0             10.0   \n",
       "\n",
       "   occupancyType originalConstructionDate originalNBDate  \\\n",
       "0              2               1975-01-01     2008-10-20   \n",
       "1              1               1991-09-17     2017-10-05   \n",
       "2              1               1949-01-01     2014-09-02   \n",
       "3              1               1949-01-01     2014-09-02   \n",
       "4              1               1960-01-01     2001-08-05   \n",
       "\n",
       "   amountPaidOnBuildingClaim  amountPaidOnContentsClaim  \\\n",
       "0                    2695.44                        0.0   \n",
       "1                    2750.18                      121.0   \n",
       "2                        NaN                        NaN   \n",
       "3                   25156.27                        0.0   \n",
       "4                        NaN                        NaN   \n",
       "\n",
       "   amountPaidOnIncreasedCostOfComplianceClaim  postFIRMConstructionIndicator  \\\n",
       "0                                         0.0                              1   \n",
       "1                                         0.0                              1   \n",
       "2                                         NaN                              0   \n",
       "3                                         0.0                              0   \n",
       "4                                         NaN                              0   \n",
       "\n",
       "  rateMethod  smallBusinessIndicatorBuilding  totalBuildingInsuranceCoverage  \\\n",
       "0          8                               0                            7400   \n",
       "1          8                               0                           10800   \n",
       "2          B                               0                          153500   \n",
       "3          B                               0                          139500   \n",
       "4          1                               0                          250000   \n",
       "\n",
       "   totalContentsInsuranceCoverage  yearOfLoss  \n",
       "0                               0        2020  \n",
       "1                             700        2020  \n",
       "2                           15800        2016  \n",
       "3                           15000        2014  \n",
       "4                               0        2018  "
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# slice through columns\n",
    "con.sql(sql).df().iloc[:, 15:30]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "0b6de0df",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>primaryResidenceIndicator</th>\n",
       "      <th>buildingDamageAmount</th>\n",
       "      <th>buildingDeductibleCode</th>\n",
       "      <th>netBuildingPaymentAmount</th>\n",
       "      <th>buildingPropertyValue</th>\n",
       "      <th>causeOfDamage</th>\n",
       "      <th>condominiumCoverageTypeCode</th>\n",
       "      <th>contentsDamageAmount</th>\n",
       "      <th>contentsDeductibleCode</th>\n",
       "      <th>netContentsPaymentAmount</th>\n",
       "      <th>contentsPropertyValue</th>\n",
       "      <th>disasterAssistanceCoverageRequired</th>\n",
       "      <th>eventDesignationNumber</th>\n",
       "      <th>ficoNumber</th>\n",
       "      <th>floodCharacteristicsIndicator</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>3945.0</td>\n",
       "      <td>F</td>\n",
       "      <td>2695.44</td>\n",
       "      <td>167531.0</td>\n",
       "      <td>1</td>\n",
       "      <td>N</td>\n",
       "      <td>0.0</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>AL0520</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>4000.0</td>\n",
       "      <td>F</td>\n",
       "      <td>2750.18</td>\n",
       "      <td>278027.0</td>\n",
       "      <td>1</td>\n",
       "      <td>N</td>\n",
       "      <td>1371.0</td>\n",
       "      <td>F</td>\n",
       "      <td>121.0</td>\n",
       "      <td>50000.0</td>\n",
       "      <td>0</td>\n",
       "      <td>AL0520</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>1698.0</td>\n",
       "      <td>5</td>\n",
       "      <td>0.00</td>\n",
       "      <td>140422.0</td>\n",
       "      <td>4</td>\n",
       "      <td>N</td>\n",
       "      <td>3690.0</td>\n",
       "      <td>5</td>\n",
       "      <td>0.0</td>\n",
       "      <td>21500.0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>30098.0</td>\n",
       "      <td>5</td>\n",
       "      <td>25156.27</td>\n",
       "      <td>140919.0</td>\n",
       "      <td>4</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2</td>\n",
       "      <td>0.00</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4</td>\n",
       "      <td>N</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>368.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   primaryResidenceIndicator  buildingDamageAmount buildingDeductibleCode  \\\n",
       "0                          0                3945.0                      F   \n",
       "1                          1                4000.0                      F   \n",
       "2                          1                1698.0                      5   \n",
       "3                          1               30098.0                      5   \n",
       "4                          1                   NaN                      2   \n",
       "\n",
       "   netBuildingPaymentAmount  buildingPropertyValue causeOfDamage  \\\n",
       "0                   2695.44               167531.0             1   \n",
       "1                   2750.18               278027.0             1   \n",
       "2                      0.00               140422.0             4   \n",
       "3                  25156.27               140919.0             4   \n",
       "4                      0.00                    NaN             4   \n",
       "\n",
       "  condominiumCoverageTypeCode  contentsDamageAmount contentsDeductibleCode  \\\n",
       "0                           N                   0.0                   None   \n",
       "1                           N                1371.0                      F   \n",
       "2                           N                3690.0                      5   \n",
       "3                           N                   NaN                      5   \n",
       "4                           N                   NaN                      0   \n",
       "\n",
       "   netContentsPaymentAmount  contentsPropertyValue  \\\n",
       "0                       0.0                    0.0   \n",
       "1                     121.0                50000.0   \n",
       "2                       0.0                21500.0   \n",
       "3                       0.0                    NaN   \n",
       "4                       0.0                    NaN   \n",
       "\n",
       "   disasterAssistanceCoverageRequired eventDesignationNumber  ficoNumber  \\\n",
       "0                                   0                 AL0520         NaN   \n",
       "1                                   0                 AL0520         NaN   \n",
       "2                                   0                   None         NaN   \n",
       "3                                   0                   None         NaN   \n",
       "4                                   0                   None       368.0   \n",
       "\n",
       "   floodCharacteristicsIndicator  \n",
       "0                            NaN  \n",
       "1                            NaN  \n",
       "2                            NaN  \n",
       "3                            NaN  \n",
       "4                            NaN  "
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# slice through columns\n",
    "con.sql(sql).df().iloc[:, 30:45]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "ca9fe289",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>floodWaterDuration</th>\n",
       "      <th>floodproofedIndicator</th>\n",
       "      <th>floodEvent</th>\n",
       "      <th>iccCoverage</th>\n",
       "      <th>netIccPaymentAmount</th>\n",
       "      <th>nfipRatedCommunityNumber</th>\n",
       "      <th>nfipCommunityNumberCurrent</th>\n",
       "      <th>nfipCommunityName</th>\n",
       "      <th>nonPaymentReasonContents</th>\n",
       "      <th>nonPaymentReasonBuilding</th>\n",
       "      <th>numberOfUnits</th>\n",
       "      <th>buildingReplacementCost</th>\n",
       "      <th>contentsReplacementCost</th>\n",
       "      <th>replacementCostBasis</th>\n",
       "      <th>stateOwnedIndicator</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>Hurricane Sally</td>\n",
       "      <td>30000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>015005</td>\n",
       "      <td>0000</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>0</td>\n",
       "      <td>185457.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>A</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>Hurricane Sally</td>\n",
       "      <td>30000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>015005</td>\n",
       "      <td>0000</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>0</td>\n",
       "      <td>314835.0</td>\n",
       "      <td>75000.0</td>\n",
       "      <td>A</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>30000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>040051</td>\n",
       "      <td>0000</td>\n",
       "      <td>None</td>\n",
       "      <td>01</td>\n",
       "      <td>01</td>\n",
       "      <td>0</td>\n",
       "      <td>175528.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>R</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>30000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>040051</td>\n",
       "      <td>0000</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>0</td>\n",
       "      <td>173974.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>R</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>Mid-Winter California Flooding</td>\n",
       "      <td>30000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>060331</td>\n",
       "      <td>0000</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>06</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>A</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   floodWaterDuration  floodproofedIndicator                      floodEvent  \\\n",
       "0                 NaN                      0                 Hurricane Sally   \n",
       "1                 NaN                      0                 Hurricane Sally   \n",
       "2                 0.0                      0                            None   \n",
       "3                 0.0                      0                            None   \n",
       "4                 0.0                      0  Mid-Winter California Flooding   \n",
       "\n",
       "   iccCoverage  netIccPaymentAmount nfipRatedCommunityNumber  \\\n",
       "0        30000                  0.0                   015005   \n",
       "1        30000                  0.0                   015005   \n",
       "2        30000                  0.0                   040051   \n",
       "3        30000                  0.0                   040051   \n",
       "4        30000                  0.0                   060331   \n",
       "\n",
       "  nfipCommunityNumberCurrent nfipCommunityName nonPaymentReasonContents  \\\n",
       "0                       0000              None                     None   \n",
       "1                       0000              None                     None   \n",
       "2                       0000              None                       01   \n",
       "3                       0000              None                     None   \n",
       "4                       0000              None                     None   \n",
       "\n",
       "  nonPaymentReasonBuilding  numberOfUnits  buildingReplacementCost  \\\n",
       "0                     None              0                 185457.0   \n",
       "1                     None              0                 314835.0   \n",
       "2                       01              0                 175528.0   \n",
       "3                     None              0                 173974.0   \n",
       "4                       06              0                      NaN   \n",
       "\n",
       "   contentsReplacementCost replacementCostBasis  stateOwnedIndicator  \n",
       "0                      0.0                    A                    0  \n",
       "1                  75000.0                    A                    0  \n",
       "2                      0.0                    R                    0  \n",
       "3                      NaN                    R                    0  \n",
       "4                      NaN                    A                    0  "
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# slice through columns\n",
    "con.sql(sql).df().iloc[:, 45:60]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "238eff68",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>waterDepth</th>\n",
       "      <th>floodZoneCurrent</th>\n",
       "      <th>buildingDescriptionCode</th>\n",
       "      <th>rentalPropertyIndicator</th>\n",
       "      <th>state</th>\n",
       "      <th>reportedCity</th>\n",
       "      <th>reportedZipCode</th>\n",
       "      <th>countyCode</th>\n",
       "      <th>censusTract</th>\n",
       "      <th>censusBlockGroupFips</th>\n",
       "      <th>latitude</th>\n",
       "      <th>longitude</th>\n",
       "      <th>id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>AL</td>\n",
       "      <td>Currently Unavailable</td>\n",
       "      <td>36542</td>\n",
       "      <td>01003</td>\n",
       "      <td>01003011407</td>\n",
       "      <td>010030114073</td>\n",
       "      <td>30.3</td>\n",
       "      <td>-87.7</td>\n",
       "      <td>e7af3d9f-b605-4653-958f-bc48f413766c</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>AE</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>AL</td>\n",
       "      <td>Currently Unavailable</td>\n",
       "      <td>36542</td>\n",
       "      <td>01003</td>\n",
       "      <td>01003011407</td>\n",
       "      <td>010030114072</td>\n",
       "      <td>30.3</td>\n",
       "      <td>-87.7</td>\n",
       "      <td>bbaeaf64-c162-41bf-a399-862d0e832447</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>None</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>AZ</td>\n",
       "      <td>Currently Unavailable</td>\n",
       "      <td>85015</td>\n",
       "      <td>04013</td>\n",
       "      <td>04013108902</td>\n",
       "      <td>040131089022</td>\n",
       "      <td>33.5</td>\n",
       "      <td>-112.1</td>\n",
       "      <td>256da746-b30b-4129-a391-93f50a6190c8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>None</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>AZ</td>\n",
       "      <td>Currently Unavailable</td>\n",
       "      <td>85015</td>\n",
       "      <td>04013</td>\n",
       "      <td>04013108902</td>\n",
       "      <td>040131089022</td>\n",
       "      <td>33.5</td>\n",
       "      <td>-112.1</td>\n",
       "      <td>e3dcbb27-a2d0-4a9e-a70d-ff7e578d4e6a</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>AE</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>CA</td>\n",
       "      <td>Currently Unavailable</td>\n",
       "      <td>93108</td>\n",
       "      <td>06083</td>\n",
       "      <td>06083001402</td>\n",
       "      <td>060830014022</td>\n",
       "      <td>34.4</td>\n",
       "      <td>-119.6</td>\n",
       "      <td>f77efb94-0188-4fa0-b52e-3aa71499baeb</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   waterDepth floodZoneCurrent  buildingDescriptionCode  \\\n",
       "0           0             None                        1   \n",
       "1           0               AE                        1   \n",
       "2           1             None                        1   \n",
       "3           1             None                        1   \n",
       "4           0               AE                        1   \n",
       "\n",
       "   rentalPropertyIndicator state           reportedCity reportedZipCode  \\\n",
       "0                        0    AL  Currently Unavailable           36542   \n",
       "1                        0    AL  Currently Unavailable           36542   \n",
       "2                        0    AZ  Currently Unavailable           85015   \n",
       "3                        0    AZ  Currently Unavailable           85015   \n",
       "4                        0    CA  Currently Unavailable           93108   \n",
       "\n",
       "  countyCode  censusTract censusBlockGroupFips  latitude  longitude  \\\n",
       "0      01003  01003011407         010030114073      30.3      -87.7   \n",
       "1      01003  01003011407         010030114072      30.3      -87.7   \n",
       "2      04013  04013108902         040131089022      33.5     -112.1   \n",
       "3      04013  04013108902         040131089022      33.5     -112.1   \n",
       "4      06083  06083001402         060830014022      34.4     -119.6   \n",
       "\n",
       "                                     id  \n",
       "0  e7af3d9f-b605-4653-958f-bc48f413766c  \n",
       "1  bbaeaf64-c162-41bf-a399-862d0e832447  \n",
       "2  256da746-b30b-4129-a391-93f50a6190c8  \n",
       "3  e3dcbb27-a2d0-4a9e-a70d-ff7e578d4e6a  \n",
       "4  f77efb94-0188-4fa0-b52e-3aa71499baeb  "
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# slice through columns\n",
    "con.sql(sql).df().iloc[:, 60:]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "20b8858a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "┌─────────┬───────┐\n",
      "│   id    │ count │\n",
      "│ varchar │ int64 │\n",
      "├─────────┴───────┤\n",
      "│     0 rows      │\n",
      "└─────────────────┘\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# count duplicate IDs\n",
    "con.sql(\"\"\"\n",
    "    SELECT\n",
    "        id,\n",
    "        COUNT(id) AS count\n",
    "    FROM\n",
    "        claims\n",
    "    GROUP BY\n",
    "        id\n",
    "    HAVING\n",
    "        count > 1\n",
    "\"\"\").show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "659a28b1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "┌─────────────────────────┬─────────────────────────┐\n",
      "│    earliestAsOfDate     │     latestAsOfDate      │\n",
      "│        timestamp        │        timestamp        │\n",
      "├─────────────────────────┼─────────────────────────┤\n",
      "│ 2019-09-19 06:12:43.388 │ 2024-10-04 00:59:55.552 │\n",
      "└─────────────────────────┴─────────────────────────┘\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# earliest and latest record effective date\n",
    "con.sql(\"\"\"\n",
    "    SELECT\n",
    "        min(asOfDate) AS earliestAsOfDate,\n",
    "        max(asOfDate) AS latestAsOfDate\n",
    "    FROM\n",
    "        claims\n",
    "\"\"\").show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "f3e83ac5",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "┌─────────────────────────┬─────────────────────┬────────────────────────┬─────────┬────────────────┐\n",
      "│        asOfDate         │     dateOfLoss      │       floodEvent       │  state  │ paidTotalClaim │\n",
      "│        timestamp        │      timestamp      │        varchar         │ varchar │     int64      │\n",
      "├─────────────────────────┼─────────────────────┼────────────────────────┼─────────┼────────────────┤\n",
      "│ 2024-10-04 00:59:55.552 │ 2024-09-16 00:00:00 │ Tropical Cyclone Eight │ NC      │              0 │\n",
      "│ 2024-10-04 00:58:55.563 │ 2024-09-11 00:00:00 │ Hurricane Francine     │ LA      │           2901 │\n",
      "│ 2024-10-04 00:57:54.885 │ 2024-09-11 00:00:00 │ Hurricane Francine     │ LA      │              0 │\n",
      "│ 2024-10-04 00:57:54.695 │ 2024-09-26 00:00:00 │ Hurricane Helene       │ FL      │           7500 │\n",
      "│ 2024-10-04 00:56:54.47  │ 2024-09-11 00:00:00 │ Hurricane Francine     │ LA      │          57179 │\n",
      "└─────────────────────────┴─────────────────────┴────────────────────────┴─────────┴────────────────┘\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# top 5 most recent effective date claim records\n",
    "con.sql(\"\"\"\n",
    "    SELECT\n",
    "        asOfDate,\n",
    "        dateOfLoss,\n",
    "        floodEvent,\n",
    "        state,\n",
    "        ROUND(\n",
    "            amountPaidOnBuildingClaim\n",
    "            + amountPaidOnContentsClaim\n",
    "            + amountPaidOnIncreasedCostOfComplianceClaim)::BIGINT AS paidTotalClaim\n",
    "    FROM\n",
    "        claims\n",
    "    ORDER BY\n",
    "        asOfDate DESC\n",
    "    LIMIT 5\n",
    "\"\"\").show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "aead7c8b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "┌─────────────────────┬─────────────────────┐\n",
      "│ earliestDateOfLoss  │  latestDateOfLoss   │\n",
      "│      timestamp      │      timestamp      │\n",
      "├─────────────────────┼─────────────────────┤\n",
      "│ 1978-01-01 00:00:00 │ 2024-10-03 00:00:00 │\n",
      "└─────────────────────┴─────────────────────┘\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# earliest and latest date of loss in dataset\n",
    "con.sql(\"\"\"\n",
    "    SELECT\n",
    "        min(dateOfLoss) AS earliestDateOfLoss,\n",
    "        max(dateOfLoss) AS latestDateOfLoss\n",
    "    FROM\n",
    "        claims\n",
    "\"\"\").show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "5f22f864",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "┌─────────────────────┬─────────────────────────┬────────────┬─────────┬────────────────┐\n",
      "│     dateOfLoss      │        asOfDate         │ floodEvent │  state  │ paidTotalClaim │\n",
      "│      timestamp      │        timestamp        │  varchar   │ varchar │     int64      │\n",
      "├─────────────────────┼─────────────────────────┼────────────┼─────────┼────────────────┤\n",
      "│ 2024-10-03 00:00:00 │ 2024-10-03 22:38:35.955 │ NULL       │ FL      │           NULL │\n",
      "│ 2024-10-03 00:00:00 │ 2024-10-03 21:40:08.5   │ NULL       │ FL      │           NULL │\n",
      "│ 2024-10-03 00:00:00 │ 2024-10-03 17:21:21.871 │ NULL       │ FL      │           NULL │\n",
      "│ 2024-10-02 00:00:00 │ 2024-10-03 21:40:08.5   │ NULL       │ FL      │           NULL │\n",
      "│ 2024-10-02 00:00:00 │ 2024-10-03 21:40:08.5   │ NULL       │ FL      │           NULL │\n",
      "└─────────────────────┴─────────────────────────┴────────────┴─────────┴────────────────┘\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# top 5 latest claim records by date of loss \n",
    "con.sql(\"\"\"\n",
    "    SELECT\n",
    "        dateOfLoss,\n",
    "        asOfDate,\n",
    "        floodEvent,\n",
    "        state,\n",
    "        ROUND(\n",
    "            amountPaidOnBuildingClaim\n",
    "            + amountPaidOnContentsClaim\n",
    "            + amountPaidOnIncreasedCostOfComplianceClaim)::BIGINT AS paidTotalClaim\n",
    "    FROM\n",
    "        claims\n",
    "    ORDER BY\n",
    "        dateOfLoss DESC, asOfDate DESC\n",
    "    LIMIT 5\n",
    "\"\"\").show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "097ffcf7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "┌──────────────────┐\n",
      "│ totalPolicyCount │\n",
      "│      int128      │\n",
      "├──────────────────┤\n",
      "│          3462669 │\n",
      "└──────────────────┘\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# total insured units in dataset\n",
    "con.sql(\"\"\"\n",
    "    SELECT\n",
    "        SUM(policyCount) AS totalPolicyCount\n",
    "    FROM\n",
    "        claims\n",
    "\"\"\").show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7df3a801",
   "metadata": {},
   "source": [
    "Policy Count:\n",
    ">Insured units in an active status. A policy contract ceases to be in an active status as of the cancellation date or the expiration date. Residential Condominium Building Association Policy (RCBAP) contracts are stored as a single policy contract but insure multiple units and therefore represent multiple policies."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ac593dd8",
   "metadata": {},
   "source": [
    "## Summary Statistics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "39581469",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>column_name</th>\n",
       "      <th>column_type</th>\n",
       "      <th>min</th>\n",
       "      <th>max</th>\n",
       "      <th>approx_unique</th>\n",
       "      <th>avg</th>\n",
       "      <th>std</th>\n",
       "      <th>q25</th>\n",
       "      <th>q50</th>\n",
       "      <th>q75</th>\n",
       "      <th>count</th>\n",
       "      <th>null_percentage</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>agricultureStructureIndicator</td>\n",
       "      <td>BIGINT</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0.00014598616650059182</td>\n",
       "      <td>0.012081593817782434</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2671486</td>\n",
       "      <td>0.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>asOfDate</td>\n",
       "      <td>TIMESTAMP</td>\n",
       "      <td>2019-09-19 06:12:43.388</td>\n",
       "      <td>2024-10-04 00:59:55.552</td>\n",
       "      <td>479449</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>2671486</td>\n",
       "      <td>0.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>basementEnclosureCrawlspaceType</td>\n",
       "      <td>BIGINT</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>1.195105377264796</td>\n",
       "      <td>1.0618378842847185</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>2671486</td>\n",
       "      <td>69.92</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>policyCount</td>\n",
       "      <td>BIGINT</td>\n",
       "      <td>1</td>\n",
       "      <td>1090</td>\n",
       "      <td>399</td>\n",
       "      <td>1.2961583927447122</td>\n",
       "      <td>6.742744486702049</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2671486</td>\n",
       "      <td>0.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>crsClassificationCode</td>\n",
       "      <td>BIGINT</td>\n",
       "      <td>1</td>\n",
       "      <td>10</td>\n",
       "      <td>10</td>\n",
       "      <td>6.708258215272243</td>\n",
       "      <td>1.526304305976192</td>\n",
       "      <td>5</td>\n",
       "      <td>7</td>\n",
       "      <td>8</td>\n",
       "      <td>2671486</td>\n",
       "      <td>80.65</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>68</th>\n",
       "      <td>censusTract</td>\n",
       "      <td>VARCHAR</td>\n",
       "      <td>01001020100</td>\n",
       "      <td>78030961200</td>\n",
       "      <td>59577</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>2671486</td>\n",
       "      <td>5.16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>69</th>\n",
       "      <td>censusBlockGroupFips</td>\n",
       "      <td>VARCHAR</td>\n",
       "      <td>010010201001</td>\n",
       "      <td>780309612002</td>\n",
       "      <td>117722</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>2671486</td>\n",
       "      <td>5.16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>70</th>\n",
       "      <td>latitude</td>\n",
       "      <td>DOUBLE</td>\n",
       "      <td>-36.0</td>\n",
       "      <td>69.9</td>\n",
       "      <td>334</td>\n",
       "      <td>33.755900244380626</td>\n",
       "      <td>5.827326735346257</td>\n",
       "      <td>29.807589746046872</td>\n",
       "      <td>30.645513794090203</td>\n",
       "      <td>39.66243830088295</td>\n",
       "      <td>2671486</td>\n",
       "      <td>1.51</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>71</th>\n",
       "      <td>longitude</td>\n",
       "      <td>DOUBLE</td>\n",
       "      <td>-170.7</td>\n",
       "      <td>174.0</td>\n",
       "      <td>715</td>\n",
       "      <td>-86.02134344014759</td>\n",
       "      <td>10.940268931584674</td>\n",
       "      <td>-91.13371113348664</td>\n",
       "      <td>-86.49085404693089</td>\n",
       "      <td>-77.44163542302745</td>\n",
       "      <td>2671486</td>\n",
       "      <td>1.51</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>72</th>\n",
       "      <td>id</td>\n",
       "      <td>VARCHAR</td>\n",
       "      <td>0000094b-1d1a-4dbe-a944-97c62b225b9c</td>\n",
       "      <td>fffffe7c-7ca2-4431-b88b-8a8216ffc934</td>\n",
       "      <td>2623176</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>2671486</td>\n",
       "      <td>0.00</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>73 rows × 12 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                        column_name column_type  \\\n",
       "0     agricultureStructureIndicator      BIGINT   \n",
       "1                          asOfDate   TIMESTAMP   \n",
       "2   basementEnclosureCrawlspaceType      BIGINT   \n",
       "3                       policyCount      BIGINT   \n",
       "4             crsClassificationCode      BIGINT   \n",
       "..                              ...         ...   \n",
       "68                      censusTract     VARCHAR   \n",
       "69             censusBlockGroupFips     VARCHAR   \n",
       "70                         latitude      DOUBLE   \n",
       "71                        longitude      DOUBLE   \n",
       "72                               id     VARCHAR   \n",
       "\n",
       "                                     min  \\\n",
       "0                                      0   \n",
       "1                2019-09-19 06:12:43.388   \n",
       "2                                      0   \n",
       "3                                      1   \n",
       "4                                      1   \n",
       "..                                   ...   \n",
       "68                           01001020100   \n",
       "69                          010010201001   \n",
       "70                                 -36.0   \n",
       "71                                -170.7   \n",
       "72  0000094b-1d1a-4dbe-a944-97c62b225b9c   \n",
       "\n",
       "                                     max  approx_unique  \\\n",
       "0                                      1              2   \n",
       "1                2024-10-04 00:59:55.552         479449   \n",
       "2                                      4              4   \n",
       "3                                   1090            399   \n",
       "4                                     10             10   \n",
       "..                                   ...            ...   \n",
       "68                           78030961200          59577   \n",
       "69                          780309612002         117722   \n",
       "70                                  69.9            334   \n",
       "71                                 174.0            715   \n",
       "72  fffffe7c-7ca2-4431-b88b-8a8216ffc934        2623176   \n",
       "\n",
       "                       avg                   std                 q25  \\\n",
       "0   0.00014598616650059182  0.012081593817782434                   0   \n",
       "1                     None                  None                None   \n",
       "2        1.195105377264796    1.0618378842847185                   0   \n",
       "3       1.2961583927447122     6.742744486702049                   1   \n",
       "4        6.708258215272243     1.526304305976192                   5   \n",
       "..                     ...                   ...                 ...   \n",
       "68                    None                  None                None   \n",
       "69                    None                  None                None   \n",
       "70      33.755900244380626     5.827326735346257  29.807589746046872   \n",
       "71      -86.02134344014759    10.940268931584674  -91.13371113348664   \n",
       "72                    None                  None                None   \n",
       "\n",
       "                   q50                 q75    count  null_percentage  \n",
       "0                    0                   0  2671486             0.00  \n",
       "1                 None                None  2671486             0.00  \n",
       "2                    1                   2  2671486            69.92  \n",
       "3                    1                   1  2671486             0.00  \n",
       "4                    7                   8  2671486            80.65  \n",
       "..                 ...                 ...      ...              ...  \n",
       "68                None                None  2671486             5.16  \n",
       "69                None                None  2671486             5.16  \n",
       "70  30.645513794090203   39.66243830088295  2671486             1.51  \n",
       "71  -86.49085404693089  -77.44163542302745  2671486             1.51  \n",
       "72                None                None  2671486             0.00  \n",
       "\n",
       "[73 rows x 12 columns]"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# calculate summary statistics of each column, pandas df for readability\n",
    "con.sql(\"\"\"\n",
    "    SELECT *\n",
    "    FROM (SUMMARIZE claims)\n",
    "\"\"\").df()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "92350b9c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "┌──────────────────────┬─────────────┬──────────────────────┬───┬────────────────────┬─────────┬─────────────────┐\n",
      "│     column_name      │ column_type │         min          │ … │        q75         │  count  │ null_percentage │\n",
      "│       varchar        │   varchar   │       varchar        │   │      varchar       │  int64  │  decimal(9,2)   │\n",
      "├──────────────────────┼─────────────┼──────────────────────┼───┼────────────────────┼─────────┼─────────────────┤\n",
      "│ agricultureStructu…  │ BIGINT      │ 0                    │ … │ 0                  │ 2671486 │            0.00 │\n",
      "│ asOfDate             │ TIMESTAMP   │ 2019-09-19 06:12:4…  │ … │ NULL               │ 2671486 │            0.00 │\n",
      "│ basementEnclosureC…  │ BIGINT      │ 0                    │ … │ 2                  │ 2671486 │           69.92 │\n",
      "│ policyCount          │ BIGINT      │ 1                    │ … │ 1                  │ 2671486 │            0.00 │\n",
      "│ crsClassificationC…  │ BIGINT      │ 1                    │ … │ 8                  │ 2671486 │           80.65 │\n",
      "│ dateOfLoss           │ TIMESTAMP   │ 1978-01-01 00:00:00  │ … │ NULL               │ 2671486 │            0.00 │\n",
      "│ elevatedBuildingIn…  │ BIGINT      │ 0                    │ … │ 0                  │ 2671486 │            0.00 │\n",
      "│ elevationCertifica…  │ VARCHAR     │ 1                    │ … │ NULL               │ 2671486 │           77.72 │\n",
      "│ elevationDifference  │ DOUBLE      │ -9989.0              │ … │ 2.8495704836170574 │ 2671486 │           72.95 │\n",
      "│ baseFloodElevation   │ DOUBLE      │ -9999.0              │ … │ 13.855669165578838 │ 2671486 │           75.70 │\n",
      "│         ·            │   ·         │ ·                    │ · │ ·                  │    ·    │              ·  │\n",
      "│         ·            │   ·         │ ·                    │ · │ ·                  │    ·    │              ·  │\n",
      "│         ·            │   ·         │ ·                    │ · │ ·                  │    ·    │              ·  │\n",
      "│ rentalPropertyIndi…  │ BIGINT      │ 0                    │ … │ 0                  │ 2671486 │            0.00 │\n",
      "│ state                │ VARCHAR     │ AK                   │ … │ NULL               │ 2671486 │            0.00 │\n",
      "│ reportedCity         │ VARCHAR     │ Currently Unavaila…  │ … │ NULL               │ 2671486 │            0.00 │\n",
      "│ reportedZipCode      │ VARCHAR     │ 00000                │ … │ NULL               │ 2671486 │            2.09 │\n",
      "│ countyCode           │ VARCHAR     │ 01001                │ … │ NULL               │ 2671486 │            2.32 │\n",
      "│ censusTract          │ VARCHAR     │ 01001020100          │ … │ NULL               │ 2671486 │            5.16 │\n",
      "│ censusBlockGroupFips │ VARCHAR     │ 010010201001         │ … │ NULL               │ 2671486 │            5.16 │\n",
      "│ latitude             │ DOUBLE      │ -36.0                │ … │ 39.68022931823674  │ 2671486 │            1.51 │\n",
      "│ longitude            │ DOUBLE      │ -170.7               │ … │ -77.51664811495554 │ 2671486 │            1.51 │\n",
      "│ id                   │ VARCHAR     │ 0000094b-1d1a-4dbe…  │ … │ NULL               │ 2671486 │            0.00 │\n",
      "├──────────────────────┴─────────────┴──────────────────────┴───┴────────────────────┴─────────┴─────────────────┤\n",
      "│ 73 rows (20 shown)                                                                        12 columns (6 shown) │\n",
      "└────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# build summarize relation\n",
    "summarize_relation = con.sql(\"SELECT * FROM (SUMMARIZE claims)\")\n",
    "\n",
    "# preview relation\n",
    "summarize_relation.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "acb4ed9b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>column_name</th>\n",
       "      <th>column_type</th>\n",
       "      <th>min</th>\n",
       "      <th>max</th>\n",
       "      <th>approx_unique</th>\n",
       "      <th>avg</th>\n",
       "      <th>std</th>\n",
       "      <th>q25</th>\n",
       "      <th>q50</th>\n",
       "      <th>q75</th>\n",
       "      <th>count</th>\n",
       "      <th>null_percentage</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>agricultureStructureIndicator</td>\n",
       "      <td>BIGINT</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0.00014598616650059182</td>\n",
       "      <td>0.012081593817782368</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2671486</td>\n",
       "      <td>0.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>asOfDate</td>\n",
       "      <td>TIMESTAMP</td>\n",
       "      <td>2019-09-19 06:12:43.388</td>\n",
       "      <td>2024-10-04 00:59:55.552</td>\n",
       "      <td>479449</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>2671486</td>\n",
       "      <td>0.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>basementEnclosureCrawlspaceType</td>\n",
       "      <td>BIGINT</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>1.195105377264796</td>\n",
       "      <td>1.0618378842847207</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>2671486</td>\n",
       "      <td>69.92</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>policyCount</td>\n",
       "      <td>BIGINT</td>\n",
       "      <td>1</td>\n",
       "      <td>1090</td>\n",
       "      <td>399</td>\n",
       "      <td>1.2961583927447122</td>\n",
       "      <td>6.74274448670205</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2671486</td>\n",
       "      <td>0.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>crsClassificationCode</td>\n",
       "      <td>BIGINT</td>\n",
       "      <td>1</td>\n",
       "      <td>10</td>\n",
       "      <td>10</td>\n",
       "      <td>6.708258215272243</td>\n",
       "      <td>1.526304305976192</td>\n",
       "      <td>5</td>\n",
       "      <td>7</td>\n",
       "      <td>8</td>\n",
       "      <td>2671486</td>\n",
       "      <td>80.65</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>dateOfLoss</td>\n",
       "      <td>TIMESTAMP</td>\n",
       "      <td>1978-01-01 00:00:00</td>\n",
       "      <td>2024-10-03 00:00:00</td>\n",
       "      <td>16868</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>2671486</td>\n",
       "      <td>0.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>elevatedBuildingIndicator</td>\n",
       "      <td>BIGINT</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0.18133877549798127</td>\n",
       "      <td>0.38529868877158163</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2671486</td>\n",
       "      <td>0.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>elevationCertificateIndicator</td>\n",
       "      <td>VARCHAR</td>\n",
       "      <td>1</td>\n",
       "      <td>E</td>\n",
       "      <td>9</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>2671486</td>\n",
       "      <td>77.72</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>elevationDifference</td>\n",
       "      <td>DOUBLE</td>\n",
       "      <td>-9989.0</td>\n",
       "      <td>998.0</td>\n",
       "      <td>371</td>\n",
       "      <td>1.253409617633019</td>\n",
       "      <td>29.091407737413885</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2.7474439219632734</td>\n",
       "      <td>2671486</td>\n",
       "      <td>72.95</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>baseFloodElevation</td>\n",
       "      <td>DOUBLE</td>\n",
       "      <td>-9999.0</td>\n",
       "      <td>9998.0</td>\n",
       "      <td>10427</td>\n",
       "      <td>126.73271926107897</td>\n",
       "      <td>767.4360045962655</td>\n",
       "      <td>6.808632423063275</td>\n",
       "      <td>9.00039181829393</td>\n",
       "      <td>13.93652892150554</td>\n",
       "      <td>2671486</td>\n",
       "      <td>75.70</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>ratedFloodZone</td>\n",
       "      <td>VARCHAR</td>\n",
       "      <td>A</td>\n",
       "      <td>X</td>\n",
       "      <td>73</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>2671486</td>\n",
       "      <td>5.20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>houseWorship</td>\n",
       "      <td>BIGINT</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0.0006876322765681722</td>\n",
       "      <td>0.026213731051500877</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2671486</td>\n",
       "      <td>0.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>locationOfContents</td>\n",
       "      <td>BIGINT</td>\n",
       "      <td>1</td>\n",
       "      <td>7</td>\n",
       "      <td>7</td>\n",
       "      <td>3.2105534797270328</td>\n",
       "      <td>0.9798814996946891</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>2671486</td>\n",
       "      <td>33.42</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>lowestAdjacentGrade</td>\n",
       "      <td>DOUBLE</td>\n",
       "      <td>-99999.9</td>\n",
       "      <td>9998.9</td>\n",
       "      <td>12638</td>\n",
       "      <td>49.6963843509632</td>\n",
       "      <td>1483.2503230525263</td>\n",
       "      <td>3.1482843304493584</td>\n",
       "      <td>6.085678173038944</td>\n",
       "      <td>10.72433481704699</td>\n",
       "      <td>2671486</td>\n",
       "      <td>81.26</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>lowestFloorElevation</td>\n",
       "      <td>DOUBLE</td>\n",
       "      <td>-9999.0</td>\n",
       "      <td>9998.9</td>\n",
       "      <td>13294</td>\n",
       "      <td>104.32951958281872</td>\n",
       "      <td>616.4135062983083</td>\n",
       "      <td>6.535693018702163</td>\n",
       "      <td>10.265360670159131</td>\n",
       "      <td>17.298499644768746</td>\n",
       "      <td>2671486</td>\n",
       "      <td>76.38</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                        column_name column_type                      min  \\\n",
       "0     agricultureStructureIndicator      BIGINT                        0   \n",
       "1                          asOfDate   TIMESTAMP  2019-09-19 06:12:43.388   \n",
       "2   basementEnclosureCrawlspaceType      BIGINT                        0   \n",
       "3                       policyCount      BIGINT                        1   \n",
       "4             crsClassificationCode      BIGINT                        1   \n",
       "5                        dateOfLoss   TIMESTAMP      1978-01-01 00:00:00   \n",
       "6         elevatedBuildingIndicator      BIGINT                        0   \n",
       "7     elevationCertificateIndicator     VARCHAR                        1   \n",
       "8               elevationDifference      DOUBLE                  -9989.0   \n",
       "9                baseFloodElevation      DOUBLE                  -9999.0   \n",
       "10                   ratedFloodZone     VARCHAR                        A   \n",
       "11                     houseWorship      BIGINT                        0   \n",
       "12               locationOfContents      BIGINT                        1   \n",
       "13              lowestAdjacentGrade      DOUBLE                 -99999.9   \n",
       "14             lowestFloorElevation      DOUBLE                  -9999.0   \n",
       "\n",
       "                        max  approx_unique                     avg  \\\n",
       "0                         1              2  0.00014598616650059182   \n",
       "1   2024-10-04 00:59:55.552         479449                    None   \n",
       "2                         4              4       1.195105377264796   \n",
       "3                      1090            399      1.2961583927447122   \n",
       "4                        10             10       6.708258215272243   \n",
       "5       2024-10-03 00:00:00          16868                    None   \n",
       "6                         1              2     0.18133877549798127   \n",
       "7                         E              9                    None   \n",
       "8                     998.0            371       1.253409617633019   \n",
       "9                    9998.0          10427      126.73271926107897   \n",
       "10                        X             73                    None   \n",
       "11                        1              2   0.0006876322765681722   \n",
       "12                        7              7      3.2105534797270328   \n",
       "13                   9998.9          12638        49.6963843509632   \n",
       "14                   9998.9          13294      104.32951958281872   \n",
       "\n",
       "                     std                 q25                 q50  \\\n",
       "0   0.012081593817782368                   0                   0   \n",
       "1                   None                None                None   \n",
       "2     1.0618378842847207                   0                   1   \n",
       "3       6.74274448670205                   1                   1   \n",
       "4      1.526304305976192                   5                   7   \n",
       "5                   None                None                None   \n",
       "6    0.38529868877158163                   0                   0   \n",
       "7                   None                None                None   \n",
       "8     29.091407737413885                 0.0                 1.0   \n",
       "9      767.4360045962655   6.808632423063275    9.00039181829393   \n",
       "10                  None                None                None   \n",
       "11  0.026213731051500877                   0                   0   \n",
       "12    0.9798814996946891                   3                   3   \n",
       "13    1483.2503230525263  3.1482843304493584   6.085678173038944   \n",
       "14     616.4135062983083   6.535693018702163  10.265360670159131   \n",
       "\n",
       "                   q75    count  null_percentage  \n",
       "0                    0  2671486             0.00  \n",
       "1                 None  2671486             0.00  \n",
       "2                    2  2671486            69.92  \n",
       "3                    1  2671486             0.00  \n",
       "4                    8  2671486            80.65  \n",
       "5                 None  2671486             0.00  \n",
       "6                    0  2671486             0.00  \n",
       "7                 None  2671486            77.72  \n",
       "8   2.7474439219632734  2671486            72.95  \n",
       "9    13.93652892150554  2671486            75.70  \n",
       "10                None  2671486             5.20  \n",
       "11                   0  2671486             0.00  \n",
       "12                   4  2671486            33.42  \n",
       "13   10.72433481704699  2671486            81.26  \n",
       "14  17.298499644768746  2671486            76.38  "
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# slice through columns for readability\n",
    "con.sql(\"SELECT * FROM summarize_relation\").df().iloc[:15, :]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "9784194b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>column_name</th>\n",
       "      <th>column_type</th>\n",
       "      <th>min</th>\n",
       "      <th>max</th>\n",
       "      <th>approx_unique</th>\n",
       "      <th>avg</th>\n",
       "      <th>std</th>\n",
       "      <th>q25</th>\n",
       "      <th>q50</th>\n",
       "      <th>q75</th>\n",
       "      <th>count</th>\n",
       "      <th>null_percentage</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>numberOfFloorsInTheInsuredBuilding</td>\n",
       "      <td>BIGINT</td>\n",
       "      <td>1</td>\n",
       "      <td>6</td>\n",
       "      <td>6</td>\n",
       "      <td>1.6958577188631916</td>\n",
       "      <td>0.8748596361444527</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>2671486</td>\n",
       "      <td>0.65</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>nonProfitIndicator</td>\n",
       "      <td>BIGINT</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0.0013876172287633176</td>\n",
       "      <td>0.0372248877216163</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2671486</td>\n",
       "      <td>0.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>obstructionType</td>\n",
       "      <td>BIGINT</td>\n",
       "      <td>10</td>\n",
       "      <td>98</td>\n",
       "      <td>20</td>\n",
       "      <td>16.553431553900655</td>\n",
       "      <td>16.107515135628212</td>\n",
       "      <td>10</td>\n",
       "      <td>10</td>\n",
       "      <td>10</td>\n",
       "      <td>2671486</td>\n",
       "      <td>43.74</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>occupancyType</td>\n",
       "      <td>BIGINT</td>\n",
       "      <td>1</td>\n",
       "      <td>19</td>\n",
       "      <td>14</td>\n",
       "      <td>1.9216470466664222</td>\n",
       "      <td>2.5068800508373754</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2671486</td>\n",
       "      <td>0.02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>originalConstructionDate</td>\n",
       "      <td>TIMESTAMP</td>\n",
       "      <td>1201-01-01 00:00:00</td>\n",
       "      <td>2024-09-16 00:00:00</td>\n",
       "      <td>31912</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>2671486</td>\n",
       "      <td>0.03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>originalNBDate</td>\n",
       "      <td>TIMESTAMP</td>\n",
       "      <td>0998-09-21 00:00:00</td>\n",
       "      <td>2024-11-09 00:00:00</td>\n",
       "      <td>17742</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>2671486</td>\n",
       "      <td>0.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>amountPaidOnBuildingClaim</td>\n",
       "      <td>DOUBLE</td>\n",
       "      <td>-162432.16</td>\n",
       "      <td>10006145.71</td>\n",
       "      <td>1286108</td>\n",
       "      <td>31531.109649180973</td>\n",
       "      <td>66190.17745513326</td>\n",
       "      <td>2317.1640557646992</td>\n",
       "      <td>9633.387051613568</td>\n",
       "      <td>36534.989849357065</td>\n",
       "      <td>2671486</td>\n",
       "      <td>22.78</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>amountPaidOnContentsClaim</td>\n",
       "      <td>DOUBLE</td>\n",
       "      <td>-41276.32</td>\n",
       "      <td>757048.95</td>\n",
       "      <td>472789</td>\n",
       "      <td>7096.517735698417</td>\n",
       "      <td>22183.209325532247</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.7398585225357502e-06</td>\n",
       "      <td>4796.289056966857</td>\n",
       "      <td>2671486</td>\n",
       "      <td>22.78</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>amountPaidOnIncreasedCostOfComplianceClaim</td>\n",
       "      <td>DOUBLE</td>\n",
       "      <td>-6450.0</td>\n",
       "      <td>60000.0</td>\n",
       "      <td>8752</td>\n",
       "      <td>459.7303803291179</td>\n",
       "      <td>3489.7836171167105</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2671486</td>\n",
       "      <td>22.78</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>postFIRMConstructionIndicator</td>\n",
       "      <td>BIGINT</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0.26588759963555864</td>\n",
       "      <td>0.441804772564026</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2671486</td>\n",
       "      <td>0.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>rateMethod</td>\n",
       "      <td>VARCHAR</td>\n",
       "      <td>1</td>\n",
       "      <td>W</td>\n",
       "      <td>22</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>2671486</td>\n",
       "      <td>1.85</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>smallBusinessIndicatorBuilding</td>\n",
       "      <td>BIGINT</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0.005793030545546561</td>\n",
       "      <td>0.07589119513192012</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2671486</td>\n",
       "      <td>0.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>totalBuildingInsuranceCoverage</td>\n",
       "      <td>BIGINT</td>\n",
       "      <td>0</td>\n",
       "      <td>243903000</td>\n",
       "      <td>12425</td>\n",
       "      <td>172980.48504127294</td>\n",
       "      <td>1253722.2614965935</td>\n",
       "      <td>39821</td>\n",
       "      <td>100316</td>\n",
       "      <td>222509</td>\n",
       "      <td>2671486</td>\n",
       "      <td>0.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>totalContentsInsuranceCoverage</td>\n",
       "      <td>BIGINT</td>\n",
       "      <td>0</td>\n",
       "      <td>6000000</td>\n",
       "      <td>3064</td>\n",
       "      <td>31127.78768283439</td>\n",
       "      <td>50873.49942270754</td>\n",
       "      <td>0</td>\n",
       "      <td>11665</td>\n",
       "      <td>46280</td>\n",
       "      <td>2671486</td>\n",
       "      <td>0.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>yearOfLoss</td>\n",
       "      <td>BIGINT</td>\n",
       "      <td>1978</td>\n",
       "      <td>2024</td>\n",
       "      <td>47</td>\n",
       "      <td>2002.6466337461623</td>\n",
       "      <td>12.783941740461886</td>\n",
       "      <td>1993</td>\n",
       "      <td>2005</td>\n",
       "      <td>2012</td>\n",
       "      <td>2671486</td>\n",
       "      <td>0.00</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                   column_name column_type  \\\n",
       "15          numberOfFloorsInTheInsuredBuilding      BIGINT   \n",
       "16                          nonProfitIndicator      BIGINT   \n",
       "17                             obstructionType      BIGINT   \n",
       "18                               occupancyType      BIGINT   \n",
       "19                    originalConstructionDate   TIMESTAMP   \n",
       "20                              originalNBDate   TIMESTAMP   \n",
       "21                   amountPaidOnBuildingClaim      DOUBLE   \n",
       "22                   amountPaidOnContentsClaim      DOUBLE   \n",
       "23  amountPaidOnIncreasedCostOfComplianceClaim      DOUBLE   \n",
       "24               postFIRMConstructionIndicator      BIGINT   \n",
       "25                                  rateMethod     VARCHAR   \n",
       "26              smallBusinessIndicatorBuilding      BIGINT   \n",
       "27              totalBuildingInsuranceCoverage      BIGINT   \n",
       "28              totalContentsInsuranceCoverage      BIGINT   \n",
       "29                                  yearOfLoss      BIGINT   \n",
       "\n",
       "                    min                  max  approx_unique  \\\n",
       "15                    1                    6              6   \n",
       "16                    0                    1              2   \n",
       "17                   10                   98             20   \n",
       "18                    1                   19             14   \n",
       "19  1201-01-01 00:00:00  2024-09-16 00:00:00          31912   \n",
       "20  0998-09-21 00:00:00  2024-11-09 00:00:00          17742   \n",
       "21           -162432.16          10006145.71        1286108   \n",
       "22            -41276.32            757048.95         472789   \n",
       "23              -6450.0              60000.0           8752   \n",
       "24                    0                    1              2   \n",
       "25                    1                    W             22   \n",
       "26                    0                    1              2   \n",
       "27                    0            243903000          12425   \n",
       "28                    0              6000000           3064   \n",
       "29                 1978                 2024             47   \n",
       "\n",
       "                      avg                  std                 q25  \\\n",
       "15     1.6958577188631916   0.8748596361444527                   1   \n",
       "16  0.0013876172287633176   0.0372248877216163                   0   \n",
       "17     16.553431553900655   16.107515135628212                  10   \n",
       "18     1.9216470466664222   2.5068800508373754                   1   \n",
       "19                   None                 None                None   \n",
       "20                   None                 None                None   \n",
       "21     31531.109649180973    66190.17745513326  2317.1640557646992   \n",
       "22      7096.517735698417   22183.209325532247                 0.0   \n",
       "23      459.7303803291179   3489.7836171167105                 0.0   \n",
       "24    0.26588759963555864    0.441804772564026                   0   \n",
       "25                   None                 None                None   \n",
       "26   0.005793030545546561  0.07589119513192012                   0   \n",
       "27     172980.48504127294   1253722.2614965935               39821   \n",
       "28      31127.78768283439    50873.49942270754                   0   \n",
       "29     2002.6466337461623   12.783941740461886                1993   \n",
       "\n",
       "                       q50                 q75    count  null_percentage  \n",
       "15                       1                   2  2671486             0.65  \n",
       "16                       0                   0  2671486             0.00  \n",
       "17                      10                  10  2671486            43.74  \n",
       "18                       1                   1  2671486             0.02  \n",
       "19                    None                None  2671486             0.03  \n",
       "20                    None                None  2671486             0.00  \n",
       "21       9633.387051613568  36534.989849357065  2671486            22.78  \n",
       "22  1.7398585225357502e-06   4796.289056966857  2671486            22.78  \n",
       "23                     0.0                 0.0  2671486            22.78  \n",
       "24                       0                   1  2671486             0.00  \n",
       "25                    None                None  2671486             1.85  \n",
       "26                       0                   0  2671486             0.00  \n",
       "27                  100316              222509  2671486             0.00  \n",
       "28                   11665               46280  2671486             0.00  \n",
       "29                    2005                2012  2671486             0.00  "
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# slice through columns for readability\n",
    "con.sql(\"SELECT * FROM summarize_relation\").df().iloc[15:30, :]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "df631c4b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>column_name</th>\n",
       "      <th>column_type</th>\n",
       "      <th>min</th>\n",
       "      <th>max</th>\n",
       "      <th>approx_unique</th>\n",
       "      <th>avg</th>\n",
       "      <th>std</th>\n",
       "      <th>q25</th>\n",
       "      <th>q50</th>\n",
       "      <th>q75</th>\n",
       "      <th>count</th>\n",
       "      <th>null_percentage</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>primaryResidenceIndicator</td>\n",
       "      <td>BIGINT</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0.4892947221134604</td>\n",
       "      <td>0.4998854774479687</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2671486</td>\n",
       "      <td>0.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>buildingDamageAmount</td>\n",
       "      <td>BIGINT</td>\n",
       "      <td>0</td>\n",
       "      <td>927700000</td>\n",
       "      <td>202772</td>\n",
       "      <td>36354.07504655704</td>\n",
       "      <td>812334.7780210077</td>\n",
       "      <td>3429</td>\n",
       "      <td>11228</td>\n",
       "      <td>40125</td>\n",
       "      <td>2671486</td>\n",
       "      <td>23.76</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>buildingDeductibleCode</td>\n",
       "      <td>VARCHAR</td>\n",
       "      <td>0</td>\n",
       "      <td>H</td>\n",
       "      <td>15</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>2671486</td>\n",
       "      <td>11.77</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>netBuildingPaymentAmount</td>\n",
       "      <td>DOUBLE</td>\n",
       "      <td>-162432.16</td>\n",
       "      <td>10006145.71</td>\n",
       "      <td>1286178</td>\n",
       "      <td>24319.109697074964</td>\n",
       "      <td>59591.4128871143</td>\n",
       "      <td>0.004801795055906804</td>\n",
       "      <td>4445.082100124046</td>\n",
       "      <td>23992.325530881346</td>\n",
       "      <td>2671486</td>\n",
       "      <td>0.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34</th>\n",
       "      <td>buildingPropertyValue</td>\n",
       "      <td>BIGINT</td>\n",
       "      <td>0</td>\n",
       "      <td>9950000000</td>\n",
       "      <td>420035</td>\n",
       "      <td>7148318.887278207</td>\n",
       "      <td>199917804.10704142</td>\n",
       "      <td>60768</td>\n",
       "      <td>109891</td>\n",
       "      <td>189858</td>\n",
       "      <td>2671486</td>\n",
       "      <td>23.76</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35</th>\n",
       "      <td>causeOfDamage</td>\n",
       "      <td>VARCHAR</td>\n",
       "      <td>0</td>\n",
       "      <td>Z</td>\n",
       "      <td>17</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>2671486</td>\n",
       "      <td>2.26</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>36</th>\n",
       "      <td>condominiumCoverageTypeCode</td>\n",
       "      <td>VARCHAR</td>\n",
       "      <td>A</td>\n",
       "      <td>U</td>\n",
       "      <td>5</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>2671486</td>\n",
       "      <td>1.40</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37</th>\n",
       "      <td>contentsDamageAmount</td>\n",
       "      <td>BIGINT</td>\n",
       "      <td>0</td>\n",
       "      <td>19230507</td>\n",
       "      <td>102299</td>\n",
       "      <td>18331.886239398373</td>\n",
       "      <td>86944.02029162779</td>\n",
       "      <td>1611</td>\n",
       "      <td>5682</td>\n",
       "      <td>18353</td>\n",
       "      <td>2671486</td>\n",
       "      <td>59.72</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>38</th>\n",
       "      <td>contentsDeductibleCode</td>\n",
       "      <td>VARCHAR</td>\n",
       "      <td>0</td>\n",
       "      <td>H</td>\n",
       "      <td>15</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>2671486</td>\n",
       "      <td>20.56</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>netContentsPaymentAmount</td>\n",
       "      <td>DOUBLE</td>\n",
       "      <td>-41276.32</td>\n",
       "      <td>757048.95</td>\n",
       "      <td>473195</td>\n",
       "      <td>5458.730475039014</td>\n",
       "      <td>19697.04273977996</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2115.4654194716413</td>\n",
       "      <td>2671486</td>\n",
       "      <td>0.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40</th>\n",
       "      <td>contentsPropertyValue</td>\n",
       "      <td>BIGINT</td>\n",
       "      <td>-1</td>\n",
       "      <td>281895323</td>\n",
       "      <td>59971</td>\n",
       "      <td>31580.59183337763</td>\n",
       "      <td>488372.3577602211</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>19589</td>\n",
       "      <td>2671486</td>\n",
       "      <td>59.72</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41</th>\n",
       "      <td>disasterAssistanceCoverageRequired</td>\n",
       "      <td>BIGINT</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>6</td>\n",
       "      <td>0.0558335502989801</td>\n",
       "      <td>0.47453698957687596</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2671486</td>\n",
       "      <td>13.45</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>42</th>\n",
       "      <td>eventDesignationNumber</td>\n",
       "      <td>VARCHAR</td>\n",
       "      <td>AL0221</td>\n",
       "      <td>WV0421</td>\n",
       "      <td>65</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>2671486</td>\n",
       "      <td>94.34</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>43</th>\n",
       "      <td>ficoNumber</td>\n",
       "      <td>BIGINT</td>\n",
       "      <td>1</td>\n",
       "      <td>998</td>\n",
       "      <td>733</td>\n",
       "      <td>500.707956505723</td>\n",
       "      <td>237.25352640186802</td>\n",
       "      <td>305</td>\n",
       "      <td>593</td>\n",
       "      <td>655</td>\n",
       "      <td>2671486</td>\n",
       "      <td>35.27</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>44</th>\n",
       "      <td>floodCharacteristicsIndicator</td>\n",
       "      <td>BIGINT</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>1.8946377938925552</td>\n",
       "      <td>0.595463294918818</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2671486</td>\n",
       "      <td>98.51</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                           column_name column_type         min          max  \\\n",
       "30           primaryResidenceIndicator      BIGINT           0            1   \n",
       "31                buildingDamageAmount      BIGINT           0    927700000   \n",
       "32              buildingDeductibleCode     VARCHAR           0            H   \n",
       "33            netBuildingPaymentAmount      DOUBLE  -162432.16  10006145.71   \n",
       "34               buildingPropertyValue      BIGINT           0   9950000000   \n",
       "35                       causeOfDamage     VARCHAR           0            Z   \n",
       "36         condominiumCoverageTypeCode     VARCHAR           A            U   \n",
       "37                contentsDamageAmount      BIGINT           0     19230507   \n",
       "38              contentsDeductibleCode     VARCHAR           0            H   \n",
       "39            netContentsPaymentAmount      DOUBLE   -41276.32    757048.95   \n",
       "40               contentsPropertyValue      BIGINT          -1    281895323   \n",
       "41  disasterAssistanceCoverageRequired      BIGINT           0            5   \n",
       "42              eventDesignationNumber     VARCHAR      AL0221       WV0421   \n",
       "43                          ficoNumber      BIGINT           1          998   \n",
       "44       floodCharacteristicsIndicator      BIGINT           1            5   \n",
       "\n",
       "    approx_unique                 avg                  std  \\\n",
       "30              2  0.4892947221134604   0.4998854774479687   \n",
       "31         202772   36354.07504655704    812334.7780210077   \n",
       "32             15                None                 None   \n",
       "33        1286178  24319.109697074964     59591.4128871143   \n",
       "34         420035   7148318.887278207   199917804.10704142   \n",
       "35             17                None                 None   \n",
       "36              5                None                 None   \n",
       "37         102299  18331.886239398373    86944.02029162779   \n",
       "38             15                None                 None   \n",
       "39         473195   5458.730475039014    19697.04273977996   \n",
       "40          59971   31580.59183337763    488372.3577602211   \n",
       "41              6  0.0558335502989801  0.47453698957687596   \n",
       "42             65                None                 None   \n",
       "43            733    500.707956505723   237.25352640186802   \n",
       "44              5  1.8946377938925552    0.595463294918818   \n",
       "\n",
       "                     q25                q50                 q75    count  \\\n",
       "30                     0                  0                   1  2671486   \n",
       "31                  3429              11228               40125  2671486   \n",
       "32                  None               None                None  2671486   \n",
       "33  0.004801795055906804  4445.082100124046  23992.325530881346  2671486   \n",
       "34                 60768             109891              189858  2671486   \n",
       "35                  None               None                None  2671486   \n",
       "36                  None               None                None  2671486   \n",
       "37                  1611               5682               18353  2671486   \n",
       "38                  None               None                None  2671486   \n",
       "39                   0.0                0.0  2115.4654194716413  2671486   \n",
       "40                     0                  0               19589  2671486   \n",
       "41                     0                  0                   0  2671486   \n",
       "42                  None               None                None  2671486   \n",
       "43                   305                593                 655  2671486   \n",
       "44                     2                  2                   2  2671486   \n",
       "\n",
       "    null_percentage  \n",
       "30             0.00  \n",
       "31            23.76  \n",
       "32            11.77  \n",
       "33             0.00  \n",
       "34            23.76  \n",
       "35             2.26  \n",
       "36             1.40  \n",
       "37            59.72  \n",
       "38            20.56  \n",
       "39             0.00  \n",
       "40            59.72  \n",
       "41            13.45  \n",
       "42            94.34  \n",
       "43            35.27  \n",
       "44            98.51  "
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# slice through columns for readability\n",
    "con.sql(\"SELECT * FROM summarize_relation\").df().iloc[30:45, :]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "bce0dd25",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>column_name</th>\n",
       "      <th>column_type</th>\n",
       "      <th>min</th>\n",
       "      <th>max</th>\n",
       "      <th>approx_unique</th>\n",
       "      <th>avg</th>\n",
       "      <th>std</th>\n",
       "      <th>q25</th>\n",
       "      <th>q50</th>\n",
       "      <th>q75</th>\n",
       "      <th>count</th>\n",
       "      <th>null_percentage</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>45</th>\n",
       "      <td>floodWaterDuration</td>\n",
       "      <td>BIGINT</td>\n",
       "      <td>0</td>\n",
       "      <td>999</td>\n",
       "      <td>261</td>\n",
       "      <td>0.8068743267334517</td>\n",
       "      <td>14.72863591340925</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2671486</td>\n",
       "      <td>9.06</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>46</th>\n",
       "      <td>floodproofedIndicator</td>\n",
       "      <td>BIGINT</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0.0001411199609505721</td>\n",
       "      <td>0.011878556264305687</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2671486</td>\n",
       "      <td>0.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>47</th>\n",
       "      <td>floodEvent</td>\n",
       "      <td>VARCHAR</td>\n",
       "      <td>2021 Mid-Spring Severe Storms</td>\n",
       "      <td>Yellowstone Flooding</td>\n",
       "      <td>177</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>2671486</td>\n",
       "      <td>28.48</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>48</th>\n",
       "      <td>iccCoverage</td>\n",
       "      <td>BIGINT</td>\n",
       "      <td>0</td>\n",
       "      <td>30000</td>\n",
       "      <td>4</td>\n",
       "      <td>27991.868422816147</td>\n",
       "      <td>4754.676432391682</td>\n",
       "      <td>30000</td>\n",
       "      <td>30000</td>\n",
       "      <td>30000</td>\n",
       "      <td>2671486</td>\n",
       "      <td>36.32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49</th>\n",
       "      <td>netIccPaymentAmount</td>\n",
       "      <td>DOUBLE</td>\n",
       "      <td>-6450.0</td>\n",
       "      <td>60000.0</td>\n",
       "      <td>8709</td>\n",
       "      <td>353.6949859478955</td>\n",
       "      <td>3066.941960282361</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2671486</td>\n",
       "      <td>0.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50</th>\n",
       "      <td>nfipRatedCommunityNumber</td>\n",
       "      <td>VARCHAR</td>\n",
       "      <td>000000</td>\n",
       "      <td>999999</td>\n",
       "      <td>16299</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>2671486</td>\n",
       "      <td>0.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>51</th>\n",
       "      <td>nfipCommunityNumberCurrent</td>\n",
       "      <td>VARCHAR</td>\n",
       "      <td>0000</td>\n",
       "      <td>815000</td>\n",
       "      <td>12065</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>2671486</td>\n",
       "      <td>72.10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>52</th>\n",
       "      <td>nfipCommunityName</td>\n",
       "      <td>VARCHAR</td>\n",
       "      <td>ABBEVILLE COUNTY *</td>\n",
       "      <td>ZUMBRO FALLS, CITY OF</td>\n",
       "      <td>9870</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>2671486</td>\n",
       "      <td>70.76</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>53</th>\n",
       "      <td>nonPaymentReasonContents</td>\n",
       "      <td>VARCHAR</td>\n",
       "      <td>01</td>\n",
       "      <td>99</td>\n",
       "      <td>23</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>2671486</td>\n",
       "      <td>68.95</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>54</th>\n",
       "      <td>nonPaymentReasonBuilding</td>\n",
       "      <td>VARCHAR</td>\n",
       "      <td>01</td>\n",
       "      <td>99</td>\n",
       "      <td>23</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>2671486</td>\n",
       "      <td>78.02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>55</th>\n",
       "      <td>numberOfUnits</td>\n",
       "      <td>BIGINT</td>\n",
       "      <td>0</td>\n",
       "      <td>99999</td>\n",
       "      <td>450</td>\n",
       "      <td>1.3967815247755893</td>\n",
       "      <td>65.44451776434711</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2671486</td>\n",
       "      <td>0.11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>56</th>\n",
       "      <td>buildingReplacementCost</td>\n",
       "      <td>BIGINT</td>\n",
       "      <td>0</td>\n",
       "      <td>9999000000</td>\n",
       "      <td>447053</td>\n",
       "      <td>8569836.741231738</td>\n",
       "      <td>235005475.35407257</td>\n",
       "      <td>0</td>\n",
       "      <td>121040</td>\n",
       "      <td>224171</td>\n",
       "      <td>2671486</td>\n",
       "      <td>23.76</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>57</th>\n",
       "      <td>contentsReplacementCost</td>\n",
       "      <td>BIGINT</td>\n",
       "      <td>0</td>\n",
       "      <td>20000000</td>\n",
       "      <td>8887</td>\n",
       "      <td>2809.0997141881894</td>\n",
       "      <td>46529.60714476816</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2671486</td>\n",
       "      <td>59.75</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>58</th>\n",
       "      <td>replacementCostBasis</td>\n",
       "      <td>VARCHAR</td>\n",
       "      <td>A</td>\n",
       "      <td>R</td>\n",
       "      <td>2</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>2671486</td>\n",
       "      <td>8.56</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59</th>\n",
       "      <td>stateOwnedIndicator</td>\n",
       "      <td>BIGINT</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0.000759502389306925</td>\n",
       "      <td>0.027548608485935006</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2671486</td>\n",
       "      <td>0.00</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   column_name column_type                            min  \\\n",
       "45          floodWaterDuration      BIGINT                              0   \n",
       "46       floodproofedIndicator      BIGINT                              0   \n",
       "47                  floodEvent     VARCHAR  2021 Mid-Spring Severe Storms   \n",
       "48                 iccCoverage      BIGINT                              0   \n",
       "49         netIccPaymentAmount      DOUBLE                        -6450.0   \n",
       "50    nfipRatedCommunityNumber     VARCHAR                         000000   \n",
       "51  nfipCommunityNumberCurrent     VARCHAR                           0000   \n",
       "52           nfipCommunityName     VARCHAR             ABBEVILLE COUNTY *   \n",
       "53    nonPaymentReasonContents     VARCHAR                             01   \n",
       "54    nonPaymentReasonBuilding     VARCHAR                             01   \n",
       "55               numberOfUnits      BIGINT                              0   \n",
       "56     buildingReplacementCost      BIGINT                              0   \n",
       "57     contentsReplacementCost      BIGINT                              0   \n",
       "58        replacementCostBasis     VARCHAR                              A   \n",
       "59         stateOwnedIndicator      BIGINT                              0   \n",
       "\n",
       "                      max  approx_unique                    avg  \\\n",
       "45                    999            261     0.8068743267334517   \n",
       "46                      1              2  0.0001411199609505721   \n",
       "47   Yellowstone Flooding            177                   None   \n",
       "48                  30000              4     27991.868422816147   \n",
       "49                60000.0           8709      353.6949859478955   \n",
       "50                 999999          16299                   None   \n",
       "51                 815000          12065                   None   \n",
       "52  ZUMBRO FALLS, CITY OF           9870                   None   \n",
       "53                     99             23                   None   \n",
       "54                     99             23                   None   \n",
       "55                  99999            450     1.3967815247755893   \n",
       "56             9999000000         447053      8569836.741231738   \n",
       "57               20000000           8887     2809.0997141881894   \n",
       "58                      R              2                   None   \n",
       "59                      1              2   0.000759502389306925   \n",
       "\n",
       "                     std    q25     q50     q75    count  null_percentage  \n",
       "45     14.72863591340925      0       0       0  2671486             9.06  \n",
       "46  0.011878556264305687      0       0       0  2671486             0.00  \n",
       "47                  None   None    None    None  2671486            28.48  \n",
       "48     4754.676432391682  30000   30000   30000  2671486            36.32  \n",
       "49     3066.941960282361    0.0     0.0     0.0  2671486             0.00  \n",
       "50                  None   None    None    None  2671486             0.00  \n",
       "51                  None   None    None    None  2671486            72.10  \n",
       "52                  None   None    None    None  2671486            70.76  \n",
       "53                  None   None    None    None  2671486            68.95  \n",
       "54                  None   None    None    None  2671486            78.02  \n",
       "55     65.44451776434711      1       1       1  2671486             0.11  \n",
       "56    235005475.35407257      0  121040  224171  2671486            23.76  \n",
       "57     46529.60714476816      0       0       0  2671486            59.75  \n",
       "58                  None   None    None    None  2671486             8.56  \n",
       "59  0.027548608485935006      0       0       0  2671486             0.00  "
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# slice through columns for readability\n",
    "con.sql(\"SELECT * FROM summarize_relation\").df().iloc[45:60, :]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "581fb652",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>column_name</th>\n",
       "      <th>column_type</th>\n",
       "      <th>min</th>\n",
       "      <th>max</th>\n",
       "      <th>approx_unique</th>\n",
       "      <th>avg</th>\n",
       "      <th>std</th>\n",
       "      <th>q25</th>\n",
       "      <th>q50</th>\n",
       "      <th>q75</th>\n",
       "      <th>count</th>\n",
       "      <th>null_percentage</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>60</th>\n",
       "      <td>waterDepth</td>\n",
       "      <td>BIGINT</td>\n",
       "      <td>-999</td>\n",
       "      <td>999</td>\n",
       "      <td>478</td>\n",
       "      <td>4.289864898924166</td>\n",
       "      <td>16.454985259103754</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>2671486</td>\n",
       "      <td>9.40</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>61</th>\n",
       "      <td>floodZoneCurrent</td>\n",
       "      <td>VARCHAR</td>\n",
       "      <td>A</td>\n",
       "      <td>X</td>\n",
       "      <td>62</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>2671486</td>\n",
       "      <td>72.15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>62</th>\n",
       "      <td>buildingDescriptionCode</td>\n",
       "      <td>BIGINT</td>\n",
       "      <td>1</td>\n",
       "      <td>21</td>\n",
       "      <td>18</td>\n",
       "      <td>1.3360927447287696</td>\n",
       "      <td>1.4946867436844309</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2671486</td>\n",
       "      <td>64.28</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>63</th>\n",
       "      <td>rentalPropertyIndicator</td>\n",
       "      <td>BIGINT</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0.012453742973012024</td>\n",
       "      <td>0.11089928702497898</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2671486</td>\n",
       "      <td>0.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>64</th>\n",
       "      <td>state</td>\n",
       "      <td>VARCHAR</td>\n",
       "      <td>AK</td>\n",
       "      <td>WY</td>\n",
       "      <td>57</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>2671486</td>\n",
       "      <td>0.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>65</th>\n",
       "      <td>reportedCity</td>\n",
       "      <td>VARCHAR</td>\n",
       "      <td>Currently Unavailable</td>\n",
       "      <td>Currently Unavailable</td>\n",
       "      <td>1</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>2671486</td>\n",
       "      <td>0.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>66</th>\n",
       "      <td>reportedZipCode</td>\n",
       "      <td>VARCHAR</td>\n",
       "      <td>00000</td>\n",
       "      <td>99999</td>\n",
       "      <td>26326</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>2671486</td>\n",
       "      <td>2.09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>67</th>\n",
       "      <td>countyCode</td>\n",
       "      <td>VARCHAR</td>\n",
       "      <td>01001</td>\n",
       "      <td>78030</td>\n",
       "      <td>2946</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>2671486</td>\n",
       "      <td>2.32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>68</th>\n",
       "      <td>censusTract</td>\n",
       "      <td>VARCHAR</td>\n",
       "      <td>01001020100</td>\n",
       "      <td>78030961200</td>\n",
       "      <td>59577</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>2671486</td>\n",
       "      <td>5.16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>69</th>\n",
       "      <td>censusBlockGroupFips</td>\n",
       "      <td>VARCHAR</td>\n",
       "      <td>010010201001</td>\n",
       "      <td>780309612002</td>\n",
       "      <td>117722</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>2671486</td>\n",
       "      <td>5.16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>70</th>\n",
       "      <td>latitude</td>\n",
       "      <td>DOUBLE</td>\n",
       "      <td>-36.0</td>\n",
       "      <td>69.9</td>\n",
       "      <td>334</td>\n",
       "      <td>33.75590024437987</td>\n",
       "      <td>5.827326735346321</td>\n",
       "      <td>29.81595144985515</td>\n",
       "      <td>30.69183610183413</td>\n",
       "      <td>39.66073414548429</td>\n",
       "      <td>2671486</td>\n",
       "      <td>1.51</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>71</th>\n",
       "      <td>longitude</td>\n",
       "      <td>DOUBLE</td>\n",
       "      <td>-170.7</td>\n",
       "      <td>174.0</td>\n",
       "      <td>715</td>\n",
       "      <td>-86.02134344014443</td>\n",
       "      <td>10.94026893158453</td>\n",
       "      <td>-91.12060166756976</td>\n",
       "      <td>-86.44710335298433</td>\n",
       "      <td>-77.48251995576663</td>\n",
       "      <td>2671486</td>\n",
       "      <td>1.51</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>72</th>\n",
       "      <td>id</td>\n",
       "      <td>VARCHAR</td>\n",
       "      <td>0000094b-1d1a-4dbe-a944-97c62b225b9c</td>\n",
       "      <td>fffffe7c-7ca2-4431-b88b-8a8216ffc934</td>\n",
       "      <td>2623176</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>2671486</td>\n",
       "      <td>0.00</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                column_name column_type                                   min  \\\n",
       "60               waterDepth      BIGINT                                  -999   \n",
       "61         floodZoneCurrent     VARCHAR                                     A   \n",
       "62  buildingDescriptionCode      BIGINT                                     1   \n",
       "63  rentalPropertyIndicator      BIGINT                                     0   \n",
       "64                    state     VARCHAR                                    AK   \n",
       "65             reportedCity     VARCHAR                 Currently Unavailable   \n",
       "66          reportedZipCode     VARCHAR                                 00000   \n",
       "67               countyCode     VARCHAR                                 01001   \n",
       "68              censusTract     VARCHAR                           01001020100   \n",
       "69     censusBlockGroupFips     VARCHAR                          010010201001   \n",
       "70                 latitude      DOUBLE                                 -36.0   \n",
       "71                longitude      DOUBLE                                -170.7   \n",
       "72                       id     VARCHAR  0000094b-1d1a-4dbe-a944-97c62b225b9c   \n",
       "\n",
       "                                     max  approx_unique                   avg  \\\n",
       "60                                   999            478     4.289864898924166   \n",
       "61                                     X             62                  None   \n",
       "62                                    21             18    1.3360927447287696   \n",
       "63                                     1              2  0.012453742973012024   \n",
       "64                                    WY             57                  None   \n",
       "65                 Currently Unavailable              1                  None   \n",
       "66                                 99999          26326                  None   \n",
       "67                                 78030           2946                  None   \n",
       "68                           78030961200          59577                  None   \n",
       "69                          780309612002         117722                  None   \n",
       "70                                  69.9            334     33.75590024437987   \n",
       "71                                 174.0            715    -86.02134344014443   \n",
       "72  fffffe7c-7ca2-4431-b88b-8a8216ffc934        2623176                  None   \n",
       "\n",
       "                    std                 q25                 q50  \\\n",
       "60   16.454985259103754                   0                   1   \n",
       "61                 None                None                None   \n",
       "62   1.4946867436844309                   1                   1   \n",
       "63  0.11089928702497898                   0                   0   \n",
       "64                 None                None                None   \n",
       "65                 None                None                None   \n",
       "66                 None                None                None   \n",
       "67                 None                None                None   \n",
       "68                 None                None                None   \n",
       "69                 None                None                None   \n",
       "70    5.827326735346321   29.81595144985515   30.69183610183413   \n",
       "71    10.94026893158453  -91.12060166756976  -86.44710335298433   \n",
       "72                 None                None                None   \n",
       "\n",
       "                   q75    count  null_percentage  \n",
       "60                   2  2671486             9.40  \n",
       "61                None  2671486            72.15  \n",
       "62                   1  2671486            64.28  \n",
       "63                   0  2671486             0.00  \n",
       "64                None  2671486             0.00  \n",
       "65                None  2671486             0.00  \n",
       "66                None  2671486             2.09  \n",
       "67                None  2671486             2.32  \n",
       "68                None  2671486             5.16  \n",
       "69                None  2671486             5.16  \n",
       "70   39.66073414548429  2671486             1.51  \n",
       "71  -77.48251995576663  2671486             1.51  \n",
       "72                None  2671486             0.00  "
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# slice through columns for readability\n",
    "con.sql(\"SELECT * FROM summarize_relation\").df().iloc[60:, :]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c6416efa",
   "metadata": {},
   "source": [
    "## Highlighted Features: Amount Paid on Claims Summary Statistics\n",
    "We will analyze these features extensively throughout this notebook.\n",
    "\n",
    "**amountPaidOnBuildingClaim**: Dollar amount paid on the building claim. In some instances, a negative amount may appear which occurs when a check issued to a policy holder is not cashed and has to be re-issued.\n",
    "\n",
    "**amountPaidOnContentsClaim**: Dollar amount paid on the contents claim. In some instances, a negative amount may appear, which occurs when a check issued to a policy holder is not cashed and has to be re-issued.\n",
    "\n",
    "**amountPaidOnIncreasedCostOfComplianceClaim**: ICC coverage is one of several flood insurance resources for policyholders who need additional help rebuilding after a flood. It provides up to $30,000 to help cover the cost of mitigation measures that will reduce the flood risk.\n",
    "\n",
    "Source: [Data Dictionary](https://www.fema.gov/openfema-data-page/fima-nfip-redacted-claims-v2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "cf6823b0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>column_name</th>\n",
       "      <th>column_type</th>\n",
       "      <th>count</th>\n",
       "      <th>null_percentage</th>\n",
       "      <th>min</th>\n",
       "      <th>max</th>\n",
       "      <th>approx_unique</th>\n",
       "      <th>avg</th>\n",
       "      <th>std</th>\n",
       "      <th>q25</th>\n",
       "      <th>q50</th>\n",
       "      <th>q75</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>amountPaidOnBuildingClaim</td>\n",
       "      <td>DOUBLE</td>\n",
       "      <td>2671486</td>\n",
       "      <td>22.78</td>\n",
       "      <td>-162432.16</td>\n",
       "      <td>10006145.71</td>\n",
       "      <td>1286108</td>\n",
       "      <td>31531.11</td>\n",
       "      <td>66190.18</td>\n",
       "      <td>2320.28</td>\n",
       "      <td>9618.31</td>\n",
       "      <td>36673.21</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>amountPaidOnContentsClaim</td>\n",
       "      <td>DOUBLE</td>\n",
       "      <td>2671486</td>\n",
       "      <td>22.78</td>\n",
       "      <td>-41276.32</td>\n",
       "      <td>757048.95</td>\n",
       "      <td>472789</td>\n",
       "      <td>7096.52</td>\n",
       "      <td>22183.21</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>4771.77</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>amountPaidOnIncreasedCostOfComplianceClaim</td>\n",
       "      <td>DOUBLE</td>\n",
       "      <td>2671486</td>\n",
       "      <td>22.78</td>\n",
       "      <td>-6450.0</td>\n",
       "      <td>60000.0</td>\n",
       "      <td>8752</td>\n",
       "      <td>459.73</td>\n",
       "      <td>3489.78</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                  column_name column_type    count  \\\n",
       "0                   amountPaidOnBuildingClaim      DOUBLE  2671486   \n",
       "1                   amountPaidOnContentsClaim      DOUBLE  2671486   \n",
       "2  amountPaidOnIncreasedCostOfComplianceClaim      DOUBLE  2671486   \n",
       "\n",
       "   null_percentage         min          max  approx_unique       avg  \\\n",
       "0            22.78  -162432.16  10006145.71        1286108  31531.11   \n",
       "1            22.78   -41276.32    757048.95         472789   7096.52   \n",
       "2            22.78     -6450.0      60000.0           8752    459.73   \n",
       "\n",
       "        std      q25      q50       q75  \n",
       "0  66190.18  2320.28  9618.31  36673.21  \n",
       "1  22183.21     0.00     0.00   4771.77  \n",
       "2   3489.78     0.00     0.00      0.00  "
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# examine summary statistics on paid total claims\n",
    "con.sql(\"\"\"\n",
    "    SELECT\n",
    "        column_name, column_type, count, null_percentage,\n",
    "        min, max, approx_unique,\n",
    "        ROUND(avg::DOUBLE, 2) AS avg,\n",
    "        ROUND(std::DOUBLE, 2) AS std,\n",
    "        ROUND(q25::DOUBLE, 2) AS q25,\n",
    "        ROUND(q50::DOUBLE, 2) AS q50,\n",
    "        ROUND(q75::DOUBLE, 2) AS q75\n",
    "    FROM\n",
    "        summarize_relation\n",
    "    WHERE column_name IN (\n",
    "        'amountPaidOnBuildingClaim',\n",
    "        'amountPaidOnContentsClaim',\n",
    "        'amountPaidOnIncreasedCostOfComplianceClaim'\n",
    "    )\n",
    "\"\"\").df()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "b2e985cc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>dateOfLoss</th>\n",
       "      <th>floodEvent</th>\n",
       "      <th>state</th>\n",
       "      <th>policyCount</th>\n",
       "      <th>paidTotalClaim</th>\n",
       "      <th>amountPaidOnBuildingClaim</th>\n",
       "      <th>amountPaidOnContentsClaim</th>\n",
       "      <th>amountPaidICC</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2022-09-28</td>\n",
       "      <td>Hurricane Ian</td>\n",
       "      <td>FL</td>\n",
       "      <td>198</td>\n",
       "      <td>10106146</td>\n",
       "      <td>10006146</td>\n",
       "      <td>100000</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2005-08-29</td>\n",
       "      <td>Hurricane Katrina</td>\n",
       "      <td>MS</td>\n",
       "      <td>48</td>\n",
       "      <td>10000000</td>\n",
       "      <td>10000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2012-10-29</td>\n",
       "      <td>Hurricane Sandy</td>\n",
       "      <td>NY</td>\n",
       "      <td>184</td>\n",
       "      <td>9467720</td>\n",
       "      <td>9467720</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2004-09-15</td>\n",
       "      <td>Hurricane Ivan</td>\n",
       "      <td>FL</td>\n",
       "      <td>64</td>\n",
       "      <td>9169507</td>\n",
       "      <td>9100033</td>\n",
       "      <td>39474</td>\n",
       "      <td>30000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2001-06-09</td>\n",
       "      <td>Tropical Storm Allison</td>\n",
       "      <td>TX</td>\n",
       "      <td>233</td>\n",
       "      <td>9023558</td>\n",
       "      <td>8973270</td>\n",
       "      <td>50288</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2022-09-28</td>\n",
       "      <td>Hurricane Ian</td>\n",
       "      <td>FL</td>\n",
       "      <td>36</td>\n",
       "      <td>9000000</td>\n",
       "      <td>9000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>2022-09-28</td>\n",
       "      <td>Hurricane Ian</td>\n",
       "      <td>FL</td>\n",
       "      <td>54</td>\n",
       "      <td>8734196</td>\n",
       "      <td>8734196</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>2005-08-29</td>\n",
       "      <td>Hurricane Katrina</td>\n",
       "      <td>MS</td>\n",
       "      <td>71</td>\n",
       "      <td>8386100</td>\n",
       "      <td>8349000</td>\n",
       "      <td>37100</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>2022-09-28</td>\n",
       "      <td>Hurricane Ian</td>\n",
       "      <td>FL</td>\n",
       "      <td>51</td>\n",
       "      <td>7778937</td>\n",
       "      <td>7778937</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>2012-10-29</td>\n",
       "      <td>Hurricane Sandy</td>\n",
       "      <td>NY</td>\n",
       "      <td>76</td>\n",
       "      <td>7447962</td>\n",
       "      <td>7447962</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  dateOfLoss              floodEvent state  policyCount  paidTotalClaim  \\\n",
       "0 2022-09-28           Hurricane Ian    FL          198        10106146   \n",
       "1 2005-08-29       Hurricane Katrina    MS           48        10000000   \n",
       "2 2012-10-29         Hurricane Sandy    NY          184         9467720   \n",
       "3 2004-09-15          Hurricane Ivan    FL           64         9169507   \n",
       "4 2001-06-09  Tropical Storm Allison    TX          233         9023558   \n",
       "5 2022-09-28           Hurricane Ian    FL           36         9000000   \n",
       "6 2022-09-28           Hurricane Ian    FL           54         8734196   \n",
       "7 2005-08-29       Hurricane Katrina    MS           71         8386100   \n",
       "8 2022-09-28           Hurricane Ian    FL           51         7778937   \n",
       "9 2012-10-29         Hurricane Sandy    NY           76         7447962   \n",
       "\n",
       "   amountPaidOnBuildingClaim  amountPaidOnContentsClaim  amountPaidICC  \n",
       "0                   10006146                     100000              0  \n",
       "1                   10000000                          0              0  \n",
       "2                    9467720                          0              0  \n",
       "3                    9100033                      39474          30000  \n",
       "4                    8973270                      50288              0  \n",
       "5                    9000000                          0              0  \n",
       "6                    8734196                          0              0  \n",
       "7                    8349000                      37100              0  \n",
       "8                    7778937                          0              0  \n",
       "9                    7447962                          0              0  "
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# top 10 paid total claim records\n",
    "con.sql(\"\"\"\n",
    "    SELECT\n",
    "        dateOfLoss,\n",
    "        floodEvent,\n",
    "        state,\n",
    "        policyCount,\n",
    "        ROUND(\n",
    "            amountPaidOnBuildingClaim\n",
    "            + amountPaidOnContentsClaim\n",
    "            + amountPaidOnIncreasedCostOfComplianceClaim, 0)::INT AS paidTotalClaim,\n",
    "        ROUND(amountPaidOnBuildingClaim, 0)::INT AS amountPaidOnBuildingClaim,\n",
    "        ROUND(amountPaidOnContentsClaim, 0)::INT AS amountPaidOnContentsClaim,\n",
    "        ROUND(amountPaidOnIncreasedCostOfComplianceClaim, 0)::INT AS amountPaidICC\n",
    "    FROM\n",
    "        claims\n",
    "    ORDER BY\n",
    "        paidTotalClaim DESC\n",
    "    LIMIT 10\n",
    "\"\"\").df()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "d55bb0c6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>dateOfLoss</th>\n",
       "      <th>floodEvent</th>\n",
       "      <th>state</th>\n",
       "      <th>policyCount</th>\n",
       "      <th>paidTotalClaim</th>\n",
       "      <th>amountPaidOnBuildingClaim</th>\n",
       "      <th>amountPaidOnContentsClaim</th>\n",
       "      <th>amountPaidICC</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2008-06-13</td>\n",
       "      <td>None</td>\n",
       "      <td>IA</td>\n",
       "      <td>1</td>\n",
       "      <td>1250000</td>\n",
       "      <td>500000</td>\n",
       "      <td>750000</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2023-04-13</td>\n",
       "      <td>April Florida Flooding</td>\n",
       "      <td>FL</td>\n",
       "      <td>1</td>\n",
       "      <td>1065210</td>\n",
       "      <td>1000000</td>\n",
       "      <td>65210</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2016-05-24</td>\n",
       "      <td>None</td>\n",
       "      <td>AR</td>\n",
       "      <td>1</td>\n",
       "      <td>1054205</td>\n",
       "      <td>500000</td>\n",
       "      <td>554205</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2012-10-29</td>\n",
       "      <td>Hurricane Sandy</td>\n",
       "      <td>NJ</td>\n",
       "      <td>1</td>\n",
       "      <td>1050000</td>\n",
       "      <td>500000</td>\n",
       "      <td>550000</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2012-10-29</td>\n",
       "      <td>Hurricane Sandy</td>\n",
       "      <td>NJ</td>\n",
       "      <td>1</td>\n",
       "      <td>1039512</td>\n",
       "      <td>500000</td>\n",
       "      <td>539512</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>1998-09-27</td>\n",
       "      <td>Hurricane Georges</td>\n",
       "      <td>AL</td>\n",
       "      <td>1</td>\n",
       "      <td>1000000</td>\n",
       "      <td>500000</td>\n",
       "      <td>500000</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>2004-09-16</td>\n",
       "      <td>Hurricane Ivan</td>\n",
       "      <td>AL</td>\n",
       "      <td>1</td>\n",
       "      <td>1000000</td>\n",
       "      <td>500000</td>\n",
       "      <td>500000</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>2004-09-16</td>\n",
       "      <td>Hurricane Ivan</td>\n",
       "      <td>AL</td>\n",
       "      <td>1</td>\n",
       "      <td>1000000</td>\n",
       "      <td>500000</td>\n",
       "      <td>500000</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>2003-09-16</td>\n",
       "      <td>None</td>\n",
       "      <td>DE</td>\n",
       "      <td>1</td>\n",
       "      <td>1000000</td>\n",
       "      <td>500000</td>\n",
       "      <td>500000</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>2003-09-15</td>\n",
       "      <td>None</td>\n",
       "      <td>DE</td>\n",
       "      <td>1</td>\n",
       "      <td>1000000</td>\n",
       "      <td>500000</td>\n",
       "      <td>500000</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  dateOfLoss              floodEvent state  policyCount  paidTotalClaim  \\\n",
       "0 2008-06-13                    None    IA            1         1250000   \n",
       "1 2023-04-13  April Florida Flooding    FL            1         1065210   \n",
       "2 2016-05-24                    None    AR            1         1054205   \n",
       "3 2012-10-29         Hurricane Sandy    NJ            1         1050000   \n",
       "4 2012-10-29         Hurricane Sandy    NJ            1         1039512   \n",
       "5 1998-09-27       Hurricane Georges    AL            1         1000000   \n",
       "6 2004-09-16          Hurricane Ivan    AL            1         1000000   \n",
       "7 2004-09-16          Hurricane Ivan    AL            1         1000000   \n",
       "8 2003-09-16                    None    DE            1         1000000   \n",
       "9 2003-09-15                    None    DE            1         1000000   \n",
       "\n",
       "   amountPaidOnBuildingClaim  amountPaidOnContentsClaim  amountPaidICC  \n",
       "0                     500000                     750000              0  \n",
       "1                    1000000                      65210              0  \n",
       "2                     500000                     554205              0  \n",
       "3                     500000                     550000              0  \n",
       "4                     500000                     539512              0  \n",
       "5                     500000                     500000              0  \n",
       "6                     500000                     500000              0  \n",
       "7                     500000                     500000              0  \n",
       "8                     500000                     500000              0  \n",
       "9                     500000                     500000              0  "
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# top 10 paid total claim records for only one policy\n",
    "con.sql(\"\"\"\n",
    "    SELECT\n",
    "        dateOfLoss,\n",
    "        floodEvent,\n",
    "        state,\n",
    "        policyCount,\n",
    "        ROUND(\n",
    "            amountPaidOnBuildingClaim\n",
    "            + amountPaidOnContentsClaim\n",
    "            + amountPaidOnIncreasedCostOfComplianceClaim, 0)::INT AS paidTotalClaim,\n",
    "        ROUND(amountPaidOnBuildingClaim, 0)::INT AS amountPaidOnBuildingClaim,\n",
    "        ROUND(amountPaidOnContentsClaim, 0)::INT AS amountPaidOnContentsClaim,\n",
    "        ROUND(amountPaidOnIncreasedCostOfComplianceClaim, 0)::INT AS amountPaidICC\n",
    "    FROM\n",
    "        claims\n",
    "    WHERE\n",
    "        policyCount == 1\n",
    "    ORDER BY\n",
    "        paidTotalClaim DESC\n",
    "    LIMIT 10\n",
    "\"\"\").df()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "93abfab7",
   "metadata": {},
   "source": [
    "## Highlighted Features: Building Damage, Values, and Replacement Costs Summary Statistics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "b9f071ab",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>column_name</th>\n",
       "      <th>column_type</th>\n",
       "      <th>count</th>\n",
       "      <th>null_percentage</th>\n",
       "      <th>min</th>\n",
       "      <th>max</th>\n",
       "      <th>approx_unique</th>\n",
       "      <th>avg</th>\n",
       "      <th>std</th>\n",
       "      <th>q25</th>\n",
       "      <th>q50</th>\n",
       "      <th>q75</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>buildingPropertyValue</td>\n",
       "      <td>BIGINT</td>\n",
       "      <td>2671486</td>\n",
       "      <td>23.76</td>\n",
       "      <td>0</td>\n",
       "      <td>9950000000</td>\n",
       "      <td>420035</td>\n",
       "      <td>7148318.89</td>\n",
       "      <td>1.999178e+08</td>\n",
       "      <td>60627.0</td>\n",
       "      <td>109791.0</td>\n",
       "      <td>190002.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>buildingReplacementCost</td>\n",
       "      <td>BIGINT</td>\n",
       "      <td>2671486</td>\n",
       "      <td>23.76</td>\n",
       "      <td>0</td>\n",
       "      <td>9999000000</td>\n",
       "      <td>447053</td>\n",
       "      <td>8569836.74</td>\n",
       "      <td>2.350055e+08</td>\n",
       "      <td>0.0</td>\n",
       "      <td>120648.0</td>\n",
       "      <td>224076.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>buildingDamageAmount</td>\n",
       "      <td>BIGINT</td>\n",
       "      <td>2671486</td>\n",
       "      <td>23.76</td>\n",
       "      <td>0</td>\n",
       "      <td>927700000</td>\n",
       "      <td>202772</td>\n",
       "      <td>36354.08</td>\n",
       "      <td>8.123348e+05</td>\n",
       "      <td>3439.0</td>\n",
       "      <td>11218.0</td>\n",
       "      <td>40195.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               column_name column_type    count  null_percentage min  \\\n",
       "0    buildingPropertyValue      BIGINT  2671486            23.76   0   \n",
       "1  buildingReplacementCost      BIGINT  2671486            23.76   0   \n",
       "2     buildingDamageAmount      BIGINT  2671486            23.76   0   \n",
       "\n",
       "          max  approx_unique         avg           std      q25       q50  \\\n",
       "0  9950000000         420035  7148318.89  1.999178e+08  60627.0  109791.0   \n",
       "1  9999000000         447053  8569836.74  2.350055e+08      0.0  120648.0   \n",
       "2   927700000         202772    36354.08  8.123348e+05   3439.0   11218.0   \n",
       "\n",
       "        q75  \n",
       "0  190002.0  \n",
       "1  224076.0  \n",
       "2   40195.0  "
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# examine summary statistics on building damage amounts, property values and replacement costs columns\n",
    "con.sql(\"\"\"\n",
    "    SELECT\n",
    "        column_name, column_type, count, null_percentage,\n",
    "        min, max, approx_unique,\n",
    "        ROUND(avg::DOUBLE, 2) AS avg,\n",
    "        ROUND(std::DOUBLE, 2) AS std,\n",
    "        ROUND(q25::DOUBLE, 2) AS q25,\n",
    "        ROUND(q50::DOUBLE, 2) AS q50,\n",
    "        ROUND(q75::DOUBLE, 2) AS q75,\n",
    "    FROM\n",
    "        summarize_relation\n",
    "    WHERE column_name IN (\n",
    "        'buildingPropertyValue',\n",
    "        'buildingReplacementCost',\n",
    "        'buildingDamageAmount'\n",
    "    )\n",
    "\"\"\").df()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "94fbd712",
   "metadata": {},
   "source": [
    "## Highlighted Features: Elevation and Water Depth Summary Statistics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "8905cc25",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>column_name</th>\n",
       "      <th>column_type</th>\n",
       "      <th>count</th>\n",
       "      <th>null_percentage</th>\n",
       "      <th>min</th>\n",
       "      <th>max</th>\n",
       "      <th>approx_unique</th>\n",
       "      <th>avg</th>\n",
       "      <th>std</th>\n",
       "      <th>q25</th>\n",
       "      <th>q50</th>\n",
       "      <th>q75</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>elevationDifference</td>\n",
       "      <td>DOUBLE</td>\n",
       "      <td>2671486</td>\n",
       "      <td>72.95</td>\n",
       "      <td>-9989.0</td>\n",
       "      <td>998.0</td>\n",
       "      <td>371</td>\n",
       "      <td>1.25</td>\n",
       "      <td>29.09</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1.00</td>\n",
       "      <td>2.78</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>baseFloodElevation</td>\n",
       "      <td>DOUBLE</td>\n",
       "      <td>2671486</td>\n",
       "      <td>75.70</td>\n",
       "      <td>-9999.0</td>\n",
       "      <td>9998.0</td>\n",
       "      <td>10427</td>\n",
       "      <td>126.73</td>\n",
       "      <td>767.44</td>\n",
       "      <td>6.85</td>\n",
       "      <td>9.00</td>\n",
       "      <td>13.88</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>lowestAdjacentGrade</td>\n",
       "      <td>DOUBLE</td>\n",
       "      <td>2671486</td>\n",
       "      <td>81.26</td>\n",
       "      <td>-99999.9</td>\n",
       "      <td>9998.9</td>\n",
       "      <td>12638</td>\n",
       "      <td>49.70</td>\n",
       "      <td>1483.25</td>\n",
       "      <td>3.15</td>\n",
       "      <td>6.08</td>\n",
       "      <td>10.77</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>lowestFloorElevation</td>\n",
       "      <td>DOUBLE</td>\n",
       "      <td>2671486</td>\n",
       "      <td>76.38</td>\n",
       "      <td>-9999.0</td>\n",
       "      <td>9998.9</td>\n",
       "      <td>13294</td>\n",
       "      <td>104.33</td>\n",
       "      <td>616.41</td>\n",
       "      <td>6.55</td>\n",
       "      <td>10.27</td>\n",
       "      <td>17.33</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>waterDepth</td>\n",
       "      <td>BIGINT</td>\n",
       "      <td>2671486</td>\n",
       "      <td>9.40</td>\n",
       "      <td>-999</td>\n",
       "      <td>999</td>\n",
       "      <td>478</td>\n",
       "      <td>4.29</td>\n",
       "      <td>16.45</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1.00</td>\n",
       "      <td>2.00</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            column_name column_type    count  null_percentage       min  \\\n",
       "0   elevationDifference      DOUBLE  2671486            72.95   -9989.0   \n",
       "1    baseFloodElevation      DOUBLE  2671486            75.70   -9999.0   \n",
       "2   lowestAdjacentGrade      DOUBLE  2671486            81.26  -99999.9   \n",
       "3  lowestFloorElevation      DOUBLE  2671486            76.38   -9999.0   \n",
       "4            waterDepth      BIGINT  2671486             9.40      -999   \n",
       "\n",
       "      max  approx_unique     avg      std   q25    q50    q75  \n",
       "0   998.0            371    1.25    29.09  0.00   1.00   2.78  \n",
       "1  9998.0          10427  126.73   767.44  6.85   9.00  13.88  \n",
       "2  9998.9          12638   49.70  1483.25  3.15   6.08  10.77  \n",
       "3  9998.9          13294  104.33   616.41  6.55  10.27  17.33  \n",
       "4     999            478    4.29    16.45  0.00   1.00   2.00  "
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# examine summary statisitics on elevation and water depth columns\n",
    "con.sql(\"\"\"\n",
    "    SELECT\n",
    "        column_name, column_type, count, null_percentage,\n",
    "        min, max, approx_unique,\n",
    "        ROUND(avg::DOUBLE, 2) AS avg,\n",
    "        ROUND(std::DOUBLE, 2) AS std,\n",
    "        ROUND(q25::DOUBLE, 2) AS q25,\n",
    "        ROUND(q50::DOUBLE, 2) AS q50,\n",
    "        ROUND(q75::DOUBLE, 2) AS q75\n",
    "    FROM\n",
    "        summarize_relation\n",
    "    WHERE column_name IN (\n",
    "        'baseFloodElevation',\n",
    "        'waterDepth',\n",
    "        'lowestAdjacentGrade',\n",
    "        'lowestFloorElevation',\n",
    "        'elevationDifference',\n",
    "    )\n",
    "\"\"\").df()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e39aed71",
   "metadata": {},
   "source": [
    "# Analysis"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "91fb68cf",
   "metadata": {},
   "source": [
    "## Total Paid Claims"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "063a47ab",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>countClaims</th>\n",
       "      <th>paidTotalClaim</th>\n",
       "      <th>paidBuildingClaim</th>\n",
       "      <th>paidContentsClaim</th>\n",
       "      <th>paidICC</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2671486</td>\n",
       "      <td>80638352603</td>\n",
       "      <td>65049593608</td>\n",
       "      <td>14640321888</td>\n",
       "      <td>948437107</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   countClaims  paidTotalClaim  paidBuildingClaim  paidContentsClaim  \\\n",
       "0      2671486     80638352603        65049593608        14640321888   \n",
       "\n",
       "     paidICC  \n",
       "0  948437107  "
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "con.sql(\"\"\"\n",
    "    SELECT\n",
    "        COUNT(id) AS countClaims,\n",
    "        ROUND(\n",
    "            SUM(amountPaidOnBuildingClaim)\n",
    "            + SUM(amountPaidOnContentsClaim)\n",
    "            + SUM(amountPaidOnIncreasedCostOfComplianceClaim), 0)::BIGINT AS paidTotalClaim, \n",
    "        ROUND(SUM(amountPaidOnBuildingClaim), 0)::BIGINT AS paidBuildingClaim,\n",
    "        ROUND(SUM(amountPaidOnContentsClaim), 0)::BIGINT AS paidContentsClaim, \n",
    "        ROUND(SUM(amountPaidOnIncreasedCostOfComplianceClaim), 0)::BIGINT AS paidICC\n",
    "    FROM\n",
    "        claims\n",
    "\"\"\").df()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "64d62770",
   "metadata": {},
   "source": [
    "## Top 20 States by Total Paid Claims"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "63b6646f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>state</th>\n",
       "      <th>countClaims</th>\n",
       "      <th>paidTotalClaim</th>\n",
       "      <th>paidBuildingClaim</th>\n",
       "      <th>paidContentsClaim</th>\n",
       "      <th>paidICC</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>LA</td>\n",
       "      <td>483956</td>\n",
       "      <td>20869805385</td>\n",
       "      <td>16459981751</td>\n",
       "      <td>4132267467</td>\n",
       "      <td>277556167</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>TX</td>\n",
       "      <td>390889</td>\n",
       "      <td>17218491455</td>\n",
       "      <td>13253973375</td>\n",
       "      <td>3906593054</td>\n",
       "      <td>57925025</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>FL</td>\n",
       "      <td>415410</td>\n",
       "      <td>11433783265</td>\n",
       "      <td>9699132485</td>\n",
       "      <td>1696552579</td>\n",
       "      <td>38098201</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>NJ</td>\n",
       "      <td>201234</td>\n",
       "      <td>6470109817</td>\n",
       "      <td>5349181327</td>\n",
       "      <td>870522071</td>\n",
       "      <td>250406420</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>NY</td>\n",
       "      <td>174823</td>\n",
       "      <td>5727350837</td>\n",
       "      <td>4955149755</td>\n",
       "      <td>719127080</td>\n",
       "      <td>53074002</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>MS</td>\n",
       "      <td>64129</td>\n",
       "      <td>3122839913</td>\n",
       "      <td>2386323813</td>\n",
       "      <td>675594228</td>\n",
       "      <td>60921873</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>NC</td>\n",
       "      <td>107850</td>\n",
       "      <td>2009961692</td>\n",
       "      <td>1698489347</td>\n",
       "      <td>274446907</td>\n",
       "      <td>37025438</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>PA</td>\n",
       "      <td>76664</td>\n",
       "      <td>1432141143</td>\n",
       "      <td>1123943137</td>\n",
       "      <td>293019403</td>\n",
       "      <td>15178602</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>AL</td>\n",
       "      <td>44754</td>\n",
       "      <td>1192064063</td>\n",
       "      <td>966786458</td>\n",
       "      <td>212390245</td>\n",
       "      <td>12887360</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>SC</td>\n",
       "      <td>49288</td>\n",
       "      <td>1017326931</td>\n",
       "      <td>882236727</td>\n",
       "      <td>131285402</td>\n",
       "      <td>3804802</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>MO</td>\n",
       "      <td>50887</td>\n",
       "      <td>965994272</td>\n",
       "      <td>717690848</td>\n",
       "      <td>230177462</td>\n",
       "      <td>18125962</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>VA</td>\n",
       "      <td>50250</td>\n",
       "      <td>756843822</td>\n",
       "      <td>623759972</td>\n",
       "      <td>110764882</td>\n",
       "      <td>22318968</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>CA</td>\n",
       "      <td>53100</td>\n",
       "      <td>739296779</td>\n",
       "      <td>634543615</td>\n",
       "      <td>103686867</td>\n",
       "      <td>1066297</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>IL</td>\n",
       "      <td>52426</td>\n",
       "      <td>588547573</td>\n",
       "      <td>492251202</td>\n",
       "      <td>81055513</td>\n",
       "      <td>15240858</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>CT</td>\n",
       "      <td>29401</td>\n",
       "      <td>549798927</td>\n",
       "      <td>474274729</td>\n",
       "      <td>66042597</td>\n",
       "      <td>9481601</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>GA</td>\n",
       "      <td>23950</td>\n",
       "      <td>476373615</td>\n",
       "      <td>398277912</td>\n",
       "      <td>75369221</td>\n",
       "      <td>2726482</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>TN</td>\n",
       "      <td>17197</td>\n",
       "      <td>448025660</td>\n",
       "      <td>368235329</td>\n",
       "      <td>76608675</td>\n",
       "      <td>3181656</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>MA</td>\n",
       "      <td>35301</td>\n",
       "      <td>440448681</td>\n",
       "      <td>380191603</td>\n",
       "      <td>59007488</td>\n",
       "      <td>1249589</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>KY</td>\n",
       "      <td>25658</td>\n",
       "      <td>412628389</td>\n",
       "      <td>333201480</td>\n",
       "      <td>76512867</td>\n",
       "      <td>2914043</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>WV</td>\n",
       "      <td>27384</td>\n",
       "      <td>367058587</td>\n",
       "      <td>272494305</td>\n",
       "      <td>92146032</td>\n",
       "      <td>2418250</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   state  countClaims  paidTotalClaim  paidBuildingClaim  paidContentsClaim  \\\n",
       "0     LA       483956     20869805385        16459981751         4132267467   \n",
       "1     TX       390889     17218491455        13253973375         3906593054   \n",
       "2     FL       415410     11433783265         9699132485         1696552579   \n",
       "3     NJ       201234      6470109817         5349181327          870522071   \n",
       "4     NY       174823      5727350837         4955149755          719127080   \n",
       "5     MS        64129      3122839913         2386323813          675594228   \n",
       "6     NC       107850      2009961692         1698489347          274446907   \n",
       "7     PA        76664      1432141143         1123943137          293019403   \n",
       "8     AL        44754      1192064063          966786458          212390245   \n",
       "9     SC        49288      1017326931          882236727          131285402   \n",
       "10    MO        50887       965994272          717690848          230177462   \n",
       "11    VA        50250       756843822          623759972          110764882   \n",
       "12    CA        53100       739296779          634543615          103686867   \n",
       "13    IL        52426       588547573          492251202           81055513   \n",
       "14    CT        29401       549798927          474274729           66042597   \n",
       "15    GA        23950       476373615          398277912           75369221   \n",
       "16    TN        17197       448025660          368235329           76608675   \n",
       "17    MA        35301       440448681          380191603           59007488   \n",
       "18    KY        25658       412628389          333201480           76512867   \n",
       "19    WV        27384       367058587          272494305           92146032   \n",
       "\n",
       "      paidICC  \n",
       "0   277556167  \n",
       "1    57925025  \n",
       "2    38098201  \n",
       "3   250406420  \n",
       "4    53074002  \n",
       "5    60921873  \n",
       "6    37025438  \n",
       "7    15178602  \n",
       "8    12887360  \n",
       "9     3804802  \n",
       "10   18125962  \n",
       "11   22318968  \n",
       "12    1066297  \n",
       "13   15240858  \n",
       "14    9481601  \n",
       "15    2726482  \n",
       "16    3181656  \n",
       "17    1249589  \n",
       "18    2914043  \n",
       "19    2418250  "
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "con.sql(\"\"\"\n",
    "    SELECT\n",
    "        state,\n",
    "        COUNT(id) AS countClaims,\n",
    "        ROUND(\n",
    "            SUM(amountPaidOnBuildingClaim)\n",
    "            + SUM(amountPaidOnContentsClaim)\n",
    "            + SUM(amountPaidOnIncreasedCostOfComplianceClaim), 0)::BIGINT AS paidTotalClaim, \n",
    "        ROUND(SUM(amountPaidOnBuildingClaim), 0)::BIGINT AS paidBuildingClaim,\n",
    "        ROUND(SUM(amountPaidOnContentsClaim), 0)::BIGINT AS paidContentsClaim, \n",
    "        ROUND(SUM(amountPaidOnIncreasedCostOfComplianceClaim), 0)::BIGINT AS paidICC\n",
    "    FROM\n",
    "        claims\n",
    "    GROUP BY\n",
    "        state\n",
    "    ORDER BY\n",
    "        paidTotalClaim DESC\n",
    "    LIMIT 20\n",
    "\"\"\").df()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "14e9840c",
   "metadata": {},
   "source": [
    "## Top 20 Counties by Total Paid Claims"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "5da1d09e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>state</th>\n",
       "      <th>countyCode</th>\n",
       "      <th>countClaims</th>\n",
       "      <th>paidTotalClaim</th>\n",
       "      <th>paidBuildingClaim</th>\n",
       "      <th>paidContentsClaim</th>\n",
       "      <th>paidICC</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>TX</td>\n",
       "      <td>48201</td>\n",
       "      <td>170625</td>\n",
       "      <td>8748468467</td>\n",
       "      <td>6794800334</td>\n",
       "      <td>1933648655</td>\n",
       "      <td>20019478</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>LA</td>\n",
       "      <td>22071</td>\n",
       "      <td>127137</td>\n",
       "      <td>7296273827</td>\n",
       "      <td>5892518413</td>\n",
       "      <td>1293261898</td>\n",
       "      <td>110493515</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>LA</td>\n",
       "      <td>22051</td>\n",
       "      <td>135361</td>\n",
       "      <td>3542723793</td>\n",
       "      <td>2633465878</td>\n",
       "      <td>847684749</td>\n",
       "      <td>61573166</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>FL</td>\n",
       "      <td>12071</td>\n",
       "      <td>43354</td>\n",
       "      <td>3413583297</td>\n",
       "      <td>3050925028</td>\n",
       "      <td>360290319</td>\n",
       "      <td>2367950</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>NJ</td>\n",
       "      <td>34029</td>\n",
       "      <td>52795</td>\n",
       "      <td>2607964807</td>\n",
       "      <td>2160867823</td>\n",
       "      <td>275256283</td>\n",
       "      <td>171840701</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>TX</td>\n",
       "      <td>48167</td>\n",
       "      <td>60552</td>\n",
       "      <td>2466669297</td>\n",
       "      <td>1896169966</td>\n",
       "      <td>549528562</td>\n",
       "      <td>20970769</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>NY</td>\n",
       "      <td>36059</td>\n",
       "      <td>51480</td>\n",
       "      <td>2279955547</td>\n",
       "      <td>1972897371</td>\n",
       "      <td>290304539</td>\n",
       "      <td>16753636</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>LA</td>\n",
       "      <td>22087</td>\n",
       "      <td>23913</td>\n",
       "      <td>2245090178</td>\n",
       "      <td>1680098976</td>\n",
       "      <td>553956976</td>\n",
       "      <td>11034225</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>LA</td>\n",
       "      <td>22103</td>\n",
       "      <td>38287</td>\n",
       "      <td>1775723597</td>\n",
       "      <td>1378279810</td>\n",
       "      <td>383168918</td>\n",
       "      <td>14274869</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>LA</td>\n",
       "      <td>22033</td>\n",
       "      <td>22493</td>\n",
       "      <td>1311706040</td>\n",
       "      <td>1140369431</td>\n",
       "      <td>169389761</td>\n",
       "      <td>1946847</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>MS</td>\n",
       "      <td>28047</td>\n",
       "      <td>15790</td>\n",
       "      <td>1286615324</td>\n",
       "      <td>982205205</td>\n",
       "      <td>283679372</td>\n",
       "      <td>20730748</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>TX</td>\n",
       "      <td>48245</td>\n",
       "      <td>22623</td>\n",
       "      <td>1187486702</td>\n",
       "      <td>877003753</td>\n",
       "      <td>308157176</td>\n",
       "      <td>2325774</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>NY</td>\n",
       "      <td>36103</td>\n",
       "      <td>33878</td>\n",
       "      <td>1106846500</td>\n",
       "      <td>971688211</td>\n",
       "      <td>120734573</td>\n",
       "      <td>14423716</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>TX</td>\n",
       "      <td>48361</td>\n",
       "      <td>12848</td>\n",
       "      <td>1042800925</td>\n",
       "      <td>740487399</td>\n",
       "      <td>301503624</td>\n",
       "      <td>809902</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>FL</td>\n",
       "      <td>12021</td>\n",
       "      <td>13243</td>\n",
       "      <td>972581605</td>\n",
       "      <td>856304732</td>\n",
       "      <td>115398671</td>\n",
       "      <td>878203</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>NJ</td>\n",
       "      <td>34025</td>\n",
       "      <td>19956</td>\n",
       "      <td>935060531</td>\n",
       "      <td>798899181</td>\n",
       "      <td>96557644</td>\n",
       "      <td>39603706</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>FL</td>\n",
       "      <td>12086</td>\n",
       "      <td>61856</td>\n",
       "      <td>841793421</td>\n",
       "      <td>661058031</td>\n",
       "      <td>180735390</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>LA</td>\n",
       "      <td>22063</td>\n",
       "      <td>14628</td>\n",
       "      <td>834936081</td>\n",
       "      <td>716190103</td>\n",
       "      <td>113401620</td>\n",
       "      <td>5344358</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>MS</td>\n",
       "      <td>28045</td>\n",
       "      <td>9837</td>\n",
       "      <td>744297690</td>\n",
       "      <td>556677410</td>\n",
       "      <td>168197640</td>\n",
       "      <td>19422640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>FL</td>\n",
       "      <td>12033</td>\n",
       "      <td>16497</td>\n",
       "      <td>731754485</td>\n",
       "      <td>596730856</td>\n",
       "      <td>126287456</td>\n",
       "      <td>8736173</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   state countyCode  countClaims  paidTotalClaim  paidBuildingClaim  \\\n",
       "0     TX      48201       170625      8748468467         6794800334   \n",
       "1     LA      22071       127137      7296273827         5892518413   \n",
       "2     LA      22051       135361      3542723793         2633465878   \n",
       "3     FL      12071        43354      3413583297         3050925028   \n",
       "4     NJ      34029        52795      2607964807         2160867823   \n",
       "5     TX      48167        60552      2466669297         1896169966   \n",
       "6     NY      36059        51480      2279955547         1972897371   \n",
       "7     LA      22087        23913      2245090178         1680098976   \n",
       "8     LA      22103        38287      1775723597         1378279810   \n",
       "9     LA      22033        22493      1311706040         1140369431   \n",
       "10    MS      28047        15790      1286615324          982205205   \n",
       "11    TX      48245        22623      1187486702          877003753   \n",
       "12    NY      36103        33878      1106846500          971688211   \n",
       "13    TX      48361        12848      1042800925          740487399   \n",
       "14    FL      12021        13243       972581605          856304732   \n",
       "15    NJ      34025        19956       935060531          798899181   \n",
       "16    FL      12086        61856       841793421          661058031   \n",
       "17    LA      22063        14628       834936081          716190103   \n",
       "18    MS      28045         9837       744297690          556677410   \n",
       "19    FL      12033        16497       731754485          596730856   \n",
       "\n",
       "    paidContentsClaim    paidICC  \n",
       "0          1933648655   20019478  \n",
       "1          1293261898  110493515  \n",
       "2           847684749   61573166  \n",
       "3           360290319    2367950  \n",
       "4           275256283  171840701  \n",
       "5           549528562   20970769  \n",
       "6           290304539   16753636  \n",
       "7           553956976   11034225  \n",
       "8           383168918   14274869  \n",
       "9           169389761    1946847  \n",
       "10          283679372   20730748  \n",
       "11          308157176    2325774  \n",
       "12          120734573   14423716  \n",
       "13          301503624     809902  \n",
       "14          115398671     878203  \n",
       "15           96557644   39603706  \n",
       "16          180735390          0  \n",
       "17          113401620    5344358  \n",
       "18          168197640   19422640  \n",
       "19          126287456    8736173  "
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "con.sql(\"\"\"\n",
    "    SELECT\n",
    "        state,\n",
    "        countyCode,\n",
    "        COUNT(id) AS countClaims,\n",
    "        ROUND(\n",
    "            SUM(amountPaidOnBuildingClaim)\n",
    "            + SUM(amountPaidOnContentsClaim)\n",
    "            + SUM(amountPaidOnIncreasedCostOfComplianceClaim), 0)::BIGINT AS paidTotalClaim,  \n",
    "        ROUND(SUM(amountPaidOnBuildingClaim), 0)::BIGINT AS paidBuildingClaim,\n",
    "        ROUND(SUM(amountPaidOnContentsClaim), 0)::BIGINT AS paidContentsClaim, \n",
    "        ROUND(SUM(amountPaidOnIncreasedCostOfComplianceClaim), 0)::BIGINT AS paidICC\n",
    "    FROM\n",
    "        claims\n",
    "    GROUP BY\n",
    "        state, countyCode\n",
    "    ORDER BY\n",
    "        paidTotalClaim DESC\n",
    "    LIMIT 20\n",
    "\"\"\").df()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "227b6ea5",
   "metadata": {},
   "source": [
    "## Top 20 Flood Events by Total Paid Claims"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "f2c0bd57",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>floodEvent</th>\n",
       "      <th>yearOfLoss</th>\n",
       "      <th>countClaims</th>\n",
       "      <th>paidTotalClaim</th>\n",
       "      <th>paidBuildingClaim</th>\n",
       "      <th>paidContentsClaim</th>\n",
       "      <th>paidICC</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Hurricane Katrina</td>\n",
       "      <td>2005</td>\n",
       "      <td>208348</td>\n",
       "      <td>16261697056</td>\n",
       "      <td>12659081935</td>\n",
       "      <td>3360020221</td>\n",
       "      <td>242594900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Hurricane Harvey</td>\n",
       "      <td>2017</td>\n",
       "      <td>92396</td>\n",
       "      <td>9055522699</td>\n",
       "      <td>6925370027</td>\n",
       "      <td>2115077279</td>\n",
       "      <td>15075393</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Hurricane Sandy</td>\n",
       "      <td>2012</td>\n",
       "      <td>144848</td>\n",
       "      <td>8956609943</td>\n",
       "      <td>7707744797</td>\n",
       "      <td>951637071</td>\n",
       "      <td>297228075</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Hurricane Ian</td>\n",
       "      <td>2022</td>\n",
       "      <td>48721</td>\n",
       "      <td>4757384580</td>\n",
       "      <td>4229314739</td>\n",
       "      <td>525543744</td>\n",
       "      <td>2526097</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Hurricane Ike</td>\n",
       "      <td>2008</td>\n",
       "      <td>58126</td>\n",
       "      <td>2702511916</td>\n",
       "      <td>2073801567</td>\n",
       "      <td>577791589</td>\n",
       "      <td>50918760</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Mid-summer severe storms</td>\n",
       "      <td>2016</td>\n",
       "      <td>30017</td>\n",
       "      <td>2533431383</td>\n",
       "      <td>2175218939</td>\n",
       "      <td>347263588</td>\n",
       "      <td>10948856</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Hurricane Irene</td>\n",
       "      <td>2011</td>\n",
       "      <td>52493</td>\n",
       "      <td>1347399996</td>\n",
       "      <td>1139897981</td>\n",
       "      <td>183189121</td>\n",
       "      <td>24312894</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Hurricane Ida</td>\n",
       "      <td>2021</td>\n",
       "      <td>28317</td>\n",
       "      <td>1346749832</td>\n",
       "      <td>1115951795</td>\n",
       "      <td>229650151</td>\n",
       "      <td>1147886</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Hurricane Ivan</td>\n",
       "      <td>2004</td>\n",
       "      <td>20137</td>\n",
       "      <td>1325419294</td>\n",
       "      <td>1083795424</td>\n",
       "      <td>221959720</td>\n",
       "      <td>19664150</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Hurricane Irma</td>\n",
       "      <td>2017</td>\n",
       "      <td>33339</td>\n",
       "      <td>1114668772</td>\n",
       "      <td>945821300</td>\n",
       "      <td>162148464</td>\n",
       "      <td>6699008</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Tropical Storm Allison</td>\n",
       "      <td>2001</td>\n",
       "      <td>35561</td>\n",
       "      <td>1104979705</td>\n",
       "      <td>820333554</td>\n",
       "      <td>279280479</td>\n",
       "      <td>5365672</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Tropical Storm Imelda</td>\n",
       "      <td>2019</td>\n",
       "      <td>10940</td>\n",
       "      <td>752448943</td>\n",
       "      <td>589796293</td>\n",
       "      <td>161684484</td>\n",
       "      <td>968166</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Hurricane Florence</td>\n",
       "      <td>2018</td>\n",
       "      <td>16801</td>\n",
       "      <td>743381994</td>\n",
       "      <td>630705096</td>\n",
       "      <td>104817034</td>\n",
       "      <td>7859865</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Flooding</td>\n",
       "      <td>1995</td>\n",
       "      <td>47489</td>\n",
       "      <td>731729839</td>\n",
       "      <td>554632707</td>\n",
       "      <td>177097132</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>Hurricane Matthew</td>\n",
       "      <td>2016</td>\n",
       "      <td>21818</td>\n",
       "      <td>662071059</td>\n",
       "      <td>563918087</td>\n",
       "      <td>92582081</td>\n",
       "      <td>5570891</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>Hurricane Isaac</td>\n",
       "      <td>2012</td>\n",
       "      <td>16926</td>\n",
       "      <td>564085634</td>\n",
       "      <td>416257051</td>\n",
       "      <td>138859651</td>\n",
       "      <td>8968932</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>Hurricane Isabel</td>\n",
       "      <td>2003</td>\n",
       "      <td>23955</td>\n",
       "      <td>500076497</td>\n",
       "      <td>408652888</td>\n",
       "      <td>59803277</td>\n",
       "      <td>31620332</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>Early summer storms</td>\n",
       "      <td>2015</td>\n",
       "      <td>8354</td>\n",
       "      <td>487749123</td>\n",
       "      <td>393750947</td>\n",
       "      <td>91265985</td>\n",
       "      <td>2732190</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>Spring severe storms</td>\n",
       "      <td>2016</td>\n",
       "      <td>8433</td>\n",
       "      <td>472881300</td>\n",
       "      <td>389316635</td>\n",
       "      <td>82963197</td>\n",
       "      <td>601467</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>Hurricane Rita</td>\n",
       "      <td>2005</td>\n",
       "      <td>14662</td>\n",
       "      <td>466353815</td>\n",
       "      <td>347426098</td>\n",
       "      <td>85449587</td>\n",
       "      <td>33478130</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                  floodEvent  yearOfLoss  countClaims  paidTotalClaim  \\\n",
       "0          Hurricane Katrina        2005       208348     16261697056   \n",
       "1           Hurricane Harvey        2017        92396      9055522699   \n",
       "2            Hurricane Sandy        2012       144848      8956609943   \n",
       "3              Hurricane Ian        2022        48721      4757384580   \n",
       "4              Hurricane Ike        2008        58126      2702511916   \n",
       "5   Mid-summer severe storms        2016        30017      2533431383   \n",
       "6            Hurricane Irene        2011        52493      1347399996   \n",
       "7              Hurricane Ida        2021        28317      1346749832   \n",
       "8             Hurricane Ivan        2004        20137      1325419294   \n",
       "9             Hurricane Irma        2017        33339      1114668772   \n",
       "10    Tropical Storm Allison        2001        35561      1104979705   \n",
       "11     Tropical Storm Imelda        2019        10940       752448943   \n",
       "12        Hurricane Florence        2018        16801       743381994   \n",
       "13                  Flooding        1995        47489       731729839   \n",
       "14         Hurricane Matthew        2016        21818       662071059   \n",
       "15           Hurricane Isaac        2012        16926       564085634   \n",
       "16          Hurricane Isabel        2003        23955       500076497   \n",
       "17       Early summer storms        2015         8354       487749123   \n",
       "18      Spring severe storms        2016         8433       472881300   \n",
       "19            Hurricane Rita        2005        14662       466353815   \n",
       "\n",
       "    paidBuildingClaim  paidContentsClaim    paidICC  \n",
       "0         12659081935         3360020221  242594900  \n",
       "1          6925370027         2115077279   15075393  \n",
       "2          7707744797          951637071  297228075  \n",
       "3          4229314739          525543744    2526097  \n",
       "4          2073801567          577791589   50918760  \n",
       "5          2175218939          347263588   10948856  \n",
       "6          1139897981          183189121   24312894  \n",
       "7          1115951795          229650151    1147886  \n",
       "8          1083795424          221959720   19664150  \n",
       "9           945821300          162148464    6699008  \n",
       "10          820333554          279280479    5365672  \n",
       "11          589796293          161684484     968166  \n",
       "12          630705096          104817034    7859865  \n",
       "13          554632707          177097132          0  \n",
       "14          563918087           92582081    5570891  \n",
       "15          416257051          138859651    8968932  \n",
       "16          408652888           59803277   31620332  \n",
       "17          393750947           91265985    2732190  \n",
       "18          389316635           82963197     601467  \n",
       "19          347426098           85449587   33478130  "
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "con.sql(\"\"\"\n",
    "    SELECT\n",
    "        floodEvent,\n",
    "        yearOfLoss,\n",
    "        COUNT(id) AS countClaims,\n",
    "        ROUND(\n",
    "            SUM(amountPaidOnBuildingClaim)\n",
    "            + SUM(amountPaidOnContentsClaim)\n",
    "            + SUM(amountPaidOnIncreasedCostOfComplianceClaim), 0)::BIGINT AS paidTotalClaim,  \n",
    "        ROUND(SUM(amountPaidOnBuildingClaim), 0)::BIGINT AS paidBuildingClaim,\n",
    "        ROUND(SUM(amountPaidOnContentsClaim), 0)::BIGINT AS paidContentsClaim, \n",
    "        ROUND(SUM(amountPaidOnIncreasedCostOfComplianceClaim), 0)::BIGINT AS paidICC\n",
    "    FROM\n",
    "        claims\n",
    "    WHERE\n",
    "        floodEvent NOT NULL\n",
    "    GROUP BY\n",
    "        floodEvent, yearOfLoss\n",
    "    ORDER BY\n",
    "        paidTotalClaim DESC\n",
    "    LIMIT 20\n",
    "\"\"\").df()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "571c3e1b",
   "metadata": {},
   "source": [
    "## Top 20 States and Flood Events by Total Paid Claims"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "552248d4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>state</th>\n",
       "      <th>floodEvent</th>\n",
       "      <th>yearOfLoss</th>\n",
       "      <th>countClaims</th>\n",
       "      <th>paidTotalClaim</th>\n",
       "      <th>paidBuildingClaim</th>\n",
       "      <th>paidContentsClaim</th>\n",
       "      <th>paidICC</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>LA</td>\n",
       "      <td>Hurricane Katrina</td>\n",
       "      <td>2005</td>\n",
       "      <td>176276</td>\n",
       "      <td>13347164644</td>\n",
       "      <td>10441743931</td>\n",
       "      <td>2721451049</td>\n",
       "      <td>183969663</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>TX</td>\n",
       "      <td>Hurricane Harvey</td>\n",
       "      <td>2017</td>\n",
       "      <td>91870</td>\n",
       "      <td>9040181063</td>\n",
       "      <td>6912549831</td>\n",
       "      <td>2112555840</td>\n",
       "      <td>15075393</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>FL</td>\n",
       "      <td>Hurricane Ian</td>\n",
       "      <td>2022</td>\n",
       "      <td>47457</td>\n",
       "      <td>4711517234</td>\n",
       "      <td>4187737200</td>\n",
       "      <td>521388936</td>\n",
       "      <td>2391097</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>NJ</td>\n",
       "      <td>Hurricane Sandy</td>\n",
       "      <td>2012</td>\n",
       "      <td>74983</td>\n",
       "      <td>4372115685</td>\n",
       "      <td>3673280769</td>\n",
       "      <td>457640033</td>\n",
       "      <td>241194883</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>NY</td>\n",
       "      <td>Hurricane Sandy</td>\n",
       "      <td>2012</td>\n",
       "      <td>57405</td>\n",
       "      <td>4214352197</td>\n",
       "      <td>3702376632</td>\n",
       "      <td>465410811</td>\n",
       "      <td>46564754</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>LA</td>\n",
       "      <td>Mid-summer severe storms</td>\n",
       "      <td>2016</td>\n",
       "      <td>30017</td>\n",
       "      <td>2533431383</td>\n",
       "      <td>2175218939</td>\n",
       "      <td>347263588</td>\n",
       "      <td>10948856</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>MS</td>\n",
       "      <td>Hurricane Katrina</td>\n",
       "      <td>2005</td>\n",
       "      <td>19051</td>\n",
       "      <td>2521512578</td>\n",
       "      <td>1910623703</td>\n",
       "      <td>558996059</td>\n",
       "      <td>51892816</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>TX</td>\n",
       "      <td>Hurricane Ike</td>\n",
       "      <td>2008</td>\n",
       "      <td>44095</td>\n",
       "      <td>2230307106</td>\n",
       "      <td>1723323896</td>\n",
       "      <td>481446923</td>\n",
       "      <td>25536286</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>FL</td>\n",
       "      <td>Hurricane Irma</td>\n",
       "      <td>2017</td>\n",
       "      <td>28755</td>\n",
       "      <td>985551621</td>\n",
       "      <td>834411306</td>\n",
       "      <td>145153321</td>\n",
       "      <td>5986994</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>TX</td>\n",
       "      <td>Tropical Storm Allison</td>\n",
       "      <td>2001</td>\n",
       "      <td>26821</td>\n",
       "      <td>978096837</td>\n",
       "      <td>723017851</td>\n",
       "      <td>250198245</td>\n",
       "      <td>4880741</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>FL</td>\n",
       "      <td>Hurricane Ivan</td>\n",
       "      <td>2004</td>\n",
       "      <td>12934</td>\n",
       "      <td>966741463</td>\n",
       "      <td>768189098</td>\n",
       "      <td>182722004</td>\n",
       "      <td>15830361</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>TX</td>\n",
       "      <td>Tropical Storm Imelda</td>\n",
       "      <td>2019</td>\n",
       "      <td>10937</td>\n",
       "      <td>752448943</td>\n",
       "      <td>589796293</td>\n",
       "      <td>161684484</td>\n",
       "      <td>968166</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>LA</td>\n",
       "      <td>Hurricane Ida</td>\n",
       "      <td>2021</td>\n",
       "      <td>14676</td>\n",
       "      <td>689007359</td>\n",
       "      <td>557400334</td>\n",
       "      <td>130739071</td>\n",
       "      <td>867954</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>NC</td>\n",
       "      <td>Hurricane Florence</td>\n",
       "      <td>2018</td>\n",
       "      <td>15337</td>\n",
       "      <td>637200566</td>\n",
       "      <td>541223398</td>\n",
       "      <td>89079169</td>\n",
       "      <td>6897999</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>NJ</td>\n",
       "      <td>Hurricane Irene</td>\n",
       "      <td>2011</td>\n",
       "      <td>18125</td>\n",
       "      <td>599488993</td>\n",
       "      <td>497230608</td>\n",
       "      <td>96994208</td>\n",
       "      <td>5264178</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>LA</td>\n",
       "      <td>Flooding</td>\n",
       "      <td>1995</td>\n",
       "      <td>36820</td>\n",
       "      <td>594357775</td>\n",
       "      <td>442113752</td>\n",
       "      <td>152244023</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>LA</td>\n",
       "      <td>Hurricane Isaac</td>\n",
       "      <td>2012</td>\n",
       "      <td>13318</td>\n",
       "      <td>513274540</td>\n",
       "      <td>375382324</td>\n",
       "      <td>129362462</td>\n",
       "      <td>8529754</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>TX</td>\n",
       "      <td>Spring severe storms</td>\n",
       "      <td>2016</td>\n",
       "      <td>8414</td>\n",
       "      <td>472718247</td>\n",
       "      <td>389154552</td>\n",
       "      <td>82962228</td>\n",
       "      <td>601467</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>TX</td>\n",
       "      <td>Early summer storms</td>\n",
       "      <td>2015</td>\n",
       "      <td>7983</td>\n",
       "      <td>472278409</td>\n",
       "      <td>380339421</td>\n",
       "      <td>89255337</td>\n",
       "      <td>2683650</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>LA</td>\n",
       "      <td>Hurricane Rita</td>\n",
       "      <td>2005</td>\n",
       "      <td>10810</td>\n",
       "      <td>415238845</td>\n",
       "      <td>308566580</td>\n",
       "      <td>73331784</td>\n",
       "      <td>33340481</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   state                floodEvent  yearOfLoss  countClaims  paidTotalClaim  \\\n",
       "0     LA         Hurricane Katrina        2005       176276     13347164644   \n",
       "1     TX          Hurricane Harvey        2017        91870      9040181063   \n",
       "2     FL             Hurricane Ian        2022        47457      4711517234   \n",
       "3     NJ           Hurricane Sandy        2012        74983      4372115685   \n",
       "4     NY           Hurricane Sandy        2012        57405      4214352197   \n",
       "5     LA  Mid-summer severe storms        2016        30017      2533431383   \n",
       "6     MS         Hurricane Katrina        2005        19051      2521512578   \n",
       "7     TX             Hurricane Ike        2008        44095      2230307106   \n",
       "8     FL            Hurricane Irma        2017        28755       985551621   \n",
       "9     TX    Tropical Storm Allison        2001        26821       978096837   \n",
       "10    FL            Hurricane Ivan        2004        12934       966741463   \n",
       "11    TX     Tropical Storm Imelda        2019        10937       752448943   \n",
       "12    LA             Hurricane Ida        2021        14676       689007359   \n",
       "13    NC        Hurricane Florence        2018        15337       637200566   \n",
       "14    NJ           Hurricane Irene        2011        18125       599488993   \n",
       "15    LA                  Flooding        1995        36820       594357775   \n",
       "16    LA           Hurricane Isaac        2012        13318       513274540   \n",
       "17    TX      Spring severe storms        2016         8414       472718247   \n",
       "18    TX       Early summer storms        2015         7983       472278409   \n",
       "19    LA            Hurricane Rita        2005        10810       415238845   \n",
       "\n",
       "    paidBuildingClaim  paidContentsClaim    paidICC  \n",
       "0         10441743931         2721451049  183969663  \n",
       "1          6912549831         2112555840   15075393  \n",
       "2          4187737200          521388936    2391097  \n",
       "3          3673280769          457640033  241194883  \n",
       "4          3702376632          465410811   46564754  \n",
       "5          2175218939          347263588   10948856  \n",
       "6          1910623703          558996059   51892816  \n",
       "7          1723323896          481446923   25536286  \n",
       "8           834411306          145153321    5986994  \n",
       "9           723017851          250198245    4880741  \n",
       "10          768189098          182722004   15830361  \n",
       "11          589796293          161684484     968166  \n",
       "12          557400334          130739071     867954  \n",
       "13          541223398           89079169    6897999  \n",
       "14          497230608           96994208    5264178  \n",
       "15          442113752          152244023          0  \n",
       "16          375382324          129362462    8529754  \n",
       "17          389154552           82962228     601467  \n",
       "18          380339421           89255337    2683650  \n",
       "19          308566580           73331784   33340481  "
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "con.sql(\"\"\"\n",
    "    SELECT\n",
    "        state,\n",
    "        floodEvent,\n",
    "        yearOfLoss,\n",
    "        COUNT(id) AS countClaims,\n",
    "        ROUND(\n",
    "            SUM(amountPaidOnBuildingClaim)\n",
    "            + SUM(amountPaidOnContentsClaim)\n",
    "            + SUM(amountPaidOnIncreasedCostOfComplianceClaim), 0)::BIGINT AS paidTotalClaim,  \n",
    "        ROUND(SUM(amountPaidOnBuildingClaim), 0)::BIGINT AS paidBuildingClaim,\n",
    "        ROUND(SUM(amountPaidOnContentsClaim), 0)::BIGINT AS paidContentsClaim, \n",
    "        ROUND(SUM(amountPaidOnIncreasedCostOfComplianceClaim), 0)::BIGINT AS paidICC\n",
    "    FROM\n",
    "        claims\n",
    "    WHERE\n",
    "        floodEvent NOT NULL\n",
    "    GROUP BY ALL\n",
    "    ORDER BY\n",
    "        paidTotalClaim DESC\n",
    "    LIMIT 20\n",
    "\"\"\").df()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3012607e",
   "metadata": {},
   "source": [
    "## Top 20 Rated Flood Zones by Total Paid Claims\n",
    "From the Data Dictionary:\n",
    ">Formerly called floodZone. NFIP Flood Zone derived from the Flood Insurance Rate Map (FIRM) used to rate the insured property. A - Special Flood with no Base Flood Elevation on FIRM; AE, A1-A30 - Special Flood with Base Flood Elevation on FIRM; A99 - Special Flood with Protection Zone; AH, AHB* - Special Flood with Shallow Ponding; AO, AOB* - Special Flood with Sheet Flow; X, B - Moderate Flood from primary water source. Pockets of areas subject to drainage problems; X, C - Minimal Flood from primary water source. Pockets of areas subject to drainage problems; D - Possible Flood; V - Velocity Flood with no Base Flood Elevation on FIRM; VE, V1-V30 - Velocity Flood with Base Flood Elevation on FIRM; AE, VE, X - New zone designations used on new maps starting January 1, 1986, in lieu of A1-A30, V1-V30, and B and C; AR - A Special Flood Hazard Area that results from the decertification of a previously accredited flood protection system that is determined to be in the process of being restored to provide base flood protection; AR Dual Zones - (AR/AE, AR/A1-A30, AR/AH, AR/AO, AR/A) Areas subject to flooding from failure of the flood protection system (Zone AR) which also overlap an existing Special Flood Hazard Area as a dual zone; *AHB, AOB, ARE, ARH, ARO, and ARA are not risk zones shown on a map, but are acceptable values for rating purposes*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "21a9b7e5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ratedFloodZone</th>\n",
       "      <th>countClaims</th>\n",
       "      <th>paidTotalClaim</th>\n",
       "      <th>paidBuildingClaim</th>\n",
       "      <th>paidContentsClaim</th>\n",
       "      <th>paidICC</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>AE</td>\n",
       "      <td>902694</td>\n",
       "      <td>35479231566</td>\n",
       "      <td>29574222634</td>\n",
       "      <td>5389301443</td>\n",
       "      <td>515707490</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>X</td>\n",
       "      <td>398598</td>\n",
       "      <td>13436983455</td>\n",
       "      <td>10383896784</td>\n",
       "      <td>3024472374</td>\n",
       "      <td>28614297</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>B</td>\n",
       "      <td>114822</td>\n",
       "      <td>3826110386</td>\n",
       "      <td>2835364624</td>\n",
       "      <td>971908875</td>\n",
       "      <td>18836887</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>A</td>\n",
       "      <td>198639</td>\n",
       "      <td>3130755422</td>\n",
       "      <td>2486940415</td>\n",
       "      <td>606541394</td>\n",
       "      <td>37273612</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>C</td>\n",
       "      <td>163832</td>\n",
       "      <td>2853210440</td>\n",
       "      <td>2119312629</td>\n",
       "      <td>723803760</td>\n",
       "      <td>10094051</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>A04</td>\n",
       "      <td>51806</td>\n",
       "      <td>2185871995</td>\n",
       "      <td>1800832671</td>\n",
       "      <td>364399597</td>\n",
       "      <td>20639726</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>A03</td>\n",
       "      <td>41872</td>\n",
       "      <td>1650992713</td>\n",
       "      <td>1337863894</td>\n",
       "      <td>284704083</td>\n",
       "      <td>28424737</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>VE</td>\n",
       "      <td>46418</td>\n",
       "      <td>1604044125</td>\n",
       "      <td>1427115498</td>\n",
       "      <td>158958851</td>\n",
       "      <td>17969776</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>A01</td>\n",
       "      <td>51362</td>\n",
       "      <td>1526019156</td>\n",
       "      <td>1188933368</td>\n",
       "      <td>320999694</td>\n",
       "      <td>16086094</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>A08</td>\n",
       "      <td>45657</td>\n",
       "      <td>1455607732</td>\n",
       "      <td>1185003260</td>\n",
       "      <td>237429765</td>\n",
       "      <td>33174708</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>A05</td>\n",
       "      <td>48604</td>\n",
       "      <td>1275294814</td>\n",
       "      <td>1059529435</td>\n",
       "      <td>186761764</td>\n",
       "      <td>29003615</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>A10</td>\n",
       "      <td>41151</td>\n",
       "      <td>1272548151</td>\n",
       "      <td>993009398</td>\n",
       "      <td>260656901</td>\n",
       "      <td>18881852</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>A07</td>\n",
       "      <td>43624</td>\n",
       "      <td>1150411323</td>\n",
       "      <td>939607938</td>\n",
       "      <td>183942076</td>\n",
       "      <td>26861309</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>A06</td>\n",
       "      <td>47085</td>\n",
       "      <td>1045591886</td>\n",
       "      <td>845127018</td>\n",
       "      <td>171603401</td>\n",
       "      <td>28861467</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>A02</td>\n",
       "      <td>31666</td>\n",
       "      <td>966418024</td>\n",
       "      <td>764371792</td>\n",
       "      <td>190319082</td>\n",
       "      <td>11727150</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>A11</td>\n",
       "      <td>30093</td>\n",
       "      <td>835448574</td>\n",
       "      <td>667150308</td>\n",
       "      <td>154405220</td>\n",
       "      <td>13893046</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>A09</td>\n",
       "      <td>26680</td>\n",
       "      <td>793640291</td>\n",
       "      <td>608541563</td>\n",
       "      <td>167178365</td>\n",
       "      <td>17920363</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>A14</td>\n",
       "      <td>24294</td>\n",
       "      <td>702934354</td>\n",
       "      <td>539529371</td>\n",
       "      <td>153188379</td>\n",
       "      <td>10216604</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>None</td>\n",
       "      <td>139025</td>\n",
       "      <td>696659870</td>\n",
       "      <td>491536385</td>\n",
       "      <td>205049316</td>\n",
       "      <td>74169</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>AH</td>\n",
       "      <td>16961</td>\n",
       "      <td>521115245</td>\n",
       "      <td>435511073</td>\n",
       "      <td>85117971</td>\n",
       "      <td>486201</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   ratedFloodZone  countClaims  paidTotalClaim  paidBuildingClaim  \\\n",
       "0              AE       902694     35479231566        29574222634   \n",
       "1               X       398598     13436983455        10383896784   \n",
       "2               B       114822      3826110386         2835364624   \n",
       "3               A       198639      3130755422         2486940415   \n",
       "4               C       163832      2853210440         2119312629   \n",
       "5             A04        51806      2185871995         1800832671   \n",
       "6             A03        41872      1650992713         1337863894   \n",
       "7              VE        46418      1604044125         1427115498   \n",
       "8             A01        51362      1526019156         1188933368   \n",
       "9             A08        45657      1455607732         1185003260   \n",
       "10            A05        48604      1275294814         1059529435   \n",
       "11            A10        41151      1272548151          993009398   \n",
       "12            A07        43624      1150411323          939607938   \n",
       "13            A06        47085      1045591886          845127018   \n",
       "14            A02        31666       966418024          764371792   \n",
       "15            A11        30093       835448574          667150308   \n",
       "16            A09        26680       793640291          608541563   \n",
       "17            A14        24294       702934354          539529371   \n",
       "18           None       139025       696659870          491536385   \n",
       "19             AH        16961       521115245          435511073   \n",
       "\n",
       "    paidContentsClaim    paidICC  \n",
       "0          5389301443  515707490  \n",
       "1          3024472374   28614297  \n",
       "2           971908875   18836887  \n",
       "3           606541394   37273612  \n",
       "4           723803760   10094051  \n",
       "5           364399597   20639726  \n",
       "6           284704083   28424737  \n",
       "7           158958851   17969776  \n",
       "8           320999694   16086094  \n",
       "9           237429765   33174708  \n",
       "10          186761764   29003615  \n",
       "11          260656901   18881852  \n",
       "12          183942076   26861309  \n",
       "13          171603401   28861467  \n",
       "14          190319082   11727150  \n",
       "15          154405220   13893046  \n",
       "16          167178365   17920363  \n",
       "17          153188379   10216604  \n",
       "18          205049316      74169  \n",
       "19           85117971     486201  "
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "con.sql(\"\"\"\n",
    "    SELECT\n",
    "        ratedFloodZone as ratedFloodZone,\n",
    "        COUNT(id) AS countClaims,\n",
    "        ROUND(\n",
    "            SUM(amountPaidOnBuildingClaim)\n",
    "            + SUM(amountPaidOnContentsClaim)\n",
    "            + SUM(amountPaidOnIncreasedCostOfComplianceClaim), 0)::BIGINT AS paidTotalClaim,  \n",
    "        ROUND(SUM(amountPaidOnBuildingClaim), 0)::BIGINT AS paidBuildingClaim,\n",
    "        ROUND(SUM(amountPaidOnContentsClaim), 0)::BIGINT AS paidContentsClaim, \n",
    "        ROUND(SUM(amountPaidOnIncreasedCostOfComplianceClaim), 0)::BIGINT AS paidICC\n",
    "    FROM\n",
    "        claims\n",
    "    GROUP BY ratedFloodZone\n",
    "    ORDER BY\n",
    "        paidTotalClaim DESC\n",
    "    LIMIT 20\n",
    "\"\"\").df()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2e5ffd65",
   "metadata": {},
   "source": [
    "## Total Paid Claims by Post-FIRM Construction Indicator\n",
    "From the Data Dictionary:\n",
    ">Indicates whether construction was started before or after publication of the FIRM. Yes is indicated with either a 'true' or '1'. No is indicated with either a 'false' or '0'. For insurance rating purposes, buildings for which the start of construction or substantial improvement was after December 31, 1974, or on or after the effective date of the initial FIRM for the community, whichever is later, are considered Post-FIRM construction."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "ebfb6c79",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>postFIRMConstructionIndicator</th>\n",
       "      <th>countClaims</th>\n",
       "      <th>paidTotalClaim</th>\n",
       "      <th>paidBuildingClaim</th>\n",
       "      <th>paidContentsClaim</th>\n",
       "      <th>paidICC</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>1961171</td>\n",
       "      <td>53017758311</td>\n",
       "      <td>43012194948</td>\n",
       "      <td>9378595126</td>\n",
       "      <td>626968236</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>710315</td>\n",
       "      <td>27620594292</td>\n",
       "      <td>22037398660</td>\n",
       "      <td>5261726761</td>\n",
       "      <td>321468871</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   postFIRMConstructionIndicator  countClaims  paidTotalClaim  \\\n",
       "0                              0      1961171     53017758311   \n",
       "1                              1       710315     27620594292   \n",
       "\n",
       "   paidBuildingClaim  paidContentsClaim    paidICC  \n",
       "0        43012194948         9378595126  626968236  \n",
       "1        22037398660         5261726761  321468871  "
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "con.sql(\"\"\"\n",
    "    SELECT\n",
    "        postFIRMConstructionIndicator,\n",
    "        COUNT(id) AS countClaims,\n",
    "        ROUND(\n",
    "            SUM(amountPaidOnBuildingClaim)\n",
    "            + SUM(amountPaidOnContentsClaim)\n",
    "            + SUM(amountPaidOnIncreasedCostOfComplianceClaim), 0)::BIGINT AS paidTotalClaim,  \n",
    "        ROUND(SUM(amountPaidOnBuildingClaim), 0)::BIGINT AS paidBuildingClaim,\n",
    "        ROUND(SUM(amountPaidOnContentsClaim), 0)::BIGINT AS paidContentsClaim, \n",
    "        ROUND(SUM(amountPaidOnIncreasedCostOfComplianceClaim), 0)::BIGINT AS paidICC\n",
    "    FROM\n",
    "        claims\n",
    "    GROUP BY\n",
    "        postFIRMConstructionIndicator\n",
    "    ORDER BY\n",
    "        paidTotalClaim DESC\n",
    "\"\"\").df()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a03cc10a",
   "metadata": {},
   "source": [
    "## Total Paid Claims by Basement Enclosure Crawlspace Types\n",
    "From the Data Dictionary:\n",
    ">Basement is defined for purposes of the NFIP as any level or story which has its floor subgrade on all sides. Basement structure values are as follows: 0 - None; 1 - Finished Basement/Enclosure; 2 - Unfinished Basement/Enclosure; 3 - Crawlspace; 4 - Subgrade Crawlspace"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "4cbe341e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>basementEnclosureCrawlspaceType</th>\n",
       "      <th>countClaims</th>\n",
       "      <th>paidTotalClaim</th>\n",
       "      <th>paidBuildingClaim</th>\n",
       "      <th>paidContentsClaim</th>\n",
       "      <th>paidICC</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NaN</td>\n",
       "      <td>1867992</td>\n",
       "      <td>55736598435</td>\n",
       "      <td>43974709709</td>\n",
       "      <td>11010313677</td>\n",
       "      <td>751575049</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.0</td>\n",
       "      <td>258080</td>\n",
       "      <td>14118916150</td>\n",
       "      <td>11621906529</td>\n",
       "      <td>2431677559</td>\n",
       "      <td>65332063</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2.0</td>\n",
       "      <td>283047</td>\n",
       "      <td>6023112182</td>\n",
       "      <td>5309665486</td>\n",
       "      <td>621057933</td>\n",
       "      <td>92388762</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1.0</td>\n",
       "      <td>218434</td>\n",
       "      <td>3411584798</td>\n",
       "      <td>2953210350</td>\n",
       "      <td>440219829</td>\n",
       "      <td>18154620</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4.0</td>\n",
       "      <td>43933</td>\n",
       "      <td>1348141039</td>\n",
       "      <td>1190101534</td>\n",
       "      <td>137052891</td>\n",
       "      <td>20986614</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   basementEnclosureCrawlspaceType  countClaims  paidTotalClaim  \\\n",
       "0                              NaN      1867992     55736598435   \n",
       "1                              0.0       258080     14118916150   \n",
       "2                              2.0       283047      6023112182   \n",
       "3                              1.0       218434      3411584798   \n",
       "4                              4.0        43933      1348141039   \n",
       "\n",
       "   paidBuildingClaim  paidContentsClaim    paidICC  \n",
       "0        43974709709        11010313677  751575049  \n",
       "1        11621906529         2431677559   65332063  \n",
       "2         5309665486          621057933   92388762  \n",
       "3         2953210350          440219829   18154620  \n",
       "4         1190101534          137052891   20986614  "
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "con.sql(\"\"\"\n",
    "    SELECT\n",
    "        basementEnclosureCrawlspaceType,\n",
    "        COUNT(id) AS countClaims,\n",
    "        ROUND(\n",
    "            SUM(amountPaidOnBuildingClaim)\n",
    "            + SUM(amountPaidOnContentsClaim)\n",
    "            + SUM(amountPaidOnIncreasedCostOfComplianceClaim), 0)::BIGINT AS paidTotalClaim,  \n",
    "        ROUND(SUM(amountPaidOnBuildingClaim), 0)::BIGINT AS paidBuildingClaim,\n",
    "        ROUND(SUM(amountPaidOnContentsClaim), 0)::BIGINT AS paidContentsClaim, \n",
    "        ROUND(SUM(amountPaidOnIncreasedCostOfComplianceClaim), 0)::BIGINT AS paidICC\n",
    "    FROM\n",
    "        claims\n",
    "    GROUP BY\n",
    "        basementEnclosureCrawlspaceType\n",
    "    ORDER BY\n",
    "        paidTotalClaim DESC\n",
    "    LIMIT 20\n",
    "\"\"\").df()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "25e7cbb9",
   "metadata": {},
   "source": [
    "## Total Paid Claims by Elevated Building Indicator\n",
    "From the Data Dictionary:\n",
    ">Indicates whether a building meets the NFIP definition of an elevated building. Yes is indicated with either a 'true' or '1'. No is indicated with either a 'false' or '0'. An elevated building is a no-basement building that was constructed so as to meet the following criteria: 1. The top of the elevated floor (all A zones) or the bottom of the lowest horizontal structural member of the lowest floor (all V zones) is above ground level. 2. The building is adequately anchored. 3. The method of elevation is pilings, columns (posts and piers), shear walls (not in V zones), or solid foundation perimeter walls (not in V zones)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "e0a298b3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>elevatedBuildingIndicator</th>\n",
       "      <th>countClaims</th>\n",
       "      <th>paidTotalClaim</th>\n",
       "      <th>paidBuildingClaim</th>\n",
       "      <th>paidContentsClaim</th>\n",
       "      <th>paidICC</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>2187042</td>\n",
       "      <td>66083874242</td>\n",
       "      <td>52806310817</td>\n",
       "      <td>12838681833</td>\n",
       "      <td>438881592</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>484444</td>\n",
       "      <td>14554478361</td>\n",
       "      <td>12243282792</td>\n",
       "      <td>1801640054</td>\n",
       "      <td>509555515</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   elevatedBuildingIndicator  countClaims  paidTotalClaim  paidBuildingClaim  \\\n",
       "0                          0      2187042     66083874242        52806310817   \n",
       "1                          1       484444     14554478361        12243282792   \n",
       "\n",
       "   paidContentsClaim    paidICC  \n",
       "0        12838681833  438881592  \n",
       "1         1801640054  509555515  "
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "con.sql(\"\"\"\n",
    "    SELECT\n",
    "        elevatedBuildingIndicator,\n",
    "        COUNT(id) AS countClaims,\n",
    "        ROUND(\n",
    "            SUM(amountPaidOnBuildingClaim)\n",
    "            + SUM(amountPaidOnContentsClaim)\n",
    "            + SUM(amountPaidOnIncreasedCostOfComplianceClaim), 0)::BIGINT AS paidTotalClaim,  \n",
    "        ROUND(SUM(amountPaidOnBuildingClaim), 0)::BIGINT AS paidBuildingClaim,\n",
    "        ROUND(SUM(amountPaidOnContentsClaim), 0)::BIGINT AS paidContentsClaim, \n",
    "        ROUND(SUM(amountPaidOnIncreasedCostOfComplianceClaim), 0)::BIGINT AS paidICC\n",
    "    FROM\n",
    "        claims\n",
    "    GROUP BY\n",
    "        elevatedBuildingIndicator\n",
    "    ORDER BY\n",
    "        paidTotalClaim DESC\n",
    "\"\"\").df()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "41bad347",
   "metadata": {},
   "source": [
    "## Total Paid Claims by Occupancy Types\n",
    "From the Data Dictionary:\n",
    ">Code indicating the use and occupancy type of the insured structure. Note, 2-digit codes are for Risk Rating 2.0 policies. 1=single family residence; 2 = 2 to 4 unit residential building; 3 = residential building with more than 4 units; 4 = Non-residential building; 6 = Non Residential - Business; 11 = Single-family residential building with the exception of a mobile home or a single residential unit within a multi unit building; 12 = A residential non-condo building with 2, 3, or 4 units seeking insurance on all units; 13 = A residential non-condo building with 5 or more units seeking insurance on all units; 14 = Residential mobile/manufactured home; 15 = Residential condo association seeking coverage on a building with one or more units; 16 = Single residential unit within a multi-unit building; 17 = Non-residential mobile/manufactured home; 18 = A non-residential building; 19 = a non-residential unit within a multi-unit building"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "78eead45",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>occupancyType</th>\n",
       "      <th>countClaims</th>\n",
       "      <th>paidTotalClaim</th>\n",
       "      <th>paidBuildingClaim</th>\n",
       "      <th>paidContentsClaim</th>\n",
       "      <th>paidICC</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2085505</td>\n",
       "      <td>58074889678</td>\n",
       "      <td>46438196346</td>\n",
       "      <td>10745221948</td>\n",
       "      <td>891471384</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>4.0</td>\n",
       "      <td>174509</td>\n",
       "      <td>6399512277</td>\n",
       "      <td>4292656524</td>\n",
       "      <td>2096028112</td>\n",
       "      <td>10827641</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3.0</td>\n",
       "      <td>93350</td>\n",
       "      <td>4398976557</td>\n",
       "      <td>4253333229</td>\n",
       "      <td>138336377</td>\n",
       "      <td>7306952</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2.0</td>\n",
       "      <td>148432</td>\n",
       "      <td>3733231026</td>\n",
       "      <td>3410379315</td>\n",
       "      <td>287688446</td>\n",
       "      <td>35163265</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6.0</td>\n",
       "      <td>44282</td>\n",
       "      <td>3360242402</td>\n",
       "      <td>2512042450</td>\n",
       "      <td>846579743</td>\n",
       "      <td>1620209</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>11.0</td>\n",
       "      <td>94093</td>\n",
       "      <td>2822556575</td>\n",
       "      <td>2447270648</td>\n",
       "      <td>373666665</td>\n",
       "      <td>1619262</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>15.0</td>\n",
       "      <td>8194</td>\n",
       "      <td>946313442</td>\n",
       "      <td>941970967</td>\n",
       "      <td>4252475</td>\n",
       "      <td>90000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>18.0</td>\n",
       "      <td>8353</td>\n",
       "      <td>504682743</td>\n",
       "      <td>413963078</td>\n",
       "      <td>90689665</td>\n",
       "      <td>30000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>12.0</td>\n",
       "      <td>4594</td>\n",
       "      <td>164787070</td>\n",
       "      <td>157682468</td>\n",
       "      <td>7040755</td>\n",
       "      <td>63847</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>13.0</td>\n",
       "      <td>1658</td>\n",
       "      <td>115997430</td>\n",
       "      <td>113511947</td>\n",
       "      <td>2485483</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>14.0</td>\n",
       "      <td>3017</td>\n",
       "      <td>59578163</td>\n",
       "      <td>50611706</td>\n",
       "      <td>8744034</td>\n",
       "      <td>222423</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>16.0</td>\n",
       "      <td>4789</td>\n",
       "      <td>50154126</td>\n",
       "      <td>15130906</td>\n",
       "      <td>35001095</td>\n",
       "      <td>22124</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>NaN</td>\n",
       "      <td>572</td>\n",
       "      <td>3516572</td>\n",
       "      <td>2450945</td>\n",
       "      <td>1065627</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>19.0</td>\n",
       "      <td>115</td>\n",
       "      <td>3509060</td>\n",
       "      <td>0</td>\n",
       "      <td>3509060</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>17.0</td>\n",
       "      <td>23</td>\n",
       "      <td>405481</td>\n",
       "      <td>393081</td>\n",
       "      <td>12400</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    occupancyType  countClaims  paidTotalClaim  paidBuildingClaim  \\\n",
       "0             1.0      2085505     58074889678        46438196346   \n",
       "1             4.0       174509      6399512277         4292656524   \n",
       "2             3.0        93350      4398976557         4253333229   \n",
       "3             2.0       148432      3733231026         3410379315   \n",
       "4             6.0        44282      3360242402         2512042450   \n",
       "5            11.0        94093      2822556575         2447270648   \n",
       "6            15.0         8194       946313442          941970967   \n",
       "7            18.0         8353       504682743          413963078   \n",
       "8            12.0         4594       164787070          157682468   \n",
       "9            13.0         1658       115997430          113511947   \n",
       "10           14.0         3017        59578163           50611706   \n",
       "11           16.0         4789        50154126           15130906   \n",
       "12            NaN          572         3516572            2450945   \n",
       "13           19.0          115         3509060                  0   \n",
       "14           17.0           23          405481             393081   \n",
       "\n",
       "    paidContentsClaim    paidICC  \n",
       "0         10745221948  891471384  \n",
       "1          2096028112   10827641  \n",
       "2           138336377    7306952  \n",
       "3           287688446   35163265  \n",
       "4           846579743    1620209  \n",
       "5           373666665    1619262  \n",
       "6             4252475      90000  \n",
       "7            90689665      30000  \n",
       "8             7040755      63847  \n",
       "9             2485483          0  \n",
       "10            8744034     222423  \n",
       "11           35001095      22124  \n",
       "12            1065627          0  \n",
       "13            3509060          0  \n",
       "14              12400          0  "
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "con.sql(\"\"\"\n",
    "    SELECT\n",
    "        occupancyType,\n",
    "        COUNT(id) AS countClaims,\n",
    "        ROUND(\n",
    "            SUM(amountPaidOnBuildingClaim)\n",
    "            + SUM(amountPaidOnContentsClaim)\n",
    "            + SUM(amountPaidOnIncreasedCostOfComplianceClaim), 0)::BIGINT AS paidTotalClaim,  \n",
    "        ROUND(SUM(amountPaidOnBuildingClaim), 0)::BIGINT AS paidBuildingClaim,\n",
    "        ROUND(SUM(amountPaidOnContentsClaim), 0)::BIGINT AS paidContentsClaim, \n",
    "        ROUND(SUM(amountPaidOnIncreasedCostOfComplianceClaim), 0)::BIGINT AS paidICC\n",
    "    FROM\n",
    "        claims\n",
    "    GROUP BY\n",
    "        occupancyType\n",
    "    ORDER BY\n",
    "        paidTotalClaim DESC\n",
    "    LIMIT 20\n",
    "\"\"\").df()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "261bbc9b",
   "metadata": {},
   "source": [
    "# Additional Analysis"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f02c2a66",
   "metadata": {},
   "source": [
    "## Total Paid Claims by Cause of Damage Code\n",
    "From the Data Dictionary:\n",
    ">Indicates the method by which the insured's property and contents were damaged. Legal values (value : description): 0 : Other causes; 1 : Tidal water overflow; 2 : Stream, river, or lake overflow; 3 : Alluvial fan overflow; 4 : Accumulation of rainfall or snowmelt; 7 : Erosion-demolition; 8 : Erosion-removal; 9 : Earth movement, landslide, land subsidence, sinkholes, etc.; A : Closed basin lake; B : Expedited claim handling process without site inspection; C : Expedited claim handling process follow-up site inspection; D : Expedited claim handling process by Adjusting Process Pilot Program (Remote Adjustment); NOTE: Due to certain provisions of the Upton Jones Amendment to the National Flood Insurance Act, cause of loss codes 7 and 8 may be used only if the date of loss is prior to September 23, 1995. More than one cause of loss code can be selected. For example, you may select 2 (stream, river, or lake overflow) and D (remote adjustment) on a single claim, or any combination of letters/numbers as appropriate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "id": "eb5d22c2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>causeOfDamage</th>\n",
       "      <th>countClaims</th>\n",
       "      <th>paidTotalClaim</th>\n",
       "      <th>paidBuildingClaim</th>\n",
       "      <th>paidContentsClaim</th>\n",
       "      <th>paidICC</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>4</td>\n",
       "      <td>1165067</td>\n",
       "      <td>3.229711e+10</td>\n",
       "      <td>2.592098e+10</td>\n",
       "      <td>6.094557e+09</td>\n",
       "      <td>281581284.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>652400</td>\n",
       "      <td>2.115363e+10</td>\n",
       "      <td>1.734079e+10</td>\n",
       "      <td>3.445278e+09</td>\n",
       "      <td>367564742.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>535954</td>\n",
       "      <td>1.849811e+10</td>\n",
       "      <td>1.479851e+10</td>\n",
       "      <td>3.483557e+09</td>\n",
       "      <td>216043929.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>219652</td>\n",
       "      <td>6.152451e+09</td>\n",
       "      <td>4.980477e+09</td>\n",
       "      <td>1.103245e+09</td>\n",
       "      <td>68728023.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>B</td>\n",
       "      <td>15809</td>\n",
       "      <td>2.101842e+09</td>\n",
       "      <td>1.650532e+09</td>\n",
       "      <td>4.400003e+08</td>\n",
       "      <td>11309655.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>3</td>\n",
       "      <td>12895</td>\n",
       "      <td>2.366988e+08</td>\n",
       "      <td>1.849697e+08</td>\n",
       "      <td>4.949010e+07</td>\n",
       "      <td>2239014.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>None</td>\n",
       "      <td>60282</td>\n",
       "      <td>5.607188e+07</td>\n",
       "      <td>5.228969e+07</td>\n",
       "      <td>3.710770e+06</td>\n",
       "      <td>71424.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>C</td>\n",
       "      <td>430</td>\n",
       "      <td>4.599313e+07</td>\n",
       "      <td>3.679560e+07</td>\n",
       "      <td>9.158730e+06</td>\n",
       "      <td>38800.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>A</td>\n",
       "      <td>684</td>\n",
       "      <td>4.026256e+07</td>\n",
       "      <td>3.331573e+07</td>\n",
       "      <td>6.465500e+06</td>\n",
       "      <td>481325.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>7</td>\n",
       "      <td>602</td>\n",
       "      <td>2.275529e+07</td>\n",
       "      <td>2.233714e+07</td>\n",
       "      <td>3.205870e+05</td>\n",
       "      <td>97560.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>D</td>\n",
       "      <td>5531</td>\n",
       "      <td>1.764484e+07</td>\n",
       "      <td>1.472252e+07</td>\n",
       "      <td>2.794027e+06</td>\n",
       "      <td>128300.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>9</td>\n",
       "      <td>1773</td>\n",
       "      <td>1.196494e+07</td>\n",
       "      <td>1.009760e+07</td>\n",
       "      <td>1.714284e+06</td>\n",
       "      <td>153050.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>8</td>\n",
       "      <td>382</td>\n",
       "      <td>3.765783e+06</td>\n",
       "      <td>3.735555e+06</td>\n",
       "      <td>3.022800e+04</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>5</td>\n",
       "      <td>17</td>\n",
       "      <td>4.662500e+04</td>\n",
       "      <td>4.662500e+04</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>6</td>\n",
       "      <td>3</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>@</td>\n",
       "      <td>2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>N</td>\n",
       "      <td>2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>Z</td>\n",
       "      <td>1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   causeOfDamage  countClaims  paidTotalClaim  paidBuildingClaim  \\\n",
       "0              4      1165067    3.229711e+10       2.592098e+10   \n",
       "1              1       652400    2.115363e+10       1.734079e+10   \n",
       "2              2       535954    1.849811e+10       1.479851e+10   \n",
       "3              0       219652    6.152451e+09       4.980477e+09   \n",
       "4              B        15809    2.101842e+09       1.650532e+09   \n",
       "5              3        12895    2.366988e+08       1.849697e+08   \n",
       "6           None        60282    5.607188e+07       5.228969e+07   \n",
       "7              C          430    4.599313e+07       3.679560e+07   \n",
       "8              A          684    4.026256e+07       3.331573e+07   \n",
       "9              7          602    2.275529e+07       2.233714e+07   \n",
       "10             D         5531    1.764484e+07       1.472252e+07   \n",
       "11             9         1773    1.196494e+07       1.009760e+07   \n",
       "12             8          382    3.765783e+06       3.735555e+06   \n",
       "13             5           17    4.662500e+04       4.662500e+04   \n",
       "14             6            3             NaN                NaN   \n",
       "15             @            2             NaN                NaN   \n",
       "16             N            2             NaN                NaN   \n",
       "17             Z            1             NaN                NaN   \n",
       "\n",
       "    paidContentsClaim      paidICC  \n",
       "0        6.094557e+09  281581284.0  \n",
       "1        3.445278e+09  367564742.0  \n",
       "2        3.483557e+09  216043929.0  \n",
       "3        1.103245e+09   68728023.0  \n",
       "4        4.400003e+08   11309655.0  \n",
       "5        4.949010e+07    2239014.0  \n",
       "6        3.710770e+06      71424.0  \n",
       "7        9.158730e+06      38800.0  \n",
       "8        6.465500e+06     481325.0  \n",
       "9        3.205870e+05      97560.0  \n",
       "10       2.794027e+06     128300.0  \n",
       "11       1.714284e+06     153050.0  \n",
       "12       3.022800e+04          0.0  \n",
       "13       0.000000e+00          0.0  \n",
       "14                NaN          NaN  \n",
       "15                NaN          NaN  \n",
       "16                NaN          NaN  \n",
       "17                NaN          NaN  "
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "con.sql(\"\"\"\n",
    "    SELECT\n",
    "        causeOfDamage,\n",
    "        COUNT(id) AS countClaims,\n",
    "        ROUND(\n",
    "            SUM(amountPaidOnBuildingClaim)\n",
    "            + SUM(amountPaidOnContentsClaim)\n",
    "            + SUM(amountPaidOnIncreasedCostOfComplianceClaim), 0)::BIGINT AS paidTotalClaim,  \n",
    "        ROUND(SUM(amountPaidOnBuildingClaim), 0)::BIGINT AS paidBuildingClaim,\n",
    "        ROUND(SUM(amountPaidOnContentsClaim), 0)::BIGINT AS paidContentsClaim, \n",
    "        ROUND(SUM(amountPaidOnIncreasedCostOfComplianceClaim), 0)::BIGINT AS paidICC\n",
    "    FROM\n",
    "        claims\n",
    "    GROUP BY\n",
    "        causeOfDamage\n",
    "    ORDER BY\n",
    "        paidTotalClaim DESC\n",
    "    LIMIT 20\n",
    "\"\"\").df()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d98926b9",
   "metadata": {},
   "source": [
    "## Total Paid Claims by Flood Characteristics Indicator\n",
    "From the Data Dictionary:\n",
    ">Indicates characteristics of the flood waters. Legal values (value : description): 1 : Velocity Flow; 2 : Low-Velocity Flow or Ponding; 3 : Wave Action; 4 : Mudflow; 5 : Erosion;"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "83d6aa78",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>floodCharacteristicsIndicator</th>\n",
       "      <th>countClaims</th>\n",
       "      <th>paidTotalClaim</th>\n",
       "      <th>paidBuildingClaim</th>\n",
       "      <th>paidContentsClaim</th>\n",
       "      <th>paidICC</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NaN</td>\n",
       "      <td>2631633</td>\n",
       "      <td>80040375269</td>\n",
       "      <td>64608337763</td>\n",
       "      <td>14488023555</td>\n",
       "      <td>944013951</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2.0</td>\n",
       "      <td>29123</td>\n",
       "      <td>392139926</td>\n",
       "      <td>284532994</td>\n",
       "      <td>105348196</td>\n",
       "      <td>2258735</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1.0</td>\n",
       "      <td>8026</td>\n",
       "      <td>169487124</td>\n",
       "      <td>126607466</td>\n",
       "      <td>41429777</td>\n",
       "      <td>1449881</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3.0</td>\n",
       "      <td>1818</td>\n",
       "      <td>28177518</td>\n",
       "      <td>23333657</td>\n",
       "      <td>4129321</td>\n",
       "      <td>714539</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4.0</td>\n",
       "      <td>649</td>\n",
       "      <td>6794025</td>\n",
       "      <td>5504227</td>\n",
       "      <td>1289798</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>5.0</td>\n",
       "      <td>237</td>\n",
       "      <td>1378741</td>\n",
       "      <td>1277501</td>\n",
       "      <td>101240</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   floodCharacteristicsIndicator  countClaims  paidTotalClaim  \\\n",
       "0                            NaN      2631633     80040375269   \n",
       "1                            2.0        29123       392139926   \n",
       "2                            1.0         8026       169487124   \n",
       "3                            3.0         1818        28177518   \n",
       "4                            4.0          649         6794025   \n",
       "5                            5.0          237         1378741   \n",
       "\n",
       "   paidBuildingClaim  paidContentsClaim    paidICC  \n",
       "0        64608337763        14488023555  944013951  \n",
       "1          284532994          105348196    2258735  \n",
       "2          126607466           41429777    1449881  \n",
       "3           23333657            4129321     714539  \n",
       "4            5504227            1289798          0  \n",
       "5            1277501             101240          0  "
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "con.sql(\"\"\"\n",
    "    SELECT\n",
    "        floodCharacteristicsIndicator,\n",
    "        COUNT(id) AS countClaims,\n",
    "        ROUND(\n",
    "            SUM(amountPaidOnBuildingClaim)\n",
    "            + SUM(amountPaidOnContentsClaim)\n",
    "            + SUM(amountPaidOnIncreasedCostOfComplianceClaim), 0)::BIGINT AS paidTotalClaim,  \n",
    "        ROUND(SUM(amountPaidOnBuildingClaim), 0)::BIGINT AS paidBuildingClaim,\n",
    "        ROUND(SUM(amountPaidOnContentsClaim), 0)::BIGINT AS paidContentsClaim, \n",
    "        ROUND(SUM(amountPaidOnIncreasedCostOfComplianceClaim), 0)::BIGINT AS paidICC\n",
    "    FROM\n",
    "        claims\n",
    "    GROUP BY\n",
    "        floodCharacteristicsIndicator\n",
    "    ORDER BY\n",
    "        paidTotalClaim DESC\n",
    "    LIMIT 20\n",
    "\"\"\").df()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9b06b452",
   "metadata": {},
   "source": [
    "## Total Paid Claims by Floodproofed Indicator\n",
    "From the Data Dictionary:\n",
    ">Indicates if the insured structure is floodproofed. Yes is indicated with either a 'true' or '1'. No is indicated with either a 'false' or '0'. NOTE: Floodproofing is any combination of structural and non-structural additions, changes, or adjustments to structures which reduce or eliminate flood damage to a property. Floodproofing may be an alternative to elevating a building to or above the BFE; however, the NFIP requires a Floodproofing Certificate prior to considering floodproofing mitigation measures in rating a structure"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "cd8e0e2d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>floodproofedIndicator</th>\n",
       "      <th>countClaims</th>\n",
       "      <th>paidTotalClaim</th>\n",
       "      <th>paidBuildingClaim</th>\n",
       "      <th>paidContentsClaim</th>\n",
       "      <th>paidICC</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>2671109</td>\n",
       "      <td>80619514527</td>\n",
       "      <td>65035738829</td>\n",
       "      <td>14635368591</td>\n",
       "      <td>948407107</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>377</td>\n",
       "      <td>18838076</td>\n",
       "      <td>13854779</td>\n",
       "      <td>4953296</td>\n",
       "      <td>30000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   floodproofedIndicator  countClaims  paidTotalClaim  paidBuildingClaim  \\\n",
       "0                      0      2671109     80619514527        65035738829   \n",
       "1                      1          377        18838076           13854779   \n",
       "\n",
       "   paidContentsClaim    paidICC  \n",
       "0        14635368591  948407107  \n",
       "1            4953296      30000  "
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "con.sql(\"\"\"\n",
    "    SELECT\n",
    "        floodproofedIndicator,\n",
    "        COUNT(id) AS countClaims,\n",
    "        ROUND(\n",
    "            SUM(amountPaidOnBuildingClaim)\n",
    "            + SUM(amountPaidOnContentsClaim)\n",
    "            + SUM(amountPaidOnIncreasedCostOfComplianceClaim), 0)::BIGINT AS paidTotalClaim,  \n",
    "        ROUND(SUM(amountPaidOnBuildingClaim), 0)::BIGINT AS paidBuildingClaim,\n",
    "        ROUND(SUM(amountPaidOnContentsClaim), 0)::BIGINT AS paidContentsClaim, \n",
    "        ROUND(SUM(amountPaidOnIncreasedCostOfComplianceClaim), 0)::BIGINT AS paidICC\n",
    "    FROM\n",
    "        claims\n",
    "    GROUP BY\n",
    "        floodproofedIndicator\n",
    "    ORDER BY\n",
    "        paidTotalClaim DESC\n",
    "\"\"\").df()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "723c44a8",
   "metadata": {},
   "source": [
    "## Total Paid Claims by Building Description Code\n",
    "From the Data Dictionary:\n",
    ">Indicates the description of the use of the insured building. Legal values (value : description): 01 : Main House; 02 : Detached Guest House; 03 : Detached Garage; 04 : Agricultural Building; 05 : Warehouse; 06 : Pool House, Clubhouse, Recreation Building; 07 : Tool/Storage Shed; 08 : Other; 09 : Barn; 10 : Apartment Building; 11 : Apartment - Unit; 12 : Cooperative Building; 13 : Cooperative - Unit; 14 : Commercial Building; 15 : Condominium (Entire Building); 16 : Condominium - Unit; 17 : House of Worship; 18 : Manufactured (Mobile) Home; 19 : Travel Trailer; 20 : Townhouse/Rowhouse;"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "d62db375",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>buildingDescriptionCode</th>\n",
       "      <th>countClaims</th>\n",
       "      <th>paidTotalClaim</th>\n",
       "      <th>paidBuildingClaim</th>\n",
       "      <th>paidContentsClaim</th>\n",
       "      <th>paidICC</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.0</td>\n",
       "      <td>905145</td>\n",
       "      <td>39094112092</td>\n",
       "      <td>32540928483</td>\n",
       "      <td>6132748786</td>\n",
       "      <td>420434822</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NaN</td>\n",
       "      <td>1717297</td>\n",
       "      <td>38230967250</td>\n",
       "      <td>29855329355</td>\n",
       "      <td>7851193461</td>\n",
       "      <td>524444435</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>8.0</td>\n",
       "      <td>37859</td>\n",
       "      <td>2754729151</td>\n",
       "      <td>2195108224</td>\n",
       "      <td>556837496</td>\n",
       "      <td>2783431</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>5.0</td>\n",
       "      <td>2367</td>\n",
       "      <td>199062000</td>\n",
       "      <td>130086791</td>\n",
       "      <td>68975209</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10.0</td>\n",
       "      <td>2330</td>\n",
       "      <td>188592994</td>\n",
       "      <td>187206254</td>\n",
       "      <td>1386741</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2.0</td>\n",
       "      <td>2258</td>\n",
       "      <td>58607574</td>\n",
       "      <td>49616808</td>\n",
       "      <td>8324777</td>\n",
       "      <td>665989</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>6.0</td>\n",
       "      <td>1324</td>\n",
       "      <td>56060632</td>\n",
       "      <td>47843930</td>\n",
       "      <td>8162650</td>\n",
       "      <td>54053</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>7.0</td>\n",
       "      <td>957</td>\n",
       "      <td>14025659</td>\n",
       "      <td>9901318</td>\n",
       "      <td>4095639</td>\n",
       "      <td>28703</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>3.0</td>\n",
       "      <td>760</td>\n",
       "      <td>13168321</td>\n",
       "      <td>11508338</td>\n",
       "      <td>1647109</td>\n",
       "      <td>12875</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>12.0</td>\n",
       "      <td>137</td>\n",
       "      <td>8563373</td>\n",
       "      <td>7334598</td>\n",
       "      <td>1215975</td>\n",
       "      <td>12800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>11.0</td>\n",
       "      <td>340</td>\n",
       "      <td>8139904</td>\n",
       "      <td>5488946</td>\n",
       "      <td>2650958</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>4.0</td>\n",
       "      <td>410</td>\n",
       "      <td>7078437</td>\n",
       "      <td>5290266</td>\n",
       "      <td>1788171</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>9.0</td>\n",
       "      <td>174</td>\n",
       "      <td>2944012</td>\n",
       "      <td>2370880</td>\n",
       "      <td>573133</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>13.0</td>\n",
       "      <td>112</td>\n",
       "      <td>1540077</td>\n",
       "      <td>913987</td>\n",
       "      <td>626090</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>15.0</td>\n",
       "      <td>4</td>\n",
       "      <td>396809</td>\n",
       "      <td>323146</td>\n",
       "      <td>73663</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>14.0</td>\n",
       "      <td>8</td>\n",
       "      <td>257190</td>\n",
       "      <td>255159</td>\n",
       "      <td>2031</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>18.0</td>\n",
       "      <td>1</td>\n",
       "      <td>70000</td>\n",
       "      <td>50000</td>\n",
       "      <td>20000</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>21.0</td>\n",
       "      <td>2</td>\n",
       "      <td>20871</td>\n",
       "      <td>20871</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>16.0</td>\n",
       "      <td>1</td>\n",
       "      <td>16255</td>\n",
       "      <td>16255</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    buildingDescriptionCode  countClaims  paidTotalClaim  paidBuildingClaim  \\\n",
       "0                       1.0       905145     39094112092        32540928483   \n",
       "1                       NaN      1717297     38230967250        29855329355   \n",
       "2                       8.0        37859      2754729151         2195108224   \n",
       "3                       5.0         2367       199062000          130086791   \n",
       "4                      10.0         2330       188592994          187206254   \n",
       "5                       2.0         2258        58607574           49616808   \n",
       "6                       6.0         1324        56060632           47843930   \n",
       "7                       7.0          957        14025659            9901318   \n",
       "8                       3.0          760        13168321           11508338   \n",
       "9                      12.0          137         8563373            7334598   \n",
       "10                     11.0          340         8139904            5488946   \n",
       "11                      4.0          410         7078437            5290266   \n",
       "12                      9.0          174         2944012            2370880   \n",
       "13                     13.0          112         1540077             913987   \n",
       "14                     15.0            4          396809             323146   \n",
       "15                     14.0            8          257190             255159   \n",
       "16                     18.0            1           70000              50000   \n",
       "17                     21.0            2           20871              20871   \n",
       "18                     16.0            1           16255              16255   \n",
       "\n",
       "    paidContentsClaim    paidICC  \n",
       "0          6132748786  420434822  \n",
       "1          7851193461  524444435  \n",
       "2           556837496    2783431  \n",
       "3            68975209          0  \n",
       "4             1386741          0  \n",
       "5             8324777     665989  \n",
       "6             8162650      54053  \n",
       "7             4095639      28703  \n",
       "8             1647109      12875  \n",
       "9             1215975      12800  \n",
       "10            2650958          0  \n",
       "11            1788171          0  \n",
       "12             573133          0  \n",
       "13             626090          0  \n",
       "14              73663          0  \n",
       "15               2031          0  \n",
       "16              20000          0  \n",
       "17                  0          0  \n",
       "18                  0          0  "
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "con.sql(\"\"\"\n",
    "    SELECT\n",
    "        buildingDescriptionCode,\n",
    "        COUNT(id) AS countClaims,\n",
    "        ROUND(\n",
    "            SUM(amountPaidOnBuildingClaim)\n",
    "            + SUM(amountPaidOnContentsClaim)\n",
    "            + SUM(amountPaidOnIncreasedCostOfComplianceClaim), 0)::BIGINT AS paidTotalClaim,  \n",
    "        ROUND(SUM(amountPaidOnBuildingClaim), 0)::BIGINT AS paidBuildingClaim,\n",
    "        ROUND(SUM(amountPaidOnContentsClaim), 0)::BIGINT AS paidContentsClaim, \n",
    "        ROUND(SUM(amountPaidOnIncreasedCostOfComplianceClaim), 0)::BIGINT AS paidICC\n",
    "    FROM\n",
    "        claims\n",
    "    GROUP BY\n",
    "        buildingDescriptionCode\n",
    "    ORDER BY\n",
    "        paidTotalClaim DESC\n",
    "    LIMIT 20\n",
    "\"\"\").df()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "af562448",
   "metadata": {},
   "source": [
    "## Total Paid Claims by Rental Property Indicator\n",
    "From the Data Dictionary:\n",
    ">Indicates if the property is a rental property. Yes is indicated with either a 'true' or '1'. No is indicated with either a 'false' or '0'. NOTE: A rental property is a property from which the owner receives payment from the occupant(s), known as tenants, in return for occupying or using the property. Rental properties may be either residential or commercial."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "id": "a26964bc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>rentalPropertyIndicator</th>\n",
       "      <th>countClaims</th>\n",
       "      <th>paidTotalClaim</th>\n",
       "      <th>paidBuildingClaim</th>\n",
       "      <th>paidContentsClaim</th>\n",
       "      <th>paidICC</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>2638216</td>\n",
       "      <td>79198225176</td>\n",
       "      <td>63777570301</td>\n",
       "      <td>14474933322</td>\n",
       "      <td>945721553</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>33270</td>\n",
       "      <td>1440127427</td>\n",
       "      <td>1272023307</td>\n",
       "      <td>165388566</td>\n",
       "      <td>2715554</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   rentalPropertyIndicator  countClaims  paidTotalClaim  paidBuildingClaim  \\\n",
       "0                        0      2638216     79198225176        63777570301   \n",
       "1                        1        33270      1440127427         1272023307   \n",
       "\n",
       "   paidContentsClaim    paidICC  \n",
       "0        14474933322  945721553  \n",
       "1          165388566    2715554  "
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "con.sql(\"\"\"\n",
    "    SELECT\n",
    "        rentalPropertyIndicator,\n",
    "        COUNT(id) AS countClaims,\n",
    "        ROUND(\n",
    "            SUM(amountPaidOnBuildingClaim)\n",
    "            + SUM(amountPaidOnContentsClaim)\n",
    "            + SUM(amountPaidOnIncreasedCostOfComplianceClaim), 0)::BIGINT AS paidTotalClaim,  \n",
    "        ROUND(SUM(amountPaidOnBuildingClaim), 0)::BIGINT AS paidBuildingClaim,\n",
    "        ROUND(SUM(amountPaidOnContentsClaim), 0)::BIGINT AS paidContentsClaim, \n",
    "        ROUND(SUM(amountPaidOnIncreasedCostOfComplianceClaim), 0)::BIGINT AS paidICC\n",
    "    FROM\n",
    "        claims\n",
    "    GROUP BY\n",
    "        rentalPropertyIndicator\n",
    "    ORDER BY\n",
    "        paidTotalClaim DESC\n",
    "\"\"\").df()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "aba112a8",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}