WebScraping_PatentAbstracts_CPCcodes.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "faa2414e-72da-45da-bb40-84c6bd987463",
   "metadata": {},
   "source": [
    "# WebScraping Patent Info"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ef27cbf6-95cf-45a5-818b-72e8e0d5322a",
   "metadata": {},
   "source": [
    "This notebook is an example of applying webscraping and API tools to collect patents data from google patent search page.\n",
    "\n",
    "First, start from Google Patent Search Page:\n",
    "+ Set search parameter\n",
    "+ Search\n",
    "+ download csv file from top-right corner\n",
    "+ The csv file should include the following information: patent id, assignee, inventors, date, and patent web page links\n",
    "\n",
    "Using \"renewable energy\" as search keyword to define the topic of patents. Set other searching parameters as follow:\n",
    "\n",
    "|search parameters|setting|description|\n",
    "|----|----|----|\n",
    "|sort by|Relevance||\n",
    "|group by|Classification|set this to True will will group searching results with CPC code.|\n",
    "|deduplicate|family|deduplicate same family patents, uses family as research unit. a patent family usually defines an invention, which could be break into many patents.|\n",
    "|date before|'filing:20240630'|we will collect all patents that have filing date before 2024.06.30|\n",
    "|date after|no selection||\n",
    "|inventor|no selection||\n",
    "|assignee|no selection||\n",
    "|patent office|'US'|focus on patents in 'US'|\n",
    "|language|'EN'|focus on patents in English language|\n",
    "|status|'Grant'|another choice is 'application'|\n",
    "|type|'patent'|another choice is 'design'|\n",
    "|litigation|no selection||"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e328072f-8c35-4f6b-b4c7-8d65c19c8bc4",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "## Load and Clean Patent List .csv File"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "id": "35aa6683-32a0-43c1-b6cc-45b6405c797c",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "df = pd.read_csv(r\"C:\\Users\\user\\Documents\\RenewableEnergy_patents.csv\", skiprows = 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "id": "82aa7e63-5ea8-4f51-86e0-fa5868b9ab5f",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>title</th>\n",
       "      <th>assignee</th>\n",
       "      <th>inventor/author</th>\n",
       "      <th>priority date</th>\n",
       "      <th>filing/creation date</th>\n",
       "      <th>publication date</th>\n",
       "      <th>grant date</th>\n",
       "      <th>result link</th>\n",
       "      <th>representative figure link</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>US-10340693-B2</td>\n",
       "      <td>Systems and methods for generating energy usin...</td>\n",
       "      <td>Lawrence D. Lansing, JR., Lawrence D. Lansing</td>\n",
       "      <td>Lawrence D. Lansing, JR., Lawrence D. Lansing</td>\n",
       "      <td>2013-03-13</td>\n",
       "      <td>2018-02-27</td>\n",
       "      <td>2019-07-02</td>\n",
       "      <td>2019-07-02</td>\n",
       "      <td>https://patents.google.com/patent/US10340693B2/en</td>\n",
       "      <td>https://patentimages.storage.googleapis.com/4c...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>US-10526056-B1</td>\n",
       "      <td>Generation of electric power using wave motion...</td>\n",
       "      <td>Physician Electronic Network, LLC</td>\n",
       "      <td>A-Hamid Hakki, Edin Dervishalidovic, Belmina H...</td>\n",
       "      <td>2019-04-29</td>\n",
       "      <td>2019-04-29</td>\n",
       "      <td>2020-01-07</td>\n",
       "      <td>2020-01-07</td>\n",
       "      <td>https://patents.google.com/patent/US10526056B1/en</td>\n",
       "      <td>https://patentimages.storage.googleapis.com/6f...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>US-11228182-B2</td>\n",
       "      <td>Converter disabling photovoltaic electrical en...</td>\n",
       "      <td>Ampt, Llc</td>\n",
       "      <td>Robert Porter, Anatoli Ledenev</td>\n",
       "      <td>2007-10-15</td>\n",
       "      <td>2019-06-12</td>\n",
       "      <td>2022-01-18</td>\n",
       "      <td>2022-01-18</td>\n",
       "      <td>https://patents.google.com/patent/US11228182B2/en</td>\n",
       "      <td>https://patentimages.storage.googleapis.com/f5...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>US-11060742-B2</td>\n",
       "      <td>PVT heat pump system capable of achieving day-...</td>\n",
       "      <td>Dalian University Of Technology</td>\n",
       "      <td>Jili ZHANG, Ruobing LIANG, Chao Zhou, Shixiang...</td>\n",
       "      <td>2017-08-03</td>\n",
       "      <td>2017-08-03</td>\n",
       "      <td>2021-07-13</td>\n",
       "      <td>2021-07-13</td>\n",
       "      <td>https://patents.google.com/patent/US11060742B2/en</td>\n",
       "      <td>https://patentimages.storage.googleapis.com/f8...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>US-10808685-B2</td>\n",
       "      <td>Dispatchable combined cycle power plant</td>\n",
       "      <td>William M. Conlon</td>\n",
       "      <td>William M. Conlon</td>\n",
       "      <td>2014-06-04</td>\n",
       "      <td>2018-10-19</td>\n",
       "      <td>2020-10-20</td>\n",
       "      <td>2020-10-20</td>\n",
       "      <td>https://patents.google.com/patent/US10808685B2/en</td>\n",
       "      <td>https://patentimages.storage.googleapis.com/d5...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               id                                              title  \\\n",
       "0  US-10340693-B2  Systems and methods for generating energy usin...   \n",
       "1  US-10526056-B1  Generation of electric power using wave motion...   \n",
       "2  US-11228182-B2  Converter disabling photovoltaic electrical en...   \n",
       "3  US-11060742-B2  PVT heat pump system capable of achieving day-...   \n",
       "4  US-10808685-B2           Dispatchable combined cycle power plant    \n",
       "\n",
       "                                        assignee  \\\n",
       "0  Lawrence D. Lansing, JR., Lawrence D. Lansing   \n",
       "1              Physician Electronic Network, LLC   \n",
       "2                                      Ampt, Llc   \n",
       "3                Dalian University Of Technology   \n",
       "4                              William M. Conlon   \n",
       "\n",
       "                                     inventor/author priority date  \\\n",
       "0      Lawrence D. Lansing, JR., Lawrence D. Lansing    2013-03-13   \n",
       "1  A-Hamid Hakki, Edin Dervishalidovic, Belmina H...    2019-04-29   \n",
       "2                     Robert Porter, Anatoli Ledenev    2007-10-15   \n",
       "3  Jili ZHANG, Ruobing LIANG, Chao Zhou, Shixiang...    2017-08-03   \n",
       "4                                  William M. Conlon    2014-06-04   \n",
       "\n",
       "  filing/creation date publication date  grant date  \\\n",
       "0           2018-02-27       2019-07-02  2019-07-02   \n",
       "1           2019-04-29       2020-01-07  2020-01-07   \n",
       "2           2019-06-12       2022-01-18  2022-01-18   \n",
       "3           2017-08-03       2021-07-13  2021-07-13   \n",
       "4           2018-10-19       2020-10-20  2020-10-20   \n",
       "\n",
       "                                         result link  \\\n",
       "0  https://patents.google.com/patent/US10340693B2/en   \n",
       "1  https://patents.google.com/patent/US10526056B1/en   \n",
       "2  https://patents.google.com/patent/US11228182B2/en   \n",
       "3  https://patents.google.com/patent/US11060742B2/en   \n",
       "4  https://patents.google.com/patent/US10808685B2/en   \n",
       "\n",
       "                          representative figure link  \n",
       "0  https://patentimages.storage.googleapis.com/4c...  \n",
       "1  https://patentimages.storage.googleapis.com/6f...  \n",
       "2  https://patentimages.storage.googleapis.com/f5...  \n",
       "3  https://patentimages.storage.googleapis.com/f8...  \n",
       "4  https://patentimages.storage.googleapis.com/d5...  "
      ]
     },
     "execution_count": 87,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "id": "f980de92-f1c4-4601-815b-2d73a2f3200e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# change column name\n",
    "df.columns = ['id','title','assignee','inventor','priority_date','filing_date','publication_date','grant_date','result_link','representative_figure_link']\n",
    "\n",
    "## cleaning id column\n",
    "# split id column to generate 'country' and 'kind_code' columns\n",
    "df[['id2']] = df[['id']]# copy id column\n",
    "df[['country','id_code','kind_code']] = df['id2'].str.split('-',expand=True)# split id column\n",
    "df = df.drop(['id_code','id2'], axis = 1)# drop id2 and id_code\n",
    "\n",
    "# remove '-' signs from id\n",
    "df['id'] = df['id'].str.replace('-', '')\n",
    "\n",
    "# remove '-' signs from date columns\n",
    "for i in ['priority_date','filing_date','publication_date','grant_date']:\n",
    "    df[i] = df[i].str.replace('-', '')\n",
    "\n",
    "# change inventor column from string to name list\n",
    "inventor_list = df['inventor'].str.split(',').to_frame()\n",
    "df[['inventor']] = inventor_list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "id": "1efc6469-64f2-4670-b675-bc76a933fab1",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df.loc[df['priority_date'].isna() == False,].reset_index()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "id": "4add95b3-b7bf-443d-8d92-6bc41cfc8b4f",
   "metadata": {},
   "outputs": [],
   "source": [
    "df['priority_date'] = df['priority_date'].astype(int).astype(str)\n",
    "df['filing_date'] = df['filing_date'].astype(int).astype(str)\n",
    "df['publication_date'] = df['publication_date'].astype(int).astype(str)\n",
    "df['grant_date'] = df['grant_date'].astype(int).astype(str)\n",
    "\n",
    "from datetime import datetime\n",
    "\n",
    "for date in ['priority_date','filing_date','publication_date','grant_date']:\n",
    "    for i in range(len(df)):\n",
    "        df.loc[i, date] = datetime.strptime(df.loc[i, date], '%Y%m%d').date()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "id": "d55df5cb-5e8f-4f20-85d3-92b00093fa43",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>title</th>\n",
       "      <th>assignee</th>\n",
       "      <th>inventor</th>\n",
       "      <th>priority_date</th>\n",
       "      <th>filing_date</th>\n",
       "      <th>publication_date</th>\n",
       "      <th>grant_date</th>\n",
       "      <th>result_link</th>\n",
       "      <th>representative_figure_link</th>\n",
       "      <th>country</th>\n",
       "      <th>kind_code</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>US10340693B2</td>\n",
       "      <td>Systems and methods for generating energy usin...</td>\n",
       "      <td>Lawrence D. Lansing, JR., Lawrence D. Lansing</td>\n",
       "      <td>[Lawrence D. Lansing,  JR.,  Lawrence D. Lansing]</td>\n",
       "      <td>2013-03-13</td>\n",
       "      <td>2018-02-27</td>\n",
       "      <td>2019-07-02</td>\n",
       "      <td>2019-07-02</td>\n",
       "      <td>https://patents.google.com/patent/US10340693B2/en</td>\n",
       "      <td>https://patentimages.storage.googleapis.com/4c...</td>\n",
       "      <td>US</td>\n",
       "      <td>B2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>US10526056B1</td>\n",
       "      <td>Generation of electric power using wave motion...</td>\n",
       "      <td>Physician Electronic Network, LLC</td>\n",
       "      <td>[A-Hamid Hakki,  Edin Dervishalidovic,  Belmin...</td>\n",
       "      <td>2019-04-29</td>\n",
       "      <td>2019-04-29</td>\n",
       "      <td>2020-01-07</td>\n",
       "      <td>2020-01-07</td>\n",
       "      <td>https://patents.google.com/patent/US10526056B1/en</td>\n",
       "      <td>https://patentimages.storage.googleapis.com/6f...</td>\n",
       "      <td>US</td>\n",
       "      <td>B1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>US11228182B2</td>\n",
       "      <td>Converter disabling photovoltaic electrical en...</td>\n",
       "      <td>Ampt, Llc</td>\n",
       "      <td>[Robert Porter,  Anatoli Ledenev]</td>\n",
       "      <td>2007-10-15</td>\n",
       "      <td>2019-06-12</td>\n",
       "      <td>2022-01-18</td>\n",
       "      <td>2022-01-18</td>\n",
       "      <td>https://patents.google.com/patent/US11228182B2/en</td>\n",
       "      <td>https://patentimages.storage.googleapis.com/f5...</td>\n",
       "      <td>US</td>\n",
       "      <td>B2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>US11060742B2</td>\n",
       "      <td>PVT heat pump system capable of achieving day-...</td>\n",
       "      <td>Dalian University Of Technology</td>\n",
       "      <td>[Jili ZHANG,  Ruobing LIANG,  Chao Zhou,  Shix...</td>\n",
       "      <td>2017-08-03</td>\n",
       "      <td>2017-08-03</td>\n",
       "      <td>2021-07-13</td>\n",
       "      <td>2021-07-13</td>\n",
       "      <td>https://patents.google.com/patent/US11060742B2/en</td>\n",
       "      <td>https://patentimages.storage.googleapis.com/f8...</td>\n",
       "      <td>US</td>\n",
       "      <td>B2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>US10808685B2</td>\n",
       "      <td>Dispatchable combined cycle power plant</td>\n",
       "      <td>William M. Conlon</td>\n",
       "      <td>[William M. Conlon]</td>\n",
       "      <td>2014-06-04</td>\n",
       "      <td>2018-10-19</td>\n",
       "      <td>2020-10-20</td>\n",
       "      <td>2020-10-20</td>\n",
       "      <td>https://patents.google.com/patent/US10808685B2/en</td>\n",
       "      <td>https://patentimages.storage.googleapis.com/d5...</td>\n",
       "      <td>US</td>\n",
       "      <td>B2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             id                                              title  \\\n",
       "0  US10340693B2  Systems and methods for generating energy usin...   \n",
       "1  US10526056B1  Generation of electric power using wave motion...   \n",
       "2  US11228182B2  Converter disabling photovoltaic electrical en...   \n",
       "3  US11060742B2  PVT heat pump system capable of achieving day-...   \n",
       "4  US10808685B2           Dispatchable combined cycle power plant    \n",
       "\n",
       "                                        assignee  \\\n",
       "0  Lawrence D. Lansing, JR., Lawrence D. Lansing   \n",
       "1              Physician Electronic Network, LLC   \n",
       "2                                      Ampt, Llc   \n",
       "3                Dalian University Of Technology   \n",
       "4                              William M. Conlon   \n",
       "\n",
       "                                            inventor priority_date  \\\n",
       "0  [Lawrence D. Lansing,  JR.,  Lawrence D. Lansing]    2013-03-13   \n",
       "1  [A-Hamid Hakki,  Edin Dervishalidovic,  Belmin...    2019-04-29   \n",
       "2                  [Robert Porter,  Anatoli Ledenev]    2007-10-15   \n",
       "3  [Jili ZHANG,  Ruobing LIANG,  Chao Zhou,  Shix...    2017-08-03   \n",
       "4                                [William M. Conlon]    2014-06-04   \n",
       "\n",
       "  filing_date publication_date  grant_date  \\\n",
       "0  2018-02-27       2019-07-02  2019-07-02   \n",
       "1  2019-04-29       2020-01-07  2020-01-07   \n",
       "2  2019-06-12       2022-01-18  2022-01-18   \n",
       "3  2017-08-03       2021-07-13  2021-07-13   \n",
       "4  2018-10-19       2020-10-20  2020-10-20   \n",
       "\n",
       "                                         result_link  \\\n",
       "0  https://patents.google.com/patent/US10340693B2/en   \n",
       "1  https://patents.google.com/patent/US10526056B1/en   \n",
       "2  https://patents.google.com/patent/US11228182B2/en   \n",
       "3  https://patents.google.com/patent/US11060742B2/en   \n",
       "4  https://patents.google.com/patent/US10808685B2/en   \n",
       "\n",
       "                          representative_figure_link country kind_code  \n",
       "0  https://patentimages.storage.googleapis.com/4c...      US        B2  \n",
       "1  https://patentimages.storage.googleapis.com/6f...      US        B1  \n",
       "2  https://patentimages.storage.googleapis.com/f5...      US        B2  \n",
       "3  https://patentimages.storage.googleapis.com/f8...      US        B2  \n",
       "4  https://patentimages.storage.googleapis.com/d5...      US        B2  "
      ]
     },
     "execution_count": 113,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = df.drop('index', axis = 1)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "id": "866cf8be-902a-4eb1-ac9b-866d9474b9b4",
   "metadata": {},
   "outputs": [],
   "source": [
    "# save cleaned patents dateframe\n",
    "df.to_csv(r\"C:\\Users\\user\\Documents\\RenewableEnergy_patents_clean.csv\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2003eed6-a12c-41a4-902f-aac26dc8099b",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "## WebScraping: Patent Abstracts"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "83094ae8-6381-4b86-af6d-6ec5bdc09ba4",
   "metadata": {},
   "outputs": [],
   "source": [
    "# reload cleaned patent list\n",
    "import pandas as pd\n",
    "df = pd.read_csv(r\"C:\\Users\\user\\Documents\\RenewableEnergy_patents_clean.csv\", index_col = 0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "c8aafd3b-77e0-4520-907e-6842f345c280",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>title</th>\n",
       "      <th>assignee</th>\n",
       "      <th>inventor</th>\n",
       "      <th>priority_date</th>\n",
       "      <th>filing_date</th>\n",
       "      <th>publication_date</th>\n",
       "      <th>grant_date</th>\n",
       "      <th>result_link</th>\n",
       "      <th>representative_figure_link</th>\n",
       "      <th>country</th>\n",
       "      <th>kind_code</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>US10340693B2</td>\n",
       "      <td>Systems and methods for generating energy usin...</td>\n",
       "      <td>Lawrence D. Lansing, JR., Lawrence D. Lansing</td>\n",
       "      <td>['Lawrence D. Lansing', ' JR.', ' Lawrence D. ...</td>\n",
       "      <td>20130313.0</td>\n",
       "      <td>20180227</td>\n",
       "      <td>20190702</td>\n",
       "      <td>20190702</td>\n",
       "      <td>https://patents.google.com/patent/US10340693B2/en</td>\n",
       "      <td>https://patentimages.storage.googleapis.com/4c...</td>\n",
       "      <td>US</td>\n",
       "      <td>B2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>US10526056B1</td>\n",
       "      <td>Generation of electric power using wave motion...</td>\n",
       "      <td>Physician Electronic Network, LLC</td>\n",
       "      <td>['A-Hamid Hakki', ' Edin Dervishalidovic', ' B...</td>\n",
       "      <td>20190429.0</td>\n",
       "      <td>20190429</td>\n",
       "      <td>20200107</td>\n",
       "      <td>20200107</td>\n",
       "      <td>https://patents.google.com/patent/US10526056B1/en</td>\n",
       "      <td>https://patentimages.storage.googleapis.com/6f...</td>\n",
       "      <td>US</td>\n",
       "      <td>B1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>US11228182B2</td>\n",
       "      <td>Converter disabling photovoltaic electrical en...</td>\n",
       "      <td>Ampt, Llc</td>\n",
       "      <td>['Robert Porter', ' Anatoli Ledenev']</td>\n",
       "      <td>20071015.0</td>\n",
       "      <td>20190612</td>\n",
       "      <td>20220118</td>\n",
       "      <td>20220118</td>\n",
       "      <td>https://patents.google.com/patent/US11228182B2/en</td>\n",
       "      <td>https://patentimages.storage.googleapis.com/f5...</td>\n",
       "      <td>US</td>\n",
       "      <td>B2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>US11060742B2</td>\n",
       "      <td>PVT heat pump system capable of achieving day-...</td>\n",
       "      <td>Dalian University Of Technology</td>\n",
       "      <td>['Jili ZHANG', ' Ruobing LIANG', ' Chao Zhou',...</td>\n",
       "      <td>20170803.0</td>\n",
       "      <td>20170803</td>\n",
       "      <td>20210713</td>\n",
       "      <td>20210713</td>\n",
       "      <td>https://patents.google.com/patent/US11060742B2/en</td>\n",
       "      <td>https://patentimages.storage.googleapis.com/f8...</td>\n",
       "      <td>US</td>\n",
       "      <td>B2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>US10808685B2</td>\n",
       "      <td>Dispatchable combined cycle power plant</td>\n",
       "      <td>William M. Conlon</td>\n",
       "      <td>['William M. Conlon']</td>\n",
       "      <td>20140604.0</td>\n",
       "      <td>20181019</td>\n",
       "      <td>20201020</td>\n",
       "      <td>20201020</td>\n",
       "      <td>https://patents.google.com/patent/US10808685B2/en</td>\n",
       "      <td>https://patentimages.storage.googleapis.com/d5...</td>\n",
       "      <td>US</td>\n",
       "      <td>B2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             id                                              title  \\\n",
       "0  US10340693B2  Systems and methods for generating energy usin...   \n",
       "1  US10526056B1  Generation of electric power using wave motion...   \n",
       "2  US11228182B2  Converter disabling photovoltaic electrical en...   \n",
       "3  US11060742B2  PVT heat pump system capable of achieving day-...   \n",
       "4  US10808685B2           Dispatchable combined cycle power plant    \n",
       "\n",
       "                                        assignee  \\\n",
       "0  Lawrence D. Lansing, JR., Lawrence D. Lansing   \n",
       "1              Physician Electronic Network, LLC   \n",
       "2                                      Ampt, Llc   \n",
       "3                Dalian University Of Technology   \n",
       "4                              William M. Conlon   \n",
       "\n",
       "                                            inventor  priority_date  \\\n",
       "0  ['Lawrence D. Lansing', ' JR.', ' Lawrence D. ...     20130313.0   \n",
       "1  ['A-Hamid Hakki', ' Edin Dervishalidovic', ' B...     20190429.0   \n",
       "2              ['Robert Porter', ' Anatoli Ledenev']     20071015.0   \n",
       "3  ['Jili ZHANG', ' Ruobing LIANG', ' Chao Zhou',...     20170803.0   \n",
       "4                              ['William M. Conlon']     20140604.0   \n",
       "\n",
       "   filing_date  publication_date  grant_date  \\\n",
       "0     20180227          20190702    20190702   \n",
       "1     20190429          20200107    20200107   \n",
       "2     20190612          20220118    20220118   \n",
       "3     20170803          20210713    20210713   \n",
       "4     20181019          20201020    20201020   \n",
       "\n",
       "                                         result_link  \\\n",
       "0  https://patents.google.com/patent/US10340693B2/en   \n",
       "1  https://patents.google.com/patent/US10526056B1/en   \n",
       "2  https://patents.google.com/patent/US11228182B2/en   \n",
       "3  https://patents.google.com/patent/US11060742B2/en   \n",
       "4  https://patents.google.com/patent/US10808685B2/en   \n",
       "\n",
       "                          representative_figure_link country kind_code  \n",
       "0  https://patentimages.storage.googleapis.com/4c...      US        B2  \n",
       "1  https://patentimages.storage.googleapis.com/6f...      US        B1  \n",
       "2  https://patentimages.storage.googleapis.com/f5...      US        B2  \n",
       "3  https://patentimages.storage.googleapis.com/f8...      US        B2  \n",
       "4  https://patentimages.storage.googleapis.com/d5...      US        B2  "
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "87fb7c65-0c72-4d50-9a48-4353232e1fcb",
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests\n",
    "from bs4 import BeautifulSoup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "83d971de-23d6-450c-95a4-c128277c6f62",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Take the first patent as an example:\n",
    "url = df.loc[0,\"result_link\"]\n",
    "response = requests.get(url) \n",
    "html_content = response.text\n",
    "\n",
    "# Create a BeautifulSoup object to parse the HTML\n",
    "soup = BeautifulSoup(html_content, 'html.parser') "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "id": "310fd8e9-bd41-43fe-bd86-80cd35605a95",
   "metadata": {},
   "outputs": [],
   "source": [
    "patent_abstract = soup.find('h2', string = 'Abstract').find_next_sibling().find('abstract').find('div').contents[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 168,
   "id": "d1cfa4ee-fa08-4546-8ef8-a0944c916381",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Systems and methods for continuously generating electric power using a renewable energy power source to continuously generate electrical energy are disclosed. An illustrative embodiment includes transmitting electrical power from the renewable energy power source to an electrolyzer to produce hydrogen gas, storing the hydrogen gas in a storage facility until production of power from the renewable energy power source drops below a predetermined threshold, and activating a secondary power generation system that converts the stored hydrogen to electrical energy. The stored hydrogen may be converted to electrical energy using a gas turbine generator or a fuel cell. The system further includes a reverse osmosis subsystem for purifying water for use in the electrolyzer and optional systems for providing the purified water to a community and for using the produced electricity to treat waste water to generate treated water that may be purified and supplied to the electrolyzer.'"
      ]
     },
     "execution_count": 168,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "patent_abstract"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "b0c60eab-69cf-4a3a-b59c-38800fe2470f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1 patent abstract:\n",
      "Systems and methods for continuously generating electric power using a renewable energy power source to continuously generate electrical energy are disclosed. An illustrative embodiment includes transmitting electrical power from the renewable energy power source to an electrolyzer to produce hydrogen gas, storing the hydrogen gas in a storage facility until production of power from the renewable energy power source drops below a predetermined threshold, and activating a secondary power generation system that converts the stored hydrogen to electrical energy. The stored hydrogen may be converted to electrical energy using a gas turbine generator or a fuel cell. The system further includes a reverse osmosis subsystem for purifying water for use in the electrolyzer and optional systems for providing the purified water to a community and for using the produced electricity to treat waste water to generate treated water that may be purified and supplied to the electrolyzer.\n",
      "\n",
      "2 patent abstract:\n",
      "A method and apparatus for producing electricity from a combination of three sources: ocean waves, wind and solar, includes converting an upward and downward motion of a buoyant platform into a rotational motion of a shaft using a plurality of blades mounted to the shaft, the blades causing the shaft to rotate from internal wind energy as the blades move up and down within a cavity. The shaft is coupled to a generator for producing electricity. A wind turbine is mounted to the buoyant platform for converting wind energy into electricity. Further, solar panels are included, for example, mounted to the buoyant platform and/or turbine blades of the wind turbine, the solar panels also generate electricity when exposed to light.\n",
      "\n",
      "3 patent abstract:\n",
      "Renewable electrical energy is provided with aspects and circuitry that can harvest maximum power from an alternative electrical energy source (\n",
      "\n",
      "4 patent abstract:\n",
      "The present invention provides a photovoltic and thermal (PVT) heat pump system capable of achieving day-night time-shared combined cooling, heating and power using solar radiation and sky cold radiation. The system utilizes a photovoltaic power generation technology and a photovoltic and thermal (PVT heat pump technology simultaneously, both of which are relatively independent and promoted to each other in the function. The main energy sources of the system are solar radiation energy and sky long-wave cold radiation energy, and the energy is respectively transformed into electric energy, thermal energy and cold energy via a photovoltic and thermal (PVT) photoelectric-evaporation/condensation module at different times in different working modes. The system of the present invention integrates power generation. heating, refrigeration and many other functions; and the equipment has simple composition, high utilization rate and remarkable energy-saving effect, thereby improving the energy utilization rate to the maximum extent, and achieving a multi-purpose machine and day-night time-shared combined cooling, heat and power.\n",
      "\n",
      "5 patent abstract:\n",
      "A combined cycle power plant comprises a combustion turbine generator, another heat source in addition to the combustion turbine generator, a steam power system, and an energy storage system. Heat from the heat source, from the energy storage system, or from the heat source and the energy storage system is used to generate steam in the steam power system. Heat from the combustion turbine generator exhaust gas may be used primarily for single phase heating of water or steam in the steam power system. Alternatively, heat from the combustion turbine generator exhaust gas may be used in parallel with the energy storage system and/or the other heat source to generate steam, and additionally to super heat steam. Both the combustion turbine generator and the steam power system may generate electricity.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# view first 5 patents abstract scraping results\n",
    "for i in range(5):\n",
    "    url = df.loc[i,\"result_link\"]\n",
    "    response = requests.get(url) \n",
    "    html_content = response.text\n",
    "    soup = BeautifulSoup(html_content, 'html.parser')\n",
    "    print(f'{i+1} patent abstract:')\n",
    "    print(soup.find('h2', string = 'Abstract').find_next_sibling().find('abstract').find('div').contents[0])\n",
    "    print('')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5d95f01c-d2fa-43e7-8a58-7cd912e1e277",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "### Problem occured in the third example, let's see what is going wrong"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "id": "cd75addc-87b0-4174-9e97-738477383dea",
   "metadata": {},
   "outputs": [],
   "source": [
    "# inspect third patent\n",
    "url = df.loc[2,\"result_link\"]\n",
    "response = requests.get(url) \n",
    "html_content = response.text\n",
    "soup = BeautifulSoup(html_content, 'html.parser') "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 132,
   "id": "e948ea4a-630a-43b7-917f-2a794db926bc",
   "metadata": {},
   "outputs": [],
   "source": [
    "abstract_contents_list = soup.find('h2', string = 'Abstract').find_next_sibling().find('abstract').find('div').contents"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 165,
   "id": "ab5d3cf5-5876-4dbd-8f8a-c331bc0e31ca",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Renewable electrical energy is provided with aspects and circuitry that can harvest maximum power from an alternative electrical energy source (',\n",
       " <b>1</b>,\n",
       " ') such as a string of solar panels (',\n",
       " <b>11</b>,\n",
       " ') for a power grid (',\n",
       " <b>10</b>,\n",
       " '). Aspects include: i) controlling electrical power creation from photovoltaic DC-AC inverter (',\n",
       " <b>5</b>,\n",
       " '), ii) operating photovoltaic DC-AC inverter (',\n",
       " <b>5</b>,\n",
       " ') at maximal efficiency even when MPP would not be, iii) protecting DC-AC inverter (',\n",
       " <b>5</b>,\n",
       " ') so input can vary over a range of insolation and temperature, and iv) providing dynamically reactive capability to react and assure operation, to permit differing components, to achieve code compliant dynamically reactive photovoltaic power control circuitry (',\n",
       " <b>41</b>,\n",
       " '). With previously explained converters, inverter control circuitry (',\n",
       " <b>38</b>,\n",
       " ') or photovoltaic power converter functionality control circuitry (',\n",
       " <b>8</b>,\n",
       " ') configured as inverter sweet spot converter control circuitry (',\n",
       " <b>46</b>,\n",
       " ') can achieve extraordinary efficiencies with substantially power isomorphic photovoltaic capability at 99.2% efficiency or even only wire transmission losses.']"
      ]
     },
     "execution_count": 165,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "abstract_contents_list"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f70df1d6-e3c7-440c-9e57-101ec85f0a60",
   "metadata": {},
   "source": [
    "The contents may be a list of strings, so using \".contents[0]\" is not a good practice. We should extract contents as a list and join all the string elements in the list.\n",
    "\n",
    "There also comes another problem in this third example, there are Tags in the contents list, which we would like to exclude from our abstract.\n",
    "\n",
    "So, we need to change our workflow. We use the contents list, only keep strings, and join all string elements to form one string, that is the whole abstract of this patent."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 166,
   "id": "7570d7a5-e4be-4257-9793-15e44e2844a3",
   "metadata": {},
   "outputs": [],
   "source": [
    "abstract_contents = [element for element in abstract_contents_list if isinstance(element, str)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 167,
   "id": "f4c6b26a-6e16-4947-a4dc-e133c0dc1e84",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Renewable electrical energy is provided with aspects and circuitry that can harvest maximum power from an alternative electrical energy source (',\n",
       " ') such as a string of solar panels (',\n",
       " ') for a power grid (',\n",
       " '). Aspects include: i) controlling electrical power creation from photovoltaic DC-AC inverter (',\n",
       " '), ii) operating photovoltaic DC-AC inverter (',\n",
       " ') at maximal efficiency even when MPP would not be, iii) protecting DC-AC inverter (',\n",
       " ') so input can vary over a range of insolation and temperature, and iv) providing dynamically reactive capability to react and assure operation, to permit differing components, to achieve code compliant dynamically reactive photovoltaic power control circuitry (',\n",
       " '). With previously explained converters, inverter control circuitry (',\n",
       " ') or photovoltaic power converter functionality control circuitry (',\n",
       " ') configured as inverter sweet spot converter control circuitry (',\n",
       " ') can achieve extraordinary efficiencies with substantially power isomorphic photovoltaic capability at 99.2% efficiency or even only wire transmission losses.']"
      ]
     },
     "execution_count": 167,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "abstract_contents"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 164,
   "id": "d1179cd4-aab5-41a9-9476-0cc2b725f336",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Renewable electrical energy is provided with aspects and circuitry that can harvest maximum power from an alternative electrical energy source ( ) such as a string of solar panels ( ) for a power grid ( ). Aspects include: i) controlling electrical power creation from photovoltaic DC-AC inverter ( ), ii) operating photovoltaic DC-AC inverter ( ) at maximal efficiency even when MPP would not be, iii) protecting DC-AC inverter ( ) so input can vary over a range of insolation and temperature, and iv) providing dynamically reactive capability to react and assure operation, to permit differing components, to achieve code compliant dynamically reactive photovoltaic power control circuitry ( ). With previously explained converters, inverter control circuitry ( ) or photovoltaic power converter functionality control circuitry ( ) configured as inverter sweet spot converter control circuitry ( ) can achieve extraordinary efficiencies with substantially power isomorphic photovoltaic capability at 99.2% efficiency or even only wire transmission losses.'"
      ]
     },
     "execution_count": 164,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "abstract = \" \".join(abstract_contents)\n",
    "abstract"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "53975589-9d70-4ce1-8fd3-6186525f7443",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "### Establish patent abstract scarping workflow"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 169,
   "id": "cb73a841-fbf8-4e1b-8b55-22860c0b234f",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import requests\n",
    "from bs4 import BeautifulSoup\n",
    "import random\n",
    "\n",
    "# reload cleaned patent list\n",
    "df = pd.read_csv(r\"C:\\Users\\user\\Documents\\RenewableEnergy_patents_clean.csv\", index_col = 0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "5c66ec70-0357-41aa-a682-426090559813",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "15815 patent abstract:\n",
      "A power generation transport includes a gas turbine, an inlet plenum coupled to an intake of the gas turbine, a generator driven by the gas turbine, and an air intake and exhaust module including an air inlet filter housing, an intake air duct coupled to the housing at a first end and to the inlet plenum at a second end, and an exhaust collector coupled to an exhaust of the gas turbine. The transport further includes at least one base frame. The frame mounts and aligns the gas turbine, the inlet plenum, the generator, and the air intake and exhaust module. The intake air duct is mounted on the base frame so as to be disposed underneath the gas turbine, and extend along the base frame from an exhaust end side of the gas turbine to an intake end side, in a longitudinal direction of the power generation transport.\n",
      "\n",
      "13451 patent abstract:\n",
      "An example device includes a housing including a first surface facing a first direction and a second surface facing a second direction; a transparent cover formed on at least a portion of the first surface of the housing; a display disposed between the transparent cover and the second surface; a sensor disposed between the display and the second surface; and a control circuit, electrically connected to the sensor, for controlling the sensor, wherein the display can include: a first region including a plurality of pixels capable of displaying color; and a second region aligned on at least a portion of the sensor such that light acquired from the outside of the electronic device passes through the sensor.\n",
      "\n",
      "5025 patent abstract:\n",
      "A force measurement system is disclosed herein. The force measurement system includes a force measurement assembly configured to receive a subject thereon, a visual display device having an output screen, and one or more data processing devices operatively coupled to the force measurement assembly and the visual display device. In the one or more embodiments, the force measurement assembly is a static force plate and the visual display device is a head-mounted visual display device or a visual display device having an output screen configured to at least partially circumscribe three sides of a torso of the subject. In one or more other embodiments, the force measurement assembly is a displaceable force plate and the visual display device is a head-mounted visual display device.\n",
      "\n",
      "4321 patent abstract:\n",
      "Provided herein are methods and systems for improved storage strategies for use of collections of storage resources, such as solid state drives, including in connection with a converged networking and storage node that may be used for virtualization of a collection of physically attached and/or network-connected storage resources.\n",
      "\n",
      "9392 patent abstract:\n",
      "An enclosure for containing a portable hemodialysis unit includes a housing suitable to support components for performing hemodialysis including a dialyzer, one or more pumps to circulate blood through the dialyzer, a source of dialysate, and one or more pumps to circulate the dialysate through the dialyzer. The housing may have a front panel at which blood circuit connections and dialysate fluidic connections are located, e.g., blood line connections for patient blood access, connections for a reagent supply, dialyzer connections for both blood flow and dialysate, etc. The enclosure may also include a pair of vertical, side-by-side doors hingedly mounted to the housing. With the doors in the closed position, access to the patient access and dialysate fluidic connections may be blocked, and the doors may allow for the retention of heat in the housing suitable for disinfection during a disinfection cycle.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# randomly sample 5 patents abstract scraping results\n",
    "# run several times to make sure there is no problem with current scraping method\n",
    "for i in random.sample(range(len(df)), 5):\n",
    "    url = df.loc[i,\"result_link\"]\n",
    "    response = requests.get(url) \n",
    "    html_content = response.text\n",
    "    soup = BeautifulSoup(html_content, 'html.parser')\n",
    "    abstract_contents_list = soup.find('h2', string = 'Abstract').find_next_sibling().find('abstract').find('div').contents\n",
    "    abstract_contents = [element for element in abstract_contents_list if isinstance(element, str)]\n",
    "    abstract = \" \".join(abstract_contents)\n",
    "    print(f'{i+1} patent abstract:')\n",
    "    print(abstract)\n",
    "    print('')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fae32dfb-1f35-4d56-9d47-7358b50b85f7",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "### Scrape Abstracts for All Patents Filed in 2023"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "88f86814-61f6-4e35-aaf5-3d42b54bc6d2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "360"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# extract patents filed in 2023 \n",
    "df_2023 = df.loc[df['filing_date'] > 20230000,]\n",
    "len(df_2023)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "a6d04d21-ee6a-4640-9d34-ef02e817df4c",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>title</th>\n",
       "      <th>assignee</th>\n",
       "      <th>inventor</th>\n",
       "      <th>priority_date</th>\n",
       "      <th>filing_date</th>\n",
       "      <th>publication_date</th>\n",
       "      <th>grant_date</th>\n",
       "      <th>result_link</th>\n",
       "      <th>representative_figure_link</th>\n",
       "      <th>country</th>\n",
       "      <th>kind_code</th>\n",
       "      <th>abstract</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>US11967653B2</td>\n",
       "      <td>Phased solar power supply system</td>\n",
       "      <td>Ampt, Llc</td>\n",
       "      <td>['Anatoli Ledenev']</td>\n",
       "      <td>20130315.0</td>\n",
       "      <td>20230905</td>\n",
       "      <td>20240423</td>\n",
       "      <td>20240423</td>\n",
       "      <td>https://patents.google.com/patent/US11967653B2/en</td>\n",
       "      <td>https://patentimages.storage.googleapis.com/c0...</td>\n",
       "      <td>US</td>\n",
       "      <td>B2</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>US11867096B2</td>\n",
       "      <td>Calcination system with thermal energy storage...</td>\n",
       "      <td>Rondo Energy, Inc.</td>\n",
       "      <td>[\"John Setel O'Donnell\", ' Peter Emery von Beh...</td>\n",
       "      <td>20201130.0</td>\n",
       "      <td>20230220</td>\n",
       "      <td>20240109</td>\n",
       "      <td>20240109</td>\n",
       "      <td>https://patents.google.com/patent/US11867096B2/en</td>\n",
       "      <td>https://patentimages.storage.googleapis.com/35...</td>\n",
       "      <td>US</td>\n",
       "      <td>B2</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>US11757404B2</td>\n",
       "      <td>Coordinated control of renewable electric gene...</td>\n",
       "      <td>8Me Nova, Llc</td>\n",
       "      <td>['Lukas Hansen', ' Philippe Garneau-Halliday',...</td>\n",
       "      <td>20190208.0</td>\n",
       "      <td>20230310</td>\n",
       "      <td>20230912</td>\n",
       "      <td>20230912</td>\n",
       "      <td>https://patents.google.com/patent/US11757404B2/en</td>\n",
       "      <td>https://patentimages.storage.googleapis.com/f6...</td>\n",
       "      <td>US</td>\n",
       "      <td>B2</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>US11973345B2</td>\n",
       "      <td>Building energy system with predictive control...</td>\n",
       "      <td>Johnson Controls Tyco IP Holdings LLP</td>\n",
       "      <td>['Robert D. Turney', ' Nishith R. Patel']</td>\n",
       "      <td>20170427.0</td>\n",
       "      <td>20230602</td>\n",
       "      <td>20240430</td>\n",
       "      <td>20240430</td>\n",
       "      <td>https://patents.google.com/patent/US11973345B2/en</td>\n",
       "      <td>https://patentimages.storage.googleapis.com/0f...</td>\n",
       "      <td>US</td>\n",
       "      <td>B2</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>US11780305B2</td>\n",
       "      <td>Tonneau system for use with a pickup truck</td>\n",
       "      <td>Worksport Ltd.</td>\n",
       "      <td>['Steven Rossi', ' Jonathan Loudon']</td>\n",
       "      <td>20151030.0</td>\n",
       "      <td>20230110</td>\n",
       "      <td>20231010</td>\n",
       "      <td>20231010</td>\n",
       "      <td>https://patents.google.com/patent/US11780305B2/en</td>\n",
       "      <td>https://patentimages.storage.googleapis.com/36...</td>\n",
       "      <td>US</td>\n",
       "      <td>B2</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             id                                              title  \\\n",
       "0  US11967653B2                  Phased solar power supply system    \n",
       "1  US11867096B2  Calcination system with thermal energy storage...   \n",
       "2  US11757404B2  Coordinated control of renewable electric gene...   \n",
       "3  US11973345B2  Building energy system with predictive control...   \n",
       "4  US11780305B2        Tonneau system for use with a pickup truck    \n",
       "\n",
       "                                assignee  \\\n",
       "0                              Ampt, Llc   \n",
       "1                     Rondo Energy, Inc.   \n",
       "2                          8Me Nova, Llc   \n",
       "3  Johnson Controls Tyco IP Holdings LLP   \n",
       "4                         Worksport Ltd.   \n",
       "\n",
       "                                            inventor  priority_date  \\\n",
       "0                                ['Anatoli Ledenev']     20130315.0   \n",
       "1  [\"John Setel O'Donnell\", ' Peter Emery von Beh...     20201130.0   \n",
       "2  ['Lukas Hansen', ' Philippe Garneau-Halliday',...     20190208.0   \n",
       "3          ['Robert D. Turney', ' Nishith R. Patel']     20170427.0   \n",
       "4               ['Steven Rossi', ' Jonathan Loudon']     20151030.0   \n",
       "\n",
       "   filing_date  publication_date  grant_date  \\\n",
       "0     20230905          20240423    20240423   \n",
       "1     20230220          20240109    20240109   \n",
       "2     20230310          20230912    20230912   \n",
       "3     20230602          20240430    20240430   \n",
       "4     20230110          20231010    20231010   \n",
       "\n",
       "                                         result_link  \\\n",
       "0  https://patents.google.com/patent/US11967653B2/en   \n",
       "1  https://patents.google.com/patent/US11867096B2/en   \n",
       "2  https://patents.google.com/patent/US11757404B2/en   \n",
       "3  https://patents.google.com/patent/US11973345B2/en   \n",
       "4  https://patents.google.com/patent/US11780305B2/en   \n",
       "\n",
       "                          representative_figure_link country kind_code  \\\n",
       "0  https://patentimages.storage.googleapis.com/c0...      US        B2   \n",
       "1  https://patentimages.storage.googleapis.com/35...      US        B2   \n",
       "2  https://patentimages.storage.googleapis.com/f6...      US        B2   \n",
       "3  https://patentimages.storage.googleapis.com/0f...      US        B2   \n",
       "4  https://patentimages.storage.googleapis.com/36...      US        B2   \n",
       "\n",
       "  abstract  \n",
       "0           \n",
       "1           \n",
       "2           \n",
       "3           \n",
       "4           "
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_2023 = df_2023.reset_index(drop = True)\n",
    "df_2023.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "45f547fb-07af-404f-8ca4-fa9134ace57d",
   "metadata": {},
   "outputs": [],
   "source": [
    "from tqdm import tqdm\n",
    "import time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "0b43b715-c80e-49f7-a565-aaa6c4364310",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|████████████████████████████████████████████████████████████████████████████████| 360/360 [14:41<00:00,  2.45s/it]\n"
     ]
    }
   ],
   "source": [
    "df_2023['abstract'] = ''\n",
    "\n",
    "for i in tqdm(range(len(df_2023))):\n",
    "    url = df_2023.loc[i,\"result_link\"]\n",
    "    response = requests.get(url) \n",
    "    html_content = response.text\n",
    "    soup = BeautifulSoup(html_content, 'html.parser')\n",
    "    abstract_contents_list = soup.find('h2', string = 'Abstract').find_next_sibling().find('abstract').find('div').contents\n",
    "    abstract_contents = [element for element in abstract_contents_list if isinstance(element, str)]\n",
    "    abstract = \" \".join(abstract_contents)\n",
    "    df_2023.loc[i, 'abstract'] = abstract\n",
    "    time.sleep(0.1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e44e50cf-ce0f-4288-85f4-19cd8b36e214",
   "metadata": {},
   "source": [
    "It takes about 15 mins to scape 360 patents' abstracts."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "6d624c5e-cca4-420e-a637-c4f406967e65",
   "metadata": {},
   "outputs": [],
   "source": [
    "# save df_2023\n",
    "df_2023.to_csv(r\"C:\\Users\\user\\Documents\\RenewableEnergy_patents_2023_abstract.csv\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "69660e0e-eeb8-411d-9659-688b8ceb0a7f",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "### Bag of Words: using scikit Learn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "id": "006349af-5736-44f9-9830-4cafa47bc26a",
   "metadata": {},
   "outputs": [],
   "source": [
    "vect = CountVectorizer()\n",
    "vect.fit(ab)\n",
    "bof = vect.transform(ab)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "id": "3c3952f4-97e9-464f-b474-05b508c03c69",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3981\n",
      "dict_keys(['high', 'efficiency', 'solar', 'power', 'system', 'combining', 'photovoltaic', 'sources', 'of', 'can', 'be', 'converted', 'by', 'base', 'phase', 'dc', 'converter', 'and', 'an', 'altered', 'that', 'have', 'outputs', 'combined', 'through', 'low', 'energy', 'storage', 'combiner', 'circuitry', 'the', 'converters', 'synchronously', 'controlled', 'synchronous', 'control', 'operates', 'switches', 'to', 'provide', 'conversion', 'output', 'provided', 'for', 'individual', 'source', 'or', 'phased', 'operational', 'modes', 'latter', 'presenting', 'at', 'string', 'panel', 'levels', 'converts', 'variable', 'renewable', 'electricity', 'vre', 'continuous', 'heat', 'over', '1000â', 'intermittent', 'electrical', 'heats', 'solid', 'medium', 'from', 'is', 'delivered', 'continuously', 'on', 'demand', 'array', 'bricks', 'incorporating', 'internal', 'radiation', 'cavities', 'directly', 'heated', 'thermal', 'facilitate', 'rapid', 'uniform', 'heating', 'via', 'reradiation', 'delivery', 'flowing', 'gas', 'establishes', 'thermocline', 'which', 'maintains', 'outlet', 'temperature', 'throughout', 'discharge', 'flows', 'structured', 'pathways', 'within', 'delivering', 'may', 'used', 'processes', 'including', 'calcination', 'hydrogen', 'electrolysis', 'steam', 'generation', 'cogeneration', 'groups', 'arrays', 'operated', 'temperatures', 'without', 'runaway', 'deep', 'sequencing', 'forecast', 'based', 'enables', 'year', 'round', 'supply', 'using', 'current', 'advance', 'information', 'weather', 'availability', 'voltage', 'distribution', 'improves', 'transfer', 'into', 'method', 'includes', 'generating', 'time', 'varying', 'charge', 'signal', 'device', 'wherein', 'comprises', 'identifying', 'prioritization', 'order', 'stack', 'simultaneously', 'operating', 'staging', 'mode', 'least', 'two', 'additional', 'each', 'comprising', 'plurality', 'candidate', 'values', 'intersection', 'one', 'more', 'according', 'determining', 'predictive', 'controller', 'building', 'processing', 'circuits', 'configured', 'obtain', 'constraint', 'defines', 'total', 'electric', 'load', 'served', 'step', 'period', 'as', 'summation', 'multiple', 'specific', 'components', 'include', 'first', 'component', 'indicating', 'amount', 'during', 'second', 'are', 'perform', 'process', 'subject', 'determine', 'operate', 'equipment', 'tonneau', 'use', 'with', 'pickup', 'truck', 'frame', 'cover', 'having', 'sections', 'moveable', 'between', 'folded', 'position', 'unfolded', 'receiving', 'material', 'secured', 'top', 'exposed', 'surface', 'moves', 'their', 'corresponding', 'positions', 'battery', 'transformer', 'cabling', 'included', 'circuit', 'systems', 'methods', 'charging', 'discharging', 'batteries', 'described', 'herein', 'in', 'some', 'embodiments', 'module', 'electrically', 'coupled', 'operable', 'state', 'transferred', 'stored', 'operably', 'monitor', 'apparatus', 'smart', 'grid', 'communication', 'disclosed', 'present', 'invention', 'element', 'constructed', 'network', 'server', 'coordinator', 'transformed', 'active', 'automatically', 'autonomously', 'after', 'initial', 'connection', 'sends', 'receives', 'messages', 'matches', 'prioritizes', 'provides', 'priority', 'flag', 'tracks', 'actual', 'introduced', 'available', 'curtailment', 'various', 'relate', 'powering', 'electrochromic', 'windows', 'panels', 'generate', 'storing', 'generated', 'controllers', 'receive', 'tint', 'transitions', 'regulating', 'controlling', 'application', 'techniques', 'design', 'networks', 'hybrid', 'off', 'otgâ', 'example', 'clamp', 'assembly', 'glass', 'tracker', 'has', 'lower', 'structure', 'characterized', 'hat', 'shaped', 'rail', 'length', 'extending', 'end', 'upper', 'sandwich', 'pair', 'edges', 'modules', 'portion', 'locking', 'spacer', 'substantially', 'key', 'structures', 'affixed', 'physically', 'lock', 'powered', 'lamp', 'radiating', 'extensions', 'being', 'formed', 'flat', 'fold', 'relative', 'other', 'along', 'lines', 'respective', 'ones', 'electronics', 'collapsible', 'collapsed', 'associated', 'wireless', 'camera', 'image', 'capture', 'images', 'further', 'if', 'down', 'store', 'captured', 'infrared', 'activation', 'triggered', 'up', 'wirelessly', 'transmit', 'detection', 'capturing', 'any', 'noise', 'motion', 'timing', 'command', 'also', 'communicate', 'utilizing', 'detecting', 'reporting', 'pollution', 'oil', 'facility', 'technology', 'utilizes', 'logic', 'read', 'sensor', 'sensed', 'data', 'pollutant', 'leak', 'leakage', 'managed', 'reduced', 'management', 'communications', 'communicates', 'internet', 'protocol', 'ip', 'real', 'less', 'than', '15', 'minutes', 'interval', 'participates', 'actively', 'consumption', 'pattern', 'comprise', 'packet', 'content', 'security', 'transport', 'route', 'attachment', 'point', 'cells', 'emitter', 'regions', 'composed', 'wide', 'bandgap', 'semiconductor', 'forming', 'tool', 'atmosphere', 'thin', 'dielectric', 'layer', 'substrate', 'cell', 'removing', 'approximately', 'electron', 'volts', 'ev', 'above', 'agricultural', 'providing', 'positive', 'air', 'pressure', 'chamber', 'prevent', 'outside', 'contaminants', 'entering', 'growing', 'crops', 'multi', 'grow', 'benches', 'connectors', 'vertical', 'hoists', 'movements', 'maintaining', 'pre', 'set', 'humidity', 'carbon', 'dioxide', 'watering', 'lighting', 'achieve', 'predetermined', 'plant', 'growth', 'motorized', 'rails', 'deliver', 'operations', 'seeding', 'harvesting', 'media', 'recovery', 'bench', 'wash', 'dispensing', 'seeds', 'mechanical', 'seeder', 'computer', 'nutrients', 'light', 'level', 'harvested', 'crop', 'selected', 'universal', 'adapter', 'enabling', 'truss', 'foundations', 'support', 'conventional', 'single', 'axis', 'trackers', 'cases', 'main', 'body', 'presents', 'standard', 'beam', 'interface', 'drilled', 'holes', 'connecting', 'portions', 'away', 'enable', 'leg', 'connect', 'driven', 'screw', 'anchors', 'located', 'below', 'it', 'orient', 'so', 'rotational', 'aligned', 'work', 'foundation', 'made', 'legs', 'this', 'will', 'many', 'different', 'much', 'like', 'plumb', 'pile', 'transmittance', 'sheet', 'transparent', 'electrode', 'spaced', 'apart', 'electrodes', 'electrophoretic', 'containing', 'ink', 'non', 'planar', 'polymer', 'elements', 'embodiment', 'charged', 'particles', 'suspending', 'fluid', '75', 'mass', 'organosilicone', 'aliphatic', 'hydrocarbon', 'fluorinated', 'elastomeric', 'devices', 'transvascular', 'ablation', 'target', 'tissue', 'examples', 'splanchnic', 'nerve', 'increase', 'venous', 'blood', 'capacitance', 'treat', 'heart', 'failure', 'hypertension', 'advanced', 'endovascularly', 'vessel', 'region', 'thoracic', 'tsn', 'such', 'greater', 'gsn', 'root', 'treating', 'hfpef', 'ablating', 'reduce', 'pulmonary', 'vehicle', 'suitable', 'sharing', 'service', '100', 'capable', 'travelling', 'driving', 'motor', 'exchangeable', 'station', 'adjusting', 'speed', 'connected', 'quantitatively', 'evaluates', 'exchangeability', 'basis', 'remaining', 'mounted', 'determines', 'evaluation', 'value', 'transmits', 'relevant', 'determined', 'memory', 'row', 'hammer', 'mitigation', 'couples', 'host', 'assist', 'sending', 'refresh', 'cycles', 'commands', 'response', 'extra', 'instead', 'integrity', 'keep', 'track', 'number', 'activate', 'sent', 'threshold', 'potential', 'victim', 'rows', 'aggressor', 'simply', 'refreshing', 'not', 'been', 'accessed', 'matter', 'specification', 'embodied', 'among', 'things', 'shifting', 'when', 'cold', 'cooled', 'model', 'obtaining', 'cost', 'describes', 'schedule', 'costs', 'future', 'refrigeration', 'maximum', 'allowed', 'cooling', 'reducing', 'usage', 'permitting', 'warmed', 'ambient', 'toward', 'excellent', 'characteristics', 'degree', 'safety', 'deterioration', 'neural', 'input', 'hidden', 'layers', 'previous', 'weight', 'coefficient', 'next', 'obtained', 'measured', 'sets', 'times', 'condition', 'changed', 'accordance', 'scalable', 'greenhouse', 'allow', 'user', 'exhaust', 'board', 'transportation', 'mechanism', 'involve', 'pumps', 'nozzle', 'port', 'upon', 'engagement', 'create', 'tight', 'seal', 'pipe', 'therethrough', 'temporarily', 'holding', 'tank', 'instantaneous', 'characteristic', 'yaw', 'rate', 'addition', 'adapting', 'autonomous', 'steering', 'acceleration', 'deceleration', 'adapted', 'implementations', 'coherent', 'ranging', 'lidar', 'monopulse', 'frequency', 'modulated', 'wave', 'fmcw', 'perovskite', 'crystal', 'lattice', 'formula', 'cxmyxz', 'alkyl', 'polyammonium', 'cations', 'disposed', 'numbers', 'group', 'consisting', 'metals', 'ammonium', 'formamidinium', 'guanidinium', 'ethene', 'tetramine', 'mg', 'ca', 'sr', 'ba', 'fe', 'cd', 'co', 'ni', 'cu', 'ag', 'au', 'hg', 'sn', 'ge', 'ga', 'pb', 'tl', 'sb', 'bi', 'ti', 'zn', 'zr', 'combinations', 'thereof', 'anions', 'halides', 'pseudohalides', 'chalcogenides', 'unmanned', 'aerial', 'robotic', 'uarv', 'fly', 'object', 'palm', 'tree', 'hover', 'place', 'adjacent', 'mount', 'itself', 'securely', 'releasably', 'mounting', 'location', 'uses', 'incorporated', 'utility', 'performing', 'utilitarian', 'functions', 'cutting', 'trim', 'branches', 'foliage', 'mobile', 'sensory', 'platform', 'propulsion', 'move', 'area', 'vertically', 'sensors', 'positioned', 'heights', 'plants', 'way', 'movement', 'operation', 'microclimate', 'sense', 'microclimates', 'around', 'stereo', 'imaging', 'invasively', 'dimensional', 'points', '3d', 'canopy', 'generally', 'relates', 'particularly', 'frustules', 'aspect', 'supercapacitor', 'electrolyte', 'thereon', 'nanostructures', 'zinc', 'oxide', 'manganese', 'nanotube', 'eyewear', 'front', 'side', 'speakers', 'connector', 'conductive', 'pads', 'contacts', 'counterpart', 'rechargeable', 'microphone', 'inside', 'touch', 'sensitive', 'function', 'another', 'headset', 'pad', 'contact', 'laminate', 'ply', 'where', 'outer', 'third', 'facing', 'fourth', 'opposite', 'inner', 'interlayer', 'plies', 'enhanced', 'polarized', 'reflective', 'coating', 'contacted', 'angle', '60â', 'normal', 'exhibits', 'lta', '70', 'reflectivity', '10', 'display', 'projecting', 'heads', 'integrated', 'wear', 'out', 'monitoring', 'capability', 'record', 'indication', 'core', 'separated', 'localized', 'diffusion', 'diffusant', 'stress', 'causes', 'how', 'people', 'entered', 'exited', 'space', 'pyroelectric', 'pir', 'patterns', 'person', 'exiting', 'whether', 'thermopile', 'radar', 'visible', 'sensing', 'monitored', 'detected', 'left', 'occupant', 'count', 'then', 'accordingly', 'head', 'guard', 'part', 'remote', 'computing', 'intelligent', 'collection', 'compostable', 'lid', 'intended', 'beverage', 'capsule', 'multilayer', 'complex', 'successively', 'woven', '50', 'fibers', 'polylactic', 'acid', 'pla', 'adhesive', 'coming', 'food', 'vegetable', 'parchment', 'loads', 'accessing', 'wiring', 'installed', 'switch', 'paddle', 'actuator', 'unit', 'removably', 'attached', 'arm', 'sleeve', 'secure', 'protruding', 'pivot', 'joint', 'about', 'proximate', 'endpoint', 'midpoint', 'measuring', 'dose', 'drug', 'patient', 'measurement', 'liquid', 'volume', 'container', 'emit', 'electromagnetic', 'optically', 'coupleable', 'detect', 'emitted', 'representing', 'convert', 'received', 'signature', 'representative', 'intensity', 'emitting', 'diode', 'led', 'rectified', 'ac', 'bus', 'regulation', 'magnitude', 'conducted', 'operatively', 'pulse', 'width', 'modulating', 'performed', 'dormant', 'resources', 'demodulating', 'decoding', 'beams', 'evaluating', 'demodulated', 'decoded', 'against', 'criterion', 'discontinuing', 'measurements', 'met', 'neither', 'nor', 'deactivating', 'receiver', 'apparatuses', 'virtualizing', 'routing', 'traffic', 'offloading', 'decisions', 'balancing', 'applications', 'make', 'both', 'destination', 'rather', 'wait', 'until', 'routed', 'before', 'disclosure', 'production', 'furandicarboxylic', 'fdca', 'intermediates', 'chemocatalytic', 'furanic', 'oxidation', 'preparing', 'derivatives', 'polymers', 'crystalline', 'preparations', 'well', 'making', 'same', 'producing', 'copper', 'catalysts', 'porous', 'substrates', 'silicon', 'nanowires', 'materials', 'related', 'compositions', 'novel', 'catalyst', 'gold', 'shell', 'does', 'patterned', 'apertured', 'web', 'land', 'areas', 'apertures', 'defined', 'surround', 'effective', 'open', 'range', '30', 'aperture', 'test', 'interaperture', 'distances', 'median', 'mean', 'endoscopic', 'deficient', 'environment', 'tube', 'lens', 'optical', 'corresponds', 'visualize', 'scene', 'engine', 'illumination', 'pulses', 'lumen', 'transmitting', 'distal', 'tip', 'endoscope', 'closed', 'loop', 'obtains', 'stores', 'transfers', 'motive', 'preferably', 'majority', 'utilized', 'supplied', 'produced', 'recharge', 'subsequent', 'controls', 'manages', 'reservoir', 'series', 'innovations', 'electronic', 'instruction', 'analyze', 'optimize', 'sequence', 'its', 'purpose', 'highly', 'reliable', 'hole', 'injection', 'share', 'common', 'anode', 'contains', 'compound', 'property', 'hence', 'likely', 'excess', 'electrons', 'here', 'accepting', 'sub', 'pixels', 'line', 'pixel', 'organic', 'transistor', 'color', 'sides', 'overlap', 'channel', 'locations', 'ducts', 'plenums', 'fan', 'duct', 'multipoint', 'sampling', 'flow', 'comparing', 'contaminant', 'concentration', 'said', 'plenum', 'action', 'setback', 'ensuring', 'accuracy', 'reliability', 'implementing', 'protective', 'exceeds', 'displaying', 'arranged', 'matrix', 'units', 'subunits', 'subunit', 'gate', 'insulating', 'interposed', 'therebetween', 'drives', 'drain', 'counter', 'face', 'whose', 'za', 'î¼m', 'room', 'per', 'micrometer', '85â', 'parts', 'cathode', 'functional', 'reacts', 'alkali', 'earth', 'properties', 'fluidic', 'holder', 'faces', 'direction', 'reference', 'surfaces', 'directions', 'xy', 'plane', 'alignment', 'movable', 'locator', 'retracted', 'biased', 'hold', 'mixed', 'film', 'transition', 'metal', 'unshared', 'atom', 'form', 'somo', 'composition', 'nanocellulose', 'very', 'essentially', 'no', 'sulfur', 'cellulose', 'nanocrystals', 'nanofibrils', 'crystallinity', '80', 'onset', 'decomposition', '300â', 'higher', '400', '700', 'nm', 'variations', 'lignin', 'coated', 'hydrophobic', 'wt', 'equivalent', 'so4', 'chemically', 'bound', 'atoms', 'structurally', 'contained', 'products', 'incorporate', 'docking', 'accessory', 'couple', 'magnetic', 'detachably', 'attach', 'back', 'extend', 'resource', 'rus', 'pilots', 'ru', 'symbol', 'odd', 'numbered', 'lowest', 'subcarrier', 'even', 'six', 'primary', 'increases', 'colors', 'increasing', 'reduces', 'metameric', 'errors', 'viewer', 'red', 'green', 'blue', 'cyan', 'yellow', 'magenta', 'primaries', 'maintain', 'compatibility', 'existing', 'backwards', 'older', 'accomplishing', 'broad', 'reproducibility', 'entire', 'losing', 'brightness', 'backlight', 'ultraviolet', 'phosphors', 'combination', 'phosphor', 'change', 'emission', 'peak', '100â', '25â', 'wavelength', 'excitation', '455', 'most', '40', 'virtual', 'waveguide', 'opposing', 'coupling', 'ic', 'collimated', 'projection', 'orthogonal', 'pupil', 'expansion', 'ope', 'splitting', 'beamlets', 'exit', 'epe', 'aerosol', 'cartridge', 'housing', 'defining', 'mouthpiece', 'bottom', 'cap', 'portal', 'contain', 'therein', 'vaporization', 'member', 'bowed', 'orientation', 'ring', 'curved', 'shape', 'interior', 'conforming', 'semi', 'flexible', 'pcb', 'physical', 'perturbations', 'executable', 'instructions', 'transceiver', 'client', 'processor', 'thereto', 'potting', 'wall', 'encapsulates', 'wearable', 'partially', 'encircle', 'limb', 'carried', 'apply', 'types', 'forms', 'represented', 'structural', 'particular', 'pharmaceutical', 'disorders', 'crm1', 'activity', 'cancer', 'glasses', 'lenses', 'temples', 'acoustic', 'driver', 'sounds', 'guiding', 'direct', 'frequencies', 'temporary', 'setpoint', 'causing', 'difference', 'analyte', 'additionally', 'regulate', 'convenient', 'variety', 'manufacture', 'complicating', 'manufacturing', 'manufactured', 'partly', 'molded', 'covering', 'useful', 'headroom', 'aggregated', 'carriers', 'bitmap', 'equal', 'path', 'loss', 'triarylamine', 'derivative', 'general', 'g1', 'given', 'note', 'ar', 'represents', 'either', 'substituted', 'unsubstituted', 'phenyl', 'biphenyl', 'naphthyl', 'î²', 'independently', 'represent', 'dot', 'nibbles', 'tile', 'operands', 'decode', 'product', 'fields', 'opcode', 'identifier', 'identify', 'matrices', 'doubleword', 'execution', 'execute', 'specified', 'eight', 'multiplying', 'nibble', 'accumulate', 'saturate', 'contents', 'there', 'monochrome', 'selective', 'filter', 'directed', 'towards', 'blockchain', 'update', 'distributed', 'nodes', 'individually', 'copies', 'updating', 'consensus', 'determination', 'fraudulently', 'modified', 'corrective', 'block', 'disseminated', 'trigger', 'predefined', 'override', 'invoked', 'manner', 'bypasses', 'illegitimate', 'while', 'neuromodulation', 'treatment', 'assessment', 'procedure', 'prior', 'stimulation', 'completed', 'senses', 'electroencephalogram', 'eeg', 'measure', 'endogenous', 'brain', 'band', 'subsystem', 'adjusts', 'parameter', 'adjusted', 'applied', 'die', 'stacked', 'accesses', 'population', 'microporous', 'separator', 'separating', 'members', 'conductor', 'le', 'we', 'height', 'he', 'ratio', '1000', 'collector', 'lc', 'program', 'ecosystem', 'credit', 'tokens', 'modelled', 'outcomes', 'attribute', 'profile', 'quantities', 'permanence', 'token', 'validated', 'verified', 'event', 'methodology', 'boundary', 'impact', 'quantification', 'updated', 'clot', 'removal', 'catheter', 'vacuum', 'proximal', 'parameters', 'filled', 'column', 'carry', 'turning', 'automatic', 'responsive', 'picture', 'expander', 'tracking', 'spatially', 'colour', 'drive', 'parallel', 'viewing', 'window', 'view', 'guide', 'reflections', 'graded', 'transmissive', 'transmissivity', 'achromatic', 'constant', 'balance', 'seen', 'improved', 'mechanisms', 'collecting', 'diverse', 'suite', 'calculating', 'precipitation', 'atmospheric', 'water', 'vapor', 'precipitable', 'phenomena', 'presence', 'fog', 'these', 'readings', 'predicting', 'estimating', 'effects', 'visibility', 'runway', 'rvr', 'estimates', 'forecasts', 'vehicular', 'forward', 'cabin', 'windshield', 'traveling', 'road', 'frames', 'processed', 'interest', 'field', 'cmos', 'ahead', 'equipped', 'spectrum', 'highest', 'gray', 'glucose', 'access', 'activating', 'establishing', 'waking', 'sleep', 'forced', 'wakeup', 'provision', 'pieces', 'advice', 'pertaining', 'household', 'appliances', 'tablet', '103', 'piece', 'auto', 'log', 'overall', 'displayed', 'results', 'format', 'list', 'sorted', 'additive', 'three', 'objects', 'extruded', 'composite', 'strand', 'completely', 'encased', 'means', 'curing', 'extrusion', 'hardens', 'paths', 'chemical', 'instill', 'capabilities', 'possible', 'only', 'shaft', 'passes', 'details', 'implementation', 'portable', 'terminal', 'code', 'interaction', 'interactive', 'render', 'programs', 'recessed', 'fixture', 'package', 'all', 'leds', 'sink', 'dissipating', 'fins', 'fin', 'straight', 'reflector', 'geometry', 'bell', 'radii', 'curvature', 'join', 'together', 'inflection', 'optic', 'coupler', 'connections', 'machine', 'learning', 'prediction', 'wound', 'healing', 'diabetic', 'foot', 'ulcers', 'wounds', 'segmentation', 'assessing', 'collect', 'reflected', 'processors', 'depicting', 'reflectance', 'subset', 'quantitative', 'features', 'predicted', 'assessed', 'compact', 'printing', 'print', 'mask', 'solidifiable', 'powder', 'compaction', 'compacting', 'stage', 'repeatedly', 'tray', 'build', 'compounds', 'heteroleptic', 'iridium', 'complexes', 'ligands', 'pyridyl', 'dibenzo', 'ligand', 'dopants', 'lifetime', 'affordances', 'workstation', 'steps', 'affordance', 'preferences', 'database', 'settings', 'zone', 'setting', 'preference', 'match', 'white', 'monochromatic', 'compensator', 'technique', 'quality', 'yield', 'spacers', 'functioning', 'partition', 'covers', 'periphery', 'formation', 'supported', 'thereby', 'preventing', 'contacting', 'due', 'twist', 'deflection', 'damage', 'crack', 'occur', 'thus', 'defect', 'shapes', 'performs', 'resolution', 'worn', 'detectors', 'photons', 'scattered', 'positioning', 'sessions', 'assistant', 'chatbot', 'software', 'developer', 'little', 'experience', 'programming', 'interact', 'aural', 'textual', 'graphical', 'translate', 'natural', 'language', 'inputs', 'representation', 'solution', 'ontology', 'artificial', 'intelligence', 'scientist', 'develop', 'train', 'refine', 'compile', 'models', 'stand', 'alone', 'infusion', 'pump', 'modify', 'alarm', 'limit', 'insulin', 'decreases', 'moreover', 'missed', 'bolusâ', 'mealâ', 'relating', 'hardware', 'interconnected', 'dies', 'separate', 'interconnect', 'interalia', 'nucleic', 'radio', 'link', 'configure', 'report', 'insufficient', 'required', 'subsequently', 'employ', 'planning', 'optimization', 'purposes', 'dynamic', 'exposure', 'video', 'hdr', 'minimum', 'meets', 'criteria', 'switching', 'bent', 'opening', 'crossing', 'modal', 'filtration', 'fresh', 'inlet', 'finally', 'diverter', 'near', 'cause', 'selectively', 'coexistence', 'transmission', 'shared', 'environments', 'applicable', 'instance', 'wpts', 'delivers', 'clients', 'signals', 'channels', 'stations', 'local', 'wlans', 'exist', 'utilize', 'do', 'interfere', 'under', 'implemented', 'standardized', 'wlan', 'protocols', 'phy', 'mac', '802', '11', 'wi', 'fiâ', 'physician', 'views', 'anatomic', 'visualizing', 'medical', 'certain', 'cardiac', 'cavity', 'transaction', 'ledger', 'regarding', 'adaptive', 'cognitive', 'decentralized', 'optionally', 'serve', 'implement', 'contract', 'entity', 'aggregation', 'involves', 'covenant', 'conditionally', 'conveys', 'rights', 'ruling', 'protecting', 'overriding', 'super', 'users', 'configuration', 'conveyed', 'obstruction', 'volumetric', 'dataset', 'myocardium', 'coronary', 'muscular', 'created', 'feature', 'functionally', 'significant', 'lesion', 'severity', 'biosensor', 'detector', 'reaction', 'sites', 'crosstalk', 'analytes', 'viewpoints', 'patches', 'patch', 'mapping', 'vector', 'minimal', 'vectors', 'mapped', 'diffuse', 'bidirectional', 'brdf', 'computed', 'outputting', 'ig', 'molecules', 'ch3', 'mutations', 'monospecific', 'bispecific', 'videos', 'associations', 'audience', 'association', 'retrieving', 'executing', 'performance', 'score', 'benchmark', 'optimized', 'reconfiguration', 'rlm', 'node', 'message', 'compared', 'articles', 'footwear', 'apparel', 'sporting', 'decorating', 'foam', 'applying', 'embossing', 'debossing', 'painting', 'dyeing', 'aspects', 'abstract', 'scanning', 'searching', 'art', 'limiting', 'graphene', 'supplying', 'reduction', 'occurs', 'working', 'immersed', 'negative', 'domain', 'conservation', 'address', 'vehicles', 'assistance', 'adas', 'imminent', 'collision', 'avoidable', 'unavoidable', 'plot', 'course', 'minimizing', 'hazard', 'ai', 'avoid', 'braking', 'accelerating', 'finds', 'best', 'avoidance', 'harm', 'estimated', 'fatalities', 'injuries', 'caused', 'situation', 'analysis', 'selection', 'human', 'emergency', 'intervention', 'fully', 'fast', 'reflexes', 'awareness', 'save', 'lives', 'highway', 'widget', 'multimedia', 'viral', 'browser', 'retrieval', 'communicatively', 'retrieve', 'files', 'metadata', 'embedded', 'commerce', 'enters', 'preview', 'streamed', 'operative', 'complete', 'commercial', 'redundancy', 'schemes', 'unified', 'integrates', 'durable', 'bulk', 'resiliency', 'raid', 'stripe', 'written', 'moving', 'surgical', 'protection', 'define', 'barrier', 'external', 'helmet', 'hood', 'shield', 'static', 'tab', 'fastener', 'circulate', 'align', 'removable', 'skin', 'console', 'handpiece', 'manifold', 'transported', 'large', 'capacity', 'mwh', 'milliseconds', 'ms', 'assays', 'droplets', 'compartmentalized', 'sample', 'suspected', 'binding', 'agent', 'conditions', 'produce', 'agents', 'priori', 'geo', 'referenced', 'worksite', 'collected', 'metric', 'qualified', 'predicative', 'cinema', 'amplifier', 'distributes', 'rendering', 'immersive', 'audio', 'sound', 'listening', 'detects', 'net', 'requirements', 'dynamically', 'gains', 'environmental', 'speaker', 'feeds', 'status', 'error', 'public', 'private', 'guidance', 'running', 'requesting', 'provider', 'able', 'proximity', 'cleaning', 'upright', 'handle', 'dispenser', 'dispense', 'wiper', 'remove', 'brushroll', 'unrelated', 'electromechanical', 'surgery', 'section', 'tower', 'house', 'resistant', 'cowling', 'enclose', 'lights', 'protectively', 'housed', 'emerge', 'upward', 'seat', 'activated', 'allowing', 'disinfect', 'staff', 'timepoints', 'notifying', 'timepoint', 'reached', 'identification', 'interrupting', 'presentation', 'resuming', 'distilling', 'concepts', 'math', 'problems', 'constructing', 'testing', 'creation', 'problem', 'extracting', 'compiling', 'concept', 'cloud', 'clis', 'mathematical', 'describe', 'udp', 'respectively', 'blocks', 'rules', 'returning', 'built', 'extracted', 'kits', 'efficiently', 'discarding', 'packets', 'destined', 'master', 'secondary', 'configures', 'discard', 'forwards', 'discards', 'expiry', 'timer', 'started', 'reception', 'cargo', 'loaded', 'cholesteryl', 'ester', 'nanoparticle', 'hollow', 'compartment', 'cholestosomeâ', 'ionic', 'encapsulated', 'cannot', 'appreciably', 'pass', 'enterocyte', 'membrane', 'absence', 'molecule', 'cholestosome', 'neutral', 'ability', 'enterocytes', 'orally', 'absorbed', 'nutrient', 'lipids', 'reach', 'golgi', 'pursuant', 'cholestosomes', 'depositing', 'effecting', 'therapy', 'diagnosis', 'ir', 'species', 'focal', 'fpa', 'behind', 'spectrally', 'incident', 'acquire', 'multispectral', 'false', 'detections', 'optimizing', 'analyzing', 'degradation', 'calculated', 'figure', 'terrain', 'rf', 'asd', 'conflict', 'edge', 'extract', 'meta', 'anomaly', 'recognition', 'assemblies', 'steerable', 'steered', 'rotation', 'axial', 'compare', 'profiles', 'calculate', 'comparison', 'result', 'interfaces', 'ascertaining', 'identity', 'competence', 'health', 'mind', 'granted', 'denied', 'creating', 'arrangements', 'improving', 'meeting', 'masks', 'openings', 'changes', 'hvac', 'climate', 'conditioned', 'compares', 'expected', 'past', 'inflatable', 'bladder', 'entirely', 'digital', 'workspace', 'ergonomics', 'dweâ', 'transform', 'touchscreen', 'gestures', 'dwe', 'collaboration', 'whiteboard', 'participating', 'collaborative', 'whiteboarding', 'session', 'parses', 'modifies', 'generates', 'viewport', 'should', 'because', 'modifying', 'chair', 'bearing', 'zones', 'closest', 'travel', 'auxiliary', 'mesh', 'vfd', 'trailer', 'ventilation', 'overpressure', 'discourage', 'entry', 'dust', 'debris', 'floor', 'vibration', 'damping', 'chassis', 'intervening', 'suspension', 'readable', 'managing', 'canvas', 'latency', 'despite', 'artwork', 'vary', 'expressiveness', 'variation', 'occupancy', 'transitioning', 'inactive', 'indicates', 'scan', 'cycle', 'recording', 'humans', 'occupying', 'but', 'absent', 'scheduler', 'wallbox', 'dimmer', 'accommodate', 'button', 'configurations', 'controllably', 'yoke', 'antenna', 'flange', 'oriented', 'angularly', 'offset', 'faceplate', 'cooperates', 'broader', 'mimo', 'ran', 'virtualized', 'baseband', 'fronthaul', 'connects', 'antennas', 'csi', 'computes', 'sparse', 'sums', 'better', 'average', 'spatial', 'subchannels', 'selects', 'hinged', 'clamps', 'employed', 'installation', 'right', 'notch', 'hinge', 'threaded', 'downwardly', 'securing', 'inserted', 'force', 'imparts', 'buildup', 'engage', 'biomarkers', 'hematoxylin', 'eosin', 'stained', 'slide', 'executed', 'segmented', 'biomarker', 'transitory', 'parser', 'communicating', 'splits', 'communicated', 'distinctly', 'encryption', 'original', 'protect', 'replacement', 'orthopedic', 'fixtures', 'locate', 'adjust', 'planes', 'bony', 'distract', 'otherwise', 'procedures', 'microlens', 'inorganic', 'collects', 'independent', 'critical', 'situations', 'encountered', 'digitally', 'verifying', 'credential', 'credentials', 'linked', 'identifiers', 'meet', 'signed', 'sole', 'discrete', 'hexagonally', 'sipes', 'transverse', 'across', 'oblique', 'subdivide', 'diamond', 'hexagonal', 'lugs', 'cemented', 'refractive', 'index', 'waves', 'impinge', 'incidence', 'angles', 'smaller', 'impinging', 'pinless', 'plug', 'jack', 'coil', 'inductively', 'shielded', 'annular', 'anchor', 'perimeter', 'concentric', 'overlapping', 'magnetically', 'snag', 'magnet', 'quantum', 'quinoxaline', 'favorably', 'el', 'substance', 'exciplex', 'homo', 'obesity', 'intragastric', 'fabricating', 'deploying', 'inflating', 'interrogate', 'gene', 'expression', 'immunogenic', 'rsv', 'protein', 'ecto', 'polypeptides', 'engineered', 'proteins', 'acids', 'encode', 'prepared', 'predominant', 'desired', 'conformation', 'inducing', 'immune', '2d', 'depiction', 'teeth', 'arrangement', 'contours', 'mouth', 'post', 'realistic', 'preserve', 'rich', 'regress', 'trained', 'predict', 'deployment', 'correspond', 'label', 'classes', 'embedding', 'algorithm', 'clusters', 'predictions', 'aid', 'understanding', 'surrounding', 'environmentâ', 'world', 'modelâ', 'allocation', 'case', 'polymerisable', 'mountable', 'programmable', 'addressable', 'configurable', 'positionable', 'irradiate', 'uncured', 'previously', 'cured', 'magnification', 'employs', 'detachable', 'precursor', 'vaporize', 'derivation', 'temporal', 'tmd', 'pu', 'encoding', 'derived', 'inclusion', 'inter', 'largest', 'coding', 'lcu', 'size', 'coordinates', 'selecting', 'neighboring', 'incentivized', 'training', 'assurance', 'incentives', 'miners', 'streams', 'enhancing', 'identified', 'augmenting', 'mining', 'technical', 'designating', 'interactions', 'visual', 'representations', 'miner', 'stream', 'printed', 'document', 'printer', 'scanner', 'printhead', 'validation', 'stronger', 'powerful', 'laser', 'mesa', 'isolated', 'automated', 'relationships', 'sender', 'domains', 'recipient', 'scoring', 'compute', 'weighted', 'grade', 'actions', 'microorganisms', 'fermentation', 'broth', 'they', 'advantageously', 'quick', 'versatile', 'efficacious', 'helpful', 'resistance', 'residue', 'concerns', 'consumers', 'ground', 'engaging', 'cab', 'rear', 'powertrain', 'center', 'gravity', 'nasal', 'formulations', 'epinephrine', 'anaphylaxis', 'retail', 'shelf', 'select', 'depicted', 'analytics', 'telemetry', 'servers', 'dependency', 'map', 'adm', 'resolve', 'flowlets', 'granularities', 'baseline', 'metrics', 'sizes', 'new', 'baselines', 'assess', 'automate', 'remediation', 'unavailability', 'issues', 'experiences', 'peripheral', 'recreated', 'peripherals', 'see', 'eye', 'technologies', 'presented', 'personal', 'relationship', 'heater', 'ended', 'reaching', 'following', 'fixed', 'nodal', 'learn', 'statistical', 'fourier', 'fft', 'trial', 'identifies', 'manager', 'participation', 'request', 'acceptance', 'shaving', 'shaver', 'close', 'integral', 'casing', 'wired', 'dedicated', 'powerline', 'illuminate', 'compression', 'carrying', 'abrasive', 'particle', 'bonded', 'register', 'reserved', 'arbitration', 'seam', 'unlock', 'logical', 'initiate', 'manifest', 'supports', 'trust', 'initialize', 'vmcs', 'vmm', 'authenticate', 'feedback', 'gps', 'commodity', 'market', 'grouping', 'hubs', 'pricing', 'audit', 'hub', 'arbitrage', 'simple', 'easy', 'mediums', 'transact', 'trade', 'seats', 'airline', 'subway', 'automobile', 'taxi', 'freight', 'tractor', 'underground', 'ship', 'sea', 'drone', 'reality', 'porting', 'locally', 'links', 'receipt', 'accomplished', 'detectable', 'uavs', 'captures', 'measures', 'transmitted', 'analyzes', 'uav', 'analyzed', 'encoded', 'designing', 'definition', 'numerical', 'simulation', 'loading', 'isotropic', 'orientations', 'behavior', 'proposals', 'precomputed', 'generable', 'modeling', 'ferroelectric', 'doped', 'polar', 'capacitor', 'turn', 'dopant', 'oxygen', 'nitrogen', 'mv', 'signaling', 'toy', 'play', 'ball', 'handheld', 'chargeable', 'contactless', 'induction', 'converting', 'kinetic', 'predictably', 'randomly', 'preset', 'numeric', 'exceeded', 'forth', 'aâ', 'libh', 'bâ', 'lix', 'câ', 'linh', 'fluorine', 'bromine', 'chloride', 'iodine', '1â', '13â', '0â', 'separators', 'lithium', 'electrochemical', 'experienced', 'events', 'indicative', 'violates', 'policy', 'conduction', 'states', 'ultrasound', 'capacitive', 'micromachined', 'transducers', 'cmuts', 'gui', 'selectable', 'menu', 'options', 'transducer', 'labeled', 'musculoskeletal', 'breast', 'carotid', 'vascular', 'abdominal', 'graphics', 'cluster', 'streaming', 'multiprocessors', 'partitioning', 'partitions', 'protected', 'coils', 'central', 'morphology', 'physiology', 'overlaps', 'statically', 'ceiling', 'downrod', 'arms', 'blade', 'holders', 'blades', 'rotatable', 'wire', 'disk', 'guy', 'retention', 'rod', 'stator', 'scheduling', 'services', 'ultra', 'urllc', 'nr', '5g', 'ue', 'restriction', 'designs', 'synchronization', 'burst', 'listen', 'paging', 'seamlessly', 'routes', 'trip', 'dependent', 'outcome', 'feed', 'specify', 'augmentation', 'authenticated', 'respect', 'authentication', 'augmented', 'fibrous', 'graphic', 'nonwoven', 'filaments', 'filament', 'releasable', 'onto', 'ear', 'wax', 'clogging', 'ports', 'hearing', 'conformal', 'houses', 'tympanic', 'expandable', 'encompass', 'circumferential', 'expanded', 'inflation', 'expand', 'actuated', 'prevents', 'contamination', 'deposited', 'deposition', 'need', 'chambers', 'chance', 'cross', 'article', 'voice', 'assistants', 'word', 'intent', 'adding', 'surrounded', 'detour', 'rigid', 'bodies', 'prosthesis', 'bone', 'resulting', 'bones', 'reflecting', 'conveyance', 'concave', 'mirror', 'correct', 'distortion', 'viewpoint', 'double', 'reducer', 'dunnage', 'pulling', 'coiled', 'ends', 'discontinuous', 'gap', 'cutter', 'phases', 'serrations', 'coextensive', 'ledges', 'focusing', 'bunching', 'stapler', 'effector', 'jaws', 'firing', 'starting', 'ending', 'rotary', 'lockout', 'unless', 'staple', 'ready', 'fire', 'confidence', 'cdns', 'allocates', 'allocated', 'vod', 'file', 'movie', 'script', 'potentially', 'biohazard', 'geographic', 'issuing', 'notification', 'overlaid', 'underlying', 'facilitating', 'replication', 'replicas', 'replica', 'manners', 'span', 'broadcast', 'distinct', 'transparently', 'replace', 'becomes', 'unavailable', 'implantable', 'transponders', 'ferromagnetic', 'implants', 'interference', 'small', 'implanted', 'readers', 'screen', 'flexibility', 'prevented', 'cracked', 'upload', 'photos', 'social', 'account', 'her', 'his', 'community', 'accounts', 'shown', 'viewers', 'clutch', 'clutched', 'engaged', 'quasi', 'passive', 'elastic', 'enter', 'released', 'torque', 'disengaged', 'inelastic', 'facilitates', 'free', 'swing', 'fluorescent', 'delayed', 'fluorescence', 'triplet', 'annihilation', '20', 'singlet', 'excited', 'benzo', 'anthracene', 'acetone', 'breath', 'bodily', 'samples', 'runs', 'smartphone', 'reports', 'cores', 'type', 'sequenced', 'maintained', 'pool', 'libraries', 'biomedical', 'prevention', 'diseases', 'threat', 'indicator', 'batch', 'rule', 'indictor', 'magnetizable', 'therapeutic', 'benefits', 'implant', 'expanding', 'joints', 'responses', 'changing', 'placed', 'electromagnet', 'energized', 'relieve', 'forces', 'modular', 'adds', 'removeably', 'plmn', 'requested', 'international', 'subscriber', 'imsi', 'offsetting', 'accepted', 'occasions', 'alternative', 'sum', 'live', 'perception', 'leveraged', 'classify', 'intersections', 'dnn', 'outputsâ', 'bounding', 'box', 'coverage', 'maps', 'boxes', 'attributes', 'distance', 'final', 'cleaner', 'lateral', 'plate', 'mop', 'writes', 'permit', 'overwrite', 'nonvolatile', 'flash', 'zoned', 'compliant', 'architectures', 'release', 'maintenance', 'buffering', 'accommodating', 'unentanglement', 'rates', 'consistent', 'newer', 'standards', 'mandate', 'addresses', 'sequential', 'uplink', 'wtru', 'pusch', 'tx', 'rx', 'trps', 'modulation', 'scheme', 'mcs', 'tpc', 'fractional', 'compensation', 'factor', 'mobility', 'codeword', 'correlation', 'discover', 'still', 'discovered', 'discoveries', 'properly', 'those', 'synergistic', 'effect', 'greatly', 'discovery', 'merging', 'self', 'extraction', 'cleaners', 'attachments', 'tools', 'wands', 'hoses', 'redirect', 'limited', 'wand', 'hose', 'reconfigurable', 'accessible', 'transactions', 'restricted', 'apportion', 'bandwidth', 'inverter', 'commanded', 'disease', 'melanoma', 'acquires', 'wavelengths', 'sweeping', 'rotating', 'clock', 'sweep', 'diagnostically', 'classifiers', 'enhance', 'fuel', 'navigation', 'arrival', 'purchase', 'full', 'stop', 'fueling', 'heel', 'medial', 'slats', 'progressive', 'gradient', 'lengths', 'widths', 'pegs', 'elongated', 'extension', 'collar', 'items', 'item', 'reflects', 'toxicity', 'solely', 'safeness', 'rating', 'exemplary', 'spreaders', 'lids', 'conducting', 'spreader', 'tim1', 'interconnects', 'gaming', 'establishment', 'write', 'start', 'volatile', 'starts', 'authenticating', 'assay', 'biological', 'abundance', 'targets', 'depend', 'multiplexing', 'reagents', 'tag', 'instrumentation', 'readout', 'nature', 'polynucleotide', 'goggles', 'nosepiece', 'fittable', 'wearer', 'strands', 'conform', 'eyepieces', 'anatomical', 'vital', 'constituted', 'acquiring', 'describing', 'tracked', 'sturdy', 'intoxication', 'sobriety', 'task', 'characterizing', 'supplementary', 'demographic', 'physiological', 'duration', 'associate', 'skip', 'later', 'playback', 'asset', 'retrieves', 'overshoot', 'compensates', 'retrieved', 'advancing', 'played', 'rotate', 'capping', 'decapping', 'decap', 'held', 'elevator', 'chuck', 'lowered', 'positon', 'opposed', 'spindles', 'beveled', 'shoulders', 'dispersion', 'producible', 'correspondence', 'characters', 'character', 'segment', 'visually', 'spinal', 'distraction', 'facet', 'articular', 'contour', 'tubular', 'distractor', 'elongate', 'separation', 'several', 'interbody', 'fusion', 'establish', 'clouds', 'providers', 'overlay', 'spans', 'divisions', 'departments', 'datacenters', 'saas', 'machines', 'destinations', 'trying', 'minimize', 'passing', 'allows', 'proven', 'biometrics', 'fraud', 'risk', 'exists', 'today', 'someone', 'license', 'lot', 'contextual', 'behavioral', 'party', 'authorization', 'termination', 'soft', 'applicator', 'fastening', 'fix', 'microprocessor', 'radiofrequency', 'muscle', 'contraction', 'chin', 'piston', 'combustion', 'engines', 'stroke', 'sufficient', 'linear', 'administrative', 'referred', 'adminâ', 'manage', 'valves', 'admin', 'fluorophore', 'configuring', 'distributions', 'zero', 'relation', 'hypothesized', 'inverse', 'stably', 'stable', 'transistors', 'subjected', 'dehydrated', 'dehydrogenated', 'impurities', 'moisture', 'supercritical', 'imagery', 'project', 'differ', 'switchable', 'directs', 'reflect', 'adsorption', 'synthetic', 'brines', 'recovering', 'brine', 'adsorbent', 'countercurrent', 'desorption', 'discusses', 'transvenous', 'transdural', 'parenchyma', 'subarachnoid', 'subdural', 'spaces', 'biopsy', 'nanofluidic', 'microelectronic', 'insertion', 'implantation', 'situ', 'abnormal', 'extravascular', 'axially', 'deflector', 'deflect', 'segmenting', 'scale', 'psilocybin', 'medicine', 'purity', 'polymorph', 'sampled', 'division', 'multitree', 'logging', 'onboard', 'recorder', 'electronically', 'mileage', 'hours', 'tax', 'wellness', 'interventions', 'metering', 'codeset', 'provisioned', 'designates', 'codes', 'designated', 'valid', 'trick', 'relatively', 'avoiding', 'forwarding', 'employing', 'sophisticated', 'download', 'what', 'versus', 'would', 'conventionally', 'cache', 'nearby', 'reverse', 'reside', 'description', 'excavation', 'site', 'navigating', 'obstacle', 'obstacles', 'unobstructed', 'circumventing', 'obstructed', 'modifiable', 'shortest', 'navigates', 'msk', 'strength', 'endurance', 'care', 'plan', 'anywhere', 'app', 'infrastructure', 'tuned', 'elevation', 'algorithms', 'class', 'category', 'bitstream', 'signaled', 'decoder', 'downsampled', 'multiplexed', 'stereoscopic', 'upconversion', 'correction', 'contribute', 'restoration', 'transcript', 'suggested', 'send', 'objectively', 'itselfâ', 'definitions', 'constitute', 'measurable', 'actionâ', 'unexplored', 'regularity', 'timescale', 'organized', 'yields', 'important', 'offers', 'insight', 'objective', 'subtle', 'alterations', 'classification', 'screening', 'study', 'early', 'toxicology', 'research', 'anxiety', 'consumer', 'publishing', 'library', 'initiated', 'permissions', 'publish', 'restrict', 'published', 'promote', 'stages', 'hypnograms', 'scores', 'recorded', 'evaluated', 'queries', 'good', 'habits', 'risky', 'versions', 'bedside', '3000', 'phone', '3002', 'methodologies', 'workload', 'historical', 'behaviors', 'construct', 'utilization', 'efficiencies', 'supplemental', 'game', 'warnings', 'player', 'haptic', 'indicate', 'satisfied', 'numerous', 'removed', 'packaging', 'acquired', 'shopper', 'packages', 'interacted', 'quantity', 'shopping', 'cart', 'calculates', 'creates', 'multiplicity', 'triggering', 'compromise', 'complexity', 'achievable', 'achieved', 'divisioning', 'namely', 'leaf', 'intermediate', 'collocated', 'tilesâ', 'regionsâ', 'however', 'explicitly', 'whole', 'separately', 'similarities', 'effectively', 'exploited', 'subdivision', 'improve', 'histogram', 'prevalent', 'barcode', 'iot', 'delay', 'timestamp', 'suction', 'separates', 'recombinant', 'collagenase', 'ii', 'preparation', 'encompasses', 'releasing', 'administration', 'television', 'framework', 'keys', 'gesture', 'functionality', 'style', 'known', 'handwriting', 'text', 'candidates', 'keyboards', 'influence', 'truth', 'grouped', 'chunks', 'temporally', 'iteratively', 'estimate', 'gaze', 'occurrences', 'advertisement', 'guest', 'lithotripsy', 'kidney', 'ureter', 'was', 'breaker', 'sscb', 'downstream', 'breakers', 'cbs', 'switched', 'short', 'square', 'rms', 'pulsed', 'regulated', 'hysteresis', 'enough', 'tripping', 'prematurely', 'isolate', 'pulsing', 'continue', 'none', 'incorporates', 'downlink', 'affected', 'initiating', 'filtering', 'encoder', 'analyzer', 'spectral', 'former', 'hydraulic', 'fracturing', 'reciprocating', 'plunger', 'prime', 'mover', 'drivetrain', 'turbine', 'concealing', 'laptop', 'gateway', 'ethernet', 'cable', 'wifi', 'disguising', 'giving', 'them', 'addressees', 'muting', 'participants', 'conference', 'call', 'watch', 'parties', 'sync', 'cpu', 'synchronize', 'presentations', 'pools', 'failover', 'floating', 'instantiate', 'instantiating', 'drawn', 'namespace', 'ingress', 'desktop', 'sit', 'desk', 'adjustable', 'scissoring', 'raise', 'unlocking', 'keyboard', 'rna', 'probe', 'immobilized', '3â', 'occupies', 'mm', 'positional', 'nucleotide', 'unique', 'complementary', 'probability', 'somatic', 'mutational', 'probabilities', 'inform', 'courses', 'maximize', 'networking', 'bridge', 'qos', 'cellular', 'gating', 'informs', 'egress', 'exclusive', 'rrc', 'grant', 'pdu', 'addressed', 'rnti', 'skipping', 'sdu', 'padding', 'buffer', 'bsr', 'patent', 'specifically', 'exchanged', 'needed', 'cis', 'dvs', 'vision', 'photoelectric', 'charges', 'exterior', 'rearview', 'fixedly', 'innermost', 'tandem', 'extends', 'circumscribes', 'encroach', 'payload', 'entries', 'payloads', 'fluorenes', 'furthermore', 'formulation', 'varies', 'favorable', 'boards', 'pairs', 'overcurrent', 'drop', 'dpp', 'sockets', 'observation', 'circumferentially', 'spots', 'nanobeam', 'diffraction', 'nmï', 'encapsulation', 'sealing', 'mechanically', 'slidingly', 'introducing', 'dsp', 'accelerator', 'caf', 'architecture', 'sculpting', 'dimension', 'dimensions', 'extracts', 'welding', 'weld', 'viewable', 'displays', 'taken', 'simulated', 'anucleated', 'platforms', 'targeted', 'achromosomal', 'fecal', 'stabilizing', 'denaturation', 'found', 'swimming', 'inkjet', 'curable', 'polymerize', 'metathesis', 'polymerization', 'romp', 'toughening', 'usable', 'drx', 'discovering', 'optimal', 'solutions', 'satisfy', 'demands', 'ngso', 'gso', 'satellite', 'constellations', 'telecommunication', 'stationary', 'constellation', 'requires', 'assignment', 'optimally', 'scarce', 'constrained', 'efficient', 'take', 'maximizing', 'geostationary', 'rebate', 'marketing', 'enroll', 'loyalty', 'completes', 'customer', 'campaigns', 'recipients', 'bonus', 'added', 'rebates', 'stud', 'forge', 'welded', 'bracket', 'ladder', 'tenant', 'business', 'handler', 'analytic', 'publisher', 'subscribers', 'tensor', 'dividing', 'inference', 'composable', 'goal', 'statements', 'quickly', 'story', 'outlines', 'nlg', 'narrative', 'author', 'conditional', 'narratives', 'intelligently', 'adapt', 'consideration', 'permits', 'appropriate', 'audible', 'look', 'waypoints', 'amf', 'handover', 'smf', 'completion', 'horizontal', 'containers', 'columns', 'beneath', 'handling', 'lifting', 'footprint', 'beyond', 'complimentary', 'jobs', 'decouples', 'job', 'effort', 'minimized', 'masters', 'computations', 'recommendations', 'improvements', 'upgrade', 'fabricate', 'poly', 'hydroxybutyrate', 'p4hb', 'copolymers', 'developed', 'fused', 'fabrication', 'overcome', 'poor', 'feeding', 'softening', 'creep', 'fed', 'melt', 'pellet', 'fpd', 'med', 'melting', 'slm', 'slurries', 'coagulation', 'bath', 'granules', 'rs', 'bler', 'thresholds', 'comparisons', 'indications', 'contactor', 'urge', 'terminals', 'relief', 'valve', 'reaches', 'moved', 'disengage', 'interengaging', 'formations', 'restricts', 'tangential', 'li', 'ion', 'si', 'areal', 'mah', 'cm', '12', '600', 'cc', '1800', 'binder', 'bind', 'stabilize', 'endpoints', 'intercepts', 'sanctioned', 'references', 'decoy', 'monitors', 'intensities', 'registered', 'just', 'decrease', 'unchanged', 'proposed', 'faster', 'traditional', 'correlate', 'bit', 'successive', 'interconnecting', 'fuse', 'insulator', 'adhered', 'supporting', 'slave', 'acts', 'carrier', 'translator', 'framing', 'formatting', 'packetization', 'recognizes', 'eyes', 'turns', 'recognized', 'recommending', 'restrictions', 'chatter', 'posted', 'posting', 'oled', 'bending', 'exposing', 'insulation', 'gesturally', 'interacting', 'biometric', 'yet', 'pose', 'grab', 'pinch', 'retractable', 'comprised', 'nonlinear', 'microphones', 'noisy', 'substantial', 'speech', 'intelligibility', 'polarity', 'whereby'])\n"
     ]
    }
   ],
   "source": [
    "print(len(vect.vocabulary_))\n",
    "print(vect.vocabulary_.keys())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "id": "fd10eda1-8069-4f47-acc6-53848affdc2a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<360x3981 sparse matrix of type '<class 'numpy.int64'>'\n",
      "\twith 18113 stored elements in Compressed Sparse Row format>\n"
     ]
    }
   ],
   "source": [
    "print(repr(bof))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c4bed57a-83f8-4418-8c3c-e8a7d8b064ef",
   "metadata": {},
   "source": [
    "As we can see, there are many uninformative words in the dictionary, like  'of', 'can', 'be' etc.\n",
    "\n",
    "We will use multiple methods to filter out uninformative words."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "id": "c1e61e76-4388-4e95-909a-027ce025386c",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "574\n",
      "dict_keys(['high', 'efficiency', 'solar', 'power', 'sources', 'base', 'phase', 'converter', 'low', 'energy', 'storage', 'circuitry', 'controlled', 'control', 'operates', 'provide', 'conversion', 'output', 'provided', 'individual', 'source', 'operational', 'modes', 'presenting', 'panel', 'variable', 'continuous', 'heat', 'electrical', 'solid', 'medium', 'continuously', 'array', 'internal', 'radiation', 'directly', 'thermal', 'facilitate', 'heating', 'delivery', 'temperature', 'used', 'processes', 'including', 'generation', 'based', 'supply', 'using', 'current', 'advance', 'information', 'voltage', 'distribution', 'transfer', 'method', 'includes', 'generating', 'time', 'signal', 'device', 'comprises', 'identifying', 'simultaneously', 'operating', 'mode', 'additional', 'comprising', 'plurality', 'values', 'according', 'determining', 'predictive', 'controller', 'processing', 'circuits', 'configured', 'obtain', 'defines', 'electric', 'load', 'period', 'multiple', 'specific', 'components', 'include', 'component', 'indicating', 'second', 'perform', 'process', 'subject', 'determine', 'equipment', 'use', 'frame', 'having', 'position', 'receiving', 'material', 'surface', 'corresponding', 'battery', 'included', 'circuit', 'systems', 'methods', 'described', 'embodiments', 'module', 'electrically', 'coupled', 'operable', 'state', 'stored', 'monitor', 'apparatus', 'smart', 'grid', 'communication', 'disclosed', 'present', 'invention', 'element', 'constructed', 'network', 'server', 'active', 'automatically', 'connection', 'sends', 'receives', 'provides', 'various', 'relate', 'generate', 'storing', 'generated', 'receive', 'controlling', 'application', 'techniques', 'networks', 'example', 'assembly', 'lower', 'structure', 'shaped', 'length', 'extending', 'end', 'upper', 'pair', 'portion', 'substantially', 'structures', 'physically', 'powered', 'formed', 'relative', 'respective', 'associated', 'wireless', 'camera', 'image', 'capture', 'images', 'store', 'captured', 'transmit', 'detection', 'motion', 'command', 'detecting', 'technology', 'logic', 'sensor', 'data', 'reduced', 'management', 'communications', 'communicates', 'real', 'pattern', 'comprise', 'content', 'security', 'transport', 'attachment', 'point', 'cells', 'regions', 'semiconductor', 'forming', 'tool', 'layer', 'substrate', 'cell', 'electron', 'providing', 'air', 'pressure', 'multi', 'maintaining', 'pre', 'set', 'carbon', 'predetermined', 'operations', 'media', 'mechanical', 'computer', 'light', 'level', 'selected', 'support', 'single', 'axis', 'cases', 'body', 'beam', 'interface', 'connecting', 'portions', 'enable', 'connect', 'located', 'work', 'different', 'like', 'transparent', 'electrode', 'spaced', 'apart', 'electrodes', 'containing', 'non', 'planar', 'elements', 'embodiment', 'particles', 'fluid', 'devices', 'target', 'tissue', 'examples', 'region', 'greater', 'treating', 'vehicle', 'suitable', 'service', 'capable', 'driving', 'motor', 'station', 'speed', 'connected', 'mounted', 'determines', 'value', 'transmits', 'determined', 'memory', 'host', 'sending', 'response', 'number', 'threshold', 'model', 'obtaining', 'cost', 'ambient', 'characteristics', 'safety', 'neural', 'input', 'layers', 'obtained', 'condition', 'accordance', 'allow', 'user', 'board', 'mechanism', 'port', 'create', 'tank', 'characteristic', 'autonomous', 'adapted', 'implementations', 'frequency', 'disposed', 'group', 'thereof', 'object', 'adjacent', 'location', 'performing', 'functions', 'mobile', 'platform', 'area', 'sensors', 'positioned', 'movement', 'operation', 'imaging', 'dimensional', 'points', '3d', 'generally', 'relates', 'particularly', 'aspect', 'thereon', 'oxide', 'connector', 'conductive', 'function', 'contact', 'enhanced', 'angle', 'display', 'integrated', 'monitoring', 'record', 'indication', 'core', 'space', 'sensing', 'detected', 'accordingly', 'head', 'remote', 'computing', 'collection', 'intended', 'acid', 'wiring', 'switch', 'actuator', 'unit', 'secure', 'joint', 'patient', 'measurement', 'liquid', 'volume', 'container', 'emit', 'detect', 'emitted', 'received', 'intensity', 'emitting', 'operatively', 'performed', 'resources', 'decoding', 'measurements', 'receiver', 'apparatuses', 'applications', 'disclosure', 'production', 'materials', 'related', 'compositions', 'novel', 'does', 'web', 'defined', 'range', 'environment', 'optical', 'engine', 'lumen', 'transmitting', 'distal', 'subsequent', 'series', 'electronic', 'sequence', 'highly', 'reliable', 'share', 'common', 'contains', 'compound', 'sub', 'pixels', 'line', 'pixel', 'organic', 'transistor', 'color', 'overlap', 'channel', 'locations', 'flow', 'comparing', 'concentration', 'said', 'reliability', 'displaying', 'arranged', 'units', 'insulating', 'face', 'direction', 'surfaces', 'plane', 'film', 'metal', 'form', 'composition', 'higher', 'products', 'magnetic', 'extend', 'resource', 'primary', 'combination', 'change', 'peak', 'wavelength', 'virtual', 'cartridge', 'housing', 'defining', 'contain', 'member', 'shape', 'flexible', 'physical', 'instructions', 'client', 'processor', 'particular', 'activity', 'driver', 'direct', 'causing', 'difference', 'analyte', 'additionally', 'manufacture', 'manufacturing', 'useful', 'equal', 'path', 'derivative', 'represents', 'independently', 'product', 'identifier', 'identify', 'execute', 'selective', 'directed', 'distributed', 'updating', 'determination', 'block', 'manner', 'treatment', 'procedure', 'measure', 'parameter', 'applied', 'members', 'ratio', 'program', 'profile', 'event', 'updated', 'proximal', 'parameters', 'automatic', 'responsive', 'picture', 'drive', 'view', 'guide', 'improved', 'processed', 'field', 'spectrum', 'access', 'displayed', 'results', 'objects', 'means', 'code', 'machine', 'learning', 'subset', 'features', 'predicted', 'printing', 'compounds', 'database', 'technique', 'quality', 'worn', 'positioning', 'software', 'graphical', 'representation', 'intelligence', 'separate', 'nucleic', 'radio', 'dynamic', 'video', 'opening', 'near', 'cause', 'transmission', 'signals', 'implemented', 'medical', 'certain', 'entity', 'users', 'configuration', 'performance', 'node', 'message', 'compared', 'articles', 'aspects', 'domain', 'address', 'analysis', 'selection', 'fast', 'external', 'sample', 'conditions', 'collected', 'audio', 'status', 'provider', 'identification', 'secondary', 'incident', 'analyzing', 'interfaces', 'identity', 'creating', 'changes', 'digital', 'generates', 'readable', 'latency', 'spatial', 'selects', 'post', 'trained', 'configurable', 'encoding', 'coding', 'size', 'identified', 'visual', 'stream', 'servers', 'new', 'presented', 'identifies', 'request', 'encoded'])\n"
     ]
    }
   ],
   "source": [
    "vect = CountVectorizer(min_df = 5, stop_words = \"english\")\n",
    "vect.fit(ab)\n",
    "bof = vect.transform(ab)\n",
    "print(len(vect.vocabulary_))\n",
    "print(vect.vocabulary_.keys())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "id": "851ce661-8453-4c06-ae44-62989348d22c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<360x574 sparse matrix of type '<class 'numpy.int64'>'\n",
      "\twith 7417 stored elements in Compressed Sparse Row format>\n"
     ]
    }
   ],
   "source": [
    "print(repr(bof))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fba10059-507a-450a-8716-54b5f0068f73",
   "metadata": {},
   "source": [
    "We could see some words should be together, like 'high efficiency', 'solar power' and 'low energy' etc.\n",
    "Let's use n-grams to allow multi-word tokens."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "id": "896baf15-83f8-4a5b-8074-92017443536c",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "653\n",
      "dict_keys(['high', 'efficiency', 'solar', 'power', 'sources', 'base', 'phase', 'converter', 'low', 'energy', 'storage', 'circuitry', 'controlled', 'control', 'operates', 'provide', 'conversion', 'output', 'provided', 'individual', 'source', 'operational', 'modes', 'presenting', 'panel', 'energy storage', 'variable', 'continuous', 'heat', 'electrical', 'solid', 'medium', 'continuously', 'array', 'internal', 'radiation', 'directly', 'thermal', 'facilitate', 'heating', 'delivery', 'temperature', 'used', 'processes', 'including', 'generation', 'based', 'supply', 'using', 'current', 'advance', 'information', 'voltage', 'distribution', 'transfer', 'method', 'includes', 'generating', 'time', 'signal', 'device', 'comprises', 'identifying', 'simultaneously', 'operating', 'mode', 'additional', 'comprising', 'plurality', 'values', 'according', 'determining', 'method includes', 'storage device', 'predictive', 'controller', 'processing', 'circuits', 'configured', 'obtain', 'defines', 'electric', 'load', 'period', 'multiple', 'specific', 'components', 'include', 'component', 'indicating', 'second', 'perform', 'process', 'subject', 'determine', 'equipment', 'use', 'frame', 'having', 'position', 'receiving', 'material', 'surface', 'corresponding', 'battery', 'included', 'circuit', 'systems', 'methods', 'described', 'embodiments', 'module', 'electrically', 'coupled', 'operable', 'state', 'stored', 'monitor', 'systems methods', 'power source', 'apparatus', 'smart', 'grid', 'communication', 'disclosed', 'present', 'invention', 'element', 'constructed', 'network', 'server', 'active', 'automatically', 'connection', 'sends', 'receives', 'provides', 'methods apparatus', 'present invention', 'systems methods apparatus', 'various', 'relate', 'generate', 'storing', 'generated', 'receive', 'controlling', 'application', 'techniques', 'networks', 'various embodiments', 'configured generate', 'device configured', 'configured receive', 'example', 'assembly', 'lower', 'structure', 'shaped', 'length', 'extending', 'end', 'upper', 'pair', 'portion', 'substantially', 'structures', 'physically', 'powered', 'formed', 'relative', 'respective', 'includes plurality', 'associated', 'wireless', 'camera', 'image', 'capture', 'images', 'store', 'captured', 'transmit', 'detection', 'motion', 'command', 'wireless communication', 'detecting', 'technology', 'logic', 'sensor', 'data', 'reduced', 'management', 'communications', 'communicates', 'real', 'pattern', 'comprise', 'content', 'security', 'transport', 'attachment', 'point', 'real time', 'cells', 'regions', 'semiconductor', 'forming', 'tool', 'layer', 'substrate', 'cell', 'electron', 'semiconductor layer', 'providing', 'air', 'pressure', 'multi', 'maintaining', 'pre', 'set', 'carbon', 'predetermined', 'operations', 'media', 'mechanical', 'computer', 'light', 'level', 'selected', 'support', 'single', 'axis', 'cases', 'body', 'beam', 'interface', 'connecting', 'portions', 'enable', 'connect', 'located', 'work', 'different', 'like', 'transparent', 'electrode', 'spaced', 'apart', 'electrodes', 'containing', 'non', 'planar', 'elements', 'embodiment', 'particles', 'fluid', 'devices', 'target', 'tissue', 'examples', 'region', 'greater', 'treating', 'devices methods', 'vehicle', 'suitable', 'service', 'capable', 'driving', 'motor', 'station', 'speed', 'connected', 'mounted', 'determines', 'value', 'transmits', 'determined', 'memory', 'host', 'sending', 'response', 'number', 'threshold', 'period time', 'model', 'obtaining', 'cost', 'ambient', 'includes determining', 'characteristics', 'safety', 'neural', 'input', 'layers', 'obtained', 'condition', 'accordance', 'neural network', 'allow', 'user', 'board', 'mechanism', 'port', 'create', 'tank', 'characteristic', 'autonomous', 'adapted', 'implementations', 'frequency', 'disposed', 'group', 'thereof', 'object', 'adjacent', 'location', 'performing', 'functions', 'mobile', 'platform', 'area', 'sensors', 'positioned', 'movement', 'operation', 'imaging', 'dimensional', 'points', '3d', 'generally', 'relates', 'particularly', 'aspect', 'thereon', 'oxide', 'active material', 'connector', 'conductive', 'function', 'contact', 'enhanced', 'angle', 'display', 'integrated', 'monitoring', 'record', 'indication', 'core', 'device includes', 'space', 'sensing', 'detected', 'accordingly', 'head', 'remote', 'computing', 'collection', 'intended', 'acid', 'wiring', 'switch', 'actuator', 'unit', 'secure', 'joint', 'patient', 'measurement', 'liquid', 'volume', 'container', 'emit', 'detect', 'emitted', 'received', 'devices systems', 'delivery device', 'plurality light', 'apparatus includes', 'intensity', 'emitting', 'operatively', 'light emitting', 'performed', 'resources', 'decoding', 'measurements', 'receiver', 'wireless device', 'apparatuses', 'applications', 'disclosure', 'production', 'present disclosure', 'disclosure provides', 'present disclosure provides', 'materials', 'related', 'compositions', 'novel', 'does', 'provided methods', 'web', 'defined', 'range', 'environment', 'optical', 'engine', 'lumen', 'transmitting', 'distal', 'device having', 'includes display', 'includes light', 'subsequent', 'series', 'electronic', 'sequence', 'disclosed apparatus', 'highly', 'reliable', 'share', 'common', 'contains', 'compound', 'emitting device', 'layer light', 'emitting layer', 'light emitting layer', 'sub', 'pixels', 'line', 'pixel', 'organic', 'transistor', 'color', 'overlap', 'channel', 'display device', 'device provided', 'emitting element', 'electrically connected', 'light emitting element', 'locations', 'flow', 'comparing', 'concentration', 'said', 'reliability', 'displaying', 'arranged', 'units', 'insulating', 'face', 'including plurality', 'insulating layer', 'display device includes', 'direction', 'surfaces', 'plane', 'film', 'metal', 'form', 'composition', 'higher', 'products', 'magnetic', 'extend', 'electronic device', 'element configured', 'resource', 'second set', 'primary', 'combination', 'change', 'peak', 'wavelength', 'device comprising', 'virtual', 'cartridge', 'housing', 'defining', 'contain', 'member', 'shape', 'flexible', 'physical', 'instructions', 'client', 'processor', 'particular', 'activity', 'invention relates', 'driver', 'direct', 'disclosure relates', 'present disclosure relates', 'causing', 'difference', 'analyte', 'additionally', 'manufacture', 'manufacturing', 'useful', 'equal', 'path', 'derivative', 'represents', 'independently', 'product', 'identifier', 'identify', 'execute', 'selective', 'image sensor', 'directed', 'distributed', 'updating', 'determination', 'block', 'manner', 'embodiments method', 'treatment', 'procedure', 'measure', 'parameter', 'applied', 'signal based', 'members', 'ratio', 'program', 'profile', 'event', 'updated', 'computer program', 'proximal', 'parameters', 'automatic', 'responsive', 'picture', 'drive', 'view', 'guide', 'improved', 'processed', 'field', 'image data', 'spectrum', 'access', 'displayed', 'results', 'objects', 'means', 'code', 'user interface', 'machine', 'learning', 'subset', 'features', 'predicted', 'machine learning', 'methods disclosed', 'systems methods disclosed', 'printing', 'compounds', 'database', 'device including', 'technique', 'quality', 'worn', 'positioning', 'software', 'graphical', 'representation', 'intelligence', 'graphical user', 'graphical user interface', 'separate', 'nucleic', 'nucleic acid', 'radio', 'dynamic', 'video', 'opening', 'near', 'cause', 'transmission', 'signals', 'implemented', 'medical', 'certain', 'layer includes', 'entity', 'users', 'configuration', 'methods systems', 'performance', 'node', 'message', 'compared', 'includes receiving', 'method includes receiving', 'articles', 'aspects', 'domain', 'address', 'analysis', 'selection', 'fast', 'client device', 'external', 'sample', 'conditions', 'collected', 'audio', 'status', 'provider', 'identification', 'secondary', 'incident', 'processor configured', 'analyzing', 'interfaces', 'identity', 'embodiments systems', 'systems apparatus', 'creating', 'changes', 'digital', 'generates', 'including display', 'readable', 'latency', 'computer readable', 'spatial', 'selects', 'post', 'trained', 'configurable', 'encoding', 'coding', 'size', 'identified', 'visual', 'stream', 'servers', 'new', 'presented', 'identifies', 'request', 'computing device', 'encoded'])\n"
     ]
    }
   ],
   "source": [
    "vect = CountVectorizer(min_df = 5, stop_words = \"english\", ngram_range = (1,3))\n",
    "vect.fit(ab)\n",
    "bof = vect.transform(ab)\n",
    "print(len(vect.vocabulary_))\n",
    "print(vect.vocabulary_.keys())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "id": "76080378-5dd0-4d47-a118-5672cb92db39",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    },
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  (0, 44)\t1\n",
      "  (0, 69)\t1\n",
      "  (0, 123)\t1\n",
      "  (0, 124)\t1\n",
      "  (0, 127)\t2\n",
      "  (0, 128)\t3\n",
      "  (0, 193)\t1\n",
      "  (0, 220)\t2\n",
      "  (0, 221)\t2\n",
      "  (0, 268)\t1\n",
      "  (0, 303)\t2\n",
      "  (0, 339)\t2\n",
      "  (0, 376)\t1\n",
      "  (0, 405)\t1\n",
      "  (0, 408)\t1\n",
      "  (0, 413)\t1\n",
      "  (0, 417)\t1\n",
      "  (0, 433)\t3\n",
      "  (0, 453)\t2\n",
      "  (0, 466)\t1\n",
      "  (0, 482)\t1\n",
      "  (0, 483)\t1\n",
      "  (0, 559)\t1\n",
      "  (0, 561)\t1\n",
      "  (0, 562)\t1\n",
      "  :\t:\n",
      "  (358, 551)\t2\n",
      "  (358, 591)\t3\n",
      "  (358, 616)\t3\n",
      "  (358, 632)\t3\n",
      "  (358, 643)\t1\n",
      "  (359, 113)\t3\n",
      "  (359, 155)\t1\n",
      "  (359, 160)\t1\n",
      "  (359, 178)\t5\n",
      "  (359, 179)\t1\n",
      "  (359, 183)\t3\n",
      "  (359, 247)\t4\n",
      "  (359, 291)\t1\n",
      "  (359, 297)\t2\n",
      "  (359, 298)\t1\n",
      "  (359, 299)\t1\n",
      "  (359, 327)\t4\n",
      "  (359, 382)\t3\n",
      "  (359, 417)\t4\n",
      "  (359, 442)\t5\n",
      "  (359, 443)\t2\n",
      "  (359, 447)\t2\n",
      "  (359, 487)\t1\n",
      "  (359, 562)\t3\n",
      "  (359, 585)\t4\n"
     ]
    }
   ],
   "source": [
    "print(bof)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "id": "03d18ebe-0c75-4695-a2f9-3db2a44825a5",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.feature_extraction.text import TfidfVectorizer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "id": "3456411e-126e-4a7c-a174-0a37ea604a4c",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    },
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "653\n",
      "dict_keys(['high', 'efficiency', 'solar', 'power', 'sources', 'base', 'phase', 'converter', 'low', 'energy', 'storage', 'circuitry', 'controlled', 'control', 'operates', 'provide', 'conversion', 'output', 'provided', 'individual', 'source', 'operational', 'modes', 'presenting', 'panel', 'energy storage', 'variable', 'continuous', 'heat', 'electrical', 'solid', 'medium', 'continuously', 'array', 'internal', 'radiation', 'directly', 'thermal', 'facilitate', 'heating', 'delivery', 'temperature', 'used', 'processes', 'including', 'generation', 'based', 'supply', 'using', 'current', 'advance', 'information', 'voltage', 'distribution', 'transfer', 'method', 'includes', 'generating', 'time', 'signal', 'device', 'comprises', 'identifying', 'simultaneously', 'operating', 'mode', 'additional', 'comprising', 'plurality', 'values', 'according', 'determining', 'method includes', 'storage device', 'predictive', 'controller', 'processing', 'circuits', 'configured', 'obtain', 'defines', 'electric', 'load', 'period', 'multiple', 'specific', 'components', 'include', 'component', 'indicating', 'second', 'perform', 'process', 'subject', 'determine', 'equipment', 'use', 'frame', 'having', 'position', 'receiving', 'material', 'surface', 'corresponding', 'battery', 'included', 'circuit', 'systems', 'methods', 'described', 'embodiments', 'module', 'electrically', 'coupled', 'operable', 'state', 'stored', 'monitor', 'systems methods', 'power source', 'apparatus', 'smart', 'grid', 'communication', 'disclosed', 'present', 'invention', 'element', 'constructed', 'network', 'server', 'active', 'automatically', 'connection', 'sends', 'receives', 'provides', 'methods apparatus', 'present invention', 'systems methods apparatus', 'various', 'relate', 'generate', 'storing', 'generated', 'receive', 'controlling', 'application', 'techniques', 'networks', 'various embodiments', 'configured generate', 'device configured', 'configured receive', 'example', 'assembly', 'lower', 'structure', 'shaped', 'length', 'extending', 'end', 'upper', 'pair', 'portion', 'substantially', 'structures', 'physically', 'powered', 'formed', 'relative', 'respective', 'includes plurality', 'associated', 'wireless', 'camera', 'image', 'capture', 'images', 'store', 'captured', 'transmit', 'detection', 'motion', 'command', 'wireless communication', 'detecting', 'technology', 'logic', 'sensor', 'data', 'reduced', 'management', 'communications', 'communicates', 'real', 'pattern', 'comprise', 'content', 'security', 'transport', 'attachment', 'point', 'real time', 'cells', 'regions', 'semiconductor', 'forming', 'tool', 'layer', 'substrate', 'cell', 'electron', 'semiconductor layer', 'providing', 'air', 'pressure', 'multi', 'maintaining', 'pre', 'set', 'carbon', 'predetermined', 'operations', 'media', 'mechanical', 'computer', 'light', 'level', 'selected', 'support', 'single', 'axis', 'cases', 'body', 'beam', 'interface', 'connecting', 'portions', 'enable', 'connect', 'located', 'work', 'different', 'like', 'transparent', 'electrode', 'spaced', 'apart', 'electrodes', 'containing', 'non', 'planar', 'elements', 'embodiment', 'particles', 'fluid', 'devices', 'target', 'tissue', 'examples', 'region', 'greater', 'treating', 'devices methods', 'vehicle', 'suitable', 'service', 'capable', 'driving', 'motor', 'station', 'speed', 'connected', 'mounted', 'determines', 'value', 'transmits', 'determined', 'memory', 'host', 'sending', 'response', 'number', 'threshold', 'period time', 'model', 'obtaining', 'cost', 'ambient', 'includes determining', 'characteristics', 'safety', 'neural', 'input', 'layers', 'obtained', 'condition', 'accordance', 'neural network', 'allow', 'user', 'board', 'mechanism', 'port', 'create', 'tank', 'characteristic', 'autonomous', 'adapted', 'implementations', 'frequency', 'disposed', 'group', 'thereof', 'object', 'adjacent', 'location', 'performing', 'functions', 'mobile', 'platform', 'area', 'sensors', 'positioned', 'movement', 'operation', 'imaging', 'dimensional', 'points', '3d', 'generally', 'relates', 'particularly', 'aspect', 'thereon', 'oxide', 'active material', 'connector', 'conductive', 'function', 'contact', 'enhanced', 'angle', 'display', 'integrated', 'monitoring', 'record', 'indication', 'core', 'device includes', 'space', 'sensing', 'detected', 'accordingly', 'head', 'remote', 'computing', 'collection', 'intended', 'acid', 'wiring', 'switch', 'actuator', 'unit', 'secure', 'joint', 'patient', 'measurement', 'liquid', 'volume', 'container', 'emit', 'detect', 'emitted', 'received', 'devices systems', 'delivery device', 'plurality light', 'apparatus includes', 'intensity', 'emitting', 'operatively', 'light emitting', 'performed', 'resources', 'decoding', 'measurements', 'receiver', 'wireless device', 'apparatuses', 'applications', 'disclosure', 'production', 'present disclosure', 'disclosure provides', 'present disclosure provides', 'materials', 'related', 'compositions', 'novel', 'does', 'provided methods', 'web', 'defined', 'range', 'environment', 'optical', 'engine', 'lumen', 'transmitting', 'distal', 'device having', 'includes display', 'includes light', 'subsequent', 'series', 'electronic', 'sequence', 'disclosed apparatus', 'highly', 'reliable', 'share', 'common', 'contains', 'compound', 'emitting device', 'layer light', 'emitting layer', 'light emitting layer', 'sub', 'pixels', 'line', 'pixel', 'organic', 'transistor', 'color', 'overlap', 'channel', 'display device', 'device provided', 'emitting element', 'electrically connected', 'light emitting element', 'locations', 'flow', 'comparing', 'concentration', 'said', 'reliability', 'displaying', 'arranged', 'units', 'insulating', 'face', 'including plurality', 'insulating layer', 'display device includes', 'direction', 'surfaces', 'plane', 'film', 'metal', 'form', 'composition', 'higher', 'products', 'magnetic', 'extend', 'electronic device', 'element configured', 'resource', 'second set', 'primary', 'combination', 'change', 'peak', 'wavelength', 'device comprising', 'virtual', 'cartridge', 'housing', 'defining', 'contain', 'member', 'shape', 'flexible', 'physical', 'instructions', 'client', 'processor', 'particular', 'activity', 'invention relates', 'driver', 'direct', 'disclosure relates', 'present disclosure relates', 'causing', 'difference', 'analyte', 'additionally', 'manufacture', 'manufacturing', 'useful', 'equal', 'path', 'derivative', 'represents', 'independently', 'product', 'identifier', 'identify', 'execute', 'selective', 'image sensor', 'directed', 'distributed', 'updating', 'determination', 'block', 'manner', 'embodiments method', 'treatment', 'procedure', 'measure', 'parameter', 'applied', 'signal based', 'members', 'ratio', 'program', 'profile', 'event', 'updated', 'computer program', 'proximal', 'parameters', 'automatic', 'responsive', 'picture', 'drive', 'view', 'guide', 'improved', 'processed', 'field', 'image data', 'spectrum', 'access', 'displayed', 'results', 'objects', 'means', 'code', 'user interface', 'machine', 'learning', 'subset', 'features', 'predicted', 'machine learning', 'methods disclosed', 'systems methods disclosed', 'printing', 'compounds', 'database', 'device including', 'technique', 'quality', 'worn', 'positioning', 'software', 'graphical', 'representation', 'intelligence', 'graphical user', 'graphical user interface', 'separate', 'nucleic', 'nucleic acid', 'radio', 'dynamic', 'video', 'opening', 'near', 'cause', 'transmission', 'signals', 'implemented', 'medical', 'certain', 'layer includes', 'entity', 'users', 'configuration', 'methods systems', 'performance', 'node', 'message', 'compared', 'includes receiving', 'method includes receiving', 'articles', 'aspects', 'domain', 'address', 'analysis', 'selection', 'fast', 'client device', 'external', 'sample', 'conditions', 'collected', 'audio', 'status', 'provider', 'identification', 'secondary', 'incident', 'processor configured', 'analyzing', 'interfaces', 'identity', 'embodiments systems', 'systems apparatus', 'creating', 'changes', 'digital', 'generates', 'including display', 'readable', 'latency', 'computer readable', 'spatial', 'selects', 'post', 'trained', 'configurable', 'encoding', 'coding', 'size', 'identified', 'visual', 'stream', 'servers', 'new', 'presented', 'identifies', 'request', 'computing device', 'encoded'])\n"
     ]
    }
   ],
   "source": [
    "tfidf_vect = TfidfVectorizer(min_df = 5, stop_words = \"english\", ngram_range = (1,3))\n",
    "tfidf_vect.fit(ab)\n",
    "tfidf_vect_bof = tfidf_vect.transform(ab)\n",
    "print(len(tfidf_vect.vocabulary_))\n",
    "print(tfidf_vect.vocabulary_.keys())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "id": "250369c6-b15f-4441-93b0-e7aafe2a7690",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "words_sortedby_tfidf = tfidf_vect_bof.toarray().max(axis = 0).argsort()\n",
    "feature_names = tfidf_vect.vocabulary_.keys()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "id": "855d7e54-5ba7-4d91-978b-fc2526fadea1",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "feautures with lowest tfidf:\n",
      "611:  external\n",
      "322:  area\n",
      "108:  methods\n",
      "298:  accordance\n",
      "595:  configuration\n",
      "414:  includes light\n",
      "12:  controlled\n",
      "258:  target\n",
      "259:  tissue\n",
      "204:  cells\n",
      "286:  model\n",
      "15:  provide\n",
      "370:  volume\n",
      "40:  delivery\n",
      "159:  length\n",
      "273:  connected\n",
      "615:  audio\n",
      "405:  range\n",
      "292:  safety\n",
      "642:  size\n"
     ]
    }
   ],
   "source": [
    "print('feautures with lowest tfidf:')\n",
    "for i in list(words_sortedby_tfidf[:20]):\n",
    "    print(f'{i}: ',list(feature_names)[i])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "id": "682752c3-d04f-4d59-b198-39d44a2c8cd7",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    },
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "feautures with highest tfidf:\n",
      "120:  apparatus\n",
      "547:  image data\n",
      "551:  results\n",
      "385:  resources\n",
      "477:  wavelength\n",
      "499:  difference\n",
      "68:  plurality\n",
      "638:  trained\n",
      "516:  directed\n",
      "572:  software\n",
      "117:  monitor\n",
      "262:  greater\n",
      "36:  directly\n",
      "646:  servers\n",
      "589:  implemented\n",
      "411:  distal\n",
      "74:  predictive\n",
      "363:  actuator\n",
      "618:  identification\n",
      "381:  emitting\n"
     ]
    }
   ],
   "source": [
    "print('feautures with highest tfidf:')\n",
    "for i in list(words_sortedby_tfidf[-20:]):\n",
    "    print(f'{i}: ',list(feature_names)[i])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9dffc7c0-5d3d-4079-8e70-694a3d7514e6",
   "metadata": {},
   "source": [
    "## WebScraping: CPC code"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "e6835f08-8eac-495d-9240-0311b0e1c160",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>title</th>\n",
       "      <th>assignee</th>\n",
       "      <th>inventor</th>\n",
       "      <th>priority_date</th>\n",
       "      <th>filing_date</th>\n",
       "      <th>publication_date</th>\n",
       "      <th>grant_date</th>\n",
       "      <th>result_link</th>\n",
       "      <th>representative_figure_link</th>\n",
       "      <th>country</th>\n",
       "      <th>kind_code</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>US10340693B2</td>\n",
       "      <td>Systems and methods for generating energy usin...</td>\n",
       "      <td>Lawrence D. Lansing, JR., Lawrence D. Lansing</td>\n",
       "      <td>['Lawrence D. Lansing', ' JR.', ' Lawrence D. ...</td>\n",
       "      <td>20130313.0</td>\n",
       "      <td>20180227</td>\n",
       "      <td>20190702</td>\n",
       "      <td>20190702</td>\n",
       "      <td>https://patents.google.com/patent/US10340693B2/en</td>\n",
       "      <td>https://patentimages.storage.googleapis.com/4c...</td>\n",
       "      <td>US</td>\n",
       "      <td>B2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>US10526056B1</td>\n",
       "      <td>Generation of electric power using wave motion...</td>\n",
       "      <td>Physician Electronic Network, LLC</td>\n",
       "      <td>['A-Hamid Hakki', ' Edin Dervishalidovic', ' B...</td>\n",
       "      <td>20190429.0</td>\n",
       "      <td>20190429</td>\n",
       "      <td>20200107</td>\n",
       "      <td>20200107</td>\n",
       "      <td>https://patents.google.com/patent/US10526056B1/en</td>\n",
       "      <td>https://patentimages.storage.googleapis.com/6f...</td>\n",
       "      <td>US</td>\n",
       "      <td>B1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>US11228182B2</td>\n",
       "      <td>Converter disabling photovoltaic electrical en...</td>\n",
       "      <td>Ampt, Llc</td>\n",
       "      <td>['Robert Porter', ' Anatoli Ledenev']</td>\n",
       "      <td>20071015.0</td>\n",
       "      <td>20190612</td>\n",
       "      <td>20220118</td>\n",
       "      <td>20220118</td>\n",
       "      <td>https://patents.google.com/patent/US11228182B2/en</td>\n",
       "      <td>https://patentimages.storage.googleapis.com/f5...</td>\n",
       "      <td>US</td>\n",
       "      <td>B2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>US11060742B2</td>\n",
       "      <td>PVT heat pump system capable of achieving day-...</td>\n",
       "      <td>Dalian University Of Technology</td>\n",
       "      <td>['Jili ZHANG', ' Ruobing LIANG', ' Chao Zhou',...</td>\n",
       "      <td>20170803.0</td>\n",
       "      <td>20170803</td>\n",
       "      <td>20210713</td>\n",
       "      <td>20210713</td>\n",
       "      <td>https://patents.google.com/patent/US11060742B2/en</td>\n",
       "      <td>https://patentimages.storage.googleapis.com/f8...</td>\n",
       "      <td>US</td>\n",
       "      <td>B2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>US10808685B2</td>\n",
       "      <td>Dispatchable combined cycle power plant</td>\n",
       "      <td>William M. Conlon</td>\n",
       "      <td>['William M. Conlon']</td>\n",
       "      <td>20140604.0</td>\n",
       "      <td>20181019</td>\n",
       "      <td>20201020</td>\n",
       "      <td>20201020</td>\n",
       "      <td>https://patents.google.com/patent/US10808685B2/en</td>\n",
       "      <td>https://patentimages.storage.googleapis.com/d5...</td>\n",
       "      <td>US</td>\n",
       "      <td>B2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             id                                              title  \\\n",
       "0  US10340693B2  Systems and methods for generating energy usin...   \n",
       "1  US10526056B1  Generation of electric power using wave motion...   \n",
       "2  US11228182B2  Converter disabling photovoltaic electrical en...   \n",
       "3  US11060742B2  PVT heat pump system capable of achieving day-...   \n",
       "4  US10808685B2           Dispatchable combined cycle power plant    \n",
       "\n",
       "                                        assignee  \\\n",
       "0  Lawrence D. Lansing, JR., Lawrence D. Lansing   \n",
       "1              Physician Electronic Network, LLC   \n",
       "2                                      Ampt, Llc   \n",
       "3                Dalian University Of Technology   \n",
       "4                              William M. Conlon   \n",
       "\n",
       "                                            inventor  priority_date  \\\n",
       "0  ['Lawrence D. Lansing', ' JR.', ' Lawrence D. ...     20130313.0   \n",
       "1  ['A-Hamid Hakki', ' Edin Dervishalidovic', ' B...     20190429.0   \n",
       "2              ['Robert Porter', ' Anatoli Ledenev']     20071015.0   \n",
       "3  ['Jili ZHANG', ' Ruobing LIANG', ' Chao Zhou',...     20170803.0   \n",
       "4                              ['William M. Conlon']     20140604.0   \n",
       "\n",
       "   filing_date  publication_date  grant_date  \\\n",
       "0     20180227          20190702    20190702   \n",
       "1     20190429          20200107    20200107   \n",
       "2     20190612          20220118    20220118   \n",
       "3     20170803          20210713    20210713   \n",
       "4     20181019          20201020    20201020   \n",
       "\n",
       "                                         result_link  \\\n",
       "0  https://patents.google.com/patent/US10340693B2/en   \n",
       "1  https://patents.google.com/patent/US10526056B1/en   \n",
       "2  https://patents.google.com/patent/US11228182B2/en   \n",
       "3  https://patents.google.com/patent/US11060742B2/en   \n",
       "4  https://patents.google.com/patent/US10808685B2/en   \n",
       "\n",
       "                          representative_figure_link country kind_code  \n",
       "0  https://patentimages.storage.googleapis.com/4c...      US        B2  \n",
       "1  https://patentimages.storage.googleapis.com/6f...      US        B1  \n",
       "2  https://patentimages.storage.googleapis.com/f5...      US        B2  \n",
       "3  https://patentimages.storage.googleapis.com/f8...      US        B2  \n",
       "4  https://patentimages.storage.googleapis.com/d5...      US        B2  "
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# reload cleaned patent list\n",
    "import pandas as pd\n",
    "df = pd.read_csv(r\"C:\\Users\\user\\Documents\\RenewableEnergy_patents_clean.csv\", index_col = 0)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "860a813a-26f9-4ad9-8443-825254db9308",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "### Establish patent cpc code scraping workflow "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "1d4c4712-09da-46c0-9cfb-0e3e56396f60",
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests\n",
    "from bs4 import BeautifulSoup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "2729cb1d-2587-4294-868a-bb3676e605a2",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Take the first patent as an example:\n",
    "url = df.loc[0,\"result_link\"]\n",
    "response = requests.get(url) \n",
    "html_content = response.text\n",
    "\n",
    "# Create a BeautifulSoup object to parse the HTML\n",
    "soup = BeautifulSoup(html_content, 'html.parser') "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "514e7b41-e07e-4bc1-a2d1-e14d11eb1be7",
   "metadata": {},
   "outputs": [],
   "source": [
    "cpc_tags = soup.find('h2', string = 'Classifications').find_next_sibling().find_all('span', itemprop=\"Code\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "id": "067b1507-64e5-403a-a117-73f794e51ce3",
   "metadata": {},
   "outputs": [],
   "source": [
    "codes = pd.unique(pd.Series([tag.contents[0] for tag in cpc_tags]))\n",
    "first_level_codes = [code for code in codes if len(code) == 1]\n",
    "second_level_codes = [code for code in codes if len(code) == 3]\n",
    "third_level_codes = [code for code in codes if len(code) == 4]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "id": "d9fe715a-cb6b-463d-a677-892121c49aaa",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['H', 'F', 'G', 'Y']"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "first_level_codes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "id": "ba980444-fd2e-4240-8f3b-5e844ba90bef",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['H02', 'F03', 'G05', 'F05', 'Y02']"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "second_level_codes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "id": "7270b659-93c8-4a94-b272-795eb80a4320",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['H02J', 'F03B', 'F03D', 'G05B', 'H02S', 'F05B', 'Y02E', 'Y02P']"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "third_level_codes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "id": "07c93d93-e626-4985-afc0-8b4203050141",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "17852 patent cpc code:\n",
      "['B', 'G', 'H']\n",
      "['B60', 'G06', 'G08', 'H04']\n",
      "['B60K', 'G06F', 'G08C', 'H04L', 'B60W']\n",
      "\n",
      "1293 patent cpc code:\n",
      "['F']\n",
      "['F01']\n",
      "['F01K']\n",
      "\n",
      "13549 patent cpc code:\n",
      "['G', 'Y']\n",
      "['G06', 'Y02']\n",
      "['G06F', 'Y02D']\n",
      "\n",
      "10372 patent cpc code:\n",
      "['H', 'C', 'Y']\n",
      "['H01', 'C01', 'Y02', 'Y10']\n",
      "['H01M', 'C01G', 'C01P', 'Y02E', 'Y02P', 'Y02T', 'Y10T']\n",
      "\n",
      "18659 patent cpc code:\n",
      "['G']\n",
      "['G09']\n",
      "['G09G']\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# view random 5 patents cpc code scraping results\n",
    "import random\n",
    "for i in random.sample(range(len(df)), 5):\n",
    "    url = df.loc[i,\"result_link\"]\n",
    "    response = requests.get(url) \n",
    "    html_content = response.text\n",
    "    soup = BeautifulSoup(html_content, 'html.parser')\n",
    "    cpc_tags = soup.find('h2', string = 'Classifications').find_next_sibling().find_all('span', itemprop=\"Code\")\n",
    "    codes = pd.unique(pd.Series([tag.contents[0] for tag in cpc_tags]))\n",
    "    first_level_codes = [code for code in codes if len(code) == 1]\n",
    "    second_level_codes = [code for code in codes if len(code) == 3]\n",
    "    third_level_codes = [code for code in codes if len(code) == 4]\n",
    "    print(f'{i+1} patent cpc code:')\n",
    "    print(first_level_codes)\n",
    "    print(second_level_codes)\n",
    "    print(third_level_codes)\n",
    "    print('')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dfcbaca6-b87b-48d6-b0ca-028158039279",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "### Scrape CPC codes for all patents filed in 2023"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "id": "450b7092-06da-4011-8e06-7aac1586a749",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>title</th>\n",
       "      <th>assignee</th>\n",
       "      <th>inventor</th>\n",
       "      <th>priority_date</th>\n",
       "      <th>filing_date</th>\n",
       "      <th>publication_date</th>\n",
       "      <th>grant_date</th>\n",
       "      <th>result_link</th>\n",
       "      <th>representative_figure_link</th>\n",
       "      <th>country</th>\n",
       "      <th>kind_code</th>\n",
       "      <th>abstract</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>US11967653B2</td>\n",
       "      <td>Phased solar power supply system</td>\n",
       "      <td>Ampt, Llc</td>\n",
       "      <td>['Anatoli Ledenev']</td>\n",
       "      <td>20130315.0</td>\n",
       "      <td>20230905</td>\n",
       "      <td>20240423</td>\n",
       "      <td>20240423</td>\n",
       "      <td>https://patents.google.com/patent/US11967653B2/en</td>\n",
       "      <td>https://patentimages.storage.googleapis.com/c0...</td>\n",
       "      <td>US</td>\n",
       "      <td>B2</td>\n",
       "      <td>A high efficiency solar power system combining...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>US11867096B2</td>\n",
       "      <td>Calcination system with thermal energy storage...</td>\n",
       "      <td>Rondo Energy, Inc.</td>\n",
       "      <td>[\"John Setel O'Donnell\", ' Peter Emery von Beh...</td>\n",
       "      <td>20201130.0</td>\n",
       "      <td>20230220</td>\n",
       "      <td>20240109</td>\n",
       "      <td>20240109</td>\n",
       "      <td>https://patents.google.com/patent/US11867096B2/en</td>\n",
       "      <td>https://patentimages.storage.googleapis.com/35...</td>\n",
       "      <td>US</td>\n",
       "      <td>B2</td>\n",
       "      <td>An energy storage system converts variable ren...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>US11757404B2</td>\n",
       "      <td>Coordinated control of renewable electric gene...</td>\n",
       "      <td>8Me Nova, Llc</td>\n",
       "      <td>['Lukas Hansen', ' Philippe Garneau-Halliday',...</td>\n",
       "      <td>20190208.0</td>\n",
       "      <td>20230310</td>\n",
       "      <td>20230912</td>\n",
       "      <td>20230912</td>\n",
       "      <td>https://patents.google.com/patent/US11757404B2/en</td>\n",
       "      <td>https://patentimages.storage.googleapis.com/f6...</td>\n",
       "      <td>US</td>\n",
       "      <td>B2</td>\n",
       "      <td>A method includes generating a time-varying ch...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>US11973345B2</td>\n",
       "      <td>Building energy system with predictive control...</td>\n",
       "      <td>Johnson Controls Tyco IP Holdings LLP</td>\n",
       "      <td>['Robert D. Turney', ' Nishith R. Patel']</td>\n",
       "      <td>20170427.0</td>\n",
       "      <td>20230602</td>\n",
       "      <td>20240430</td>\n",
       "      <td>20240430</td>\n",
       "      <td>https://patents.google.com/patent/US11973345B2/en</td>\n",
       "      <td>https://patentimages.storage.googleapis.com/0f...</td>\n",
       "      <td>US</td>\n",
       "      <td>B2</td>\n",
       "      <td>A predictive controller for a building energy ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>US11780305B2</td>\n",
       "      <td>Tonneau system for use with a pickup truck</td>\n",
       "      <td>Worksport Ltd.</td>\n",
       "      <td>['Steven Rossi', ' Jonathan Loudon']</td>\n",
       "      <td>20151030.0</td>\n",
       "      <td>20230110</td>\n",
       "      <td>20231010</td>\n",
       "      <td>20231010</td>\n",
       "      <td>https://patents.google.com/patent/US11780305B2/en</td>\n",
       "      <td>https://patentimages.storage.googleapis.com/36...</td>\n",
       "      <td>US</td>\n",
       "      <td>B2</td>\n",
       "      <td>A tonneau system for use with a pickup truck. ...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             id                                              title  \\\n",
       "0  US11967653B2                  Phased solar power supply system    \n",
       "1  US11867096B2  Calcination system with thermal energy storage...   \n",
       "2  US11757404B2  Coordinated control of renewable electric gene...   \n",
       "3  US11973345B2  Building energy system with predictive control...   \n",
       "4  US11780305B2        Tonneau system for use with a pickup truck    \n",
       "\n",
       "                                assignee  \\\n",
       "0                              Ampt, Llc   \n",
       "1                     Rondo Energy, Inc.   \n",
       "2                          8Me Nova, Llc   \n",
       "3  Johnson Controls Tyco IP Holdings LLP   \n",
       "4                         Worksport Ltd.   \n",
       "\n",
       "                                            inventor  priority_date  \\\n",
       "0                                ['Anatoli Ledenev']     20130315.0   \n",
       "1  [\"John Setel O'Donnell\", ' Peter Emery von Beh...     20201130.0   \n",
       "2  ['Lukas Hansen', ' Philippe Garneau-Halliday',...     20190208.0   \n",
       "3          ['Robert D. Turney', ' Nishith R. Patel']     20170427.0   \n",
       "4               ['Steven Rossi', ' Jonathan Loudon']     20151030.0   \n",
       "\n",
       "   filing_date  publication_date  grant_date  \\\n",
       "0     20230905          20240423    20240423   \n",
       "1     20230220          20240109    20240109   \n",
       "2     20230310          20230912    20230912   \n",
       "3     20230602          20240430    20240430   \n",
       "4     20230110          20231010    20231010   \n",
       "\n",
       "                                         result_link  \\\n",
       "0  https://patents.google.com/patent/US11967653B2/en   \n",
       "1  https://patents.google.com/patent/US11867096B2/en   \n",
       "2  https://patents.google.com/patent/US11757404B2/en   \n",
       "3  https://patents.google.com/patent/US11973345B2/en   \n",
       "4  https://patents.google.com/patent/US11780305B2/en   \n",
       "\n",
       "                          representative_figure_link country kind_code  \\\n",
       "0  https://patentimages.storage.googleapis.com/c0...      US        B2   \n",
       "1  https://patentimages.storage.googleapis.com/35...      US        B2   \n",
       "2  https://patentimages.storage.googleapis.com/f6...      US        B2   \n",
       "3  https://patentimages.storage.googleapis.com/0f...      US        B2   \n",
       "4  https://patentimages.storage.googleapis.com/36...      US        B2   \n",
       "\n",
       "                                            abstract  \n",
       "0  A high efficiency solar power system combining...  \n",
       "1  An energy storage system converts variable ren...  \n",
       "2  A method includes generating a time-varying ch...  \n",
       "3  A predictive controller for a building energy ...  \n",
       "4  A tonneau system for use with a pickup truck. ...  "
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_2023 = pd.read_csv(r\"C:\\Users\\user\\Documents\\RenewableEnergy_patents_2023_abstract.csv\", index_col = 0)\n",
    "df_2023.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "id": "dfb6951f-4d0b-40af-a815-0633cc391258",
   "metadata": {},
   "outputs": [],
   "source": [
    "from tqdm import tqdm\n",
    "import time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "id": "15a201b8-9f40-45e2-b839-c18972fcf0dd",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|████████████████████████████████████████████████████████████████████████████████| 360/360 [19:02<00:00,  3.17s/it]\n"
     ]
    }
   ],
   "source": [
    "df_2023['cpc_1'] = ''\n",
    "df_2023['cpc_2'] = ''\n",
    "df_2023['cpc_3'] = ''\n",
    "\n",
    "for i in tqdm(range(len(df_2023))):\n",
    "    url = df.loc[i,\"result_link\"]\n",
    "    response = requests.get(url) \n",
    "    html_content = response.text\n",
    "    soup = BeautifulSoup(html_content, 'html.parser')\n",
    "    cpc_tags = soup.find('h2', string = 'Classifications').find_next_sibling().find_all('span', itemprop=\"Code\")\n",
    "    codes = pd.unique(pd.Series([tag.contents[0] for tag in cpc_tags]))\n",
    "    first_level_codes = [code for code in codes if len(code) == 1]\n",
    "    second_level_codes = [code for code in codes if len(code) == 3]\n",
    "    third_level_codes = [code for code in codes if len(code) == 4]\n",
    "    df_2023.at[i, 'cpc_1'] = first_level_codes\n",
    "    df_2023.at[i, 'cpc_2'] = second_level_codes\n",
    "    df_2023.at[i, 'cpc_3'] = third_level_codes\n",
    "    time.sleep(0.1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "id": "ad5cf251-7b2f-4b1c-b60b-a5e7330de3da",
   "metadata": {},
   "outputs": [],
   "source": [
    "# count on number of code categories on each level\n",
    "df_2023['cpc_1_num'] = ''\n",
    "df_2023['cpc_2_num'] = ''\n",
    "df_2023['cpc_3_num'] = ''\n",
    "\n",
    "for i in range(len(df_2023)):\n",
    "    df_2023.loc[i, 'cpc_1_num'] = len(df_2023.loc[i,'cpc_1'])\n",
    "    df_2023.loc[i, 'cpc_2_num'] = len(df_2023.loc[i,'cpc_2'])\n",
    "    df_2023.loc[i, 'cpc_3_num'] = len(df_2023.loc[i,'cpc_3'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "id": "13525f15-8f41-49a2-bdd2-77c66f60ff1d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>priority_date</th>\n",
       "      <th>filing_date</th>\n",
       "      <th>publication_date</th>\n",
       "      <th>grant_date</th>\n",
       "      <th>cpc_1_num</th>\n",
       "      <th>cpc_2_num</th>\n",
       "      <th>cpc_3_num</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>3.600000e+02</td>\n",
       "      <td>3.600000e+02</td>\n",
       "      <td>3.600000e+02</td>\n",
       "      <td>3.600000e+02</td>\n",
       "      <td>360.000000</td>\n",
       "      <td>360.000000</td>\n",
       "      <td>360.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>2.014607e+07</td>\n",
       "      <td>2.023034e+07</td>\n",
       "      <td>2.023853e+07</td>\n",
       "      <td>2.023853e+07</td>\n",
       "      <td>2.800000</td>\n",
       "      <td>3.905556</td>\n",
       "      <td>5.258333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>3.761036e+04</td>\n",
       "      <td>2.244709e+02</td>\n",
       "      <td>3.694609e+03</td>\n",
       "      <td>3.694609e+03</td>\n",
       "      <td>0.925648</td>\n",
       "      <td>1.728661</td>\n",
       "      <td>2.578557</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>2.003101e+07</td>\n",
       "      <td>2.023010e+07</td>\n",
       "      <td>2.023061e+07</td>\n",
       "      <td>2.023061e+07</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>2.012111e+07</td>\n",
       "      <td>2.023013e+07</td>\n",
       "      <td>2.024011e+07</td>\n",
       "      <td>2.024011e+07</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>3.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>2.015103e+07</td>\n",
       "      <td>2.023031e+07</td>\n",
       "      <td>2.024031e+07</td>\n",
       "      <td>2.024031e+07</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>5.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>2.017104e+07</td>\n",
       "      <td>2.023042e+07</td>\n",
       "      <td>2.024043e+07</td>\n",
       "      <td>2.024043e+07</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>7.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>2.021083e+07</td>\n",
       "      <td>2.023120e+07</td>\n",
       "      <td>2.024062e+07</td>\n",
       "      <td>2.024062e+07</td>\n",
       "      <td>7.000000</td>\n",
       "      <td>12.000000</td>\n",
       "      <td>16.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       priority_date   filing_date  publication_date    grant_date  \\\n",
       "count   3.600000e+02  3.600000e+02      3.600000e+02  3.600000e+02   \n",
       "mean    2.014607e+07  2.023034e+07      2.023853e+07  2.023853e+07   \n",
       "std     3.761036e+04  2.244709e+02      3.694609e+03  3.694609e+03   \n",
       "min     2.003101e+07  2.023010e+07      2.023061e+07  2.023061e+07   \n",
       "25%     2.012111e+07  2.023013e+07      2.024011e+07  2.024011e+07   \n",
       "50%     2.015103e+07  2.023031e+07      2.024031e+07  2.024031e+07   \n",
       "75%     2.017104e+07  2.023042e+07      2.024043e+07  2.024043e+07   \n",
       "max     2.021083e+07  2.023120e+07      2.024062e+07  2.024062e+07   \n",
       "\n",
       "        cpc_1_num   cpc_2_num   cpc_3_num  \n",
       "count  360.000000  360.000000  360.000000  \n",
       "mean     2.800000    3.905556    5.258333  \n",
       "std      0.925648    1.728661    2.578557  \n",
       "min      1.000000    1.000000    1.000000  \n",
       "25%      2.000000    3.000000    3.000000  \n",
       "50%      3.000000    4.000000    5.000000  \n",
       "75%      3.000000    5.000000    7.000000  \n",
       "max      7.000000   12.000000   16.000000  "
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_2023['cpc_1_num'] = df_2023['cpc_1_num'].astype(int)\n",
    "df_2023['cpc_2_num'] = df_2023['cpc_2_num'].astype(int)\n",
    "df_2023['cpc_3_num'] = df_2023['cpc_3_num'].astype(int)\n",
    "\n",
    "df_2023.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "id": "5eaff86f-fe1e-44a9-8c2c-0f7ff6b81eb1",
   "metadata": {},
   "outputs": [],
   "source": [
    "df_2023['priority_date'] = df_2023['priority_date'].astype(int).astype(str)\n",
    "df_2023['filing_date'] = df_2023['filing_date'].astype(int).astype(str)\n",
    "df_2023['publication_date'] = df_2023['publication_date'].astype(int).astype(str)\n",
    "df_2023['grant_date'] = df_2023['grant_date'].astype(int).astype(str)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "id": "bfd54946-ce91-4082-978f-8c9ecc9a11eb",
   "metadata": {},
   "outputs": [],
   "source": [
    "from datetime import datetime\n",
    "\n",
    "for date in ['priority_date','filing_date','publication_date','grant_date']:\n",
    "    for i in range(len(df_2023)):\n",
    "        df_2023.loc[i, date] = datetime.strptime(df_2023.loc[i, date], '%Y%m%d').date()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "id": "d64e49f2-ec6f-4ed8-b53e-c0e1c35fb631",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>title</th>\n",
       "      <th>assignee</th>\n",
       "      <th>inventor</th>\n",
       "      <th>priority_date</th>\n",
       "      <th>filing_date</th>\n",
       "      <th>publication_date</th>\n",
       "      <th>grant_date</th>\n",
       "      <th>result_link</th>\n",
       "      <th>representative_figure_link</th>\n",
       "      <th>country</th>\n",
       "      <th>kind_code</th>\n",
       "      <th>abstract</th>\n",
       "      <th>cpc_1</th>\n",
       "      <th>cpc_2</th>\n",
       "      <th>cpc_3</th>\n",
       "      <th>cpc_1_num</th>\n",
       "      <th>cpc_2_num</th>\n",
       "      <th>cpc_3_num</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>US11967653B2</td>\n",
       "      <td>Phased solar power supply system</td>\n",
       "      <td>Ampt, Llc</td>\n",
       "      <td>['Anatoli Ledenev']</td>\n",
       "      <td>2013-03-15</td>\n",
       "      <td>2023-09-05</td>\n",
       "      <td>2024-04-23</td>\n",
       "      <td>2024-04-23</td>\n",
       "      <td>https://patents.google.com/patent/US11967653B2/en</td>\n",
       "      <td>https://patentimages.storage.googleapis.com/c0...</td>\n",
       "      <td>US</td>\n",
       "      <td>B2</td>\n",
       "      <td>A high efficiency solar power system combining...</td>\n",
       "      <td>[H, F, G, Y]</td>\n",
       "      <td>[H02, F03, G05, F05, Y02]</td>\n",
       "      <td>[H02J, F03B, F03D, G05B, H02S, F05B, Y02E, Y02P]</td>\n",
       "      <td>4</td>\n",
       "      <td>5</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>US11867096B2</td>\n",
       "      <td>Calcination system with thermal energy storage...</td>\n",
       "      <td>Rondo Energy, Inc.</td>\n",
       "      <td>[\"John Setel O'Donnell\", ' Peter Emery von Beh...</td>\n",
       "      <td>2020-11-30</td>\n",
       "      <td>2023-02-20</td>\n",
       "      <td>2024-01-09</td>\n",
       "      <td>2024-01-09</td>\n",
       "      <td>https://patents.google.com/patent/US11867096B2/en</td>\n",
       "      <td>https://patentimages.storage.googleapis.com/35...</td>\n",
       "      <td>US</td>\n",
       "      <td>B2</td>\n",
       "      <td>An energy storage system converts variable ren...</td>\n",
       "      <td>[H, B, F, Y]</td>\n",
       "      <td>[H02, B63, F03, F24, F05, Y02]</td>\n",
       "      <td>[H02N, B63B, F03B, F03D, F24S, H02K, H02S, F05...</td>\n",
       "      <td>4</td>\n",
       "      <td>6</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>US11757404B2</td>\n",
       "      <td>Coordinated control of renewable electric gene...</td>\n",
       "      <td>8Me Nova, Llc</td>\n",
       "      <td>['Lukas Hansen', ' Philippe Garneau-Halliday',...</td>\n",
       "      <td>2019-02-08</td>\n",
       "      <td>2023-03-10</td>\n",
       "      <td>2023-09-12</td>\n",
       "      <td>2023-09-12</td>\n",
       "      <td>https://patents.google.com/patent/US11757404B2/en</td>\n",
       "      <td>https://patentimages.storage.googleapis.com/f6...</td>\n",
       "      <td>US</td>\n",
       "      <td>B2</td>\n",
       "      <td>A method includes generating a time-varying ch...</td>\n",
       "      <td>[H, Y]</td>\n",
       "      <td>[H02, Y02, Y04, Y10]</td>\n",
       "      <td>[H02J, H02M, Y02E, Y02P, Y04S, Y10S]</td>\n",
       "      <td>2</td>\n",
       "      <td>4</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>US11973345B2</td>\n",
       "      <td>Building energy system with predictive control...</td>\n",
       "      <td>Johnson Controls Tyco IP Holdings LLP</td>\n",
       "      <td>['Robert D. Turney', ' Nishith R. Patel']</td>\n",
       "      <td>2017-04-27</td>\n",
       "      <td>2023-06-02</td>\n",
       "      <td>2024-04-30</td>\n",
       "      <td>2024-04-30</td>\n",
       "      <td>https://patents.google.com/patent/US11973345B2/en</td>\n",
       "      <td>https://patentimages.storage.googleapis.com/0f...</td>\n",
       "      <td>US</td>\n",
       "      <td>B2</td>\n",
       "      <td>A predictive controller for a building energy ...</td>\n",
       "      <td>[F, H, Y]</td>\n",
       "      <td>[F25, F24, H02, Y02]</td>\n",
       "      <td>[F25B, F24F, H02S, Y02E]</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>US11780305B2</td>\n",
       "      <td>Tonneau system for use with a pickup truck</td>\n",
       "      <td>Worksport Ltd.</td>\n",
       "      <td>['Steven Rossi', ' Jonathan Loudon']</td>\n",
       "      <td>2015-10-30</td>\n",
       "      <td>2023-01-10</td>\n",
       "      <td>2023-10-10</td>\n",
       "      <td>2023-10-10</td>\n",
       "      <td>https://patents.google.com/patent/US11780305B2/en</td>\n",
       "      <td>https://patentimages.storage.googleapis.com/36...</td>\n",
       "      <td>US</td>\n",
       "      <td>B2</td>\n",
       "      <td>A tonneau system for use with a pickup truck. ...</td>\n",
       "      <td>[F, Y]</td>\n",
       "      <td>[F03, F01, F02, F22, F05, Y02]</td>\n",
       "      <td>[F03G, F01K, F02C, F22B, F05D, Y02E]</td>\n",
       "      <td>2</td>\n",
       "      <td>6</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             id                                              title  \\\n",
       "0  US11967653B2                  Phased solar power supply system    \n",
       "1  US11867096B2  Calcination system with thermal energy storage...   \n",
       "2  US11757404B2  Coordinated control of renewable electric gene...   \n",
       "3  US11973345B2  Building energy system with predictive control...   \n",
       "4  US11780305B2        Tonneau system for use with a pickup truck    \n",
       "\n",
       "                                assignee  \\\n",
       "0                              Ampt, Llc   \n",
       "1                     Rondo Energy, Inc.   \n",
       "2                          8Me Nova, Llc   \n",
       "3  Johnson Controls Tyco IP Holdings LLP   \n",
       "4                         Worksport Ltd.   \n",
       "\n",
       "                                            inventor priority_date  \\\n",
       "0                                ['Anatoli Ledenev']    2013-03-15   \n",
       "1  [\"John Setel O'Donnell\", ' Peter Emery von Beh...    2020-11-30   \n",
       "2  ['Lukas Hansen', ' Philippe Garneau-Halliday',...    2019-02-08   \n",
       "3          ['Robert D. Turney', ' Nishith R. Patel']    2017-04-27   \n",
       "4               ['Steven Rossi', ' Jonathan Loudon']    2015-10-30   \n",
       "\n",
       "  filing_date publication_date  grant_date  \\\n",
       "0  2023-09-05       2024-04-23  2024-04-23   \n",
       "1  2023-02-20       2024-01-09  2024-01-09   \n",
       "2  2023-03-10       2023-09-12  2023-09-12   \n",
       "3  2023-06-02       2024-04-30  2024-04-30   \n",
       "4  2023-01-10       2023-10-10  2023-10-10   \n",
       "\n",
       "                                         result_link  \\\n",
       "0  https://patents.google.com/patent/US11967653B2/en   \n",
       "1  https://patents.google.com/patent/US11867096B2/en   \n",
       "2  https://patents.google.com/patent/US11757404B2/en   \n",
       "3  https://patents.google.com/patent/US11973345B2/en   \n",
       "4  https://patents.google.com/patent/US11780305B2/en   \n",
       "\n",
       "                          representative_figure_link country kind_code  \\\n",
       "0  https://patentimages.storage.googleapis.com/c0...      US        B2   \n",
       "1  https://patentimages.storage.googleapis.com/35...      US        B2   \n",
       "2  https://patentimages.storage.googleapis.com/f6...      US        B2   \n",
       "3  https://patentimages.storage.googleapis.com/0f...      US        B2   \n",
       "4  https://patentimages.storage.googleapis.com/36...      US        B2   \n",
       "\n",
       "                                            abstract         cpc_1  \\\n",
       "0  A high efficiency solar power system combining...  [H, F, G, Y]   \n",
       "1  An energy storage system converts variable ren...  [H, B, F, Y]   \n",
       "2  A method includes generating a time-varying ch...        [H, Y]   \n",
       "3  A predictive controller for a building energy ...     [F, H, Y]   \n",
       "4  A tonneau system for use with a pickup truck. ...        [F, Y]   \n",
       "\n",
       "                            cpc_2  \\\n",
       "0       [H02, F03, G05, F05, Y02]   \n",
       "1  [H02, B63, F03, F24, F05, Y02]   \n",
       "2            [H02, Y02, Y04, Y10]   \n",
       "3            [F25, F24, H02, Y02]   \n",
       "4  [F03, F01, F02, F22, F05, Y02]   \n",
       "\n",
       "                                               cpc_3  cpc_1_num  cpc_2_num  \\\n",
       "0   [H02J, F03B, F03D, G05B, H02S, F05B, Y02E, Y02P]          4          5   \n",
       "1  [H02N, B63B, F03B, F03D, F24S, H02K, H02S, F05...          4          6   \n",
       "2               [H02J, H02M, Y02E, Y02P, Y04S, Y10S]          2          4   \n",
       "3                           [F25B, F24F, H02S, Y02E]          3          4   \n",
       "4               [F03G, F01K, F02C, F22B, F05D, Y02E]          2          6   \n",
       "\n",
       "   cpc_3_num  \n",
       "0          8  \n",
       "1          9  \n",
       "2          6  \n",
       "3          4  \n",
       "4          6  "
      ]
     },
     "execution_count": 85,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_2023.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "id": "6d9e52ec-270e-491a-bf96-9e9a14d33de9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>cpc_1_num</th>\n",
       "      <th>cpc_2_num</th>\n",
       "      <th>cpc_3_num</th>\n",
       "      <th>file_grant_days</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>360.000000</td>\n",
       "      <td>360.000000</td>\n",
       "      <td>360.000000</td>\n",
       "      <td>360.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>2.800000</td>\n",
       "      <td>3.905556</td>\n",
       "      <td>5.258333</td>\n",
       "      <td>342.533333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.925648</td>\n",
       "      <td>1.728661</td>\n",
       "      <td>2.578557</td>\n",
       "      <td>82.588715</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>126.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>2.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>294.750000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>3.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>351.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>3.000000</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>7.000000</td>\n",
       "      <td>396.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>7.000000</td>\n",
       "      <td>12.000000</td>\n",
       "      <td>16.000000</td>\n",
       "      <td>524.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        cpc_1_num   cpc_2_num   cpc_3_num  file_grant_days\n",
       "count  360.000000  360.000000  360.000000       360.000000\n",
       "mean     2.800000    3.905556    5.258333       342.533333\n",
       "std      0.925648    1.728661    2.578557        82.588715\n",
       "min      1.000000    1.000000    1.000000       126.000000\n",
       "25%      2.000000    3.000000    3.000000       294.750000\n",
       "50%      3.000000    4.000000    5.000000       351.000000\n",
       "75%      3.000000    5.000000    7.000000       396.000000\n",
       "max      7.000000   12.000000   16.000000       524.000000"
      ]
     },
     "execution_count": 116,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# count days from file to grant\n",
    "df_2023['file_grant_days'] = ''\n",
    "for i in range(len(df_2023)):\n",
    "    filed_date = df_2023.loc[i, 'filing_date']\n",
    "    granted_date = df_2023.loc[i, 'grant_date']\n",
    "    difference = granted_date - filed_date\n",
    "    df_2023.loc[i, 'file_grant_days'] = difference.days\n",
    "\n",
    "df_2023['file_grant_days'] = df_2023['file_grant_days'].astype(int)\n",
    "df_2023.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "id": "2835509f-57e4-42a9-93a5-e2f0d6aecad4",
   "metadata": {},
   "outputs": [],
   "source": [
    "df_2023.to_csv(r\"C:\\Users\\user\\Documents\\RenewableEnergy_patents_2023_abstract_cpc.csv\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}