Data_modeling_notebook.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "8b1d419e-09bd-4a1f-a8ae-d12d594a8d24",
   "metadata": {},
   "source": [
    "This section is used to model the Covid-19 disease and plot the results <br>\n",
    "<b><font size=7>Table of content</font></b><br>\n",
    "<font size=3.5>\n",
    "3. [Modeling](#3)<br>\n",
    "&emsp;3.1. [Methodology](#3.1)<br>\n",
    "&emsp;3.2. [Modeling databases](#3.2)<br>\n",
    "&emsp;&emsp;3.2.1. [World database](#3.2.1)<br>\n",
    "&emsp;&emsp;3.2.2. [VietNam database](#3.2.2)<br>\n",
    "&emsp;3.3. [Forecasting & Visualization](#3.3)<br>\n",
    "&emsp;&emsp;3.3.1. [World data](#3.3.1)<br>\n",
    "&emsp;&emsp;&ensp;a. [By world grand total](#3.3.1.a)<br>\n",
    "&emsp;&emsp;&ensp;b. [By each country](#3.3.1.b)<br>\n",
    "&emsp;&emsp;3.3.2. [VietNam data](#3.3.2)<br>\n",
    "&emsp;&emsp;&ensp;a. [By country grand total](#3.3.2.a)<br>\n",
    "&emsp;&emsp;&ensp;b. [By each province](#3.3.2.b)<br>\n",
    "</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f9ac70ae-0266-4770-8014-caa119ede7ff",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import os,time\n",
    "from IPython.display import clear_output\n",
    "from functools import partial\n",
    "from multiprocessing import Manager,Pool\n",
    "import plotly.graph_objects as go\n",
    "from plotly.subplots import make_subplots\n",
    "\n",
    "from covid_science.modeling import SVID_modeling,get_arima_order\n",
    "from covid_science.workers import wrapper_SVID_predict"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a8cd71fb-f64c-4bc6-9b7a-619a459c7486",
   "metadata": {},
   "source": [
    "<font size=10>**3.Modeling**</font><a name=\"3\"></a><br>\n",
    "<ins><font size=6>**3.1.Methodology**</font></ins><a name=\"3.1\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c8772c7a-5d68-4a6c-8d0d-dbd4e3f97abf",
   "metadata": {
    "tags": []
   },
   "source": [
    "A ordinary differential equation system is built to estimate the spread of Covid-19 virus base on the following information.\n",
    "\n",
    "The population will be broken down into 4 groups (S+V+I+D=N):\n",
    "* Susceptible (**S**): The main population which is susceptible to the disease without any protection **(person)**.\n",
    "* Vaccinated (**V**): The population which has been vaccinated or recovers from the disease and has protection against the disease to some degree **(person)**.\n",
    "* Infectious (**I**): The infected population, whom can infect the non-infected population **(person)**.\n",
    "* Dead (**D**): The population which passes away due to the disease (disease-induced mortality) **(person)**.\n",
    "* Affected population (**N**): The total population which is affected by the disease **(person)**.\n",
    "\n",
    "<ins>The ODE (ordinary differential equation) model:</ins>\n",
    "<font size=5>\n",
    "$$\n",
    "\\begin{align} \n",
    "&\\frac{dS}{dt} = \\frac{-\\beta SI}{N} - (\\alpha+\\mu)S + \\alpha_0V +\\pi p \\\\\n",
    "&\\frac{dV}{dt} = \\frac{-\\beta_vVI}{N} -(\\alpha_0 + \\mu)V + \\theta I + \\alpha S + \\pi(1-p) \\\\\n",
    "&\\frac{dI}{dt} = -(\\mu +\\gamma +\\theta)I + \\frac{(\\beta S + \\beta_V V)I}{N} \\\\\n",
    "&\\frac{dD}{dt} = -\\mu D + \\gamma I\n",
    "\\end{align} \n",
    "$$\n",
    "</font>\n",
    "\n",
    "<ins>With initial conditions as follows:</ins>\n",
    "\n",
    "<font size=5>\n",
    "$$\n",
    "\\begin{align} \n",
    "&S_0 \\ge 0 \\\\\n",
    "&V_0 \\ge 0 \\\\\n",
    "&I_0 \\ge 0 \\\\\n",
    "&D_0 \\ge0\n",
    "\\end{align} \n",
    "$$\n",
    "</font>\n",
    "\n",
    "<ins>And model parameter as follows:</ins>\n",
    "\n",
    "$\\pi$: recruitment rate $(person/time)$ <br>\n",
    "$p$: recruitment non-vaccinated percent *(%)* <br>\n",
    "$\\alpha$: vaccination rate $(time^{-1})$ <br>\n",
    "$\\alpha_0$: disminissing rate of vaccine effectiveness $(time^{-1})$ <br>\n",
    "$\\beta$: transmission rate $(person/time)$ <br>\n",
    "$\\mu$: natural death rate $(time^{-1})$ <br>\n",
    "$\\gamma$: Covid-19 death rate $(time^{-1})$ <br>\n",
    "$\\theta$: post-infected immunity rate $(time^{-1})$ <br>\n",
    "$\\beta_v = \\beta(1-\\eta)$: vaccinated tranmission rate $(person/time)$ <br>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "10a2b553-c225-4182-aa12-9068b8d8e34c",
   "metadata": {},
   "source": [
    "By solving the ODE system we can get the stability points:\n",
    "\n",
    "<font size=4>\n",
    "$$\n",
    "\\begin{align}\n",
    "&\\frac{dS}{dt} = 0 \\\\\n",
    "&\\frac{dV}{dt} = 0 \\\\\n",
    "&\\frac{dI}{dt} = 0 \\\\\n",
    "&\\frac{dD}{dt} = 0\n",
    "\\end{align}\n",
    "$$\n",
    "</font>\n",
    "\n",
    "[The stability of typical equilibria of smooth ODEs is determined by the sign of real part of eigenvalues of the Jacobian matrix. An equilibrium is asymptotically stable if all eigenvalues have negative real parts; it is unstable if at least one eigenvalue has positive real part.](http://www.scholarpedia.org/article/Equilibrium#:~:text=Jacobian%20Matrix,-The%20stability%20of&text=where%20all%20derivatives%20are%20evaluated,eigenvalue%20has%20positive%20real%20part.)\n",
    "\n",
    "<font size=5>\n",
    "$$\n",
    "J=D_xf = f_x = \\frac{\\partial f_i}{\\partial x_j} =\n",
    "\\begin{pmatrix}\n",
    "\\frac{\\partial f_1}{\\partial x_1} & \\frac{\\partial f_1}{\\partial x_2} & \\ldots & \\frac{\\partial f_1}{\\partial x_n} \\\\\n",
    "\\frac{\\partial f_2}{\\partial x_1} & \\frac{\\partial f_2}{\\partial x_2} & \\ldots & \\frac{\\partial f_2}{\\partial x_n} \\\\\n",
    "\\vdots & \\vdots & \\ddots & \\vdots \\\\\n",
    "\\frac{\\partial f_n}{\\partial x_1} & \\frac{\\partial f_n}{\\partial x_2} & \\ldots & \\frac{\\partial f_n}{\\partial x_n} \n",
    "\\end{pmatrix}\n",
    "$$\n",
    "</font>\n",
    "\n",
    "From there, we can possibly have two answers:\n",
    "\n",
    "* Disease free equilibrium (DFE): The point at which the population is disease-free,where as:\n",
    "\n",
    "<font size=4>\n",
    "$$\n",
    "\\begin{align} \n",
    "&S = \\frac{\\alpha_0\\pi + p\\mu\\pi}{\\alpha\\mu + \\alpha_0\\mu +\\mu^2} \\\\\n",
    "&V = \\frac{\\alpha\\pi + (1-p)\\mu\\pi}{\\alpha\\mu + \\alpha_0\\mu + \\mu^2} \\\\\n",
    "&I=0 \\\\\n",
    "&D=0 \\\\\n",
    "&R_0 = \\frac{\\alpha\\beta_v + \\alpha_0\\beta + p\\beta\\mu + \\beta_v\\mu}{(\\gamma + \\mu + \\theta)(\\alpha + \\alpha_0 + \\mu) + p\\beta_v\\mu} \\lt 1\n",
    "\\end{align} \n",
    "$$\n",
    "</font>\n",
    "\n",
    "* Endemic Equilibrium (EE): The point at which the disease is at equilibrium point (the disease is not totally eradicated and remains in the population).\n",
    "\n",
    "<font size=4>\n",
    "$$\n",
    "\\begin{align} \n",
    "&S^*\\geq 0 \\\\\n",
    "&V^*\\geq 0 \\\\\n",
    "&I^*\\geq 0 \\\\\n",
    "&D^*\\geq 0 \\\\\n",
    "&R_0 = \\frac{\\alpha\\beta_v + \\alpha_0\\beta + p\\beta\\mu + \\beta_v\\mu}{(\\gamma + \\mu + \\theta)(\\alpha + \\alpha_0 + \\mu)+p\\beta_v\\mu} \\gt 1\n",
    "\\end{align} \n",
    "$$\n",
    "</font>\n",
    "*where: $R_0$ is The Basic Reproduction Number that describes the transmissability or contagiousness of an infectious disease. If $R_0$<1, the disease will eventually die out and vice versa."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "312a67cf-36ea-4569-be91-d7f2587c6cf3",
   "metadata": {},
   "source": [
    "<ins><font size=6>**3.2.Modeling databases**</font></ins><a name=\"3.2\"></a><br>\n",
    "<ins><font size=5>**3.2.1.World database**</font></ins><a name=\"3.2.1\"></a><br>\n",
    "* <ins>Census database</ins>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b381eb55-d1c5-41cc-be17-7d4bb6d09e32",
   "metadata": {},
   "outputs": [],
   "source": [
    "#use the mean of birth and death rate from a number of years\n",
    "mean_years = range(2019,2022)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9ccc5f8d-76f3-4f54-bc16-9976f9f811f0",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "birth_df = pd.read_csv('database/Covid-19_raw_data/census_data/crude_birth_rate.csv',encoding='utf-8',\n",
    "                        usecols=['Region, subregion, country or area *',\n",
    "                                 'Location code',\n",
    "                                 'ISO3 Alpha-code',\n",
    "                                 'Year',\n",
    "                                 'Crude Birth Rate (births per 1,000 population)'])\n",
    "birth_df = birth_df.loc[birth_df.Year.isin(mean_years),['Region, subregion, country or area *',\n",
    "                                                        'Location code',\n",
    "                                                        'ISO3 Alpha-code',\n",
    "                                                        'Crude Birth Rate (births per 1,000 population)']].copy()\n",
    "birth_df = birth_df.groupby(['Region, subregion, country or area *',\n",
    "                             'Location code',\n",
    "                             'ISO3 Alpha-code'], dropna=False).mean().reset_index()\n",
    "birth_df.head(3) # number per 1000 person "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "161f69ee-7cc7-4e96-baf7-198ac5f08f77",
   "metadata": {},
   "outputs": [],
   "source": [
    "death_df = pd.read_csv('database/Covid-19_raw_data/census_data/crude_death_rate.csv',encoding='utf-8',\n",
    "                       usecols=['Region, subregion, country or area *',\n",
    "                                'Location code',\n",
    "                                'ISO3 Alpha-code',\n",
    "                                'Year',\n",
    "                                'Crude Death Rate (deaths per 1,000 population)'])\n",
    "death_df = death_df.loc[death_df.Year.isin(mean_years),['Region, subregion, country or area *',\n",
    "                                                        'Location code',\n",
    "                                                        'ISO3 Alpha-code',\n",
    "                                                        'Crude Death Rate (deaths per 1,000 population)']].copy()\n",
    "death_df = death_df.groupby(['Region, subregion, country or area *',\n",
    "                             'Location code',\n",
    "                             'ISO3 Alpha-code'], dropna=False).mean().reset_index()\n",
    "death_df.head(3) # number per 1000 person "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f825d480-164f-4684-869d-c430b78172d3",
   "metadata": {},
   "source": [
    "* <ins>Modeling input database</ins>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0e96bb74-f0ae-4e82-a452-4f2f99613534",
   "metadata": {},
   "outputs": [],
   "source": [
    "#modeling data of total world\n",
    "world_df = pd.read_csv(\"database/Covid-19_modeling_input/WorldData/w_modeling_data.csv\",parse_dates=['date'])\n",
    "# world_df = world_df.iloc[:-1].copy()\n",
    "\n",
    "#world average birth_rate/1000 person/year\n",
    "w_b_rate = birth_df.loc[birth_df['Location code']==900,'Crude Birth Rate (births per 1,000 population)'].values[0]\n",
    "w_birth_rate = (1+(w_b_rate/1000))**(1/365)-1 #(convert to %/day)\n",
    "#world average death_rate/1000 person/year\n",
    "w_d_rate = death_df.loc[death_df['Location code']==900,'Crude Death Rate (deaths per 1,000 population)'].values[0]\n",
    "w_death_rate = (1+(w_d_rate/1000))**(1/365)-1 #(convert to %/day)\n",
    "\n",
    "#modeling data by each country\n",
    "bc_df = pd.read_csv(\"database/Covid-19_modeling_input/WorldData/w_modeling_data_by_country.csv\",parse_dates=['date'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "86d1c2dc-23c5-4940-a871-c6d9d994f7be",
   "metadata": {},
   "source": [
    "<ins><font size=5>**3.2.2.VietNam database**</font></ins><a name=\"3.2.2\"></a><br>\n",
    "* <ins>Modeling input database</ins>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6da24934-caf6-4f9c-8f99-2b09a5629427",
   "metadata": {},
   "outputs": [],
   "source": [
    "vn_df = pd.read_csv(\"database/Covid-19_modeling_input/VietNamData/vn_modeling_data.csv\",parse_dates=['date'])\n",
    "# vn_df = vn_df.iloc[:-1].copy()\n",
    "\n",
    "#average birth_rate/1000 person/year\n",
    "b_rate = birth_df.loc[birth_df['ISO3 Alpha-code']=='VNM','Crude Birth Rate (births per 1,000 population)'].values[0] \n",
    "vn_birth_rate = (1+(b_rate/1000))**(1/365)-1 #(convert to %/day)\n",
    "#average death_rate/1000 person/year\n",
    "d_rate = death_df.loc[death_df['ISO3 Alpha-code']=='VNM','Crude Death Rate (deaths per 1,000 population)'].values[0] \n",
    "vn_death_rate = (1+(d_rate/1000))**(1/365)-1 #(convert to %/day)\n",
    "\n",
    "#modeling data by each province\n",
    "vn_bp_df = pd.read_csv(\"database/Covid-19_modeling_input/VietNamData/vn_modeling_data_by_province.csv\",parse_dates=['date'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "32051cc1-904a-470b-8ee9-0caac127d8ee",
   "metadata": {},
   "source": [
    "<ins><font size=6>**3.3.Forecasting & Visualization**</font></ins><a name=\"3.3\"></a><br>\n",
    "<ins><font size=5>**3.3.1.World data**</font></ins><a name=\"3.3.1\"></a><br>\n",
    "<font size=4><ins>**a. By world grand total**</ins></font><a name=\"3.3.1.a\"></a><br>\n",
    "* <ins>General input parameters</ins>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d2d78756-6a10-4b38-833b-ee7cd447790d",
   "metadata": {},
   "outputs": [],
   "source": [
    "#input for world data processing\n",
    "w_p = 1.0\n",
    "\n",
    "#input for SVID_modeling\n",
    "w_start_date = world_df.iloc[-1].date\n",
    "w_days = 7\n",
    "w_model_number=[1,2,3]\n",
    "w_scoring = 'neg_mean_absolute_percentage_error'\n",
    "w_greater_better = False\n",
    "w_best_model = True #if no best model is chosen, the data of the first model will be chosen for plotting\n",
    "w_multioutput = \"variance_weighted\"\n",
    "w_preset_order = None #for application, this can be preset for faster model calculation\n",
    "w_param_kwargs = None\n",
    "w_plot_point = 30\n",
    "\n",
    "#conversion to real data\n",
    "w_r_protect_time = 150\n",
    "w_n_days = 1000 "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "07dcb3df-0ca1-449e-91f7-516ce4c5570c",
   "metadata": {},
   "source": [
    "* <ins>Data processing and forecast SVID model future values</ins>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7fde8736-5b7b-4d8f-b21b-7a2ff7b26f3a",
   "metadata": {},
   "outputs": [],
   "source": [
    "start = time.time()\n",
    "w_manager = Manager()\n",
    "w_counter= w_manager.Array('i',[0,0])\n",
    "w_time_list = w_manager.list([])\n",
    "w_error_dict = w_manager.dict()\n",
    "w_shared_output = w_manager.Namespace()\n",
    "w_shared_output.shared_df = pd.DataFrame()\n",
    "w_shared_output.equil_df = pd.DataFrame()\n",
    "w_lock = w_manager.Lock()\n",
    "\n",
    "#extra argument for function input\n",
    "w_input_kwargs = {'predict_days':w_days,\n",
    "                   'date_col':'date',\n",
    "                   'initial_state':['S0','V0','I0','D0'],\n",
    "                   'model_param':['beta','beta_v','gamma','theta','alpha','alpha0','pi'],\n",
    "                   'model_number':w_model_number,\n",
    "                   'best_model':w_best_model,\n",
    "                   'scoring':w_scoring,\n",
    "                   'multioutput':w_multioutput,\n",
    "                   'greater_better':w_greater_better,\n",
    "                   'preset_arima_order': w_preset_order, #for fast modeling, if set to None, it will be much slower \n",
    "                   'param_kwargs':w_param_kwargs\n",
    "                  }\n",
    "\n",
    "#worker function\n",
    "worker_func = partial(wrapper_SVID_predict,\n",
    "                      shared_count=w_counter,\n",
    "                      time_list = w_time_list,\n",
    "                      error_dict=w_error_dict,\n",
    "                      shared_output=w_shared_output,\n",
    "                      lock=w_lock,\n",
    "                      input_data=(world_df,'iso_code'),\n",
    "                      start_date=w_start_date,\n",
    "                      birth_data=w_birth_rate,\n",
    "                      death_data=w_death_rate,\n",
    "                      p=w_p,\n",
    "                      r_protect_time=w_r_protect_time,\n",
    "                      n_days = w_n_days,\n",
    "                      **w_input_kwargs)\n",
    "\n",
    "worker_func('OWID_WRL')\n",
    "\n",
    "w_predict_df = w_shared_output.shared_df.copy()\n",
    "w_predict_df = w_predict_df.sort_values(['iso_code','method','date']).reset_index(drop=True)\n",
    "w_equil_df = w_shared_output.equil_df.copy()\n",
    "w_error_result = dict(w_error_dict)\n",
    "w_time = list(w_time_list)\n",
    "\n",
    "w_manager.shutdown()\n",
    "\n",
    "end = time.time()\n",
    "\n",
    "print(f\"Total processing time: {end-start}s\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "af1d678d-9156-4e94-8af7-b848b5eeecb1",
   "metadata": {},
   "source": [
    "* <ins>Store modeling data</ins>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6a171dd9-4a4c-46bf-82b6-841e7030be1f",
   "metadata": {},
   "outputs": [],
   "source": [
    "folder_path = \"database/Covid-19_modeling_output/WorldData\"\n",
    "if os.path.exists(str(folder_path)):\n",
    "    pass\n",
    "else:\n",
    "    os.makedirs(str(folder_path))\n",
    "    print(\"New folder created.\")\n",
    "w_predict_df.to_csv(folder_path +\"/\"+\"w_prediction.csv\",encoding=\"utf-8\",index=False)\n",
    "w_equil_df.to_csv(folder_path +\"/\"+\"w_equilibrium.csv\",encoding=\"utf-8\",index=False)\n",
    "w_predict_df.head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "394391b4-fb8b-4fa3-9572-bf909c273148",
   "metadata": {},
   "source": [
    "* <ins>Plot forecast data</ins>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c8cedecd-49dc-40fa-9684-91b3d6aea2db",
   "metadata": {},
   "outputs": [],
   "source": [
    "##making frame for the subplots\n",
    "\n",
    "w_specs = [[{},{}],\n",
    "            [{},{}],\n",
    "            [{},{}],\n",
    "            [{},{}],\n",
    "            [{},{}],\n",
    "            [{\"colspan\": 2},None]]\n",
    "sub_title = [\"Forecasted S\",\n",
    "             \"Forecasted V\",\n",
    "             \"Forecasted I\",\n",
    "             \"Forecasted D\",\n",
    "             \"Forecasted daily_case_non_vaccinated\",\n",
    "             \"Forecasted daily_case_vaccinated\",\n",
    "             \"Forecasted death_avg\",\n",
    "             \"Forecasted recovery_case\",\n",
    "             \"Forecasted new_full_vaccinated\",\n",
    "             \"Forecasted new_boost_req\",\n",
    "             \"Equilibrium plot\"]\n",
    "w_fig = make_subplots(rows=6,cols=2,specs=w_specs,subplot_titles=sub_title,vertical_spacing=0.05,horizontal_spacing=0.05)\n",
    "\n",
    "for plot_color in ['blue','red']:\n",
    "    for i in range(0,10):\n",
    "        if i%2==0:\n",
    "            add_col=1\n",
    "            showlegend=False\n",
    "        else:\n",
    "            add_col=2\n",
    "            showlegend=True\n",
    "        add_row = (i//2)+1\n",
    "        if plot_color=='blue':\n",
    "            data_name=\"real\"\n",
    "        else:\n",
    "            data_name=\"forecast\"\n",
    "        w_fig.add_trace(go.Scatter(name=data_name,line={'color':plot_color},legendgroup=str(i),showlegend=showlegend,mode='lines'),\n",
    "                         row=add_row,col=add_col)\n",
    "w_fig.add_trace(go.Scatter(name=\"S\",line={'color':'green'},legendgroup='10',mode='lines'),row=6,col=1)\n",
    "w_fig.add_trace(go.Scatter(name=\"V\",line={'color':'blue'},legendgroup='10',mode='lines'),row=6,col=1)\n",
    "w_fig.add_trace(go.Scatter(name=\"I\",line={'color':'orange'},legendgroup='10',mode='lines'),row=6,col=1)\n",
    "\n",
    "##create buttons data for dropdown list\n",
    "\n",
    "w_real_data = (world_df.loc[world_df.date<=w_start_date].iloc[-w_plot_point:,[0,4,5,6,7,19,20,15,16,17,18]].to_numpy())\n",
    "\n",
    "#forecast data\n",
    "if w_best_model:\n",
    "    w_forecast_data = w_predict_df.loc[w_predict_df.iso_code=='OWID_WRL'].\\\n",
    "    loc[:,['S0','V0','I0','D0','daily_case_non_vaccinated','daily_case_vaccinated','death_avg','recovery_case','new_full_vaccinated','new_boost_req']].to_numpy()\n",
    "else: #if no best model is chosen, the data of the first model will be chosen\n",
    "    w_forecast_data = w_predict_df.loc[(w_predict_df.iso_code=='OWID_WRL')&(w_predict_df.method==w_model_number[0])].\\\n",
    "    loc[:,['S0','V0','I0','D0','daily_case_non_vaccinated','daily_case_vaccinated','death_avg','recovery_case','new_full_vaccinated','new_boost_req']].to_numpy()\n",
    "w_forecast_date = pd.date_range(start=w_start_date,periods=w_days+1)\n",
    "\n",
    "#get reproduction rate\n",
    "w_R0 = world_df.loc[world_df.date==w_start_date,['daily_R0', 'avg_R0']].to_numpy().squeeze().tolist()\n",
    "\n",
    "w_date_range = pd.date_range(w_start_date,periods=w_n_days)\n",
    "\n",
    "x_list=[]\n",
    "x_list.extend([w_real_data[:,0]]*10)\n",
    "x_list.extend([w_forecast_date]*10)\n",
    "x_list.extend([w_equil_df.date.to_numpy()]*3)\n",
    "\n",
    "y_list = []\n",
    "y_list.extend(w_real_data[:,1:].T)\n",
    "y_list.extend(w_forecast_data.T)\n",
    "y_list.extend(w_equil_df[['S','V','I']].to_numpy().T)\n",
    "\n",
    "#update plot data\n",
    "for i,data in enumerate(zip(x_list,y_list)):\n",
    "    w_fig.data[i].update(x=data[0],y=data[1])\n",
    "\n",
    "w_fig.layout.annotations[10].update(text=f'Equilibrium plot (daily_R0:{round(w_R0[0],2)} - average_R0:{round(w_R0[1],2)})')\n",
    "clear_output()\n",
    "w_fig.update_layout(title_text=\"Total world forecast\",title_x=0.5,title_font_size=50,\n",
    "                   height=2000,\n",
    "                   legend_tracegroupgap=285)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d41dff05-b234-4802-bf55-4445533748f3",
   "metadata": {},
   "source": [
    "<font size=4><ins>**b. By each country**</ins></font><a name=\"3.3.1.b\"></a><br>\n",
    "* <ins>General input parameters</ins>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "aaecd253-f3f4-4fef-a034-747002b9b7c8",
   "metadata": {},
   "outputs": [],
   "source": [
    "#input for each country data processing\n",
    "bc_p = 1.0\n",
    "\n",
    "#input for SVID_modeling\n",
    "bc_start_date = None \n",
    "bc_days = 7\n",
    "bc_model_number=[1,2,3]\n",
    "bc_scoring = 'neg_mean_absolute_percentage_error'\n",
    "bc_greater_better = False\n",
    "bc_best_model = True\n",
    "bc_multioutput = \"variance_weighted\"\n",
    "bc_param_kwargs = None\n",
    "bc_plot_point = 30\n",
    "\n",
    "#conversion to real data\n",
    "bc_r_protect_time = 150\n",
    "bc_n_days = 1000 \n",
    "\n",
    "# get general ARIMA order, seasonal order from world total data using for each country\n",
    "bc_preset_order = get_arima_order(input_df=world_df,\n",
    "                                   cols=['beta','beta_v','gamma','theta','alpha','alpha0','pi'],\n",
    "                                   start=world_df.loc[world_df.date==w_start_date].index[0])\n",
    "\n",
    "bc_max_worker = 6"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "71cff7c4-89a1-4d0f-9784-aa9cd11a17ba",
   "metadata": {},
   "source": [
    "* <ins>Data processing and forecast SVID model future values</ins>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b41d6c2d-2743-4f5d-bbcc-320ae6675fa6",
   "metadata": {},
   "outputs": [],
   "source": [
    "start_cell = time.perf_counter()\n",
    "\n",
    "bc_manager = Manager()\n",
    "bc_counter= bc_manager.Array('i',[0,0])\n",
    "bc_time_list = bc_manager.list([])\n",
    "bc_error_dict = bc_manager.dict()\n",
    "bc_shared_output = bc_manager.Namespace()\n",
    "bc_shared_output.shared_df = pd.DataFrame()\n",
    "bc_shared_output.equil_df = pd.DataFrame()\n",
    "bc_lock = bc_manager.Lock()\n",
    "\n",
    "#list of country iso_code\n",
    "bc_iso_code = bc_df.iso_code.unique()\n",
    "\n",
    "\n",
    "#extra argument for function input\n",
    "bc_input_kwargs = {'predict_days':bc_days,\n",
    "                   'date_col':'date',\n",
    "                   'initial_state':['S0','V0','I0','D0'],\n",
    "                   'model_param':['beta','beta_v','gamma','theta','alpha','alpha0','pi'],\n",
    "                   'model_number':bc_model_number,\n",
    "                   'best_model':bc_best_model,\n",
    "                   'scoring':bc_scoring,\n",
    "                   'multioutput':bc_multioutput,\n",
    "                   'greater_better':bc_greater_better,\n",
    "                   'preset_arima_order': bc_preset_order, #for fast modeling, if set to None, it will be much slower \n",
    "                   'param_kwargs':bc_param_kwargs\n",
    "                  }\n",
    "\n",
    "#worker function\n",
    "worker_func = partial(wrapper_SVID_predict,\n",
    "                      shared_count=bc_counter,\n",
    "                      time_list = bc_time_list,\n",
    "                      error_dict=bc_error_dict,\n",
    "                      shared_output=bc_shared_output,\n",
    "                      lock=bc_lock,\n",
    "                      input_data=(bc_df,'iso_code'),\n",
    "                      start_date=bc_start_date,\n",
    "                      birth_data=(birth_df,'ISO3 Alpha-code','Crude Birth Rate (births per 1,000 population)'),\n",
    "                      death_data=(death_df,'ISO3 Alpha-code','Crude Death Rate (deaths per 1,000 population)'),\n",
    "                      p=bc_p,\n",
    "                      r_protect_time=bc_r_protect_time,\n",
    "                      n_days = bc_n_days,\n",
    "                      **bc_input_kwargs)\n",
    "\n",
    "executor= Pool(processes=bc_max_worker)\n",
    "start_worker = time.perf_counter()\n",
    "pool_result = executor.map_async(worker_func,bc_iso_code)\n",
    "\n",
    "max_country = len(bc_iso_code)\n",
    "while bc_counter[0]<max_country:\n",
    "    time.sleep(1)\n",
    "    n_done = bc_counter[0]\n",
    "    n_wait = max_country\n",
    "    total_time = round(sum(bc_time_list),2)\n",
    "    if n_done!=0:\n",
    "        avg_time = round(total_time/n_done,2)\n",
    "    if n_done!=0:\n",
    "        remain_country = n_wait - n_done\n",
    "        remain_time = round(remain_country*avg_time,2)\n",
    "        real_remain = round(remain_time/bc_max_worker,2)\n",
    "    else:\n",
    "        remain_country = \"estimating...\"\n",
    "        remain_time = \"-- \"\n",
    "        real_remain = \"-- \"\n",
    "        \n",
    "    elapsed_time = round(time.perf_counter()-start_worker,2)\n",
    "    a_string = (f\"Elapsed time: {elapsed_time}s    \" +\n",
    "                f\"Total workers processing time: {total_time}s    \" + \n",
    "                f\" Remaining countries: {remain_country}    \" + \n",
    "                f\" Estimated remaining worker time: {remain_time}s    \" +\n",
    "                f\" Estimated remaining real time: {real_remain}s\")\n",
    "    \n",
    "    print(\" \"*(len(a_string)+10),end=\"\\r\")\n",
    "    print(a_string,end=\"\\r\")\n",
    "\n",
    "print(\" \"*(len(a_string)+10),end=\"\\r\")\n",
    "print(\"---Forecast processing completed---\")\n",
    "print(f\"Number of workers with error happened: {bc_counter[1]}\")\n",
    "print(f\"Total worker processing time taken: {round(sum(bc_time_list),2)}s\")\n",
    "\n",
    "bc_predict_df = bc_shared_output.shared_df.copy()\n",
    "bc_predict_df = bc_predict_df.sort_values(['iso_code','method','date']).reset_index(drop=True)\n",
    "bc_equil_df = bc_shared_output.equil_df.copy()\n",
    "bc_equil_df = bc_equil_df.sort_values(['iso_code','date']).reset_index(drop=True)\n",
    "bc_error_result = dict(bc_error_dict)\n",
    "bc_time = list(bc_time_list)\n",
    "\n",
    "executor.close()\n",
    "executor.join()\n",
    "bc_manager.shutdown()\n",
    "\n",
    "end_cell = time.perf_counter()\n",
    "print(f\"Total elapsed time on this code block: {round(end_cell-start_cell,2)}s\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a2fde39d-174a-4edb-a4c9-21225e276129",
   "metadata": {},
   "source": [
    "* <ins>Store modeling data</ins>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "81fb087a-345f-41e4-a710-63ccbbe38665",
   "metadata": {},
   "outputs": [],
   "source": [
    "folder_path = \"database/Covid-19_modeling_output/WorldData\"\n",
    "if os.path.exists(str(folder_path)):\n",
    "    pass\n",
    "else:\n",
    "    os.makedirs(str(folder_path))\n",
    "    print(\"New folder created.\")\n",
    "bc_predict_df.to_csv(folder_path +\"/\"+\"w_prediction_by_country.csv\",encoding=\"utf-8\",index=False)\n",
    "bc_equil_df.to_csv(folder_path +\"/\"+\"w_equilibrium_by_country.csv\",encoding=\"utf-8\",index=False)\n",
    "bc_predict_df.head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2d782aee-af0e-4ac1-95b2-e8da981bec15",
   "metadata": {},
   "source": [
    "* <ins>Plot forecast data</ins>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "75a4550d-0bbb-4c4a-8144-75a545cc6004",
   "metadata": {},
   "outputs": [],
   "source": [
    "##making frame for the subplots\n",
    "bc_col_loc1 = [bc_df.columns.get_loc('date'),\n",
    "              bc_df.columns.get_loc('S0'),\n",
    "              bc_df.columns.get_loc('V0'),\n",
    "              bc_df.columns.get_loc('I0'),\n",
    "              bc_df.columns.get_loc('D0'),\n",
    "              bc_df.columns.get_loc('daily_case_non_vaccinated'),\n",
    "              bc_df.columns.get_loc('daily_case_vaccinated'),\n",
    "              bc_df.columns.get_loc('death_avg'),\n",
    "              bc_df.columns.get_loc('recovery_case'),\n",
    "              bc_df.columns.get_loc('new_full_vaccinated'),\n",
    "              bc_df.columns.get_loc('new_boost_req')]\n",
    "bc_col_loc2 = [bc_predict_df.columns.get_loc('date'),\n",
    "              bc_predict_df.columns.get_loc('S0'),\n",
    "              bc_predict_df.columns.get_loc('V0'),\n",
    "              bc_predict_df.columns.get_loc('I0'),\n",
    "              bc_predict_df.columns.get_loc('D0'),\n",
    "              bc_predict_df.columns.get_loc('daily_case_non_vaccinated'),\n",
    "              bc_predict_df.columns.get_loc('daily_case_vaccinated'),\n",
    "              bc_predict_df.columns.get_loc('death_avg'),\n",
    "              bc_predict_df.columns.get_loc('recovery_case'),\n",
    "              bc_predict_df.columns.get_loc('new_full_vaccinated'),\n",
    "              bc_predict_df.columns.get_loc('new_boost_req')]\n",
    "\n",
    "bc_specs = [[{},{}],\n",
    "            [{},{}],\n",
    "            [{},{}],\n",
    "            [{},{}],\n",
    "            [{},{}],\n",
    "            [{\"colspan\": 2},None]]\n",
    "sub_title = [\"Forecasted S\",\n",
    "             \"Forecasted V\",\n",
    "             \"Forecasted I\",\n",
    "             \"Forecasted D\",\n",
    "             \"Forecasted daily_case_non_vaccinated\",\n",
    "             \"Forecasted daily_case_vaccinated\",\n",
    "             \"Forecasted death_avg\",\n",
    "             \"Forecasted recovery_case\",\n",
    "             \"Forecasted new_full_vaccinated\",\n",
    "             \"Forecasted new_boost_req\",\n",
    "             \"Equilibrium plot\"]\n",
    "bc_fig = make_subplots(rows=6,cols=2,specs=bc_specs,subplot_titles=sub_title,vertical_spacing=0.05,horizontal_spacing=0.05)\n",
    "bc_fig['layout']['annotations'] += (dict(text=\"Country:\", x=-0.02, xref=\"paper\",xanchor='left', y=1.042, yref=\"paper\",yanchor=\"top\",\n",
    "                                                                                     align=\"left\", showarrow=False),)\n",
    "\n",
    "for plot_color in ['blue','red']:\n",
    "    for i in range(0,10):\n",
    "        if i%2==0:\n",
    "            add_col=1\n",
    "            showlegend=False\n",
    "        else:\n",
    "            add_col=2\n",
    "            showlegend=True\n",
    "        add_row = (i//2)+1\n",
    "        if plot_color=='blue':\n",
    "            data_name=\"real\"\n",
    "        else:\n",
    "            data_name=\"forecast\"\n",
    "        bc_fig.add_trace(go.Scatter(name=data_name,line={'color':plot_color},legendgroup=str(i),showlegend=showlegend,mode='lines'),\n",
    "                         row=add_row,col=add_col)\n",
    "bc_fig.add_trace(go.Scatter(name=\"S\",line={'color':'green'},legendgroup='10',mode='lines'),row=6,col=1)\n",
    "bc_fig.add_trace(go.Scatter(name=\"V\",line={'color':'blue'},legendgroup='10',mode='lines'),row=6,col=1)\n",
    "bc_fig.add_trace(go.Scatter(name=\"I\",line={'color':'orange'},legendgroup='10',mode='lines'),row=6,col=1)\n",
    "\n",
    "##create buttons data for dropdown list\n",
    "\n",
    "update_menu=[]\n",
    "country_list = bc_df[['iso_code','location']].drop_duplicates(ignore_index=True)\n",
    "annotation_list = []\n",
    "for note in bc_fig.layout.annotations:\n",
    "    annotation_list.append(go.layout.Annotation(note))\n",
    "    \n",
    "for code in bc_predict_df.iso_code.unique():\n",
    "    if bc_start_date is not None:\n",
    "        bc_start_stamp = pd.Timestamp(bc_start_date)\n",
    "        real_data = (bc_df.loc[bc_df.iso_code==code]\n",
    "                     .loc[(bc_df.date<=bc_start_stamp)&(bc_df.date>=(bc_start_stamp-pd.Timedelta(bc_plot_point,unit=\"D\")))]\n",
    "                     .iloc[-bc_plot_point:,bc_col_loc1].to_numpy())\n",
    "    else:\n",
    "        bc_start_stamp = pd.Timestamp(bc_df.loc[bc_df.iso_code==code].date.values[-1])\n",
    "        real_data = bc_df.loc[bc_df.iso_code==code].iloc[-bc_plot_point:,bc_col_loc1].to_numpy()\n",
    "    \n",
    "    #forecast data\n",
    "    if bc_best_model:\n",
    "        forecast_data = bc_predict_df.loc[bc_predict_df.iso_code==code].iloc[:,bc_col_loc2].to_numpy()\n",
    "    else: #if no best model is chosen, the data of the first model will be chosen\n",
    "        forecast_data = bc_predict_df.loc[(bc_predict_df.iso_code==code)&(bc_predict_df.method==bc_model_number[0])].iloc[:,bc_col_loc2].to_numpy()\n",
    "    \n",
    "    #get reproduction rate\n",
    "    bc_R0 = bc_df.loc[bc_df.iso_code==code].loc[bc_df.date==bc_start_stamp,['daily_R0', 'avg_R0']].to_numpy().squeeze().tolist()\n",
    "        \n",
    "    x_list=[]\n",
    "    x_list.extend([real_data[:,0]]*10)\n",
    "    x_list.extend([forecast_data[:,0]]*10)\n",
    "    x_list.extend([bc_equil_df.loc[bc_equil_df.iso_code==code,'date'].to_numpy()]*3)\n",
    "    \n",
    "    y_list = []\n",
    "    y_list.extend(real_data[:,1:].T)\n",
    "    y_list.extend(forecast_data[:,1:].T)\n",
    "    y_list.extend(bc_equil_df.loc[bc_equil_df.iso_code==code,['S','V','I']].to_numpy().T)\n",
    "    \n",
    "    #The main title of each button\n",
    "    plot_title=country_list.loc[country_list.iso_code==code,'location'].values[0]\n",
    "    \n",
    "    button_annotation = annotation_list.copy()\n",
    "    button_annotation[10] = go.layout.Annotation(bc_fig.layout.annotations[10]).update(text=f'Equilibrium plot (daily_R0:{round(bc_R0[0],2)} - average_R0:{round(bc_R0[1],2)})')\n",
    "    button =  dict(label=code,\n",
    "                   method=\"update\",\n",
    "                   args=[{\"y\":y_list,\n",
    "                          \"x\":x_list},\n",
    "                         {\"title\":{\"text\":plot_title,\"x\":0.5,\"font\":{\"size\":50}},\n",
    "                          \"annotations\":button_annotation}\n",
    "                   ]\n",
    "                  )\n",
    "    update_menu.append(button)\n",
    "    \n",
    "##plotting\n",
    "\n",
    "bc_fig.update_layout(updatemenus=[dict(active=-1,\n",
    "                                      buttons=update_menu,\n",
    "                                      pad={\"r\": 10, \"t\": 10},\n",
    "                                      showactive=True,\n",
    "                                      xanchor=\"left\",\n",
    "                                      x=0.05,\n",
    "                                      yanchor=\"top\",\n",
    "                                      y=1.05)\n",
    "                                  ],\n",
    "                     legend_tracegroupgap=285,\n",
    "                     height=2000\n",
    "                    )\n",
    "clear_output()\n",
    "bc_fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "99839e0c-d019-4318-a13b-2fff2377a9a4",
   "metadata": {},
   "source": [
    "<ins><font size=5>**3.3.2.VietNam data**</font></ins><a name=\"3.3.2\"></a><br>\n",
    "<font size=4><ins>**a. By country grand total**</ins></font><a name=\"3.3.2.a\"></a><br>\n",
    "* <ins>General input parameters</ins>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1af59e26-2502-4f2d-ac6b-ab5aba506b74",
   "metadata": {},
   "outputs": [],
   "source": [
    "#input for Vietnam data processing\n",
    "vn_p=1.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9ddd78f1-e76d-4109-ae8d-d07123aac14b",
   "metadata": {},
   "outputs": [],
   "source": [
    "#input for SVID_modeling\n",
    "vn_start_date = vn_df.iloc[-1].date\n",
    "vn_days = 7\n",
    "vn_model_number=[1,2,3]\n",
    "vn_scoring = 'neg_mean_absolute_percentage_error'\n",
    "vn_greater_better = False\n",
    "vn_best_model = True #if no best model is chosen, the data of the first model will be chosen for plotting\n",
    "vn_multioutput = \"variance_weighted\"\n",
    "vn_preset_order = None #for application, this can be preset for faster model calculation\n",
    "vn_param_kwargs = None\n",
    "vn_plot_point = 30\n",
    "\n",
    "#conversion to real data\n",
    "vn_r_protect_time = 150\n",
    "vn_n_days = 1000"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ccbf5fb7-d56f-429e-86ae-55c8df3d9d71",
   "metadata": {},
   "source": [
    "* <ins>Data processing and forecast SVID model future values</ins>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9ecec8af-85d6-46cc-aa5e-b77d318e2766",
   "metadata": {},
   "outputs": [],
   "source": [
    "start = time.time()\n",
    "\n",
    "vn_manager = Manager()\n",
    "vn_counter= vn_manager.Array('i',[0,0])\n",
    "vn_time_list = vn_manager.list([])\n",
    "vn_error_dict = vn_manager.dict()\n",
    "vn_shared_output = vn_manager.Namespace()\n",
    "vn_shared_output.shared_df = pd.DataFrame()\n",
    "vn_shared_output.equil_df = pd.DataFrame()\n",
    "\n",
    "vn_lock = vn_manager.Lock()\n",
    "\n",
    "\n",
    "#extra argument for function input\n",
    "vn_input_kwargs = {'predict_days':vn_days,\n",
    "                   'date_col':'date',\n",
    "                   'initial_state':['S0','V0','I0','D0'],\n",
    "                   'model_param':['beta','beta_v','gamma','theta','alpha','alpha0','pi'],\n",
    "                   'model_number':vn_model_number,\n",
    "                   'best_model':vn_best_model,\n",
    "                   'scoring':vn_scoring,\n",
    "                   'multioutput':vn_multioutput,\n",
    "                   'greater_better':vn_greater_better,\n",
    "                   'preset_arima_order': vn_preset_order, #for fast modeling, if set to None, it will be much slower \n",
    "                   'param_kwargs':vn_param_kwargs\n",
    "                  }\n",
    "\n",
    "#worker function\n",
    "worker_func = partial(wrapper_SVID_predict,\n",
    "                      shared_count=vn_counter,\n",
    "                      time_list = vn_time_list,\n",
    "                      error_dict=vn_error_dict,\n",
    "                      shared_output=vn_shared_output,\n",
    "                      lock=vn_lock,\n",
    "                      input_data=(vn_df,'iso_code'),\n",
    "                      start_date=vn_start_date,\n",
    "                      birth_data=vn_birth_rate,\n",
    "                      death_data=vn_death_rate,\n",
    "                      p=vn_p,\n",
    "                      r_protect_time=vn_r_protect_time,\n",
    "                      n_days = vn_n_days,\n",
    "                      **vn_input_kwargs)\n",
    "\n",
    "worker_func('VNM')\n",
    "\n",
    "vn_predict_df = vn_shared_output.shared_df.copy()\n",
    "vn_predict_df = vn_predict_df.sort_values(['iso_code','method','date']).reset_index(drop=True)\n",
    "vn_equil_df = vn_shared_output.equil_df.copy()\n",
    "vn_error_result = dict(vn_error_dict)\n",
    "vn_time = list(vn_time_list)\n",
    "\n",
    "vn_manager.shutdown()\n",
    "\n",
    "end = time.time()\n",
    "\n",
    "print(f\"Total processing time: {end-start}s\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "608933e2-1f21-4707-8323-1b5c6ece3a8b",
   "metadata": {},
   "source": [
    "* <ins>Store modeling data</ins>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "72cc353f-fc9e-44bc-b9c8-25500facfae4",
   "metadata": {},
   "outputs": [],
   "source": [
    "folder_path = \"database/Covid-19_modeling_output/VietNamData\"\n",
    "if os.path.exists(str(folder_path)):\n",
    "    pass\n",
    "else:\n",
    "    os.makedirs(str(folder_path))\n",
    "    print(\"New folder created.\")\n",
    "vn_predict_df.to_csv(folder_path +\"/\"+\"vn_prediction.csv\",encoding=\"utf-8\",index=False)\n",
    "vn_equil_df.to_csv(folder_path +\"/\"+\"vn_equilibrium.csv\",encoding=\"utf-8\",index=False)\n",
    "vn_predict_df.head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d84a2d9e-3804-4887-969f-955a8cd2bb1b",
   "metadata": {},
   "source": [
    "* <ins>Plot forecast data</ins>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b2f49612-9f99-4b00-80bb-623dc8df002a",
   "metadata": {},
   "outputs": [],
   "source": [
    "##making frame for the subplots\n",
    "\n",
    "vn_specs = [[{},{}],\n",
    "            [{},{}],\n",
    "            [{},{}],\n",
    "            [{},{}],\n",
    "            [{},{}],\n",
    "            [{\"colspan\": 2},None]]\n",
    "sub_title = [\"Forecasted S\",\n",
    "             \"Forecasted V\",\n",
    "             \"Forecasted I\",\n",
    "             \"Forecasted D\",\n",
    "             \"Forecasted daily_case_non_vaccinated\",\n",
    "             \"Forecasted daily_case_vaccinated\",\n",
    "             \"Forecasted death_avg\",\n",
    "             \"Forecasted recovery_case\",\n",
    "             \"Forecasted new_full_vaccinated\",\n",
    "             \"Forecasted new_boost_req\",\n",
    "             \"Equilibrium plot\"]\n",
    "vn_fig = make_subplots(rows=6,cols=2,specs=vn_specs,subplot_titles=sub_title,vertical_spacing=0.05,horizontal_spacing=0.05)\n",
    "\n",
    "for plot_color in ['blue','red']:\n",
    "    for i in range(0,10):\n",
    "        if i%2==0:\n",
    "            add_col=1\n",
    "            showlegend=False\n",
    "        else:\n",
    "            add_col=2\n",
    "            showlegend=True\n",
    "        add_row = (i//2)+1\n",
    "        if plot_color=='blue':\n",
    "            data_name=\"real\"\n",
    "        else:\n",
    "            data_name=\"forecast\"\n",
    "        vn_fig.add_trace(go.Scatter(name=data_name,line={'color':plot_color},legendgroup=str(i),showlegend=showlegend,mode='lines'),\n",
    "                         row=add_row,col=add_col)\n",
    "vn_fig.add_trace(go.Scatter(name=\"S\",line={'color':'green'},legendgroup='10',mode='lines'),row=6,col=1)\n",
    "vn_fig.add_trace(go.Scatter(name=\"V\",line={'color':'blue'},legendgroup='10',mode='lines'),row=6,col=1)\n",
    "vn_fig.add_trace(go.Scatter(name=\"I\",line={'color':'orange'},legendgroup='10',mode='lines'),row=6,col=1)\n",
    "\n",
    "##create buttons data for dropdown list\n",
    "\n",
    "vn_real_data = (vn_df.loc[vn_df.date<=vn_start_date].iloc[-vn_plot_point:,[0,2,3,4,5,17,18,13,14,15,16]].to_numpy())\n",
    "\n",
    "#forecast data\n",
    "if vn_best_model:\n",
    "    vn_forecast_data = vn_predict_df.loc[vn_predict_df.iso_code=='VNM'].\\\n",
    "    loc[:,['S0','V0','I0','D0','daily_case_non_vaccinated','daily_case_vaccinated','death_avg','recovery_case','new_full_vaccinated','new_boost_req']].to_numpy()\n",
    "else: #if no best model is chosen, the data of the first model will be chosen\n",
    "    vn_forecast_data = vn_predict_df.loc[(vn_predict_df.iso_code=='VNM')&(vn_predict_df.method==vn_model_number[0])].\\\n",
    "    loc[:,['S0','V0','I0','D0','daily_case_non_vaccinated','daily_case_vaccinated','death_avg','recovery_case','new_full_vaccinated','new_boost_req']].to_numpy()\n",
    "vn_forecast_date = pd.date_range(start=vn_start_date,periods=vn_days+1)\n",
    "\n",
    "#get reproduction rate\n",
    "vn_R0 = vn_df.loc[vn_df.date==vn_start_date,['daily_R0', 'avg_R0']].to_numpy().squeeze().tolist()\n",
    "\n",
    "x_list=[]\n",
    "x_list.extend([vn_real_data[:,0]]*10)\n",
    "x_list.extend([vn_forecast_date]*10)\n",
    "x_list.extend([vn_equil_df.date.to_numpy()]*3)\n",
    "\n",
    "y_list = []\n",
    "y_list.extend(vn_real_data[:,1:].T)\n",
    "y_list.extend(vn_forecast_data.T)\n",
    "y_list.extend(vn_equil_df[['S','V','I']].to_numpy().T)\n",
    "\n",
    "#update plot data\n",
    "for i,data in enumerate(zip(x_list,y_list)):\n",
    "    vn_fig.data[i].update(x=data[0],y=data[1])\n",
    "\n",
    "vn_fig.layout.annotations[10].update(text=f'Equilibrium plot (daily_R0:{round(vn_R0[0],2)} - average_R0:{round(vn_R0[1],2)})')\n",
    "clear_output()\n",
    "vn_fig.update_layout(title_text=\"VietNam forecast\",title_x=0.5,title_font_size=50,\n",
    "                   height=2000,\n",
    "                   legend_tracegroupgap=285)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6b7b191e-802b-4c2a-b643-8cfe58939ddf",
   "metadata": {},
   "source": [
    "<font size=4><ins>**b. By each province**</ins></font><a name=\"3.3.2.b\"></a><br>\n",
    "* <ins>General input parameters</ins>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "09b0d4bb-942d-46e8-8519-593d88b7b2ca",
   "metadata": {},
   "outputs": [],
   "source": [
    "#input for each province data processing\n",
    "vn_bp_p = 1.0\n",
    "\n",
    "#input for SVID_modeling\n",
    "vn_bp_start_date = None\n",
    "vn_bp_days = 7\n",
    "vn_bp_model_number=[1,2,3]\n",
    "vn_bp_scoring = 'neg_mean_absolute_percentage_error'\n",
    "vn_bp_greater_better = False\n",
    "vn_bp_best_model = True #if no best model is chosen, the data of the first model will be chosen for plotting\n",
    "vn_bp_multioutput = \"variance_weighted\"\n",
    "vn_bp_param_kwargs = None\n",
    "vn_bp_plot_point = 30\n",
    "\n",
    "#conversion to real data\n",
    "vn_bp_r_protect_time = 150\n",
    "vn_bp_n_days = 1000 \n",
    "\n",
    "# get general ARIMA order, seasonal order from world total data using for each country\n",
    "vn_bp_preset_order = get_arima_order(input_df=vn_df,\n",
    "                                   cols=['beta','beta_v','gamma','theta','alpha','alpha0','pi'],\n",
    "                                   start=vn_df.loc[vn_df.date==vn_start_date].index[0])\n",
    "\n",
    "vn_bp_max_worker = 6"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ebd8671d-ffdb-4458-899b-9f84b9f1fe64",
   "metadata": {},
   "source": [
    "* <ins>Data processing and forecast SVID model future values</ins>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4c54d581-d6bf-4ec6-8824-7fb533cabd1d",
   "metadata": {},
   "outputs": [],
   "source": [
    "start_cell = time.perf_counter()\n",
    "\n",
    "vn_bp_manager = Manager()\n",
    "vn_bp_counter= vn_bp_manager.Array('i',[0,0])\n",
    "vn_bp_time_list = vn_bp_manager.list([])\n",
    "vn_bp_error_dict = vn_bp_manager.dict()\n",
    "vn_bp_shared_output = vn_bp_manager.Namespace()\n",
    "vn_bp_shared_output.shared_df = pd.DataFrame()\n",
    "vn_bp_shared_output.equil_df = pd.DataFrame()\n",
    "vn_bp_lock = vn_bp_manager.Lock()\n",
    "\n",
    "#list of province code\n",
    "vn_bp_code = vn_bp_df.id.unique()\n",
    "\n",
    "\n",
    "#extra argument for function input\n",
    "vn_bp_input_kwargs = {'predict_days':vn_bp_days,\n",
    "                   'date_col':'date',\n",
    "                   'initial_state':['S0','V0','I0','D0'],\n",
    "                   'model_param':['beta','beta_v','gamma','theta','alpha','alpha0','pi'],\n",
    "                   'model_number':vn_bp_model_number,\n",
    "                   'best_model':vn_bp_best_model,\n",
    "                   'scoring':vn_bp_scoring,\n",
    "                   'multioutput':vn_bp_multioutput,\n",
    "                   'greater_better':vn_bp_greater_better,\n",
    "                   'preset_arima_order': vn_bp_preset_order, #for fast modeling, if set to None, it will be much slower \n",
    "                   'param_kwargs':vn_bp_param_kwargs\n",
    "                  }\n",
    "\n",
    "#worker function\n",
    "worker_func = partial(wrapper_SVID_predict,\n",
    "                      shared_count=vn_bp_counter,\n",
    "                      time_list = vn_bp_time_list,\n",
    "                      error_dict=vn_bp_error_dict,\n",
    "                      shared_output=vn_bp_shared_output,\n",
    "                      lock=vn_bp_lock,\n",
    "                      input_data=(vn_bp_df,'id'),\n",
    "                      start_date=vn_bp_start_date,\n",
    "                      birth_data=vn_birth_rate,\n",
    "                      death_data=vn_death_rate,\n",
    "                      p=vn_bp_p,\n",
    "                      r_protect_time=vn_bp_r_protect_time,\n",
    "                      n_days = vn_bp_n_days,\n",
    "                      **vn_bp_input_kwargs)\n",
    "\n",
    "executor= Pool(processes=vn_bp_max_worker)\n",
    "start_worker = time.perf_counter()\n",
    "pool_result = executor.map_async(worker_func,vn_bp_code)\n",
    "\n",
    "max_province = len(vn_bp_code)\n",
    "while vn_bp_counter[0]<max_province:\n",
    "    time.sleep(1)\n",
    "    n_done = vn_bp_counter[0]\n",
    "    n_wait = max_province\n",
    "    total_time = round(sum(vn_bp_time_list),2)\n",
    "    if n_done!=0:\n",
    "        avg_time = round(total_time/n_done,2)\n",
    "    if n_done!=0:\n",
    "        remain_province = n_wait - n_done\n",
    "        remain_time = round(remain_province*avg_time,2)\n",
    "        real_remain = round(remain_time/vn_bp_max_worker,2)\n",
    "    else:\n",
    "        remain_province = \"estimating...\"\n",
    "        remain_time = \"-- \"\n",
    "        real_remain = \"-- \"\n",
    "        \n",
    "    elapsed_time = round(time.perf_counter()-start_worker,2)\n",
    "    a_string = (f\"Elapsed time: {elapsed_time}s    \" +\n",
    "                f\"Total workers processing time: {total_time}s    \" + \n",
    "                f\" Remaining provinces: {remain_province}    \" + \n",
    "                f\" Estimated remaining worker time: {remain_time}s    \" +\n",
    "                f\" Estimated remaining real time: {real_remain}s\")\n",
    "    \n",
    "    print(\" \"*(len(a_string)+10),end=\"\\r\")\n",
    "    print(a_string,end=\"\\r\")\n",
    "\n",
    "print(\" \"*(len(a_string)+10),end=\"\\r\")\n",
    "print(\"---Forecast processing completed---\")\n",
    "print(f\"Number of workers with error happened: {vn_bp_counter[1]}\")\n",
    "print(f\"Total worker processing time taken: {round(sum(vn_bp_time_list),2)}s\")\n",
    "\n",
    "vn_bp_predict_df = vn_bp_shared_output.shared_df.copy()\n",
    "vn_bp_predict_df = vn_bp_predict_df.sort_values(['id','method','date']).reset_index(drop=True)\n",
    "vn_bp_equil_df = vn_bp_shared_output.equil_df.copy()\n",
    "vn_bp_equil_df = vn_bp_equil_df.sort_values(['id','date']).reset_index(drop=True)\n",
    "vn_bp_error_result = dict(vn_bp_error_dict)\n",
    "vn_bp_time = list(vn_bp_time_list)\n",
    "\n",
    "executor.close()\n",
    "executor.join()\n",
    "vn_bp_manager.shutdown()\n",
    "\n",
    "end_cell = time.perf_counter()\n",
    "print(f\"Total elapsed time on this code block: {round(end_cell-start_cell,2)}s\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a5527c58-df6e-4f0b-a9a9-4832f8bcbd25",
   "metadata": {},
   "source": [
    "* <ins>Store modeling data</ins>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6636ea60-9d35-4c2f-8270-03e022f36d08",
   "metadata": {},
   "outputs": [],
   "source": [
    "folder_path = \"database/Covid-19_modeling_output/VietNamData\"\n",
    "if os.path.exists(str(folder_path)):\n",
    "    pass\n",
    "else:\n",
    "    os.makedirs(str(folder_path))\n",
    "    print(\"New folder created.\")\n",
    "vn_bp_predict_df.to_csv(folder_path +\"/\"+\"vn_prediction_by_province.csv\",encoding=\"utf-8\",index=False)\n",
    "vn_bp_equil_df.to_csv(folder_path +\"/\"+\"vn_equilibrium_by_province.csv\",encoding=\"utf-8\",index=False)\n",
    "vn_bp_predict_df.head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d3b3fe5e-6646-4915-bd80-3b8fc11e037c",
   "metadata": {},
   "source": [
    "* <ins>Plot forecast data</ins>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1ab04426-2fc0-4bd9-b2e2-f8192866bdb6",
   "metadata": {},
   "outputs": [],
   "source": [
    "##making frame for the subplots\n",
    "\n",
    "vn_col_loc1 = [vn_bp_df.columns.get_loc('date'),\n",
    "              vn_bp_df.columns.get_loc('S0'),\n",
    "              vn_bp_df.columns.get_loc('V0'),\n",
    "              vn_bp_df.columns.get_loc('I0'),\n",
    "              vn_bp_df.columns.get_loc('D0'),\n",
    "              vn_bp_df.columns.get_loc('daily_case_non_vaccinated'),\n",
    "              vn_bp_df.columns.get_loc('daily_case_vaccinated'),\n",
    "              vn_bp_df.columns.get_loc('death_avg'),\n",
    "              vn_bp_df.columns.get_loc('recovery_case'),\n",
    "              vn_bp_df.columns.get_loc('new_full_vaccinated'),\n",
    "              vn_bp_df.columns.get_loc('new_boost_req')]\n",
    "vn_col_loc2 = [vn_bp_predict_df.columns.get_loc('date'),\n",
    "              vn_bp_predict_df.columns.get_loc('S0'),\n",
    "              vn_bp_predict_df.columns.get_loc('V0'),\n",
    "              vn_bp_predict_df.columns.get_loc('I0'),\n",
    "              vn_bp_predict_df.columns.get_loc('D0'),\n",
    "              vn_bp_predict_df.columns.get_loc('daily_case_non_vaccinated'),\n",
    "              vn_bp_predict_df.columns.get_loc('daily_case_vaccinated'),\n",
    "              vn_bp_predict_df.columns.get_loc('death_avg'),\n",
    "              vn_bp_predict_df.columns.get_loc('recovery_case'),\n",
    "              vn_bp_predict_df.columns.get_loc('new_full_vaccinated'),\n",
    "              vn_bp_predict_df.columns.get_loc('new_boost_req')]\n",
    "\n",
    "vn_bp_specs = [[{},{}],\n",
    "              [{},{}],\n",
    "              [{},{}],\n",
    "              [{},{}],\n",
    "              [{},{}],\n",
    "              [{\"colspan\": 2},None]]\n",
    "sub_title = [\"Forecasted S\",\n",
    "             \"Forecasted V\",\n",
    "             \"Forecasted I\",\n",
    "             \"Forecasted D\",\n",
    "             \"Forecasted daily_case_non_vaccinated\",\n",
    "             \"Forecasted daily_case_vaccinated\",\n",
    "             \"Forecasted death_avg\",\n",
    "             \"Forecasted recovery_case\",\n",
    "             \"Forecasted new_full_vaccinated\",\n",
    "             \"Forecasted new_boost_req\",\n",
    "             \"Equilibrium plot\"]\n",
    "vn_bp_fig = make_subplots(rows=6,cols=2,specs=vn_bp_specs,subplot_titles=sub_title,vertical_spacing=0.05,horizontal_spacing=0.05)\n",
    "vn_bp_fig['layout']['annotations'] += (dict(text=\"Province:\", x=-0.02, xref=\"paper\",xanchor='left', y=1.042, yref=\"paper\",yanchor=\"top\",\n",
    "                                                                                     align=\"left\", showarrow=False),)\n",
    "\n",
    "for plot_color in ['blue','red']:\n",
    "    for i in range(0,10):\n",
    "        if i%2==0:\n",
    "            add_col=1\n",
    "            showlegend=False\n",
    "        else:\n",
    "            add_col=2\n",
    "            showlegend=True\n",
    "        add_row = (i//2)+1\n",
    "        if plot_color=='blue':\n",
    "            data_name=\"real\"\n",
    "        else:\n",
    "            data_name=\"forecast\"\n",
    "        vn_bp_fig.add_trace(go.Scatter(name=data_name,line={'color':plot_color},legendgroup=str(i),showlegend=showlegend,mode='lines'),\n",
    "                         row=add_row,col=add_col)\n",
    "vn_bp_fig.add_trace(go.Scatter(name=\"S\",line={'color':'green'},legendgroup='10',mode='lines'),row=6,col=1)\n",
    "vn_bp_fig.add_trace(go.Scatter(name=\"V\",line={'color':'blue'},legendgroup='10',mode='lines'),row=6,col=1)\n",
    "vn_bp_fig.add_trace(go.Scatter(name=\"I\",line={'color':'orange'},legendgroup='10',mode='lines'),row=6,col=1)\n",
    "\n",
    "##create buttons data for dropdown list\n",
    "\n",
    "update_menu=[]\n",
    "province_list = vn_bp_df[['id','provinceName']].drop_duplicates(ignore_index=True)\n",
    "annotation_list = []\n",
    "for note in vn_bp_fig.layout.annotations:\n",
    "    annotation_list.append(go.layout.Annotation(note))\n",
    "    \n",
    "for code in vn_bp_predict_df.id.unique():\n",
    "    if vn_bp_start_date is not None:\n",
    "        vn_bp_start_stamp = pd.Timestamp(vn_bp_start_date)\n",
    "        real_data = (vn_bp_df.loc[vn_bp_df.id==code]\n",
    "                     .loc[(vn_bp_df.date<=vn_bp_start_stamp)&(vn_bp_df.date>=(vn_bp_start_stamp-pd.Timedelta(vn_bp_plot_point,unit=\"D\")))]\n",
    "                     .iloc[-vn_bp_plot_point:,vn_col_loc1].to_numpy())\n",
    "    else:\n",
    "        vn_bp_start_stamp = pd.Timestamp(vn_bp_df.loc[vn_bp_df.id==code].date.values[-1])\n",
    "        real_data = vn_bp_df.loc[vn_bp_df.id==code].iloc[-vn_bp_plot_point:,vn_col_loc1].to_numpy()\n",
    "    \n",
    "    #forecast data\n",
    "    if vn_bp_best_model:\n",
    "        forecast_data = vn_bp_predict_df.loc[vn_bp_predict_df.id==code].iloc[:,vn_col_loc2].to_numpy()\n",
    "    else: #if no best model is chosen, the data of the first model will be chosen\n",
    "        forecast_data = vn_bp_predict_df.loc[(vn_bp_predict_df.id==code)&(vn_bp_predict_df.method==vn_bp_model_number[0])].iloc[:,vn_col_loc2].to_numpy()\n",
    "    \n",
    "    #get reproduction rate\n",
    "    vn_bp_R0 = vn_bp_df.loc[vn_bp_df.id==code].loc[vn_bp_df.date==vn_bp_start_stamp,['daily_R0', 'avg_R0']].to_numpy().squeeze().tolist()\n",
    "        \n",
    "    x_list=[]\n",
    "    x_list.extend([real_data[:,0]]*10)\n",
    "    x_list.extend([forecast_data[:,0]]*10)\n",
    "    x_list.extend([vn_bp_equil_df.loc[vn_bp_equil_df.id==code,'date'].to_numpy()]*3)\n",
    "    \n",
    "    y_list = []\n",
    "    y_list.extend(real_data[:,1:].T)\n",
    "    y_list.extend(forecast_data[:,1:].T)\n",
    "    y_list.extend(vn_bp_equil_df.loc[vn_bp_equil_df.id==code,['S','V','I']].to_numpy().T)\n",
    "    \n",
    "    #The main title of each button\n",
    "    plot_title=province_list.loc[province_list.id==code,'provinceName'].values[0]\n",
    "    \n",
    "    button_annotation = annotation_list.copy()\n",
    "    button_annotation[10] = go.layout.Annotation(vn_bp_fig.layout.annotations[10]).update(text=f'Equilibrium plot (daily_R0:{round(vn_bp_R0[0],2)} - average_R0:{round(vn_bp_R0[1],2)})')\n",
    "    button =  dict(label=plot_title,\n",
    "                   method=\"update\",\n",
    "                   args=[{\"y\":y_list,\n",
    "                          \"x\":x_list},\n",
    "                         {\"title\":{\"text\":plot_title,\"x\":0.5,\"font\":{\"size\":50}},\n",
    "                          \"annotations\":button_annotation}\n",
    "                   ]\n",
    "                  )\n",
    "    update_menu.append(button)\n",
    "    \n",
    "##plotting\n",
    "\n",
    "vn_bp_fig.update_layout(updatemenus=[dict(active=-1,\n",
    "                                      buttons=update_menu,\n",
    "                                      pad={\"r\": 10, \"t\": 10},\n",
    "                                      showactive=True,\n",
    "                                      xanchor=\"left\",\n",
    "                                      x=0.05,\n",
    "                                      yanchor=\"top\",\n",
    "                                      y=1.05)\n",
    "                                  ],\n",
    "                     legend_tracegroupgap=285,\n",
    "                     height=2000\n",
    "                    )\n",
    "clear_output()\n",
    "vn_bp_fig.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c10d48d5-5255-4a33-9486-fed197fbe894",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}