WorldClim - Projected Climatologies MAPS_21.03.11.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "ready-identification",
   "metadata": {},
   "source": [
    "# WorldClim data : Historical & future Climatologies (1970-2100) - MAPS\n",
    "## 4 SSP scenarios and 9 Global Climatic Models (Precipitation, Temperature)\n",
    "- Data: https://www.worldclim.org/data/index.html\n",
    "- author: Loïc Duffar https://github.com/loicduffar\n",
    "\n",
    "This python notebook uses data from the wordclim web site to plot future climatologies maps on a given area. The outputs are stored as image files (png) and raster files (geotif).\n",
    "\n",
    "### How to PROCEED ?\n",
    "The data files must be divided into different sub-folders:\n",
    "- a folder containing the 12 tif files with historical monthly interannual averages (1 file per month of the year)\n",
    "- a folder with sub-folders for the various future periods (e.g. 1921-1940, 1961-1981, 1981-2100). Each subfolder contains 4 tif files for each model (and each file contains 12 bands for the the different months).\n",
    "- Optionnaly, the global DEM file can be downloaded from WorldClim to plot the elevation map of the area\n",
    "\n",
    "Run the first script, then one of the following ones:\n",
    "- <a href=#section01>1)</a> Customization\n",
    "- <a href=#section02>2)</a> Plot annual map for a unique scenario/period/model (annual and monthly maps)\n",
    "- <a href=#section03>3)</a> Generate a bunch of maps of all scenarios/periods/models (annual maps only)\n",
    "- <a href=#section04>4)</a> Plot the elevation map\n",
    "- <a href=#section05>5)</a> "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "twelve-solution",
   "metadata": {},
   "source": [
    "###  <a name=section01></a>1) Customization\n",
    "\n",
    "NB: The cell below does not produce any result, but its execution is necessary before to run any one of the next cell\n",
    "\n",
    "Customize the code below before to run it:\n",
    "- Choose between precipitation or temperature, and between anomaly or not (deviation from historical baseline)\n",
    "- Set the geographical area to be drawn\n",
    "- Choose the folders for input data and output files\n",
    "- Set the file names composition:\n",
    "  - prefix of the file names\n",
    "  - List of models, scenarios and time periods\n",
    "- Set the legend for historical period\n",
    "\n",
    "- Contour interval for annual map\n",
    "- vmin, vmax : values to anchor the colormap in order to compare the different maps. If set to (None, None), they are automaticaly computed from the extreme values of each map, in which case the colormaps of the differente maps are not comparable\n",
    "    - vmin_annual, vmax_annual: minimum and maximum values of the annual data \n",
    "    - vmin_monthly, vmax_monthly: minimum and maximum values of the monthly data\n",
    "\n",
    "- Set the vector files to be drawn for map wrapping (optional): e.g. line/point (river, borders, localities) and/or polygon (watershed or project area). The Coordinate Reference System must be WGS84 (4326)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "utility-affiliate",
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "Created on March 2021\n",
    "@author: Loïc Duffar\n",
    "\"\"\"\n",
    "import numpy as np\n",
    "import geopandas as gpd\n",
    "import os\n",
    "import cartopy\n",
    "\n",
    "######### Customization\n",
    "# ----------- data -----------\n",
    "variable= 'Temperature'# Precipitation or Temperature\n",
    "plot_anomaly= True      # set it to True to plot anomaly (deviation from baseline) instead of raw data\n",
    "# ----------- Spatial cropping\n",
    "# minx, miny, maxx, maxy=[-85.91, 7.81, -82.61, 11.31]# COSTA RICA\n",
    "# minx, miny, maxx, maxy= [-72.1, 17.3, -68.2, 20.2] # République dominicaine\n",
    "minx, miny, maxx, maxy= [-74.6, 17.3, -68.2, 20.2] # HISPANIOLA\n",
    "# ----------- Files -----------\n",
    "folder_out= r'C:\\Users\\DUFFAR\\Documents\\A\\ETUDES\\2021 Yaque del Sur'\n",
    "# ........... folder path for the HISTORICAL tif files (WITHOUT \"\\\" CHARACTER AT THE END !!!)\n",
    "fld_historical= r'X:\\1-COMMUN\\DIS\\Documentation\\Hydrologie\\Documentation externe\\Climat Monde\\WorldClim\\Historical Climatologies 1970-2000\\wc2.1_5m'\n",
    "# ........... folder path for the FUTURE tif files (WITHOUT \"\\\" CHARACTER AT THE END !!!)\n",
    "fld_future= r'X:\\1-COMMUN\\DIS\\Documentation\\Hydrologie\\Documentation externe\\Climat Monde\\WorldClim\\Future data 2021-2100'\n",
    "# ........... Composition of the tif file names\n",
    "file_prefix= 'wc2.1_5m_'# e.g. 'wc2.1_5m_'\n",
    "model_list = ['BCC-CSM2-MR', 'CNRM-CM6-1', 'CNRM-ESM2-1', 'CanESM5', 'GFDL-ESM4', 'IPSL-CM6A-LR', 'MIROC-ES2L', 'MIROC6', 'MRI-ESM2-0']\n",
    "scenario_list = ['ssp126', 'ssp245', 'ssp370', 'ssp585']\n",
    "scenario_list_long = ['ssp1-2.6', 'ssp2-4.5', 'ssp3-7.0', 'ssp5-8.5']\n",
    "period_list = ['2021-2040', '2041-2060', '2061-2080', '2081-2100']\n",
    "historical_period= '1970-2000'\n",
    "# ------- MAP -----------\n",
    "# ........... Min and max values to anchor the MONTHLY colormap, or (None, None) for automatic determination from the data, \n",
    "#             in which case the colormaps of the differente maps are not comparable\n",
    "#    TIPS:\n",
    "#    - Precipitation monthly: (0, 300).  if (None, None), the values are computed from 2nd and 98th percentiles of the data instead of the extreme values\n",
    "#    - Temperature monthly: (9, 33)\n",
    "vmin_monthly, vmax_monthly = (9, 33)\n",
    "#........ Contour interval for annual map\n",
    "#    TIPS:\n",
    "#    - Precipitation Yearly: 300 and 50 or 100 for anomaly\n",
    "#    - Temperature Yearly: 2 and 0.1 for anomaly\n",
    "contour_int_annual= 0.1\n",
    "#........ Min and max values to anchor the ANNUAL colormap, or (None, None) for automatic determination from the data,\n",
    "#             in which case the colormaps of the differente maps are not comparable\n",
    "#    TIPS: e.g. (200, 2600) for Precipitation, (11, 31) / (0.6, 3.2) for Temperature / anomaly\n",
    "#    - Min value should not be greater than the minimum of the data\n",
    "#    - Max value should be fixed according to contour interval and vmin (even much greater than the maximum of the data)\n",
    "#    - For Precipitation anomalies, abosolute values of vmin and vmax should be equal (e.g. -800, +800), because the color map is automaticaly set to divergent on either side of zero\n",
    "vmin_annual, vmax_annual = (0.6, 3.2)\n",
    "# ........ Vector files (optionnal but very usefull)\n",
    "#          - Polygon file (shape file) for plotting and average computation (path INCLUDING folder)\n",
    "polygon_file = r'X:\\4-PROJ_INTERNATIONAL\\2020_11_25-DO-FAI-GIRE YAQUE DEL SUR\\CARTOPLAN\\Hidrografia_REPDOM\\Subcuencas_YaqueSur.shp'\n",
    "#          - Shape file (point, line or polygon) for plotting (path INCLUDING FOLDER)\n",
    "wrapping_file = r'X:\\4-PROJ_INTERNATIONAL\\2020_11_25-DO-FAI-GIRE YAQUE DEL SUR\\CARTOPLAN\\Hidrografia_REPDOM\\República Dominicana.shp'\n",
    "\n",
    "######### Initializations\n",
    "#...... Files\n",
    "file_prefix = file_prefix + '_' if file_prefix.strip()[-1:]!= '_' else file_prefix.strip()\n",
    "if variable.upper()== 'PRECIPITATION':\n",
    "    file_prefix = file_prefix + 'prec'\n",
    "    unit= 'mm'\n",
    "elif variable.upper()== 'TEMPERATURE':\n",
    "    file_prefix = file_prefix + 't'\n",
    "    unit= '°C'\n",
    "#...... Map\n",
    "proj = cartopy.crs.PlateCarree()\n",
    "nlevels_annual=  int((vmax_annual - vmin_annual) / contour_int_annual) + 1\n",
    "levels= np.linspace(vmin_annual, vmax_annual, nlevels_annual)\n",
    "gdf_polyg = gpd.read_file(polygon_file) if os.path.exists(polygon_file) else None\n",
    "gdf_wrapping = gpd.read_file(wrapping_file) if os.path.exists(wrapping_file) else None\n",
    "res = '10m'                  # resolution for naturalearth basemap with cartopy: 10m, 50m or 110m (10m = 1/ 10 000 000)\n",
    "\n",
    "if variable.upper()== 'PRECIPITATION' and plot_anomaly== True:\n",
    "    cmap = 'seismic_r'\n",
    "else:\n",
    "    cmap= 'jet'    # https://matplotlib.org/stable/tutorials/colors/colormaps.html\n",
    "if variable.upper()== 'PRECIPITATION':\n",
    "    clabel_fmt= '%d'\n",
    "else:\n",
    "    clabel_fmt= '%1.1f'\n",
    "\n",
    "print('Cartopy version: ', cartopy.__version__)\n",
    "\n",
    "######### Function definitions ##################################################################\n",
    "#======= annual map (cartopy library) ============\n",
    "def plot_annual(da_annual, title='WorldClim - Annual mean map', levels=10, vmin= None, vmax = None, cmap= 'jet', clabel_fmt= '%1.1f',\n",
    "                shapes= dict(polyg= dict(shape= None, edgecolor='pink', facecolor= 'grey', alpha= 0.4), \n",
    "                             vector= dict(shape= None, edgecolor='grey', facecolor='none', color='blue', linewidth=0.5, alpha=1)), \n",
    "                folder_out= '', file_out= 'WorldClim - Annual mean map.png',\n",
    "               ):\n",
    "    fig = plt.figure(figsize=(18, 14))\n",
    "    ax = fig.add_subplot(projection= proj)\n",
    "    #ax.set_extent([lon_min-x_ext, lon_max+x_ext, lat_min-y_ext, lat_max+y_ext], crs=proj)\n",
    "\n",
    "    #. . . . . . . . plot contours\n",
    "    cs1 = xr.plot.contourf(da_annual, 'lon', 'lat', levels=levels, vmin= vmin, vmax = vmax,\n",
    "                               cmap= cmap, alpha =1, cbar_kwargs= dict(shrink=0.6, label= unit, pad=0.2))# contour fills\n",
    "    cs2 = xr.plot.contour(da_annual, 'lon', 'lat', levels=levels, vmin= vmin, vmax = vmax,\n",
    "                          linewidths=1)# contour allows to plot contour lines (contourf does not)\n",
    "    plt.clabel(cs2, fontsize=10, inline=True, fmt= clabel_fmt)\n",
    "\n",
    "    #. . . . . . . . Map customization (Polygon and shape file defined by user)\n",
    "    if shapes['polyg']['shape'] is not None:\n",
    "        shapes['polyg']['shape'].boundary.plot(ax=ax, edgecolor= shapes['polyg']['edgecolor'], facecolor= shapes['polyg']['facecolor'], alpha= shapes['polyg']['alpha'])\n",
    "    if shapes['vector']['shape'] is not None: \n",
    "        shapes['vector']['shape'].plot(ax=ax, edgecolor= shapes['vector']['edgecolor'], facecolor= shapes['vector']['facecolor'], color= shapes['vector']['color'], linewidth= shapes['vector']['linewidth'], alpha= shapes['vector']['alpha']) \n",
    "\n",
    "    #. . . . . . . . Map wrapping (NaturalEarth)\n",
    "    ax.add_feature(cfeature.NaturalEarthFeature('physical', 'land', res, facecolor='antiquewhite', edgecolor='black'),\n",
    "                    alpha=0.3)\n",
    "    ax.add_feature(cfeature.NaturalEarthFeature('physical', 'rivers_lake_centerlines', res, facecolor='none', \n",
    "                    edgecolor='blue'), alpha=0.5)\n",
    "    ax.add_feature(cfeature.NaturalEarthFeature('physical', 'rivers_europe', res, facecolor='none', edgecolor='blue'))\n",
    "    ax.add_feature(cfeature.NaturalEarthFeature('physical', 'lakes', res, facecolor=cfeature.COLORS['water'], edgecolor='blue'), \n",
    "                    alpha=0.5)\n",
    "    ax.add_feature(cfeature.NaturalEarthFeature('physical', 'ocean', res, facecolor= 'aliceblue', edgecolor='black'), \n",
    "                    alpha=0.5)\n",
    "    plt.title(title, loc='center')\n",
    "    plt.savefig(os.path.join(folder_out, file_out), dpi=150)\n",
    "    return fig\n",
    "\n",
    "#======= Maps by month (Facet plot by cartopy library) ============\n",
    "def plot_by_month(da, title= 'worldclim - Interannual Mean maps by month', \n",
    "                  cmap= 'jet', vmin= None, vmax= None,\n",
    "                  folder_out= '', file_out= 'worldclim - Interannual Mean maps by month.png'):\n",
    "    grid= xr.plot.imshow(da, x='lon', y='lat', aspect=2, size=2, vmin= vmin, vmax= vmax,# figsize= (15, 18),\n",
    "    #da.plot(\n",
    "                         col= 'month', col_wrap= 3, cmap= cmap, robust= True,\n",
    "                         cbar_kwargs= dict(label= unit, aspect= 30),\n",
    "                         transform= proj,\n",
    "                         subplot_kws= dict(projection= proj)# clim(da.min(), da.max())\n",
    "                  )\n",
    "    ax= plt.gca()\n",
    "    #plt.tight_layout()\n",
    "    # plt.subplots_adjust(wspace=0., hspace=0.2)\n",
    "    # for im in plt.gca().get_images():\n",
    "    #     im.set_clim(da.min(), da.max())\n",
    "\n",
    "    for ax in grid.axes.flat:\n",
    "    #     ax.add_feature(cfeature.NaturalEarthFeature('physical', 'coastlines', res, facecolor='none', edgecolor='black'))\n",
    "        ax.add_feature(cfeature.NaturalEarthFeature('physical', 'ocean', res, facecolor= 'aliceblue', edgecolor='black'), alpha=1)\n",
    "    plt.gcf().suptitle(title, y=1)\n",
    "    plt.savefig(os.path.join(folder_out, file_out), dpi=150)\n",
    "    fig= plt.gcf()\n",
    "    return fig\n",
    "######### End of functions definition ##################################################################"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "olive-parallel",
   "metadata": {},
   "source": [
    "###  <a name=section02></a>2) Maps for a unique scenario/model/period (annual AND MONTHLY)\n",
    "This cell allow to plot a unique scenario/model/period, but also provides the monthly maps (wich are not plotted by the next cell)\n",
    "\n",
    "NB: The cell # 1 of this notebook must be ran before the cell below\n",
    "\n",
    "Customize the code below before to run it:\n",
    "- Choose the model, scenario and period, among the lists set in the cell above"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "complicated-helicopter",
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "Created on March 2021\n",
    "@author: Loïc Duffar\n",
    "\"\"\"\n",
    "import pandas as pd\n",
    "import hvplot.pandas\n",
    "import hvplot.xarray\n",
    "import xarray as xr\n",
    "import rioxarray\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import cartopy.feature as cfeature\n",
    "print('Cartopy version: ', cartopy.__version__)\n",
    "\n",
    "######### Customization \n",
    "# ........... Choose the model, scenario and period\n",
    "model= model_list[0]\n",
    "scenario= scenario_list[3]\n",
    "period= period_list[3]\n",
    "\n",
    "######### Historical Data (mean): Reading and Preprocessing\n",
    "for i, month in enumerate(range(1, 13)):\n",
    "    if variable.upper()== 'PRECIPITATION':\n",
    "        file = os.path.join(fld_historical, file_prefix + '_' + \"{:0>2d}\".format(month) + '.tif')\n",
    "    elif variable.upper()== 'TEMPERATURE':\n",
    "        file = os.path.join(fld_historical, file_prefix + 'avg_' + \"{:0>2d}\".format(month) + '.tif')\n",
    "    if i==0:\n",
    "        da= rioxarray.open_rasterio(os.path.join(fld_historical, file), masked= True)\n",
    "        da= da.rio.clip_box(minx, miny, maxx, maxy)\n",
    "    else:\n",
    "        new_da= rioxarray.open_rasterio(os.path.join(fld_historical, file), masked= True)\n",
    "        new_da= new_da.rio.clip_box(minx, miny, maxx, maxy)\n",
    "        liste= list(da['band'].values)\n",
    "        liste.append(month)\n",
    "        da= xr.concat([da, new_da], dim= pd.Index( liste, name= 'band'))\n",
    "# +++++++ Rename x/y coordinates and set spatial dimensions to the new names\n",
    "da_historical= da.rename(dict(band= 'month', x= 'lon', y= 'lat'))\n",
    "da_historical.rio.set_spatial_dims(x_dim= 'lon', y_dim= 'lat', inplace= True)\n",
    "# +++++++ Annual aggregation and Save climatology maps into a geotif file\n",
    "if variable.upper()== 'PRECIPITATION':\n",
    "    da_historical_annual = da_historical.sum('month', skipna= False)\n",
    "elif variable.upper()== 'TEMPERATURE':\n",
    "    da_historical_annual= da_historical.mean('month', skipna= False)\n",
    "da_historical_annual.rio.set_spatial_dims(x_dim= 'lon', y_dim= 'lat', inplace= True)\n",
    "\n",
    "# +++++++ Save maps to a geotif file\n",
    "file = 'WorldClim - Monthly ' + variable + ' Climatology Map - Historical.tif'\n",
    "da_historical.rio.to_raster(os.path.join(folder_out, file))\n",
    "file = 'WorldClim - Annual ' + variable + ' Climatology Map - Historical.tif'\n",
    "da_historical_annual.rio.to_raster(os.path.join(folder_out, file))\n",
    "\n",
    "#########  Future Data: Reading and Preprocessing\n",
    "# +++++++ Read file(s)\n",
    "path = fld_future + '/' + period\n",
    "if variable.upper()== 'PRECIPITATION':\n",
    "    file = os.path.join(path, '_'.join([file_prefix, model, scenario, period]) + '.tif')\n",
    "    da= rioxarray.open_rasterio(os.path.join(fld_future, file), masked= True)\n",
    "elif variable.upper()== 'TEMPERATURE':\n",
    "    file = os.path.join(path, '_'.join([file_prefix + 'min', model, scenario, period]) + '.tif')\n",
    "    file2 = os.path.join(path, '_'.join([file_prefix + 'max', model, scenario, period]) + '.tif')\n",
    "    da= rioxarray.open_rasterio(os.path.join(fld_future, file), masked= True)\n",
    "    da2= rioxarray.open_rasterio(os.path.join(fld_future, file2), masked= True)\n",
    "# +++++++ Crop raster\n",
    "da= da.rio.clip_box(minx, miny, maxx, maxy)\n",
    "# +++++++ Average min/max temperatures\n",
    "if variable.upper()== 'TEMPERATURE':\n",
    "    da2= da2.rio.clip_box(minx, miny, maxx, maxy)\n",
    "    da= (da + da2) / 2\n",
    "# +++++++ Rename x/y coordinates and set spatial dimensions to the new names\n",
    "da= da.rename(dict(band= 'month', x='lon', y= 'lat'))\n",
    "da.rio.set_spatial_dims(x_dim= 'lon', y_dim= 'lat', inplace= True)\n",
    "# +++++++ Annual aggregation\n",
    "if variable.upper()== 'PRECIPITATION':\n",
    "    da_future_annual = da.sum('month', skipna= False)\n",
    "elif variable.upper()== 'TEMPERATURE':\n",
    "    da_future_annual = da.mean('month', skipna= False)\n",
    "#display(da_annual)\n",
    "\n",
    "if plot_anomaly==True:\n",
    "    da_to_plot= da_future_annual-da_historical_annual\n",
    "    base= variable + ' Anomaly '\n",
    "else:\n",
    "    da_to_plot= da_future_annual\n",
    "    base= variable + ' '\n",
    "#########  Historical Data: Plot annual map (cartopy)\n",
    "title_annual= 'WorldClim - Annual ' + base\n",
    "plt.close()\n",
    "if plot_anomaly== False:\n",
    "    file_out= title_annual + 'Climatology Map - Historical.png'\n",
    "    title= title_annual + 'Climatology (' + historical_period + ')'\n",
    "    fig= plot_annual(da_historical_annual, title= title,  levels=levels, cmap= cmap, vmin= vmin_annual, vmax = vmax_annual,\n",
    "                    shapes= dict(polyg= dict(shape= gdf_polyg, edgecolor='magenta', facecolor= 'none', alpha= 1), \n",
    "                                 vector= dict(shape= gdf_wrapping, edgecolor='grey', facecolor='none', color='none', linewidth=0.5, alpha=1)),\n",
    "                     folder_out= folder_out, file_out= file_out, clabel_fmt= clabel_fmt )\n",
    "    plt.show()\n",
    "\n",
    "#########  Future Data: Plot annual map (cartopy)\n",
    "plt.close()\n",
    "\n",
    "file_out=  title_annual +  'Climatology Map - 1 scenario_model_period.png'\n",
    "title= title_annual + '- ' + '_'.join([period, scenario, model])\n",
    "fig= plot_annual(da_to_plot, title= title, levels=levels, cmap= cmap, vmin= vmin_annual, vmax = vmax_annual, \n",
    "                shapes= dict(polyg= dict(shape= gdf_polyg, edgecolor='magenta', facecolor= 'none', alpha= 1), \n",
    "                             vector= dict(shape= gdf_wrapping, edgecolor='grey', facecolor='none', color='none', linewidth=0.5, alpha=1)),\n",
    "                 folder_out= folder_out, file_out= file_out, clabel_fmt= clabel_fmt)\n",
    "plt.show()\n",
    "\n",
    "#########  Historical Data: Monthly Plot (cartopy facet plot)\n",
    "title_monthly= 'WorldClim - Monthly ' + base\n",
    "plt.close()\n",
    "file_out= title_monthly + 'Climatology Map - Historical.png'\n",
    "title= title_monthly + 'Climatology (' + historical_period + ')'\n",
    "fig= plot_by_month(da_historical, title= title, cmap= cmap, vmin=vmin_monthly, vmax=vmax_monthly, \n",
    "                   folder_out= folder_out, file_out= file_out)\n",
    "plt.show()\n",
    "\n",
    "#########  Future Data: Montly Plot (cartopy facet plot)\n",
    "plt.close()\n",
    "file_out= title_monthly + 'Climatology Map - Future.png'\n",
    "title= title_monthly + 'Climatology ' + '_'.join([period, scenario, model])\n",
    "fig= plot_by_month(da, title= title, cmap= cmap, vmin=vmin_monthly, vmax=vmax_monthly, \n",
    "                   folder_out= folder_out, file_out= file_out)\n",
    "plt.show()\n",
    "\n",
    "#########  Plot months animation (hvplot)\n",
    "hv2= da.hvplot(x='lon', y='lat', width= 700, geo= True,  crs= proj,\n",
    "                coastline= res, \n",
    "                tiles= 'CartoLight', # CartoLight, StamenTerrain, StamenTerrainRetina, EsriTerrain, EsriNatGeo\n",
    "                cmap= cmap, clabel= unit, alpha=0.6,\n",
    "#                title='WorldClim',\n",
    "                clim=(da.min(), da.max()),\n",
    "                groupby= \"month\",  # Coordinates to group by, for animation plot\n",
    "                widget_type= \"scrubber\",\n",
    "                widget_location= \"bottom\",\n",
    "                 )\n",
    "hv1= da_historical.hvplot(x='lon', y='lat', width= 700, geo= True,  crs= proj,\n",
    "                coastline= res, \n",
    "                tiles= 'CartoLight', # CartoLight, StamenTerrain, StamenTerrainRetina, EsriTerrain, EsriNatGeo\n",
    "                cmap= cmap, clabel= unit, alpha=0.6,\n",
    "#                title='WorldClim',\n",
    "                clim=(da.min(), da.max()),\n",
    "                groupby= \"month\",  # Coordinates to group by, for animation plot\n",
    "                widget_type= \"scrubber\",\n",
    "                widget_location= \"bottom\",\n",
    "                 )\n",
    "hv1 + hv2"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "reasonable-antigua",
   "metadata": {},
   "source": [
    "###  <a name=section03></a>3) Generate a bunch of maps for all scenarios/periods/models (into png files)\n",
    "NB: The cell # 1 of this notebook must be ran before the cell below\n",
    "\n",
    "Run this cell and wait the message \"Process completed\" (one minute or more). \n",
    "\n",
    "Maps files are not displayed but stored in the user defined output folder"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "enclosed-congress",
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "Created on March 2021\n",
    "@author: Loïc Duffar\n",
    "\"\"\"\n",
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\")# prevents multiples warnings caused by non updated cartopy module (v 0.17)\n",
    "\n",
    "import datetime\n",
    "import pandas as pd\n",
    "import hvplot.pandas\n",
    "import hvplot.xarray\n",
    "import xarray as xr\n",
    "import rioxarray\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import cartopy.feature as cfeature\n",
    "\n",
    "######### Initializations\n",
    "start= datetime.datetime.now()# Start Timer\n",
    "print('Process in progress (please wait the ending message)')\n",
    "\n",
    "######### Historical data\n",
    "# +++++++ Read tif files, crop the rasters (12 months) and store them in a single DataArray\n",
    "for i, month in enumerate(range(1, 13)):\n",
    "    if variable.upper()== 'PRECIPITATION':\n",
    "        file = os.path.join(fld_historical, file_prefix + '_' + \"{:0>2d}\".format(month) + '.tif')\n",
    "    elif variable.upper()== 'TEMPERATURE':\n",
    "        file = os.path.join(fld_historical, file_prefix + 'avg_' + \"{:0>2d}\".format(month) + '.tif')\n",
    "    \n",
    "    if i==0:\n",
    "        da= rioxarray.open_rasterio(os.path.join(fld_historical, file), masked= True)\n",
    "        da= da.rio.clip_box(minx, miny, maxx, maxy)\n",
    "    else:\n",
    "        new_da= rioxarray.open_rasterio(os.path.join(fld_historical, file), masked= True)\n",
    "        new_da= new_da.rio.clip_box(minx, miny, maxx, maxy)\n",
    "        liste= list(da['band'].values)\n",
    "        liste.append(month)\n",
    "        da= xr.concat([da, new_da], dim= pd.Index( liste, name= 'band'))\n",
    "# +++++++ Rename coordinates and set spatial dimensions to 'lon' and 'lat'\n",
    "da_historical= da.rename(dict(band= 'month', x= 'lon', y= 'lat'))\n",
    "da_historical.rio.set_spatial_dims(x_dim= 'lon', y_dim= 'lat', inplace= True)\n",
    "# +++++++ Use da_historical to initialize a new DataArray for the future data, with an additional dimension 'period'\n",
    "# da_tot= da_historical.expand_dims( {'period': 1, 'model': 1, 'scenario': 1}, \n",
    "#                                   axis= range(3, 6))\n",
    "# da_tot= da_tot.assign_coords( {'period': ('period', [historical_period]), \n",
    "#                                'scenario': ('scenario', ['-']),\n",
    "#                                'model': ('model', ['-']),\n",
    "#                               })\n",
    "# display(da_tot)\n",
    "# +++++++ Annual aggregation\n",
    "if variable.upper()== 'PRECIPITATION':\n",
    "    da_historical_annual = da_historical.sum('month', skipna= False)\n",
    "elif variable.upper()== 'TEMPERATURE':\n",
    "    da_historical_annual = da_historical.mean('month', skipna= False)\n",
    "da_historical_annual.rio.set_spatial_dims(x_dim= 'lon', y_dim= 'lat', inplace= True)\n",
    "\n",
    "# +++++++ Save maps to a geotif file\n",
    "file = 'WorldClim - Monthly ' + variable + ' Climatology Map - Historical.tif'\n",
    "da_historical.rio.to_raster(os.path.join(folder_out, file))\n",
    "file = 'WorldClim - Annual ' + variable + ' Climatology Map - Historical.tif'\n",
    "da_historical_annual.rio.to_raster(os.path.join(folder_out, file))\n",
    "\n",
    "#########  Historical Data: Plot annual map (cartopy)\n",
    "plt.close()\n",
    "if plot_anomaly== False:\n",
    "    title= 'WordlClim - ' + variable + ' - ' + 'Annual mean (' + historical_period + ')'\n",
    "    file_out= 'WorldClim - Annual mean map - Historical.png'\n",
    "    fig= plot_annual(da_historical_annual, title= title,  levels=levels, cmap= cmap, vmin= vmin_annual, vmax = vmax_annual,\n",
    "                    shapes= dict(polyg= dict(shape= gdf_polyg, edgecolor='magenta', facecolor= 'none', alpha= 1), \n",
    "                                 vector= dict(shape= gdf_wrapping, edgecolor='grey', facecolor='none', color='none', linewidth=0.5, alpha=1)),\n",
    "                     folder_out= folder_out, file_out= file_out, clabel_fmt= clabel_fmt )\n",
    "\n",
    "#########  Historical Data: Plot monthly animation (hvplot)\n",
    "hv= da_historical.hvplot(x='lon', y='lat', width= 700, geo= True,  crs= proj,\n",
    "                coastline= res, \n",
    "                tiles= 'CartoLight', # CartoLight, StamenTerrain, StamenTerrainRetina, EsriTerrain, EsriNatGeo\n",
    "                cmap= 'jet', clabel= unit, alpha=0.6,\n",
    "#                title='WorldClim - Historical mean by month',\n",
    "                clim=(da.min(), da.max()),\n",
    "                groupby= \"month\",  # Coordinate to group by, for animation plot\n",
    "                widget_type= \"scrubber\",\n",
    "                widget_location= \"bottom\",\n",
    "                 )\n",
    "\n",
    "######### Future Data: loop for one map per scenario-period-model\n",
    "plt.ioff()# desactivate interactive plot to prevent displaying maps (maps are saved in png files)\n",
    "first_array= True\n",
    "#period_coordinates= [period]\n",
    "for period in period_list:\n",
    "    print('Period --------------', period)\n",
    "    for i, scenario in enumerate(scenario_list):\n",
    "        for model in model_list:\n",
    "            path = fld_future + '/' + period\n",
    "            if variable.upper()== 'PRECIPITATION':\n",
    "                file = os.path.join(path, '_'.join([file_prefix, model, scenario, period]) + '.tif')\n",
    "            elif variable.upper()== 'TEMPERATURE':\n",
    "                file = os.path.join(path, '_'.join([file_prefix + 'min', model, scenario, period]) + '.tif')\n",
    "                file2 = os.path.join(path, '_'.join([file_prefix + 'max', model, scenario, period]) + '.tif')\n",
    "            ######### Preprocess\n",
    "            # +++++++ Read file(s)\n",
    "            if os.path.exists(os.path.join(fld_future, file)):\n",
    "                da= rioxarray.open_rasterio(os.path.join(fld_future, file), masked= True)\n",
    "                if variable.upper()== 'TEMPERATURE':\n",
    "                    da2= rioxarray.open_rasterio(os.path.join(fld_future, file2), masked= True)\n",
    "            else:\n",
    "                continue\n",
    "                # Skip the plot in case of missing file\n",
    "            # +++++++ Crop raster\n",
    "            da= da.rio.clip_box(minx, miny, maxx, maxy)           \n",
    "            # +++++++ Average min/max temperatures\n",
    "            if variable.upper()== 'TEMPERATURE':\n",
    "                da2= da2.rio.clip_box(minx, miny, maxx, maxy)\n",
    "                da= (da + da2) / 2            \n",
    "            # +++++++ Rename coordinates and set spatial dimensions to 'lon' and 'lat'\n",
    "            da= da.rename(dict(band= 'month', x='lon', y= 'lat'))\n",
    "            da.rio.set_spatial_dims(x_dim= 'lon', y_dim= 'lat', inplace= True)\n",
    "            # +++++++ Stack the raster into an other DataArray with 3 additionnal dimensions (period, model, scenario)\n",
    "#             da= da.expand_dims( {'period': 1, \n",
    "#                                  'scenario': 1,\n",
    "#                                  'model': 1,\n",
    "#                                 }, \n",
    "#                                axis= range(3, 6)\n",
    "#                               )\n",
    "#             da= da.assign_coords( {'period': ('period', [period]), \n",
    "#                                    'scenario': ('scenario', [scenario]),\n",
    "#                                    'model': ('model', [model]),  \n",
    "#                                   })\n",
    "#             display(da)            \n",
    "#             liste= list(da_tot['period'].values)\n",
    "#             liste.extend([period for i in range(12)])\n",
    "#             print(liste)\n",
    "#             da_tot= xr.concat([da_tot, da], dim= pd.Index( liste, name= 'period'))\n",
    "            if first_array== True:\n",
    "                first_array= False\n",
    "                da_tot= da\n",
    "            else:\n",
    "                da_tot= da\n",
    "                #da_tot= xr.combine_nested([[da_tot], [da]], concat_dim= [[historical_period, '-', '-'], [period, model, scenario]])\n",
    "#            period_coordinates.append(period)\n",
    "            # +++++++ Annual aggregation\n",
    "            if variable.upper()== 'PRECIPITATION':\n",
    "                da_annual = da.sum('month', skipna= False)\n",
    "            elif variable.upper()== 'TEMPERATURE':\n",
    "                da_annual = da.mean('month', skipna= False)\n",
    "\n",
    "            #########  Plot annual map (cartopy)\n",
    "            if plot_anomaly==True:\n",
    "                da_to_plot= da_annual-da_historical_annual\n",
    "                title= variable + ' Anomaly '\n",
    "            else:\n",
    "                da_to_plot= da_annual\n",
    "                title= variable + ' '\n",
    "            title= 'WordlClim - Annual ' + title + '_'.join([period, scenario, model])\n",
    "            fig= plot_annual(da_to_plot, title= title, levels=levels, cmap= cmap, vmin= vmin_annual, vmax = vmax_annual,\n",
    "                            shapes= dict(polyg= dict(shape= gdf_polyg, edgecolor='magenta', facecolor= 'none', alpha= 1), \n",
    "                                         vector= dict(shape= gdf_wrapping, edgecolor='grey', facecolor='none', color='none', linewidth=0.5, alpha=1)),\n",
    "                             folder_out= folder_out, file_out= title, clabel_fmt= clabel_fmt)\n",
    "            plt.close()# close plot data, otherwise the map would be displayed after reactivation of interactive plot\n",
    "plt.ion()# reactivate interactive plot\n",
    "#da_tot= xr.concatenate([da_historical, da_tot], dim= period_coordinates)\n",
    "#da_tot= da_tot.assign_coords(period= period_coordinates)\n",
    "# display(da_tot)\n",
    "print(\"Process completed in \", datetime.datetime.now()-start)\n",
    "print('')\n",
    "print('Historical data - Mean by month')\n",
    "hv"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "earned-clearing",
   "metadata": {},
   "source": [
    "###  <a name=section04></a>4) Elevation map\n",
    "The DEM tif file can be downloaded from WorldClim https://www.worldclim.org/data/worldclim21.html\n",
    "\n",
    "NB: The cell # 1 of this notebook must be ran before the cell below\n",
    "\n",
    "Customize the code below before to run it:\n",
    "- folder and name of the elevation raster file\n",
    "- Min/Max elevations to anchor the color map"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "academic-victor",
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "Created on March 2021\n",
    "@author: Loïc Duffar\n",
    "\"\"\"\n",
    "import hvplot.xarray\n",
    "import hvplot.pandas\n",
    "import xarray as xr\n",
    "import rioxarray\n",
    "import numpy as np\n",
    "import matplotlib as mpl\n",
    "import matplotlib.pyplot as plt\n",
    "import cartopy\n",
    "import cartopy.feature as cfeature\n",
    "import hvplot.xarray\n",
    "\n",
    "######### Customization\n",
    "# ----------- DEM global TIF file (FOLDER WITHOUT \"\\\" CHARACTER AT THE END !!!)\n",
    "fld_elv= r'X:\\1-COMMUN\\DIS\\Documentation\\Hydrologie\\Documentation externe\\Climat Monde\\WorldClim\\Elevation'\n",
    "file_elv= 'wc2.1_30s_elev.tif'\n",
    "# -------- Min/Max values to anchor the color map\n",
    "#          - vmin should be negative to avoid low land to be colored in blue\n",
    "#          - vmax should be adjusted according to the relief (lower than max elevation in case of high montains, otherwise greater than or equal to the max)\n",
    "vmin_elv, vmax_elv= (-50, 2800)\n",
    "\n",
    "######### Initializations\n",
    "proj = cartopy.crs.PlateCarree()\n",
    "title= 'Elevation map'\n",
    "file_out= 'WorldClim - Elevation Map.png'\n",
    "cmap_terrain= 'terrain'\n",
    "res= '10m'\n",
    "\n",
    "######### Process\n",
    "# +++++++ Open file\n",
    "da= rioxarray.open_rasterio(os.path.join(fld_elv, file_elv), masked= True)\n",
    "da= da.sel(band= 1)\n",
    "\n",
    "# +++++++ Clip data\n",
    "da= da.rio.clip_box(minx, miny, maxx, maxy)\n",
    "\n",
    "# +++++++ Rename x/y coordinates and set spatial dimensions to the new names\n",
    "da= da.rename(dict(x= 'lon', y= 'lat'))\n",
    "da.rio.set_spatial_dims(x_dim= 'lon', y_dim= 'lat', inplace= True)\n",
    "da= da.reset_coords(names= ['band', 'spatial_ref'], drop=True)\n",
    "\n",
    "# +++++++ Plot (cartopy)\n",
    "plt.close()\n",
    "#. . . . . . . . Plot elevation\n",
    "da.plot.imshow(vmin= vmin_elv, vmax= vmax_elv,\n",
    "#               cbar_kwargs= dict(shrink=0.6, label= 'Elv. (m)', pad=0.2),\n",
    "               cmap= cmap_terrain, norm= mpl.colors.PowerNorm(0.3, clip=False), robust= True,\n",
    "               cbar_kwargs= dict(shrink=0.6, label= 'Elv. (m)', aspect= 30),\n",
    "               transform= proj, subplot_kws= dict(projection= proj),\n",
    "              )\n",
    "plt.gcf().set_size_inches(15, 12)\n",
    "ax= plt.gca()\n",
    "\n",
    "#. . . . . . . . Map custumization\n",
    "plt.gcf().suptitle(title, y=1)\n",
    "if gdf_polyg is not None:\n",
    "    gdf_polyg.boundary.plot(ax=ax, edgecolor= 'magenta', facecolor= 'none', alpha= 1)\n",
    "if gdf_wrapping is not None: \n",
    "    gdf_wrapping.plot(ax=ax, edgecolor= 'grey', facecolor= 'none', color= 'none', linewidth= 0.6, alpha= 1) \n",
    "\n",
    "#. . . . . . . . Map wrapping (NaturalEarth)\n",
    "ax.add_feature(cfeature.NaturalEarthFeature('physical', 'rivers_lake_centerlines', res, facecolor='none', \n",
    "                edgecolor='blue'), alpha=0.5)\n",
    "ax.add_feature(cfeature.NaturalEarthFeature('physical', 'rivers_europe', res, facecolor='none', edgecolor='blue'))\n",
    "ax.add_feature(cfeature.NaturalEarthFeature('physical', 'lakes', res, facecolor=cfeature.COLORS['water'], edgecolor='blue'), \n",
    "                alpha=0.5)\n",
    "ax.add_feature(cfeature.NaturalEarthFeature('physical', 'ocean', res, facecolor= 'aliceblue', edgecolor='black'), \n",
    "                alpha= 1)\n",
    "    \n",
    "plt.savefig(os.path.join(folder_out, file_out), dpi=150)\n",
    "\n",
    "# +++++++ Plot (hvplot)\n",
    "hv= da.hvplot(x='lon', y='lat', width= 950, height=700, geo= True,  crs= proj,\n",
    "                coastline= res, \n",
    "                tiles=  'EsriTerrain', # CartoLight, StamenTerrain, StamenTerrainRetina, EsriTerrain, EsriNatGeo,  Wikipedia, OSM, StamenLabels, EsriReference\n",
    "                cmap= cmap_terrain, clabel= 'm', alpha=0.6,\n",
    "                title= title,\n",
    "                clim=(-700, vmax_elv), \n",
    "         )\n",
    "if gdf_polyg is not None :\n",
    "    hv= hv * gdf_polyg.hvplot(x='lon', y='lat', width= 950, height=700, geo= True,  crs= proj,\n",
    "                line_width= 2, line_color= 'magenta', fill_color= 'none',\n",
    "                tiles=  'EsriReference', # CartoLight, StamenTerrain, StamenTerrainRetina, EsriTerrain, EsriNatGeo,  Wikipedia, OSM, StamenLabels, EsriReference\n",
    "                clim=(-700, vmax_elv), \n",
    "         )\n",
    "hv"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}