control_points_sampling.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "9d28001d-6550-4103-8e45-667d02546bff",
   "metadata": {},
   "source": [
    "## Sampling of Control Points\n",
    "- **Input**: a raster file representing the study area, where the value of each pixel is assigned to one of 5 vegetation classes. These classes were obtained by partitioning the area using K-means clustering based on NDVI, MSAVI, albedo and TGSI values over the study period.\n",
    "- **Output**: We sample an equal number of control points with an arbitrary date in each class. The resulting json and shapefile are used to generate the corresponding spectral indices feeded to the linear regression model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "d2f92411-c44a-4912-a6c6-06d9508cec9b",
   "metadata": {},
   "outputs": [],
   "source": [
    "from osgeo import gdal\n",
    "import random\n",
    "import json\n",
    "import shapefile"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "45f7eebb-94ba-4250-bd9f-bf13281ba907",
   "metadata": {},
   "source": [
    "Global variables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "87225786-be39-4dcd-973e-7351e2b68c00",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "tiff_file = './rasters/clustered_study_area.tif'\n",
    "out_json = './models/control-points.json'\n",
    "out_shp_prefix = './models/control-points'\n",
    "\n",
    "cp_number = 5000\n",
    "cp_per_class = 1000\n",
    "first_year = 2002\n",
    "last_year = 2022"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "20fe826b-348e-4bda-891b-45396caee8a2",
   "metadata": {},
   "source": [
    "Open the raster file and get image dimensions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "bb253816-0e88-4973-9a36-8eda9d16b7d6",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "raster = gdal.Open(tiff_file)\n",
    "xsize = raster.RasterXSize\n",
    "ysize = raster.RasterYSize\n",
    "num_pixels = xsize * ysize\n",
    "num_bands = raster.RasterCount"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "a5a893bc-90cf-4f11-899c-eea568edc288",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "the number of bands: 1\n",
      "The number of rows: 39675\n",
      "The number of columns: 14808\n",
      "The number of pixels in the image: 587507400\n"
     ]
    }
   ],
   "source": [
    "print(f\"the number of bands: {num_bands}\")\n",
    "print(f\"The number of rows: {xsize}\")\n",
    "print(f\"The number of columns: {ysize}\")\n",
    "print(f\"The number of pixels in the image: {num_pixels}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "21116c97-3902-494f-8ff4-c38f10b2f86f",
   "metadata": {},
   "source": [
    "---\n",
    "Get the CRS of the raster"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "6d7f7075-ca9f-4d28-83f9-8f82e1ac21e0",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The CRS of the GeoTIFF file is: GEOGCS[\"WGS 84\",DATUM[\"WGS_1984\",SPHEROID[\"WGS 84\",6378137,298.257223563,AUTHORITY[\"EPSG\",\"7030\"]],AUTHORITY[\"EPSG\",\"6326\"]],PRIMEM[\"Greenwich\",0],UNIT[\"degree\",0.0174532925199433,AUTHORITY[\"EPSG\",\"9122\"]],AXIS[\"Latitude\",NORTH],AXIS[\"Longitude\",EAST],AUTHORITY[\"EPSG\",\"4326\"]]\n"
     ]
    }
   ],
   "source": [
    "crs = raster.GetProjection()\n",
    "print(f\"The CRS of the GeoTIFF file is: {crs}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4780aab4-ef2f-4caa-9a4a-c0d8641b4a0f",
   "metadata": {},
   "source": [
    "The **[GetGeoTransform](https://gdal.org/tutorials/geotransforms_tut.html)** function returns a tuple with following attributes:\\\n",
    "- geo_tr(0) x-coordinate of the upper-left corner of the upper-left pixel.\n",
    "- geo_tr(1) w-e pixel resolution / pixel width.\n",
    "- geo_tr(2) row rotation (typically zero).\n",
    "- geo_tr(3) y-coordinate of the upper-left corner of the upper-left pixel.\n",
    "- geo_tr(4) column rotation (typically zero).\n",
    "- geo_tr(5) n-s pixel resolution / pixel height (negative value for a north-up image)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "4fdde000-e8ec-46a5-a8ea-0e803aafe084",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Coordinates of the upper-left corner of the upper-left pixel: -2.21551,36.04490\n",
      "Pixel size: 0.00027,-0.00027\n"
     ]
    }
   ],
   "source": [
    "# Get the geo-transform matrix\n",
    "geo_tr = raster.GetGeoTransform()\n",
    "print(f\"Coordinates of the upper-left corner of the upper-left pixel: {geo_tr[0]:.5f},{geo_tr[3]:.5f}\")\n",
    "print(f\"Pixel size: {geo_tr[1]:.5f},{geo_tr[5]:.5f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3c4d2f7e-2c0c-4631-a8d9-1b21c4e07ad1",
   "metadata": {},
   "source": [
    "Read the raster as an array"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "c132c074-a062-40bc-8f18-ad9b111d515a",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "band_data = raster.GetRasterBand(1)\n",
    "pixel_array = band_data.ReadAsArray()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d61baca3-8865-4f98-81c4-11ec90d98a35",
   "metadata": {},
   "source": [
    "This function takes the coordinates of a pixel in the raster and returns its geographic coordinates"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "565fc181-1aa6-4dec-aea1-4ad665b9167e",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def geo_coord(x,y):\n",
    "    x_origin = geo_tr[0]\n",
    "    y_origin = geo_tr[3]\n",
    "    pixel_width = geo_tr[1]\n",
    "    pixel_height = geo_tr[5]\n",
    "    x_geo = x_origin + x * pixel_width + (pixel_width / 2)\n",
    "    y_geo = y_origin + y * pixel_height + (pixel_height / 2)\n",
    "    return (x_geo,y_geo)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "859b488b-6801-4134-adab-e4d487f5be6c",
   "metadata": {
    "tags": []
   },
   "source": [
    "An example of the calculation of the geographical coordinates of a pixel"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "a735a3ea-bb84-46b1-9a65-22d900de7058",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The geographical coordinates of pixel (7400,19000) are (-0.22112, 30.92437)\n"
     ]
    }
   ],
   "source": [
    "pixel_x = 7400  \n",
    "pixel_y = 19000 \n",
    "(x_geo,y_geo) = geo_coord(pixel_x,pixel_y)\n",
    "print(f\"The geographical coordinates of pixel ({pixel_x},{pixel_y}) are ({x_geo:.5f}, {y_geo:.5f})\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ae488800-86cc-42b4-9619-f43a04b513aa",
   "metadata": {},
   "source": [
    "---\n",
    "The following loop performs a stratified sampling of control points to represent the 5 vegetation classes over the study area and returns a json file. A random date within the study period is given to attributed to each point."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "4220a3d3-8034-408d-8229-f53e06b7d4f2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of samples per class: [1000, 1000, 1000, 1000, 1000]\n"
     ]
    }
   ],
   "source": [
    "iter = 0\n",
    "current_samples_per_class = [0,0,0,0,0]\n",
    "samples_list = []\n",
    "while iter < cp_number: \n",
    "    rnd_pixel_x= random.randint(0, xsize-1)\n",
    "    rnd_pixel_y = random.randint(0, ysize-1)\n",
    "    rnd_pixel_val = pixel_array[rnd_pixel_y,rnd_pixel_x]\n",
    "    if rnd_pixel_val != 0 and current_samples_per_class[rnd_pixel_val-1]<cp_per_class :\n",
    "        current_samples_per_class[rnd_pixel_val-1] += 1\n",
    "        geo_coord_pixel = geo_coord(rnd_pixel_x,rnd_pixel_y)        \n",
    "        random_date = \"{:04d}\".format(random.randint(first_year, last_year))+'-'+\"{:02d}\".format(random.randint(1, 12))+'-'+\"{:02d}\".format(random.randint(1, 28))\n",
    "        samples_list.append({'latitude': str(geo_coord_pixel[1]), 'longitude': str(geo_coord_pixel[0]), \n",
    "                          'class': str(rnd_pixel_val),'map_date': random_date})\n",
    "        iter += 1\n",
    "print(f\"Number of samples per class: {current_samples_per_class}\")\n",
    "with open(out_json, 'w') as json_file:\n",
    "    json.dump(samples_list, json_file)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "66430ff0-bd35-4fd9-84c8-41a156ac0818",
   "metadata": {},
   "source": [
    "A shapefile of the generated control points is also generated"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "1226c7e7-bc91-4f95-826b-04375bf8c86b",
   "metadata": {},
   "outputs": [],
   "source": [
    " # Define the shapefile attributes\n",
    "fields = [(\"ID\", \"N\", 6, 0), (\"Class\", \"N\", 1, 0),(\"map_date\", \"C\", 10)]\n",
    "\n",
    "# Define the shapefile writer\n",
    "w = shapefile.Writer(out_shp_prefix, shapeType=shapefile.POINT)\n",
    "for field in fields:\n",
    "    w.field(*field)\n",
    "    \n",
    "id = 1 \n",
    "for point in samples_list:\n",
    "    w.point(float(point[\"longitude\"]), float(point[\"latitude\"]))\n",
    "    w.record(id,point['class'],point[\"map_date\"])\n",
    "    id += 1\n",
    "w.crs = crs\n",
    "# Save the shapefile\n",
    "w.close()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}