The Match-up Data Base (MDB) includes match-ups between different satellite sensors and in situ data. It was developed as a way to improve the exchange, and processing of match-up data among different entities based on the concept by EUMETSAT (https://ocdb.readthedocs.io/en/latest/ocdb-MDB-user-manual.html). There is one MDB file for each site and satellite sensor.
The workflow of the MDB creation:
- Create satellite extract
- Add in situ data to the satellite extract
- Concatenation of all extract files to one single MDB file
The hypernets_val/SAT_EXTRACT/sat_extract_OLCI.py
script creates the single satellite extracts in a NetCDF format. It can be run as:
cd SAT_EXTRACT
./sat_extract_OLCI.py -c CONFIG_FILE
where the config file has the parameters needed to select and extract the data from the satellite granules. An example of config file (hypernets_val/SAT_EXTRACT/config_sat_extract_OLCI.ini) used for the extract of EUMETSAT Sentinel-3/OLCI WFR data:
## config file for creating satellite extracts
[file_path]
sat_source_dir: /path/to/sat/data
output_dir: /path/where/to/save/output
[Time_and_sites_selection]
#first and last dates for analysis. Format: YYYY-MM-DD
time_start: 2021-01-01
time_stop: 2021-07-07
sites: VEIT
[satellite_options]
sat_prefix: satellite_
satellite: S3
platform: A,B
sensor: olci
# WRR o WFR for olci
resolution: WFR
# processor version. Note: without zeros to the left
proc_version: 7.00
extract_size: 25
window_size: 3
n_bands: 16
#apply BRDF to OLCI reflectance
BRDF: F
The hypernets_val//MDB_builder/MDB_builder.py
adds the in situ data to the satellite extracts and concatenates them to a single MDB file in NetCDF format. It can be run as:
cd MDB_builder
MDB_builder.py -c CONFIG_FILE
where the config file has the parameters needed to select the satellite extracts and the in situ data. An example of config file (hypernets_val//MDB_builder/cconfig_MDB_builder_OLCI_HYPSTAR.ini.template) used for creating a MDB for EUMETSAT Sentinel-3/OLCI WFR data and HYPSTAR data:
## config file for MDBs builder
[file_path]
sat_extract_dir: /path/to/sat/extracts
ins_source_dir: /path/to/in/situ/data
output_dir: /path/where/to/save/output
[Time_and_sites_selection]
#first and last dates for analysis. Format: YYYY-MM-DD
time_start: 2021-01-01
time_stop: 2021-07-07
#insitu_type: PANTHYR, AERONET
insitu_type: HYPERNETS
#AERONET-OC or PANTHYR sites: ALL or any from AERONET-OC list. Default is ALL
sites: BEFR
[satellite_options]
sat_prefix: satellite_
satellite: S3
platform: A,B
sensor: olci
# WRR o WFR for olci
resolution: WFR
# processor version. Note: without zeros to the left
proc_version: 7.00
extract_size: 25
window_size: 3
n_bands: 16
#apply BRDF to OLCI reflectance
BRDF: F