Skip to content

Run your first Local Test Scenario

Emmanuel Mathot edited this page Mar 22, 2024 · 6 revisions

In this page, we will run the some of the Local Test Scenarios testing the Target Site availability, data search and download. We assume you have followed the first part of this guide and the Test Site Preparation

We are going to run some Test Scenarios with 2 Target Sites:

Tip

All Test Scenarios and their Test Cases are described in the Test Scenarios Description page.

The Local Test Scenarios are run using cdab-client command line tool.

Start the Test Site Container

First, we will start the Test Site docker container we will use for this benchmark session. Once started, the container keeps running in the background and we can control it with the docker command line tool. It is then easy to copy files to the container, start a test scenario and retrieve the results.

docker run --detach --name testsite-1 ghcr.io/esacdab/cdab-testsuite:latest

Your Test Site Container should be ready. You can control it with the following command and you should have a similar output

$ docker ps

CONTAINER ID        IMAGE                                               COMMAND                  CREATED              STATUS              PORTS                    NAMES
0a7c0ab615fe        ghcr.io/esacdab/cdab-testsuite:latest               "sudo /usr/bin/supervisor…"   About a minute ago   Up About a minute   22/tcp                   testsite-1

Let's now copy the config.yaml file previously prepared to the container

docker cp config.yaml testsite-1:/home/jenkins/config.yaml

Note

The container is running as a non-root user. The config.yaml file is copied to the /home/jenkins directory which is the workspace directory where the user has write access.

Run Query & Download Test Scenarios

In this first benchmark, we will run the 2 first Test Scenarios TS01 and TS02 on Copernicus Data Space Ecosystem.

The first one will benchmark basic data access functions with several catalog searches and then a single download:

docker exec -it testsite-1 cdab-client -v -tsn=testsite-1 -tn=cdse TS01

The Test will begin and quickly you console will output something similar to

2024-03-18 08:54:47,910 [1] INFO  cdabtesttools.MainClass - ===============================
2024-03-18 08:54:47,920 [1] INFO  cdabtesttools.MainClass - Validating Test Options...
2024-03-18 08:54:47,920 [1] INFO  cdabtesttools.MainClass - [1] Loading & Checking configuration
2024-03-18 08:54:48,015 [1] DEBUG cdabtesttools.MainClass - Configuration found in /home/jenkins/config.yaml
2024-03-18 08:54:48,015 [1] INFO  cdabtesttools.MainClass - [2] Configuring target das-od-cf (https://catalogue.dataspace.copernicus.eu/odata/v1)
2024-03-18 08:54:48,018 [1] DEBUG Terradue.OpenSearch.Engine.OpenSearchEngine - Registering extension: atom
2024-03-18 08:54:48,018 [1] DEBUG Terradue.OpenSearch.Engine.OpenSearchEngine - Registering extension: json
2024-03-18 08:54:48,022 [1] DEBUG cdabtesttools.Target.TargetSiteWrapper - TARGET TYPE: DIAS
2024-03-18 08:54:48,022 [1] INFO  cdabtesttools.MainClass - Scenario is executed on an instance outside the target, external target-specific data access will be used if possible
2024-03-18 08:54:48,022 [1] INFO  cdabtesttools.MainClass - [3] Check scenarios compatibility...
2024-03-18 08:54:48,023 [1] INFO  cdabtesttools.MainClass - 1 Test Scenarios : TS01
2024-03-18 08:54:48,023 [1] INFO  cdabtesttools.MainClass - Test Options VALIDATED
2024-03-18 08:54:48,023 [1] INFO  cdabtesttools.MainClass - ===============================
2024-03-18 08:54:48,024 [1] INFO  cdabtesttools.MainClass - Setting ThreadPool Min/Max : 16/32767
2024-03-18 08:54:48,028 [1] INFO  cdabtesttools.MainClass - ********************************
2024-03-18 08:54:48,028 [1] INFO  cdabtesttools.MainClass - *   EXECUTING TEST SCENARIOS   *
2024-03-18 08:54:48,028 [1] INFO  cdabtesttools.MainClass - ********************************
2024-03-18 08:54:48,028 [1] INFO  cdabtesttools.MainClass - Creating Test Cases for Test Scenario [TS01] Simple data search and single download
2024-03-18 08:54:49,408 [1] INFO  cdabtesttools.MainClass - 3 Test Case(s) for Test Scenario [TS01]
2024-03-18 08:54:49,408 [1] INFO  cdabtesttools.MainClass - Queuing Tasks for Test Case [TC101] Service Reachability
2024-03-18 08:54:49,408 [1] INFO  cdabtesttools.MainClass - Queuing Tasks for Test Case [TC201] Basic catalogue query
2024-03-18 08:54:49,408 [1] INFO  cdabtesttools.MainClass - Queuing Tasks for Test Case [TC301] Single Remote Download
2024-03-18 08:54:49,411 [.NET ThreadPool Worker] INFO  cdabtesttools.MainClass - Tests Ignition ...
2024-03-18 08:54:49,411 [.NET ThreadPool Worker] INFO  cdabtesttools.MainClass - 3
2024-03-18 08:54:50,411 [.NET ThreadPool Worker] INFO  cdabtesttools.MainClass - 2
2024-03-18 08:54:51,412 [.NET ThreadPool Worker] INFO  cdabtesttools.MainClass - 1
2024-03-18 08:54:52,412 [.NET ThreadPool Worker] INFO  cdabtesttools.MainClass - 0
2024-03-18 08:54:53,413 [.NET ThreadPool Worker] INFO  cdabtesttools.MainClass - Tests FIRED !!!
2024-03-18 08:54:53,414 [.NET ThreadPool Worker] DEBUG cdabtesttools.MainClass - Prepare TC101
2024-03-18 08:54:53,418 [.NET ThreadPool Worker] DEBUG cdabtesttools.MainClass - > HTTP HEAD/GET https://catalogue.dataspace.copernicus.eu/odata/v1/Products ...
2024-03-18 08:54:53,418 [.NET ThreadPool Worker] DEBUG cdabtesttools.MainClass - > HTTP HEAD/GET https://catalogue.dataspace.copernicus.eu/odata/v1/Products ...
2024-03-18 08:54:53,764 [.NET ThreadPool Worker] DEBUG cdabtesttools.MainClass - Connected to catalogue.dataspace.copernicus.eu
2024-03-18 08:54:53,764 [.NET ThreadPool Worker] DEBUG cdabtesttools.MainClass - Reply from catalogue.dataspace.copernicus.eu
2024-03-18 08:54:53,765 [.NET ThreadPool Worker] DEBUG cdabtesttools.MainClass - SP CC 0/6
2024-03-18 08:54:53,765 [.NET ThreadPool Worker] DEBUG cdabtesttools.MainClass - Connected to catalogue.dataspace.copernicus.eu
2024-03-18 08:54:53,765 [.NET ThreadPool Worker] DEBUG cdabtesttools.MainClass - Reply from catalogue.dataspace.copernicus.eu
2024-03-18 08:54:53,765 [.NET ThreadPool Worker] DEBUG cdabtesttools.MainClass - SP CC 0/6
2024-03-18 08:54:53,767 [.NET ThreadPool Worker] DEBUG cdabtesttools.MainClass - < HTTP/1.1 OK OK 344ms
2024-03-18 08:54:53,770 [.NET ThreadPool Worker] DEBUG cdabtesttools.MainClass - < HTTP/1.1 OK OK 346ms
2024-03-18 08:54:53,786 [.NET ThreadPool Worker] DEBUG cdabtesttools.Measurement.MeasurementsAnalyzer - Average Response Time : 345ms
2024-03-18 08:54:53,786 [.NET ThreadPool Worker] DEBUG cdabtesttools.Measurement.MeasurementsAnalyzer - Peak Response Time : 346ms
2024-03-18 08:54:53,787 [.NET ThreadPool Worker] DEBUG cdabtesttools.Measurement.MeasurementsAnalyzer - Error Rate : 0%
2024-03-18 08:54:53,788 [.NET ThreadPool Worker] DEBUG cdabtesttools.Measurement.MeasurementsAnalyzer - average Concurrency : 1.98#
2024-03-18 08:54:53,789 [.NET ThreadPool Worker] DEBUG cdabtesttools.Measurement.MeasurementsAnalyzer - peak Concurrency : 2#
2024-03-18 08:54:53,790 [.NET ThreadPool Worker] DEBUG cdabtesttools.MainClass - Prepare TC201
2024-03-18 08:54:53,806 [.NET ThreadPool Worker] DEBUG cdabtesttools.MainClass - [1] > Query Simple-Random-1 Sentinel-5P L1B Online...
2024-03-18 08:54:53,806 [.NET ThreadPool Worker] DEBUG cdabtesttools.MainClass - [2] > Query Simple-Random-0 Sentinel-3 SY L2 VGP NTC Online...
2024-03-18 08:54:53,817 [.NET ThreadPool Worker] INFO  Terradue.OpenSearch.DataHub.Dias.CopernicusOdataSearchClient - Performing OData search
...

Test Scenario TS01 should be executed quite quickly and once completed, you should see the last lines like this:

2024-03-18 08:54:57,672 [.NET ThreadPool Worker] DEBUG cdabtesttools.MainClass - [5] 3.7MB downloaded (file #1:96.4% / total:100.0%) [11MB/s]
2024-03-18 08:54:57,672 [.NET ThreadPool Worker] DEBUG cdabtesttools.MainClass - [5] 3.8MB downloaded (file #1:97.4% / total:101.0%) [11.1MB/s]
2024-03-18 08:54:57,673 [.NET ThreadPool Worker] DEBUG cdabtesttools.MainClass - [5] 3.8MB downloaded (file #1:98.4% / total:102.1%) [11.2MB/s]
2024-03-18 08:54:57,674 [.NET ThreadPool Worker] DEBUG cdabtesttools.MainClass - [5] 3.8MB downloaded (file #1:99.4% / total:103.1%) [11.3MB/s]
2024-03-18 08:54:57,769 [.NET ThreadPool Worker] DEBUG cdabtesttools.Measurement.MeasurementsAnalyzer - Average Response Time : 445ms
2024-03-18 08:54:57,770 [.NET ThreadPool Worker] DEBUG cdabtesttools.Measurement.MeasurementsAnalyzer - Peak Response Time : 445ms
2024-03-18 08:54:57,770 [.NET ThreadPool Worker] DEBUG cdabtesttools.Measurement.MeasurementsAnalyzer - Error Rate : 0%
2024-03-18 08:54:57,770 [.NET ThreadPool Worker] DEBUG cdabtesttools.Measurement.MeasurementsAnalyzer - average Concurrency : 1#
2024-03-18 08:54:57,770 [.NET ThreadPool Worker] DEBUG cdabtesttools.Measurement.MeasurementsAnalyzer - peak Concurrency : 1#
2024-03-18 08:54:57,770 [.NET ThreadPool Worker] DEBUG cdabtesttools.Measurement.MeasurementsAnalyzer - Max Size : 4040064bytes
2024-03-18 08:54:57,770 [.NET ThreadPool Worker] DEBUG cdabtesttools.Measurement.MeasurementsAnalyzer - Total Size : 4040064bytes
2024-03-18 08:54:57,770 [.NET ThreadPool Worker] DEBUG cdabtesttools.Measurement.MeasurementsAnalyzer - Total Throughput : 5133499bytes/second
2024-03-18 08:54:57,770 [.NET ThreadPool Worker] DEBUG cdabtesttools.Measurement.MeasurementsAnalyzer - Data Collection Division : S5P_RPRO_L1B_IR_SIR_20180430T001950_20180430T020120_02818_93_020100_20220417T162224 S5P TROPOMI L1B_IR_SIR L1B starting at 04/30/2018 01:34:18 ending at 04/30/2018 01:37:28
2024-03-18 08:54:57,770 [.NET ThreadPool Worker] DEBUG cdabtesttools.Measurement.MeasurementsAnalyzer - Data Access Methods : http
2024-03-18 08:54:57,801 [1] INFO  cdabtesttools.MainClass - ********************************
2024-03-18 08:54:57,801 [1] INFO  cdabtesttools.MainClass - *   COMPLETED TEST SCENARIOS   *
2024-03-18 08:54:57,801 [1] INFO  cdabtesttools.MainClass - ********************************
Bye

We will now copy the benchmark results of this Test Scenario run.

docker cp testsite-1:/home/jenkins/TS01Results.json testsite-1-TS01-cdse-results.json

The file contains all the metrics for each Test Case measured during the benchmark.

Tip

The Test Results document produced by the tools are described in theTest Results Description page.

Let's now start the second Test Scenario TS02

This second one will benchmark more advanced data access function with several catalog searches and then multiple parallel downloads:

docker exec -it testsite-1 cdab-client -v -tsn=testsite-1 -tn=cdse TS02

Again the output of the console will show the progress of the Test Case

2024-03-18 09:00:00,402 [1] INFO  cdabtesttools.MainClass - ===============================
2024-03-18 09:00:00,411 [1] INFO  cdabtesttools.MainClass - Validating Test Options...
2024-03-18 09:00:00,411 [1] INFO  cdabtesttools.MainClass - [1] Loading & Checking configuration
2024-03-18 09:00:00,507 [1] DEBUG cdabtesttools.MainClass - Configuration found in /home/jenkins/config.yaml
2024-03-18 09:00:00,507 [1] INFO  cdabtesttools.MainClass - [2] Configuring target das-od-cf (https://catalogue.dataspace.copernicus.eu/odata/v1)
2024-03-18 09:00:00,510 [1] DEBUG Terradue.OpenSearch.Engine.OpenSearchEngine - Registering extension: atom
2024-03-18 09:00:00,510 [1] DEBUG Terradue.OpenSearch.Engine.OpenSearchEngine - Registering extension: json
2024-03-18 09:00:00,513 [1] DEBUG cdabtesttools.Target.TargetSiteWrapper - TARGET TYPE: DIAS
2024-03-18 09:00:00,513 [1] INFO  cdabtesttools.MainClass - Scenario is executed on an instance outside the target, external target-specific data access will be used if possible
2024-03-18 09:00:00,513 [1] INFO  cdabtesttools.MainClass - [3] Check scenarios compatibility...
2024-03-18 09:00:00,514 [1] INFO  cdabtesttools.MainClass - 1 Test Scenarios : TS02
2024-03-18 09:00:00,514 [1] INFO  cdabtesttools.MainClass - Test Options VALIDATED
2024-03-18 09:00:00,514 [1] INFO  cdabtesttools.MainClass - ===============================
2024-03-18 09:00:00,515 [1] INFO  cdabtesttools.MainClass - Setting ThreadPool Min/Max : 16/32767
2024-03-18 09:00:00,519 [1] INFO  cdabtesttools.MainClass - ********************************
2024-03-18 09:00:00,519 [1] INFO  cdabtesttools.MainClass - *   EXECUTING TEST SCENARIOS   *
2024-03-18 09:00:00,519 [1] INFO  cdabtesttools.MainClass - ********************************
2024-03-18 09:00:00,519 [1] INFO  cdabtesttools.MainClass - Creating Test Cases for Test Scenario [TS02] Complex data search and bulk download
2024-03-18 09:00:01,944 [1] INFO  cdabtesttools.MainClass - 2 Test Case(s) for Test Scenario [TS02]
2024-03-18 09:00:01,944 [1] INFO  cdabtesttools.MainClass - Queuing Tasks for Test Case [TC202] Complex Query
2024-03-18 09:00:01,945 [1] INFO  cdabtesttools.MainClass - Queuing Tasks for Test Case [TC302] Multiple Remote Download
2024-03-18 09:00:01,947 [.NET ThreadPool Worker] INFO  cdabtesttools.MainClass - Tests Ignition ...
2024-03-18 09:00:01,948 [.NET ThreadPool Worker] INFO  cdabtesttools.MainClass - 3
2024-03-18 09:00:02,948 [.NET ThreadPool Worker] INFO  cdabtesttools.MainClass - 2
2024-03-18 09:00:03,948 [.NET ThreadPool Worker] INFO  cdabtesttools.MainClass - 1
2024-03-18 09:00:04,949 [.NET ThreadPool Worker] INFO  cdabtesttools.MainClass - 0
2024-03-18 09:00:05,949 [.NET ThreadPool Worker] INFO  cdabtesttools.MainClass - Tests FIRED !!!
2024-03-18 09:00:05,950 [.NET ThreadPool Worker] DEBUG cdabtesttools.MainClass - Prepare TC202
2024-03-18 09:00:05,972 [.NET ThreadPool Worker] DEBUG cdabtesttools.MainClass - [4] > Query Complex-Random-0 Sentinel-1 GRD intersecting Venezuela Online...
2024-03-18 09:00:05,972 [.NET ThreadPool Worker] DEBUG cdabtesttools.MainClass - [2] > Query Complex-Random-3 Sentinel-5P L2 Offline From Tuesday, 31 August 2021 To Friday, 15 December 2023 intersecting Dominican Republic Online...
2024-03-18 09:00:05,972 [.NET ThreadPool Worker] DEBUG cdabtesttools.MainClass - [1] > Query Complex-Random-2 Sentinel-2 L1C intersecting Sudan Online...
2024-03-18 09:00:05,972 [.NET ThreadPool Worker] DEBUG cdabtesttools.MainClass - [5] > Query Complex-Random-4 Sentinel-2 L1C From Wednesday, 19 October 2016 intersecting Belize Online...
2024-03-18 09:00:05,972 [.NET ThreadPool Worker] DEBUG cdabtesttools.MainClass - [3] > Query Complex-Random-1 Sentinel-2 L1C To Friday, 15 January 2016 intersecting Democratic Republic of the Congo Online...
2024-03-18 09:00:05,985 [.NET ThreadPool Worker] INFO  Terradue.OpenSearch.DataHub.Dias.CopernicusOdataSearchClient - Performing OData search
2024-03-18 09:00:05,985 [.NET ThreadPool Worker] INFO  Terradue.OpenSearch.DataHub.Dias.CopernicusOdataSearchClient - Performing OData search
2024-03-18 09:00:05,985 [.NET ThreadPool Worker] INFO  Terradue.OpenSearch.DataHub.Dias.CopernicusOdataSearchClient - Performing OData search
2024-03-18 09:00:05,985 [.NET ThreadPool Worker] INFO  Terradue.OpenSearch.DataHub.Dias.CopernicusOdataSearchClient - Performing OData search
2024-03-18 09:00:05,985 [.NET ThreadPool Worker] INFO  Terradue.OpenSearch.DataHub.Dias.CopernicusOdataSearchClient - Performing OData search
2024-03-18 09:00:05,986 [.NET ThreadPool Worker] DEBUG Terradue.OpenSearch.DataHub.Dias.CopernicusOdataSearchClient - Request parameters:
2024-03-18 09:00:05,986 [.NET ThreadPool Worker] DEBUG Terradue.OpenSearch.DataHub.Dias.CopernicusOdataSearchClient - Request parameters:
2024-03-18 09:00:05,986 [.NET ThreadPool Worker] DEBUG Terradue.OpenSearch.DataHub.Dias.CopernicusOdataSearchClient - Request parameters:
2024-03-18 09:00:05,986 [.NET ThreadPool Worker] DEBUG Terradue.OpenSearch.DataHub.Dias.CopernicusOdataSearchClient - Request parameters:
2024-03-18 09:00:05,986 [.NET ThreadPool Worker] DEBUG Terradue.OpenSearch.DataHub.Dias.CopernicusOdataSearchClient - Request parameters:
2024-03-18 09:00:05,986 [.NET ThreadPool Worker] DEBUG Terradue.OpenSearch.DataHub.Dias.CopernicusOdataSearchClient - - $filter = Collection/Name eq 'SENTINEL-5P' and ContentDate/End ge 2021-08-31T23:00:00Z and ContentDate/Start lt 2023-12-15T17:00:00Z and OData.CSC.Intersects(area=geography'SRID=4326;POLYGON ((-71.7 19.7, -69.9 19.7, -68.3 18.6, -71.1 18.3, -71.4 17.6, -71.7 19.7))') and Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'processingLevel' and att/OData.CSC.StringAttribute/Value eq 'L2') and Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'processingMode' and att/OData.CSC.StringAttribute/Value eq 'OFFL') and Online eq true
...

After some minutes, once completed...

2024-03-18 09:00:49,442 [.NET ThreadPool Worker] DEBUG cdabtesttools.MainClass - [21] 357.5MB downloaded (file #1:94.0% / total:94.0%) [11.5MB/s]
2024-03-18 09:00:49,651 [.NET ThreadPool Worker] DEBUG cdabtesttools.MainClass - [21] 361.3MB downloaded (file #1:95.0% / total:95.0%) [11.6MB/s]
2024-03-18 09:00:49,963 [.NET ThreadPool Worker] DEBUG cdabtesttools.MainClass - [21] 365.1MB downloaded (file #1:96.0% / total:96.0%) [11.6MB/s]
2024-03-18 09:00:50,228 [.NET ThreadPool Worker] DEBUG cdabtesttools.MainClass - [21] 368.9MB downloaded (file #1:97.1% / total:97.1%) [11.6MB/s]
2024-03-18 09:00:50,435 [.NET ThreadPool Worker] DEBUG cdabtesttools.MainClass - [21] 372.7MB downloaded (file #1:98.1% / total:98.1%) [11.6MB/s]
2024-03-18 09:00:50,643 [.NET ThreadPool Worker] DEBUG cdabtesttools.MainClass - [21] 376.5MB downloaded (file #1:99.1% / total:99.1%) [11.7MB/s]
2024-03-18 09:00:54,098 [.NET ThreadPool Worker] DEBUG cdabtesttools.Measurement.MeasurementsAnalyzer - Average Response Time : 879ms
2024-03-18 09:00:54,098 [.NET ThreadPool Worker] DEBUG cdabtesttools.Measurement.MeasurementsAnalyzer - Peak Response Time : 879ms
2024-03-18 09:00:54,098 [.NET ThreadPool Worker] DEBUG cdabtesttools.Measurement.MeasurementsAnalyzer - Error Rate : 0%
2024-03-18 09:00:54,098 [.NET ThreadPool Worker] DEBUG cdabtesttools.Measurement.MeasurementsAnalyzer - average Concurrency : 1#
2024-03-18 09:00:54,098 [.NET ThreadPool Worker] DEBUG cdabtesttools.Measurement.MeasurementsAnalyzer - peak Concurrency : 1#
2024-03-18 09:00:54,098 [.NET ThreadPool Worker] DEBUG cdabtesttools.Measurement.MeasurementsAnalyzer - Max Size : 398584249bytes
2024-03-18 09:00:54,098 [.NET ThreadPool Worker] DEBUG cdabtesttools.Measurement.MeasurementsAnalyzer - Total Size : 398584249bytes
2024-03-18 09:00:54,098 [.NET ThreadPool Worker] DEBUG cdabtesttools.Measurement.MeasurementsAnalyzer - Total Throughput : 10923109bytes/second
2024-03-18 09:00:54,098 [.NET ThreadPool Worker] DEBUG cdabtesttools.Measurement.MeasurementsAnalyzer - Data Collection Division : S2A_MSIL1C_20150708T081016_N0204_R078_T35NQD_20150708T082158 S2A MSI S2MSI1C LEVEL-1C starting at 07/08/2015 08:10:16 ending at 07/08/2015 08:10:16
2024-03-18 09:00:54,098 [.NET ThreadPool Worker] DEBUG cdabtesttools.Measurement.MeasurementsAnalyzer - Data Access Methods : http
2024-03-18 09:00:54,131 [1] INFO  cdabtesttools.MainClass - ********************************
2024-03-18 09:00:54,131 [1] INFO  cdabtesttools.MainClass - *   COMPLETED TEST SCENARIOS   *
2024-03-18 09:00:54,131 [1] INFO  cdabtesttools.MainClass - ********************************
Bye

... we copy the results

docker cp testsite-1:/home/jenkins/TS02Results.json testsite-1-TS02-cdse-results.json

Run Cataloge Coverage Test Scenario

In this section, we will run the Test Scenarios TS05 on ONDA.

This Test Scenario will measure the coverage of the Target Site catalogue in comparison of the reference catalogue on Copernicus Data Space Ecosystem.

docker exec -it testsite-1 cdab-client -v -tsn=testsite-1 -tn=onda TS05

the output of the console will show the progress of the Test Cases. This test scenario will produce a lot of output as it will scan the entire catalogue of the Target Site and compare it with the reference catalogue.

2024-03-18 09:01:39,535 [1] INFO  cdabtesttools.MainClass - ===============================
2024-03-18 09:01:39,544 [1] INFO  cdabtesttools.MainClass - Validating Test Options...
2024-03-18 09:01:39,545 [1] INFO  cdabtesttools.MainClass - [1] Loading & Checking configuration
2024-03-18 09:01:39,639 [1] DEBUG cdabtesttools.MainClass - Configuration found in /home/jenkins/config.yaml
2024-03-18 09:01:39,639 [1] INFO  cdabtesttools.MainClass - [2] Configuring target onda (https://catalogue.onda-dias.eu/dias-catalogue/)
2024-03-18 09:01:39,642 [1] DEBUG Terradue.OpenSearch.Engine.OpenSearchEngine - Registering extension: atom
2024-03-18 09:01:39,642 [1] DEBUG Terradue.OpenSearch.Engine.OpenSearchEngine - Registering extension: json
2024-03-18 09:01:39,646 [1] DEBUG Terradue.OpenSearch.DataHub.DHuS.DataHubWrapperSettings - ONDA ODATA V2 API identified
2024-03-18 09:01:39,647 [1] DEBUG cdabtesttools.Target.TargetSiteWrapper - TARGET TYPE: DIAS
2024-03-18 09:01:39,647 [1] INFO  cdabtesttools.MainClass - Scenario is executed on an instance outside the target, external target-specific data access will be used if possible
2024-03-18 09:01:39,647 [1] INFO  cdabtesttools.MainClass - [3] Check scenarios compatibility...
2024-03-18 09:01:39,647 [1] INFO  cdabtesttools.MainClass - 1 Test Scenarios : TS05
2024-03-18 09:01:39,647 [1] INFO  cdabtesttools.MainClass - Test Options VALIDATED
2024-03-18 09:01:39,647 [1] INFO  cdabtesttools.MainClass - ===============================
2024-03-18 09:01:39,649 [1] INFO  cdabtesttools.MainClass - Setting ThreadPool Min/Max : 16/32767
2024-03-18 09:01:39,654 [1] INFO  cdabtesttools.MainClass - ********************************
2024-03-18 09:01:39,654 [1] INFO  cdabtesttools.MainClass - *   EXECUTING TEST SCENARIOS   *
2024-03-18 09:01:39,654 [1] INFO  cdabtesttools.MainClass - ********************************
2024-03-18 09:01:39,654 [1] INFO  cdabtesttools.MainClass - Creating Test Cases for Test Scenario [TS05] Coverage Analysis
2024-03-18 09:01:39,655 [1] INFO  cdabtesttools.MainClass - 3 Test Case(s) for Test Scenario [TS05]
2024-03-18 09:01:39,655 [1] INFO  cdabtesttools.MainClass - Queuing Tasks for Test Case [TC501] Catalogue Coverage
2024-03-18 09:01:39,656 [1] INFO  cdabtesttools.MainClass - Queuing Tasks for Test Case [TC502] Target Local Data Coverage
2024-03-18 09:01:39,656 [1] INFO  cdabtesttools.MainClass - Queuing Tasks for Test Case [TC503] Target Local Data Offer Consistency
2024-03-18 09:01:39,657 [.NET ThreadPool Worker] INFO  cdabtesttools.MainClass - Tests Ignition ...
2024-03-18 09:01:39,657 [.NET ThreadPool Worker] INFO  cdabtesttools.MainClass - 3
2024-03-18 09:01:40,658 [.NET ThreadPool Worker] INFO  cdabtesttools.MainClass - 2
2024-03-18 09:01:41,659 [.NET ThreadPool Worker] INFO  cdabtesttools.MainClass - 1
2024-03-18 09:01:42,659 [.NET ThreadPool Worker] INFO  cdabtesttools.MainClass - 0
2024-03-18 09:01:43,660 [.NET ThreadPool Worker] INFO  cdabtesttools.MainClass - Tests FIRED !!!
2024-03-18 09:01:43,662 [.NET ThreadPool Worker] DEBUG cdabtesttools.MainClass - Prepare TC501
2024-03-18 09:01:43,664 [.NET ThreadPool Worker] DEBUG Terradue.OpenSearch.Engine.OpenSearchEngine - Registering extension: atom
2024-03-18 09:01:43,664 [.NET ThreadPool Worker] DEBUG Terradue.OpenSearch.Engine.OpenSearchEngine - Registering extension: json
2024-03-18 09:01:43,665 [.NET ThreadPool Worker] DEBUG cdabtesttools.Target.TargetSiteWrapper - TARGET TYPE: DIAS
2024-03-18 09:01:43,666 [.NET ThreadPool Worker] DEBUG Terradue.OpenSearch.Engine.OpenSearchEngine - Registering extension: atom
2024-03-18 09:01:43,666 [.NET ThreadPool Worker] DEBUG Terradue.OpenSearch.Engine.OpenSearchEngine - Registering extension: json
2024-03-18 09:01:43,666 [.NET ThreadPool Worker] DEBUG cdabtesttools.Target.TargetSiteWrapper - TARGET TYPE: DIAS
2024-03-18 09:01:43,670 [.NET ThreadPool Worker] DEBUG Terradue.OpenSearch.DataHub.DHuS.DataHubWrapperSettings - ONDA ODATA V2 API identified
2024-03-18 09:01:43,672 [.NET ThreadPool Worker] DEBUG cdabtesttools.MainClass - [1] > Query [reference] Sentinel-1 RAW before 1D  before 1D...
2024-03-18 09:01:43,672 [.NET ThreadPool Worker] DEBUG cdabtesttools.MainClass - [2] > Query [target] Sentinel-1 RAW before 1D  before 1D...
2024-03-18 09:01:43,685 [.NET ThreadPool Worker] DEBUG Terradue.OpenSearch.DataHub.DHuS.DHuSODataClient - Querying (try = 3)...
2024-03-18 09:01:43,686 [.NET ThreadPool Worker] INFO  Terradue.OpenSearch.DataHub.Dias.CopernicusOdataSearchClient - Performing OData search
2024-03-18 09:01:43,686 [.NET ThreadPool Worker] DEBUG Terradue.OpenSearch.DataHub.Dias.CopernicusOdataSearchClient - Request parameters:
2024-03-18 09:01:43,687 [.NET ThreadPool Worker] DEBUG Terradue.OpenSearch.DataHub.Dias.CopernicusOdataSearchClient - - $filter = Collection/Name eq 'SENTINEL-1' and ContentDate/End ge 2014-01-01T00:00:00Z and ContentDate/Start lt 2014-04-01T00:00:00Z and Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'productType' and att/OData.CSC.StringAttribute/Value eq 'RAW')
2024-03-18 09:01:43,687 [.NET ThreadPool Worker] DEBUG Terradue.OpenSearch.DataHub.Dias.CopernicusOdataSearchClient - - $top = 0
2024-03-18 09:01:43,687 [.NET ThreadPool Worker] DEBUG Terradue.OpenSearch.DataHub.Dias.CopernicusOdataSearchClient - - $expand = Attributes
2024-03-18 09:01:43,687 [.NET ThreadPool Worker] DEBUG Terradue.OpenSearch.DataHub.DHuS.DataHubWebClient - Querying https://catalogue.onda-dias.eu/dias-catalogue/Products?$search="platformName:*Sentinel-1* AND beginPosition:[* TO 2024-03-17T09:01:43.0000000Z] AND productType:*RAW*"&$top=0&$orderby=creationDate desc
2024-03-18 09:01:43,702 [.NET ThreadPool Worker] DEBUG Terradue.OpenSearch.DataHub.DHuS.DataHubWebClient - Connected to catalogue.onda-dias.eu
2024-03-18 09:01:43,850 [.NET ThreadPool Worker] DEBUG Terradue.OpenSearch.DataHub.DHuS.DataHubWebClient - Reply from catalogue.onda-dias.eu
2024-03-18 09:01:43,883 [.NET ThreadPool Worker] DEBUG Terradue.OpenSearch.DataHub.DHuS.DHuSODataClient - Querying Count (Try = 3) : https://catalogue.onda-dias.eu/dias-catalogue/Products/$count?$search="platformName:*Sentinel-1* AND beginPosition:[* TO 2024-03-17T09:01:43.0000000Z] AND productType:*RAW*"
2024-03-18 09:01:43,969 [.NET ThreadPool Worker] DEBUG Terradue.OpenSearch.DataHub.DHuS.DataHubWrapperSettings - ONDA ODATA V2 API identified
2024-03-18 09:01:43,983 [.NET ThreadPool Worker] DEBUG cdabtesttools.MainClass - [3] < 0/0 entries for [target] Sentinel-1 RAW before 1D  before 1D. 297bytes in 147ms
2024-03-18 09:01:43,984 [.NET ThreadPool Worker] DEBUG Terradue.OpenSearch.DataHub.DHuS.DataHubWrapperSettings - ONDA ODATA V2 API identified
2024-03-18 09:01:43,984 [.NET ThreadPool Worker] DEBUG cdabtesttools.MainClass - [4] > Query [reference] Sentinel-1 SLC before 1D  before 1D...
2024-03-18 09:01:43,984 [.NET ThreadPool Worker] DEBUG cdabtesttools.MainClass - [5] > Query [target] Sentinel-1 SLC before 1D  before 1D...
2024-03-18 09:01:43,985 [.NET ThreadPool Worker] INFO  Terradue.OpenSearch.DataHub.Dias.CopernicusOdataSearchClient - Performing OData search
2024-03-18 09:01:43,985 [.NET ThreadPool Worker] DEBUG Terradue.OpenSearch.DataHub.DHuS.DHuSODataClient - Querying (try = 3)...
2024-03-18 09:01:43,985 [.NET ThreadPool Worker] DEBUG Terradue.OpenSearch.DataHub.Dias.CopernicusOdataSearchClient - Request parameters:
2024-03-18 09:01:43,985 [.NET ThreadPool Worker] DEBUG Terradue.OpenSearch.DataHub.Dias.CopernicusOdataSearchClient - - $filter = Collection/Name eq 'SENTINEL-1' and ContentDate/End ge 2014-01-01T00:00:00Z and ContentDate/Start lt 2014-04-01T00:00:00Z and Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'productType' and att/OData.CSC.StringAttribute/Value eq 'SLC')
2024-03-18 09:01:43,985 [.NET ThreadPool Worker] DEBUG Terradue.OpenSearch.DataHub.Dias.CopernicusOdataSearchClient - - $top = 0
2024-03-18 09:01:43,985 [.NET ThreadPool Worker] DEBUG Terradue.OpenSearch.DataHub.Dias.CopernicusOdataSearchClient - - $expand = Attributes
2024-03-18 09:01:43,985 [.NET ThreadPool Worker] DEBUG Terradue.OpenSearch.DataHub.DHuS.DataHubWebClient - Querying https://catalogue.onda-dias.eu/dias-catalogue/Products?$search="platformName:*Sentinel-1* AND beginPosition:[* TO 2024-03-17T09:01:43.0000000Z] AND productType:*SLC*"&$top=0&$orderby=creationDate desc
2024-03-18 09:01:43,985 [.NET ThreadPool Worker] DEBUG Terradue.OpenSearch.DataHub.DHuS.DataHubWebClient - Connected to catalogue.onda-dias.eu
...

After some minutes, once completed...

2024-03-18 09:21:48,426 [.NET ThreadPool Worker] DEBUG cdabtesttools.Measurement.MeasurementsAnalyzer - Data Collection Division : Sentinel-1 RAW last 3D,Sentinel-1 SLC,Sentinel-1 GRD,Sentinel-1 OCN,Sentinel-2 L1C,Sentinel-2 L2A,Sentinel-3 OLCI L1 NTC,Sentinel-3 OLCI L2 Land FR NTC,Sentinel-3 OLCI L2 Land RR NTC,Sentinel-3 SRAL L1 NTC,Sentinel-3 SRAL L2 Land NTC,Sentinel-3 SLSTR L1 NTC,Sentinel-3 SLSTR L2 Land NTC,Sentinel-3 SYNERGY L2 NTC,Sentinel-3 STC,Sentinel-5P L1B,Sentinel-5P L2 Offline,Sentinel-5P L2 NRT,Sentinel-5P L2 RPROstring
2024-03-18 09:21:48,426 [.NET ThreadPool Worker] DEBUG cdabtesttools.Measurement.MeasurementsAnalyzer - Target Total Results : 0,625573,75807,0,0,3500467,0,0,0,0,4927,0,0,6343,0,0,53058,0,0#
2024-03-18 09:21:48,426 [.NET ThreadPool Worker] DEBUG cdabtesttools.Measurement.MeasurementsAnalyzer - Reference Total Results : 1852,2449487,3081342,1175482,41189860,40775169,1141042,1163014,73965,431300,152242,1717925,1997748,747954,1105892,405154,329728,2617282,368929#
2024-03-18 09:21:48,426 [.NET ThreadPool Worker] DEBUG cdabtesttools.Measurement.MeasurementsAnalyzer - Data Offer Consistency : 0,25.54,2.46,0,0,8.58,0,0,0,0,3.24,0,0,0.85,0,0,16.09,0,0%
2024-03-18 09:21:48,455 [1] INFO  cdabtesttools.MainClass - ********************************
2024-03-18 09:21:48,455 [1] INFO  cdabtesttools.MainClass - *   COMPLETED TEST SCENARIOS   *
2024-03-18 09:21:48,455 [1] INFO  cdabtesttools.MainClass - ********************************

... we copy the results

docker cp testsite-1:/home/jenkins/TS05Results.json testsite-1-TS05-onda-results.json

Stop and Clean the Test Site Container

At the end of the benchmark session, when all reaults have been copied, we can stop and remove the the container.

docker stop testsite-1
docker rm testsite-1

Warning

It is important to clean container after a benchmark session. Indeed, it contains its own image and all the data downloaded during the tests. Severall gigabytes of disk space may be used!

⏭️ You can now run the Remote Test Scenarios.