Multiple Maximum Generation and Pumping for Hydro (Merging with develop branch)

docs/reference-guide/03-commands.md

@@ -178,7 +178,7 @@ this command allows to state, for each kind of time-series, whether it should be
 the available set (be it ready-made or Antares-generated) _**OR**_ should take a user-defined value
 (in the former case, the default "rand" value should be kept; in the latter, the value should be the reference number of the time-series to use). Multiple simulation profiles can be defined and archived. The default active profile gives the "rand" status for all time-series in all areas (full probabilistic simulation).
 
-  Regarding Hydro time-series, the scenario builder gives, in addition to the assignment of a specific number to use for the inflows time-series, the ability to define the initial reservoir level to use for each MC year.
+  Regarding Hydro time-series, the scenario builder gives, in addition to the assignment of a specific number to use for the inflows time-series, the ability to define the initial reservoir level to use for each MC year, also hydro max power scenario builder is available to support time-series for Maximum Generation and Maximum Pumping because the number of TS's for ROR, Hydro Storage and Minimum Generation can be different than the  number of TS's for Maximum Generation and Maximum Pumping.
 
 - **MC Scenario playlist** For each Monte-Carlo year of the simulation defined in the "Simulation" active window,
 this command allows to state whether a MC year prepared for the simulation should be actually simulated or not.

docs/reference-guide/04-active_windows.md

@@ -99,10 +99,10 @@ These two parts are detailed hereafter.
 
 ### RIGHT PART: Time-series management
 
-For the different kinds of time-series that Antares manages in a non-deterministic way (load, thermal generation, hydro power, wind power, solar power or renewable depending on the option chosen):
+For the different kinds of time-series that Antares manages in a non-deterministic way (load, thermal generation, hydro power, hydro max power, wind power, solar power or renewable depending on the option chosen):
 
 1. **Choice of the kind of time-series to use**
-Either « ready-made » or «stochastic » (i.e. Antares-generated), defined by setting the value to either "on" or "off". Note that for Thermal time-series, the cluster-wise parameter may overrule this global parameter (see Thermal window description below).
+Either « ready-made » or «stochastic » (i.e. Antares-generated), defined by setting the value to either "on" or "off". Exception is hydro max power that can only be « ready-made ». Note that for Thermal time-series, the cluster-wise parameter may overrule this global parameter (see Thermal window description below).
 
 2. **For stochastic TS only**:
     - **Number** Number of TS to generate
@@ -334,11 +334,11 @@ which explains the comparatively long length of this chapter.
 
 In the main Window, the user may pick any area appearing in the list and is then given access to different tabs:
 
-- The "time-series" tab displays the "ready-made" time-series already available for simulation purposes. There are two categories of time-series (displayed in two different subtabs): the Run of River (ROR) time-series on the one hand and the Storage power (SP) time-series on the other hand.
+- The "time-series" tab displays the "ready-made" time-series already available for simulation purposes. There are five categories of time-series (displayed in five different subtabs): the Run of River (ROR) time-series, the Storage power (SP) time-series, the Minimum Generation power, the Maximum Generation power and the Maximum Pumping Power.
 
-    ROR time-series are defined at the hourly scale; each of the 8760 values represents the ROR power expected at a given hour, expressed in round number and in MW. The SP time-series are defined at the daily scale; each of the 365 values represents an overall SP energy expected in the day, expressed in round number and in MWh. These natural inflows are considered to be storable into a reservoir for later use.
+    ROR time-series are defined at the hourly scale; each of the 8760 values represents the ROR power expected at a given hour, expressed in round number and in MW. The SP time-series are defined at the daily scale; each of the 365 values represents an overall SP energy expected in the day, expressed in round number and in MWh. These natural inflows are considered to be storable into a reservoir for later use. The Minimum Generation time-series are defined at the hourly scale; each of the 8760 values represents the Minimum Generation power expected at a given hour expressed in round number and in MW. The Maximum Generation time-series are defined at the hourly scale; each of the 8760 values represents the Maximum Generation power expected at a given hour expressed in round number and in MW. The Maximum Pumping time-series are defined at the hourly scale; each of the 8760 values represents the Maximum Pumping power expected at a given hour expressed in round number and in MW.
 
-    Both types of data may come from any origin outside Antares, or may have been formerly generated by the Antares time-series stochastic generator and stored as input data on the user's request. Different ways to update data are:
+    ROR time-series and SP time-series may come from any origin outside Antares, or may have been formerly generated by the Antares time-series stochastic generator and stored as input data on the user's request. Minimum Generation, Maximum Generation and Maximum Pumping may come from any origin outside Antares, but they can not be generated by the Antares time-series stochastic generator. Different ways to update data are:
 
     - direct typing
     - copy/paste a selected field to/from the clipboard
@@ -352,7 +352,8 @@ In the main Window, the user may pick any area appearing in the list and is then
 
     - _Note that:_
 
-        - _For a given area, the number of ROR time-series and SP times-series **must** be identical_
+        - _For a given area, the number of ROR time-series, SP times-series and Minimum Generation **must** be identical_
+        - _For a given area, the number of Maximum Generation and Maximum Pumping **must** be identical_
         - _If the "intra-modal correlated draws" option was not selected in the_ **simulation** _window,
       MC adequacy or economy simulations can take place even if the number of hydro time-series is not the same
       in all areas (e.g. 2 , 5 , 1 , 45 ,...)_
@@ -415,10 +416,10 @@ the weekly optimal hydro-thermal unit-commitment and dispatch process.
 Standard credits (Bottom part)
 
 The bottom part displays two daily time-series (365 values) defined for energy generation/storage
-(hydro turbines or hydro pumps). In each case, the first array defines the maximum power (generated or absorbed),
-and the second defines the maximum daily energy (either generated or stored).
+(hydro turbines or hydro pumps). Both arrays represents the maximum daily energy (either generated or stored).
 
-For the sake of clarity, maximum daily energies are expressed as a number of hours at maximum power.
+For the sake of clarity, maximum daily energies are expressed as a number of hours at maximum power and these values
+are used along with the Maximum Generation and Maximum Pumping TS's to calculate daily mean energy.
 
 Credit modulation (Upper part)
 

docs/reference-guide/13-file-format.md

@@ -1,5 +1,16 @@
 # Study format changes
 This is a list of all recent changes that came with new Antares Simulator features. The main goal of this document is to lower the costs of changing existing interfaces, both GUI and scripts.
+## v9.1.0
+### (Input) Hydro Maximum Generation/Pumping Power
+* For each area, new files are added **input/hydro/series/<area>/maxHourlyGenPower.txt** and **input/hydro/series/<area>/maxHourlyPumpPower.txt**. These files have one or more columns, and 8760 rows. The number of columns in these two files must be the same if there is more than one column in each file, but if there is just one column for example in maxHourlyGenPower.txt file, maxHourlyPumpPower.txt file can have more than one column and vice versa. Starting from v9.0, file **input/hydro/common/capacity/maxpower_<area>** is no longer used.
+### How to upgrade capacity time-series ?
+Source file = maxpower_<area>, destination files are maxHourlyGenPower.txt and maxHourlyPumpPower.txt. To upgrade time-series, you need to 
+1. For generation, multiply 1st and 2nd rows. For pumping, multiply 3rd and 4rd rows.
+2. Duplicate values 24 times : source has 365 rows, destination has 365 * 24 = 8760 rows
+* Also for each area, new files are added **input/hydro/common/capacity/maxDailyGenEnergy_<area>** and **input/hydro/common/capacity/maxDailyPumpEnergy_<area>**. These files have just one column and 365 rows. For old studies, file **input/hydro/common/capacity/maxpower_<area>** will be deleted after cleaning a study.
+* Under `Configure/MC Scenario Builder` new section is added `Hydro Max Power`
+* In the existing file **settings/scenariobuilder.dat**, under **<ruleset>** section following properties added: **hgp,<area>,<year> = <hgp-value>**
+
 ## v9.0.0
 ### Input
 ### Study version
@@ -13,6 +24,7 @@ version = 9.0
 ```
 Compatibility is kept with versions up to 8.8.0. Starting from version 9.0.0, the new format must be used.
 
+
 ## v8.8.0
 ### Input
 #### Short-term storage

src/libs/antares/series/series.cpp

@@ -135,5 +135,4 @@ uint64_t TimeSeries::memoryUsage() const
 {
     return timeSeries.memoryUsage();
 }
-
 } // namespace Antares::Data

src/libs/antares/study/CMakeLists.txt

@@ -23,6 +23,8 @@ set(SRC_STUDY_SCENARIO_BUILDER
         include/antares/study/scenario-builder/ThermalTSNumberData.h
         include/antares/study/scenario-builder/HydroTSNumberData.h
         scenario-builder/HydroTSNumberData.cpp
+        include/antares/study/scenario-builder/HydroMaxPowerTSNumberData.h
+        scenario-builder/HydroMaxPowerTSNumberData.cpp
         scenario-builder/SolarTSNumberData.cpp
         include/antares/study/scenario-builder/solarTSNumberData.h
         include/antares/study/scenario-builder/WindTSNumberData.h
@@ -127,6 +129,8 @@ set(SRC_STUDY_PART_HYDRO
         include/antares/study/parts/hydro/allocation.h
         include/antares/study/parts/hydro/allocation.hxx
         parts/hydro/allocation.cpp
+        include/antares/study/parts/hydro/hydromaxtimeseriesreader.h
+        parts/hydro/hydromaxtimeseriesreader.cpp
 )
 source_group("study\\part\\hydro" FILES ${SRC_STUDY_PART_HYDRO})
 

src/libs/antares/study/area/area.cpp

@@ -259,6 +259,7 @@ void Area::resizeAllTimeseriesNumbers(uint nbYears)
     solar.series.timeseriesNumbers.reset(1, nbYears);
     wind.series.timeseriesNumbers.reset(1, nbYears);
     hydro.series->timeseriesNumbers.reset(1, nbYears);
+    hydro.series->timeseriesNumbersHydroMaxPower.reset(1, nbYears);
     for (auto& namedLink : links)
     {
         AreaLink* link = namedLink.second;

src/libs/antares/study/area/list.cpp

@@ -874,11 +874,38 @@ static bool AreaListLoadFromFolderSingleArea(Study& study,
             buffer.clear() << study.folderInput << SEP << "hydro" << SEP << "prepro";
             ret = area.hydro.prepro->loadFromFolder(study, area.id, buffer.c_str()) && ret;
         }
-        if (area.hydro.series && (!options.loadOnlyNeeded || !area.hydro.prepro)) // Series
+
+        auto* hydroSeries = area.hydro.series;
+        if (!options.loadOnlyNeeded || !area.hydro.prepro) // Series
         {
             buffer.clear() << study.folderInput << SEP << "hydro" << SEP << "series";
-            ret = area.hydro.series->loadFromFolder(study, area.id, buffer) && ret;
+            ret = hydroSeries->loadGenerationTS(area.id, buffer, study.header.version) && ret;
+
+            hydroSeries->EqualizeGenerationTSsizes(area, study.usedByTheSolver);
         }
+
+        if (study.header.version < StudyVersion(9,1))
+        {
+            buffer.clear() << study.folderInput << SEP << "hydro";
+
+            HydroMaxTimeSeriesReader reader(area.hydro, area.id.to<std::string>(), area.name.to<std::string>());
+            ret = reader.read(buffer, study.usedByTheSolver) && ret;
+        }
+        else
+        {
+            buffer.clear() << study.folderInput << SEP << "hydro" << SEP << "series";
+            ret = hydroSeries->LoadMaxPower(area.id, buffer) && ret;
+
+            if (study.usedByTheSolver)
+            {
+                hydroSeries->EqualizeMaxPowerTSsizes(area);
+            }
+            else
+                hydroSeries->setHydroModulability(area);
+        }
+
+        hydroSeries->resizeTSinDeratedMode(
+            study.parameters.derated, study.header.version, study.usedByTheSolver);
     }
 
     // Wind
@@ -1522,9 +1549,7 @@ void AreaList::removeLoadTimeseries()
 
 void AreaList::removeHydroTimeseries()
 {
-    each([](Data::Area& area) {
-        area.hydro.series->reset();
-    });
+    each([](Data::Area& area) { area.hydro.series->reset(); });
 }
 
 void AreaList::removeSolarTimeseries()

src/libs/antares/study/area/scratchpad.cpp

@@ -29,7 +29,36 @@ using namespace Yuni;
 namespace Antares::Data
 {
 
-AreaScratchpad::AreaScratchpad(const StudyRuntimeInfos& rinfos, Area& area)
+bool doWeHaveOnePositiveMaxDailyEnergy(const Matrix<double>& dailyPower,
+                                       const Matrix<double>::ColumnType& nbHoursAtPmaxPerDay)
+{
+    for (uint tsNumber = 0; tsNumber < dailyPower.width; ++tsNumber)
+    {
+        for (uint day = 0; day < DAYS_PER_YEAR; ++day)
+        {
+            if (dailyPower[tsNumber][day] * nbHoursAtPmaxPerDay[day] > 0.)
+                return true;
+        }
+    }
+
+    return false;
+}
+
+void CalculateDailyMeanPower(const Matrix<double>::ColumnType& hourlyColumn,
+                             Matrix<double>::ColumnType& dailyColumn)
+{
+    for (uint day = 0; day < DAYS_PER_YEAR; ++day)
+    {
+        dailyColumn[day] = std::accumulate(hourlyColumn + day * HOURS_PER_DAY,
+                                           hourlyColumn + day * HOURS_PER_DAY + HOURS_PER_DAY,
+                                           0)
+                           / 24.;
+    }
+}
+
+AreaScratchpad::AreaScratchpad(const StudyRuntimeInfos& rinfos, Area& area) :
+ meanMaxDailyGenPower(area.hydro.series->timeseriesNumbersHydroMaxPower),
+ meanMaxDailyPumpPower(area.hydro.series->timeseriesNumbersHydroMaxPower)
 {
     // alias to the simulation mode
     auto mode = rinfos.mode;
@@ -44,12 +73,6 @@ AreaScratchpad::AreaScratchpad(const StudyRuntimeInfos& rinfos, Area& area)
         originalMustrunSum[h] = std::numeric_limits<double>::quiet_NaN();
     }
 
-    for (uint d = 0; d != DAYS_PER_YEAR; ++d)
-    {
-        optimalMaxPower[d] = std::numeric_limits<double>::quiet_NaN();
-        pumpingMaxPower[d] = std::numeric_limits<double>::quiet_NaN();
-    }
-
     // Fatal hors hydro
     {
         double sum;
@@ -71,6 +94,37 @@ AreaScratchpad::AreaScratchpad(const StudyRuntimeInfos& rinfos, Area& area)
         }
     }
 
+    //*******************************************************************************
+    // TODO : about computing hydro max power daily mean from hourly max power TS.
+    //*******************************************************************************
+    //   - This computation is done here, but we don't want it here.
+    //     We want Scratchpad to shrink and even disappear.
+    //     So a possible solution to move this computation to some place else is to host 
+    //     these means TS in the hydro part of areas, and compute them right after 
+    //     their the hourly TS (max power).
+    //     Note that scratchpad instances are duplicated for multi-threading purpose,
+    //     and that moving these TS elsewhere could create concurrency issues.
+    //     But these daily TS, once computed, are then only read (in daily.cpp 
+    //     and when building the weekly optimization problem).
+    //     Thus we don't have to fear such issues.
+    //   - Besides, there is a performance problem here : for a given area, we compute
+    //     the max power daily means for each call to scratchpad constructor, that is 
+    //     the same computation for each thread.
+    //     This is another reason to move the computation from here.
+    //*******************************************************************************
+
+    //  Hourly maximum generation/pumping power matrices and their number of TS's (width of matrices)
+    auto const& maxHourlyGenPower = area.hydro.series->maxHourlyGenPower.timeSeries;
+    auto const& maxHourlyPumpPower = area.hydro.series->maxHourlyPumpPower.timeSeries;
+    uint nbOfMaxPowerTimeSeries = area.hydro.series->maxPowerTScount();
+
+    //  Setting width and height of daily mean maximum generation/pumping power matrices
+    meanMaxDailyGenPower.timeSeries.reset(nbOfMaxPowerTimeSeries, DAYS_PER_YEAR);
+    meanMaxDailyPumpPower.timeSeries.reset(nbOfMaxPowerTimeSeries, DAYS_PER_YEAR);
+
+    // Instantiate daily mean maximum generation/pumping power matrices
+    CalculateMeanDailyMaxPowerMatrices(maxHourlyGenPower, maxHourlyPumpPower, nbOfMaxPowerTimeSeries);
+
     // ===============
     // hydroHasMod
     // ===============
@@ -81,20 +135,10 @@ AreaScratchpad::AreaScratchpad(const StudyRuntimeInfos& rinfos, Area& area)
     // Useful whether we use a heuristic target or not
     bool hydroGenerationPermission = false;
 
-    // ... Getting hydro max power
-    auto const& maxPower = area.hydro.maxPower;
+    // ... Getting hydro max energy
+    auto const& dailyNbHoursAtGenPmax = area.hydro.dailyNbHoursAtGenPmax[0];
 
-    // ... Hydro max generating power and energy
-    auto const& maxGenP = maxPower[Data::PartHydro::genMaxP];
-    auto const& maxGenE = maxPower[Data::PartHydro::genMaxE];
-
-    double value = 0.;
-    for (uint d = 0; d < DAYS_PER_YEAR; ++d)
-        value += maxGenP[d] * maxGenE[d];
-
-    // If generating energy is nil over the whole year, hydroGenerationPermission is false, true
-    // otherwise.
-    hydroGenerationPermission = (value > 0.);
+    hydroGenerationPermission = doWeHaveOnePositiveMaxDailyEnergy(meanMaxDailyGenPower.timeSeries, dailyNbHoursAtGenPmax);
 
     // ---------------------
     // Hydro has inflows
@@ -123,30 +167,31 @@ AreaScratchpad::AreaScratchpad(const StudyRuntimeInfos& rinfos, Area& area)
     // --------------------------
     hydroHasMod = hydroHasInflows || hydroGenerationPermission;
 
-
     // ===============
     // Pumping
     // ===============
-    // ... Hydro max power
 
-    // ... Hydro max pumping power and energy
-    auto const& maxPumpingP = maxPower[Data::PartHydro::pumpMaxP];
-    auto const& maxPumpingE = maxPower[Data::PartHydro::pumpMaxE];
+    //  Hydro max pumping energy
+    auto const& dailyNbHoursAtPumpPmax = area.hydro.dailyNbHoursAtPumpPmax[0];
 
-    // ... Pumping max power
-    for (uint d = 0; d != DAYS_PER_YEAR; ++d)
-        pumpingMaxPower[d] = maxPumpingP[d];
-
-    double valuePumping = 0.;
-    // ... Computing 'pumpHasMod' parameter
-    for (uint d = 0; d < DAYS_PER_YEAR; ++d)
-        valuePumping += maxPumpingP[d] * maxPumpingE[d];
-
-    // If pumping energy is nil over the whole year, pumpHasMod is false, true otherwise.
-    pumpHasMod = (valuePumping > 0.);
+    //  If pumping energy is nil over the whole year, pumpHasMod is false, true otherwise.
+    pumpHasMod = doWeHaveOnePositiveMaxDailyEnergy(meanMaxDailyPumpPower.timeSeries, dailyNbHoursAtPumpPmax);
 }
 
-AreaScratchpad::~AreaScratchpad() = default;
+void AreaScratchpad::CalculateMeanDailyMaxPowerMatrices(const Matrix<double>& hourlyMaxGenMatrix,
+                                                        const Matrix<double>& hourlyMaxPumpMatrix,
+                                                        uint nbOfMaxPowerTimeSeries)
+{
+    for (uint nbOfTimeSeries = 0; nbOfTimeSeries < nbOfMaxPowerTimeSeries; ++nbOfTimeSeries)
+    {
+        auto& hourlyMaxGenColumn = hourlyMaxGenMatrix[nbOfTimeSeries];
+        auto& hourlyMaxPumpColumn = hourlyMaxPumpMatrix[nbOfTimeSeries];
+        auto& MeanMaxDailyGenPowerColumn = meanMaxDailyGenPower.timeSeries[nbOfTimeSeries];
+        auto& MeanMaxDailyPumpPowerColumn = meanMaxDailyPumpPower.timeSeries[nbOfTimeSeries];
 
+        CalculateDailyMeanPower(hourlyMaxGenColumn, MeanMaxDailyGenPowerColumn);
+        CalculateDailyMeanPower(hourlyMaxPumpColumn, MeanMaxDailyPumpPowerColumn);
+    }
+}
 } // namespace Antares::Data