feat(ts-gen): add timeseries generation #2112
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
MartinBelthle
force-pushed
the
feat/add-timeseries-generation
branch
2 times, most recently
from
39e9095
to
027978e
Compare
MartinBelthle
force-pushed
the
feat/add-timeseries-generation
branch
from
027978e
to
89e9d30
Compare
sylvlecl
requested changes
Aug 20, 2024
antarest/study/storage/variantstudy/model/command/generate_thermal_cluster_timeseries.py
Outdated
Show resolved
Hide resolved
antarest/study/storage/variantstudy/model/command/generate_thermal_cluster_timeseries.py
Show resolved
Hide resolved
antarest/study/storage/variantstudy/model/command/generate_thermal_cluster_timeseries.py
Show resolved
Hide resolved
| modulation_matrix = study_data.tree.get( | ||
| ["input", "thermal", "prepro", area_id, thermal.id.lower(), "modulation"] | ||
| )["data"] | ||
| modulation_capacity = np.array([row[2] for row in modulation_matrix]) |
There was a problem hiding this comment.
We should use an enum or constants for the various column numbers
There was a problem hiding this comment.
I used constants
antarest/study/storage/variantstudy/model/command/generate_thermal_cluster_timeseries.py
Outdated
Show resolved
Hide resolved
antarest/study/storage/variantstudy/model/command/generate_thermal_cluster_timeseries.py
Outdated
Show resolved
Hide resolved
antarest/study/storage/variantstudy/model/command/generate_thermal_cluster_timeseries.py
Outdated
Show resolved
Hide resolved
| # 8- Generates the UUID for the `get_inner_matrices` method | ||
| uuid = study_data.tree.context.matrix.create(generated_matrix) | ||
| self._INNER_MATRICES.append(uuid) | ||
| # 9- Write the matrix inside the input folder. |
There was a problem hiding this comment.
It's a shame that we need to:
- write the matrix twice, in the matrix store AND as a file here
- duplicate here the code that writes a matrix to a file: it's only 1 line but we need to ensure consistency between here and other places where we write matrice, it's error-prone for future evolutions
For 2, we should probably have a small utility method shared at least beween matrix.py and here.
For 1, I have not easy solution, we probaby need to postpone this ... Note that this kind of direct write will be an issue the day we want to store input matrices with a different format.
There was a problem hiding this comment.
Now that we don't have inner_matrices anymore, the 1. is no longer relevant.
Also for 2. I created a function dump_dataframe inside matrix.py that I reused here
antarest/study/storage/variantstudy/model/command/generate_thermal_cluster_timeseries.py
Show resolved
Hide resolved
antarest/study/storage/variantstudy/model/command/generate_thermal_cluster_timeseries.py
Outdated
Show resolved
Hide resolved
sylvlecl
approved these changes
Aug 21, 2024
maugde
pushed a commit
that referenced
this pull request
Aug 23, 2024
Uses antares-timeseries-generation to perform thermal clusters timeseries generation. The generation is performed as a background task on file studies, since it can take several seconds or more. Note that generated matrices are not "inner matrices" of the command since they are generated on each command execution. Generated matrices that will be stored later in the matrix store will be saved from garbage collection as all other snaphot matrices.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Here's an exemple of generation duration time with this PR performed after the commit
add little tests.It's clear to see why we need to use
tasks.We'll probably encounter some timeouts during variants safe generation. To avoid that, we'll have to increase this limit which is hard-coded at 60 seconds inside the code