[ENH] Sensitivity DataFrame #486
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Preserve development version 0.10.0.dev0 in docs/conf.py, setup.py, and serpentTools/__init__.py
…1-develop DEV/REL/BUG Merge 0.9.1 into develop
Introduced a helper class, ListOfArrays, to improve performance of the ResultsReader. Instead of using ``numpy.vstack`` each time a new row is appended into the ``ResultsReader.resdata``, the row is simply appended into a list. Some checking is done to ensure that the rows have consistent shape and data type. After reading, the final arrays are constructed and placed back in the ``resdata`` dictionary. This approach improves performance and scalability, especially for longer rows, e.g. more burnup steps.
DEV Use list of arrays to improve results parsing
The serpentTools.data.getFile and readDataFile functions now rely on the evnironment variable SERPENT_TOOLS_DATA. This will allow the files to be stored in any location, including outside the repository, but more importantly, outside the python package. Since this change is done inside the getFile function, none of the tests have to be modified at all. The travis build commands will have to be slightly modified to use this directory, and those changes will come shortly. Notes have been added to the developer guide to indicate the use of this variable, and how to set it. Related: GH Issue CORE-GATECH-GROUP#336
The files are no longer included in the python package, and thus won't be bundled for distribution nor install!! The interface file serpentTools/data/__init__.py has been moved to simply serpentTools/data.py, while all other files have been moved into the data/ directory. In the future, this directory may (and hopefully will) be removed from the entire repository. But for now, at least the python package is cleaner.
Notebook test script takes place in a temporary directory, where tmp/data likely does not exist.
Need the variables.yaml file to be shipped, but we no longer need versioneer files.
…ata-files DEV Unbundle data files from python package
PEP 518 introduced the pyproject.toml file for containing the bare minimum of build requirements. Some packages that use cython or f2py or numpy in the build process include those. We don't so our pyproject.toml file is very minimal. In doing this, we are ensured to work in a build environment that has setuptools, removing some ugly logic from the setup file. The toml file is kind of like an INI file, and is described more through the links in PEP 518 - https://www.python.org/dev/peps/pep-0518/ - https://github.com/toml-lang/toml This file has also been included in our MANIFEST file to ensure it is shipped with source distributions
Installs a serpentTools executable that provides the same functionality as "python -m serpentTools" Documentation has been updated in the command line section
DEV Better build configuration
Conflicts:
docs/changelog.rst
docs/conf.py
serpentTools/__init__.py
setup.cfg
setup.py
Reserve 0.10.0.dev0 version
Update python versions in setup.cfg
Keep minimal setup.py
* master: DOC Add NSE publication to readme, docs
Backport of features included since 0.9.3 release Includes the following PRs git log --online --pretty="%s" --merges 0.9.3... Merge pull request CORE-GATECH-GROUP#407 from drewejohnson/fix-readme Merge pull request CORE-GATECH-GROUP#406 from drewejohnson/r0.9.3 Merge pull request CORE-GATECH-GROUP#405 from nicoloabrate/master Merge pull request CORE-GATECH-GROUP#404 from drewejohnson/update-git-attr Merge pull request CORE-GATECH-GROUP#402 from drewejohnson/xsplot-labels Merge pull request CORE-GATECH-GROUP#400 from drewejohnson/lib2-notes Merge pull request CORE-GATECH-GROUP#399 from CORE-GATECH-GROUP/docs-sphinx-2.0 Merge pull request CORE-GATECH-GROUP#397 from drewejohnson/port-list-of-arrays Merge pull request CORE-GATECH-GROUP#398 from drewejohnson/det-supers Merge pull request CORE-GATECH-GROUP#396 from drewejohnson/df-arg-renaming Merge pull request CORE-GATECH-GROUP#394 from drewejohnson/0.9.3-update-docs Merge pull request CORE-GATECH-GROUP#395 from drewejohnson/hist-branch-dictlike Merge pull request CORE-GATECH-GROUP#393 from drewejohnson/less-metadata Merge pull request CORE-GATECH-GROUP#392 from drewejohnson/better-access Merge pull request CORE-GATECH-GROUP#391 from drewejohnson/dep-dataframe Merge pull request CORE-GATECH-GROUP#390 from drewejohnson/api-dep-attributes Merge pull request CORE-GATECH-GROUP#388 from drewejohnson/xsplot-branch Merge pull request CORE-GATECH-GROUP#378 from drewejohnson/better-build Merge pull request CORE-GATECH-GROUP#376 from drewejohnson/unbundle-data-files Merge pull request CORE-GATECH-GROUP#370 from drewejohnson/results-loa Merge pull request CORE-GATECH-GROUP#369 from drewejohnson/merge-0.9.1-develop
DEV Merge 0.9.3 into develop
The new class resides can be imported as serpentTools.next.base.SerpentFile, and describes the interface for the serpentTools.next design. Concrete classes are expected to provide a method for reading a Serpent source file, presented as an IOBase. The abstract class handles converting strings or pathlib.Path instances to readable streams
Part of drewtils package, 0.2.0
The new classes are contained in serpentTools/next/detector.py and are fully implemented. The DetectorFile inherits from a dictionary and SerpentFile, enforcing that all values in the dictionary are Detector instances. The __setitem__ and update methods are over-written to back this up. The dedicated DetectorReader is intended more for repeated use, and can be created externally to the DetectorFile. The main benefit is likely to be experienced if many files are to be read, and if a subset of detectors are to be processed from those files. A set of strings is used to control what detectors are processed from the file, defaulting to all detectors.
Contains just SerpentFile, DetectorFile, and DetectorReader classes at the moment
Only effects depletion parsing for ZAIS at the momement
Other cases are less likely across this package, and can be taken care of usually with some flavor of list comprehension. This removes a lot of logic inside this function, which is used a lot throughout this project. In most cases, we could probably remove the function as it is a one-liner now. Tests in test_utils.py relating to this function have been removed as they involved a lot of boilerplate for setting up and testing the now defunct modes for str2vec. Instead, a simple pytest test has been written that creates a random vector of three sizes (mainly to see how it handles vectors of a single entry), converts to a space-separated string, and then back to vector. In a couple cases across the project, the use of this function has been changed to just use list-comprehension
Primarily impacts results reader, but also touches the microxs file as well. This has the benefit of not-relying on the span (start and end) of the matched substring, and allows us to capture both the variable name and the desired quantities with match.groups()
Demonstrates that issue CORE-GATECH-GROUP#411 was fixed with 79a65ca by adding a small test comparing some scalars captured and then compared to the expected value. No absolute nor relative tolerance is used with pytest for as strict as a test as possible
BUG Capture full precision of scalars in ResultsReader
…on-3-letters [BUG] Try using the whole string if getting PlotGrid fails using only the first letter. (issue CORE-GATECH-GROUP#415)
Interface outside of the rc object for expanding variable groups. Abstracted away because the setting interface will eventually be removed (GH CORE-GATECH-GROUP#339) as each reader (built from CORE-GATECH-GROUP#335 and CORE-GATECH-GROUP#400) will control their own settings.
DEV Add variable expander in settings.py
(apply CORE-GATECH-GROUP#421) (cherry picked from commit f1607d2)
…ort-master CI Test with pytest<6.0
- Fix env vars - Fix SERPENT_TOOLS_DATA path for develop branch - Remove bdist_wheel, bdist_egg from testing - Fix coverage call
yerr and drawstyle are incompatible after 3.3.4 - matplotlib/matplotlib#19879 - matplotlib/matplotlib#19888
…ions-off-develop CI Transition from Travis ci to github actions
Signed-off-by: Andrew Johnson <a.johnson@usnc-tech.com>
DOC Switch to GHA test badge from travis
Signed-off-by: Andrew Johnson <a.johnson@usnc-tech.com>
…ns-usnc-docs DOC Update publications, add USNC-Tech sponsored development
[ENH] Sensitivity dataframe
Update as a consequence of my first contribution
|
@GrimFe it looks like you have made a PR from your develop branch into the main branch here. The problem is that those two branches have diverged to heavily that git has made this PR far too large. I think we both would expect this addition to really just be a few commits touching a few files. But because you have (admirably) started reconciling the git histories between I'll put some thoughts in #483 but I think our best move is to close this PR for now, and try to reconcile changes from #483 into |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The
toDataFramemethod was added to theSensitivityReaderclass.This allows for easy energy dependent sensitivity manipulation in a pandas context and follows what was implemented for the
DepletedMaterialclass.