Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for 4D arrays in ZFP compressor #4344

Merged
merged 5 commits into from
Oct 3, 2024

Conversation

gregorweiss
Copy link
Contributor

This PR introduces a small yet potentially valuable enhancement for users who work with 4D data arrays and require support for ZFP data compression.

While working on data compression for Neko, we encountered the need to compress 4D data arrays using the ZFP compressor, specifically the zfp_field_4d() function. This feature is available starting with ZFP version 0.5.4 https://zfp.readthedocs.io/en/release0.5.4/versions.html

The current ADIOS2 release supports ZFP version 0.5.3. Therefore, a CMake source check has been introduced to conditionally enable this feature if the system's ZFP version is 0.5.4 or higher.

The branch has been successfully tested with our Neko use case.

cmake/DetectOptions.cmake Outdated Show resolved Hide resolved
@vicentebolea
Copy link
Collaborator

feel free to merge

@gregorweiss
Copy link
Contributor Author

Test 1130: Bindings.C.ADIOS2_C_API.ADIOS2BPWriteStatsOnly.MPI in Ubunut20.04, gcc8, ompi is failing after merging. I don't see that this test would cover the code in this PR. What are your thoughts?

@eisenhauer
Copy link
Member

Probably not a real failure. I've forced a rerun of that job.

@vicentebolea
Copy link
Collaborator

WriteStateOnly is a new test, it maybe flaky. It should not hold this PR from being merged.

@anagainaru
Copy link
Contributor

anagainaru commented Oct 3, 2024

WriteStateOnly is a new test, it maybe flaky. It should not hold this PR from being merged.

I agree it should not hold this PR from being merged. But the test should not be flaky, what was the error? I didn't see it.

@eisenhauer
Copy link
Member

The reader reported steps=0, then a series of failures after that. Perhaps the writer died prematurely. But a rerun fixed it.

@eisenhauer eisenhauer merged commit 2b40572 into ornladios:master Oct 3, 2024
37 checks passed
@gregorweiss gregorweiss deleted the feature/zfp-4d branch October 3, 2024 14:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants