Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Output XDMF that conforms to spec #820

Merged
merged 23 commits into from
Feb 6, 2023

Conversation

BenWibking
Copy link
Collaborator

@BenWibking BenWibking commented Jan 25, 2023

PR Summary

This changes the XDMF output to conform to the current XDMF specification (available here: https://gitlab.kitware.com/xdmf/xdmf/blob/master/Xdmf.dtd).

To fix the data format issues, this also incorporates the changes proposed in #796.

Fixes #819.

PR Checklist

  • Code passes cpplint
  • New features are documented.
  • Adds a test for any bugs fixed. Adds tests for new features.
  • Code is formatted
  • Changes are summarized in CHANGELOG.md
  • CI has been triggered on Darwin for performance regression tests.
  • Docs build
  • (@lanl.gov employees) Update copyright on changed files

@BenWibking
Copy link
Collaborator Author

I've tested this on the outputs from the advection example with Paraview 5.11.0 and it works fine. With the same output files, VisIt 3.22 still crashes.

@BenWibking
Copy link
Collaborator Author

The crashes are unrelated, and are fixed by #796.

Copy link
Collaborator

@Yurlungur Yurlungur left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So long as Visit's happy, I'm happy. I see some chatter on the Matrix channel, so I defer to you and @brryan about how to proceed.

@BenWibking
Copy link
Collaborator Author

Visit works, but Paraview is still unhappy (emits lots of identical warnings).

@Yurlungur
Copy link
Collaborator

Ah ok.

@BenWibking
Copy link
Collaborator Author

Ok, it works for me now in both VisIt and Paraview without any warnings. @Yurlungur @brryan does this look good to you?

Copy link
Collaborator

@Yurlungur Yurlungur left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this makes sense.

What output did you try this with? Before we merge I want to make sure visit/paraview are happy with both scalar and vector output and with 1 and many blocks. But assuming that's the case, I approve.

@BenWibking
Copy link
Collaborator Author

I tried this with both the advection example outputs and a problem in AthenaPK. I think (?) that covers both scalar and vector output, but I didn't test a single block simulation.

@Yurlungur
Copy link
Collaborator

I tried this with both the advection example outputs and a problem in AthenaPK. I think (?) that covers both scalar and vector output, but I didn't test a single block simulation.

Good enough for me.

Copy link
Collaborator

@pgrete pgrete left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly requesting cleanups. I can also confirm that the patch works (though I didn't test non-vector variables).

@@ -264,11 +264,12 @@ static std::string stringXdmfArrayRef(const std::string &prefix,
const std::string &hdfPath,
const std::string &label, const hsize_t *dims,
const int &ndims, const std::string &theType,
const int &precision) {
const int &precision, bool isVector) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this change?
isVector does not seem to be used.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes this was a mistake, thanks for noticing this

src/outputs/parthenon_hdf5.cpp Outdated Show resolved Hide resolved
Comment on lines -317 to -321
// TODO(JL) 3 and 5 are literals here, careful if they change.
// "3" rows for START, STRIDE, and COUNT for each slab with "5" (H5_NDIM) entries.
// START: iblock variable(_component) 0 0 0
// STRIDE: 1 1 1 1 1
// COUNT: 1 vector_size nx3 nx2 nx1
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the comment is still valid.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Comment on lines 319 to 321
for (int i = 2; i < ndims; i++) {
fid << dims[i] << " ";
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this change (compared to using dims321)?
I think we should remain consistent, i.e., either switch to a loop everywhere and get rid of dims321 or use dims321 wherever possible (which should make future changes easier as all places can be identified by the same pattern)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No reason now for this change, and in fact it was overly specialized to the 1D tensor dimension case -- reverted. I agree about dims321 consistency.

src/outputs/parthenon_hdf5.cpp Outdated Show resolved Hide resolved
for (const auto &vinfo : var_list) {
// TODO(BRR) just let vinfo provide this
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@brryan is that comment still valid (as vinfo is already used)?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had thought about storing alldims itself in vinfo but I don't think that's a meaningful change -- comment removed

BenWibking and others added 2 commits February 3, 2023 08:41
Co-authored-by: Philipp Grete <pgrete@hs.uni-hamburg.de>
Co-authored-by: Philipp Grete <pgrete@hs.uni-hamburg.de>
@BenWibking
Copy link
Collaborator Author

@brryan Can you address @pgrete's comments above? I think these changes originally came from your PR. (Alternatively, you could merge my changes into your PR.)

@brryan
Copy link
Collaborator

brryan commented Feb 3, 2023

@brryan Can you address @pgrete's comments above? I think these changes originally came from your PR. (Alternatively, you could merge my changes into your PR.)

Thanks for pinging me on this @BenWibking, yes I am responsible for these issues :). I made the fixes in response to @pgrete's comments in #796. While looking at this I noticed that scalar variable output wasn't working with XDMF, e.g. the in_or_out variable from the calculate_pi example (I think both advection and any AthenaPK problem only produce vector-type output (AthenaPK has one big state vector for all variables as I understand it)). I modified writeXdmfSlabVariableRef to work for the scalar case as well. Similar modifications would need to be made for 2- and 3-tensor dimension variables but since this work is already spread across two PRs and currently only I (?) use higher-tensor-dimension variables I'll open an issue instead.

Since the changes I made weren't entirely trivial could I trouble you to merge in my latest brryan/xdmf_fix and check that paraview is still happy (I only checked burgers, advection, and calculate_pi in visit)? Then if everything works I think we can just merge this PR in.

@BenWibking
Copy link
Collaborator Author

Since the changes I made weren't entirely trivial could I trouble you to merge in my latest brryan/xdmf_fix and check that paraview is still happy (I only checked burgers, advection, and calculate_pi in visit)? Then if everything works I think we can just merge this PR in.

I've checked that burgers, advection and calculate_pi all load into Paraview without any issues.

@pgrete
Copy link
Collaborator

pgrete commented Feb 6, 2023

I removed the unused parameter, formatted the code, and added a changelog.
Enabling auto-merge now.

@pgrete pgrete enabled auto-merge (squash) February 6, 2023 13:14
@brryan
Copy link
Collaborator

brryan commented Feb 6, 2023

Since the changes I made weren't entirely trivial could I trouble you to merge in my latest brryan/xdmf_fix and check that paraview is still happy (I only checked burgers, advection, and calculate_pi in visit)? Then if everything works I think we can just merge this PR in.

I've checked that burgers, advection and calculate_pi all load into Paraview without any issues.

Thanks @BenWibking!

I removed the unused parameter, formatted the code, and added a changelog. Enabling auto-merge now.

Thanks @pgrete! (sorry I missed the unused parameter)

@Yurlungur Yurlungur disabled auto-merge February 6, 2023 17:31
@Yurlungur Yurlungur enabled auto-merge February 6, 2023 17:31
@Yurlungur
Copy link
Collaborator

The Python/C++ formatting static check does not seem to be triggering. I tried enabling/disabling auto-merge to no effect. Is something wrong?

@Yurlungur
Copy link
Collaborator

Looks like the format check was skipped for some reason.

@Yurlungur
Copy link
Collaborator

I cloned the branch and ran formatting manually. It passes, so I'm going to force merge. @pgrete I think there maybe some extra step needed to get the formatting workflow to work on forks?

@Yurlungur Yurlungur disabled auto-merge February 6, 2023 18:30
@Yurlungur Yurlungur merged commit 4bb0299 into parthenon-hpc-lab:develop Feb 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

XDMF output is non-compliant with spec
4 participants