Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add global reduction for "magic heating" #31

Closed
wants to merge 117 commits into from

Conversation

BenWibking
Copy link
Contributor

This adds a global reduction in the hydro driver to compute a 1D vertical profile of the cooling rate.

Due to the way reductions are implemented in Parthenon, this requires adding tasks in hydro driver. However, this is somewhat of a specialized application, so I'm not sure how best to handle this special case. Feedback is welcome!

@BenWibking
Copy link
Contributor Author

BenWibking commented Mar 8, 2023

When manually running the regression tests on HPCC, they all pass. Waiting on an MPI build for the automated CI runner.

@BenWibking
Copy link
Contributor Author

@pgrete I can reproduce the weird values using the cloud.in input file with this PR. This suggests there is a bug in the reductions code, or else in how I am using them.

src/hydro/hydro.hpp Outdated Show resolved Hide resolved
Comment on lines 200 to 217
// global reduction of a vector
const int numHist = pin->GetOrAddInteger("precipitator", "numHist", 8);
if (parthenon::Globals::my_rank == 0) {
std::cout << "Using numHist = " << numHist << std::endl;
}
parthenon::AllReduce<PinnedArray1D<Real>> profile_reduce;
profile_reduce.val = PinnedArray1D<Real>("Reduce me", numHist);
pkg->AddParam("profile_reduce", profile_reduce, true);

PinnedArray1D<Real> profile_reduce_zbins("Bin centers", numHist);
const Real x3min = pin->GetReal("parthenon/mesh", "x3min");
const Real x3max = pin->GetReal("parthenon/mesh", "x3max");
const Real dz_hist = (x3max - x3min) / numHist;
for (int i = 0; i < numHist; ++i) {
profile_reduce_zbins(i) = dz_hist * (Real(i) + 0.5) + x3min;
}
pkg->AddParam("profile_reduce_zbins", profile_reduce_zbins, false);

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that this is (currently) still a "single problem generator" related feature, I suggest to move it to that pgen (this also applies to the code below in the hydro_driver where I suggest to use a problem related callback function for the reduction).

If you don't object I'm happy to move stuff around.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't object, but I don't think I fully understand -- do you mean create a new callback function interface for this?

src/hydro/hydro.cpp Outdated Show resolved Hide resolved
src/hydro/hydro_driver.cpp Outdated Show resolved Hide resolved
Comment on lines +98 to +99
const cooling::TabularCooling &tabular_cooling =
pkg->Param<cooling::TabularCooling>("tabular_cooling");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this function will only work for tabular cooling, I'd move the logic upwards (i.e., where you already check for none) and make sure it's only working in that case (and otherwise return/fail).
I'm thinking of future us' here who might implemented other cooling methods and assume that this works silently.

src/hydro/srcterms/tabular_cooling.hpp Show resolved Hide resolved
src/hydro/hydro.cpp Outdated Show resolved Hide resolved
src/hydro/hydro_driver.cpp Outdated Show resolved Hide resolved
src/hydro/hydro_driver.cpp Outdated Show resolved Hide resolved
src/hydro/srcterms/tabular_cooling.cpp Outdated Show resolved Hide resolved
@pgrete
Copy link
Contributor

pgrete commented Mar 15, 2023

Have you tried updating the tasks as discussed on Matrix?
I don't quite see how just calculating some numbers/and or mixing tasks up, end ups messing around with the values in the actual data views.

What's the input file you used for the render above? Is that the cloud one?

@BenWibking
Copy link
Contributor Author

BenWibking commented Mar 15, 2023

Have you tried updating the tasks as discussed on Matrix? I don't quite see how just calculating some numbers/and or mixing tasks up, end ups messing around with the values in the actual data views.

Testing this now.
Update: does not fix the issue.

What's the input file you used for the render above? Is that the cloud one?

Ah yes, it's the cloud input file.

@BenWibking
Copy link
Contributor Author

Ok, so the output issue exists also on the main branch, at least for the cloud problem.

@BenWibking
Copy link
Contributor Author

BenWibking commented Mar 16, 2023

With the changes you suggested on the Matrix chat, the outputs for test problems run with this branch look fine in yt.

@BenWibking
Copy link
Contributor Author

BenWibking commented Mar 17, 2023

The MPI tests appear to be failing due to a memory issue. Might be fixed by parthenon-hpc-lab/parthenon#841.

@BenWibking
Copy link
Contributor Author

I've found a cleaner way to implement this that doesn't require changing the hydro driver, so I'm closing this PR for now.

@BenWibking BenWibking closed this May 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants