Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

C48 fails on WCOSS2 #1129

Closed
aerorahul opened this issue Nov 22, 2022 · 1 comment · Fixed by #1131
Closed

C48 fails on WCOSS2 #1129

aerorahul opened this issue Nov 22, 2022 · 1 comment · Fixed by #1131
Assignees
@aerorahul
Labels
bug Something isn't working

Comments

@aerorahul
Copy link
Contributor

Expected behavior
C48 should run on all platforms.

Current behavior
C48 is currently the ensemble resolution test case and fails on WCOSS2. (All) members successfully complete integration on Hera and Orion.

Machines affected
WCOSS2

To Reproduce
Try and run a cycled experiment at C96/C48 on WCOSS2.

Detailed Description
It is found that the number of threads is set to 2 for C48 and that seems to be causing the failures on WCOSS2.

Possible Implementation
Reduce number of threads for C48 to 1 for all platforms unless someone from the modeling teams can address the threading issue. Running w/ 1 thread should allow development and testing of the global-workflow.
@aerorahul aerorahul added the bug Something isn't working label Nov 22, 2022
@aerorahul aerorahul self-assigned this Nov 22, 2022
@aerorahul aerorahul mentioned this issue Nov 22, 2022
Merged
4 tasks
@WalterKolczynski-NOAA
Copy link
Contributor

Is there an issue open with UFS to fix the underlying problem?

WalterKolczynski-NOAA pushed a commit that referenced this issue Nov 22, 2022
@aerorahul
Update C48 forecast to run with one thread (#1131)
3a9c27a 
C48 forecasts currently fail on WCOSS2 when running with two threads. Two threads are likely not really needed, so the forecast job is reduced to single-threaded on all platforms, at least until the problem can be corrected.

Also moves the block capping the write block size to the node size outside the resolution-specific block, since that is needed for all resolutions.

Fixes #1129
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@aerorahul aerorahul

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Update C48 forecast to run with one thread NOAA-EMC/global-workflow
2 participants
@aerorahul @WalterKolczynski-NOAA