Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add analysis task for waves #869

Merged
merged 26 commits into from
Dec 13, 2022
Merged

Add analysis task for waves #869

merged 26 commits into from
Dec 13, 2022

Conversation

sbrus89
Copy link
Contributor

@sbrus89 sbrus89 commented Apr 1, 2022

This adds climatology and regional analyses for E3SM runs with WAVEWATCH III.

Checklist (for @xylar) before merging:

  • copy obs from /lcrc/group/e3sm/diagnostics/observations/Ocean/* to /lcrc/group/e3sm/public_html/diagnostics/observations/Ocean/
  • generate README from obs.xml and copy to /lcrc/group/e3sm/public_html/diagnostics/observations/Ocean/*
  • sync obs to
    • Compy
    • Cori

@pep8speaks
Copy link

Hello @sbrus89! Thanks for opening this PR. We checked the lines you've touched for PEP 8 issues, and found:

Line 82:19: E265 block comment should start with '# '
Line 86:20: E124 closing bracket does not match visual indentation
Line 115:59: W291 trailing whitespace
Line 118:1: W293 blank line contains whitespace
Line 120:9: E303 too many blank lines (2)
Line 121:13: E265 block comment should start with '# '
Line 129:35: W291 trailing whitespace
Line 133:9: E303 too many blank lines (2)
Line 135:9: E265 block comment should start with '# '
Line 146:9: E303 too many blank lines (2)
Line 156:37: W291 trailing whitespace
Line 157:38: W291 trailing whitespace
Line 161:81: E501 line too long (83 > 80 characters)
Line 175:41: W291 trailing whitespace
Line 183:21: E303 too many blank lines (2)
Line 192:25: E265 block comment should start with '# '
Line 195:25: E265 block comment should start with '# '
Line 204:8: E111 indentation is not a multiple of four
Line 207:8: E114 indentation is not a multiple of four (comment)
Line 208:8: E114 indentation is not a multiple of four (comment)
Line 209:8: E114 indentation is not a multiple of four (comment)
Line 210:8: E114 indentation is not a multiple of four (comment)
Line 212:8: E114 indentation is not a multiple of four (comment)
Line 213:8: E114 indentation is not a multiple of four (comment)
Line 214:8: E111 indentation is not a multiple of four
Line 216:8: E111 indentation is not a multiple of four
Line 217:1: W293 blank line contains whitespace
Line 218:12: E111 indentation is not a multiple of four
Line 227:62: E261 at least two spaces before inline comment
Line 230:5: E303 too many blank lines (2)
Line 232:81: E501 line too long (111 > 80 characters)
Line 242:1: E303 too many blank lines (4)
Line 264:30: W291 trailing whitespace
Line 297:9: E265 block comment should start with '# '
Line 305:55: W291 trailing whitespace
Line 316:9: E265 block comment should start with '# '
Line 318:9: E265 block comment should start with '# '
Line 321:9: E265 block comment should start with '# '
Line 371:81: E501 line too long (84 > 80 characters)
Line 379:81: E501 line too long (82 > 80 characters)
Line 396:17: E303 too many blank lines (2)
Line 435:1: W293 blank line contains whitespace
Line 450:9: E266 too many leading '#' for block comment
Line 450:31: W291 trailing whitespace
Line 452:43: E231 missing whitespace after ','
Line 453:45: E231 missing whitespace after ','
Line 454:49: E231 missing whitespace after ','
Line 455:47: E231 missing whitespace after ','
Line 457:30: E231 missing whitespace after ':'
Line 457:33: E231 missing whitespace after ','
Line 457:35: E231 missing whitespace after ','
Line 457:37: E231 missing whitespace after ','
Line 457:39: E231 missing whitespace after ','
Line 457:41: E231 missing whitespace after ','
Line 457:43: E231 missing whitespace after ','
Line 457:45: E231 missing whitespace after ','
Line 457:47: E231 missing whitespace after ','
Line 457:49: E231 missing whitespace after ','
Line 457:52: E231 missing whitespace after ','
Line 457:55: E231 missing whitespace after ','
Line 457:60: W291 trailing whitespace
Line 458:30: E231 missing whitespace after ':'
Line 458:33: E231 missing whitespace after ','
Line 458:35: E231 missing whitespace after ','
Line 459:30: E231 missing whitespace after ':'
Line 459:33: E231 missing whitespace after ','
Line 459:35: E231 missing whitespace after ','
Line 463:81: E501 line too long (91 > 80 characters)
Line 465:36: E231 missing whitespace after ','
Line 467:81: E501 line too long (95 > 80 characters)
Line 468:1: W293 blank line contains whitespace
Line 470:9: E303 too many blank lines (2)
Line 478:1: W293 blank line contains whitespace
Line 481:1: W293 blank line contains whitespace
Line 488:1: W293 blank line contains whitespace
Line 500:29: E225 missing whitespace around operator
Line 500:34: E231 missing whitespace after ','
Line 501:22: W291 trailing whitespace
Line 503:29: E225 missing whitespace around operator
Line 503:34: E231 missing whitespace after ','
Line 507:1: W293 blank line contains whitespace
Line 509:39: E231 missing whitespace after ','
Line 509:41: E231 missing whitespace after ','
Line 509:44: W291 trailing whitespace
Line 511:31: E231 missing whitespace after ','
Line 511:46: E231 missing whitespace after ','
Line 511:54: E231 missing whitespace after ','
Line 512:81: E501 line too long (86 > 80 characters)
Line 515:81: E501 line too long (116 > 80 characters)
Line 515:104: E231 missing whitespace after ','
Line 516:81: E501 line too long (117 > 80 characters)
Line 516:101: E231 missing whitespace after ','
Line 516:113: E231 missing whitespace after ','
Line 520:22: E231 missing whitespace after ','
Line 520:24: E231 missing whitespace after ','
Line 520:41: E231 missing whitespace after ','
Line 520:48: E231 missing whitespace after ','
Line 521:20: E111 indentation is not a multiple of four
Line 521:49: E231 missing whitespace after ','
Line 521:51: E231 missing whitespace after ','
Line 524:19: W291 trailing whitespace
Line 528:2: E114 indentation is not a multiple of four (comment)
Line 528:2: E303 too many blank lines (3)

@sbrus89
Copy link
Contributor Author

sbrus89 commented Apr 1, 2022

@xylar, we should talk about how to clean this up before merging, but I wanted to get a PR started so I don't forget about it.

@sbrus89 sbrus89 requested a review from xylar April 1, 2022 14:04
Copy link
Collaborator

@xylar xylar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sbrus89, this is really great! I'm excited to get this analysis in.

I hope my comments aren't too overwhelming or confusing but we can discuss them in a bit when we have our usual call.

mpas_analysis/default.cfg Outdated Show resolved Hide resolved
mpas_analysis/default.cfg Outdated Show resolved Hide resolved
mpas_analysis/default.cfg Outdated Show resolved Hide resolved
mpas_analysis/default.cfg Outdated Show resolved Hide resolved
mpas_analysis/ocean/climatology_map_waves.py Outdated Show resolved Hide resolved
Comment on lines 498 to 503
if field['mpas'] == 'timeMonthly_avg_peakWaveFrequency':
maskedVar = (cellMasks*(1.0/ds[field['prefix']])).compute()
varRange=(0.0,15.0)
else:
maskedVar = (cellMasks*ds[field['prefix']]).compute()
varRange=(0.0,10.0)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

varRange shouldn't be hard coded here. It should come from a config option.

field['prefix'] is not the variable you want with the changes I suggested above:

Suggested change
if field['mpas'] == 'timeMonthly_avg_peakWaveFrequency':
maskedVar = (cellMasks*(1.0/ds[field['prefix']])).compute()
varRange=(0.0,15.0)
else:
maskedVar = (cellMasks*ds[field['prefix']]).compute()
varRange=(0.0,10.0)
var = ds[mpas]
if mpas == 'peakWaveFrequency':
var = 1.0/var
varRange=(0.0,15.0)
else:
varRange=(0.0,10.0)
maskedVar = (cellMasks*var).compute()

varRange=(0.0,10.0)

for index, regionName in enumerate(regionNames):
print(' '+regionName)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
print(' '+regionName)
print(f' {regionName}')

mpas_analysis/ocean/climatology_map_waves.py Outdated Show resolved Hide resolved
mpas_analysis/ocean/climatology_map_waves.py Outdated Show resolved Hide resolved
Comment on lines 515 to 713
plt.savefig(outDirectory+'/'+field['prefix']+'_'+self.season+'_'+regionName.replace(' ','_')+'.png')
f = open(outDirectory+'/'+field['prefix']+'_'+self.season+'_'+regionName.replace(' ','_')+'.txt','w')
heights = [patch.get_height() for patch in ax.patches]
widths = [patch.get_width() for patch in ax.patches]
xloc = [patch.get_xy()[0] for patch in ax.patches]
for h,w,x in zip(heights,widths,xloc):
f.write('{} {} {}\n'.format(x,w,h))
f.close()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
plt.savefig(outDirectory+'/'+field['prefix']+'_'+self.season+'_'+regionName.replace(' ','_')+'.png')
f = open(outDirectory+'/'+field['prefix']+'_'+self.season+'_'+regionName.replace(' ','_')+'.txt','w')
heights = [patch.get_height() for patch in ax.patches]
widths = [patch.get_width() for patch in ax.patches]
xloc = [patch.get_xy()[0] for patch in ax.patches]
for h,w,x in zip(heights,widths,xloc):
f.write('{} {} {}\n'.format(x,w,h))
f.close()
regionSuffix = regionName.replace(' ','_')
filePrefix = os.path.join(
outDirectory, f'{prefix}_{self.season}_{regionSuffix}')
plt.savefig(f'{filePrefix}.png')
with open(f'{filePrefix}.txt', 'w') as f:
heights = [patch.get_height() for patch in ax.patches]
widths = [patch.get_width() for patch in ax.patches]
xloc = [patch.get_xy()[0] for patch in ax.patches]
for h, w, x in zip(heights, widths, xloc):
f.write(f'{x} {w} {h}\n')

@cbegeman
Copy link
Collaborator

cbegeman commented Oct 3, 2022

Noting that the histogram plot type, and possibly the ocean histogram task, could be leveraged by PR #917.

@sbrus89 Let me know if you have any questions about how this functionality works, or if there is something I can add that would make it possible to make the kind of plots that you want.

@xylar
Copy link
Collaborator

xylar commented Nov 11, 2022

@sbrus89, it looks like you rebased onto master, not develop?

@sbrus89
Copy link
Contributor Author

sbrus89 commented Nov 11, 2022

@xylar, yes. thanks for catching that. Force of habit, I guess.

@sbrus89
Copy link
Contributor Author

sbrus89 commented Nov 11, 2022

@xylar - I hope this is getting closer, but I'm sure it needs another round of suggestions from you when you have a chance. If you want to test, I have a config file here: /lcrc/group/acme/sbrus/scratch/anvil/20221027.submeso.langmuir.momentum.piControl.ne30pg2_EC30to60E2r2_wQU225EC30to60E2r2.anvil/config.waves_50yrs

Here's an example of what this should produce for generate = ['only_waves']:
Screen Shot 2022-11-11 at 5 25 56 PM

@xylar
Copy link
Collaborator

xylar commented Nov 23, 2022

Yay, with my latest suggestions, I'm no longer seeing seams in the Arctic/Antarctic observational plots:
https://web.lcrc.anl.gov/public/e3sm/diagnostic_output/ac.xasay-davis/analysis/waves/20221027.submeso.langmuir.momentum.piControl.ne30pg2_EC30to60E2r2_wQU225EC30to60E2r2.anvil/yrs_31-50/ocean/index.html

@sbrus89
Copy link
Contributor Author

sbrus89 commented Nov 23, 2022

@xylar, I incorporated all your suggestions and tested again. The output is here:

https://web.lcrc.anl.gov/public/e3sm/diagnostic_output/sbrus/langmuir_momentum_Bcase/html_waves_100yrs/ocean/index.html#global_waves

Copy link
Collaborator

@xylar xylar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks fantastic! I ran a test with just waves and things look great this time. Thanks for making the changes I requested.

I'm going to run my usual test suite. I won't merge just yet as we discussed because this won't go in until the next version (1.9.0).

@xylar
Copy link
Collaborator

xylar commented Nov 24, 2022

@sbrus89, one thing I forgot is that you need to add some documentation. Could you find a similar analysis tasks in the docs and make a copy? The observations also need to be documented. That's a process we should talk through.

@sbrus89
Copy link
Contributor Author

sbrus89 commented Nov 24, 2022

@xylar, I was going to ask about that. Yes, I'll take care of that documentation for the analysis task and follow up with you on the observations.

Copy link
Collaborator

@xylar xylar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm seeing failures in main vs. control analysis. The issue is that the gallery title includes the obs_type, but that isn't defined when you aren't comparing with obs.

It also might not make sense to have galleries for both sets of obs (and the names of the gallery groups may also not make sense) when you are doing a main vs. control run since the obs aren't included in the plots.

mpas_analysis/ocean/climatology_map_waves.py Show resolved Hide resolved
mpas_analysis/ocean/climatology_map_waves.py Outdated Show resolved Hide resolved
mpas_analysis/ocean/climatology_map_waves.py Outdated Show resolved Hide resolved
@xylar
Copy link
Collaborator

xylar commented Nov 24, 2022

The situation with the obs isn't actually all that complicated. You need to fill in something in this XML file:
https://github.com/MPAS-Dev/MPAS-Analysis/blob/develop/mpas_analysis/obs/observational_datasets.xml
Just pick an existing one as an example and ask if you have questions about what goes there.

Then, we need to run the python script there to generate the readme and the bibtex file that we'll put in the WAVES directory next to the obs files. Some documentation will also get generated automatically from the xml file. You should refer to that in your documentation for the analysis task. Again, find an example of another analysis task with obs to use as a template.

@xylar
Copy link
Collaborator

xylar commented Dec 13, 2022

@sbrus89, could you please change the ERA5 and SSCCI directories and their contents in /lcrc/group/e3sm/diagnostics/observations/Ocean so they have group write permission? Thanks!

@xylar
Copy link
Collaborator

xylar commented Dec 13, 2022

Retesting looks good. I reran the same analysis I had done before and the changes look fine. I also ran the test suite again and everything except the docs worked as expected, and the docs were fine when I ran them manually so I think it's a fluke.

@xylar
Copy link
Collaborator

xylar commented Dec 13, 2022

Docs are also fine in CI here.

@xylar xylar merged commit b30abfa into MPAS-Dev:develop Dec 13, 2022
@xylar
Copy link
Collaborator

xylar commented Dec 13, 2022

Excellent work on this, @sbrus89!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants