Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up wave.resource module #352

Open
wants to merge 33 commits into
base: develop
Choose a base branch
from

Conversation

akeeste
Copy link
Contributor

@akeeste akeeste commented Sep 17, 2024

@ssolson This is a follow-up to my other wave PRs and resolves #331. Handling the various edge cases robustly in pure numpy is difficult, so I want to first resolve #331 by using DataArrays throughout the wave resource functions instead of Datasets.

Similar to Ryan's testing mentioned in #331, I found that using DataArrays/Pandas has a 1000x speed up vs Datasets for very large input data. This should restore MHKiT's speed to it's previous state. Using a pure numpy base would have an additional 5-10x speed up from DataArrays, but I think the current work with DataArrays will:

  • be sufficient for our users
  • be easier to develop with
  • be easier to handle edge cases

@akeeste akeeste marked this pull request as ready for review October 2, 2024 18:34
@akeeste akeeste marked this pull request as draft October 2, 2024 18:44
@akeeste akeeste marked this pull request as ready for review October 10, 2024 17:24
@akeeste
Copy link
Contributor Author

akeeste commented Oct 10, 2024

@ssolson This PR is ready for review. Tests should pass now

With some modifications to the type handling functions, and an appropriate frequency_dimension input if required, the wave.resource functions should handle Pandas Series, Pandas DataFrames, and xarray DataArrays regardless of input shape, dimensions names, dimension order, etc. I largely moved away from converting data to xarraay Datasets because they are slow and more difficult to work with.

@akeeste
Copy link
Contributor Author

akeeste commented Oct 10, 2024

mhkit.loads.graphics is unchanged, so I'm not sure why the pylint loads test is now failing on the number of positional arguments there. This branch is up to date with develop

@ssolson
Copy link
Contributor

ssolson commented Oct 14, 2024

mhkit.loads.graphics is unchanged, so I'm not sure why the pylint loads test is now failing on the number of positional arguments there. This branch is up to date with develop

Pylint added new warnings around the way it wants users to handle positional arguments. I'll address it.

@ssolson
Copy link
Contributor

ssolson commented Oct 14, 2024

mhkit.loads.graphics is unchanged, so I'm not sure why the pylint loads test is now failing on the number of positional arguments there. This branch is up to date with develop

Pylint added new warnings around the way it wants users to handle positional arguments. I'll address it.

Addressed in #357

Let's merge #357 then make sure the tests pass here.

@akeeste
Copy link
Contributor Author

akeeste commented Oct 14, 2024

Thanks @ssolson. I'll merge that in here and fix a couple minor items with some examples

@akeeste
Copy link
Contributor Author

akeeste commented Oct 14, 2024

@akeeste TODO:

  • fix a couple last issues with example notebooks
  • reduce time required for example notebooks to enforce speed back to previous benchmark

@akeeste
Copy link
Contributor Author

akeeste commented Oct 14, 2024

@ssolson this PR is now ready for review and all tests are passing. I tightened up the timing on the environmental contours, 3 extreme response, and PacWave examples.

A straight forward test case on the difference in computational expense is using a wave resource function (e.g. energy_period) with a year of NDBC spectral data, or repeating Ryan's script in #331

@ssolson ssolson added wave module Clean Up Improve code consistency and readability labels Oct 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Clean Up Improve code consistency and readability wave module
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants