-
Notifications
You must be signed in to change notification settings - Fork 262
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add advanced HPC functionality based on PIO library #720
Add advanced HPC functionality based on PIO library #720
Conversation
Merge of changes in Unidata netcdf-c
Is there any chance of getting a design document? |
Yes, more documentation is coming. Now that I've got everything working, I'll get it documented. ;-) |
OK, there are now two .md files in the libpio directory: using_pio.md for a user-level overview, and developer_docs.md for developer level documentation. These documents will develop over the coming days. |
@DennisHeimbigner take a look at the two docs I added, and most importantly the example examples/C/pio_simple_async.c. This example shows the simplest async arrangement with 1 task doing all IO for 1 computation component, which consists of 3 tasks. The computation component then writes a simple file with one variable, using distributed arrays. Note that, due to the simplicity of this example the decomposition is created in the midst of the netCDF code that creates the file. In general the decomposition will be created before any files are created, and will be used for many files. |
I am going to take this PR down for a while. |
This PR contains the changes needed to get PIO working within the netCDF library.
This brings some very powerful new features for HPC. These features will be very helpful to weather modelers, climate modelers, and anyone else using netCDF on many-core supercomputers.
Documentation is good on a per-function level. Doxygen documentation of PIO is not enabled yet pending merging of another PR I have with many Doxygen changes. Once that PR is merged I will turn on PIO documentation in the Doxyfile.in file.
New functions are added to netcdf_par.h to support the new features. The PIO features are only available on builds which include both netCDF-4 and parallel.
Fixes #555.