Appending to an existing dataset

[njchim]

I am looking for documentation or an example on how to append data to an existing dataset. I looked through the samples and didn't find anything that seemed to make sense. I think "chunking" is maybe what I want but haven't found a good explanation of that either. I have defined a compound type for the data type I am looking to store and am able to write a static size number of records. But how would I be able to save new records as they are created on the fly? Thanks in advance!

[1uc]

There's the following example which I hope will help you or others:
https://github.com/BlueBrain/HighFive/blob/master/src/examples/create_extensible_dataset.cpp

Please let us know if there's something missing from the example.

Chunking gives HDF5 the permission to store the array in chunks, e.g. elements 0, ..., 15, 16, .., 31, etc. will be stored in contiguous chunks. (The chunk size is configurable; and 32 is likely not a good choice, too small.) Only once the need to store the entire array contiguously is lifted, can one imagine increasing the size of the array on disk. Conceptually it might be something like: add new chunks to the end of the file and update some internal datastructure that allows HDF5 to assemble the array from the chunks when read into (contiguous) memory. However, us users of HDF5 don't need to know the details of where it stores chunks and how the metadata is handled. The API is simply to resize the dataset and write to the dataset using selections or hyperslabs.