How to load data into a std::vector

[pf4d]

Hi there,

I know this may not be the right place to ask about using the software, but there does not exist a netcdf-cxx support group.

I'm trying to understand the easiest way to load data into a std::vector. The dimensions of the data are obtained using NcVar::getDim, but I cannot create C++ variable-length array; thus I would like to load the data into a std::vector.

I see here https://www2.nrel.colostate.edu/projects/irc/public/Documents/Software/netCDF/cpp4/html/classnetcdf_1_1NcVar.html#a15362861cb578da74756927805d7d46e
that some version of the software had a NcVar::Get method that seemed to to the trick, but the lastest version does not have this method.

Can anyone explain what the process would be to do this?

Thank you very much!

[ZedThree]

You can use std::vector::data to get a pointer to the underlying storage:

// Create a vector of type T with the appropriate size reserved.
std::vector<T> value(dims[0].getSize());

// Assuming `var` is your `NcVar`
var.getVar(value.data());

You could even wrap this up in a helper template function to have something that feels more like idiomatic C++:

/// Return variable data as a flat 1D array.
template <typename T>
std::vector<T> get_variable(const NcVar& var) {
  auto dims = var.getDims();
  // Get the product of all the dimension sizes
  auto total_size = std::accumulate(dims.begin(), dims.end(), 1, 
                                    [](const NcDim& dim, std::size_t size) {
                                        return dim.getSize() * size;
                                    });
  std::vector<T> values(total_size);
  var.getVar(values.data());
  return values;
}

// To call our helper, we need to specify the type.
// Note that this does no checking you've given the correct type!
auto data = get_variable<double>(var);

This is just an example that I've not checked actually compiles. For real production use, you might not want a flat 1D array, and you almost certainly want to do some error checking, especially of the types!

[pf4d]

Thank you for the quick answer. Do you know how this could be done with two-dimensional data (a vector<vector<T>>)? I assume I can pull each row one at a time from the NcVar using one of the NcVar::getVar methods and populate each vector with those elements.

Ideally I would like to put each row from the NcVar into a single "flat" vector.

Thanks again!

[pf4d]

Wow, that was a silly question, you already answered it. I never looked at lambda functions, it took a minute to look though your code to realize that var::getVar loaded the data as flattened (meaning I suppose that netCDF-4 stores the data as flat arrays?).

Thank you.

[ZedThree]

NcVar::getVar has an overload set that takes start and count vectors which you can use to pull out some N-D chunk from the file. The version I showed does indeed just read the whole N-D variable as a flat array. How they're actually stored on disk is a bit more complicated, but luckily we don't need to care about that.

In our code, we actually have a set of classes that wrap 2D and 3D arrays, storing them internally as 1D, and providing indexing operators for access. This is much better for data locality than using vector<vector<T>> for example.