Improve testing infrastructure.

[1uc]

Adds/reimplements abstractions for the following:

• Create multi-dimensional array filled with suitable values.
• Traits for accessing values.
• Traits for hiding the difference of DataSet and Attribute.
• Useful utilities such as ravel, unravel and flat_size.

[codecov]

Codecov Report

Attention: 16 lines in your changes are missing coverage. Please review.

Comparison is base (30a9d14) 84.97% compared to head (586310f) 85.62%.

Files	Patch %	Lines
tests/unit/test_all_types.cpp	90.69%	12 Missing ⚠️
tests/unit/data_generator.hpp	97.76%	4 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #871      +/-   ##
==========================================
+ Coverage   84.97%   85.62%   +0.65%     
==========================================
  Files          71       73       +2     
  Lines        5118     5461     +343     
==========================================
+ Hits         4349     4676     +327     
- Misses        769      785      +16     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[tdegeus]

I would use a library for this! For example, you can use the NumPy equivalent xtensor. Writing and maintaining some extra custom code is extra work and error-prone. I would say that testable data are as simple as

xt::xarray<int> a = xt::arange<int>(5 * 4 * 3).reshape({5, 4, 3}); 

Then you can use functions like ravel, ravel_index, ... just like in NumPy. No new API, no new code.

[1uc]

I'd like to test the following:

template<class Container>
void check_write_dataset(const Container& container) {
  file.createDataSet("foo", container);
}

Hence, I need to able to create arrays of all supported types

std::vector<T>
std::vector<std::array<T, 4>>
std::array<std::vector<T>, 4>
boost::
// xt:: (once supported in core)

and so forth, and be able to fill them with exactly the correct values. Being able to have any container filled with sensible values, requires being able to a) allocate containers of a given shape, b) assign values. Those two feature make up a good chunk of the PR. Then there very minor niceties like copying, etc.

The second issue is that I need a somewhat uniform API for creating the product of template<class T> Container with a bunch of different scalar types T. That's the other large chunk of the PR.

How does an xt::array help in this situation? Yes, I can test XTensor, but I can't test writing of any of the other containers. If I use XTensor arrays to write, I can read the other containers, but I still can't check that the values are correct, because I don't have access to some API unifying trait to hide the difference of a[i][j], a(i,j) etc. (gets worse in 3D).

[tdegeus]

If there is a pressing reason to use a custom implementation, of course, it's fine. Anyway, this is not a public implementation, nor will change a lot.

I just thought it would be easier to write some casters but avoid implementing (un)ravel(index) functions and equality operators.

[tdegeus]

Following the command above:

auto x = a(1, 0);

there is also

auto y = a.flat(10)

to read directly from the flat storage.

(comment applicable below)

[tdegeus]

It's a good idea to streamline testing. However, as I stated in some comments, there are existing libraries for this, and I would strongly suggest not to seek a custom implementation. Using a library comes with the advantage that:

1. It does not imply extra code that has to be maintained.
2. It comes with a fixed API that is well-documented.
3. It comes with print functions.
4. The libraries are usually themselves intensively tested.

[1uc]

In defense of the complexity: it found several bugs/quirks or things to follow up on:

• Segfault for std::array<std::vector>. #875
• Review boost::multi_array for ordering. #874
• Fix create_datatype<bool>(). #869
• Improve error for boost::numeric::ublas::matrix<std::string>. #879

It would trivially fix:

• Failure to build unit-test with half. #895

Any non-uniformity is very painful to us (not just our users), and therefore has a higher chance of getting noticed and fixed. The following come to mind:

• Datatype of char[][N]. #865
• Unify read(T*) and write_raw(T*). #870

[ferdonline]

I think it's a nice idea to unify Dataset and Property testing. You also added tests convering that, so I really don't have anything to nitpick about. If you want to wait for other reviews, great, but from my side good to go