Read whole leaf-level arrays at once when possible, rather than growing lists

[adamreeve]

This PR changes how array values are read to avoid growing lists dynamically, and instead create arrays directly from spans of the buffered logical values.

This is based on the work done by @philjdf in philjdf#4 and philjdf#5. The new benchmark has mostly been copied as-is, but the array reading changes have been adapted to work with the latest master code.

On my machine (AMD Ryzen 9 5900 X with 64 GB RAM running Fedora Linux 36 with dotnet SDK 6.0.107), I get the following benchmark results:

Test case	Time taken (s)	Time taken with server GC (s)
Parquet.Net	10.8	10.1
ParquetSharp 7.0	20.4	20.9
ParquetSharp 8.0 beta	12.5	10.9
ParquetSharp with this PR	4.5	2.6

The speed up from ParquetSharp 7.0 to 8.0 looks to be due to changes in Arrow rather than any of the changes we've made in ParquetSharp. I've also included benchmark results when enabling the dotnet server GC as this can improve performance with ParquetSharp significantly.

[philjdf]

Nice speed improvement, thanks Adam!

[marcin-krystianc]

Nice improvement. See my comment, maybe it is possible to make it even better?

[marcin-krystianc]

I've experimented with some further optimisations, but I didn't see an actual improvement.
The most promising approach is https://github.com/G-Research/ParquetSharp/compare/master...marcin-krystianc:ParquetSharp:marcink-20220810-arraypool?expand=1&w=1 where I used ArrayPool to reduce allocations for temporary arrays. Unfortunately in benchmark it shows no actual improvement, I think it is mainly because temporary arrays are not used that often as most of the times there is a single chunk being created and returned.