feat: offset indices in sparse tensor

[itzhakstern]

Change the indices of sparse tensors from positions of non-zero elements to offsets.

For example, if we have a tensor:

[0, 29, 0, 0, 17, 6]

its sparse representation would be:

• values: 29,17,6
• indices: 1,4,5

Instead, we want to store the differences (offsets) between consecutive indices:

• values: 29,17,6
• indices: 1,3,1

This approach seems to significantly reduce the size of compressed Parquet files, as the indices diffs contain repetitive patterns, which compress well.

[codspeed-hq]

CodSpeed Performance Report

Merging #3725 will improve performances by 82.23%

_{Comparing itzhakstern:itzhaks/sparse-tensors-indices-to-offsets (5629ece) with main (86adc44)}

Summary

⚡ 2 improvements
✅ 25 untouched benchmarks

Benchmarks breakdown

	Benchmark	BASE	HEAD	Change
⚡	test_count[1 Small File]	3.7 ms	3.4 ms	+10%
⚡	test_iter_rows_first_row[100 Small Files]	342.5 ms	188 ms	+82.23%

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 77.86%. Comparing base (6deb87e) to head (5629ece).
Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3725      +/-   ##
==========================================
+ Coverage   77.31%   77.86%   +0.54%     
==========================================
  Files         734      735       +1     
  Lines       93737    93128     -609     
==========================================
+ Hits        72475    72513      +38     
+ Misses      21262    20615     -647     

Files with missing lines	Coverage Δ
src/daft-core/src/array/ops/cast.rs	88.60% <100.00%> (+0.16%)	⬆️

... and 20 files with indirect coverage changes

[raunakab]

I'm still combing through the PR, but just have a minor nit that I'd like to get out of the way first.

[raunakab]

Minor nit: typo here. ofssets_indices_arr should be offsets_indices_arr.

[raunakab]

@itzhakstern I know the PR is still in draft, but if you don't mind, could you please add a description on what this PR is trying to achieve?

I can gather that it's enabling registering index offsets in sparse tensors, but having a succinct description would be helpful for readability into the PR for other reviewers as well. Thanks in advance!