You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
The DeltaBitPackEncoder has a couple of parameters that users may wish to override, in particular:
Block size - the number of values per block (default: 128)
Miniblock count - the number of miniblocks per block (default: 4)
These defaults result in 32 elements per miniblock, the smallest possible. Small miniblocks may lead to smaller representation as the bit width is per miniblock, similarly small blocks may lead to smaller representation as the min delta is per block. However, for input data with a uniform distribution, larger blocks may yield better compression and faster decode.
See here for more information on what these parameters control
Describe the solution you'd like
ColumnProperties currently allows setting Encoding for a specific column, we should provide the ability to control the settings passed to the DeltaBitPackEncoder.
Describe alternatives you've considered
Additional context
#2276 adds vectorised unpacking of runs of 64x 64-bit numbers, however, this would need miniblocks containing 64 elements to benefit from this
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
The DeltaBitPackEncoder has a couple of parameters that users may wish to override, in particular:
These defaults result in 32 elements per miniblock, the smallest possible. Small miniblocks may lead to smaller representation as the bit width is per miniblock, similarly small blocks may lead to smaller representation as the min delta is per block. However, for input data with a uniform distribution, larger blocks may yield better compression and faster decode.
See here for more information on what these parameters control
Describe the solution you'd like
ColumnProperties
currently allows settingEncoding
for a specific column, we should provide the ability to control the settings passed to the DeltaBitPackEncoder.Describe alternatives you've considered
Additional context
#2276 adds vectorised unpacking of runs of 64x 64-bit numbers, however, this would need miniblocks containing 64 elements to benefit from this
The text was updated successfully, but these errors were encountered: