feat: compact segment

[dantengsky]

I hereby agree to the terms of the CLA available at: https://databend.rs/dev/policies/cla/

Summary

• introduce optimize table compact segment statement:
  optimize table t compact SEGMENT
• which compacts segments according to the table option block_per_segment , default value is 1000 (blocks per segment)
• resolves conflicts with append only operations in a way of OOC retry (retry without backoff, 10 times at most)
  the idea of conflict detection is stolen from PR fix(storage): support mutation during insertion #8205, originating from @zhyass

NOTE:

To increase the possibility of success committing (of compact operation), the compaction is done in the target_select phase, and if concurrent resolvable conflicts(append-only txs) are detected during the commit phase, the newly appended segments will NOT be compacted again during retries, just merged with the compaction (and recalculates the statistics). the non-compacted newly appended segments are hopefully being compacted in the subsequence optimize compact segment operations.

Fixes #8247

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Updated
databend	⬜️ Ignored (Inspect)		Oct 19, 2022 at 8:11AM (UTC)

[zhyass]

LGTM