Removal of sequence number

[antekresic]

Sequence numbers were an optimization for ordering batches based on the orderby configuration setting. It was used for ordered append and avoiding sorting compressed data when it matched the query ordering. However, with enabling changes to compressed data, bookkeeping of sequence numbers is becoming more of a hassle. Removing them and using the metadata columns for ordering reduces that burden while keeping all the existing optimizations that relied on the sequences in place.

This change does not include downgrade script which will be added in a followup PR.

There are slight planning regressions but otherwise benchmark looks OK:
https://grafana.ops.savannah-dev.timescale.com/d/fasYic_4z/compare-akuzm?orgId=1&var-branch=All&var-run1=3725&var-run2=3735&var-threshold=0&var-use_historical_thresholds=true&var-threshold_expression=2.5%20%2A%20percentile_cont%280.90%29&var-exact_suite_version=false

Disable-check: force-changelog-file

[akuzm]

Some questions:

• Can you do backwards scan using a compressed index scan? Technically the required order on compressed data is segmentby DESC, max_orderby1 DESC, min_orderby1 DESC, and it cannot be satisfied with an index on segmentby, min_orderby1, max_orderby1.

• We're going to have to support reading sequence numbers for a long time, not to make people recompress all the tables, right? I wonder how we should best test it, maybe we should keep the option to create chunks with sequence numbers for now, and use it in a dedicated test. I think there's a good chance we break something on a timeframe of e.g. a year.

[erimatnor]

Just a few nits so far.

What is the plan for update/downgrade scripts? Do we need to block downgrades somehow, so that people can't downgrade if they have the new format?

[erimatnor]

should there be an explicit DESC also in the new sort key? Or should it be ASC?

[antekresic]

Previous sort was a DESC becuase compression is set to "time DESC" so to get ASC ordering, you needed reverse order on the sequence number. New ordering should always mirror the column ordering because we are using actual values instead of a sequence which can have its own ordering.

[antekresic]

@erimatnor added the missing downgrade script check. Please take a look when you can.