[exporter/clickhouse] Update default logs table schema (2)

[SpencerTorres]

Description:

Previously updated in #33611, I am opening this to start a discussion on further improvements that can be made to the table.

Notable changes:

• Changed from monthly partitions to daily. With ttl_only_drop_parts=1, this will help drop data for TTLs shorter than 1 month (such as when your log retention is only 7 days).
• Changed idx_body granularity to 8, which should reduce the index size (especially beneficial for cloud services with separate storage)
• Removed TimestampDate column
• Simplified primary key to only use TimestampTime. Performance difference is negligible if not better. Also makes queries easier to write-- with the current version it requires that you provide both TimestampDate and TimestampTime for optimal sorting performance.
• Separated and updated order by. Now it matches the primary key, with the addition of Timestamp, so that nanoseconds sorting is preserved by default.

Let me know if you have any more suggestions.

Link to tracking Issue:

Testing:

Documentation:

[crobert-1]

Question out of ignorance: How does changing DB schema impact upgrade? Will this receiver still work on a DB that was created with the old schema? Are there any tests to ensure functionality works as expected with this?

[SpencerTorres]

How does changing DB schema impact upgrade? Will this receiver still work on a DB that was created with the old schema? Are there any tests to ensure functionality works as expected with this?

Yep, it's compatible with the old schema as none of the column names have changed. We have some integration tests built in. I believe these are disabled for now due to inconsistencies in the CI, but they can still be run manually to confirm everything works the same.

The intent of changing the schema in this way is to improve performance. In production deployments, it's recommended that users tune the schema to match their query patterns. The logs table we provide here is intended to be a good default.