-
Notifications
You must be signed in to change notification settings - Fork 2.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Remove support for Cassandra 3.x and add Cassandra 5.x #5962
Merged
yurishkuro
merged 6 commits into
jaegertracing:main
from
mahadzaryab1:deprecate-cassandra-3.x
Sep 10, 2024
Merged
Changes from 1 commit
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
d9e15e4
Deprecate Cassandra 3.x And Enable Test For 5.x
mahadzaryab1 3dc668a
Merge branch 'main' into deprecate-cassandra-3.x
mahadzaryab1 9366ac0
Add v5 To Dependabot And Renovate
mahadzaryab1 87626f5
Add docker-compose File For V5
mahadzaryab1 cc4120e
Remove Redundant Template File
mahadzaryab1 422d091
Fix Argument In Github Actions File
mahadzaryab1 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,222 @@ | ||
-- | ||
-- Creates Cassandra keyspace with tables for traces and dependencies. | ||
-- | ||
-- Required parameters: | ||
-- | ||
-- keyspace | ||
-- name of the keyspace | ||
-- replication | ||
-- replication strategy for the keyspace, such as | ||
-- for prod environments | ||
-- {'class': 'NetworkTopologyStrategy', '$datacenter': '${replication_factor}' } | ||
-- for test environments | ||
-- {'class': 'SimpleStrategy', 'replication_factor': '1'} | ||
-- trace_ttl | ||
-- default time to live for trace data, in seconds | ||
-- dependencies_ttl | ||
-- default time to live for dependencies data, in seconds (0 for no TTL) | ||
-- | ||
-- Non-configurable settings: | ||
-- gc_grace_seconds is non-zero, see: http://www.uberobert.com/cassandra_gc_grace_disables_hinted_handoff/ | ||
-- For TTL of 2 days, compaction window is 1 hour, rule of thumb here: http://thelastpickle.com/blog/2016/12/08/TWCS-part1.html | ||
|
||
CREATE KEYSPACE IF NOT EXISTS ${keyspace} WITH replication = ${replication}; | ||
|
||
CREATE TYPE IF NOT EXISTS ${keyspace}.keyvalue ( | ||
key text, | ||
value_type text, | ||
value_string text, | ||
value_bool boolean, | ||
value_long bigint, | ||
value_double double, | ||
value_binary blob | ||
); | ||
|
||
CREATE TYPE IF NOT EXISTS ${keyspace}.log ( | ||
ts bigint, -- microseconds since epoch | ||
fields frozen<list<frozen<${keyspace}.keyvalue>>> | ||
); | ||
|
||
CREATE TYPE IF NOT EXISTS ${keyspace}.span_ref ( | ||
ref_type text, | ||
trace_id blob, | ||
span_id bigint | ||
); | ||
|
||
CREATE TYPE IF NOT EXISTS ${keyspace}.process ( | ||
service_name text, | ||
tags frozen<list<frozen<${keyspace}.keyvalue>>> | ||
); | ||
|
||
-- Notice we have span_hash. This exists only for zipkin backwards compat. Zipkin allows spans with the same ID. | ||
-- Note: Cassandra re-orders non-PK columns alphabetically, so the table looks differently in CQLSH "describe table". | ||
-- start_time is bigint instead of timestamp as we require microsecond precision | ||
CREATE TABLE IF NOT EXISTS ${keyspace}.traces ( | ||
trace_id blob, | ||
span_id bigint, | ||
span_hash bigint, | ||
parent_id bigint, | ||
operation_name text, | ||
flags int, | ||
start_time bigint, -- microseconds since epoch | ||
duration bigint, -- microseconds | ||
tags list<frozen<keyvalue>>, | ||
logs list<frozen<log>>, | ||
refs list<frozen<span_ref>>, | ||
process frozen<process>, | ||
PRIMARY KEY (trace_id, span_id, span_hash) | ||
) | ||
WITH compaction = { | ||
'compaction_window_size': '${compaction_window_size}', | ||
'compaction_window_unit': '${compaction_window_unit}', | ||
'class': 'org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy' | ||
} | ||
AND default_time_to_live = ${trace_ttl} | ||
AND speculative_retry = 'NONE' | ||
AND gc_grace_seconds = 10800; -- 3 hours of downtime acceptable on nodes | ||
|
||
CREATE TABLE IF NOT EXISTS ${keyspace}.service_names ( | ||
service_name text, | ||
PRIMARY KEY (service_name) | ||
) | ||
WITH compaction = { | ||
'min_threshold': '4', | ||
'max_threshold': '32', | ||
'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy' | ||
} | ||
AND default_time_to_live = ${trace_ttl} | ||
AND speculative_retry = 'NONE' | ||
AND gc_grace_seconds = 10800; -- 3 hours of downtime acceptable on nodes | ||
|
||
CREATE TABLE IF NOT EXISTS ${keyspace}.operation_names_v2 ( | ||
service_name text, | ||
span_kind text, | ||
operation_name text, | ||
PRIMARY KEY ((service_name), span_kind, operation_name) | ||
) | ||
WITH compaction = { | ||
'min_threshold': '4', | ||
'max_threshold': '32', | ||
'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy' | ||
} | ||
AND default_time_to_live = ${trace_ttl} | ||
AND speculative_retry = 'NONE' | ||
AND gc_grace_seconds = 10800; -- 3 hours of downtime acceptable on nodes | ||
|
||
-- index of trace IDs by service + operation names, sorted by span start_time. | ||
CREATE TABLE IF NOT EXISTS ${keyspace}.service_operation_index ( | ||
service_name text, | ||
operation_name text, | ||
start_time bigint, -- microseconds since epoch | ||
trace_id blob, | ||
PRIMARY KEY ((service_name, operation_name), start_time) | ||
) WITH CLUSTERING ORDER BY (start_time DESC) | ||
AND compaction = { | ||
'compaction_window_size': '1', | ||
'compaction_window_unit': 'HOURS', | ||
'class': 'org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy' | ||
} | ||
AND default_time_to_live = ${trace_ttl} | ||
AND speculative_retry = 'NONE' | ||
AND gc_grace_seconds = 10800; -- 3 hours of downtime acceptable on nodes | ||
|
||
CREATE TABLE IF NOT EXISTS ${keyspace}.service_name_index ( | ||
service_name text, | ||
bucket int, | ||
start_time bigint, -- microseconds since epoch | ||
trace_id blob, | ||
PRIMARY KEY ((service_name, bucket), start_time) | ||
) WITH CLUSTERING ORDER BY (start_time DESC) | ||
AND compaction = { | ||
'compaction_window_size': '1', | ||
'compaction_window_unit': 'HOURS', | ||
'class': 'org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy' | ||
} | ||
AND default_time_to_live = ${trace_ttl} | ||
AND speculative_retry = 'NONE' | ||
AND gc_grace_seconds = 10800; -- 3 hours of downtime acceptable on nodes | ||
|
||
CREATE TABLE IF NOT EXISTS ${keyspace}.duration_index ( | ||
service_name text, -- service name | ||
operation_name text, -- operation name, or blank for queries without span name | ||
bucket timestamp, -- time bucket, - the start_time of the given span rounded to an hour | ||
duration bigint, -- span duration, in microseconds | ||
start_time bigint, -- microseconds since epoch | ||
trace_id blob, | ||
PRIMARY KEY ((service_name, operation_name, bucket), duration, start_time, trace_id) | ||
) WITH CLUSTERING ORDER BY (duration DESC, start_time DESC) | ||
AND compaction = { | ||
'compaction_window_size': '1', | ||
'compaction_window_unit': 'HOURS', | ||
'class': 'org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy' | ||
} | ||
AND default_time_to_live = ${trace_ttl} | ||
AND speculative_retry = 'NONE' | ||
AND gc_grace_seconds = 10800; -- 3 hours of downtime acceptable on nodes | ||
|
||
-- a bucketing strategy may have to be added for tag queries | ||
-- we can make this table even better by adding a timestamp to it | ||
CREATE TABLE IF NOT EXISTS ${keyspace}.tag_index ( | ||
service_name text, | ||
tag_key text, | ||
tag_value text, | ||
start_time bigint, -- microseconds since epoch | ||
trace_id blob, | ||
span_id bigint, | ||
PRIMARY KEY ((service_name, tag_key, tag_value), start_time, trace_id, span_id) | ||
) | ||
WITH CLUSTERING ORDER BY (start_time DESC) | ||
AND compaction = { | ||
'compaction_window_size': '1', | ||
'compaction_window_unit': 'HOURS', | ||
'class': 'org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy' | ||
} | ||
AND default_time_to_live = ${trace_ttl} | ||
AND speculative_retry = 'NONE' | ||
AND gc_grace_seconds = 10800; -- 3 hours of downtime acceptable on nodes | ||
|
||
CREATE TYPE IF NOT EXISTS ${keyspace}.dependency ( | ||
parent text, | ||
child text, | ||
call_count bigint, | ||
source text | ||
); | ||
|
||
-- compaction strategy is intentionally different as compared to other tables due to the size of dependencies data | ||
CREATE TABLE IF NOT EXISTS ${keyspace}.dependencies_v2 ( | ||
ts_bucket timestamp, | ||
ts timestamp, | ||
dependencies list<frozen<dependency>>, | ||
PRIMARY KEY (ts_bucket, ts) | ||
) WITH CLUSTERING ORDER BY (ts DESC) | ||
AND compaction = { | ||
'min_threshold': '4', | ||
'max_threshold': '32', | ||
'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy' | ||
} | ||
AND default_time_to_live = ${dependencies_ttl}; | ||
|
||
-- adaptive sampling tables | ||
-- ./plugin/storage/cassandra/samplingstore/storage.go | ||
CREATE TABLE IF NOT EXISTS ${keyspace}.operation_throughput ( | ||
bucket int, | ||
ts timeuuid, | ||
throughput text, | ||
PRIMARY KEY(bucket, ts) | ||
) WITH CLUSTERING ORDER BY (ts desc); | ||
|
||
CREATE TABLE IF NOT EXISTS ${keyspace}.sampling_probabilities ( | ||
bucket int, | ||
ts timeuuid, | ||
hostname text, | ||
probabilities text, | ||
PRIMARY KEY(bucket, ts) | ||
) WITH CLUSTERING ORDER BY (ts desc); | ||
|
||
-- distributed lock | ||
-- ./plugin/pkg/distributedlock/cassandra/lock.go | ||
CREATE TABLE IF NOT EXISTS ${keyspace}.leases ( | ||
name text, | ||
owner text, | ||
PRIMARY KEY (name) | ||
); |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we want to drop schema creation for v3?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These are not Cassandra versions but our db schema evolutions. Does something need to change for Cassandra v5?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
no i just copied the same file to maintain the separation - let me know if you want me to combine it
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If the db schema is not changing there's no need to create a new file.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
should i just use the v4 file then?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes - schema version and Cassandra version are not coupled.