Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(outcomes): Add TTL to outcomes dataset #3615

Merged
merged 2 commits into from
Jan 23, 2023
Merged

fix(outcomes): Add TTL to outcomes dataset #3615

merged 2 commits into from
Jan 23, 2023

Conversation

nikhars
Copy link
Member

@nikhars nikhars commented Jan 19, 2023

Since there is no TTL on the outcomes dataset, the storage size can grow unbounded. This PR adds a TTL of 30 days for the raw tables and 90 days for the aggregate tables.

Since there is no TTL on the outcomes dataset, the storage size can grow unbounded. This PR adds a TTL of 30 days for the raw tables and 90 days for the aggregate tables
@github-actions
Copy link

This PR has a migration; here is the generated SQL

-- start migrations

-- forward migration outcomes : 0005_outcomes_ttl
Local op: ALTER TABLE outcomes_raw_local MODIFY TTL timestamp + toIntervalDay(30);
Local op: ALTER TABLE outcomes_hourly_local MODIFY TTL timestamp + toIntervalDay(90);
-- end forward migration outcomes : 0005_outcomes_ttl




-- backward migration outcomes : 0005_outcomes_ttl
-- end backward migration outcomes : 0005_outcomes_ttl

@codecov
Copy link

codecov bot commented Jan 19, 2023

Codecov Report

Base: 92.35% // Head: 92.35% // Increases project coverage by +0.00% 🎉

Coverage data is based on head (8d58205) compared to base (2d353e0).
Patch coverage: 100.00% of modified lines in pull request are covered.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #3615   +/-   ##
=======================================
  Coverage   92.35%   92.35%           
=======================================
  Files         741      742    +1     
  Lines       34293    34303   +10     
=======================================
+ Hits        31671    31681   +10     
  Misses       2622     2622           
Impacted Files Coverage Δ
snuba/migrations/groups.py 95.61% <ø> (ø)
...uba/snuba_migrations/outcomes/0005_outcomes_ttl.py 100.00% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@nikhars nikhars marked this pull request as ready for review January 19, 2023 18:43
@nikhars nikhars requested a review from a team as a code owner January 19, 2023 18:43
@dbanda
Copy link
Contributor

dbanda commented Jan 22, 2023

Is this change also being added to SaaS? I ask because I have a PR #3605 that does something similar by adding a TTL to a columns. Dont know if that will be repeating work.

@nikhars
Copy link
Member Author

nikhars commented Jan 23, 2023

Is this change also being added to SaaS? I ask because I have a PR #3605 that does something similar by adding a TTL to a columns. Dont know if that will be repeating work.

Yes, I will apply the TTL settings to SaaS. In your PR, the TTL change I see is
Local op: ALTER TABLE outcomes_raw_local MODIFY COLUMN event_id Nullable(UUID) CODEC (LZ4HC(0)) TTL timestamp + toIntervalDay(30);
Is that TTL change not on the SaaS environment yet? I thought the reconciliation was getting code to match SaaS. Could you provide some additional context.

@nikhars nikhars merged commit f0dae68 into master Jan 23, 2023
@nikhars nikhars deleted the fix/outcomes-ttl branch January 23, 2023 18:21
@dbanda
Copy link
Contributor

dbanda commented Jan 23, 2023

Is this change also being added to SaaS? I ask because I have a PR #3605 that does something similar by adding a TTL to a columns. Dont know if that will be repeating work.

Yes, I will apply the TTL settings to SaaS. In your PR, the TTL change I see is Local op: ALTER TABLE outcomes_raw_local MODIFY COLUMN event_id Nullable(UUID) CODEC (LZ4HC(0)) TTL timestamp + toIntervalDay(30); Is that TTL change not on the SaaS environment yet? I thought the reconciliation was getting code to match SaaS. Could you provide some additional context.

yes the change is in SaaS. but to get them to match I will also have to add it to what we have here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants