Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sessions_raw storage usage grows indefinitely #3191

Closed
maths22 opened this issue Sep 27, 2022 · 1 comment · Fixed by #3223
Closed

sessions_raw storage usage grows indefinitely #3191

maths22 opened this issue Sep 27, 2022 · 1 comment · Fixed by #3223
Labels
bug Something isn't working

Comments

@maths22
Copy link

maths22 commented Sep 27, 2022

Environment

Snuba 22.6.0 in a sentry k8s environment and also snuba 22.7.0 in the onpremise docker compose environment.

Steps to Reproduce

  1. Use sentry for a long time with activity that generates sessions.

Expected Result

Session data would be purged somewhere around after it reaches retention_days old.

Actual Result

Session data is never purged by any process and as a result it takes up large amounts of disk space. On my prod k8s installation, the last ~90 days is ~400 gb, so obviously additional retention quickly adds up on the disk space front. I need to periodically. manually drop partitions to keep disk space in check The docker-compose installation is more lightly used so the disk space is not notable but shows that it has sessions_raw data that is ~16 months old, even though retention_days is set to 90.

It's possible there's some cleanup task in snuba that I don't have configured, but i can't find any evidence of such a thing in snuba's codebase.

@nikhars
Copy link
Member

nikhars commented Oct 4, 2022

We have identified the problem to be a bug in the table definition. There is no TTL defined on the table due to which it does not get deleted. We will work on fixing the bug on our end. Thanks for reporting the problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants