Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider bringing back deduplication check; per store type. #1635

Closed
bwplotka opened this issue Oct 11, 2019 · 2 comments
Closed

Consider bringing back deduplication check; per store type. #1635

bwplotka opened this issue Oct 11, 2019 · 2 comments
Labels

Comments

@bwplotka
Copy link
Member

bwplotka commented Oct 11, 2019

This removes deduplicate check we had as there was a bug there: #1636

Ideally, we want to bring this back, but with more advanced logic: e.g per component there should be no duplicate, but also Querier vs others should not duplicate (as Querier is just a proxy in StoreAPI mode)

Goals:

  • To notify user early about misconfiguration. Essentially it does not make sense and it will pollute the system (compactor will halt) if two Prometheus-es has same data.
  • Pointing to the same StoreAPI is inefficient.
  • Pointing to the StoreAPIs with same data (e.g 2 StoreGW operating on the same bucket) is inefficient

Notify user, and block all but one of the duplicates.

We consider flag to opt-out from this check.

@bwplotka bwplotka changed the title Bring back deduplication check; per store type. Consider bringing back deduplication check; per store type. Oct 13, 2019
@bwplotka
Copy link
Member Author

This is no go as to do it right, we would need to check duplicates for certain time ranges. Plus all works fine if you duplicate except write path which we don't really touch in Querier.

I think it's fine to just keep Querier without any check. This adds warning for the single case of Ruler or Sidecar having same external labels which is not ideal.

@stale
Copy link

stale bot commented Jan 11, 2020

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Jan 11, 2020
@stale stale bot closed this as completed Jan 18, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant