Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add pluggable extension point for TranslogDeletionPolicy and move CCR specific logic to plugin. #1260

Closed
mch2 opened this issue Sep 20, 2021 · 4 comments · Fixed by #1404
Closed
Assignees
Labels
distributed framework v1.2.0 Issues related to version 1.2.0

Comments

@mch2
Copy link
Member

mch2 commented Sep 20, 2021

#1100 added translog pruning logic behind an index setting that is specific to CCR. This logic and setting should be scoped only to the CCR plugin.

Suggested solution:
We need to create an extension point for plugins to provide their own TranslogDeletionPolicy logic. Right now we have the limitation that only one plugin may override engine (A suggestion to break this limitation in #1254 ). Until that is available, we should restrict only a single plugin from overriding deletion policy.

@adnapibar
Copy link
Contributor

Instead of creating a separate plugin API for this, we can modify EnginePlugin to provide the extension points. Nick came up with an idea of creating an EngineConfigFactory to solve the issue.

@adnapibar
Copy link
Contributor

Reopening it for 1416

@adnapibar adnapibar reopened this Oct 25, 2021
@saikaranam-amazon
Copy link
Member

saikaranam-amazon commented Oct 26, 2021

@adnapibar Thanks for doing these changes.

Instead of creating a separate plugin API for this, we can modify EnginePlugin to provide the extension points

Does this mean that the only one plugin can override Engine? If that is the case, we already have use-case to override Engine by two plugins - `CCR` and `k-NN` and as only one plugin can override engine, these will need additional changes in the related plugins to avoid the conflict.

Individual plugins can still override the translog deletion policy.

@nknize
Copy link
Collaborator

nknize commented Oct 26, 2021

Individual plugins can still override the translog deletion policy.

Correct. This avoids the Engine override conflict and enables CCR and KNN to co-exist. This design maintains that only one plugin can override the translog deletion policy. We've discussed the possibility of adding a merge that enables multiple plugins to define min gen criteria; but that's out of scope for now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
distributed framework v1.2.0 Issues related to version 1.2.0
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants