Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Resource Cleanup on delete #90

Merged
merged 6 commits into from
Oct 18, 2024

Conversation

danielgerlag
Copy link
Contributor

Description

This PR adds infrastructure to clean up resources when sources / reactions are deleted. It takes a 2 pronged approach.

The first is to add an optional deprovisionHandler to any resource service. This is defined on the Source / Reaction Provider spec. When the resource is deleted, the control plane will do a best effort attempt to call the deprovision method on the service, using standard Dapr retry semantics. If the handler is not callable, to resource will still be deleted.

To second, is to include an INSTANCE_ID environment variable on each service. This value remains the same for the lifetime of the resource (source/reaction). If you delete and recreate the resource with the same name, the instance id will change. Thus the service can inspect if the instance id is as expected, if not then any existing state should be considered stale.

Both mechanisms have been implemented for the debezium reactivator.

control-planes/mgmt_api/src/main.rs Dismissed Show dismissed Hide dismissed
infrastructure/comms-dapr/src/comms.rs Dismissed Show dismissed Hide dismissed
@ruokun-niu
Copy link
Contributor

Should we set deprovisionHandler to be true for all sources?

@ruokun-niu
Copy link
Contributor

Quick question (I might be jumping ahead since I vaguely remember you mentioning that there will be multiple PRs for source cleanups): If I set deprovisionHandler to true for one of my source reactivator, will the dapr resources for other components of the source be deleted when the source is deleted? For example, when I delete the source, will the dapr state data of the change router be cleared?

@danielgerlag
Copy link
Contributor Author

Should we set deprovisionHandler to be true for all sources?

Only once we have implemented handlers for them

@danielgerlag
Copy link
Contributor Author

Quick question (I might be jumping ahead since I vaguely remember you mentioning that there will be multiple PRs for source cleanups): If I set deprovisionHandler to true for one of my source reactivator, will the dapr resources for other components of the source be deleted when the source is deleted? For example, when I delete the source, will the dapr state data of the change router be cleared?

It's per service. Since change router is a universal service across all sources, if we implemented a handler, it wouldn't need a special config, so we could to call it if we wanted to.

@agentofreality
Copy link
Contributor

I think, in future, we should do separate PRs for things like the changes to the shared infrastructure abstractions.

@danielgerlag danielgerlag merged commit dda3a66 into drasi-project:main Oct 18, 2024
30 checks passed
ruokun-niu added a commit to ruokun-niu/drasi-platform that referenced this pull request Oct 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants