Skip to content
This repository has been archived by the owner on Jun 19, 2022. It is now read-only.

Reconcilers delete old Topics and PullSubscriptions #1066

Merged

Conversation

Harwayne
Copy link
Contributor

@Harwayne Harwayne commented May 13, 2020

Fixes #905.

Proposed Changes

  • Reconcilers delete owned and controlled Topic and PullSubscription in pubsub.cloud.google.com.
    • Topic
      • First have their PropagationPolicy updated to NoDelete, so that the Topic is not deleted in GCP.
      • Deleted before the creation of the new Topic. This is because the old and new Topics both use the same Service name, so we will never have both be Ready=true at the same time.
        • This will lead to potential lost events for Channels during upgrade while the new Topic's reconciler tries to create the Service.
    • PullSubscription
      • Deleted only after the new PullSubscription is Ready=true.
        • This means there will likely be double event delivery for a period of time.
        • The old PullSubscription is not drained, so anything in it that was not delivered will be lost.

Release Note

`Source`s and `Channel`s will delete deprecated `Topic`s and `PullSubscription`s. `Channel`s may become unavailable during the upgrade, but should quickly reconcile back to being healthy. `Source`s will continue sending events and may for a short period of time send the same events multiple times. Any old events in the `PullSubscription` will be lost (e.g. an event from four hours ago that has not yet been acknowledge by the Source's sink).

Docs

Harwayne added 30 commits May 1, 2020 16:51
This is being done because the intevents version will soon need the same functions and will be given the non-prefixed names (as they will continue to exist past the next release).
@Harwayne
Copy link
Contributor Author

This is ready for review.

@Harwayne
Copy link
Contributor Author

/retest

1 similar comment
@Harwayne
Copy link
Contributor Author

/retest

Copy link
Member

@nachocano nachocano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/hold
in case you want to do some of the changes

@@ -175,15 +189,101 @@ func (psb *PubSubBase) ReconcilePullSubscription(ctx context.Context, pubsubable
}
}

// TODO Compare the actual PullSubscription we found with the desired one and update as needed.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this TODO still relevant?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope :)

Bad merge with your change that just added this.

return ps, nil
}

func (psb *PubSubBase) deleteOldPubSubTopicCO(_ context.Context, pubsubable duck.PubSubable, t *inteventsv1alpha1.Topic) pkgreconciler.Event {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: need the CO at the end? same for the pullSubscription deletion

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

something that has been useful for me in this deprecation stuff has been to add TODOs saying remove after 0.x is cut, or something like that... not strong opinion, but might be easier for others to know what to remove

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

"knative.dev/pkg/apis"
"knative.dev/pkg/kmeta"

duckv1alpha1 "github.com/google/knative-gcp/pkg/apis/duck/v1alpha1"
)

type Identifiable interface {
// runtime.Object can be removed once the old Topic and PullSubscription are removed.
runtime.Object
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this one needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The event recorder requires a runtime.Object. In the deleteOldPubSub*CO methods, I write an event if the old CO is not owned by the pubsubable.

@Harwayne
Copy link
Contributor Author

/unhold

Comments addressed.

@knative-metrics-robot
Copy link

The following is the coverage report on the affected files.
Say /test pull-google-knative-gcp-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/intevents/reconciler.go Do not exist 79.9%

@nachocano
Copy link
Member

nachocano commented May 21, 2020 via email

@knative-prow-robot knative-prow-robot merged commit 82e445e into google:master May 21, 2020
@Harwayne Harwayne deleted the 905-reconcilers-delete-old-crds branch May 21, 2020 19:58
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
approved cla: yes (override cla status due to multiple authors bug) lgtm size/XL
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Switch API Groups - pubsub.cloud.google.com -> internal.events.cloud.google.com
5 participants