Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(controller): Cron re-apply update #3883

Merged
merged 4 commits into from
Aug 31, 2020
Merged

Conversation

alexec
Copy link
Contributor

@alexec alexec commented Aug 27, 2020

Checklist:

  • Either (a) I've created an enhancement proposal and discussed it with the community, (b) this is a bug fix, or (c) this is a chore.
  • The title of the PR is (a) conventional, (b) states what changed, and (c) suffixes the related issues number. E.g. "fix(controller): Updates such and such. Fixes #1234".
  • I've signed the CLA.
  • I have written unit and/or e2e tests for my change. PRs without these are unlikely to be merged.
  • My builds are green. Try syncing with master if they are not.
  • My organization is added to USERS.md.

@@ -98,7 +98,6 @@ func (woc *cronWfOperationCtx) validateCronWorkflow() error {
woc.reportCronWorkflowError(v1alpha1.ConditionTypeSpecError, fmt.Sprint(err))
} else {
woc.cronWf.Status.Conditions.RemoveCondition(v1alpha1.ConditionTypeSpecError)
woc.persistUpdate()
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is the bug I think, cannot reapplyUpdates twice

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch

@@ -117,58 +116,68 @@ func getWorkflowObjectReference(wf *v1alpha1.Workflow, runWf *v1alpha1.Workflow)
}

func (woc *cronWfOperationCtx) persistUpdate() {
_, err := woc.cronWfIf.Update(woc.cronWf)
if woc.origCronWf.ResourceVersion != woc.cronWf.ResourceVersion {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a guard-rail

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might also not be necessary depending on https://github.com/argoproj/argo/pull/3883/files#r480375881

@alexec alexec added this to the v2.11 milestone Aug 28, 2020
@alexec alexec marked this pull request as ready for review August 28, 2020 20:57
@alexec
Copy link
Contributor Author

alexec commented Aug 28, 2020

Flaky: TestPendingRetryWorkflowWithRetryStrategy

@simster7 simster7 self-assigned this Aug 31, 2020
@@ -98,7 +98,6 @@ func (woc *cronWfOperationCtx) validateCronWorkflow() error {
woc.reportCronWorkflowError(v1alpha1.ConditionTypeSpecError, fmt.Sprint(err))
} else {
woc.cronWf.Status.Conditions.RemoveCondition(v1alpha1.ConditionTypeSpecError)
woc.persistUpdate()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch

}
}
woc.cronWf = cronWf
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, I'm not too sure what the purpose of this is. In theory, once persistUpdates is called, it should be the end of the cron operation context and nothing else should access woc.cronWf. Could you explain a bit why this line is necessary?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've included this so that subsequent calls to this func will have the updated version, and as a result cannot accidentally reapplyUpdate twice.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've included this so that subsequent calls to this func will have the updated version, and as a result cannot accidentally reapplyUpdate twice.

In that case wouldn't the guard rail here (#3883 (comment)) always fail? The resource version of the new woc.cronWf will always be different than the old.

It seems like we don't intend to call persistUpdates more than once at all. Should we instead ensure that it doesn't happen altogether?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My goals to make sure we don't regress by putting in place checks. I would want the guard condition to fail - as it indicates a programming error.

@alexec alexec added the P2 label Aug 31, 2020
Copy link
Member

@simster7 simster7 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants