Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG-5656] Annotate Jobs with parent ScaledJob generation #5876

Merged
merged 8 commits into from
Jul 30, 2024

Conversation

josefkarasek
Copy link
Member

@josefkarasek josefkarasek commented Jun 10, 2024

As reported in #5656, keda-operator Pod restart causes the deletion of existing Jobs (child resources of ScaledJob). The Jobs are re-created immediately, but the state/progress of the running Jobs is lost.

My approach to fixing this is to maintain relation between the ScaledJob and child Jobs using an Annotation:
"scaledjob.keda.sh/generation": $(ScaledJob.Generation)
This way the operator, during startup, will only delete Jobs with "scaledjob.keda.sh/generation" that doesn't match the generation of the parent ScaledJob.

Checklist

Fixes #5656

@josefkarasek josefkarasek marked this pull request as ready for review June 17, 2024 09:15
@josefkarasek josefkarasek requested a review from a team as a code owner June 17, 2024 09:15
@josefkarasek josefkarasek changed the title Annotate Jobs with parent ScaledJob generation [BUG-5656] Annotate Jobs with parent ScaledJob generation Jun 17, 2024
Copy link
Member

@zroubalik zroubalik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good, thanks! Could you please also update changelog?

pkg/scaling/executor/scale_jobs.go Outdated Show resolved Hide resolved
@zroubalik
Copy link
Member

zroubalik commented Jun 25, 2024

/run-e2e internal
Update: You can check the progress here

zroubalik

This comment was marked as outdated.

Copy link
Member

@zroubalik zroubalik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@josefkarasek the static check si failing on the Changelog entry, it needs to be alphabetically sorted, also please move this into Improvements section - it is an user-facing issue.

The rest looks good, thanks

@zroubalik zroubalik self-requested a review June 25, 2024 19:51
Signed-off-by: Josef Karasek <josef@kedify.io>
Signed-off-by: Josef Karasek <josef@kedify.io>
Signed-off-by: Josef Karasek <josef@kedify.io>
Signed-off-by: Josef Karasek <josef@kedify.io>
Signed-off-by: Josef Karasek <josef@kedify.io>
Signed-off-by: Josef Karasek <josef@kedify.io>
Signed-off-by: Josef Karasek <josef@kedify.io>
@zroubalik
Copy link
Member

zroubalik commented Jul 30, 2024

/run-e2e internal
Update: You can check the progress here

Signed-off-by: Zbynek Roubalik <zroubalik@gmail.com>
@zroubalik zroubalik merged commit 7ca6708 into kedacore:main Jul 30, 2024
16 checks passed
@JorTurFer JorTurFer mentioned this pull request Jul 30, 2024
23 tasks
JorTurFer pushed a commit to JorTurFer/keda that referenced this pull request Jul 30, 2024
)

* Annotate Jobs with parent ScaledJob generation

Signed-off-by: Josef Karasek <josef@kedify.io>

* fix tests

Signed-off-by: Josef Karasek <josef@kedify.io>

* fix lint

Signed-off-by: Josef Karasek <josef@kedify.io>

* fix log message

Signed-off-by: Josef Karasek <josef@kedify.io>

* update changelog

Signed-off-by: Josef Karasek <josef@kedify.io>

* update changelog

Signed-off-by: Josef Karasek <josef@kedify.io>

* update changelog

Signed-off-by: Josef Karasek <josef@kedify.io>

---------

Signed-off-by: Josef Karasek <josef@kedify.io>
Signed-off-by: Zbynek Roubalik <zroubalik@gmail.com>
Co-authored-by: Zbynek Roubalik <zroubalik@gmail.com>
JorTurFer pushed a commit to JorTurFer/keda that referenced this pull request Jul 30, 2024
)

* Annotate Jobs with parent ScaledJob generation

Signed-off-by: Josef Karasek <josef@kedify.io>

* fix tests

Signed-off-by: Josef Karasek <josef@kedify.io>

* fix lint

Signed-off-by: Josef Karasek <josef@kedify.io>

* fix log message

Signed-off-by: Josef Karasek <josef@kedify.io>

* update changelog

Signed-off-by: Josef Karasek <josef@kedify.io>

* update changelog

Signed-off-by: Josef Karasek <josef@kedify.io>

* update changelog

Signed-off-by: Josef Karasek <josef@kedify.io>

---------

Signed-off-by: Josef Karasek <josef@kedify.io>
Signed-off-by: Zbynek Roubalik <zroubalik@gmail.com>
Co-authored-by: Zbynek Roubalik <zroubalik@gmail.com>
Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es>
JorTurFer added a commit that referenced this pull request Jul 31, 2024
* bump golang

Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es>

* chore: build with keda-tools:1.22.5 (#5971)

* chore: build with keda-tools:1.22.5
to resolve CVE-2024-24790, CVE-2024-24789, and CVE-2024-24791
bump github.com/Azure/azure-sdk-for-go/sdk/azidentity to resolve CVE-2024-35255

Signed-off-by: Paul Yu <paul.d.yu@gmail.com>

* chore: use go install instead of go get and replacing deprecated tools

Signed-off-by: Paul Yu <paul.d.yu@gmail.com>

* chore: vendor dependency cleanup

Signed-off-by: Paul Yu <paul.d.yu@gmail.com>

* Update missing references to 1.21

Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es>

---------

Signed-off-by: Paul Yu <paul.d.yu@gmail.com>
Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es>
Co-authored-by: Jorge Turrado Ferrero <Jorge_turrado@hotmail.es>
Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es>

* [BUG-5922] Report failing ScaledJob triggers in status (#5916)

Signed-off-by: Josef Karasek <josef@kedify.io>
Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es>

* [BUG-5656] Annotate Jobs with parent ScaledJob generation (#5876)

* Annotate Jobs with parent ScaledJob generation

Signed-off-by: Josef Karasek <josef@kedify.io>

* fix tests

Signed-off-by: Josef Karasek <josef@kedify.io>

* fix lint

Signed-off-by: Josef Karasek <josef@kedify.io>

* fix log message

Signed-off-by: Josef Karasek <josef@kedify.io>

* update changelog

Signed-off-by: Josef Karasek <josef@kedify.io>

* update changelog

Signed-off-by: Josef Karasek <josef@kedify.io>

* update changelog

Signed-off-by: Josef Karasek <josef@kedify.io>

---------

Signed-off-by: Josef Karasek <josef@kedify.io>
Signed-off-by: Zbynek Roubalik <zroubalik@gmail.com>
Co-authored-by: Zbynek Roubalik <zroubalik@gmail.com>
Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es>

* fix: `+srv` mongodb url scheme parsing bug (#5773)

This commit fixs issue #5760. where OP was facing problem with +srv schema

Signed-off-by: Rishikesh Betigeri <53863619+Rishikesh01@users.noreply.github.com>
Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es>

* fix: issue when GitHub organization contains more than 30 repos (#5746)

Signed-off-by: Simon Kobler <github@kobler.me>
Signed-off-by: Simon Kobler <32038731+KoblerS@users.noreply.github.com>
Co-authored-by: Jorge Turrado Ferrero <Jorge_turrado@hotmail.es>
Co-authored-by: Simon Kobler <github@kobler.me>
Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es>

* Fix scaler leak during cache refresh (#5807)

Signed-off-by: Guillaume Jacquet <guillaume.jacquet@gmail.com>
Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es>

* Prepare release v2.14.1

Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es>

* add missing change

Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es>

* update changelog

Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es>

* fix: e2e test regex check tag (#5831)

Signed-off-by: Jan Wozniak <wozniak.jan@gmail.com>
Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es>

* Validate regex before building image for e2e test (#5783)

* added regex pre check before building image

Signed-off-by: Yaxhveer <yaxhcod@gmail.com>

* updated changelog

Signed-off-by: Yaxhveer <yaxhcod@gmail.com>

* refactored

Signed-off-by: Yaxhveer <yaxhcod@gmail.com>

* corrected

Signed-off-by: Yaxhveer <yaxhcod@gmail.com>

* corrected changelog

Signed-off-by: Yaxhveer <yaxhcod@gmail.com>

* updated the workflow

Signed-off-by: Yaxhveer <yaxhcod@gmail.com>

* updated the workflow

Signed-off-by: Yaxhveer <yaxhcod@gmail.com>

---------

Signed-off-by: Yaxhveer <yaxhcod@gmail.com>
Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es>

* fix some pending tasks

Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es>

* use AAD-Pod-Identity always

Signed-off-by: Jorge Turrado <jorge.turrado@scrm.lidl>

* use AAD-Pod-Identity always

Signed-off-by: Jorge Turrado <jorge.turrado@scrm.lidl>

---------

Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es>
Signed-off-by: Paul Yu <paul.d.yu@gmail.com>
Signed-off-by: Josef Karasek <josef@kedify.io>
Signed-off-by: Zbynek Roubalik <zroubalik@gmail.com>
Signed-off-by: Rishikesh Betigeri <53863619+Rishikesh01@users.noreply.github.com>
Signed-off-by: Simon Kobler <github@kobler.me>
Signed-off-by: Simon Kobler <32038731+KoblerS@users.noreply.github.com>
Signed-off-by: Guillaume Jacquet <guillaume.jacquet@gmail.com>
Signed-off-by: Jan Wozniak <wozniak.jan@gmail.com>
Signed-off-by: Yaxhveer <yaxhcod@gmail.com>
Signed-off-by: Jorge Turrado <jorge.turrado@scrm.lidl>
Co-authored-by: Paul Yu <paul.d.yu@gmail.com>
Co-authored-by: Josef Karasek <josef@kedify.io>
Co-authored-by: Zbynek Roubalik <zroubalik@gmail.com>
Co-authored-by: Rishikesh <53863619+Rishikesh01@users.noreply.github.com>
Co-authored-by: Simon Kobler <32038731+KoblerS@users.noreply.github.com>
Co-authored-by: Simon Kobler <github@kobler.me>
Co-authored-by: Guillaume Jacquet <guillaume.jacquet@gmail.com>
Co-authored-by: Jan Wozniak <wozniak.jan@gmail.com>
Co-authored-by: Yashveer <101015836+Yaxhveer@users.noreply.github.com>
jkyros pushed a commit to jkyros/keda that referenced this pull request Aug 2, 2024
* bump golang

Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es>

* chore: build with keda-tools:1.22.5 (kedacore#5971)

* chore: build with keda-tools:1.22.5
to resolve CVE-2024-24790, CVE-2024-24789, and CVE-2024-24791
bump github.com/Azure/azure-sdk-for-go/sdk/azidentity to resolve CVE-2024-35255

Signed-off-by: Paul Yu <paul.d.yu@gmail.com>

* chore: use go install instead of go get and replacing deprecated tools

Signed-off-by: Paul Yu <paul.d.yu@gmail.com>

* chore: vendor dependency cleanup

Signed-off-by: Paul Yu <paul.d.yu@gmail.com>

* Update missing references to 1.21

Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es>

---------

Signed-off-by: Paul Yu <paul.d.yu@gmail.com>
Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es>
Co-authored-by: Jorge Turrado Ferrero <Jorge_turrado@hotmail.es>
Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es>

* [BUG-5922] Report failing ScaledJob triggers in status (kedacore#5916)

Signed-off-by: Josef Karasek <josef@kedify.io>
Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es>

* [BUG-5656] Annotate Jobs with parent ScaledJob generation (kedacore#5876)

* Annotate Jobs with parent ScaledJob generation

Signed-off-by: Josef Karasek <josef@kedify.io>

* fix tests

Signed-off-by: Josef Karasek <josef@kedify.io>

* fix lint

Signed-off-by: Josef Karasek <josef@kedify.io>

* fix log message

Signed-off-by: Josef Karasek <josef@kedify.io>

* update changelog

Signed-off-by: Josef Karasek <josef@kedify.io>

* update changelog

Signed-off-by: Josef Karasek <josef@kedify.io>

* update changelog

Signed-off-by: Josef Karasek <josef@kedify.io>

---------

Signed-off-by: Josef Karasek <josef@kedify.io>
Signed-off-by: Zbynek Roubalik <zroubalik@gmail.com>
Co-authored-by: Zbynek Roubalik <zroubalik@gmail.com>
Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es>

* fix: `+srv` mongodb url scheme parsing bug (kedacore#5773)

This commit fixs issue kedacore#5760. where OP was facing problem with +srv schema

Signed-off-by: Rishikesh Betigeri <53863619+Rishikesh01@users.noreply.github.com>
Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es>

* fix: issue when GitHub organization contains more than 30 repos (kedacore#5746)

Signed-off-by: Simon Kobler <github@kobler.me>
Signed-off-by: Simon Kobler <32038731+KoblerS@users.noreply.github.com>
Co-authored-by: Jorge Turrado Ferrero <Jorge_turrado@hotmail.es>
Co-authored-by: Simon Kobler <github@kobler.me>
Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es>

* Fix scaler leak during cache refresh (kedacore#5807)

Signed-off-by: Guillaume Jacquet <guillaume.jacquet@gmail.com>
Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es>

* Prepare release v2.14.1

Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es>

* add missing change

Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es>

* update changelog

Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es>

* fix: e2e test regex check tag (kedacore#5831)

Signed-off-by: Jan Wozniak <wozniak.jan@gmail.com>
Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es>

* Validate regex before building image for e2e test (kedacore#5783)

* added regex pre check before building image

Signed-off-by: Yaxhveer <yaxhcod@gmail.com>

* updated changelog

Signed-off-by: Yaxhveer <yaxhcod@gmail.com>

* refactored

Signed-off-by: Yaxhveer <yaxhcod@gmail.com>

* corrected

Signed-off-by: Yaxhveer <yaxhcod@gmail.com>

* corrected changelog

Signed-off-by: Yaxhveer <yaxhcod@gmail.com>

* updated the workflow

Signed-off-by: Yaxhveer <yaxhcod@gmail.com>

* updated the workflow

Signed-off-by: Yaxhveer <yaxhcod@gmail.com>

---------

Signed-off-by: Yaxhveer <yaxhcod@gmail.com>
Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es>

* fix some pending tasks

Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es>

* use AAD-Pod-Identity always

Signed-off-by: Jorge Turrado <jorge.turrado@scrm.lidl>

* use AAD-Pod-Identity always

Signed-off-by: Jorge Turrado <jorge.turrado@scrm.lidl>

---------

Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es>
Signed-off-by: Paul Yu <paul.d.yu@gmail.com>
Signed-off-by: Josef Karasek <josef@kedify.io>
Signed-off-by: Zbynek Roubalik <zroubalik@gmail.com>
Signed-off-by: Rishikesh Betigeri <53863619+Rishikesh01@users.noreply.github.com>
Signed-off-by: Simon Kobler <github@kobler.me>
Signed-off-by: Simon Kobler <32038731+KoblerS@users.noreply.github.com>
Signed-off-by: Guillaume Jacquet <guillaume.jacquet@gmail.com>
Signed-off-by: Jan Wozniak <wozniak.jan@gmail.com>
Signed-off-by: Yaxhveer <yaxhcod@gmail.com>
Signed-off-by: Jorge Turrado <jorge.turrado@scrm.lidl>
Co-authored-by: Paul Yu <paul.d.yu@gmail.com>
Co-authored-by: Josef Karasek <josef@kedify.io>
Co-authored-by: Zbynek Roubalik <zroubalik@gmail.com>
Co-authored-by: Rishikesh <53863619+Rishikesh01@users.noreply.github.com>
Co-authored-by: Simon Kobler <32038731+KoblerS@users.noreply.github.com>
Co-authored-by: Simon Kobler <github@kobler.me>
Co-authored-by: Guillaume Jacquet <guillaume.jacquet@gmail.com>
Co-authored-by: Jan Wozniak <wozniak.jan@gmail.com>
Co-authored-by: Yashveer <101015836+Yaxhveer@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Restart of keda-operator causes ScaledJob object to be updated
2 participants