Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KEP: time dependent job priority #884

Closed
wants to merge 4 commits into from

Conversation

Gekko0114
Copy link
Member

What type of PR is this?

/kind documentation
/kind api-change

What this PR does / why we need it:

KEP for #754

Special notes for your reviewer:

Thanks for your review.

Does this PR introduce a user-facing change?

NONE

@k8s-ci-robot k8s-ci-robot added kind/documentation Categorizes issue or PR as related to documentation. release-note-none Denotes a PR that doesn't merit a release note. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API labels Jun 21, 2023
@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Jun 21, 2023
@k8s-ci-robot k8s-ci-robot requested a review from trasc June 21, 2023 14:23
@netlify
Copy link

netlify bot commented Jun 21, 2023

Deploy Preview for kubernetes-sigs-kueue canceled.

Name Link
🔨 Latest commit 9087f5b
🔍 Latest deploy log https://app.netlify.com/sites/kubernetes-sigs-kueue/deploys/6496e40f5df8d70008efe25c

@k8s-ci-robot
Copy link
Contributor

Hi @Gekko0114. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jun 21, 2023
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Gekko0114
Once this PR has been reviewed and has the lgtm label, please assign ahg-g for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@Gekko0114 Gekko0114 changed the title WIP: KEP: time dependent job priority KEP: time dependent job priority Jun 23, 2023
@Gekko0114 Gekko0114 marked this pull request as ready for review June 23, 2023 13:34
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 23, 2023
@k8s-ci-robot k8s-ci-robot requested a review from tenzen-y June 23, 2023 13:34
@Gekko0114
Copy link
Member Author

Gekko0114 commented Jun 23, 2023

@alculquicondor, @kerthcet, @tenzen-y, @trasc,

Thank you so much for your discussion.
I've created KEP. Could you review it?
Thanks,

Copy link
Contributor

@alculquicondor alculquicondor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/assign @mwielgus

@tenzen-y
Copy link
Member

I will check after the release of v0.4.

This proposal allows job priorities to be dynamically adjusted based on time.
This functionality aims to prioritize the execution of jobs that have been waiting
for an extended period, even if they have a lower initial priority.
By introducing time-dependent priority adjustments, the system can ensure that jobs
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about introducing just a priority/ordering annotation and keeping the updates outside of Kueue, in a simple controller? I can imagine quite a few different use cases for custom ordering in a queue and obviously not all of them can be added to the API and implemented.

### Other points
If a Job has both the `spec.PriorityClass` and WorkloadPriorityClass defined, `spec.PriorityClass` is ignored.
Since the `creationTimeStamp` of the Job is always available, the calculation can be performed every time
without updating the values of Job and Workload Priority.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can make troubleshooting harder. In case of a mix of jobs with different workload priority class it will be impossible to tell what the current order of workloads actually is at the moment (without copying them to a spreadsheet or running some additional custom scripts).

@alculquicondor
Copy link
Contributor

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jul 11, 2023

}

type DynamicPriorityPolicy struct {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can quickly become either insufficient or very cluttered. For example, I may want to have this logic outside of business hours and on weekends. Plus maybe deprioritise workloads that have been waiting for too long as their results may not actually be that relevant anymore. And so on.

@alculquicondor alculquicondor mentioned this pull request Jul 11, 2023
3 tasks
@Gekko0114
Copy link
Member Author

Gekko0114 commented Jul 13, 2023

Hi @alculquicondor, @mwielgus,

Thanks for your comment.
I might not fully understand the recent discussion, so I would appreciate it if you could clarify.

I've seen the proposal for #973. Should we prioritize that proposal over this KEP?
Or is it possible to implement this KEP after considering your comments for improvement?

@alculquicondor
Copy link
Contributor

Yes, I think we should start with #973 as a simpler version of this and wait for user feedback for one or two releases.

WDYT @kerthcet, since you initially proposed #754

@Gekko0114
Copy link
Member Author

@alculquicondor ,
Sure, thanks. Then, I will keep this PR as it is without making any changes.

@kerthcet
Copy link
Contributor

Yes, I think we should start with #973 as a simpler version of this and wait for user feedback for one or two releases.

WDYT @kerthcet, since you initially proposed #754

I left comments there.

@Gekko0114
Copy link
Member Author

I will close this PR since workloadPriorityClass was merged

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API kind/documentation Categorizes issue or PR as related to documentation. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. release-note-none Denotes a PR that doesn't merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants