Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement default LocalQueue #3610

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Implement default LocalQueue #3610

wants to merge 1 commit into from
+563 −39

Conversation

yaroslava-serdiuk
Copy link
Contributor

@yaroslava-serdiuk yaroslava-serdiuk commented Nov 21, 2024 •
edited
Loading

What type of PR is this?

/kind feature

What this PR does / why we need it:

Which issue(s) this PR fixes:

Fixes #2936

Does this PR introduce a user-facing change?

If KueueDefaulting feature gate is set to true, the LocalQueue with name default will be used as default LocalQueue in the corresponding namespace"

cc: @mwielgus @KPostOffice

@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. kind/feature Categorizes issue or PR as related to a new feature. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Nov 21, 2024
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: yaroslava-serdiuk
Once this PR has been reviewed and has the lgtm label, please assign mimowo for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Nov 21, 2024
@netlify Netlify
Copy link

netlify bot commented Nov 21, 2024 •
edited
Loading

✅ Deploy Preview for kubernetes-sigs-kueue canceled.

Name Link
🔨 Latest commit a2e7c62
🔍 Latest deploy log https://app.netlify.com/sites/kubernetes-sigs-kueue/deploys/674dc822c3afc400083ff130

@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 21, 2024
kannon92
kannon92 reviewed Nov 22, 2024
View reviewed changes
pkg/queue/local_queue.go Outdated Show resolved Hide resolved
kannon92
kannon92 reviewed Nov 22, 2024
View reviewed changes
pkg/controller/jobframework/base_webhook.go Outdated
@@ -55,7 +59,11 @@ func (w *BaseWebhook) Default(ctx context.Context, obj runtime.Object) error {
job := w.FromObject(obj)
log := ctrl.LoggerFrom(ctx)
log.V(5).Info("Applying defaults", "job", klog.KObj(job.Object()))
ApplyDefaultForSuspend(job, w.ManageJobsWithoutQueueName)
if features.Enabled(features.KueueDefaulting) && w.Queues.DefaultLocalQueue(job.Object().GetNamespace()) != "" {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe I miss the discussion but I read this change to mean:

Once KueueDefaulting is true, we will no longer use w.ManageJobsWithoutQueueName?

Should we consider deprecation of w.ManageJobsWithoutQueueName?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In k/k, we always take feature gates to eventually mean that they will be true so we should think through when we want to remove this feature gate and always opt in for this feature.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. With KueueDefaulting ManageJobsWithoutQueueName doesn't make sense, so in case the feature go to GA, ManageJobsWithoutQueueName should be deprecated.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

However, if the default queue doesn't exist, ManageJobsWithoutQueueName is still useful.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

^^ agree here. I was thinking of this as just a QoL improvement especially for cases where users have access to only 1 LocalQueue. It allows them to ignore any underlying Kueue details and submit their jobs as normal

@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 25, 2024
@kannon92 kannon92 mentioned this pull request Nov 25, 2024
Merged
@k8s-ci-robot k8s-ci-robot added needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Nov 26, 2024
@yaroslava-serdiuk yaroslava-serdiuk mentioned this pull request Nov 28, 2024
Open
PBundyra
PBundyra reviewed Nov 29, 2024
View reviewed changes
pkg/controller/jobframework/base_webhook_test.go
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use JobWrapper from pkg/util/testingjobs/job/wrappers.go?

PBundyra
PBundyra reviewed Nov 29, 2024
View reviewed changes
pkg/controller/jobs/jobset/jobset_webhook_test.go
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use JobSet wrapper from pkg/util/testingjobs/jobset/wrappers.go

PBundyra
PBundyra reviewed Nov 29, 2024
View reviewed changes
pkg/controller/jobs/mpijob/mpijob_webhook_test.go
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here, can we use MPIJobWrapper?

PBundyra
PBundyra reviewed Nov 29, 2024
View reviewed changes
pkg/controller/jobs/pod/pod_webhook.go Outdated
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add tests for that as well?

PBundyra
PBundyra reviewed Nov 29, 2024
View reviewed changes
pkg/queue/manager.go
@@ -206,6 +206,16 @@ func (m *Manager) DeleteClusterQueue(cq *kueue.ClusterQueue) {
metrics.ClearClusterQueueMetrics(cq.Name)
}

func (m *Manager) DefaultLocalQueue(namespace string) string {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we always check the result of this function in if statement and compare it with "", can this function return true if there is the default LocalQueue in the namespace and false otherwise. If so, then can we rename it to something along the lines IsDefaultLQInNamespace

PBundyra
PBundyra reviewed Nov 29, 2024
View reviewed changes
pkg/features/kube_features.go
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you also update the docs in site/content/en/docs/installation/_index.md?

@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Dec 2, 2024
mimowo
mimowo reviewed Dec 2, 2024
View reviewed changes
pkg/features/kube_features.go Outdated
// alpha: v0.9
//
// Enable to set default LocalQueue.
KueueDefaulting featuregate.Feature = "KueueDefaulting"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please update as in the KEP and make sure the feature-gate is used to guard the code

@k8s-ci-robot
Copy link
Contributor

@yaroslava-serdiuk: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-kueue-verify-main a2e7c62 link true /test pull-kueue-verify-main
pull-kueue-test-integration-main a2e7c62 link true /test pull-kueue-test-integration-main
pull-kueue-test-unit-main a2e7c62 link true /test pull-kueue-test-unit-main

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@PBundyra PBundyra PBundyra left review comments

@KPostOffice KPostOffice KPostOffice left review comments

@mimowo mimowo mimowo left review comments

@kannon92 kannon92 kannon92 left review comments

@trasc trasc Awaiting requested review from trasc

Assignees
No one assigned
Labels
cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/feature Categorizes issue or PR as related to a new feature. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Default LocalQueue
6 participants
@yaroslava-serdiuk @k8s-ci-robot @kannon92 @mimowo @KPostOffice @PBundyra