Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade Kubernetes to v1.30.7 #2463

Merged
merged 20 commits into from
Jan 10, 2025
Merged

Upgrade Kubernetes to v1.30.7 #2463

Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
f81a1a2
chore: update go.mod & go mod tidy.
Electronic-Waste Dec 8, 2024
b608ee0
chore: replace source.Kind and EnqueueRequestForXxx with typed func c…
Electronic-Waste Dec 8, 2024
bc49242
chore: update admission.Decoder in webhook.
Electronic-Waste Dec 8, 2024
dc3625c
chore: update Makefile.
Electronic-Waste Dec 8, 2024
0a83183
fix: update codegen script.
Electronic-Waste Dec 9, 2024
538cec6
chore: execute update-codegen.sh.
Electronic-Waste Dec 9, 2024
11e577e
chore: update openapigen & generate new openapi definitions.
Electronic-Waste Dec 9, 2024
ed01b40
fix: fix typo error.
Electronic-Waste Dec 9, 2024
b915471
chore: update k8s version in CI.
Electronic-Waste Dec 9, 2024
146148c
test(codegen): output CODEGEN_PKG.
Electronic-Waste Dec 9, 2024
5c48e4b
fix(codegen): move shell check annotation.
Electronic-Waste Dec 9, 2024
532ab00
fix(ci): change k8s version in go test to 1.30.0.
Electronic-Waste Dec 9, 2024
8c2a15c
fix: remove toolchain declaration in go.mod
Electronic-Waste Dec 11, 2024
f3088b2
fix: remove codegen dependency in openapigen.sh.
Electronic-Waste Dec 11, 2024
a63e300
fix: fix bugs in recursive dir detection.
Electronic-Waste Dec 11, 2024
aa9c300
fix: remove a blank line.
Electronic-Waste Dec 11, 2024
000729d
fix: remove klog/v2
Electronic-Waste Dec 11, 2024
251c6f9
chore(codegen): add three dots in the comment.
Electronic-Waste Jan 9, 2025
0de551d
fix(codegen): fix package dependency on k8s.io/code-generator.
Electronic-Waste Jan 9, 2025
01e41d4
chore(Makefile): add go-mod-download.
Electronic-Waste Jan 10, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/e2e-test-darts-cifar10.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,6 @@ jobs:
strategy:
fail-fast: false
matrix:
kubernetes-version: ["v1.27.11", "v1.28.7", "v1.29.2"]
kubernetes-version: ["v1.28.7", "v1.29.2", "v1.30.7"]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you want to migrate to k8s 1.31 in the followup PR ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I'd like to. But I'm busy this week. If it's not urgent, I'll raise another PR to migrate to k8s 1.31 next week.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that step by step upgrading would be better so that we can easily pick the commit when we find the issue on the specific K8s version.

So, I would recommend upgrading 1 K8s version in 1 PR.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can also add one that tests whatever is the cutting edge version so we can get notified of any incompatibility earlier

Copy link
Member Author

@Electronic-Waste Electronic-Waste Dec 16, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SGTM. The newest version of k8s is 1.32.0 now. WDYT👀 @kubeflow/wg-automl-leads

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the late response. We upgrade the Kube version step by step.
So, I think that it would be great to use the 1.32 in separate issue.

I believe that @shaowei-su can open a PR to update Kube to 1.31 after this PR.
Then you can open PR to support 4th supporting Kube version, which meant that we support 1.29, 1.30, 1.31, 1.32 versions the same as training-operator.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we discussed it today during the call.
Let's initially support 1.28 - 1.31 in Katib 0.18 as we discussed here: #2457.
In the future, we can migrate to k8s 1.32.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SGTM. Can we work on this PR and merge it as soon as possible? The deadline for Kubeflow 1.10 feature freeze is Jan 20th. We may need to reserve some time for 1.31 version:)

# Comma Delimited
experiments: ["darts-cpu"]
2 changes: 1 addition & 1 deletion .github/workflows/e2e-test-enas-cifar10.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,6 @@ jobs:
strategy:
fail-fast: false
matrix:
kubernetes-version: ["v1.27.11", "v1.28.7", "v1.29.2"]
kubernetes-version: ["v1.28.7", "v1.29.2", "v1.30.7"]
# Comma Delimited
experiments: ["enas-cpu"]
2 changes: 1 addition & 1 deletion .github/workflows/e2e-test-pytorch-mnist.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ jobs:
strategy:
fail-fast: false
matrix:
kubernetes-version: ["v1.27.11", "v1.28.7", "v1.29.2"]
kubernetes-version: ["v1.28.7", "v1.29.2", "v1.30.7"]
# Comma Delimited
experiments:
# suggestion-hyperopt
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/e2e-test-simple-pbt.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,6 @@ jobs:
fail-fast: false
matrix:
# Detail: https://hub.docker.com/r/kindest/node
kubernetes-version: ["v1.27.11", "v1.28.7", "v1.29.2"]
kubernetes-version: ["v1.28.7", "v1.29.2", "v1.30.7"]
# Comma Delimited
experiments: ["simple-pbt"]
2 changes: 1 addition & 1 deletion .github/workflows/e2e-test-tf-mnist-with-summaries.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,6 @@ jobs:
strategy:
fail-fast: false
matrix:
kubernetes-version: ["v1.27.11", "v1.28.7", "v1.29.2"]
kubernetes-version: ["v1.28.7", "v1.29.2", "v1.30.7"]
# Comma Delimited
experiments: ["tfjob-mnist-with-summaries"]
2 changes: 1 addition & 1 deletion .github/workflows/e2e-test-tune-api.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,4 +31,4 @@ jobs:
fail-fast: false
matrix:
# Detail: https://hub.docker.com/r/kindest/node
kubernetes-version: ["v1.27.11", "v1.28.7", "v1.29.2"]
kubernetes-version: ["v1.28.7", "v1.29.2", "v1.30.7"]
2 changes: 1 addition & 1 deletion .github/workflows/e2e-test-ui-random-search-postgres.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,4 +32,4 @@ jobs:
strategy:
fail-fast: false
matrix:
kubernetes-version: ["v1.27.11", "v1.28.7", "v1.29.2"]
kubernetes-version: ["v1.28.7", "v1.29.2", "v1.30.7"]
2 changes: 1 addition & 1 deletion .github/workflows/test-go.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ jobs:
fail-fast: false
matrix:
# Detail: `setup-envtest list`
kubernetes-version: ["1.27.1", "1.28.3", "1.29.3"]
kubernetes-version: ["1.28.3", "1.29.3", "1.30.0"]

# notifies that all test jobs are finished.
finish:
Expand Down
6 changes: 3 additions & 3 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ HAS_MOCKGEN := $(shell command -v mockgen;)
COMMIT := v1beta1-$(shell git rev-parse --short=7 HEAD)
KATIB_REGISTRY := docker.io/kubeflowkatib
CPU_ARCH ?= linux/amd64,linux/arm64
ENVTEST_K8S_VERSION ?= 1.29
ENVTEST_K8S_VERSION ?= 1.30
MOCKGEN_VERSION ?= $(shell grep 'go.uber.org/mock' go.mod | cut -d ' ' -f 2)
GO_VERSION=$(shell grep '^go' go.mod | cut -d ' ' -f 2)
GOPATH ?= $(shell go env GOPATH)
Expand All @@ -21,7 +21,7 @@ test: envtest

envtest:
ifndef HAS_SETUP_ENVTEST
go install sigs.k8s.io/controller-runtime/tools/setup-envtest@bf15e44028f908c790721fc8fe67c7bf2d06a611 #v0.17.3
go install sigs.k8s.io/controller-runtime/tools/setup-envtest@release-0.18
$(info "setup-envtest has been installed")
endif
$(info "setup-envtest has already installed")
Expand Down Expand Up @@ -82,7 +82,7 @@ sync-go-mod:
CONTROLLER_GEN = $(shell pwd)/bin/controller-gen
.PHONY: controller-gen
controller-gen:
@GOBIN=$(shell pwd)/bin GO111MODULE=on go install sigs.k8s.io/controller-tools/cmd/controller-gen@v0.14.0
@GOBIN=$(shell pwd)/bin GO111MODULE=on go install sigs.k8s.io/controller-tools/cmd/controller-gen@v0.16.5

# Run this if you update any existing controller APIs.
# 1. Generate deepcopy, clientset, listers, informers for the APIs (hack/update-codegen.sh)
Electronic-Waste marked this conversation as resolved.
Show resolved Hide resolved
Expand Down
2 changes: 1 addition & 1 deletion cmd/db-manager/v1beta1/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ import (
api_pb "github.com/kubeflow/katib/pkg/apis/manager/v1beta1"
db "github.com/kubeflow/katib/pkg/db/v1beta1"
"github.com/kubeflow/katib/pkg/db/v1beta1/common"
"k8s.io/klog"
"k8s.io/klog/v2"

"google.golang.org/grpc"
"google.golang.org/grpc/reflection"
Expand Down
2 changes: 1 addition & 1 deletion cmd/metricscollector/v1beta1/file-metricscollector/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ import (
psutil "github.com/shirou/gopsutil/v3/process"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials/insecure"
"k8s.io/klog"
"k8s.io/klog/v2"

commonv1beta1 "github.com/kubeflow/katib/pkg/apis/controller/common/v1beta1"
api "github.com/kubeflow/katib/pkg/apis/manager/v1beta1"
Expand Down
2 changes: 1 addition & 1 deletion cmd/suggestion/goptuna/v1beta1/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ import (
api_v1_beta1 "github.com/kubeflow/katib/pkg/apis/manager/v1beta1"
suggestion "github.com/kubeflow/katib/pkg/suggestion/v1beta1/goptuna"
"google.golang.org/grpc"
"k8s.io/klog"
"k8s.io/klog/v2"
)

const (
Expand Down
2 changes: 1 addition & 1 deletion examples/v1beta1/kind-cluster/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Install the following tools to run the example:

- [Docker](https://docs.docker.com/get-docker) >= 20.10
- [Kind](https://kind.sigs.k8s.io/docs/user/quick-start/#installation) >= 0.13
- [`kubectl`](https://kubernetes.io/docs/tasks/tools/#kubectl) >= 1.27
- [`kubectl`](https://kubernetes.io/docs/tasks/tools/#kubectl) >= 1.28

## Installation

Expand Down
81 changes: 39 additions & 42 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
module github.com/kubeflow/katib

go 1.22
go 1.22.0

toolchain go1.23.2
Electronic-Waste marked this conversation as resolved.
Show resolved Hide resolved

require (
github.com/DATA-DOG/go-sqlmock v1.5.0
Expand All @@ -14,29 +16,28 @@ require (
github.com/lib/pq v1.10.6
github.com/mattbaird/jsonpatch v0.0.0-20171005235357-81af80346b1a
github.com/nxadm/tail v1.4.11
github.com/onsi/gomega v1.30.0
github.com/open-policy-agent/cert-controller v0.10.1
github.com/prometheus/client_golang v1.18.0
github.com/onsi/gomega v1.34.1
github.com/open-policy-agent/cert-controller v0.11.0
github.com/prometheus/client_golang v1.19.1
github.com/shirou/gopsutil/v3 v3.22.5
github.com/spf13/viper v1.9.0
github.com/tidwall/gjson v1.14.1
go.uber.org/mock v0.4.0
google.golang.org/grpc v1.58.3
google.golang.org/protobuf v1.33.0
k8s.io/api v0.29.3
k8s.io/apimachinery v0.29.3
k8s.io/client-go v0.29.3
k8s.io/code-generator v0.29.3
k8s.io/klog v1.0.0
k8s.io/kube-openapi v0.0.0-20231010175941-2dd684a91f00
sigs.k8s.io/controller-runtime v0.17.3
google.golang.org/protobuf v1.34.1
k8s.io/api v0.30.7
k8s.io/apimachinery v0.30.7
k8s.io/client-go v0.30.7
k8s.io/code-generator v0.30.7
k8s.io/klog/v2 v2.130.1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reverting klog version, then create dedicated issue for the klog version updating.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it

k8s.io/kube-openapi v0.0.0-20240430033511-f0e62f92d13f
sigs.k8s.io/controller-runtime v0.18.5
sigs.k8s.io/structured-merge-diff/v4 v4.4.1
sigs.k8s.io/yaml v1.4.0
)

require (
cloud.google.com/go/compute v1.21.0 // indirect
cloud.google.com/go/compute/metadata v0.2.3 // indirect
cloud.google.com/go/compute/metadata v0.3.0 // indirect
github.com/Azure/azure-sdk-for-go v68.0.0+incompatible // indirect
github.com/Azure/go-autorest v14.2.0+incompatible // indirect
github.com/Azure/go-autorest/autorest v0.11.29 // indirect
Expand All @@ -62,7 +63,7 @@ require (
github.com/aws/smithy-go v1.13.5 // indirect
github.com/awslabs/amazon-ecr-credential-helper/ecr-login v0.0.0-20230510185313-f5e39e5f34c7 // indirect
github.com/beorn7/perks v1.0.1 // indirect
github.com/cespare/xxhash/v2 v2.2.0 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/chrismellard/docker-credential-acr-env v0.0.0-20230304212654-82a0ddb27589 // indirect
github.com/containerd/stargz-snapshotter/estargz v0.14.3 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
Expand All @@ -71,24 +72,24 @@ require (
github.com/docker/distribution v2.8.2+incompatible // indirect
github.com/docker/docker v26.1.5+incompatible // indirect
github.com/docker/docker-credential-helpers v0.7.0 // indirect
github.com/emicklei/go-restful/v3 v3.11.0 // indirect
github.com/emicklei/go-restful/v3 v3.12.0 // indirect
github.com/evanphx/json-patch v5.6.0+incompatible // indirect
github.com/evanphx/json-patch/v5 v5.8.0 // indirect
github.com/evanphx/json-patch/v5 v5.9.0 // indirect
github.com/fsnotify/fsnotify v1.7.0 // indirect
github.com/go-logr/logr v1.4.1 // indirect
github.com/go-logr/zapr v1.3.0 // indirect
github.com/go-ole/go-ole v1.2.6 // indirect
github.com/go-openapi/jsonpointer v0.19.6 // indirect
github.com/go-openapi/jsonreference v0.20.2 // indirect
github.com/go-openapi/swag v0.22.3 // indirect
github.com/go-openapi/jsonpointer v0.21.0 // indirect
github.com/go-openapi/jsonreference v0.21.0 // indirect
github.com/go-openapi/swag v0.23.0 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang-jwt/jwt/v4 v4.5.1 // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/golang/protobuf v1.5.4 // indirect
github.com/google/gnostic-models v0.6.8 // indirect
github.com/google/go-containerregistry/pkg/authn/kubernetes v0.0.0-20230516205744-dbecb1de8cfa // indirect
github.com/google/gofuzz v1.2.0 // indirect
github.com/google/uuid v1.3.0 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/hashicorp/hcl v1.0.0 // indirect
github.com/imdario/mergo v0.3.12 // indirect
github.com/jmespath/go-jmespath v0.4.0 // indirect
Expand All @@ -98,7 +99,6 @@ require (
github.com/lufia/plan9stats v0.0.0-20211012122336-39d0f177ccd0 // indirect
github.com/magiconair/properties v1.8.5 // indirect
github.com/mailru/easyjson v0.7.7 // indirect
github.com/matttproud/golang_protobuf_extensions/v2 v2.0.0 // indirect
github.com/mitchellh/go-homedir v1.1.0 // indirect
github.com/mitchellh/mapstructure v1.4.2 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
Expand All @@ -109,9 +109,9 @@ require (
github.com/pelletier/go-toml v1.9.4 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/power-devops/perfstat v0.0.0-20210106213030-5aafc221ea8c // indirect
github.com/prometheus/client_model v0.5.0 // indirect
github.com/prometheus/common v0.45.0 // indirect
github.com/prometheus/procfs v0.12.0 // indirect
github.com/prometheus/client_model v0.6.1 // indirect
github.com/prometheus/common v0.53.0 // indirect
github.com/prometheus/procfs v0.15.0 // indirect
github.com/sirupsen/logrus v1.9.1 // indirect
github.com/spf13/afero v1.6.0 // indirect
github.com/spf13/cast v1.4.1 // indirect
Expand All @@ -127,30 +127,27 @@ require (
go.uber.org/atomic v1.11.0 // indirect
go.uber.org/multierr v1.11.0 // indirect
go.uber.org/zap v1.26.0 // indirect
golang.org/x/crypto v0.21.0 // indirect
golang.org/x/exp v0.0.0-20220722155223-a9213eeb770e // indirect
golang.org/x/mod v0.14.0 // indirect
golang.org/x/net v0.23.0 // indirect
golang.org/x/oauth2 v0.12.0 // indirect
golang.org/x/sync v0.5.0 // indirect
golang.org/x/sys v0.18.0 // indirect
golang.org/x/term v0.18.0 // indirect
golang.org/x/text v0.14.0 // indirect
golang.org/x/time v0.3.0 // indirect
golang.org/x/tools v0.16.1 // indirect
golang.org/x/crypto v0.25.0 // indirect
golang.org/x/exp v0.0.0-20240719175910-8a7402abbf56 // indirect
golang.org/x/mod v0.19.0 // indirect
golang.org/x/net v0.27.0 // indirect
golang.org/x/oauth2 v0.20.0 // indirect
golang.org/x/sync v0.7.0 // indirect
golang.org/x/sys v0.22.0 // indirect
golang.org/x/term v0.22.0 // indirect
golang.org/x/text v0.16.0 // indirect
golang.org/x/time v0.5.0 // indirect
golang.org/x/tools v0.23.0 // indirect
gomodules.xyz/jsonpatch/v2 v2.4.0 // indirect
gonum.org/v1/gonum v0.8.2 // indirect
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20230822172742-b8732ec3820d // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/ini.v1 v1.63.2 // indirect
gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
k8s.io/apiextensions-apiserver v0.29.2 // indirect
k8s.io/component-base v0.29.2 // indirect
k8s.io/gengo v0.0.0-20230829151522-9cce18d56c01 // indirect
k8s.io/klog/v2 v2.110.1 // indirect
k8s.io/utils v0.0.0-20230726121419-3b25d923346b // indirect
k8s.io/apiextensions-apiserver v0.30.3 // indirect
k8s.io/gengo/v2 v2.0.0-20240228010128-51d4e06bde70 // indirect
k8s.io/utils v0.0.0-20240502163921-fe8a2dddb1d0 // indirect
sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd // indirect
)
Loading
Loading