Skip to content

OCPBUGS50970 Configuring maxParallelImagePulls value using KubeletConfig in RHOCP 4 #95384

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
133 changes: 133 additions & 0 deletions modules/nodes-nodes-max-image-pulls.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
// Module included in the following assemblies:
//
// * nodes/nodes/nodes-nodes-managing.adoc

:_mod-docs-content-type: PROCEDURE
[id="nodes-nodes-max-image-pulls_{context}"]

= Configuring parallel container image pulls

To help control bandwidth issues, you can configure the number of workload images that can be pulled at the same time.

By default, the cluster can pull images in parallel, which allows multiple workloads to pull images at the same time. Pulling multiple images in parallel can improve workload start up time, by allowing your workloads to pull needed images without waiting. Also, updates However, pulling too many images at once can use excessive network bandwidth and cause latency issues throughout your cluster.

You can configure the maximum number of images that can be pulled in parallel or force serial image pulling, which allows only one image to be pulled at a time.

To control the number of images that can be pulled simultaneously, you can use a kubelet config to set the `maxParallelImagePulls` to specify a limit. Additional image pulls above this limit are held until one of the current pulls is complete.

To force serial image pulls, use a kubelet config to set `serializeImagePulls` field to `true`.

.Prerequisites

* You have a running {product-title} cluster.

* You are logged in to the cluster as a user with administrative privileges.

.Procedure

. Apply a custom label to the machine config pool where you want to configure parallel pulls by running a command similar to the following.
+
[source,terminal]
----
$ oc label machineconfigpool <mcp_name> parallel-pulls=set
----

. Create a custom resource (CR) to enable and configure parallel image pulling.
+
[source,yaml]
----
apiVersion: machineconfiguration.openshift.io/v1
kind: KubeletConfig
metadata:
name: parallel-image-pulls
# ...
spec:
machineConfigPoolSelector:
matchLabels:
parallel-pulls: set
kubeletConfig:
serializeImagePulls: false <1>
maxParallelImagePulls: 3 <2>
# ...
----
<1> Set to `false` to enable parallel image pulls. Set to `true` to force serial image pulling. The default is `false`.
<2> Specify the maximum number of images that can be pulled in parallel.

. Create the new machine config by running a command similar to the following:
+
[source,terminal]
----
$ oc create -f <file_name>.yaml
----

.Verification

. Check the machine configs to see that a new one was added by running the following command:
+
[source,terminal]
----
$ oc get MachineConfig
----
+
.Example output
+
[source,terminal]
----
NAME GENERATEDBYCONTROLLER IGNITIONVERSION AGE
00-master 70025364a114fc3067b2e82ce47fdb0149630e4b 3.5.0 133m
00-worker 70025364a114fc3067b2e82ce47fdb0149630e4b 3.5.0 133m
# ...
99-parallel-generated-kubelet 70025364a114fc3067b2e82ce47fdb0149630e4b 3.5.0 15s <1>
# ...
rendered-parallel-c634a80f644740974ceb40c054c79e50 70025364a114fc3067b2e82ce47fdb0149630e4b 3.5.0 10s <2>
----
<1> The new machine config. In this example, the machine config is for the `parallel` custom machine config pool.
<2> The new rendered machine config. In this example, the machine config is for the `parallel` custom machine config pool.

. Check to see that the nodes in the `parallel` machine config pool are being updated by running the following command:
+
[source,terminal]
----
$ oc get machineconfigpool
----
+
.Example output
+
[source,terminal]
----
NAME CONFIG UPDATED UPDATING DEGRADED MACHINECOUNT READYMACHINECOUNT UPDATEDMACHINECOUNT DEGRADEDMACHINECOUNT AGE
parallel rendered-parallel-3904f0e69130d125b3b5ef0e981b1ce1 False True False 1 0 0 0 65m
master rendered-master-7536834c197384f3734c348c1d957c18 True False False 3 3 3 0 140m
worker rendered-worker-c634a80f644740974ceb40c054c79e50 True False False 2 2 2 0 140m
----

. When the nodes are updated, check that the parallel pull maximum was configured:

.. Open an `oc debug` session to a node by running a command similar to the following:
+
[source,terminal]
----
$ oc debug node/<node_name>
----

.. Set `/host` as the root directory within the debug shell by running the following command:
+
[source,terminal]
----
sh-5.1# chroot /host
----

.. Examine the `kubelet.conf` file by running the following command:
+
[source,terminal]
----
sh-5.1# cat /etc/kubernetes/kubelet.conf | grep -i maxParallelImagePulls
----
+
.Example output
+
[source,terminal]
----
maxParallelImagePulls: 3
----

1 change: 1 addition & 0 deletions nodes/nodes/nodes-nodes-managing.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ include::modules/nodes-nodes-rtkernel-arguments.adoc[leveloffset=+1]
endif::openshift-webscale[]

include::modules/nodes-nodes-swap-memory.adoc[leveloffset=+1]
include::modules/nodes-nodes-max-image-pulls.adoc[leveloffset=+1]
include::modules/nodes-control-plane-osp-migrating.adoc[leveloffset=+1]
[role="_additional-resources"]
.Additional resources
Expand Down