Merge pull request #1519 from roycaihw/release18

Pull master into release-18.0

Author: k8s-ci-robot

.github/workflows/e2e-master.yaml

@@ -19,7 +19,7 @@ jobs:
       with:
         submodules: true
     - name: Create Kind Cluster
-      uses: helm/kind-action@v1.1.0
+      uses: helm/kind-action@v1.2.0
       with:
         cluster_name: kubernetes-python-e2e-master-${{ matrix.python-version }}
         # The kind version to be used to spin the cluster up

.github/workflows/e2e-release-11.0.yaml

@@ -19,7 +19,7 @@ jobs:
       with:
         submodules: true
     - name: Create Kind Cluster
-      uses: helm/kind-action@v1.1.0
+      uses: helm/kind-action@v1.2.0
       with:
         cluster_name: kubernetes-python-e2e-release-11.0-${{ matrix.python-version }}
         # The kind version to be used to spin the cluster up

.github/workflows/e2e-release-12.0.yaml

@@ -19,7 +19,7 @@ jobs:
       with:
         submodules: true
     - name: Create Kind Cluster
-      uses: helm/kind-action@v1.1.0
+      uses: helm/kind-action@v1.2.0
       with:
         cluster_name: kubernetes-python-e2e-release-12.0-${{ matrix.python-version }}
         # The kind version to be used to spin the cluster up

.github/workflows/e2e-release-17.0.yaml

@@ -19,7 +19,7 @@ jobs:
       with:
         submodules: true
     - name: Create Kind Cluster
-      uses: helm/kind-action@v1.1.0
+      uses: helm/kind-action@v1.2.0
       with:
         cluster_name: kubernetes-python-e2e-release-17.0-${{ matrix.python-version }}
         # The kind version to be used to spin the cluster up

.github/workflows/e2e-release-18.0.yaml

@@ -19,7 +19,7 @@ jobs:
       with:
         submodules: true
     - name: Create Kind Cluster
-      uses: helm/kind-action@v1.1.0
+      uses: helm/kind-action@v1.2.0
       with:
         cluster_name: kubernetes-python-e2e-release-18.0-${{ matrix.python-version }}
         # The kind version to be used to spin the cluster up

CHANGELOG.md

@@ -1,3 +1,16 @@
+# v18.0.0-snapshot
+
+Kubernetes API Version: To Be Updated
+
+### Feature
+- Support for the dryRun parameter has been added to the dynamic client. ([kubernetes-client/python-base#247](https://github.com/kubernetes-client/python-base/pull/247), [@gravesm](https://github.com/gravesm))
+- The `python2` support will be removed in 18.0.0 beta release. All the tests will use `python3` versions. ([kubernetes-client/python-base#238](https://github.com/kubernetes-client/python-base/pull/238), [@Priyankasaggu11929](https://github.com/Priyankasaggu11929))
+- The dynamic client now supports customizing http "Accept" header through the `header_params` parameter, which can be used to customizing API server response, e.g. retrieving object metadata only. ([kubernetes-client/python-base#236](https://github.com/kubernetes-client/python-base/pull/236), [@Yashks1994](https://github.com/Yashks1994))
+- Allow optional list of YAML objects as param to `create_from_yaml` util [kubernetes-client/python#1403](https://github.com/kubernetes-client/python/pull/1403)
+
+**Bug Fix:**
+- Fix empty yaml document in `create_from_yaml` [kubernetes-client/python#1506](https://github.com/kubernetes-client/python/pull/1506)
+
 # v18.20.0b1
 
 Kubernetes API Version: 1.18.20

examples/pod_namespace_watch.py renamed to examples/watch/pod_namespace_watch.py

@@ -17,6 +17,12 @@
 The script will wait for 10 events related to namespaces to occur within
 the `timeout_seconds` threshold and then move on to wait for another 10 events
 related to pods to occur within the `timeout_seconds` threshold.
+
+
+Refer to the document below to understand the server-side & client-side
+timeout settings for the watch request handler: ~
+
+https://github.com/github.com/kubernetes-client/python/blob/master/examples/watch/timeout-settings.md
 """
 
 from kubernetes import client, config, watch

examples/watch/timeout-settings.md

@@ -0,0 +1,71 @@
+
+**This documentation briefly provides information about the `server side` & `client side` connection timeout settings, in the watch request handler.**
+
+---
+
+There are two inputs available in the client, that could be used to set connection timeouts:
+
+- `timeout_seconds`
+- `_request_timeout`
+
+---
+
+#### Sever-side timeout (`kwargs['timeout_seconds'] = n`)
+
+- The value of the argument `timeout_seconds`, **n**, (which is time duration in seconds) is consumed at the server side. It is included in the request URL to the server. 
+  
+  *For eg.* ~ `https://localhost:6443/api/v1/namespaces/default/pods?labelSelector=app%3Ddemo&timeoutSeconds=100&watch=True`
+
+- In case, if the `timeout_seconds` value is set, the value `n` would determine the server-side connection timeout duration.
+
+  *For eg.* ~ if `kwargs['timeout_seconds'] = 3600`, then the server-side connection timeout will be equal to 1 hour.
+  
+  This timeout duration is determined by the expression ~ `timeout = time.Duration(3600) * time.seconds`, *i.e.* `timeout = 1 hour`
+
+  ***Refer:*** 
+  - *[https://github.com/kubernetes/apiserver/blob/release-1.20/pkg/endpoints/handlers/get.go#L254](https://github.com/kubernetes/apiserver/blob/release-1.20/pkg/endpoints/handlers/get.go#L254)*
+
+- In case, if the `timeout_seconds` value is not set, then the connection timeout will be a randomized value (in seconds) between `minRequestTimeout` and 2*`minRequestTimeout`, to spread out the load.
+
+  It is determined using the expression ~ `timeout = time.Duration(float64(minRequestTimeout) * (rand.Float64() + 1.0))`
+
+  Where `minRequestTimeout` indicates the minimum number of seconds a handler must keep a request open before timing it out.
+  
+  The default value of `minRequestTimeout` is 1800 seconds.
+
+  ***Refer:***
+  - *[https://github.com/kubernetes/apiserver/blob/release-1.20/pkg/endpoints/handlers/get.go#L257](https://github.com/kubernetes/apiserver/blob/release-1.20/pkg/endpoints/handlers/get.go#L257)*
+  - *[https://github.com/kubernetes/kubernetes/blob/release-1.20/staging/src/k8s.io/apiserver/pkg/server/config.go#L320](https://github.com/kubernetes/kubernetes/blob/release-1.20/staging/src/k8s.io/apiserver/pkg/server/config.go#L320)*
+
+- In case of a network outage, the server side timeout value will have no effect & the client will hang indefinitely without raising any exception. Note, that this is the case provided when there is no other client-side timeout (i.e., `_request_timeout`) value specified.
+
+  (*See the section below for information on `client side timeout`*)
+
+- It is recommended to set this timeout value to a higher number such as 3600 seconds (1 hour).
+
+---
+
+#### Client-side timeout (`kwargs['_request_timeout'] = n`)
+
+- The value of the argument `_request_timeout`, **n** (which is time duration in seconds) is set to the socket used for the connection.
+
+- In case, if the `_request_timeout` value is set, this argument can accept 2 types of input values ~
+    - float, 
+    - a tuple (with a length of 2)
+    
+  ***Refer***
+  - *[https://github.com/kubernetes-client/python/blob/v17.17.0/kubernetes/client/api_client.py#L336-L339](https://github.com/kubernetes-client/python/blob/v17.17.0/kubernetes/client/api_client.py#L336-L339)*
+
+- In case of network outage, leading to dropping all packets with no RST/FIN, the timeout value (in seconds) determined by the `request_timeout` argument, would be the time duration for how long the client will wait before dropping the connection.
+
+- When the timeout happens, an exception will be raised, for eg. ~
+  
+  `urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='localhost', port=6443): Read timed out.`
+  
+- In case, if the `_request_timeout` value is not set, then the default value is **`None`** & socket will have no timeout.
+
+  ***Refer:***
+  - *[https://docs.python.org/3/library/socket.html#socket.getdefaulttimeout](https://docs.python.org/3/library/socket.html#socket.getdefaulttimeout)*
+
+- It is recommended to set this timeout value to a lower number (for eg. ~ maybe 60 seconds).
+

kubernetes/base

@@ -99,9 +99,6 @@ def sources(self, sources):
         :param sources: The sources of this V1ProjectedVolumeSource.  # noqa: E501
         :type: list[V1VolumeProjection]
         """
-        if self.local_vars_configuration.client_side_validation and sources is None:  # noqa: E501
-            raise ValueError("Invalid value for `sources`, must not be `None`")  # noqa: E501
-
         self._sources = sources
 
     def to_dict(self):

kubernetes/client/models/v1_projected_volume_source.py

@@ -13,8 +13,10 @@
 # under the License.
 
 import unittest
+from os import path
 
 import yaml
+
 from kubernetes import utils, client
 from kubernetes.client.rest import ApiException
 from kubernetes.e2e_test import base
@@ -37,6 +39,7 @@ def tearDownClass(cls):
         k8s_client = client.api_client.ApiClient(configuration=cls.config)
         core_v1 = client.CoreV1Api(api_client=k8s_client)
         core_v1.delete_namespace(name=cls.test_namespace)
+
     # Tests for creating individual API objects
 
     def test_create_apps_deployment_from_yaml(self):
@@ -59,6 +62,31 @@ def test_create_apps_deployment_from_yaml(self):
             except ApiException:
                 continue
 
+    def test_create_apps_deployment_from_yaml_object(self):
+        """
+        Should be able to pass YAM objects directly to helper function.
+        """
+        k8s_client = client.api_client.ApiClient(configuration=self.config)
+        _path = self.path_prefix + "apps-deployment.yaml"
+        with open(path.abspath(_path)) as f:
+            yaml_objects = yaml.safe_load_all(f)
+            utils.create_from_yaml(
+                k8s_client,
+                yaml_objects=yaml_objects,
+            )
+        app_api = client.AppsV1Api(k8s_client)
+        dep = app_api.read_namespaced_deployment(name="nginx-app",
+                                                 namespace="default")
+        self.assertIsNotNone(dep)
+        while True:
+            try:
+                app_api.delete_namespaced_deployment(
+                    name="nginx-app", namespace="default",
+                    body={})
+                break
+            except ApiException:
+                continue
+
     def test_create_apps_deployment_from_yaml_obj(self):
         k8s_client = client.api_client.ApiClient(configuration=self.config)
         with open(self.path_prefix + "apps-deployment.yaml") as f:

kubernetes/e2e_test/test_utils.py

@@ -26,7 +26,8 @@
 
 def create_from_yaml(
         k8s_client,
-        yaml_file,
+        yaml_file=None,
+        yaml_objects=None,
         verbose=False,
         namespace="default",
         **kwargs):
@@ -36,6 +37,8 @@ def create_from_yaml(
     Input:
     yaml_file: string. Contains the path to yaml file.
     k8s_client: an ApiClient object, initialized with the client args.
+    yaml_objects: List[dict]. Optional list of YAML objects; used instead
+        of reading the `yaml_file`. Default is None.
     verbose: If True, print confirmation from the create action.
         Default is False.
     namespace: string. Contains the namespace to create all
@@ -62,12 +65,13 @@ def create_from_yaml(
         FailToCreateError which holds list of `client.rest.ApiException`
         instances for each object that failed to create.
     """
-    with open(path.abspath(yaml_file)) as f:
-        yml_document_all = yaml.safe_load_all(f)
 
+    def create_with(objects):
         failures = []
         k8s_objects = []
-        for yml_document in yml_document_all:
+        for yml_document in objects:
+            if yml_document is None:
+                continue
             try:
                 created = create_from_dict(k8s_client, yml_document, verbose,
                                            namespace=namespace,
@@ -77,9 +81,19 @@ def create_from_yaml(
                 failures.extend(failure.api_exceptions)
         if failures:
             raise FailToCreateError(failures)
-
         return k8s_objects
 
+    if yaml_objects:
+        yml_document_all = yaml_objects
+        return create_with(yml_document_all)
+    elif yaml_file:
+        with open(path.abspath(yaml_file)) as f:
+            yml_document_all = yaml.safe_load_all(f)
+            return create_with(yml_document_all)
+    else:
+        raise ValueError(
+            'One of `yaml_file` or `yaml_objects` arguments must be provided')
+
 
 def create_from_dict(k8s_client, data, verbose=False, namespace='default',
                      **kwargs):

kubernetes/utils/create_from_yaml.py

@@ -74,4 +74,17 @@ else
     exit 1
 fi;
 
+# Patching commits for Tolerating Null Sources on Projected Volumes
+# TODO: remove this patch when we release v20 clients
+# Ref: https://github.com/kubernetes-client/python/pull/1497
+git cherry-pick -n f3dbc8cbf1ab2aaf5e3bd8c0f0fc068e67823971
+if [ $? -eq 0 ]
+then
+    echo Succesfully patched changes for Tolerating Null Sources on Projected Volumes
+else
+    echo Failed to patch changes for Tolerating Null Sources on Projected Volumes
+    git restore --staged .
+    exit 1
+fi;
+
 git commit -m "Apply hotfixes"