[Cilium/ RKE2] Unstable connection inside cluster

[huy105]

Environmental Info:
RKE2 Version: v1.30.6+rke2r1
Cilium v1.16.2

Node(s) CPU architecture, OS, and Version:
OS: RHEL 8
cpu: amd64
Virtual environment: VMware

Cluster Configuration:
1 master, 10 agent

Describe the bug:
Unstable connection when check services with curl command

Steps To Reproduce:

• Installed K8s Nginx controller: v1.11.3,
• Nginx pod schedule on node for example A, B, then expose with NodePort service at 30080, 30443.
• Execute command "curl -v node-ip:30443" (node-ip is any node ip of cluster)

Expected behavior:
Connect to service success

Actual behavior:
Sometime the connection is success, sometime the result is connection timeout
The connection regularly fails only on node which scheduled nginx pod (for example A, B), except the node which pod nginx doesn't schedule on

Additional context / logs:
Log doesn't show anything when connection is timeout.

[dereknola]

Without a reliable way to reproduce this, or with some amount of logging, resolving this issue will be difficult.

[brandond]

Have you confirmed that host firewalls (firewalld/ufw) are disabled on all nodes?