help request: service health check interval

[hgaong]

Description

service

{
  "id": "557101231575663294",
  "create_time": 1741587810,
  "update_time": 1742293721,
  "name": "test-api",
  "upstream": {
    "nodes": [
      {
        "host": "172.31.19.1",
        "port": 80,
        "weight": 100
      },
      {
        "host": "172.31.20.9",
        "port": 80,
        "weight": 100
      }
    ],
    "retries": 0,
    "timeout": {
      "connect": 30,
      "send": 30,
      "read": 30
    },
    "type": "roundrobin",
    "checks": {
      "active": {
        "concurrency": 2,
        "healthy": {
          "http_statuses": [
            200,
            302,
            301,
            404
          ],
          "interval": 1,
          "successes": 2
        },
        "http_path": "/apisix/hearbeat",
        "port": 80,
        "timeout": 1,
        "type": "http",
        "unhealthy": {
          "http_failures": 2,
          "http_statuses": [
            429,
            500,
            501,
            502,
            503,
            504,
            505
          ],
          "interval": 1,
          "tcp_failures": 2,
          "timeouts": 3
        }
      }
    },
    "scheme": "http",
    "pass_host": "pass",
    "keepalive_pool": {
      "idle_timeout": 60,
      "requests": 1000,
      "size": 640
    }
  },
  "plugins": {
    "ip-restriction": {
      "_meta": {
        "disable": true
      },
      "blacklist": [
        "108.136.33.21"
      ]
    },
    "limit-req": {
      "_meta": {
        "disable": true
      },
      "burst": 0,
      "key": "http_x_forwarded_for",
      "key_type": "var",
      "rate": 10,
      "rejected_code": 429,
      "rejected_msg": "ffffff"
    },
    "ua-restriction": {
      "_meta": {
        "disable": true
      },
      "bypass_missing": false,
      "denylist": [
        "curl"
      ]
    }
  }
} 

Then I configured around 600 routes, each of which referenced this service. The observed situation is that each node of the upstream service has about 20 heartbeat requests per second, which is inconsistent with the configured interval of 1 second. why ?

Environment

• APISIX version (run apisix version):3.11
• Operating system (run uname -a):
• OpenResty / Nginx version (run openresty -V or nginx -V):
• etcd version, if relevant (run curl http://127.0.0.1:9090/v1/server_info):
• APISIX Dashboard version, if relevant:
• Plugin runner version, for issues related to plugin runners:
• LuaRocks version, for installation issues (run luarocks --version):

[Baoyuantop]

The observed situation is that each node of the upstream service has about 20 heartbeat requests per second

How did you tally this information? Maybe try adding upstream.checks.active.req_headers configuration to make sure all requests are issued by APISIX's health checks

[hgaong]

The observed situation is that each node of the upstream service has about 20 heartbeat requests per second

How did you tally this information? Maybe try adding upstream.checks.active.req_headers configuration to make sure all requests are issued by APISIX's health checks

Through SkyWalking and the logs of my service, I have counted 20 heartbeat requests per second. It can be confirmed that only APISix will call this path "http_path": "/apisix/heartbeat".

[Baoyuantop]

Can you provide the configuration file for APISIX? It would help to reproduce the problem.

[hgaong]

Can you provide the configuration file for APISIX? It would help to reproduce the problem.
config.yaml

apisix:
  node_listen: 9080              # APISIX listening port
  enable_ipv6: false

deployment:
  admin:
    allow_admin:               # https://nginx.org/en/docs/http/ngx_http_access_module.html#allow
      - 0.0.0.0/0              # We need to restrict ip access rules for security. 0.0.0.0/0 is for test.

    admin_key:
      - name: "admin"
        key: secret                 # apikey
        role: admin                 # admin: manage all configuration data

  etcd:
    host:                           # it's possible to define multiple etcd hosts addresses of the same etcd cluster.
      - "http://etcd:2379"          # multiple etcd address
    prefix: "/apisix"               # apisix configurations prefix
    timeout: 30                     # 30 seconds

plugin_attr:
  skywalking:
    service_name: APISIX
    service_instance_name: "APISIX Instance Name"
    endpoint_addr: http://sw:12800