🌱 Add metrics to Runtime SDK hook client

[chrischdi]

What this PR does / why we need it:

Adds request related metrics to the Runtime SDK hook client which will be helpful to analyse request durations to Runtime SDK extensions.

Adds the following metrics:

1. capi_runtime_sdk_request_latency_seconds, example:

# HELP capi_runtime_sdk_request_duration_seconds Request duration in seconds, broken down by hook and host.
# TYPE capi_runtime_sdk_request_duration_seconds histogram
capi_runtime_sdk_request_duration_seconds_bucket{group="hooks.runtime.cluster.x-k8s.io",hook="Discovery",host="webhook-service.test-extension.svc:443",version="v1alpha1",le="0.001"} 0
capi_runtime_sdk_request_duration_seconds_bucket{group="hooks.runtime.cluster.x-k8s.io",hook="Discovery",host="webhook-service.test-extension.svc:443",version="v1alpha1",le="0.002"} 0
capi_runtime_sdk_request_duration_seconds_bucket{group="hooks.runtime.cluster.x-k8s.io",hook="Discovery",host="webhook-service.test-extension.svc:443",version="v1alpha1",le="0.004"} 0
capi_runtime_sdk_request_duration_seconds_bucket{group="hooks.runtime.cluster.x-k8s.io",hook="Discovery",host="webhook-service.test-extension.svc:443",version="v1alpha1",le="0.008"} 2
capi_runtime_sdk_request_duration_seconds_bucket{group="hooks.runtime.cluster.x-k8s.io",hook="Discovery",host="webhook-service.test-extension.svc:443",version="v1alpha1",le="0.016"} 2
capi_runtime_sdk_request_duration_seconds_bucket{group="hooks.runtime.cluster.x-k8s.io",hook="Discovery",host="webhook-service.test-extension.svc:443",version="v1alpha1",le="0.032"} 3
capi_runtime_sdk_request_duration_seconds_bucket{group="hooks.runtime.cluster.x-k8s.io",hook="Discovery",host="webhook-service.test-extension.svc:443",version="v1alpha1",le="0.064"} 3
capi_runtime_sdk_request_duration_seconds_bucket{group="hooks.runtime.cluster.x-k8s.io",hook="Discovery",host="webhook-service.test-extension.svc:443",version="v1alpha1",le="0.128"} 3
capi_runtime_sdk_request_duration_seconds_bucket{group="hooks.runtime.cluster.x-k8s.io",hook="Discovery",host="webhook-service.test-extension.svc:443",version="v1alpha1",le="0.256"} 3
capi_runtime_sdk_request_duration_seconds_bucket{group="hooks.runtime.cluster.x-k8s.io",hook="Discovery",host="webhook-service.test-extension.svc:443",version="v1alpha1",le="0.512"} 3
capi_runtime_sdk_request_duration_seconds_bucket{group="hooks.runtime.cluster.x-k8s.io",hook="Discovery",host="webhook-service.test-extension.svc:443",version="v1alpha1",le="+Inf"} 3
capi_runtime_sdk_request_duration_seconds_sum{group="hooks.runtime.cluster.x-k8s.io",hook="Discovery",host="webhook-service.test-extension.svc:443",version="v1alpha1"} 0.028844958
capi_runtime_sdk_request_duration_seconds_count{group="hooks.runtime.cluster.x-k8s.io",hook="Discovery",host="webhook-service.test-extension.svc:443",version="v1alpha1"} 3

2. capi_runtime_sdk_requests_total example:

# HELP capi_runtime_sdk_requests_total Number of HTTP requests, partitioned by status code, host and hook.
# TYPE capi_runtime_sdk_requests_total counter
capi_runtime_sdk_requests_total{code="200",group="hooks.runtime.cluster.x-k8s.io",hook="Discovery",host="webhook-service.test-extension.svc:443",version="v1alpha1"} 3

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Part of #6330

Additional info

Some prior art from client-go:

• rest_client_ request_latency_seconds
• rest_client_requests_total

[killianmuldoon]

This is great - using it now to get a feel for how the e2e tests run!

Just a couple of nits around comments.

[sbueringer]

General approach looks fine. Mostly questions/nits as I don't really remember the metric best practices.

[sbueringer]

I would take another look once CI is green or ping me otherwise if you want another review :)

[chrischdi]

/retitle 🌱 Add metrics to Runtime SDK hook client

[sbueringer]

/lgtm

/assign @fabriziopandini

[sbueringer]

@ykakarap @killianmuldoon If you have some time, would be good if you can review this PR

[killianmuldoon]

/lgtm

[ykakarap]

@ykakarap @killianmuldoon If you have some time, would be good if you can review this PR

Will take a look at this today :)

[ykakarap]

/lgtm

I am not very familiar with metric and practices to follow. But the overall approach here looks good.

[sbueringer]

/lgtm
/approve
/cherry-pick release-1.2

[k8s-infra-cherrypick-robot]

@sbueringer: once the present PR merges, I will cherry-pick it on top of release-1.2 in a new PR and assign it to you.

In response to this:

/lgtm
/approve
/cherry-pick release-1.2

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[sbueringer]

@chrischdi please rebase when you've got some time :)

[k8s-ci-robot]

@chrischdi: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-cluster-api-e2e-informing-main	6ffd863	link	false	/test pull-cluster-api-e2e-informing-main

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[sbueringer]

/lgtm
/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: sbueringer

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [sbueringer]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[k8s-infra-cherrypick-robot]

@sbueringer: new pull request created: #6755

In response to this:

/lgtm
/approve
/cherry-pick release-1.2

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.