add support for prometheus metrics #25

Fijxu · 2024-12-18T03:18:28Z

Useful for troubleshooting blockages in a visual manner using Grafana.

It doesn't count how many times poTokenGenerate has failed between lines 69-83, I didn't wanted to touch that much code. But I'll make a new commit to count on those lines too so you can review it.

unixfox · 2024-12-18T07:38:37Z

Maybe you can includes some "default" dashboards :)

unixfox · 2025-03-16T15:02:05Z

Could you allow this to be toggleable? I think people running private instances don't need it, it's more towards public instances.

Fijxu · 2025-03-18T19:46:00Z

This should be better, tell me if you want to change or add something

Edit: I'll add the grafana dashboard JSON in a few minutes

deno.json

alexmaras

I'm going to pull this down and test it too, but here's some comments in the mean time. Nothing looks wildly problematic.

src/routes/index.ts

src/routes/metrics.ts

src/main.ts

src/lib/helpers/metrics.ts

alexmaras · 2025-03-20T13:30:21Z

Just an update - pulled the branch and tested with prometheus and things look like they're correctly read in, so the format of the output all looks good.

src/routes/index.ts

src/lib/helpers/youtubePlayerHandling.ts

src/main.ts

alexmaras · 2025-03-21T01:00:55Z

src/lib/helpers/metrics.ts

+        switch (true) {
+            // CONTENT_CHECK_REQUIRED: Sensitive content videos.
+            case (videoData.playabilityStatus?.status ===
+                "CONTENT_CHECK_REQUIRED"): {
+                break;
+            }
+            case (videoData.playabilityStatus?.status === "LOGIN_REQUIRED"): {
+                switch (true) {
+                    // Age restricted videos, we don't need to track those.
+                    case videoData.playabilityStatus?.reason?.includes(
+                        "Sign in to confirm your age",
+                    ): {
+                        break;
+                    }
+
+                    case videoData.playabilityStatus?.reason?.includes(
+                        "Sign in to confirm you’re not a bot",
+                    ): {
+                        this.innertubeErrorReasonSignIn.inc();
+
+                        switch (true) {
+                            case videoData.playabilityStatus?.errorScreen
+                                ?.playerErrorMessageRenderer
+                                ?.subreason?.runs?.[0]?.text?.includes(
+                                    "This helps protect our community",
+                                ): {
+                                this.innertubeErrorSubreasonProtectCommunity
+                                    .inc();
+                                break;
+                            }
+                            default: {
+                                this.innertubeErrorSubreasonUnknown.inc();
+                                break;
+                            }
+                        }
+
+                        break;
+                    }
+
+                    default: {
+                        this.innertubeErrorReasonUnknown.inc();
+                        break;
+                    }
+                }
+                break;
+            }
+            default:
+                this.innertubeErrorStatusUnknown.inc();
+                break;
+        }


this is tricky, I'm going to have a play with this and see if I can find a nice way to do it that feels like it'll extend a bit more easily without becoming impossible to understand.

This is the only bit I'm stuck on now, the rest of the PR looks good to me. I'm playing around with some options, these kinds of logical mazes are always painful to parse and implement.

interested too in a "better" format

Done, this looks better, unless you want to do the checking inside the own checkStatus, checkReason and checkSubreason functions.

Edit: It seems to work well, but I don't have any IP banned so I can't test the other reason and subreasons regarding ip blocks.

unixfox · 2025-03-27T20:25:36Z

Please rebase thank you :)

unixfox · 2025-03-27T20:26:25Z

config/config.example.toml

@@ -15,6 +15,7 @@
 # # secret key needs to be 16 characters long or more
 # secret_key = "CHANGE_ME" # env variable: SERVER_SECRET_KEY
 # verify_requests = false
+# enable_metrics = false # env variable: ENABLE_METRICS


Same as #65, please rename to SERVER_ENABLE_METRICS

unixfox · 2025-03-27T20:30:17Z

deno.json

@@ -6,6 +6,7 @@
  "imports": {
    "hono": "jsr:@hono/[email protected]",
    "@std/toml": "jsr:@std/[email protected]",
+    "prom-client": "npm:[email protected]",


I would really like to avoid npm as much as possible, would it be possible for you to point to the github repo: https://github.com/siimon/prom-client/blob/v15.1.3/index.js?

You may use https://esm.sh to help you.

Using npm in deno requires enabling many many useless environment variables that I would like to get rid of at some point if we can stop using jsdom from npm.

fix deno lint and typo

Fijxu · 2025-03-27T20:53:29Z

I'll add a default Grafana dashboard, do not merge it yet ;)

I got a warning from grafana saying that "metric may not be a counter, name does not end with _total/_sum/ ...". This changes the metric names using the prometheus naming convention (https://prometheus.io/docs/practices/naming/)

unixfox · 2025-03-28T07:44:45Z

src/lib/helpers/config.ts

+        enable_metrics: z.boolean().default(
+            Deno.env.get("SERVER_ENABLE_METRICS") === "true"
+                ? true
+                : Deno.env.get("SERVER_ENABLE_METRICS") === "false"


Why do you check for enable metrics env to be false?

If you remove the parameter it's false by default.

yeah, default of Deno.env.get("SERVER_ENABLE_METRICS") === "true" will make it a boolean correctly. At the moment it'd always be true unless the env variable SERVER_ENABLE_METRICS is "false"

This reverts commit a7991b4.

Fijxu force-pushed the metrics branch 2 times, most recently from 64b46dc to 88c2296 Compare March 18, 2025 19:42

unixfox requested changes Mar 19, 2025

View reviewed changes

deno.json Outdated Show resolved Hide resolved

Fijxu force-pushed the metrics branch from 88c2296 to 78d0b3b Compare March 19, 2025 19:29

alexmaras requested changes Mar 20, 2025

View reviewed changes

src/routes/index.ts Outdated Show resolved Hide resolved

src/routes/metrics.ts Outdated Show resolved Hide resolved

src/main.ts Outdated Show resolved Hide resolved

src/lib/helpers/metrics.ts Outdated Show resolved Hide resolved

src/lib/helpers/metrics.ts Outdated Show resolved Hide resolved

alexmaras reviewed Mar 20, 2025

View reviewed changes

src/lib/helpers/metrics.ts Outdated Show resolved Hide resolved

Fijxu force-pushed the metrics branch from 716feb6 to 95d167f Compare March 20, 2025 21:15

alexmaras requested changes Mar 21, 2025

View reviewed changes

Fijxu force-pushed the metrics branch from 7aef5e8 to d69396e Compare March 26, 2025 15:40

unixfox requested changes Mar 27, 2025

View reviewed changes

Fijxu added 10 commits March 27, 2025 17:32

add support for prometheus metrics

379b1c0

fix deno lint and typo

chore: fmt

487c8aa

apply suggestions

f03614d

better innertube response checking

ad9ac3e

rename route _metrics to metrics

d5c3561

use Hono context to pass Metrics onto functions

33a0941

fix: add missing metrics argument

ed74e7a

prefix config option with SERVER_

0f93368

handle SERVER_ENABLE_METRICS explicitly

a7991b4

replace npm:prom-client with esm.sh

ade4afe

Fijxu force-pushed the metrics branch from d69396e to ade4afe Compare March 27, 2025 20:51

Fijxu added 2 commits March 27, 2025 18:04

use prometheus naming convention for metrics

301228c

I got a warning from grafana saying that "metric may not be a counter, name does not end with _total/_sum/ ...". This changes the metric names using the prometheus naming convention (https://prometheus.io/docs/practices/naming/)

add grafana dashboard

898856a

Fijxu added 5 commits March 27, 2025 18:49

better format on the checking of the innertube response

ae8ef58

add 'UNPLAYABLE' playabilityStatus.status

7c9acf0

add 'LOGIN_REQUIRED' status and fix logic

12e4a19

update grafana dashboard

8963155

set videoData objects as a const

69a6f9d

unixfox mentioned this pull request Mar 28, 2025

[Question] Rotate IP on Gluetun Proxy upon block #91

Open

unixfox requested changes Mar 28, 2025

View reviewed changes

Revert "handle SERVER_ENABLE_METRICS explicitly"

8fd8922

This reverts commit a7991b4.

unixfox mentioned this pull request Mar 28, 2025

[Enhancement] Invidious states it cannot connect to companion if initial potoken fetch fails #92

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add support for prometheus metrics #25

add support for prometheus metrics #25

Fijxu commented Dec 18, 2024

unixfox commented Dec 18, 2024

unixfox commented Mar 16, 2025

Fijxu commented Mar 18, 2025 •

edited

Loading

alexmaras left a comment

alexmaras commented Mar 20, 2025

alexmaras Mar 21, 2025

alexmaras Mar 21, 2025

unixfox Mar 27, 2025

Fijxu Mar 27, 2025 •

edited

Loading

unixfox commented Mar 27, 2025

unixfox Mar 27, 2025

unixfox Mar 27, 2025

Fijxu commented Mar 27, 2025

unixfox Mar 28, 2025

alexmaras Mar 28, 2025

add support for prometheus metrics #25

Are you sure you want to change the base?

add support for prometheus metrics #25

Conversation

Fijxu commented Dec 18, 2024

unixfox commented Dec 18, 2024

unixfox commented Mar 16, 2025

Fijxu commented Mar 18, 2025 • edited Loading

alexmaras left a comment

Choose a reason for hiding this comment

alexmaras commented Mar 20, 2025

alexmaras Mar 21, 2025

Choose a reason for hiding this comment

alexmaras Mar 21, 2025

Choose a reason for hiding this comment

unixfox Mar 27, 2025

Choose a reason for hiding this comment

Fijxu Mar 27, 2025 • edited Loading

Choose a reason for hiding this comment

unixfox commented Mar 27, 2025

unixfox Mar 27, 2025

Choose a reason for hiding this comment

unixfox Mar 27, 2025

Choose a reason for hiding this comment

Fijxu commented Mar 27, 2025

unixfox Mar 28, 2025

Choose a reason for hiding this comment

alexmaras Mar 28, 2025

Choose a reason for hiding this comment

Fijxu commented Mar 18, 2025 •

edited

Loading

Fijxu Mar 27, 2025 •

edited

Loading