Skip to content

grafana dashboard broken for SLOs with dots in the name #527

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
problame opened this issue Oct 31, 2023 · 2 comments
Closed

grafana dashboard broken for SLOs with dots in the name #527

problame opened this issue Oct 31, 2023 · 2 comments
Labels

Comments

@problame
Copy link

problame commented Oct 31, 2023

Problem

One of our SLOs has the name xyz-latency-10ms-99.99%.

The Grafana Dashboard linked from the README.md is broken for such SLOs.

Specifically, it sends queries like this

sloth_slo_info{sloth_service="pageserver", sloth_slo="xxx-latency-10ms-99\\.99%"}

because the underlying query (quoting from the dashboard JSON, hence double-escaping)

          "expr": "sloth_slo_info{sloth_service=\"${service}\", sloth_slo=\"${slo}\"}",

Root-Cause

Before queries are sent to your data source the query is interpolated, meaning the variable is replaced with its current value. During interpolation, the variable value might be escaped in order to conform to the syntax of the query language and where it is used. For example, a variable used in a regex expression in an InfluxDB or Prometheus query will be regex escaped. Read the data source specific documentation topic for details on value escaping during interpolation.

https://grafana.com/docs/grafana/latest/dashboards/variables/variable-syntax/

Solution

Use the regex label filter =~ instead of a plain =.

          "expr": "sloth_slo_info{sloth_service=~\"${service}\", sloth_slo=~\"${slo}\"}",

I have done the work already to fix the dashboard JSON, we're using it internally.

I don't know what the process is to make a PR for the dashboard.

IMO it would be good if the JSON is versioned in this repository, instead of just being on Grafana.

@problame problame changed the title grafana dashboard doesn't use correct interpolation grafana dashboard broken for SLOs with dots in the name Oct 31, 2023
Copy link

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 15 days.

@github-actions github-actions bot added the stale label Mar 27, 2025
Copy link

This issue was closed because it has been stale for 15 days with no activity.

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Apr 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant