Ad hoc filters not working with Tempo

[Dexiion]

Describe the bug
Hi!

We are switching from ApplicationInisghts in Azure to Tempo in grafana using OpenTelemetry, in AppInsights we had on the TransactionSearch page some filters where we can find by HTTP Code, Custom Properties, Duration, Success, etc.

We found that in grafana we have this also and they are called Ad Hoc filters (image 1) but when we add them they do not filter nothing as you can see on the image 2

To Reproduce
Steps to reproduce the behavior:

1. Create a pannel in grafana with tempo data source
2. Add an ad hoc variable
3. Try to filter by some of the labels

Expected behavior
The filters on the ad hoc are applied correctly to the data source so in this specific case It will only show the traces with resource.service.name of deployment-tool-agent

Environment:

• Infrastructure: Kubernetes
• Deployment tool: helm

[joe-elliott]

This seems to be a frontend issue. Is Grafana executing new queries when you apply the ad hoc filter? or is it doing live filtering in the frontend?