Show RoutingGroup info in query history #607

andythsu · 2025-02-03T05:48:00Z

Description

It is useful to show what routing group a request is being routed to because if two routing groups have the same routedTo url, it will show the wrong information on the history page

Old:

New:

Additional context and related issues

Release notes

( ) This is not user-visible or is docs only, and no release notes are required.
(X) Release notes are required. Please propose a release note for me.
( ) Release notes are required, with the following suggested text:

willmostly · 2025-02-10T17:15:33Z

gateway-ha/src/main/java/io/trino/gateway/ha/handler/RoutingTargetHandler.java

    {
        Optional<String> previousBackend = getPreviousBackend(request);
-        String clusterHost = previousBackend.orElseGet(() -> getBackendFromRoutingGroup(request));
+        // This falls back on adhoc routing group if no routing group can be determined
+        String routingGroup = routingGroupSelector.findRoutingGroup(request).orElse("adhoc");


moving this call out of the orElseGet will cause the routing logic to be triggered for every incoming request, including those containing query IDs. Please revert

willmostly · 2025-02-10T17:21:22Z

gateway-ha/src/main/java/io/trino/gateway/ha/handler/RoutingDestinationInfo.java

+ */
+package io.trino.gateway.ha.handler;
+
+public record RoutingDestinationInfo(String group, String clusterUri) {}


It would expand the scope of this change, but it seems like a good time to add some audibility on which rules were triggered. For example, you could have RoutingDestinationInfo(String group, String clusterUri, List<String> rulesFired), where rulesFired would contain the names of all rules who's condition evaluated to true. We can chat about this offline

Should this be a simple String instead of List<String since only the first matching rule gets triggered?

willmostly · 2025-02-10T18:54:52Z

gateway-ha/src/main/java/io/trino/gateway/ha/handler/RoutingDestinationInfo.java

+ */
+package io.trino.gateway.ha.handler;
+
+public record RoutingDestinationInfo(String group, String clusterUri) {}


Can clusterUri be URIinstead of aString`?

gateway-ha/src/main/java/io/trino/gateway/proxyserver/ProxyRequestHandler.java

gateway-ha/src/main/resources/mysql/V2__add_routingGroup_to_query_history.sql

willmostly · 2025-02-10T19:24:52Z

gateway-ha/src/test/java/io/trino/gateway/ha/router/TestRoutingGroupSelector.java

@@ -85,11 +85,11 @@ void testByRoutingGroupHeader()
        // If the header is present the routing group is the value of that header.
        when(mockRequest.getHeader(ROUTING_GROUP_HEADER)).thenReturn("batch_backend");
        assertThat(RoutingGroupSelector.byRoutingGroupHeader().findRoutingGroup(mockRequest))
-                .isEqualTo("batch_backend");
+                .contains("batch_backend");


why do we need to change this?

changing this because findRoutingGroup returns an Optional<String> now

vishalya · 2025-02-03T15:02:03Z

gateway-ha/src/main/java/io/trino/gateway/ha/handler/RoutingTargetHandler.java

-        String clusterHost = previousBackend.orElseGet(() -> getBackendFromRoutingGroup(request));
+        // This falls back on adhoc routing group if no routing group can be determined
+        String routingGroup = routingGroupSelector.findRoutingGroup(request).orElse("adhoc");
+        String user = request.getHeader(USER_HEADER);


This may not be always true, especially in case of bearer tokens (JWT). But I see that this logic is from before and it's not newly added as a part of the PR.

We can take care of this in a different PR

gateway-ha/src/main/java/io/trino/gateway/ha/handler/RoutingDestinationInfo.java

mosabua · 2025-02-26T23:04:51Z

Let us know when you need another review cycle @andythsu

andythsu · 2025-02-28T04:45:13Z

gateway-ha/src/main/java/io/trino/gateway/ha/handler/RoutingTargetHandler.java

+        Optional<String> previousCluster = getPreviousCluster(queryId, request);
+        RoutingDestination routingDestination = previousCluster.map(cluster -> {
+            String routingGroup = queryId.map(routingManager::findRoutingGroupForQueryId)
+                    .orElse("adhoc");


@willmostly in the case where previousCluster is found but queryId is not found (i.e., cookie-based routing), how should we handle the value for routing group?

Chaho12

LGTM 👍

Chaho12 · 2025-03-05T07:08:19Z

@andythsu

It is useful to show what routing group a request is being routed to because if two routing groups have the same routedTo url, it will show the wrong information on the history page

I notice that if two routing groups have same proxyTo url, then history page shows wrong routedTo(Name) too. Have you seen this happen?

andythsu · 2025-03-05T19:40:58Z

@andythsu

It is useful to show what routing group a request is being routed to because if two routing groups have the same routedTo url, it will show the wrong information on the history page

I notice that if two routing groups have same proxyTo url, then history page shows wrong routedTo(Name) too. Have you seen this happen?

@Chaho12 yes I'm aware of this. The reason is because we don't store name in the query_history table, and when we pull history in here it maintains a mapping from proxyTo to name, essentially overwriting previous name with a new name.

andythsu · 2025-03-05T19:42:18Z

Also I think we should probably change the language here to match the column names in db to make things consistent... Or at least call it a different name because it's weird to have two RoutedTo

Chaho12 · 2025-03-05T23:15:24Z

Also I think we should probably change the language here to match the column names in db to make things consistent... Or at least call it a different name because it's weird to have two RoutedTo

I agree.. was thinking simply like Name. I'll send a pr on changing name, as this is different issue.

Btw while I was looking code I notice that not only Name but also the issue is with the data that gets overwritten when proxyTo is not unique.

cla-bot · 2025-03-14T14:24:14Z

Thank you for your pull request and welcome to our community. We could not parse the GitHub identity of the following contributors: asu80.
This is most likely caused by a git client misconfiguration; please make sure to:

check if your git client is configured with an email to sign commits git config --list | grep email
If not, set it up using git config --global user.email [email protected]
Make sure that the git commit email is configured in your GitHub account settings, see https://github.com/settings/emails

vishalya

LGTM

cla-bot bot added the cla-signed label Feb 3, 2025

andythsu force-pushed the routing_group branch 4 times, most recently from f57f2ad to d23d058 Compare February 3, 2025 11:48

andythsu requested review from vishalya, ebyhr and willmostly February 3, 2025 11:49

andythsu force-pushed the routing_group branch from d23d058 to ce90b2f Compare February 3, 2025 14:17

ebyhr removed their request for review February 5, 2025 23:24

willmostly requested changes Feb 10, 2025

View reviewed changes

vishalya reviewed Feb 12, 2025

View reviewed changes

andythsu force-pushed the routing_group branch 4 times, most recently from cb61497 to 8d6609b Compare February 28, 2025 04:43

andythsu commented Feb 28, 2025

View reviewed changes

Chaho12 approved these changes Mar 4, 2025

View reviewed changes

Chaho12 mentioned this pull request Mar 7, 2025

Rename RoutedTo to Name field in Query History page #639

Open

andythsu force-pushed the routing_group branch 2 times, most recently from e8e492d to 4157116 Compare March 14, 2025 14:24

cla-bot bot removed the cla-signed label Mar 14, 2025

Show RoutingGroup info in query history

792ecd0

andythsu force-pushed the routing_group branch from 4157116 to 792ecd0 Compare March 14, 2025 14:24

cla-bot bot added the cla-signed label Mar 14, 2025

vishalya approved these changes Mar 18, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Show RoutingGroup info in query history #607

Show RoutingGroup info in query history #607

andythsu commented Feb 3, 2025 •

edited

Loading

willmostly Feb 10, 2025

willmostly Feb 10, 2025

andythsu Feb 27, 2025 •

edited

Loading

willmostly Feb 10, 2025

andythsu Feb 27, 2025

willmostly Feb 10, 2025

andythsu Feb 27, 2025

vishalya Feb 3, 2025

andythsu Mar 5, 2025

mosabua commented Feb 26, 2025

andythsu Feb 28, 2025

Chaho12 left a comment

Chaho12 commented Mar 5, 2025 •

edited

Loading

andythsu commented Mar 5, 2025

andythsu commented Mar 5, 2025 •

edited

Loading

Chaho12 commented Mar 5, 2025 •

edited

Loading

cla-bot bot commented Mar 14, 2025

vishalya left a comment

Show RoutingGroup info in query history #607

Are you sure you want to change the base?

Show RoutingGroup info in query history #607

Conversation

andythsu commented Feb 3, 2025 • edited Loading

Description

Additional context and related issues

Release notes

Choose a reason for hiding this comment

Choose a reason for hiding this comment

andythsu Feb 27, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mosabua commented Feb 26, 2025

Choose a reason for hiding this comment

Chaho12 left a comment

Choose a reason for hiding this comment

Chaho12 commented Mar 5, 2025 • edited Loading

andythsu commented Mar 5, 2025

andythsu commented Mar 5, 2025 • edited Loading

Chaho12 commented Mar 5, 2025 • edited Loading

cla-bot bot commented Mar 14, 2025

vishalya left a comment

Choose a reason for hiding this comment

andythsu commented Feb 3, 2025 •

edited

Loading

andythsu Feb 27, 2025 •

edited

Loading

Chaho12 commented Mar 5, 2025 •

edited

Loading

andythsu commented Mar 5, 2025 •

edited

Loading

Chaho12 commented Mar 5, 2025 •

edited

Loading