[Proposal] Query Resource Isolation in Pinot

[vvivekiyer]

Hi, sharing the design for Query Resource Isolation in Pinot - https://docs.google.com/document/d/1bUr_l4B8gJCiUOKw97HFB9EEWvTU0Fw70HuZ0jq7YVw/edit?tab=t.0#heading=h.y9hbiahmee08

Problem Statement:
The noisy-neighbor problem is a serious limitation in dedicated tenant and shared tenant setups. In shared tenant setups, changes to query patterns/QPS/etc in one table can negatively impact the queries of other tables, increasing the blast-radius for incidents. In dedicated tenant use-cases, queries from a single use-case/upstream caller can impact other callers sharing the table.

Given a steady state workload being run on a Pinot cluster, Pinot should be able to protect itself and maintain the SLOs (availability and latency) and degrade gracefully to handle scenarios like unexpected changes in cost of workloads which can be a function of:

• New queries being ramped
• Changes to existing queries/query patterns
• New data characteristics changing query selectivity
• QPS spikes

Various versions of this problem have been discussed in the past. For example in #8618, #5627

High Level Proposal

Through this effort, table owners can define query workloads and attach enforcementProfiles to each workload. Pinot will provide read-path isolation for these query workloads. This will allow Pinot maintainers to provide better SLO guarantees and improve HW utilization.

For more details, please refer to the doc.

cc: @siddharthteotia @Jackie-Jiang @mayankshriv @mcvsubbu @ankitsultana @jasperjiaguo