Query Frontend For Logical Query Plan Support #6884
Conversation
rubywtl
force-pushed
the
frontend/middleware-logicalplan-support
branch
from
5e6383f to
3631c26
Compare
…ed execution feature flag Signed-off-by: rubywtl <[email protected]>
…tests Signed-off-by: rubywtl <[email protected]>
rubywtl
force-pushed
the
frontend/middleware-logicalplan-support
branch
from
7c03859 to
0d12680
Compare
| @@ -533,12 +533,16 @@ func (t *Cortex) initQueryFrontendTripperware() (serv services.Service, err erro | |||
| prometheusCodec, | |||
| shardedPrometheusCodec, | |||
| t.Cfg.Querier.LookbackDelta, | |||
| t.Cfg.Querier.EnablePerStepStats, | |||
| t.Cfg.Frontend.DistributedExecEnabled, | |||
| t.Cfg.Frontend.DisableDuplicateLabelChecks, | |||
There was a problem hiding this comment.
I would not expose this as a config for now. Let's always disable duplicate label check
| lookbackDelta: lookbackDelta, | ||
| enabledPerStepStats: enablePerStepStats, | ||
| next: next, | ||
| disableDuplicateLabelChecks: disableDuplicateLabelChecks, |
There was a problem hiding this comment.
I think we can just remove lookbackDelta, enabledPerStepStats and disableDuplicateLabelChecks. Those options only make sense in execution time but not the logicalplan and optimize phase. So it doesn't matter if they are true or false. We can just hardcode to false
|
|
||
| // TODO: Add distributed optimizer for remote node insertion | ||
|
|
||
| return byteLP, nil |
There was a problem hiding this comment.
Can we just combine NewInstantLogicalPlan and NewRangeLogicalPlan into one NewLogicalPlan function?
| return byteLP, nil | ||
| } | ||
|
|
||
| func (l logicalPlanGen) NewRangeLogicalPlan(qs string, start, end time.Time, interval time.Duration) ([]byte, error) { |
There was a problem hiding this comment.
I would recommend separating creating logical plan and marshal the logical plan. NewLogicalPlan function should return the plan itself instead of the serialized version.
| @@ -96,6 +96,8 @@ type Request interface { | |||
| GetStep() int64 | |||
| // GetQuery returns the query of the request. | |||
| GetQuery() string | |||
| // GetLogicalPlan returns the serialized logical plan | |||
| GetLogicalPlan() []byte | |||
There was a problem hiding this comment.
It probably makes more sense if GetLogicalPlan returns the plan type instead of the serialized version. So we need to keep logical plan as a Plan type here.
We can do the serialization in the Codec encode request part
| @@ -26,7 +26,7 @@ import ( | |||
| ) | |||
|
|
|||
| const ( | |||
| query = "/api/v1/query_range?end=1536716898&query=sum%28container_memory_rss%29+by+%28namespace%29&start=1536673680&stats=all&step=120" | |||
| queryAll = "/api/v1/query_range?end=1536716898&query=sum%28container_memory_rss%29+by+%28namespace%29&start=1536673680&stats=all&step=120" | |||
There was a problem hiding this comment.
Why do we need to rename this?
| @@ -29,6 +31,8 @@ func (cfg *CombinedFrontendConfig) RegisterFlags(f *flag.FlagSet) { | |||
| cfg.FrontendV2.RegisterFlags(f) | |||
|
|
|||
| f.StringVar(&cfg.DownstreamURL, "frontend.downstream-url", "", "URL of downstream Prometheus.") | |||
| f.BoolVar(&cfg.DistributedExecEnabled, "frontend.distributed_exec_enabled", false, "Experimental: Enables distributed execution of queries by passing logical query plan fragments to downstream components.") | |||
There was a problem hiding this comment.
You need to run make doc to generate docs. Otherwise CI will fail
| "net/http" | ||
| "strconv" | ||
| "testing" | ||
| "time" |
There was a problem hiding this comment.
Let's try to sort and group the imports. Otherwise lint will fail
| "github.com/thanos-io/promql-engine/query" | ||
| "github.com/weaveworks/common/httpgrpc" | ||
| "net/http" | ||
| "time" |
There was a problem hiding this comment.
Let's try to sort and group the imports. Otherwise lint will fail
What this PR does:
This PR adds logical plan generation support to the query frontend as part of the distributed query execution implementation.
Features
Configuration Changes
Testing Coverage
Integration Tests
Unit Tests
Which issue(s) this PR fixes:
Related to #6789
Checklist
CHANGELOG.mdupdated