api: add a new scheduler to balance the regions of the given key range #8988

bufferflies · 2025-01-10T01:51:53Z

What problem does this PR solve?

Issue Number: Close #8987

What is changed and how does it work?

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)

Code changes

Side effects

Related changes

Release note

None.

Signed-off-by: 童剑 <1045931706@qq.com>

ti-chi-bot · 2025-01-10T01:51:55Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

Signed-off-by: 童剑 <1045931706@qq.com>

codecov · 2025-01-13T09:24:54Z

Codecov Report

Attention: Patch coverage is 61.74242% with 101 lines in your changes missing coverage. Please review.

Project coverage is 76.22%. Comparing base (c92da83) to head (33c12bd).
Report is 1 commits behind head on master.

❌ Your patch status has failed because the patch coverage (61.74%) is below the target coverage (74.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #8988      +/-   ##
==========================================
- Coverage   76.23%   76.22%   -0.02%     
==========================================
  Files         467      468       +1     
  Lines       71155    71415     +260     
==========================================
+ Hits        54247    54436     +189     
- Misses      13515    13575      +60     
- Partials     3393     3404      +11

Flag	Coverage Δ
unittests	`76.22% <61.74%> (-0.02%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Signed-off-by: 童剑 <1045931706@qq.com>

pkg/mcs/scheduling/server/cluster.go

pkg/schedule/operator/kind.go

rleungx · 2025-01-15T07:17:51Z

pkg/schedule/schedulers/balance_key_range.go

+type balanceKeyRangeSchedulerConfig struct {
+	syncutil.RWMutex
+	schedulerConfig
+	balanceKeyRangeSchedulerParam


Shall we use a slice to support multiple key ranges with different roles or engines?

We need to support a multi-key range with same role and engine.

pkg/schedule/types/type.go

tools/pd-ctl/tests/scheduler/scheduler_test.go

Signed-off-by: 童剑 <1045931706@qq.com>

rleungx · 2025-01-15T10:25:59Z

tools/pd-ctl/pdctl/command/scheduler.go

@@ -374,6 +375,17 @@ func NewBalanceWitnessSchedulerCommand() *cobra.Command {
 	return c
 }

+// NewBalanceRangeSchedulerCommand returns a command to add a balance-key-range-scheduler.
+func NewBalanceRangeSchedulerCommand() *cobra.Command {


Do we need to support it? It's hard to use.

yes, it's helpful to debug it in the first step, and then it recommends the SQL function description

How about using API to test now? I don't think we need to add a new command that's going to be deprecated soon.

Signed-off-by: 童剑 <1045931706@qq.com>

pkg/schedule/schedulers/balance_key_range.go

Signed-off-by: 童剑 <1045931706@qq.com>

nolouch · 2025-01-23T09:01:09Z

pkg/schedule/schedulers/balance_range.go

+	Engine    string          `json:"engine"`
+	Timeout   time.Duration   `json:"timeout"`
+	Ranges    []core.KeyRange `json:"ranges"`
+	TableName string          `json:"table-name"`


Name it as alias for better, since pd does not understand the table concept.

cc @bufferflies

rleungx

The rest LGTM except the left comment.

rleungx · 2025-01-23T09:11:33Z

tools/pd-ctl/pdctl/command/scheduler.go

+		Use:        "balance-range-scheduler [--format=raw|encode|hex] <engine> <role> <table-name> <start_key> <end_key>",
+		Short:      "add a scheduler to balance region for given range",
+		Run:        addSchedulerForBalanceRangeCommandFunc,
+		Deprecated: "balance-range will be deprecated in the future, please use sql instead",


It's strange to add Deprecated now.

removed and add one todo for sql implement

Signed-off-by: 童剑 <1045931706@qq.com>

server/api/scheduler.go

Signed-off-by: 童剑 <1045931706@qq.com>

ti-chi-bot · 2025-02-10T08:50:51Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: nolouch, rleungx

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [nolouch,rleungx]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

ti-chi-bot · 2025-02-10T08:50:51Z

[LGTM Timeline notifier]

Timeline:

2025-01-23 09:15:28.807279505 +0000 UTC m=+345056.138198909: ☑️ agreed by rleungx.
2025-02-10 08:50:50.910522013 +0000 UTC m=+260093.306744076: ☑️ agreed by nolouch.

bufferflies added 2 commits January 7, 2025 18:24

add scheduler config

e693b3e

Signed-off-by: 童剑 <1045931706@qq.com>

add new scheduler for key range

23ff7d0

Signed-off-by: 童剑 <1045931706@qq.com>

ti-chi-bot bot added release-note-none dco-signoff: yes do-not-merge/work-in-progress size/XL labels Jan 10, 2025

bufferflies added 2 commits January 10, 2025 14:32

pass ut

1e6d628

Signed-off-by: 童剑 <1045931706@qq.com>

lint

d1da5b5

Signed-off-by: 童剑 <1045931706@qq.com>

bufferflies force-pushed the balance_key_range branch 2 times, most recently from fe71a54 to cb8a4b9 Compare January 14, 2025 08:21

pass ut

d0cfc2d

Signed-off-by: 童剑 <1045931706@qq.com>

bufferflies force-pushed the balance_key_range branch from cb8a4b9 to d0cfc2d Compare January 14, 2025 09:32

bufferflies marked this pull request as ready for review January 14, 2025 09:56

ti-chi-bot bot removed the do-not-merge/work-in-progress label Jan 14, 2025

rleungx reviewed Jan 15, 2025

View reviewed changes

rename balance-key-range to balance-range

d86148f

Signed-off-by: 童剑 <1045931706@qq.com>

rleungx reviewed Jan 15, 2025

View reviewed changes

use hex encode

8bdb7bc

Signed-off-by: 童剑 <1045931706@qq.com>

bufferflies force-pushed the balance_key_range branch from 0bb25b9 to 8bdb7bc Compare January 16, 2025 07:36

bufferflies changed the title ~~api: add new scheduler for balance the regions of the given key range~~ api: add a new scheduler to balance the regions of the given key range Jan 17, 2025

nolouch reviewed Jan 17, 2025

View reviewed changes

pkg/schedule/schedulers/balance_key_range.go Outdated Show resolved Hide resolved

pkg/schedule/schedulers/balance_key_range.go Outdated Show resolved Hide resolved

rleungx reviewed Jan 17, 2025

View reviewed changes

pkg/schedule/schedulers/balance_key_range.go Outdated Show resolved Hide resolved

pkg/schedule/schedulers/balance_key_range.go Outdated Show resolved Hide resolved

pkg/schedule/schedulers/balance_key_range.go Outdated Show resolved Hide resolved

rename

0696ba6

Signed-off-by: 童剑 <1045931706@qq.com>

bufferflies force-pushed the balance_key_range branch from eabfb27 to 0696ba6 Compare January 21, 2025 07:05

add table configuration

5d5ee0f

Signed-off-by: 童剑 <1045931706@qq.com>

nolouch reviewed Jan 23, 2025

View reviewed changes

rleungx approved these changes Jan 23, 2025

View reviewed changes

ti-chi-bot bot added the needs-1-more-lgtm label Jan 23, 2025

ti-chi-bot bot added the approved label Jan 23, 2025

bufferflies added 3 commits January 26, 2025 08:16

multi config

225184c

Signed-off-by: 童剑 <1045931706@qq.com>

add multi jobs

39ec9b2

Signed-off-by: 童剑 <1045931706@qq.com>

pass ut

e509705

Signed-off-by: 童剑 <1045931706@qq.com>

nolouch reviewed Feb 8, 2025

View reviewed changes

server/api/scheduler.go Outdated Show resolved Hide resolved

rename table-name

439f692

Signed-off-by: 童剑 <1045931706@qq.com>

nolouch approved these changes Feb 10, 2025

View reviewed changes

ti-chi-bot bot added lgtm and removed needs-1-more-lgtm labels Feb 10, 2025

Merge branch 'master' into balance_key_range

33c12bd

ti-chi-bot bot merged commit 517afe0 into tikv:master Feb 10, 2025
23 of 25 checks passed

bufferflies deleted the balance_key_range branch February 11, 2025 06:57

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

api: add a new scheduler to balance the regions of the given key range #8988

api: add a new scheduler to balance the regions of the given key range #8988

bufferflies commented Jan 10, 2025

ti-chi-bot bot commented Jan 10, 2025

codecov bot commented Jan 13, 2025 •

edited

Loading

rleungx Jan 15, 2025

bufferflies Jan 15, 2025

rleungx Jan 15, 2025

bufferflies Jan 16, 2025

okJiang Feb 10, 2025

nolouch Jan 23, 2025

nolouch Feb 5, 2025

bufferflies Feb 6, 2025

rleungx left a comment

rleungx Jan 23, 2025

bufferflies Feb 6, 2025

ti-chi-bot bot commented Feb 10, 2025

ti-chi-bot bot commented Feb 10, 2025

api: add a new scheduler to balance the regions of the given key range #8988

api: add a new scheduler to balance the regions of the given key range #8988

Conversation

bufferflies commented Jan 10, 2025

What problem does this PR solve?

What is changed and how does it work?

Check List

Release note

ti-chi-bot bot commented Jan 10, 2025

codecov bot commented Jan 13, 2025 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rleungx left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ti-chi-bot bot commented Feb 10, 2025

ti-chi-bot bot commented Feb 10, 2025

[LGTM Timeline notifier]

codecov bot commented Jan 13, 2025 •

edited

Loading