-
Notifications
You must be signed in to change notification settings - Fork 1.2k
ticdc: add new doc for TiCDC data replication capabilities #20345
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
20 commits
Select commit
Hold shift + click to select a range
66b14d8
update
hongyunyan edb5e40
update
hongyunyan 1c2e8a4
update
hongyunyan 99f6d70
Update wording and format
lilin90 3fb1e75
ticdc: rename and update descriptions
lilin90 f3f33d4
ticdc: reorganize content
lilin90 5edd128
ticdc: reorganize content
lilin90 1180776
ticdc: improve format
lilin90 5d818eb
toc: add new doc to toc
lilin90 cef5c83
ticdc: update wording
lilin90 d8437bc
Update ticdc/ticdc-data-replication-capabilities.md
hongyunyan 050e5ed
Update ticdc/ticdc-data-replication-capabilities.md
hongyunyan 93bfc9d
ticdc: update format
lilin90 3529e85
ticdc: update wording
lilin90 787be98
ticdc: update format
lilin90 5eda7bd
Update format
lilin90 a1edebd
Fix a link
lilin90 b11ec20
update
hongyunyan dbe38eb
Update ticdc/ticdc-data-replication-capabilities.md
hongyunyan 678833c
Update ticdc/ticdc-data-replication-capabilities.md
hongyunyan File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,48 @@ | ||
--- | ||
title: TiCDC 数据同步能力详解 | ||
summary: 了解 TiCDC 的数据同步能力。 | ||
--- | ||
|
||
# TiCDC 数据同步能力详解 | ||
|
||
[TiCDC](/ticdc/ticdc-overview.md) (TiDB Change Data Capture) 是 TiDB 生态中用于实时数据同步的核心组件。本文详细解释 TiCDC 的数据同步能力。 | ||
|
||
## 工作原理 | ||
|
||
- TiCDC 监听 TiKV 的变更日志 (Raft Log),将行数据的增删改操作转换为下游兼容的 SQL 语句,并不基于上游的 SQL 获取数据变更。详情请参考 [TiCDC 处理数据变更的实现原理](/ticdc/ticdc-overview.md#ticdc-处理数据变更的实现原理)。 | ||
|
||
- TiCDC 生成与 SQL 语义等效的逻辑操作(如 `INSERT`、`UPDATE`、`DELETE`),而非逐条还原上游执行的原始 SQL 语句。详情请参考 [TiCDC 处理数据变更的实现原理](/ticdc/ticdc-overview.md#ticdc-处理数据变更的实现原理)。 | ||
|
||
- TiCDC 提供事务最终一致性的保证。开启 [redo log](/ticdc/ticdc-sink-to-mysql.md#灾难场景的最终一致性复制) 后,TiCDC 可以保证容灾场景下的最终一致性;开启 [Syncpoint](/ticdc/ticdc-upstream-downstream-check.md#启用-syncpoint) 后,TiCDC 提供一致性快照读和数据一致性校验。 | ||
|
||
## 支持的下游 | ||
|
||
TiCDC 支持同步数据到多类下游,包括: | ||
|
||
- [TiDB 及兼容 MySQL 协议的数据库](/ticdc/ticdc-sink-to-mysql.md) | ||
- [Apache Kafka](/ticdc/ticdc-sink-to-kafka.md) | ||
- [Message Queue (MQ) 类的下游](/ticdc/ticdc-changefeed-config.md#sink),如 [Pulsar](/ticdc/ticdc-sink-to-pulsar.md) | ||
- [存储服务(Amazon S3、GCS、Azure Blob Storage 和 NFS)](/ticdc/ticdc-sink-to-cloud-storage.md) | ||
- [通过 Confluent Cloud 同步至 Snowflake、ksqlDB、SQL Server](/ticdc/integrate-confluent-using-ticdc.md) | ||
- [使用 Apache Flink 消费同步至 Kafka 的数据](/replicate-data-to-kafka.md) | ||
|
||
## 数据同步范围 | ||
|
||
TiCDC 对上游数据变更的支持范围如下: | ||
|
||
+ 支持: | ||
|
||
- DDL 和 DML 语句(非系统表)。 | ||
- 索引操作 (`ADD INDEX`, `CREATE INDEX`):为了减少对 Changefeed 同步延迟的影响,当下游为 TiDB 时,TiCDC 会[异步执行创建和添加索引的 DDL 操作](/ticdc/ticdc-ddl.md#创建和添加索引-ddl-的异步执行)。 | ||
- 外键约束 DDL 语句 (`ADD FOREIGN KEY`):TiCDC 不会同步上游系统变量的设置,需要在下游手动设置 [`foreign_key_checks`](/system-variables.md#foreign_key_checks) 来决定是否开启下游的外键约束检查。TiCDC 不会检查数据是否满足下游的外键约束。 | ||
|
||
+ 不支持: | ||
|
||
- 系统表(如 `mysql.*` 和 `information_schema.*`)的 DDL 和 DML 语句。 | ||
- 临时表的 DDL 和 DML 语句。 | ||
- DQL (Data Query Language) 语句 和 DCL (Data Control Language) 语句。 | ||
|
||
## 使用限制 | ||
|
||
- TiCDC 对一些场景暂不支持,详见[暂不支持的场景](/ticdc/ticdc-overview.md#暂不支持的场景)。 | ||
- TiCDC 只检查上游数据变更的完整性,不检查数据变更是否符合上游或下游的约束。如果遇到不满足下游约束的数据变更,TiCDC 会在写入下游时报错。例如,TiCDC 不会进行任何外键约束的检查。 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.