Skip to content

Remove 'enterprise' and 'basic' terms from v24.3 cdc docs #19790

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 18, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/current/_data/redirects.yml
Original file line number Diff line number Diff line change
Expand Up @@ -253,7 +253,7 @@

- destination: how-does-a-changefeed-work.md
sources: ['how-does-an-enterprise-changefeed-work.md']
versions: ['v25.2', 'v25.1']
versions: ['v25.2', 'v25.1', 'v24.3']

- destination: kubernetes-overview.md
sources: ['operate-cockroachdb-kubernetes.md']
Expand Down
4 changes: 2 additions & 2 deletions src/current/_includes/releases/v24.3/v24.3.0-alpha.1.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,9 +58,9 @@ Release Date: October 9, 2024
- Updated the cluster setting [`changefeed.sink_io_workers`]({% link v24.3/cluster-settings.md %}#setting-changefeed-sink-io-workers) with all the [sinks]({% link v24.3/changefeed-sinks.md %}) that support the setting. [#129946][#129946]
- Added a LDAP authentication method to complement password-based login for the [DB Console]({% link v24.3/ui-overview.md %}) if HBA configuration has an entry for LDAP for the user attempting login, along with other matching criteria (like the requests originating IP address) for authentication to the DB Console. [#130418][#130418]
- Added timers around key parts of the [changefeed]({% link v24.3/change-data-capture-overview.md %}) pipeline to help debug feeds experiencing issues. The `changefeed.stage.<stage>.latency` metrics now emit latency histograms for each stage. The metric respects the [changefeed `scope` label]({% link v24.3/monitor-and-debug-changefeeds.md %}#using-changefeed-metrics-labels) for debugging specific feeds. [#128794][#128794]
- For [enterprise changefeeds]({% link v24.3/how-does-an-enterprise-changefeed-work.md %}), [events]({% link v24.3/eventlog.md %}) `changefeed_failed` and `create_changefeed` now include a `JobId` field. [#131396][#131396]
- For [enterprise changefeeds]({% link v24.3/how-does-a-changefeed-work.md %}), [events]({% link v24.3/eventlog.md %}) `changefeed_failed` and `create_changefeed` now include a `JobId` field. [#131396][#131396]
- The new [metric]({% link v24.3/metrics.md %}) `seconds_until_license_expiry` allows you to monitor the status of a cluster's Enterprise license. [#129052][#129052].
- Added the `changefeed.total_ranges` metric, which [monitors]({% link v24.3/monitor-and-debug-changefeeds.md %}) the number of [ranges]({% link v24.3/architecture/overview.md %}#architecture-range) that are watched by [changefeed aggregators]({% link v24.3/how-does-an-enterprise-changefeed-work.md %}). It shares the same polling interval as [`changefeed.lagging_ranges`]({% link v24.3/advanced-changefeed-configuration.md %}#lagging-ranges), which is controlled by the existing `lagging_ranges_polling_interval` option. [#130897][#130897]
- Added the `changefeed.total_ranges` metric, which [monitors]({% link v24.3/monitor-and-debug-changefeeds.md %}) the number of [ranges]({% link v24.3/architecture/overview.md %}#architecture-range) that are watched by [changefeed aggregators]({% link v24.3/how-does-a-changefeed-work.md %}). It shares the same polling interval as [`changefeed.lagging_ranges`]({% link v24.3/advanced-changefeed-configuration.md %}#lagging-ranges), which is controlled by the existing `lagging_ranges_polling_interval` option. [#130897][#130897]

<h3 id="v24-3-0-alpha-1-sql-language-changes">SQL language changes</h3>

Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
Use the `schema_locked` [storage parameter]({% link {{ page.version.version }}/with-storage-parameter.md %}) to disallow [schema changes]({% link {{ page.version.version }}/online-schema-changes.md %}) on a watched table, which allows the changefeed to take a fast path that avoids checking if there are schema changes that could require synchronization between [changefeed aggregators]({% link {{ page.version.version }}/how-does-an-enterprise-changefeed-work.md %}). This helps to decrease the latency between a write committing to a table and it emitting to the [changefeed's sink]({% link {{ page.version.version }}/changefeed-sinks.md %}). Enabling `schema_locked`
Use the `schema_locked` [storage parameter]({% link {{ page.version.version }}/with-storage-parameter.md %}) to disallow [schema changes]({% link {{ page.version.version }}/online-schema-changes.md %}) on a watched table, which allows the changefeed to take a fast path that avoids checking if there are schema changes that could require synchronization between [changefeed aggregators]({% link {{ page.version.version }}/how-does-a-changefeed-work.md %}). This helps to decrease the latency between a write committing to a table and it emitting to the [changefeed's sink]({% link {{ page.version.version }}/changefeed-sinks.md %}). Enabling `schema_locked`

Enable `schema_locked` on the watched table with the [`ALTER TABLE`]({% link {{ page.version.version }}/alter-table.md %}) statement:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,9 @@ In this example, you'll set up a basic changefeed for a single-node cluster that
$ cockroach sql --url="postgresql://[email protected]:26257?sslmode=disable" --format=csv
~~~

{% include {{ page.version.version }}/cdc/core-url.md %}
{% include {{ page.version.version }}/cdc/sinkless-url.md %}

{% include {{ page.version.version }}/cdc/core-csv.md %}
{% include {{ page.version.version }}/cdc/sinkless-csv.md %}

1. Enable the `kv.rangefeed.enabled` [cluster setting]({% link {{ page.version.version }}/cluster-settings.md %}):

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,9 @@ In this example, you'll set up a basic changefeed for a single-node cluster.
--format=csv
~~~

{% include {{ page.version.version }}/cdc/core-url.md %}
{% include {{ page.version.version }}/cdc/sinkless-url.md %}

{% include {{ page.version.version }}/cdc/core-csv.md %}
{% include {{ page.version.version }}/cdc/sinkless-csv.md %}

1. Enable the `kv.rangefeed.enabled` [cluster setting]({% link {{ page.version.version }}/cluster-settings.md %}):

Expand Down
2 changes: 1 addition & 1 deletion src/current/_includes/v24.3/cdc/lagging-ranges.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ Use the `changefeed.lagging_ranges` metric to track the number of [ranges]({% li
- `lagging_ranges_polling_interval` sets the interval rate for when lagging ranges are checked and the `lagging_ranges` metric is updated. Polling adds latency to the `lagging_ranges` metric being updated. For example, if a range falls behind by 3 minutes, the metric may not update until an additional minute afterward.
- **Default:** `1m`

{% include_cached new-in.html version="v24.3" %} Use the `changefeed.total_ranges` metric to monitor the number of ranges that are watched by [aggregator processors]({% link {{ page.version.version }}/how-does-an-enterprise-changefeed-work.md %}) participating in the changefeed job. If you're experiencing lagging ranges, `changefeed.total_ranges` may indicate that the number of ranges watched by aggregator processors in the job is unbalanced. You may want to try [pausing]({% link {{ page.version.version }}/pause-job.md %}) the changefeed and then [resuming]({% link {{ page.version.version }}/resume-job.md %}) it, so that the changefeed replans the work in the cluster. `changefeed.total_ranges` shares the same polling interval as the `changefeed.lagging_ranges` metric, which is controlled by the `lagging_ranges_polling_interval` option.
{% include_cached new-in.html version="v24.3" %} Use the `changefeed.total_ranges` metric to monitor the number of ranges that are watched by [aggregator processors]({% link {{ page.version.version }}/how-does-a-changefeed-work.md %}) participating in the changefeed job. If you're experiencing lagging ranges, `changefeed.total_ranges` may indicate that the number of ranges watched by aggregator processors in the job is unbalanced. You may want to try [pausing]({% link {{ page.version.version }}/pause-job.md %}) the changefeed and then [resuming]({% link {{ page.version.version }}/resume-job.md %}) it, so that the changefeed replans the work in the cluster. `changefeed.total_ranges` shares the same polling interval as the `changefeed.lagging_ranges` metric, which is controlled by the `lagging_ranges_polling_interval` option.

{{site.data.alerts.callout_success}}
You can use the [`metrics_label`]({% link {{ page.version.version }}/monitor-and-debug-changefeeds.md %}#using-changefeed-metrics-labels) option to track the `lagging_ranges` and `total_ranges` metric per changefeed.
Expand Down
4 changes: 2 additions & 2 deletions src/current/_includes/v24.3/sidebar-data/stream-data.json
Original file line number Diff line number Diff line change
Expand Up @@ -139,9 +139,9 @@
"title": "Technical Overview",
"items": [
{
"title": "How Does an Enterprise Changefeed Work?",
"title": "How Does a Changefeed Work?",
"urls": [
"/${VERSION}/how-does-an-enterprise-changefeed-work.html"
"/${VERSION}/how-does-a-changefeed-work.html"
]
}
]
Expand Down
Binary file modified src/current/images/v24.3/changefeed-structure.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
8 changes: 4 additions & 4 deletions src/current/v24.3/advanced-changefeed-configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,13 +63,13 @@ Adjusting `kv.closed_timestamp.target_duration` could have a detrimental impact

`kv.closed_timestamp.target_duration` controls the target [closed timestamp]({% link {{ page.version.version }}/architecture/transaction-layer.md %}#closed-timestamps) lag duration, which determines how far behind the current time CockroachDB will attempt to maintain the closed timestamp. For example, with the default value of `3s`, if the current time is `12:30:00` then CockroachDB will attempt to keep the closed timestamp at `12:29:57` by possibly retrying or aborting ongoing writes that are below this time.

A changefeed aggregates checkpoints across all ranges, and once the timestamp on all the ranges advances, the changefeed can then [checkpoint]({% link {{ page.version.version }}/how-does-an-enterprise-changefeed-work.md %}). In the context of changefeeds, `kv.closed_timestamp.target_duration` affects how old the checkpoints will be, which will determine the latency before changefeeds can consider the history of an event complete.
A changefeed aggregates checkpoints across all ranges, and once the timestamp on all the ranges advances, the changefeed can then [checkpoint]({% link {{ page.version.version }}/how-does-a-changefeed-work.md %}). In the context of changefeeds, `kv.closed_timestamp.target_duration` affects how old the checkpoints will be, which will determine the latency before changefeeds can consider the history of an event complete.

#### `kv.rangefeed.closed_timestamp_refresh_interval`

**Default:** `3s`

This setting controls the interval at which [closed timestamp]({% link {{ page.version.version }}/architecture/transaction-layer.md %}#closed-timestamps) updates are delivered to [rangefeeds]({% link {{ page.version.version }}/create-and-configure-changefeeds.md %}#enable-rangefeeds) and in turn emitted as a [changefeed checkpoint]({% link {{ page.version.version }}/how-does-an-enterprise-changefeed-work.md %}).
This setting controls the interval at which [closed timestamp]({% link {{ page.version.version }}/architecture/transaction-layer.md %}#closed-timestamps) updates are delivered to [rangefeeds]({% link {{ page.version.version }}/create-and-configure-changefeeds.md %}#enable-rangefeeds) and in turn emitted as a [changefeed checkpoint]({% link {{ page.version.version }}/how-does-a-changefeed-work.md %}).

Increasing the interval value will lengthen the delay between each checkpoint, which will increase the latency of changefeed checkpoints, but reduce the impact on SQL latency due to [overload]({% link {{ page.version.version }}/admission-control.md %}#use-cases-for-admission-control) on the cluster. This happens because every range with a rangefeed has to emit a checkpoint event with this `3s` interval. As an example, 1 million ranges would result in 330,000 events per second, which would use more CPU resources.

Expand Down Expand Up @@ -117,7 +117,7 @@ Before tuning these settings, we recommend reading details on our [changefeed at

### Pausing changefeeds and garbage collection

By default, [protected timestamps]({% link {{ page.version.version }}/architecture/storage-layer.md %}#protected-timestamps) will protect changefeed data from [garbage collection]({% link {{ page.version.version }}/architecture/storage-layer.md %}#garbage-collection) up to the time of the [_checkpoint_]({% link {{ page.version.version }}/how-does-an-enterprise-changefeed-work.md %}). Protected timestamps will protect changefeed data from garbage collection if the downstream [changefeed sink]({% link {{ page.version.version }}/changefeed-sinks.md %}) is unavailable until you either [cancel]({% link {{ page.version.version }}/cancel-job.md %}) the changefeed or the sink becomes available once again.
By default, [protected timestamps]({% link {{ page.version.version }}/architecture/storage-layer.md %}#protected-timestamps) will protect changefeed data from [garbage collection]({% link {{ page.version.version }}/architecture/storage-layer.md %}#garbage-collection) up to the time of the [_checkpoint_]({% link {{ page.version.version }}/how-does-a-changefeed-work.md %}). Protected timestamps will protect changefeed data from garbage collection if the downstream [changefeed sink]({% link {{ page.version.version }}/changefeed-sinks.md %}) is unavailable until you either [cancel]({% link {{ page.version.version }}/cancel-job.md %}) the changefeed or the sink becomes available once again.

However, if the changefeed lags too far behind, the protected changes could lead to an accumulation of garbage. This could result in increased disk usage and degraded performance for some workloads.

Expand Down Expand Up @@ -175,7 +175,7 @@ When designing a system that needs to emit a lot of changefeed messages, whether

When a changefeed emits a [resolved]({% link {{ page.version.version }}/create-changefeed.md %}#resolved) message, it force flushes all outstanding messages that have buffered, which will diminish your changefeed's throughput while the flush completes. Therefore, if you are aiming for higher throughput, we suggest setting the duration higher (e.g., 10 minutes), or **not** using the `resolved` option.

If you are setting the `resolved` option when you are aiming for high throughput, you must also consider the [`min_checkpoint_frequency`]({% link {{ page.version.version }}/create-changefeed.md %}#min-checkpoint-frequency) option, which defaults to `30s`. This option controls how often nodes flush their progress to the [coordinating changefeed node]({% link {{ page.version.version }}/how-does-an-enterprise-changefeed-work.md %}). As a result, `resolved` messages will not be emitted more frequently than the configured `min_checkpoint_frequency`. Set this option to at least as long as your `resolved` option duration.
If you are setting the `resolved` option when you are aiming for high throughput, you must also consider the [`min_checkpoint_frequency`]({% link {{ page.version.version }}/create-changefeed.md %}#min-checkpoint-frequency) option, which defaults to `30s`. This option controls how often nodes flush their progress to the [coordinating changefeed node]({% link {{ page.version.version }}/how-does-a-changefeed-work.md %}). As a result, `resolved` messages will not be emitted more frequently than the configured `min_checkpoint_frequency`. Set this option to at least as long as your `resolved` option duration.

### Batching and buffering messages

Expand Down
2 changes: 1 addition & 1 deletion src/current/v24.3/cdc-queries.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ Function | Description
--------------------------+----------------------
`changefeed_creation_timestamp()` | Returns the decimal MVCC timestamp when the changefeed was created. Use this function to build CDC queries that restrict emitted events by time. `changefeed_creation_timestamp()` can serve a similar purpose to the [`now()` time function]({% link {{ page.version.version }}/functions-and-operators.md %}#date-and-time-functions), which is not supported with CDC queries.
`event_op()` | Returns a string describing the type of event. If a changefeed is running with the [`diff`]({% link {{ page.version.version }}/create-changefeed.md %}#diff) option, then this function returns `'insert'`, `'update'`, or `'delete'`. If a changefeed is running without the `diff` option, it is not possible to determine an update from an insert, so `event_op()` returns [`'upsert'`](https://www.cockroachlabs.com/blog/sql-upsert/) or `'delete'`.
`event_schema_timestamp()` | Returns the timestamp of [schema change]({% link {{ page.version.version }}/online-schema-changes.md %}) events that cause a [changefeed message]({% link {{ page.version.version }}/changefeed-messages.md %}) to emit. When the schema change event does not result in a table backfill or scan, `event_schema_timestamp()` will return the event's timestamp. When the schema change event does result in a table backfill or scan, `event_schema_timestamp()` will return the timestamp at which the backfill/scan is read — the [high-water mark time]({% link {{ page.version.version }}/how-does-an-enterprise-changefeed-work.md %}) of the changefeed.
`event_schema_timestamp()` | Returns the timestamp of [schema change]({% link {{ page.version.version }}/online-schema-changes.md %}) events that cause a [changefeed message]({% link {{ page.version.version }}/changefeed-messages.md %}) to emit. When the schema change event does not result in a table backfill or scan, `event_schema_timestamp()` will return the event's timestamp. When the schema change event does result in a table backfill or scan, `event_schema_timestamp()` will return the timestamp at which the backfill/scan is read — the [high-water mark time]({% link {{ page.version.version }}/how-does-a-changefeed-work.md %}) of the changefeed.

You can also use the following functions in CDC queries:

Expand Down
Loading
Loading