Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[docs] sensors docs: clarify run_key behavior in a single sensor evaluation #26404

Closed
wants to merge 1 commit into from
+2 −0
Closed

[docs] sensors docs: clarify run_key behavior in a single sensor evaluation #26404

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/content/concepts/partitions-schedules-sensors/sensors.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,8 @@ In the example, a <PyObject object="RunRequest"/> is requested for each file dur

Run keys allow you to write sensor evaluation functions that declaratively describe what runs should exist, and helps you avoid the need for more complex logic that manages state. However, when dealing with high-volume external events, some state-tracking optimizations might be necessary.

NOTE: when a sensor evaluation yields multiple <PyObject object="RunRequest"/> objects with the same `run_key`, Dagster will execute all of them. The `run_key` will not be used to deduplicate or skip <PyObject object="RunRequest"/> objects within the same sensor evaluation.

### Sensor optimizations using cursors

When writing a sensor that deals with high-volume events, it might not be feasible to yield a <PyObject object="RunRequest"/> during every sensor evaluation. For example, you may have an `s3` storage bucket that contains thousands of files.
Expand Down