-
Notifications
You must be signed in to change notification settings - Fork 1.5k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
4da28c4
commit 9b762f6
Showing
6 changed files
with
75 additions
and
10 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,6 @@ | ||
--- | ||
title: Asset Sensors | ||
sidebar_position: 50 | ||
--- | ||
|
||
### Basic Asset Sensor Example | ||
|
4 changes: 2 additions & 2 deletions
4
...ext/docs/guides/automation/creating-dynamic-pipelines-based-on-external-data.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,6 +1,6 @@ | ||
--- | ||
title: "Creating dynamic pipelines based on external data" | ||
sidebar_position: 3 | ||
sidebar_position: 30 | ||
--- | ||
|
||
# Creating dynamic pipelines based on external data | ||
# Creating dynamic pipelines based on external data |
4 changes: 0 additions & 4 deletions
4
docs/docs-next/docs/guides/automation/running-pipelines-on-a-schedule.md
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,11 +1,74 @@ | ||
--- | ||
title: Scheduling Examples | ||
title: "Scheduling pipelines" | ||
sidebar_label: "Running pipelines on a schedule" | ||
sidebar_position: 10 | ||
--- | ||
|
||
### Basic Schedule Example | ||
## Basic Schedule Example | ||
|
||
A basic schedule is defined by a `JobDefinition` and a `cron_schedule` using the `ScheduleDefinition` class. | ||
|
||
<CodeExample filePath="guides/automation/simple-schedule-example.py" language="python" title="Simple Schedule Example" /> | ||
|
||
For more examples of schedules, see the [How-To Use Schedules](/guides/automation/schedules) guide. | ||
## How to Set Custom Timezones | ||
|
||
By default, schedules without a timezone will run in UTC. If you want to run a schedule in a different timezone, you can | ||
set the `timezone` parameter. | ||
|
||
```python | ||
ecommerce_schedule = ScheduleDefinition( | ||
job=ecommerce_job, | ||
cron_schedule="15 5 * * 1-5", | ||
timezone="America/Los_Angeles", | ||
) | ||
``` | ||
|
||
## How to Create Partitioned Schedules | ||
|
||
If you have a partitioned asset and job, you can create a schedule from the partition using `build_schedule_from_partitioned_job`. | ||
The schedule will execute as the same cadence specified by the partition definition. | ||
|
||
```python | ||
from dagster import ( | ||
asset, | ||
build_schedule_from_partitioned_job, | ||
define_asset_job, | ||
DailyPartitionsDefinition, | ||
) | ||
|
||
daily_partition = DailyPartitionsDefinition(start_date="2024-05-20") | ||
|
||
|
||
@asset(partitions_def=daily_partition) | ||
def daily_asset(): ... | ||
|
||
partitioned_asset_job = define_asset_job("partitioned_job", selection=[daily_asset]) | ||
|
||
# highlight-start | ||
# This partition will run daily | ||
asset_partitioned_schedule = build_schedule_from_partitioned_job( | ||
partitioned_asset_job, | ||
) | ||
# highlight-end | ||
|
||
``` | ||
|
||
If you have a partitioned job, you can create a schedule from the partition using `build_schedule_from_partitioned_job`. | ||
|
||
```python | ||
from dagster import build_schedule_from_partitioned_job, job | ||
|
||
|
||
@job(config=partitioned_config) | ||
def partitioned_op_job(): ... | ||
|
||
# highlight-start | ||
partitioned_op_schedule = build_schedule_from_partitioned_job( | ||
partitioned_op_job, | ||
) | ||
# highlight-end | ||
``` | ||
|
||
--- | ||
|
||
For more information about how Schedules work, see the [About Schedules](/concepts/schedules) concept page. |
2 changes: 1 addition & 1 deletion
2
docs/docs-next/docs/guides/automation/triggering-pipeline-runs-using-events.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,4 +1,4 @@ | ||
--- | ||
title: "Creating event-based pipelines" | ||
sidebar_position: 2 | ||
sidebar_position: 20 | ||
--- |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
extends: capitalization | ||
message: "'%s' should be in sentence case" | ||
level: error | ||
scope: heading | ||
match: $sentence |