Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[docs] - Add Declarative Automation concept page (DOC-321) #23425

Merged
merged 12 commits into from
Aug 7, 2024
4 changes: 2 additions & 2 deletions docs/content/_navigation.json
Original file line number Diff line number Diff line change
Expand Up @@ -170,8 +170,8 @@
"path": "/concepts/partitions-schedules-sensors/sensors"
},
{
"title": "Auto-materialization policies",
"path": "/concepts/assets/asset-auto-execution"
"title": "Declarative Automation (Experimental)",
"path": "/concepts/automation/declarative-automation"
},
{
"title": "Asset Sensors",
Expand Down
4 changes: 2 additions & 2 deletions docs/content/concepts.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -67,8 +67,8 @@ Dagster offers several ways to run data pipelines without manual intervention, i
href="/concepts/partitions-schedules-sensors/sensors"
></ArticleListItem>
<ArticleListItem
title="Auto-materialize policies"
href="/concepts/assets/asset-auto-execution"
title="Declarative Automation"
href="/concepts/automation/declarative-automation"
></ArticleListItem>
<ArticleListItem
title="Asset Sensors"
Expand Down
234 changes: 0 additions & 234 deletions docs/content/concepts/assets/asset-auto-execution.mdx

This file was deleted.

28 changes: 13 additions & 15 deletions docs/content/concepts/automation.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -41,18 +41,16 @@ You can use sensors to run a job or materialize an asset in response to specific

You can also use sensors to act on the status of a job run. Refer to the [Sensors documentation][sensors] to learn more.

### Auto-materialize policies <Experimental />
### Declarative Automation <Experimental />

If you want a declarative approach to automating your pipelines, Auto-materialize policies (AMP) may be a good fit. AMPs allow you to assign policies to assets and let Dagster determine the best approach to keeping assets up-to-date while adhering to those policies.
Declarative Automation allows you to automatically materialize assets when specified criteria are met. Using Declarative Automation, you could update assets:

For example, with AMPs, you can update assets based on:
- When the asset hasn't yet been materialized
- When an asset's upstream dependency has been updated
- After an asset's parents have been updated since a cron tick
- ... based on your own custom conditions

- Whether an upstream dependency has been updated
- Whether an upstream dependency has the latest data from its dependencies
- Whether a materialization has occured since the last tick of a cron schedule
- ... and more

AMPs are declared on an asset-by-asset basis, but can be applied to multiple assets at once. Refer to the [Auto-materializing Assets documentation][auto-materialize-policies] to learn more.
Materialization conditions are declared on an asset-by-asset basis. Refer to the [Declarative Automation documentation][declarative-automation] to learn more.

### Asset Sensors <Experimental />

Expand Down Expand Up @@ -83,7 +81,7 @@ The following cheatsheet contains high-level details about each of the automatio
<tr>
<th
style={{
width: "20%",
width: "15%",
}}
>
Method
Expand Down Expand Up @@ -175,13 +173,13 @@ The following cheatsheet contains high-level details about each of the automatio
</tr>
<tr>
<td>
<a href="/concepts/assets/asset-auto-execution">
Auto-materialize policies
<a href="/concepts/automation/declarative-automation">
Declarative Automation
</a>
</td>
<td>
Automatically materializes an asset or selection of assets when
specified criteria (ex: upstream changes) are met
Automatically materializes an asset when specified criteria (ex:
upstream changes) are met
</td>
<td>
<ul
Expand Down Expand Up @@ -251,4 +249,4 @@ The following cheatsheet contains high-level details about each of the automatio

[asset-sensors]: /concepts/partitions-schedules-sensors/asset-sensors

[auto-materialize-policies]: /concepts/assets/asset-auto-execution
[declarative-automation]: /concepts/automation/declarative-automation
Loading