Skip to content

Commit

Permalink
[docs] - Add Declarative Automation concept page (DOC-321) (#23425)
Browse files Browse the repository at this point in the history
## Summary & Motivation

This PR:

- Adds a new concept page for Declarative Automation (DA)
- Updates the **Automation** page with info about DA
- Removes the AMP page
- Updates links from AMP to the DA page
- Adds a redirect from AMP to DA

TODO:

- [x] Add screenshots
- [ ] Add customization guide + update links
- [x] Update sidenav
- [x] TBD: Remove AMP page + add redirect?

## How I Tested These Changes
  • Loading branch information
erinkcochran87 authored Aug 7, 2024
1 parent 49d46e6 commit db05469
Show file tree
Hide file tree
Showing 15 changed files with 232 additions and 304 deletions.
4 changes: 2 additions & 2 deletions docs/content/_navigation.json
Original file line number Diff line number Diff line change
Expand Up @@ -170,8 +170,8 @@
"path": "/concepts/partitions-schedules-sensors/sensors"
},
{
"title": "Auto-materialization policies",
"path": "/concepts/assets/asset-auto-execution"
"title": "Declarative Automation (Experimental)",
"path": "/concepts/automation/declarative-automation"
},
{
"title": "Asset Sensors",
Expand Down
4 changes: 2 additions & 2 deletions docs/content/concepts.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -67,8 +67,8 @@ Dagster offers several ways to run data pipelines without manual intervention, i
href="/concepts/partitions-schedules-sensors/sensors"
></ArticleListItem>
<ArticleListItem
title="Auto-materialize policies"
href="/concepts/assets/asset-auto-execution"
title="Declarative Automation"
href="/concepts/automation/declarative-automation"
></ArticleListItem>
<ArticleListItem
title="Asset Sensors"
Expand Down
234 changes: 0 additions & 234 deletions docs/content/concepts/assets/asset-auto-execution.mdx

This file was deleted.

28 changes: 13 additions & 15 deletions docs/content/concepts/automation.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -41,18 +41,16 @@ You can use sensors to run a job or materialize an asset in response to specific

You can also use sensors to act on the status of a job run. Refer to the [Sensors documentation][sensors] to learn more.

### Auto-materialize policies <Experimental />
### Declarative Automation <Experimental />

If you want a declarative approach to automating your pipelines, Auto-materialize policies (AMP) may be a good fit. AMPs allow you to assign policies to assets and let Dagster determine the best approach to keeping assets up-to-date while adhering to those policies.
Declarative Automation allows you to automatically materialize assets when specified criteria are met. Using Declarative Automation, you could update assets:

For example, with AMPs, you can update assets based on:
- When the asset hasn't yet been materialized
- When an asset's upstream dependency has been updated
- After an asset's parents have been updated since a cron tick
- ... based on your own custom conditions

- Whether an upstream dependency has been updated
- Whether an upstream dependency has the latest data from its dependencies
- Whether a materialization has occured since the last tick of a cron schedule
- ... and more

AMPs are declared on an asset-by-asset basis, but can be applied to multiple assets at once. Refer to the [Auto-materializing Assets documentation][auto-materialize-policies] to learn more.
Materialization conditions are declared on an asset-by-asset basis. Refer to the [Declarative Automation documentation][declarative-automation] to learn more.

### Asset Sensors <Experimental />

Expand Down Expand Up @@ -83,7 +81,7 @@ The following cheatsheet contains high-level details about each of the automatio
<tr>
<th
style={{
width: "20%",
width: "15%",
}}
>
Method
Expand Down Expand Up @@ -175,13 +173,13 @@ The following cheatsheet contains high-level details about each of the automatio
</tr>
<tr>
<td>
<a href="/concepts/assets/asset-auto-execution">
Auto-materialize policies
<a href="/concepts/automation/declarative-automation">
Declarative Automation
</a>
</td>
<td>
Automatically materializes an asset or selection of assets when
specified criteria (ex: upstream changes) are met
Automatically materializes an asset when specified criteria (ex:
upstream changes) are met
</td>
<td>
<ul
Expand Down Expand Up @@ -251,4 +249,4 @@ The following cheatsheet contains high-level details about each of the automatio

[asset-sensors]: /concepts/partitions-schedules-sensors/asset-sensors

[auto-materialize-policies]: /concepts/assets/asset-auto-execution
[declarative-automation]: /concepts/automation/declarative-automation
Loading

1 comment on commit db05469

@github-actions
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deploy preview for dagster-docs ready!

✅ Preview
https://dagster-docs-9uglxawop-elementl.vercel.app
https://master.dagster.dagster-docs.io

Built with commit db05469.
This pull request is being automatically deployed with vercel-action

Please sign in to comment.