Skip to content

Commit

Permalink
more links
Browse files Browse the repository at this point in the history
Signed-off-by: nikki everett <[email protected]>
  • Loading branch information
neverett committed Dec 19, 2024
1 parent 0896fab commit 59419ac
Show file tree
Hide file tree
Showing 14 changed files with 24 additions and 24 deletions.
2 changes: 1 addition & 1 deletion docs/docs-beta/docs/dagster-plus/features/catalog-views.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ In this guide, you'll learn how to create, access, and share catalog views with
<summary>Prerequisites</summary>

- **Organization Admin**, **Admin**, or **Editor** permissions on Dagster+
- Familiarity with [Assets](/guides/build/create-asset-pipelines/assets-concepts/index.mdx and [Asset metadata](/guides/build/create-a-pipeline/metadata)
- Familiarity with [Assets](/guides/build/create-asset-pipelines/assets-concepts/index.mdx and [Asset metadata](/guides/build/create-asset-pipelines/metadata)

</details>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ You'll need one or more assets that emit the same metadata key at run time. Insi
are most valuable when you have multiple assets that emit the same kind of metadata, such as
such as the number of rows processed or the size of a file uploaded to object storage.

Follow [the metadata guide](/guides/build/create-a-pipeline/metadata#runtime-metadata) to add numeric metadata
Follow [the metadata guide](/guides/build/create-asset-pipelines/metadata#runtime-metadata) to add numeric metadata
to your asset materializations.

## Step 2: Enable viewing your metadata in Dagster+ Insights
Expand Down
2 changes: 1 addition & 1 deletion docs/docs-beta/docs/getting-started/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,4 +70,4 @@ If you encounter any issues during the installation process:
## Next steps

- Get up and running with your first Dagster project in the [Quickstart](/getting-started/quickstart)
- Learn to [create data assets in Dagster](/guides/build/create-a-pipeline/data-assets)
- Learn to [create data assets in Dagster](/guides/build/create-asset-pipelines/data-assets)
2 changes: 1 addition & 1 deletion docs/docs-beta/docs/guides/automate/schedules.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ If using partitions and jobs, you can create a schedule using the partition with
<Tabs>
<TabItem value="assets" label="Assets">

If you have a [partitioned asset](/guides/build/create-a-pipeline/partitioning) and job:
If you have a [partitioned asset](/guides/build/create-asset-pipelines/partitioning) and job:

<CodeExample filePath="guides/automation/schedule-with-partition.py" language="python" />

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ This guide demonstrates how to standardize database connections and customize th

To follow the steps in this guide, you'll need:

- Familiarity with [Assets](/guides/build/create-a-pipeline/data-assets)
- Familiarity with [Assets](/guides/build/create-asset-pipelines/data-assets)

To run the examples in this guide, you'll need:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ You'll often want to be able to adjust parameters when materializing assets, whi

To follow the steps in this guide, you'll need familiarity with:

- [Assets](/guides/build/create-a-pipeline/data-assets)
- [Assets](/guides/build/create-asset-pipelines/data-assets)
- [Pydantic](https://docs.pydantic.dev/latest/)

</details>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -72,5 +72,5 @@ flowchart LR
## Next steps

- Learn to create [dependencies between assets](/guides/build/create-asset-pipelines/assets-concepts/asset-dependencies)
- Enrich Dagster's built-in data catalog with [asset metadata](/guides/build/create-a-pipeline/metadata)
- Enrich Dagster's built-in data catalog with [asset metadata](/guides/build/create-asset-pipelines/metadata)
- Learn to use a [factory pattern](/guides/build/configure/asset-factories) to create multiple, similar assets
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ For example, external assets could be:

To follow the steps in this guide, you'll need:

- Familiarity with [Assets](/guides/build/create-a-pipeline/data-assets)
- Familiarity with [Assets](/guides/build/create-asset-pipelines/data-assets)
- Familiarity with [Sensors](/guides/automate/sensors)
</details>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -46,10 +46,10 @@ Most Dagster pipelines follow these steps:

To start building your Dagster pipeline, dive into the following guides:

- [Define data assets](/guides/build/create-a-pipeline/data-assets)
- [Define data assets](/guides/build/create-asset-pipelines/data-assets)
- [Create dependencies between assets](/guides/build/create-asset-pipelines/assets-concepts/asset-dependencies)
- [Enrich assets with metadata](/guides/build/create-a-pipeline/metadata)
- [Partition assets](/guides/build/create-a-pipeline/partitioning)
- [Represent external data sources](/guides/build/create-a-pipeline/external-assets)
- [Enrich assets with metadata](/guides/build/create-asset-pipelines/metadata)
- [Partition assets](/guides/build/create-asset-pipelines/partitioning)
- [Represent external data sources](/guides/build/create-asset-pipelines/external-assets)

By following these guides, you'll learn how to create powerful, maintainable data pipelines using Dagster's asset-based approach.
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ Using metadata in Dagster, you can:

To follow the steps in this guide, you'll need:

- Familiarity with [Assets](/guides/build/create-a-pipeline/data-assets)
- Familiarity with [Assets](/guides/build/create-asset-pipelines/data-assets)
</details>

## Adding owners to assets \{#owners}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,8 @@ Partitioned assets in Dagster can have dependencies on other partitioned assets,

To follow the steps in this guide, you'll need:

- Familiarity with [Assets](/guides/build/create-a-pipeline/data-assets)
- Familiarity with [Partitions](/guides/build/create-a-pipeline/partitioning)
- Familiarity with [Assets](/guides/build/create-asset-pipelines/data-assets)
- Familiarity with [Partitions](/guides/build/create-asset-pipelines/partitioning)

</details>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ There are several ways to partition your data in Dagster:

To follow the steps in this guide, you'll need:

- Familiarity with [Assets](/guides/build/create-a-pipeline/data-assets)
- Familiarity with [Assets](/guides/build/create-asset-pipelines/data-assets)

</details>

Expand Down
10 changes: 5 additions & 5 deletions docs/docs-beta/docs/guides/test/asset-checks.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ A asset check is defined using the `@asset_check` decorator.

The following example defines an asset check on an asset that fails if the `order_id` column of the asset contains a null value. The asset check will run after the asset has been materialized.

<CodeExample filePath="guides/build/create-a-pipeline/data-assets/quality-testing/asset-checks/single-asset-check.py" language="python" />
<CodeExample filePath="guides/build/create-asset-pipelines/data-assets/quality-testing/asset-checks/single-asset-check.py" language="python" />

## Defining multiple asset checks \{#multiple-checks}

Expand All @@ -53,29 +53,29 @@ The following example defines two asset checks using the `@multi_asset_check` de

In this example, both asset checks will run in a single operation after the asset has been materialized.

<CodeExample filePath="guides/build/create-a-pipeline/data-assets/quality-testing/asset-checks/multiple-asset-checks.py" language="python" />
<CodeExample filePath="guides/build/create-asset-pipelines/data-assets/quality-testing/asset-checks/multiple-asset-checks.py" language="python" />

## Programmatically generating asset checks \{#factory-pattern}

Defining multiple checks can also be done using a factory pattern. The example below defines the same two asset checks as in the previous example, but this time using a factory pattern and the `@multi_asset_check` decorator.

<CodeExample filePath="guides/build/create-a-pipeline/data-assets/quality-testing/asset-checks/asset-checks-factory.py" language="python" />
<CodeExample filePath="guides/build/create-asset-pipelines/data-assets/quality-testing/asset-checks/asset-checks-factory.py" language="python" />

## Blocking downstream materialization

By default, if a parent's asset check fails during a run, the run will continue and downstream assets will be materialized. To prevent this behavior, set the `blocking` argument to `True` in the `@asset_check` decorator.

In the example bellow, if the `orders_id_has_no_nulls` check fails, the downstream `augmented_orders` asset won't be materialized.

<CodeExample filePath="guides/build/create-a-pipeline/data-assets/quality-testing/asset-checks/block-downstream-with-asset-checks.py" language="python" />
<CodeExample filePath="guides/build/create-asset-pipelines/data-assets/quality-testing/asset-checks/block-downstream-with-asset-checks.py" language="python" />

## Scheduling and monitoring asset checks

In some cases, running asset checks separately from the job materializing the assets can be useful. For example, running all data quality checks once a day and sending an alert if they fail. This can be achieved using schedules and sensors.

In the example below, two jobs are defined: one for the asset and another for the asset check. Schedules are defined to materialize the asset and execute the asset check independently. A sensor is defined to send an email alert when the asset check job fails.

<CodeExample filePath="guides/build/create-a-pipeline/data-assets/quality-testing/asset-checks/asset-checks-with-schedule-and-sensor.py" language="python" />
<CodeExample filePath="guides/build/create-asset-pipelines/data-assets/quality-testing/asset-checks/asset-checks-with-schedule-and-sensor.py" language="python" />

## Next steps

Expand Down
6 changes: 3 additions & 3 deletions docs/docs-beta/docs/guides/test/data-freshness-testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,8 @@ Freshness checks can also communicate SLAs for their data freshness. For example

To follow the steps in this guide, you'll need familiarity with:

- [Assets](/guides/build/create-a-pipeline/data-assets)
- [External assets](/guides/build/create-a-pipeline/external-assets)
- [Assets](/guides/build/create-asset-pipelines/data-assets)
- [External assets](/guides/build/create-asset-pipelines/external-assets)
- [Asset checks](/guides/test/asset-checks)

</details>
Expand All @@ -45,7 +45,7 @@ The example below defines a freshness check on an asset that fails if the asset'

## External asset freshness \{#external-assets}

[External assets](/guides/build/create-a-pipeline/external-assets) are assets orchestrated by systems other than Dagster.
[External assets](/guides/build/create-asset-pipelines/external-assets) are assets orchestrated by systems other than Dagster.

To run freshness checks on external assets, the checks need to know when the external assets were last updated. Emitting these update timestamps as values for the `dagster/last_updated_timestamp` observation metadata key allows Dagster to calculate whether the asset is overdue.

Expand Down

0 comments on commit 59419ac

Please sign in to comment.