-
Notifications
You must be signed in to change notification settings - Fork 1.5k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[docs-beta] Update to dagster as dg (#23864)
## Summary & Motivation Update the beta docs code snippets to use the new `import dagster as dg` code pattern. This should be the new approach to all documentation that references dagster code, as consistency is important. Here are a few reasons why I like this approach: 0) There is prior art in many libraries of this approach, notably `pandas`, `sklearn`, `numpy`, `matplotlib`, `pytest` and many other packages that export many objects. 1) Fewer import lines in code samples/snippets. Compare the first image which is 12 lines shorter. This makes code easier for users to understand parse. ![image.png](https://graphite-user-uploaded-assets-prod.s3.amazonaws.com/80phFRWNTJBzbgP9Xj13/1c9cc21d-3b04-4eab-b174-77d23cf74d63.png) 2) Another side benefit is that type-ahead experience is improved. In VS Code, for example, typing `dg.sensor` and pressing Ctrl+Space provides you with all the exported Sensor-like objects ![image.png](https://graphite-user-uploaded-assets-prod.s3.amazonaws.com/80phFRWNTJBzbgP9Xj13/3c0776cb-8006-488e-849a-36ae64c193a1.png) Compare to typing `sensor` <ctrl-space> instead with `import dagster` ![image.png](https://graphite-user-uploaded-assets-prod.s3.amazonaws.com/80phFRWNTJBzbgP9Xj13/8de32f3e-7ac3-4902-84a5-ebba88ebea24.png) 3) Finally, we can always revert this change in the future by searching the examples for `dg.` if we decide this approach is fundamentally broken. ## How I Tested These Changes bk, pytest ## Changelog [New | Bug | Docs] NOCHANGELOG
- Loading branch information
1 parent
8fa1b6e
commit 9dcf00e
Showing
22 changed files
with
158 additions
and
220 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
38 changes: 13 additions & 25 deletions
38
examples/docs_beta_snippets/docs_beta_snippets/guides/automation/asset-sensor-custom-eval.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
23 changes: 8 additions & 15 deletions
23
examples/docs_beta_snippets/docs_beta_snippets/guides/automation/multi-asset-sensor.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,35 +1,28 @@ | ||
from dagster import ( | ||
AssetKey, | ||
MultiAssetSensorEvaluationContext, | ||
RunRequest, | ||
asset, | ||
define_asset_job, | ||
multi_asset_sensor, | ||
) | ||
import dagster as dg | ||
|
||
|
||
@asset | ||
@dg.asset | ||
def target_asset(): | ||
pass | ||
|
||
|
||
downstream_job = define_asset_job("downstream_job", [target_asset]) | ||
downstream_job = dg.define_asset_job("downstream_job", [target_asset]) | ||
|
||
|
||
@multi_asset_sensor( | ||
@dg.multi_asset_sensor( | ||
monitored_assets=[ | ||
AssetKey("upstream_asset_1"), | ||
AssetKey("upstream_asset_2"), | ||
dg.AssetKey("upstream_asset_1"), | ||
dg.AssetKey("upstream_asset_2"), | ||
], | ||
job=downstream_job, | ||
) | ||
def my_multi_asset_sensor(context: MultiAssetSensorEvaluationContext): | ||
def my_multi_asset_sensor(context: dg.MultiAssetSensorEvaluationContext): | ||
run_requests = [] | ||
for ( | ||
asset_key, | ||
materialization, | ||
) in context.latest_materialization_records_by_key().items(): | ||
if materialization: | ||
run_requests.append(RunRequest(asset_selection=[asset_key])) | ||
run_requests.append(dg.RunRequest(asset_selection=[asset_key])) | ||
context.advance_cursor({asset_key: materialization}) | ||
return run_requests |
15 changes: 5 additions & 10 deletions
15
examples/docs_beta_snippets/docs_beta_snippets/guides/automation/schedule-with-partition.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,22 +1,17 @@ | ||
from dagster import ( | ||
DailyPartitionsDefinition, | ||
asset, | ||
build_schedule_from_partitioned_job, | ||
define_asset_job, | ||
) | ||
import dagster as dg | ||
|
||
daily_partition = DailyPartitionsDefinition(start_date="2024-05-20") | ||
daily_partition = dg.DailyPartitionsDefinition(start_date="2024-05-20") | ||
|
||
|
||
@asset(partitions_def=daily_partition) | ||
@dg.asset(partitions_def=daily_partition) | ||
def daily_asset(): ... | ||
|
||
|
||
partitioned_asset_job = define_asset_job("partitioned_job", selection=[daily_asset]) | ||
partitioned_asset_job = dg.define_asset_job("partitioned_job", selection=[daily_asset]) | ||
|
||
# highlight-start | ||
# This partition will run daily | ||
asset_partitioned_schedule = build_schedule_from_partitioned_job( | ||
asset_partitioned_schedule = dg.build_schedule_from_partitioned_job( | ||
partitioned_asset_job, | ||
) | ||
# highlight-end |
26 changes: 9 additions & 17 deletions
26
...es/docs_beta_snippets/docs_beta_snippets/guides/automation/simple-asset-sensor-example.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
12 changes: 6 additions & 6 deletions
12
examples/docs_beta_snippets/docs_beta_snippets/guides/automation/simple-schedule-example.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
8 changes: 4 additions & 4 deletions
8
..._snippets/docs_beta_snippets/guides/data-assets/passing-data-assets/passing-data-avoid.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,20 +1,20 @@ | ||
from dagster import asset | ||
import dagster as dg | ||
|
||
|
||
# Warning! This is not the right way to create assets | ||
@asset | ||
@dg.asset | ||
def download_files(): | ||
# Download files from S3, the web, etc. | ||
... | ||
|
||
|
||
@asset | ||
@dg.asset | ||
def unzip_files(): | ||
# Unzip files to local disk or persistent storage | ||
... | ||
|
||
|
||
@asset | ||
@dg.asset | ||
def load_data(): | ||
# Read data previously written and store in a data warehouse | ||
... |
Oops, something went wrong.
9dcf00e
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Deploy preview for dagster-docs-beta ready!
✅ Preview
https://dagster-docs-beta-ivjpuqbv1-elementl.vercel.app
https://dagster-docs-beta.dagster-docs.io
Built with commit 9dcf00e.
This pull request is being automatically deployed with vercel-action