Skip to content

Commit

Permalink
[dagster-dlt] allow op_tag customization for dlt_assets to be used in…
Browse files Browse the repository at this point in the history
… concurrency limits (#26638)

## Summary & Motivation

I realize now that we we're not exposing `op_tags` for `dlt_assets`
which would be beneficial in setting concurrency limits for dlt assets.
This exposes this parameter to the `multi_asset`

## How I Tested These Changes

## Changelog

- [dagster-dlt] allow customization of `op_tags` in `dlt_assets`
  • Loading branch information
cmpadden authored Dec 20, 2024
1 parent 49c5480 commit 54acb8f
Show file tree
Hide file tree
Showing 2 changed files with 6 additions and 2 deletions.
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
from typing import Any, Callable, Optional, Sequence
from typing import Any, Callable, Mapping, Optional, Sequence

from dagster import (
AssetsDefinition,
Expand Down Expand Up @@ -67,6 +67,7 @@ def dlt_assets(
group_name: Optional[str] = None,
dagster_dlt_translator: Optional[DagsterDltTranslator] = None,
partitions_def: Optional[PartitionsDefinition] = None,
op_tags: Optional[Mapping[str, Any]] = None,
) -> Callable[[Callable[..., Any]], AssetsDefinition]:
"""Asset Factory for using data load tool (dlt).
Expand All @@ -76,6 +77,8 @@ def dlt_assets(
name (Optional[str], optional): The name of the op.
group_name (Optional[str], optional): The name of the asset group.
dagster_dlt_translator (DagsterDltTranslator, optional): Customization object for defining asset parameters from dlt resources.
partitions_def (Optional[PartitionsDefinition]): Optional partitions definition.
op_tags (Optional[Mapping[str, Any]]): The tags for the underlying op.
Examples:
Loading Hubspot data to Snowflake with an auto materialize policy using the dlt verified source:
Expand Down Expand Up @@ -142,6 +145,7 @@ def github_reactions_dagster_assets(context: AssetExecutionContext, dlt: Dagster
group_name=group_name,
can_subset=True,
partitions_def=partitions_def,
op_tags=op_tags,
specs=build_dlt_asset_specs(
dlt_source=dlt_source,
dlt_pipeline=dlt_pipeline,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ def sling_assets(
name (Optional[str]: The name of the op.
partitions_def (Optional[PartitionsDefinition]): The partitions definition for this asset.
backfill_policy (Optional[BackfillPolicy]): The backfill policy for this asset.
op_tags (Optional[Mapping[str, Any]]): The tags for this asset.
op_tags (Optional[Mapping[str, Any]]): The tags for the underlying op.
Examples:
Running a sync by providing a path to a Sling Replication config:
Expand Down

0 comments on commit 54acb8f

Please sign in to comment.