How to define an asset with dynamic partition definition?

[dodid]

I have the following definitions:

# assets.py
START_DATE = "2000-01-01"

partitions_defintion = DailyPartitionsDefinition(start_date=START_DATE)


@asset(partitions_def=partitions_defintion)
def start_date(context) -> str:
    today = context.partition_key
    return today

# definitions.py
id_map = {
    "a": "2020-01-01",
    "b": "2021-01-01",
}

for id, start_date in id_map.items():
    # Loaded asset are all partitioned with start date as ""2000-01-01""
    loaded_assets = load_assets_from_modules([assets], key_prefix=id)

How can I create assets with different partition definitions here? Ideally, we could do:

    loaded_assets = load_assets_from_modules(
        [assets], key_prefix=key, partition_def=DailyPartitionsDefinition(start_date=start_date))

What are possible options? Thanks.

[garethbrickman]

In Dagster, each asset within a single @asset decorator must have a single partitions_def. If you want to create assets with different partition definitions, you would need to define separate assets for each partition definition. Unfortunately, you cannot dynamically change the partition definition for assets loaded from modules using load_assets_from_modules.

But you could programmatically create the assets with the respective partitions_def. Example:

from dagster import DailyPartitionsDefinition, asset

id_map = {
    "a": "2020-01-01",
    "b": "2021-01-01",
}

assets = []

for id, start_date in id_map.items():
    partitions_def = DailyPartitionsDefinition(start_date=start_date)

    @asset(name=f"{id}_asset", partitions_def=partitions_def)
    def dynamic_asset(context):
        today = context.partition_key
        return today

    assets.append(dynamic_asset)