Handling MultiPartitions with DuckDB IO Manager

[dagsir[bot]]

I'm setting up a Dagster pipeline in a local environment, and using the DuckDBPandasIOManager because it seems to be pretty good for local environments. I've got an upstream asset with a MultiPartitionsDefinition (a MonthlyPartitionsDefinition and StaticPartitionsDefinition) that will feed into a downstream asset with a single MonthlyPartitionsDefinition partition. The downstream asset is defined as follows:

    group_name="statements",
    description="The statements",
    ins={
        "upstream": AssetIn(
            key=upstream.key,
            partition_mapping=MultiToSingleDimensionPartitionMapping(
                partition_dimension_name="date"
            )
        ),
    },
    partitions_def=monthly_partition_def,
)
def downstream(upstream: pd.DataFrame):```
Whenever I try to materialize this asset I get the following error:
> dagster._check.CheckError: Failure condition: Tried to access partition key for asset 'AssetKey(['upstream'])', but the number of input partitions != 1
<https://github.com/dagster-io/dagster/issues/13290|This looks to be very similar to this error, which has its own issue>. It seems the `DuckDBPandasIOManager`  may also not support inputs with multiple partitions. For now I just want to build this pipeline without worrying about where the data will be written to. Is there an I/O manager that supports inputs with multiple partitions and that works well in a local environment? Am I better writing my own version of the DuckDB I/O manager that has this support, as was originally done in that issue?

---

*The question was originally asked in [Dagster Slack](https://dagster.slack.com/archives/C01U954MEER/p1699467348788989?thread_ts=1699467348.788989&cid=C01U954MEER).*

[jamiedemaria]

duplicate of #17834