Support for n-dimensional partition definitions

[chrishiste]

What's the use case?

Dagster currently only supports multi-partitions definitions with 2 partitions definitions.

While this works for many use cases, it limits scenarios where workflows depend on more complex partitioning schemas involving three or more dimensions.

Extending support to "n-dimensional partitions" would unlock greater flexibility and utility for partition-based workflows.

Ideas of implementation

No response

Additional information

To work around the limitation, I'm currently encoding multiple dimensions into a single dimension.

Example: Combining region and product into a single encoded string like [region]-[product], while keeping time into it's own dimension.

While functional, this approach is far from ideal, especially in the UI.

Message from the maintainers

Impacted by this issue? Give it a 👍! We factor engagement into prioritization.