Help needed with default resources for graphs #17863
-
Hi folks! So imagine we have a dagster graph and a dict of pre-defined "default" resources to re-use when constructing jobs from this graph. I can think of some solutions for this, like using resources-dict fabrics instead of just dicts, but it still looks like a complication over cool dagster design. The question was originally asked in Dagster Slack. |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
You can set default resources on your class Resource1(ConfigurableResource):
foo: str
@op
def op1(context, my_resource: Resource1):
context.log.info(my_resource.foo)
@op
def op2(context, my_resource: Resource1):
context.log.info(my_resource.foo)
@graph
def my_graph():
op1()
op2()
job_with_default_resources = my_graph.to_job(name="job_with_default_resources")
job_with_custom_resources = my_graph.to_job(name="job_with_custom_resource", resource_defs={"my_resource": Resource1(foo="hello world")})
defs = Definitions(
jobs=[job_with_custom_resources, job_with_default_resources],
resources={
"my_resource": Resource1(foo="bar")
}
) With this setup, if you run |
Beta Was this translation helpful? Give feedback.
You can set default resources on your
Definitions
object, and then define custom resources on the individual jobs that need them. for example