how to trigger a job 'last' when selected from dagsters ui

[JohnStokes228]

i have a use case in which my users may pick to run any number of independent jobs from the job list. my users are all using the dagster ui to run these commands. what i want, is for no matter which jobs or combination of jobs my users pick to run, a final job can be triggered after the fact that effectively shuts down some azure resources. i dont mind if this job is one they must also pick to run manually, what matters is that it will run after all other selected jobs, and that those jobs may not be known ahead of time since the user picks dynamically.

what is the best way to achieve this with dagster?

[garethbrickman]

One idea is to have a sensor monitoring the independent jobs. When one or more of them has a successful run, the sensor can automatically trigger the final job to run.

Example implementation:

from dagster import sensor, RunRequest, DagsterRunStatus, SensorEvaluationContext

@sensor(job=final_job)
def monitor_jobs_sensor(context: SensorEvaluationContext):
    monitored_jobs = ["independent_job_1", "independent_job_2"]
    completed_jobs = []

    for job_name in monitored_jobs:
        runs = context.instance.get_runs(filters={"job_name": job_name, "status": DagsterRunStatus.SUCCESS})
        if runs:
            completed_jobs.append(job_name)

    if len(completed_jobs) == len(monitored_jobs):
        yield RunRequest(run_key=None, run_config={})

[JohnStokes228]

thanks for the response Gareth. is there documented anywhere all the possible values of DagsterRunStatus at all? and is there any means of evaluating the run status of other jobs from within a job or does it have to be a sensor?

[garethbrickman]

You can see all the values for DagsterRunStatus here:

dagster/python_modules/dagster/dagster/_core/storage/dagster_run.py

Line 48 in e30f056

class DagsterRunStatus(Enum):

You could, also by using the context from the Dagster Instance however that may not be performant. Probably it's better to use a Run Status Sensor as that's purpose designed for your kind of use case.