Bodywork and pipeline re-run / conditions / parameter forwarding?

[jondoering]

Hi,

I just discovered bodywork and it looks really interesting.
I am more used to kubeflow or ARGO style ML pipelines and was hence wondering what bodyworks
take on following features is (and if this is already supported or on the roadmap):

• re-running pipelines which did not run successfully due to e.g. technical problems (e.g. not enough resources, network errors etc.)
• is there a plan to support conditionial pipelines (if-then-else pattern)?
• can I easily forward parameters/variables (e.g. filenames or model-name) from one step to another?

I read through the documentation, but could not find anything yet.

Thanks,
Jonathan

[AlexIoannides]

Hi Jonathan,

Thanks for taking an interest in Bodywork.

You can configure schedule pipelines to be retired N times using the --retries variable. For example,

$ bodywork cronjob create \
    --namespace=MY_NAMESPACE \
    --name=MY_PIPELINE \
    --git-repo-url=https://github.com/MY_USERNAME/MY_REPO \
    --git-repo-branch=BRANCH \
    --retries=N

It appears we forgot to document this feature... sorry! You can also configure each individual stage to be retried using the RETRIES variable in a batch stage's config.ini file.

There's no ability for passing parameters from one stage to the next as you can with Argo Workflows or AirFlow. This was a deliberate choice for two reasons: firstly, we have found these sorts of features difficult to work with in the past and got better traction (and easier testing) via persisting state in a database (or even just using files on cloud storage); and secondly, we tried to avoid the temptation to re-invent Argo/AirFlow. The intent was to provide a greatly simplified workflow engine that is much easier to configure and covers ~90% of the most common use-cases in ML pipelines. We may have measured incorrectly, however, so we're open to suggestions - do you think this is a deal-breaker in you opinion?

We applied similar reasoning with the conditional pipelines - in our experience we never used them within ML pipelines, so we decided to leave them out of the initial roadmap. Would you be able to share a use-case that we can use as the basis for a story/ticket to hang on our roadmap?

We're very keen to gain feedback from the community, so if there's anything we can do to help enable you then please let us know.

Many thanks!

Alex