Skip to content

added appgen dag #390

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Apr 16, 2025
Merged

added appgen dag #390

merged 9 commits into from
Apr 16, 2025

Conversation

grallewellyn
Copy link
Contributor

Purpose

This purpose of this PR is to publish a DAG that uses the Docker image running docker in docker to make packaging an algorithm: https://unity-sds.gitbook.io/docs/mdps-overview/tutorials/algorithms-and-algorithm-development/packaging-an-algorithm easier for users
I published the Docker image that we are using in the DAG to our jplmdps account: https://hub.docker.com/r/jplmdps/unity-app-gen

My code for the DAG and Dockerfile is here: unity-sds/unity-app-generator#31

Proposed Changes

  • [ADD] Functioning AppGen DAG

Issues

Relevant issues:
unity-sds/unity-project-management#228
#275
unity-sds/unity-app-generator#21

Testing

Copy link
Collaborator

@LucaCinquini LucaCinquini left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@grallewellyn : if you redeploy your SPS with the latest Docker image version (2.6.0 - as specified in the Airflow tfvars file) you will be able to simplify this DAG by importing POD_LABEL, DEFAULT_LOG_LEVEL and EC2_TYPES from the plugin file.

Additionally, there might not be a need to ask the user to select a disk size and EC2 type - should those be fixed to reasonable values that work for all app-generator jobs? Let's talk to @mcduffie about it.

@LucaCinquini LucaCinquini merged commit 033e325 into develop Apr 16, 2025
2 checks passed
@LucaCinquini LucaCinquini deleted the appgen-dag branch April 16, 2025 20:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants