Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding autoscaling to the CWL DAG and SBG end-to-end step-by-step DAG #114

Merged
merged 5 commits into from
Jun 26, 2024

Conversation

LucaCinquini
Copy link
Collaborator

@LucaCinquini LucaCinquini commented Jun 25, 2024

Purpose

  • This PR introduces autoscaling behaviour for 2 existing DAGs: the "cwl_dag" and the "sbg_L1_to_L2_e2e_cwl_step_by_step_dag"

Proposed Changes

  • [CHANGE] Modify "cwl_dag" to use Karpenter for autoscaling
  • [CHANGE] Modify "sbg_L1_to_L2_e2e_cwl_step_by_step_dag" to use Karpenter for autoscaling

Issues

Testing

  • After the changes, both the "cwl_dag" and the "sbg_L1_to_L2_e2e_cwl_step_by_step_dag" completed successfully (single execution only).

@LucaCinquini LucaCinquini requested a review from nikki-t June 26, 2024 16:06
Copy link
Collaborator

@nikki-t nikki-t left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me (although I don't have a ton of experience with Kubernetes and Karpenter). I also saw that it was successfully executed in issue #64.

@LucaCinquini LucaCinquini merged commit 83f7b1d into develop Jun 26, 2024
2 checks passed
@LucaCinquini LucaCinquini deleted the 64-scaling-cwl branch June 26, 2024 16:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants