App Build Automation

[mcduffie]

Automating the app-package build

Acceptance Criteria:

• Ability for a user to trigger an app-package build (authentication?)
• app-package is built from project provided source code in an open repository
• app-package is stored in docker registry
• app-package is registered in Dockstore
• CWL is available through dockstore
• source code provided is stored in a repository for provenance/re-creation

Work Tickets

• Create an API endpoint to push foreign Git repositories into Gitlab to trigger automated application building unity-app-build-trigger#1
• Create an API endpoint to provide status on application build status per Git repository unity-app-build-trigger#2
• Seperate out Gitlab YML process into seperate steps unity-app-build-trigger#3
• Modify automation pipeline such that can be used with keys per subgroup unity-app-build-trigger#4
• Document app build automation pipeline unity-app-build-trigger#5
• Fix error that occurs when the app build pipeline pushes into a repo already existing into Gitlab unity-app-build-trigger#6
• Automate pipeline application building lambda and api gateway deployment unity-ads-deployment#153

[GodwinShen]

@mcduffie please create and/or link the work tickets for the acceptance criteria noted above.

[GodwinShen]

@mcduffie I noticed that the one work ticket (unity-sds/unity-app-build-trigger#1) associated to this task is complete. Is that the only ticket required to close this one out? If not, can you please add any other related work tickets under the "Work Tickets" section here?

[GodwinShen]

Seems like we still need to complete unity-sds/unity-app-build-trigger#2 before closing out this ticket, if so then I will add that ticket to the "Work Tickets" section here.

[mike-gangl]

@edwinsarkissian - we'll need to clean up the EC2 installs for the gitlab runners at somepoint. there is absolutely no version control happening and that's going to be an issue when trying to support dev/test/prod deployments - as what's installed on them will be a factor of when they were installed.

[mike-gangl]

see https://github.com/unity-sds/unity-ads-deployment/blob/main/ci_cd/install_group_runner_x86_64_docker.tftpl#L171-L174 for examples fo what i'm talking about.

[edwinsarkissian]

Yes, I agree. I will need to deal with version control. However, for now the problem is that registering image with the local image registry fails without any error message, and I have no idea what is causing it. I would have expected to see some error message or warning coming out of app-pack-gen or unity-app-gen. Unfortunately, I won't be able to spend more than 5% of my time on Unity, which I have already done. I have to come back to this early next week.