Airflow integration into Unity Marketplace

[mike-gangl]

add Airflow and OGC Endpoints to Unity Marketplace and Management Console Interface

Acceptance Criteria:

• Airflow is not deployed automatically to a venue until explicitly selected
• Airflow nodes are configurable (number of instances, types, spot, etc via management console)
• Market place entry and artifacts are created (See U-CS)
• Deployment form Management Interface can be done to a project venue
• Deployment integrates with Venue API Gateway (e.g. venue httpd backend)
• Deployment is linked with Unity A&A / cognito (TBD)
• Health check endpoint is available and defined in marketplace entry
• Integration question How does Airflow/CWL DAG know what U-DS bucket it should be writing to with the new OGC App Package/stage-out updates? Is this an SSM param? is it passed in at deploy time?
• NEW Gherkin Test case(s) added to System Test DURING THE PLANNING - how will we test the marketplace feature. Is it manual? or can it be automated?
• NEW Gherkin Test case(s) implementation added to System Test during PI. If automated, these must implement the above test cases(s)
• What Validation test can we use to test the deployment is correctly working?

[GodwinShen]

@jpl-btlunsfo is working on the EKS and Carpenter stuff.