Setup role based authentication and authorization for Airflow UI with Cognito

[ramesh-maddegoda]

💡 Description

At the moment, those who have access to the MCP AWS account can use many functionalities of MWAA. It is required restrict the features based on roles of users.

⚔️ Parent Epic / Related Tickets

No response

[jordanpadams]

Status: Tested with 2 different roles but issues with MCP roles. To follow with MCP regarding adding/remove Admin.

• Document how to set this up.

[nutjob4life]

Had a chat with Gabe from MCP, considering temporary users to test with various roles (because we always have the admin role). Need the extra roles to test each situation. Some data persistence issue currently being troubleshooted.

[tloubrieu-jpl]

To be discussed with Gabe when Ramesh is back.

[tloubrieu-jpl]

Ramesh looks at Cognito authentication as an alternative to solve the issue. That allows to attach specific IAM policies to users which match with their actual authorization.

[ramesh-maddegoda]

Tested COgnito integration with the approach proposed with the following Amazon articles.

Accessing a private Amazon MWAA environment using federated identities
https://d1.awsstatic.com/whitepapers/accessing-a-private-amazon-mwaa-environment-using-federated-identities.pdf

Application load balancer single-sign-on for Amazon MWAA
https://github.com/aws-samples/alb-sso-mwaa

There was a problem related with validating the digital signature of the access token. The documentation says, that issue will not be there, when the lambda zip file is built on a linux system with required cryptography libraries.

Regardless of that, it was possible to test the MWAA Role-based access with Cognito users.

[ramesh-maddegoda]

If we do not want to use Cognito for Nucleus users, another option available creating AWS accounts for user, create MWAA IAM for those users such as View, Ops, Admin etc. This option is much easier to implement technically, compared to the Cognito option.

[tloubrieu-jpl]

@ramesh-maddegoda need to present the architecture to the JPL SA's to validate it for production.

[tloubrieu-jpl]

@ramesh-maddegoda provided the architecture proposed to the SAs.

[tloubrieu-jpl]

We decided to move forward with the Cognito based authentication.

[tloubrieu-jpl]

The SSO works but not the redirection between nucleus and cognito.

[tloubrieu-jpl]

@ramesh-maddegoda is still investigating the redirection issue.

[jordanpadams]

Standup status: blocked by intermittent issues with Cognito / Lambda solution for authentication.

[ramesh-maddegoda]

Created pull request #122

[ramesh-maddegoda]

The pull request #122 was merged.

As a part of this ticket, an ALB based approach to enable Cognito authentication for Nucleus Airflow UI was implemented. However, the ALB based approach only worked sometimes and currently there is an Amazon support ticket Case 172781777100323) to troubleshoot it. At the moment, a python script is used to get a web token URL to access Airflow UI.

A new ticket Web based Cognito authentication for Nucleus Airflow UI was created to focus on resolving this ALB related problem and eventually implement web based Cognito authentication for Nucleus Airflow UI.