Skip to content

Conversation

sarahdasko
Copy link

@sarahdasko sarahdasko commented Apr 11, 2025

Summary

Added role_arn to be configurable at the project or model level
Added the ability to configure aws_access_key_id and aws_secret_access_key at the project or model level

Checklist

  • Unit and integration tests covering the common scenarios were added

@CLAassistant
Copy link

CLAassistant commented Apr 11, 2025

CLA assistant check
All committers have signed the CLA.

Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR allows for the configuration of the S3 role ARN at either the global (project) or model level.

  • Added a new integration test class (TestS3RoleArnGlobal) that confirms the compiled SQL contains the correct extra_credentials invocation.
  • Updated the clickhouse adapter implementation to retrieve the role ARN from either the provided parameter or the project configuration.

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
tests/integration/adapter/clickhouse/test_clickhouse_s3.py Introduces new integration tests to verify role ARN injection in compiled SQL.
dbt/adapters/clickhouse/impl.py Updates the S3 source clause generation to include role ARN from configuration.

@sarahdasko sarahdasko requested a review from Copilot April 21, 2025 18:40
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR enhances S3 source configurations by allowing the role_arn to be defined globally or at the model level. Key changes include:

  • Adding integration tests to verify the role_arn and AWS access key configurations.
  • Updating the ClickHouse adapter's S3 source clause function to support role_arn from configuration.
  • Ensuring that the SQL compilation properly includes the new role_arn and AWS credential format.

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
tests/integration/adapter/clickhouse/test_clickhouse_s3.py Added tests to validate AWS access key and role_arn configurations
dbt/adapters/clickhouse/impl.py Updated s3source_clause to integrate role_arn from config

@BentsiLeviav
Copy link
Contributor

Hi @sarahdasko
Would you mind syncing your fork?

@sarahdasko
Copy link
Author

happy to @BentsiLeviav

@BentsiLeviav
Copy link
Contributor

@sarahdasko would need you to merge it one more time :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants