Auth Authentication Fails in GitHub Action Pipeline

[Masoud-Ghodrati]

Issue: OAuth Authentication Fails in GitHub Action Pipeline

Description:

I am trying to use OAuth Authentication for schemachange in my GitHub Action pipeline but encounter the following error message:

Traceback (most recent call last):
  File "/opt/actions-runner/_work/_tool/Python/3.8.18/x64/bin/schemachange", line 8, in <module>
    sys.exit(main())
  File "/opt/actions-runner/_work/_tool/Python/3.8.18/x64/lib/python3.8/site-packages/schemachange/cli.py", line 1309, in main
    deploy_command(config)
  File "/opt/actions-runner/_work/_tool/Python/3.8.18/x64/lib/python3.8/site-packages/schemachange/cli.py", line 601, in deploy_command
    session = SnowflakeSchemachangeSession(config)
  File "/opt/actions-runner/_work/_tool/Python/3.8.18/x64/lib/python3.8/site-packages/schemachange/cli.py", line 283, in __init__
    if self.set_connection_args():
  File "/opt/actions-runner/_work/_tool/Python/3.8.18/x64/lib/python3.8/site-packages/schemachange/cli.py", line 372, in set_connection_args
    oauth_token = self.get_oauth_token()
  File "/opt/actions-runner/_work/_tool/Python/3.8.18/x64/lib/python3.8/site-packages/schemachange/cli.py", line 329, in get_oauth_token
    "url": self.oauth_config["token-provider-url"],
TypeError: 'NoneType' object is not subscriptable

This suggests that the oauth_config is not being correctly passed or is None. However, I have verified that the required environment variables are correctly set and passed in the GitHub Actions pipeline.

Steps to Reproduce:

1. I have the following GitHub Actions pipeline configured:

name: testing

on:
  push:
    branches:
      - feat/*

jobs:
  Deploy:
    runs-on: ubuntu-latest
    permissions:
      id-token: write
      contents: read
    environment: 
      name: 'Dev-SF-DB' 
    env:
      SF_ACCOUNT: ${{ secrets.SF_ACCOUNT }}
      SF_USERNAME: ${{ secrets.SF_USERNAME }}
      SF_ROLE: ${{ secrets.SF_ROLE_MULESOFT }}
      SF_WAREHOUSE: ${{ vars.SF_WAREHOUSE_MULESOFT }}
      SF_DATABASE: ${{ vars.SF_DATABASE }}
      CLIENT_ID: ${{ vars.CLIENT_ID }}
      CLIENT_SECRET: ${{ vars.CLIENT_SECRET }}
      SCOPE_URL_TEST: ${{ vars.SCOPE_URL_TEST }}
      AZURE_ORG_GUID: ${{ vars.AZURE_ORG_GUID }}
      SNOWFLAKE_AUTHENTICATOR: oauth
    steps:
      - name: Checkout repository
        uses: actions/checkout@v4
      
      - name: Use Python 3.8.x
        uses: actions/setup-python@v5
        with:
          python-version: 3.8.x
      
      - name: Install Python dependencies
        run: |
          pip install --upgrade pip
          pip install -r code/python/requirements.txt

      - name: Run python
              run: |
                python code/python/src/snowflake_conn_tocken.py

      - name: Run schemachange to deploy the snowflake Stored Procedures
        run: |
          schemachange -f ${{ github.workspace }}/code/schemachange/stored_procedure \
          -a $SF_ACCOUNT \
          -r $SF_ROLE \
          -u $SF_USERNAME \
          -w $SF_WAREHOUSE \
          -d $SF_DATABASE \
          --create-change-history-table \

2. I have also confirmed that a separate Python script using the same credentials works to authenticate and connect to Snowflake (in the pipeline). Here’s the relevant Python code that works:

import snowflake.connector
import requests
import os

AUTH_CLIENT_ID = os.getenv("CLIENT_ID")
AUTH_CLIENT_SECRET = os.getenv("CLIENT_SECRET")
AUTH_GRANT_TYPE = "client_credentials"
SCOPE_URL = os.getenv("SCOPE_URL_TEST")
AZURE_ORG_GUID = os.getenv("AZURE_ORG_GUID")
url = f"https://login.microsoftonline.com/{AZURE_ORG_GUID}/oauth2/v2.0/token"

req_info = {
            "url": url,
            "headers": {"Content-Type": "application/x-www-form-urlencoded", "User-Agent": "python/schemachange"},
            "data": {"client_id": AUTH_CLIENT_ID, "client_secret": AUTH_CLIENT_SECRET, "grant_type": "client_credentials", "scope": SCOPE_URL},
        }
response = requests.post(**req_info)
TOKEN = response.json()['access_token']

ctx = snowflake.connector.connect(
    account=os.getenv("SF_ACCOUNT"),
    authenticator="OAUTH",
    token=TOKEN,
    warehouse=os.getenv("SF_WAREHOUSE"),
    database=os.getenv("SF_DATABASE"),
    role=os.getenv("SF_ROLE")
)

Potential Cause:

It appears that the schemachange OAuth config is not parsing the environment variables correctly. I suspect the issue might be related to how the OAuth config is being passed within schemachange.

Additional Information:

Here's the config file I’m using with schemachange:

config-version: 1
snowflake-account: ''
snowflake-user: 'user'
snowflake-role: 'role'
snowflake-warehouse: 'warehouse'
oauthconfig:
  token-provider-url: 'https://login.microsoftonline.com/{{ env_var('AZURE_ORG_GUID', 'default') }}/oauth2/v2.0/token'
  token-response-name: 'access_token'
  token-request-headers:
    Content-Type: "application/x-www-form-urlencoded"
    User-Agent: "python/schemachange"
  token-request-payload:
    client_id: '{{ env_var('CLIENT_ID', 'default') }}'
    client_secret: '{{ env_var('CLIENT_SECRET', 'default') }}'
    grant_type: 'client_credentials'
    scope: '{{ env_var('SCOPE_URL_TEST', 'default') }}'

Request for Help:

Can someone please provide guidance on what might be causing the issue? Specifically, it seems like schemachange is unable to parse the OAuth configuration properly in the GitHub Actions pipeline.

[herakilla]

@Masoud-Ghodrati I ran into the same problem myself. When I looked at the code for how the cli is grabbing the config I found that the property in the yml file needs to be oauth_config instead of oauthconfig The CLI line in question

Once I made that change in my schemachange-config.yml the github action started working 😄

I opened PR #286 to fix this so hopefully a maintainer will merge it

[Masoud-Ghodrati]

@Masoud-Ghodrati I ran into the same problem myself. When I looked at the code for how the cli is grabbing the config I found that the property in the yml file needs to be oauth_config instead of oauthconfig The CLI line in question

Once I made that change in my schemachange-config.yml the github action started working 😄

I opened PR #286 to fix this so hopefully a maintainer will merge it

Thanks @herakilla for this, I can't believe my eyes didn't caught this differece.

It seems it's getting the config now, however I have another issue, which is not related to this but it's related to schemachange. I get this error now:

I don't know why I need -u $SF_USERNAME when I use oauth. It should work without username as far as I understand.

Schemachange uses python connector

ctx = snowflake.connector.connect(
    account=os.getenv("SF_ACCOUNT"),
    authenticator="OAUTH",
    token=TOKEN,
    warehouse=os.getenv("SF_WAREHOUSE"),
    database=os.getenv("SF_DATABASE"),
    role=os.getenv("SF_ROLE")
)

and it doesn't need username.