Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Core] Drop Track 1 SDK authentication #29631

Merged
merged 1 commit into from
Feb 13, 2025
Merged

[Core] Drop Track 1 SDK authentication #29631

merged 1 commit into from
Feb 13, 2025

Conversation

jiasli
Copy link
Member

@jiasli jiasli commented Aug 1, 2024

Description
Close #20460

After migrating the last Track 1 SDK azure-batch to Track 2 (#30501), Azure CLI no longer depends on any Track 1 SDK. This PR drops Track 1 SDK authentication support.

But before this, there are several incorrect/incomplete Track 2 migrations that must be fixed:

They are all caused by passing resource via get_login_credentials(), instead of passing credential scopes when creating the client. To make things worse, different data-plane SDKs have different implementations and forms of credential scopes.

Copy link

azure-client-tools-bot-prd bot commented Aug 1, 2024

️✔️AzureCLI-FullTest
️✔️acr
️✔️2020-09-01-hybrid
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.12
️✔️3.9
️✔️acs
️✔️2020-09-01-hybrid
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.12
️✔️3.9
️✔️advisor
️✔️latest
️✔️3.12
️✔️3.9
️✔️ams
️✔️latest
️✔️3.12
️✔️3.9
️✔️apim
️✔️latest
️✔️3.12
️✔️3.9
️✔️appconfig
️✔️latest
️✔️3.12
️✔️3.9
️✔️appservice
️✔️latest
️✔️3.12
️✔️3.9
️✔️aro
️✔️latest
️✔️3.12
️✔️3.9
️✔️backup
️✔️latest
️✔️3.12
️✔️3.9
️✔️batch
️✔️latest
️✔️3.12
️✔️3.9
️✔️batchai
️✔️latest
️✔️3.12
️✔️3.9
️✔️billing
️✔️latest
️✔️3.12
️✔️3.9
️✔️botservice
️✔️latest
️✔️3.12
️✔️3.9
️✔️cdn
️✔️latest
️✔️3.12
️✔️3.9
️✔️cloud
️✔️latest
️✔️3.12
️✔️3.9
️✔️cognitiveservices
️✔️latest
️✔️3.12
️✔️3.9
️✔️compute_recommender
️✔️latest
️✔️3.12
️✔️3.9
️✔️computefleet
️✔️latest
️✔️3.12
️✔️3.9
️✔️config
️✔️latest
️✔️3.12
️✔️3.9
️✔️configure
️✔️latest
️✔️3.12
️✔️3.9
️✔️consumption
️✔️latest
️✔️3.12
️✔️3.9
️✔️container
️✔️latest
️✔️3.12
️✔️3.9
️✔️containerapp
️✔️latest
️✔️3.12
️✔️3.9
️✔️core
️✔️2018-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.12
️✔️3.9
️✔️cosmosdb
️✔️latest
️✔️3.12
️✔️3.9
️✔️databoxedge
️✔️2019-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.12
️✔️3.9
️✔️dls
️✔️latest
️✔️3.12
️✔️3.9
️✔️dms
️✔️latest
️✔️3.12
️✔️3.9
️✔️eventgrid
️✔️latest
️✔️3.12
️✔️3.9
️✔️eventhubs
️✔️latest
️✔️3.12
️✔️3.9
️✔️feedback
️✔️latest
️✔️3.12
️✔️3.9
️✔️find
️✔️latest
️✔️3.12
️✔️3.9
️✔️hdinsight
️✔️latest
️✔️3.12
️✔️3.9
️✔️identity
️✔️latest
️✔️3.12
️✔️3.9
️✔️iot
️✔️2019-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.12
️✔️3.9
️✔️keyvault
️✔️2018-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.12
️✔️3.9
️✔️lab
️✔️latest
️✔️3.12
️✔️3.9
️✔️managedservices
️✔️latest
️✔️3.12
️✔️3.9
️✔️maps
️✔️latest
️✔️3.12
️✔️3.9
️✔️marketplaceordering
️✔️latest
️✔️3.12
️✔️3.9
️✔️monitor
️✔️latest
️✔️3.12
️✔️3.9
️✔️mysql
️✔️latest
️✔️3.12
️✔️3.9
️✔️netappfiles
️✔️latest
️✔️3.12
️✔️3.9
️✔️network
️✔️2018-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.12
️✔️3.9
️✔️policyinsights
️✔️latest
️✔️3.12
️✔️3.9
️✔️privatedns
️✔️latest
️✔️3.12
️✔️3.9
️✔️profile
️✔️latest
️✔️3.12
️✔️3.9
️✔️rdbms
️✔️latest
️✔️3.12
️✔️3.9
️✔️redis
️✔️latest
️✔️3.12
️✔️3.9
️✔️relay
️✔️latest
️✔️3.12
️✔️3.9
️✔️resource
️✔️2018-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.12
️✔️3.9
️✔️role
️✔️latest
️✔️3.12
️✔️3.9
️✔️search
️✔️latest
️✔️3.12
️✔️3.9
️✔️security
️✔️latest
️✔️3.12
️✔️3.9
️✔️servicebus
️✔️latest
️✔️3.12
️✔️3.9
️✔️serviceconnector
️✔️latest
️✔️3.12
️✔️3.9
️✔️servicefabric
️✔️latest
️✔️3.12
️✔️3.9
️✔️signalr
️✔️latest
️✔️3.12
️✔️3.9
️✔️sql
️✔️latest
️✔️3.12
️✔️3.9
️✔️sqlvm
️✔️latest
️✔️3.12
️✔️3.9
️✔️storage
️✔️2018-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.12
️✔️3.9
️✔️synapse
️✔️latest
️✔️3.12
️✔️3.9
️✔️telemetry
️✔️2018-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.12
️✔️3.9
️✔️util
️✔️latest
️✔️3.12
️✔️3.9
️✔️vm
️✔️2018-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.12
️✔️3.9

Copy link

azure-client-tools-bot-prd bot commented Aug 1, 2024

️✔️AzureCLI-BreakingChangeTest
️✔️Non Breaking Changes

@yonzhan
Copy link
Collaborator

yonzhan commented Aug 1, 2024

Core

@@ -97,6 +97,7 @@ def scopes_to_resource(scopes):


def _normalize_scopes(scopes):
# TODO: Drop this function
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function is to support very old Track 2 SDKs that doesn't comply with the current authentication interface. We should consider dropping their support as well.

@evelyn-ys
Copy link
Member

Storage track1 use customized token updater which calls Profile.get_raw_token() each time

def timer_callback(self):
# call to get a new token and set a timer
from azure.cli.core._profile import Profile
from datetime import datetime
# should give back token that is valid for at least 5 mins
token = Profile(cli_ctx=self.cli_ctx).get_raw_token(
resource="https://storage.azure.com", subscription=self.cli_ctx.data['subscription_id'])[0][2]
try:
self.token_credential.token = token['accessToken']
expire = token['expiresOn']
seconds_left = (datetime.strptime(expire, "%Y-%m-%d %H:%M:%S.%f") - datetime.now()).total_seconds()
except KeyError: # needed to deal with differing unserialized MSI token payload
self.token_credential.token = token['access_token']
expire = datetime.fromtimestamp(int(token['expires_on']))
seconds_left = (expire - datetime.now()).total_seconds()
if seconds_left < 180:
logger.warning("Acquired token will expire on %s. Current time is %s.", expire, datetime.now())
with self.lock:
self.timer = threading.Timer(seconds_left - 180, self.timer_callback)
self.timer.daemon = True
self.timer.start()

So it won't block signed_session() deprecation from cli core

Comment on lines +16 to +15
def __init__(self, credential, auxiliary_credentials=None):
"""Cross-tenant credential adaptor. It takes a main credential and auxiliary credentials.

It implements Track 2 SDK's azure.core.credentials.TokenCredential by exposing get_token.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even though Track 1 SDK auth support is dropped, CredentialAdaptor is preserved as it will be repurposed as an SDK-MSAL adaptor in #29955.

profile = Profile(cli_ctx=cli_ctx)
credential, _, _ = profile.get_login_credentials()
bearer_token = credential.get_token().token
scopes = resource_to_scopes(cli_ctx.cloud.endpoints.active_directory_resource_id)
bearer_token = credential.get_token(*scopes).token
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

get_token should never be called without scopes. After Track 1 SDK auth support removal, these tests fails

FAILED src/azure-cli/azure/cli/command_modules/appservice/tests/latest/test_webapp_commands.py::AppServiceLogTest::test_download_win_web_log
FAILED src/azure-cli/azure/cli/command_modules/appservice/tests/latest/test_webapp_commands.py::WebappZipDeployScenarioTest::test_deploy_zip
FAILED src/azure-cli/azure/cli/command_modules/appservice/tests/latest/test_webapp_commands.py::WebappDeploymentLogsScenarioTest::test_webapp_list_deployment_logs
FAILED src/azure-cli/azure/cli/command_modules/appservice/tests/latest/test_webapp_commands.py::WebappDeploymentLogsScenarioTest::test_webapp_show_deployment_logs
FAILED src/azure-cli/azure/cli/command_modules/appservice/tests/latest/test_webapp_commands.py::WebappOneDeployScenarioTest::test_one_deploy_arm
FAILED src/azure-cli/azure/cli/command_modules/appservice/tests/latest/test_webapp_commands.py::WebappOneDeployScenarioTest::test_one_deploy_scm
FAILED src/azure-cli/azure/cli/command_modules/appservice/tests/latest/test_webapp_commands.py::TrackRuntimeStatusTest::test_webapp_deployment_source_disable_tracking_runtimestatus
FAILED src/azure-cli/azure/cli/command_modules/appservice/tests/latest/test_webapp_commands.py::TrackRuntimeStatusTest::test_webapp_deployment_source_track_runtimestatus_buildfailed
FAILED src/azure-cli/azure/cli/command_modules/appservice/tests/latest/test_webapp_commands.py::TrackRuntimeStatusTest::test_webapp_deployment_source_track_runtimestatus_runtimefailed
FAILED src/azure-cli/azure/cli/command_modules/appservice/tests/latest/test_webapp_commands.py::TrackRuntimeStatusTest::test_webapp_deployment_source_track_runtimestatus_runtimesucessful
FAILED src/azure-cli/azure/cli/command_modules/appservice/tests/latest/test_webapp_commands.py::TrackRuntimeStatusTest::test_webapp_track_runtimestatus_buildfailed
FAILED src/azure-cli/azure/cli/command_modules/appservice/tests/latest/test_webapp_commands.py::TrackRuntimeStatusTest::test_webapp_track_runtimestatus_runtimefailed
FAILED src/azure-cli/azure/cli/command_modules/appservice/tests/latest/test_webapp_commands.py::TrackRuntimeStatusTest::test_webapp_track_runtimestatus_runtimesucessful

https://dev.azure.com/azclitools/public/_build/results?buildId=219257&view=logs&j=3791f883-8843-5e94-fc79-c8ca993c0a42&t=fc099b28-42bc-5603-6ee8-d61b88ef47c8

self = <azure.cli.testsdk.patches.patch_retrieve_token_for_user.<locals>.get_user_credential_mock.<locals>.UserCredentialMock object at 0x7f7307d80fe0>
scopes = (), kwargs = {}

    def get_token(self, *scopes, **kwargs):  # pylint: disable=unused-argument
        # Old Track 2 SDKs are no longer supported. ***/pull/29690
>       assert len(scopes) == 1, "'scopes' must contain only one element."
E       AssertionError: 'scopes' must contain only one element.

src/azure-cli-testsdk/azure/cli/testsdk/patches.py:73: AssertionError

Copy link
Member Author

@jiasli jiasli Feb 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The access token is used to call scm_url (such as https://webapp-win-log000002.scm.azurewebsites.net), but the token's audience is ARM (https://management.core.windows.net/). This seems incorrect.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kamperiadis Could you please help take a look at this question or involve the person who can help take a look?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tagging @amberwang113 since this is related to webapps

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I created another PR #30797 to address this.

@jiasli
Copy link
Member Author

jiasli commented Feb 5, 2025

azure-mgmt-devtestlabs is still Track 1, causing failure in src/azure-cli/azure/cli/command_modules/vm/tests/latest/test_vm_commands.py::VMAutoShutdownScenarioTest::test_vm_auto_shutdown:

https://dev.azure.com/azclitools/public/_build/results?buildId=219257&view=logs&j=c2322915-ca0d-5dd0-b94a-383d9b2059d1&t=a59ece2d-3a65-582c-2324-007952d49e0f

self = <msrest.pipeline.requests.RequestsCredentialsPolicy object at 0x7f6f7aa6c6b0>
request = <msrest.pipeline.Request object at 0x7f6f7aa6cd40>
kwargs = {'stream': False}
session = <requests.sessions.Session object at 0x7f6f7aa6c710>

    def send(self, request, **kwargs):
        session = request.context.session
        try:
>           self._creds.signed_session(session)
E           AttributeError: 'CredentialAdaptor' object has no attribute 'signed_session'

env/lib/python3.12/site-packages/msrest/pipeline/requests.py:65: AttributeError

@evelyn-ys
Copy link
Member

azure-mgmt-devtestlabs is still Track 1, causing failure in src/azure-cli/azure/cli/command_modules/vm/tests/latest/test_vm_commands.py::VMAutoShutdownScenarioTest::test_vm_auto_shutdown:

https://dev.azure.com/azclitools/public/_build/results?buildId=219257&view=logs&j=c2322915-ca0d-5dd0-b94a-383d9b2059d1&t=a59ece2d-3a65-582c-2324-007952d49e0f

Will be fixed by #30771

cred, subscription_id, _ = profile.get_login_credentials(
subscription_id=subscription_id,
resource=cli_ctx.cloud.endpoints.active_directory_data_lake_resource_id)
cred, subscription_id, _ = profile.get_login_credentials(subscription_id=subscription_id)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will be covered by #30770.

@@ -130,8 +130,7 @@ def cf_log_analytics_data_plane(cli_ctx, _):
from azure.monitor.query import LogsQueryClient
from azure.cli.core._profile import Profile
profile = Profile(cli_ctx=cli_ctx)
cred, _, _ = profile.get_login_credentials(
resource=cli_ctx.cloud.endpoints.log_analytics_resource_id)
cred, _, _ = profile.get_login_credentials()
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please double check if cli_ctx.cloud.endpoints.log_analytics_resource_id needs to be set as credential_scopes when creating LogsQueryClient.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

azure.monitor.query._logs_query_client.LogsQueryClient uses audience to accept scopes instead of credential_scopes.

When audience is not provided, it falls back to the domain of the URL.

    def __init__(self, credential: TokenCredential, **kwargs: Any) -> None:
        endpoint = kwargs.pop("endpoint", "https://api.loganalytics.io/v1")
        if not endpoint.startswith("https://") and not endpoint.startswith("http://"):
            endpoint = "https://" + endpoint
        parsed_endpoint = urlparse(endpoint)
        audience = kwargs.pop("audience", f"{parsed_endpoint.scheme}://{parsed_endpoint.netloc}")
        self._endpoint = endpoint
        auth_policy = kwargs.pop("authentication_policy", None)
        self._client = MonitorQueryClient(
            credential=credential,
            authentication_policy=auth_policy or get_authentication_policy(credential, audience),

So resource=cli_ctx.cloud.endpoints.log_analytics_resource_id actually never worked.

@jiasli
Copy link
Member Author

jiasli commented Feb 7, 2025

@@ -385,16 +385,13 @@ def logout_all(self):
identity.logout_all_users()
identity.logout_all_service_principal()

def get_login_credentials(self, resource=None, subscription_id=None, aux_subscriptions=None, aux_tenants=None):
def get_login_credentials(self, subscription_id=None, aux_subscriptions=None, aux_tenants=None):
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For Track 1 SDK, the resource of the access token is managed by Azure CLI, not Track 1 SDK. resource is kept as a property of the credential returned by get_login_credentials(). When SDK client calls signed_session() to acquire the access token, CLI provides the access token for resource.

                                returns
get_login_credentials(resource) ------>  credential[resource]
                                             ^
                                             |   calls signed_session()
                                             |
                                  Track 1 SDK client()

For Track 2 SDK, the scopes(resource/.default) of the access token is managed by SDK client instead. The scopes is passed to Track 2 SDK via credential_scopes argument when creating the client instance. The SDK client keeps it and passes it to get_token() when acquiring the access token from the credential.

                            returns
get_login_credentials()  ------------->  credential()
                                             ^
                                             |   calls get_token(scopes)
                                             |
                             Track 2 SDK client(credential_scopes)

Since Track 1 SDK is no longer supported, Azure CLI doesn't need to manage resource anymore, so resource argument is dropped.

@jiasli jiasli changed the title [Core] Drop Track 1 SDK authentication support [Core] Drop Track 1 SDK authentication Feb 10, 2025
logger.debug("CredentialAdaptor.get_token: scopes=%r, kwargs=%r", scopes, kwargs)

# Discard unsupported kwargs: tenant_id, enable_cae
filtered_kwargs = {}
if 'data' in kwargs:
filtered_kwargs['data'] = kwargs['data']

token, _ = self._get_token(scopes, **filtered_kwargs)
return token
return self._credential.get_token(*scopes, **filtered_kwargs)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed that SSLError try catch part is removed. Any reason?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SSLError should be handled by azure.cli.core.util.handle_exception:

# SSLError is raised when making HTTP requests with 'requests' lib behind a proxy that intercepts HTTPS traffic.
# - SSLError is raised when directly calling 'requests' lib, such as MSAL or `az rest`
# - azure.core.exceptions.ServiceRequestError is raised when indirectly calling 'requests' lib with azure.core,
# which wraps the original SSLError
elif isinstance(ex, SSLError) or isinstance(ex, ServiceRequestError) and isinstance(ex.inner_exception, SSLError):
az_error = azclierror.AzureConnectionError(error_msg)
az_error.set_recommendation(SSLERROR_TEMPLATE)

SSLError handling comes from ADAL age: #11093. I can't remember why I added it to src/azure-cli-core/azure/cli/core/adal_authentication.py instead of src/azure-cli-core/azure/cli/core/util.py.

evelyn-ys
evelyn-ys previously approved these changes Feb 12, 2025
@jiasli jiasli merged commit c824dfb into Azure:dev Feb 13, 2025
53 checks passed
@jiasli jiasli deleted the track1-auth branch February 13, 2025 09:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Auto-Assign Auto assign by bot Core CLI core infrastructure
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Drop Track 1 Azure SDK support
6 participants