Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add disable-instance-discovery option in interactive pop mode #593

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

Aijing2333
Copy link

@Aijing2333 Aijing2333 commented Jan 13, 2025

For Winfield, we need to disable instance discovery in the auth flow, or we will hit error. See details in #555

Context
Project Winfield is an appliance Virtual Machine (VM) that provides a local control plane for your datacenter. It allows you to deploy and manage the appliance easily and use Azure Arc and Azure management capabilities in a disconnected or air-gapped environment. Project Winfield is designed to help you meet specific regulatory requirements when you need a local control plane.

@Aijing2333 Aijing2333 changed the title Disable instance discovery on Winfield Disable instance discovery on Winfield in interactive pop mode Jan 13, 2025
@Aijing2333 Aijing2333 changed the title Disable instance discovery on Winfield in interactive pop mode Disable instance discovery(interactive pop mode) on Winfield Jan 13, 2025
@@ -18,10 +18,11 @@ func AcquirePoPTokenInteractive(
authority,
clientID string,
options *azcore.ClientOptions,
isWinfieldEnv bool,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This parameter is too specific to winfield here, this should be more generic such as WithInstanceDiscovery for example.

I also think that we should start passing optional option or struct here.

@@ -34,14 +35,14 @@ func NewTokenProvider(o *Options) (TokenProvider, error) {
case DeviceCodeLogin:
return newDeviceCodeTokenProvider(*oAuthConfig, o.ClientID, o.ServerID, o.TenantID)
case InteractiveLogin:
return newInteractiveTokenProvider(*oAuthConfig, o.ClientID, o.ServerID, o.TenantID, popClaimsMap)
return newInteractiveTokenProvider(*oAuthConfig, o.ClientID, o.ServerID, o.TenantID, popClaimsMap, isWinfieldEnvironment(o.Environment))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

overall I would like to avoid naming winfield here, and find a broader approach.

@@ -87,3 +88,10 @@ func getAzureEnvironment(environment string) (azure.Environment, error) {
}
return azure.EnvironmentFromName(environment)
}

func isPrivateEnvironment(environment string) bool {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isAirGappedCloud is likely better naming

@@ -38,6 +38,7 @@ type Options struct {

const (
defaultEnvironmentName = "AzurePublicCloud"
privateEnvironmentName = "AzureStackCloud"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

who sets the value AzureStackCloud? where is this being consumed?

Copy link
Author

@Aijing2333 Aijing2333 Jan 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When using cli cmd az aksarc get-credentials, the AKSArc operator will return a kubeconfig file, the environment name is in this file.
This is local access(can access API server) for AKSArc cluster:

az aksarc get-credentials --name "aksarc-test1" --resource-group $resource_group --file C:\Aksarc\config-aksarc-test1-user
kubectl --kubeconfig C:\Aksarc\config-aksarc-test1-user get ns

kubelogin info:

users:

name: aad-user
user:
exec:
apiVersion: client.authentication.k8s.io/v1beta1
args:

  • get-token
  • --login
  • interactive
  • --server-id
  • xxx
  • --client-id
  • xxx
  • --tenant-id
  • xxx
  • --environment
  • AzureStackCloud
  • --pop-enabled
  • --pop-claims
  • xxxxxxxx
    command: kubelogin
    env: null

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose AKS should also set this argument in AGC environment?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, I don't think AzureStackCloud is appropriate for Airgapped environment.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This name is hardcode in https://github.com/Azure/go-autorest/blob/main/autorest/azure/environments.go#L293 to load env from file(retrieve ActiveDirectoryEndpoint), so we set this name.
For AKS in AGC environment, not very familiar with it, not sure if it needs to load own ActiveDirectoryEndpoint and disable discovery instance

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thinking back, AirGappedCloud is a bit overloaded. The proposed change here is specifically for the environment Azure Local Disconnected (https://learn.microsoft.com/en-us/azure/azure-local/manage/disconnected-operations-overview)

It is also referred internally as Arc Autonomous. This disconnected environment has its own simple Identity Provider(not AAD) which does not have instance metadata discovery endpoint.

@Aijing2333 Is it passible to parameterize DisableInstanceDiscovery to kubelogin.exe ? who will be setting that option?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for now, exposing this parameter directly in cli and api and let the end user decide is most appropriate, rather than deriving it from the environment

Copy link
Author

@Aijing2333 Aijing2333 Jan 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @weinong @bganapa for your suggestion, I will make the change to expose the DisableInstanceDiscovery option.
The name AzureStackCloud is still needed as it is the only way to load env from file https://github.com/Azure/go-autorest/blob/main/autorest/azure/environments.go#L293, is this fine?

Copy link
Member

@bganapa bganapa Jan 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to refer AzureStackCloud within Kubelogin codebase or outside of it ? I know we have this reference of AzureStackCloud in many codebases, CLI, Custom Location RP etc.. If we cant avoid it, I think we will have to make that case and take it forward..

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Never mind, I suddenly realize after I expose the DisableInstanceDiscovery option, this code change removed. Kubelogin already contains the logic to the load env file when env name is azurestackcloud

type PublicClientOptions struct {
Authority string
ClientID string
DisableInstanceDiscovery bool
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dont we also need TenantId here? wondering as I introduced MsalClientOptions in my other PR and the possibility of reusing it here..

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Aijing2333 Aijing2333 changed the title Disable instance discovery(interactive pop mode) on Winfield Add disable-instance-discovery option in interactive pop mode Jan 23, 2025
@@ -20,6 +20,7 @@ ZURE_CLIENT_CERTIFICATE_PASSWORD environment variable
--client-id string AAD client application ID. It may be specified in AAD_SERVICE_PRINCIPAL_CLIENT_ID or AZURE_CLIENT_ID environment variable
--client-secret string AAD client application secret. Used in spn login. It may be specified in AAD_SERVICE_PRINCIPAL_CLIENT_SECRET or AZURE_CLIENT_S
ECRET environment variable
--disable-instance-discovery set to true to disable instance discovery in environments with their own simple Identity Provider (not AAD) that do not have instance metadata discovery endpoint.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what do you mean by simple?

@@ -20,6 +20,7 @@ ZURE_CLIENT_CERTIFICATE_PASSWORD environment variable
--client-id string AAD client application ID. It may be specified in AAD_SERVICE_PRINCIPAL_CLIENT_ID or AZURE_CLIENT_ID environment variable
--client-secret string AAD client application secret. Used in spn login. It may be specified in AAD_SERVICE_PRINCIPAL_CLIENT_SECRET or AZURE_CLIENT_S
ECRET environment variable
--disable-instance-discovery set to true to disable instance discovery in environments with their own simple Identity Provider (not AAD) that do not have instance metadata discovery endpoint.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
--disable-instance-discovery set to true to disable instance discovery in environments with their own simple Identity Provider (not AAD) that do not have instance metadata discovery endpoint.
--disable-instance-discovery set to true to disable instance discovery in environments with their own Identity Provider (not Entra ID/AAD) that does not have instance metadata discovery endpoint.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants