How can you use kubelogin with multiple azure tenancies?

[pearj]

We have our dev and prod AKS clusters in different azure tenancies, and I can't figure out a way that works with kubelogin. Before kubelogin I just needed to do a device code login for each cluster. But now only one tenancy works at a time.

When I try to use kubelogin convert-kubeconfig -l azurecli with multiple tenancies only one tenancy works and the rest complain with error: You must be logged in to the server (Unauthorized). Is this simply an unsupported scenario?

I tried kubelogin convert-kubeconfig -l devicecode but then I got Error: --client-id is required

So at the moment I have had to revert back to the built-in behaviour of az aks get-credentials and ignoring those warnings about how it won't work in v1.25+.

How are other people handling multiple tenancies?

[weinong]

@pearj can you share the repro for Error: --client-id is required please?

[pearj]

The repro is:

1. Run az aks get-credentials for all your aks clusters
2. Run kubelogin convert-kubeconfig -l devicecode, which works
3. Run kubelogin convert-kubeconfig -l azurecli which also works
4. Run kubelogin convert-kubeconfig -l devicecode, which fails, I think because the azurecli version deletes data.

The solution seems to be logging into the clusters again.

I do like the idea of using azure cli option though, is there a way to make it work if you've signed into multiple tenancies with the azure cli? I think get-token would need to include the tenant id or something like that?

[weinong]

@pearj the conversion is based on what's provided in kubeconfig, and conversion from devicecode to azurecli will lose client-id as azure cli uses its own client id. when azurecli is used, get-token does nothing but invoking azure cli which doesn't support multi tenants. your best bet is do az login --tenant $tenantID

[weinong]

#122 should help. Can you give it a try?

[pearj]

Ooh thanks! I'll try that tomorrow at work

[pearj]

It worked perfectly! Thanks for doing that for me.

[pearj]

It turns out it's a little more complicated. When a token expires it can't fetch a new one unless the current tenant is set with az account set --subscription. I guess I'll need to have a bash script that switches the azure cli primary subscription when I change the kubernetes context.
But I think it's still workable. I don't think it's something that kubelogin can fix.

[weinong]

oops, the introduction of tenantID to azurecli broke the azurecli using msi login. #123