Add support for token object for ProviderAzure. Fixes #195

[SokolovAnatoliy]

This pull request addresses issue #195. The problem solved is that the token passed to the chat object can expire, and then the chat can no longer be continued.

I propose that the AzureAuth token object be passed instead, which allows for the validate method to determine if the token is expired and the refresh method to refresh the token.

Help is requested on the prop_azure_token() function, to eliminate the dependency on the AzureAuth package.

[hadley]

Since we're only using this in a single place, there's no need to pull out into a separate function. And indeed, since we expect that pretty much all ProviderAzure objects are created by chat_azure(), I'd just do any validation there and not worry about it in the class.

[SokolovAnatoliy]

Agreed. That is a good idea. I added a local version of is_azure_token from the AzureAuth package to utils, and removed the class validation from utils-S7. Now the validation is done in the chat_azure function and is validated as a string in the ProviderAzure class.

I also renamed the argument token to azure_token, because I thought it was confusing with the term "token" being used for LLM tokens. Once the access_token is extracted from the object, I called it access_token to make it clear that it was no longer the azure_token object.

[hadley]

But I think we'd prefer not to take a dependency on AzureAuth, so we might need to do this from first principles.

[atheriel]

@SokolovAnatoliy What form of authentication are you using from AzureAuth? Service principals? Managed identities? Azure CLI authentication?

[SokolovAnatoliy]

@SokolovAnatoliy What form of authentication are you using from AzureAuth? Service principals? Managed identities? Azure CLI authentication?

@atheriel - we mainly use Azure Active Directory authentication via authorization_code (thanks to you) and device_code flows.
These are the first two methods described in the README here:
https://github.com/Azure/AzureAuth

We also use managed identities as well. It depends on the use case.

I am not aware of us using service principals or Azure CLI, but that just may be a limitation of my knowledge based the practices of the users I interact with.

[atheriel]

@SokolovAnatoliy If you're using auth code & device code flows with elmer, is that for local development/testing? Or have you gotten that wired up in a deployed app of some kind, too?

[SokolovAnatoliy]

@atheriel - for elmer we are currently using managed identities in Azure, but we do have device_code working in apps, as well as using the auth code flow from the integrations pane in Connect. So I imagine, that both would just work if that was the auth type for LLMs in Azure.

For us, they all use the exact same structure of creating an R6 object token of the AzureToken class. In fact, I convert the token retrieved from Connect Integrations into the R6 AzureToken, so that there is no change in experience for the users who utilize such authentications in their work.

[atheriel]

@SokolovAnatoliy Thank you, that's very helpful. Great to hear that you're using Connect's OAuth integrations! I'm hoping to get them working with elmer automatically soon, too.

[JamesHWade]

It's exciting to see all the continued progress on the package!

@hadley, are you waiting on any more code changes before merging? I'm eager to use the new token handling for Azure calls. Currently, any app using chat_azure() will stop working once the token expires, so this is a big improvement.

[hadley]

@JamesHWade I'm waiting for @atheriel to say it's ok

[atheriel]

@JamesHWade I don't want to take this PR in its current form, not least because it doesn't actually solve the refresh issue. We're also hesitant to add AzureAuth-related functionality because it's a risky dependency for ellmer to take.

However, I do have an alternative way to add an escape hatch for AzureAuth: by allowing chat_azure() to take a credentials function, like chat_cortex(). This would allow you to use the AzureToken class in a fairly natural way while actually handling refresh correctly, and keeps us out of the business of blessing that dependency. It also leaves the door open to our implementing more complex Azure authentication features ourselves. I'll try to get that PR up soon.

[SokolovAnatoliy]

@atheriel - your solution sounds good, and I'd like to check it out, but I just want to comment that my updated code no longer has a dependency on AzureAuth at all.

I updated it to allow chat_azure to accept an R6 token object instead of a string as the authentication token. There is a basic check that the object is of class AzureToken. That way you can handle the token stuff outside of chat_azure (like you proposed). If the token object is no longer valid, the function will attempt to use the token refresh method. In my testing I was able to create a chat with an expired token and confirm that the token refresh method indeed refreshes the token.

So in the short-medium term, this does not add any dependencies to the ellmer package, but does allow us to use the code with a provided token and keep our chat_azure object from dying every 30 minutes.

[atheriel]

See #248.