Skip to content

Provide default image reponse usage values, if they are not returned by the API #618

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

svendecabooter
Copy link

Provide default image reponse usage values, if they are not returned by the API (e.g. for LiteLLM)

What:

  • Bug Fix
  • New Feature

Description:

We are using LiteLLM as an AI provider. Supposedly this software would provide the same API as OpenAI.
However, we are encountering issues with image response usage statistics.
The PHP client expects various token usage stats to be returned (such as total_tokens, input_tokens, ...).
In our default setup of LiteLLM we only retrieve ["total_tokens" => 0], not the other expected data.
Therefore, it would probably be best not to rely on these values always being there, and providing default values in case they are not, instead of crashing.

Other responses seem to work fine for us for now, but we haven't tested all types yet...

@iBotPeaches
Copy link
Collaborator

You'll find the problem with just adding null checks is the types don't suggest they are null, so it becomes a bit more of a fix to make things nullable as shown by test failures. In this case falling back to a 0 seems misleading, but making them nullable makes downstream users adapt for a nullable case - so a tough call.

In this case is a 0 or maybe -1 better?

@svendecabooter
Copy link
Author

Hmm I'm not familiar enough with this library or the OpenAI API to know what the best approach would be.
In case we set the default value to -1, wouldn't the tests also fail then?

Or should we make sure there are always (default) values in the $attributes array earlier on in the code flow?

@iBotPeaches
Copy link
Collaborator

I'm torn between nullable or -1 at this point. Nullable because its semantically better, but that causes all downstream users to type for something nullable that generally won't be for OpenAI.

-1 because it keeps the typing and basically says "I don't know"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants