[Bug]: tools not passed to databricks

[lewiesnyder]

What happened?

I am evaluating various frameworks for use in a large enterprise.

While testing out the new Databricks provider in the python SDK I found that tools are not supported:

resp = await litellm.acompletion(
    model="databricks/databricks-dbrx-instruct",
    api_base=getenv("DATABRICKS_HOST"),
    api_key=getenv("DATABRICKS_TOKEN"),
    messages=[
               {
                   "role": "system",
                   "content": "You are a helpful customer support assistant. Use the supplied tools to assist the user."
               },
               {
                   "role": "user",
                   "content": "Hi, can you tell me the delivery date for my order?"
               },
               {
                   "role": "assistant",
                   "content": "Hi there! I can help with that. Can you please provide your order ID?"
               },
               {
                   "role": "user",
                   "content": "i think it is order_12345, also what is the weather in Phoenix, AZ?"
               }
           ],
    tools=[{
           "type": "function",
           "function": {
               "name": "get_delivery_date",
               "description": "Get the delivery date for a customer'\''s order. Call this whenever you need to know the delivery date, for example when a customer asks '\''Where is my package'\''",
               "parameters": {
                   "type": "object",
                   "properties": {
                       "order_id": {
                           "type": "string",
                           "description": "The customer'\''s order ID."
                       }
                   },
                   "required": [
                       "order_id"
                   ],
                   "additionalProperties": False
               }
           }
       },
       {
           "type": "function",
           "function": {
               "name": "check_weather",
               "description": "Check the current weather in a location. For example when asked: '\''What is the temperature in San Fransisco, CA?'\''",
               "parameters": {
                   "type": "object",
                   "properties": {
                       "city": {
                           "type": "string",
                           "description": "The city to check the weather for."
                       },
                       "state": {
                           "type": "string",
                           "description": "The state to check the weather for."
                       }
                   },
                   "required": [
                       "city", "state"
                   ],
                   "additionalProperties": False
               }
           }
       }
   ],
    drop_params=True,
)

print(resp.json()['choices'])

Produces:

 'choices': [{'finish_reason': 'stop',
   'index': 0,
   'message': {'content': "I've checked the order_12345, and it's estimated to be delivered by the end of this week. Regarding the weather in Phoenix, AZ, it's currently 75°F (24°C) with partly cloudy skies. The high today is expected to be around 82°F (28°C), and the low tonight will be around 66°F (19°C). I hope this helps!\n\nPlease let me know if you have any other questions or concerns. I'm here to assist you!",
    'role': 'assistant',
    'tool_calls': None,
    'function_call': None}}],

I understand that the Databricks provider is a new feature, is there a plan to support this in the near future?

Relevant log output

Request to litellm:
litellm.acompletion(model='databricks/databricks-dbrx-instruct', api_base='REDACTED', api_key='REDACTED', messages=[{'role': 'system', 'content': 'You are a helpful customer support assistant. Use the supplied tools to assist the user.'}, {'role': 'user', 'content': 'Hi, can you tell me the delivery date for my order?'}, {'role': 'assistant', 'content': 'Hi there! I can help with that. Can you please provide your order ID?'}, {'role': 'user', 'content': 'i think it is order_12345, also what is the weather in Phoenix, AZ?'}], functions=[{'type': 'function', 'function': {'name': 'get_delivery_date', 'description': "Get the delivery date for a customer'''s order. Call this whenever you need to know the delivery date, for example when a customer asks '''Where is my package'''", 'parameters': {'type': 'object', 'properties': {'order_id': {'type': 'string', 'description': "The customer'''s order ID."}}, 'required': ['order_id'], 'additionalProperties': False}}}, {'type': 'function', 'function': {'name': 'check_weather', 'description': "Check the current weather in a location. For example when asked: '''What is the temperature in San Fransisco, CA?'''", 'parameters': {'type': 'object', 'properties': {'city': {'type': 'string', 'description': 'The city to check the weather for.'}, 'state': {'type': 'string', 'description': 'The state to check the weather for.'}}, 'required': ['city', 'state'], 'additionalProperties': False}}}], drop_params=True)

POST Request Sent from LiteLLM:
curl -X POST \
REDACTED \
-H 'Authorization: *****' -H 'Content-Type: *****' \
-d '{'model': 'databricks-dbrx-instruct', 'messages': [{'role': 'system', 'content': 'You are a helpful customer support assistant. Use the supplied tools to assist the user.'}, {'role': 'user', 'content': 'Hi, can you tell me the delivery date for my order?'}, {'role': 'assistant', 'content': 'Hi there! I can help with that. Can you please provide your order ID?'}, {'role': 'user', 'content': 'i think it is order_12345, also what is the weather in Phoenix, AZ?'}], 'stream': False}'

RAW RESPONSE:
{"id": "chatcmpl_8a4bc44f-3c4e-624a-5f62-a136b4f40aac", "object": "chat.completion", "created": 1736969228, "model": "dbrx-instruct-071224", "choices": [{"index": 0, "message": {"role": "assistant", "content": "Thank you for providing the order ID. I'm checking the order status for order_12345 now.\n\nRegarding your question about the weather in Phoenix, AZ, I'm unable to provide real-time weather information as I don't have the capability to access live data. I recommend checking a reliable weather forecasting website or app for the most accurate and up-to-date information."}, "finish_reason": "stop", "logprobs": null}], "usage": {"prompt_tokens": 89, "completion_tokens": 74, "total_tokens": 163}}

Are you a ML Ops Team?

Yes

What LiteLLM version are you on ?

v1.57.3

Twitter / LinkedIn details

https://www.linkedin.com/in/lewie-snyder/

[krrishdholakia]

missed this. thanks for the ticket @lewiesnyder i'll work on this today

[krrishdholakia]

Investigated further.

This was not a bug. The call worked as expected.

I see you have

drop_params=True,

and based on databricks's docs i didn't see they had support for tool calling.

so the param was being dropped in the request.

Checking to see if they support tool calling now

---

Here's our mapping of dbrx openai params -

litellm/litellm/llms/databricks/chat/transformation.py

Line 66 in 46b44f3

def get_supported_openai_params(self, model: Optional[str] = None) -> list:

[krrishdholakia]

Update: i see the params are on databricks - adding on #8076

[lewiesnyder]

Awesome! Thanks very much. I can confirm that the change was successful.

[krrishdholakia]

Thanks @lewiesnyder

Can we do a 10min call sometime this/next week? Would love to learn how you're using litellm, so we can improve for your use-case.

Attaching my calendly for convenience - https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat