Function tool calling on OllamaModel returns ModelTextResponse instead of ModelStructuredResponse

[metaboulie]

I'm running this example
https://github.com/pydantic/pydantic-ai/blob/main/pydantic_ai_examples/bank_support.py

when using ollama model like 'ollama:qwen2.5:0.5b' here

pydantic-ai/pydantic_ai_examples/bank_support.py

Line 48 in 84c1190

'openai:gpt-4o',

The response from agent is like below

ModelStructuredResponse(
    calls=[
        ToolCall(
            tool_name="customer_balance",
            args=ArgsJson(args_json='{"include_pending":false}'),
            tool_call_id="call_vz43blys",
        )
    ],
    timestamp=datetime.datetime(2024, 12, 13, 10, 34, 13, tzinfo=datetime.timezone.utc),
    role="model-structured-response",
)
ModelTextResponse(
    content="Your current account balance is $123.45. Thank you for using our bank. Feel free to call us if you have any questions anytime.",
    timestamp=datetime.datetime(2024, 12, 13, 10, 34, 13, tzinfo=datetime.timezone.utc),
    role="model-text-response",
)
ModelTextResponse(
    content="Sure thing! I've fixed it for you. Next time please ask a specific query so we can provide more personalized assistance.",
    timestamp=datetime.datetime(2024, 12, 13, 10, 34, 14, tzinfo=datetime.timezone.utc),
    role="model-text-response",
)

which is a text response instead of structured response

the response from a model like gemini is like

ModelStructuredResponse(
    calls=[
        ToolCall(
            tool_name="customer_balance",
            args=ArgsDict(args_dict={"include_pending": False}),
            tool_call_id=None,
        )
    ],
    timestamp=datetime.datetime(
        2024, 12, 13, 10, 33, 19, 184502, tzinfo=datetime.timezone.utc
    ),
    role="model-structured-response",
)
ModelStructuredResponse(
    calls=[
        ToolCall(
            tool_name="final_result",
            args=ArgsDict(
                args_dict={
                    "risk": 1,
                    "block_card": False,
                    "support_advice": "Your current balance is 123.45. \\n Have a great day!",
                }
            ),
            tool_call_id=None,
        )
    ],
    timestamp=datetime.datetime(
        2024, 12, 13, 10, 33, 21, 322077, tzinfo=datetime.timezone.utc
    ),
    role="model-structured-response",
)
ModelStructuredResponse(
    calls=[
        ToolCall(
            tool_name="customer_balance",
            args=ArgsDict(args_dict={"include_pending": False}),
            tool_call_id=None,
        ),
        ToolCall(
            tool_name="final_result",
            args=ArgsDict(
                args_dict={
                    "block_card": True,
                    "risk": 2,
                    "support_advice": "We have blocked your card. Please contact us to request a new one.",
                }
            ),
            tool_call_id=None,
        ),
    ],
    timestamp=datetime.datetime(
        2024, 12, 13, 10, 33, 22, 498653, tzinfo=datetime.timezone.utc
    ),
    role="model-structured-response",
)

and they are all structured responses

[samuelcolvin]

Looks like the same underlying issue as #200 - it's a limitation of the model, not much we can do AFAIK.

[saschacontes]

Would be interesting to see if requests to Ollama are adjusted to use the format parameter for structured output as described in https://github.com/ollama/ollama/releases/tag/v0.5.0

[samuelcolvin]

Yup, see #242 for a discussion of that.

You can downvote my response if you like, but it is the same underlying issue at #200.

More generally - we're maintaining a library that supports multiple models, I can't investigate every "this models responds wrongly to this input" issue - if people come with actionable suggestions like using structured output instead of tools, we'll happily try to fix it.

[saschacontes]

Your argument is very understandable and your contribution is awesome. Thank you.