Ollama Chat Completion not invoking plugin functions

[eeadie]

Hi,

I have been investigating using Ollama chat completion. I am struggling to get plugin functions to exectue when using Ollama chat completion. Everything works as expected when using AzureOpenAIChatCompletion.

Is invoking of plugin functions supported with OllamaChatCompletion?

Thanks,

Euan

[eeadie]

Example project I am testing with:

SemanticKernelLightBlub.zip

[sophialagerkranspandey]

Thanks for reaching out, we are currently working on this. I'm tagging @RogerBarreto @RogerBarret0 to elaborate.

[RogerBarreto]

@eeadie Currently our Ollama dedicated connector doesn't support function calling.

We have the PR below adding that support for our Ollama Connector, keep posted, potentially available in our next release.

• .Net: MS AI Ollama with Function calling #9488

[frazur]

Hi @RogerBarreto,
I'm trying to use the Ollama chat completion with llama3.2 [edit] with auto function call [/edit] but I see a different behavior than GPT4.
Just to make you understand, let's say my plugin is Weather(city_name, day) and the query is "Tell me what the weather is like in Rome tomorrow"

• with GPT4, the behavior is that the model understands the parameters, understands what the plugin is, invokes it and returns "tomorrow in Rome it will be partly cloudy"
• with Ollama + Llama3.2, the result is "name:Weather-plugin, parameters {city_name:Rome, day:tomorrow}", that is, it recognizes plugin and parameters, but does not perform the invocation.

So it seems like something is missing. Any suggestion?

[tomfanara]

Hi I am having this issue too any release date on this enhancement?

[deepinderdeol]

@RogerBarreto I tried with the latest packages (version 1.30) and function calling did not work for me. I am getting responses using Ollama but functions are not executed.
Is this supported in the current version?

[deepinderdeol]

Update - I was able to get this working with version 1.30

[eeadie]

@deepinderdeol can you spot anything I am doing wrong in the attachment SemanticKernelLightBlub.zip below? I cannot get the function calling to work using Ollama in that example.

[deepinderdeol]

@eeadie you are using phi3. I am not sure this model supports tool calls.
Try using llama3.2 or qwen2 or some other model that supports tool calls.

[eeadie]

@RogerBarreto tested with v1.32 and the plugin is still not getting invoked. Do you know what version the PR above is likely to be included in? Attached project shows an example of the issue. Using AddAzureOpenAIChatCompletion the plugin is invoked, using AddOllamaChatCompletion plugin does not get invoked.
SemanticKernelLightBlub.zip

[bauann]

Hi @eeadie,

I have tested the code you provided and made a few modifications. First of all, the Phi3 model seems not to support tool calls, so I changed the model to Llama3.1 (8b) for testing purposes. I also changed the PromptExecutionSettings from OpenAIPromptExecutionSettings to OllamaPromptExecutionSettings as shown below:

#pragma warning disable SKEXP0070 
OllamaPromptExecutionSettings executionSettings = new()
{
    FunctionChoiceBehavior = FunctionChoiceBehavior.Auto()
};
#pragma warning restore SKEXP0070

I then used the question "How many lights are there?" to chat with the model, and it successfully calls the lights plugin as expected.
These details are provided for your reference, hope they help.

[bauann]

By the way, you can reference here to get the information about the difference between FunctionChoiceBehavior and ToolCallBehavior .

[sapiovesanunivision]

I am not able to make it working too.
I tested the linked project above with Ollama 0.4.6, llama3.1:8b as suggested, with these packages:

  <ItemGroup>
    <PackageReference Include="Microsoft.SemanticKernel" Version="1.33.0" />
    <PackageReference Include="Microsoft.SemanticKernel.Connectors.OpenAI" Version="1.33.0" />
    <PackageReference Include="Microsoft.SemanticKernel.Plugins.OpenApi" Version="1.33.0-preview" />
    <PackageReference Include="Microsoft.SemanticKernel.Connectors.Ollama" Version="1.33.0-alpha" />
    <PackageReference Include="Microsoft.SemanticKernel.Agents.OpenAI" Version="1.33.0-alpha" />
    <PackageReference Include="Microsoft.SemanticKernel.Agents.Core" Version="1.33.0-alpha" />
    <PackageReference Include="Microsoft.SemanticKernel.Agents.Abstractions" Version="1.33.0-alpha" />
  </ItemGroup>

I cannot hit a breakpoint in the plugin.
What am I missing?

[virex-84]

Example ollama function calling: https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Demos/OllamaFunctionCalling/Program.cs

First, you need add system prompt to your model for add function calling ("tools"): for quen2.5 family look this url, for llama 3.1 example prompt for ollama model and etc.

For other types of models (Phi3, etc.) you need to search another prompts to add function calls.

You can add your model with your system prompt (with function calling support) to ollama, see url.

1. download model for example "QVikhr-2.5-1.5B-Instruct-r-Q8_0.gguf" from huggingface.co to "c:\models"
2. create file "Modelfile" (without extension) in"c:\models"
3. insert into file:

from c:\models\QVikhr-2.5-1.5B-Instruct-r-Q8_0.gguf
PARAMETER temperature 0.4
TEMPLATE """
{{- if .Messages }}
{{- if or .System .Tools }}<|im_start|>system
{{- if .System }}
{{ .System }}
{{- end }}
{{- if .Tools }}
...

4. run cmd.exe
5. ollama create MYMODEL -f c:\models\Modelfile
6. ollama serve
7. ollama rm MYMODEL - remove model

If model not include function calling prompt (see "Modelfile"), ollama server print message "registry.ollama.ai MYMODEL does not support tools".

Weak models usually don't support calling functions even if you use this prompt, because they don't think well, see this url

Sorry for my english, translation from google.