Reducing verbosity of function schema (in function calling) by using function advertising

[ankiga-MSFT]

I am using function calling (auto behavior) in Semantic Kernel C# SDK. The plugins that I have written have grown a lot in terms of the number of functions which got added to the plugins. This is increasing the verbosity of function JSON schema and increasing the token usage, thereby also degrading the performance. I might eventually start hitting model token size limits as well.

I had an idea of generating embeddings for the function descriptions and performing Top 'N' retrieval by using vector search based on the incoming query and then supply only those functions for function calling.

I came across this feature. I wanted to understand if using 'function advertising' will actually reduce the function schema to include only the supplied functions? Or is it a filtering layer after all the functions from plugins are serialized in the JSON schema.

[dmytrostruk]

@ankiga-MSFT

I had an idea of generating embeddings for the function descriptions and performing Top 'N' retrieval by using vector search based on the incoming query and then supply only those functions for function calling.

We have an already existing example with this case:
https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Concepts/Optimization/PluginSelectionWithFilters.cs

You could try to update it to include your plugins instead of sample plugins from the example and test how it works for you.

I wanted to understand if using 'function advertising' will actually reduce the function schema to include only the supplied functions?

That's correct. Only those functions you pass in FunctionChoiceBehavior.Auto(functions) will be included in the request to AI model. In the example I shared above you can compare the token usage for both cases when all functions are included and when only specific functions are included.

[ankiga-MSFT]

Thanks @dmytrostruk! This helps.