Use models such as Paligemma that are not instruction fine-tuned

[AayGpt]

Hi,

I want to use models such as Paligemma-3b that have not been explicitly instruction finetuned and hence do not support the OpenAI Chat completions API.

Is there a way to run such models with baml? Thanks!

Some related open issues on other repos which might be useful:-

1. [Bug]: PaliGemma2 not working with OpenAI Docker serve vllm-project/vllm#12052
2. [Bug]: PaliGemma serving vllm-project/vllm#6644

[linear]

BAML-230 Use models such as Paligemma that are not instruction fine-tuned

[sxlijin]

If they don't support the chat API or completions API, we'd have to add a new client integration type for it. Do you have any pointers to documentation for how to communicate with these models?

[AayGpt]

Hi. Would something like this help:- https://huggingface.co/google/paligemma2-3b-pt-224