Add support for Snowflake's "Cortex inference" APIs, backed by Claude #255
Comments
atheriel
changed the title
|
Renaming proposals sound good to me. |
atheriel
added a commit
to atheriel/elmer
that referenced
this issue
Jan 17, 2025
This commit adds `chat_snowflake()` for chatting with models hosted
through Snowflake's [Cortex LLM REST API][0]:
chat <- chat_snowflake()
chat$chat("Tell me a joke in the form of a SQL query.")
On the backend it looks fairly similar to OpenAI, though it has only the
basic textual functionality, and so many advanced ellmer features are
not available. I also reused quite a bit of the credential support and
utilities from `chat_cortex()`, so this commit also includes some minor
refactoring of that provider.
Right now the default model for `chat_snowflake()` is Llama 3.1 70B, but
we should change it to Claude 3.5 Sonnet when that gets rolled out more
widely; it's only available to customers in the `us-west-1` Snowflake
region right now.
Unit tests are included.
Part of tidyverse#255.
[0]: https://docs.snowflake.com/en/user-guide/snowflake-cortex/cortex-llm-rest-api
Signed-off-by: Aaron Jacobs <[email protected]>
atheriel
added a commit
to atheriel/elmer
that referenced
this issue
Jan 21, 2025
This commit adds `chat_snowflake()` for chatting with models hosted
through Snowflake's [Cortex LLM REST API][0]:
chat <- chat_snowflake()
chat$chat("Tell me a joke in the form of a SQL query.")
On the backend it looks fairly similar to OpenAI, though it has only the
basic textual functionality, and so many advanced ellmer features are
not available. I also reused quite a bit of the credential support and
utilities from `chat_cortex()`, so this commit also includes some minor
refactoring of that provider.
Right now the default model for `chat_snowflake()` is Llama 3.1 70B, but
we should change it to Claude 3.5 Sonnet when that gets rolled out more
widely; it's only available to customers in the `us-west-1` Snowflake
region right now.
Unit tests are included.
Part of tidyverse#255.
[0]: https://docs.snowflake.com/en/user-guide/snowflake-cortex/cortex-llm-rest-api
Signed-off-by: Aaron Jacobs <[email protected]>
atheriel
added a commit
to atheriel/elmer
that referenced
this issue
Jan 22, 2025
This commit adds `chat_snowflake()` for chatting with models hosted
through Snowflake's [Cortex LLM REST API][0]:
chat <- chat_snowflake()
chat$chat("Tell me a joke in the form of a SQL query.")
On the backend it looks fairly similar to OpenAI, though it has only the
basic textual functionality, and so many advanced ellmer features are
not available. I also reused quite a bit of the credential support and
utilities from `chat_cortex()`, so this commit also includes some minor
refactoring of that provider.
Right now the default model for `chat_snowflake()` is Llama 3.1 70B, but
we should change it to Claude 3.5 Sonnet when that gets rolled out more
widely; it's only available to customers in the `us-west-1` Snowflake
region right now.
Unit tests are included.
Part of tidyverse#255.
[0]: https://docs.snowflake.com/en/user-guide/snowflake-cortex/cortex-llm-rest-api
Signed-off-by: Aaron Jacobs <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Snowflake just announced that Claude 3.5 Sonnet is now generally available through Cortex AI, which is a significant upgrade over any of their existing models. It looks like a lot of their documentation hasn't been updated yet, either, but the starting points would be:
We should probably add support for these new "inference" APIs, because I think some folks would be quite excited to route to a high-quality model through Snowflake, much like they're doing today through Azure, Bedrock, or Databricks. These new APIs look to function much more like the other OpenAI-like APIs that
ellmersupports, too, as opposed to the "Cortex Analyst" APIs that I implemented forchat_cortex().Maybe we should introduce a new
chat_snowflake()function for this API and drop the confusing "Cortex" branding ofchat_cortex()-- or rename it tochat_cortex_analyst(), since Snowflake is apparently going to majorly upgrade that API using Claude as well.The text was updated successfully, but these errors were encountered: