diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 06a2309c412..5a413078014 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -259,6 +259,11 @@ /sdk/python/featurestore_sample/notebooks/sdk_only/3.\ Enable\ recurrent\ materialization\ and\ run\ batch\ inference.ipynb @sdgilley @msakande @Blackmist @ssalgadodev @lgayhardt @fbsolo-ms1 /sdk/python/featurestore_sample/notebooks/sdk_only/4.\ Enable\ online\ store\ and\ run\ online\ inference.ipynb @sdgilley @msakande @Blackmist @ssalgadodev @lgayhardt @fbsolo-ms1 /sdk/python/featurestore_sample/notebooks/sdk_only/5.\ Develop\ a\ feature\ set\ with\ custom\ source.ipynb @sdgilley @msakande @Blackmist @ssalgadodev @lgayhardt @fbsolo-ms1 +/sdk/python/foundation-models/cohere/cohere-aisearch-rag.ipynb @stewart-co @kseniia-cohere +/sdk/python/foundation-models/cohere/cohere_faiss_langchain_embed.ipynb @stewart-co @kseniia-cohere +/sdk/python/foundation-models/cohere/cohere-aisearch-langchain-rag.ipynb @stewart-co @kseniia-cohere +sdk/python/foundation-models/cohere/command_faiss_langchain.ipynb @stewart-co @kseniia-cohere +sdk/python/foundation-models/cohere/command_tools-langchain.ipynb @stewart-co @kseniia-cohere /sdk/python/featurestore_sample/notebooks/sdk_only/7.\ Develop\ a\ feature\ set\ using\ Domain\ Specific\ Language\ DSL @sdgilley @msakande @Blackmist @ssalgadodev @lgayhardt @fbsolo-ms1 /sdk/python/jobs/automl-standalone-jobs/automl-classification-task-bankmarketing/automl-classification-task-bankmarketing-serverless.ipynb @sdgilley @msakande @Blackmist @ssalgadodev @lgayhardt @fbsolo-ms1 /sdk/python/jobs/automl-standalone-jobs/automl-image-object-detection-task-fridge-items/automl-image-object-detection-task-fridge-items.ipynb @sdgilley @msakande @Blackmist @ssalgadodev @lgayhardt @fbsolo-ms1 diff --git a/sdk/python/foundation-models/cohere/README.md b/sdk/python/foundation-models/cohere/README.md new file mode 100644 index 00000000000..8426ff3e162 --- /dev/null +++ b/sdk/python/foundation-models/cohere/README.md @@ -0,0 +1,22 @@ +## Cohere model samples + +### Basic samples + +Sample | Description | SDKs +--|--|-- +[webrequests.ipynb](./webrequests.ipynb)|Use Command R family of LLMs with the native Python packages.|`urllib.request`, `json` +[cohere-cmdR.ipynb](./cohere-cmdR.ipynb)|Use Command R family of LLMs with the Cohere python SDK.|`cohere` +[cohere-embed.ipynb](./cohere-embed.ipynb)| Use Embedding models with the Cohere python SDK.|`cohere` +[langchain.ipynb](./langchain.ipynb)|Use Command R family of LLMs with the LangChain SDK and Cohere's LangChain integration.|`langchain`, `langchain_cohere` +[litellm.ipynb](./litellm.ipynb)|Use Command R family of LLMs with the LiteLLM SDK |`litellm` +[openaisdk.ipynb](./openaisdk.ipynb)|Use Command R family of LLMs with the Open AI SDK. Note that this is exprimental and can break if there is breaking change with the OpenAI APIs. |`openai` + + +### Retrieval Augmented Generation (RAG) and Tool-Use samples +Sample | Description | SDKs +--|--|-- +[cohere_faiss_langchain_embed.ipynb](./cohere_faiss_langchain_embed.ipynb)|Create a local (FAISS) vector index using Cohere embeddings - Langchain|`langchain`, `langchain_cohere` +[command_faiss_langchain.ipynb](./command_faiss_langchain.ipynb)|Use Cohere Command R/R+ to answer questions from data in local (FAISS) vector index - Langchain|`langchain`, `langchain_cohere` +[cohere-aisearch-langchain-rag.ipynb](./cohere-aisearch-langchain-rag.ipynb)|Use Cohere Command R/R+ to answer questions from data in AI search vector index - Langchain|`langchain`, `langchain_cohere` +[cohere-aisearch-rag.ipynb](./cohere-aisearch-rag.ipynb)|Use Cohere Command R/R+ to answer questions from data in AI search vector index - Cohere SDK| `cohere`, `azure_search_documents` +[command_tools-langchain.ipynb](./command_tools-langchain.ipynb)|Command R+ tool/function calling using LangChain|`cohere`, `langchain`, `langchain_cohere` diff --git a/sdk/python/foundation-models/cohere/cohere-aisearch-langchain-rag.ipynb b/sdk/python/foundation-models/cohere/cohere-aisearch-langchain-rag.ipynb new file mode 100644 index 00000000000..2cf98881325 --- /dev/null +++ b/sdk/python/foundation-models/cohere/cohere-aisearch-langchain-rag.ipynb @@ -0,0 +1,376 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Use Cohere Command R or Cohere Command R+ with LangChain and Azure AI Search to Answer Questions Using Your Data\n", + "\n", + "You can use Cohere Command R or Cohere Command R+ models deployed in Azure AI and Azure ML with `langchain` to create advanced retrieval augmented generation (RAG) pipelines.\n", + "\n", + "> Review the [documentation](https://learn.microsoft.com/en-us/azure/ai-studio/how-to/deploy-models-cohere-command) for the Cohere family of models at for AI Studio and for ML Studio for details on how to provision inference endpoints, regional availability, pricing and inference schema reference." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Prerequisites\n", + "\n", + "Before we start, there are certain steps we need to take to deploy the models:\n", + "\n", + "* Register for a valid Azure account with subscription \n", + "* Make sure you have access to [Azure AI Studio](https://learn.microsoft.com/en-us/azure/ai-studio/what-is-ai-studio?tabs=home)\n", + "* Create a project and resource group\n", + "* Select `Cohere Command R` or `Cohere Command R+`.\n", + "\n", + " > Notice that some models may not be available in all the regions in Azure AI and Azure Machine Learning. On those cases, you can create a workspace or project in the region where the models are available and then consume it with a connection from a different one. To learn more about using connections see [Consume models with connections](https://learn.microsoft.com/en-us/azure/ai-studio/concepts/connections)\n", + "\n", + "* Deploy with \"Pay-as-you-go\"\n", + "* Follow the same steps for `Cohere-embed-v3-english`\n", + "\n", + "Once deployed successfully, you should be assigned for an API endpoint and a security key for inference.\n", + "\n", + "For more information, you should consult Azure's official documentation [here](https://learn.microsoft.com/en-us/azure/ai-studio/how-to/deploy-models-cohere-command) for model deployment and inference.\n", + "\n", + "### Services\n", + "\n", + "You will need to ensure the following services have been created in your Azure environment:\n", + "* Ensure you have created a search service. This can be done in the `Azure Portal` and more instructions can be found here: https://learn.microsoft.com/en-us/azure/search/search-what-is-azure-search **NOTE: You do not need to create the index, this will be done below**\n", + "\n", + "* Create a Cohere `Embed` and `Command` endpoint in the `Azure AI Studio`. Instructions can be found here: https://learn.microsoft.com/en-us/azure/ai-studio/how-to/deploy-models-cohere-command " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Example\n", + "\n", + "The following example demonstrate how to create a RAG workflow that uses a `Cohere Command R` or `Cohere Command R+` model deployed in Azure AI and Azure ML. We will also leverage Azure AI Search to store our documents along with LangChain to orchestrate the process" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Install Dependencies" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "! pip install --quiet langchain langchain-cohere azure-search-documents azure-identity" + ] + }, + { + "cell_type": "code", + "execution_count": 139, + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "from azure.core.credentials import AzureKeyCredential\n", + "from langchain_community.vectorstores.azuresearch import AzureSearch\n", + "from langchain_cohere import ChatCohere\n", + "from langchain_cohere import CohereEmbeddings\n", + "from langchain_core.runnables import RunnablePassthrough, RunnableLambda\n", + "from langchain.schema import HumanMessage" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "## Constants\n", + "We will set the values for the keys and models that we will use in our RAG pipeline" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's create instances of our Chat and Embed models deployed in Azure AI or Azure ML. Use the `langchain_cohere` package and configure it as follows:\n", + "\n", + "- `embed_endpoint` and `command_endpoint`: Use the endpoint URL from your deployment. Include `/v1` at the end of the endpoint URL." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# set constants\n", + "search_service_endpoint = \"https://.search.windows.net\"\n", + "key_credential = AzureKeyCredential(\"AZURE_SEARCH_ADMIN_KEY\")\n", + "index_name = \"AZURE_SEARCH_INDEX\" # this index does not have to be created yet\n", + "\n", + "# Embed\n", + "azure_cohere_embed_endpoint = \"https://..inference.ai.azure.com/v1\"\n", + "azure_cohere_embed_key = \"\"\n", + "\n", + "# Command\n", + "azure_cohere_command_endpoint = \"https://..inference.ai.azure.com/v1\"\n", + "azure_cohere_command_key = \"\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### AI Search Index\n", + "\n", + "We need to set up the index in our `Azure AI Search`\n", + "\n", + "We will use the `LangChain` `AzureSearch` package as well as the `CohereEmbeddings` package, which will serve as the embedding model for our index" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Set up the embedding model to be used in the vector index\n", + "embed_model = CohereEmbeddings(\n", + " base_url=azure_cohere_embed_endpoint,\n", + " cohere_api_key=azure_cohere_embed_key,\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# create the vector store using AzureSearch\n", + "vector_store = AzureSearch(\n", + " azure_search_endpoint=search_service_endpoint,\n", + " azure_search_key=key_credential,\n", + " index_name=index_name,\n", + " embedding_function=embed_model,\n", + " semantic_configuration_name=\"default\",\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Lets create some sample data to add to our index" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "docs_to_index = [\n", + " {\n", + " \"hotelId\": \"1\",\n", + " \"hotelName\": \"Fancy Stay\",\n", + " \"description\": \"Best hotel in town if you like luxury hotels.\",\n", + " \"category\": \"Luxury\",\n", + " },\n", + " {\n", + " \"hotelId\": \"2\",\n", + " \"hotelName\": \"Roach Motel\",\n", + " \"description\": \"Cheapest hotel in town. Infact, a motel.\",\n", + " \"category\": \"Budget\",\n", + " },\n", + " {\n", + " \"hotelId\": \"3\",\n", + " \"hotelName\": \"EconoStay\",\n", + " \"description\": \"Very popular hotel in town.\",\n", + " \"category\": \"Budget\",\n", + " },\n", + " {\n", + " \"hotelId\": \"4\",\n", + " \"hotelName\": \"Modern Stay\",\n", + " \"description\": \"Modern architecture, very polite staff and very clean. Also very affordable.\",\n", + " \"category\": \"Luxury\",\n", + " },\n", + " {\n", + " \"hotelId\": \"5\",\n", + " \"hotelName\": \"Secret Point\",\n", + " \"description\": \"One of the best hotel in town. The hotel is ideally located on the main commercial artery of the city in the heart of New York.\",\n", + " \"category\": \"Boutique\",\n", + " },\n", + "]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Finally, we will add the texts into our vector index. This will automatically embed the field we choose (description in this case) and attach the associated metadata" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "vector_store.add_texts(\n", + " texts=[d[\"description\"] for d in docs_to_index], metadatas=docs_to_index\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We will create a langchain retriever with our newly-populated Azure AI vector store" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "retriever = vector_store.as_retriever()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "retriever.get_relevant_documents(\"Best luxury hotel in town\", top_k=3)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Create our Chat function\n", + "Next, we will create a basic chat function using the `ChatCohere` class\n", + "\n", + "Cohere's `chat endpoint` can accept documents directly and will return a grounded answer that includes citations against those documents.\n", + "\n", + "No prompt is needed since the `Cohere` model will automatically use a RAG prompt when documents are passed in.\n", + "\n", + "Because we are using `LangChain's expression language (LCEL)`, we will also wrap our function at the end with a `RunableLambda` function. Learn more about LCEL here: https://python.langchain.com/docs/expression_language/get_started/" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "def ask(inputs):\n", + " \"\"\"\n", + " Ask a question to the chatbot, expecting a dictionary with 'question' and 'documents'.\n", + "\n", + " Args:\n", + " inputs (dict): A dictionary containing 'question' and 'documents'.\n", + "\n", + " Returns:\n", + " str: The response from the chatbot.\n", + " \"\"\"\n", + " question = inputs[\"question\"]\n", + " documents = inputs[\"documents\"]\n", + "\n", + " documents = [d.metadata for d in documents]\n", + "\n", + " chatbot = ChatCohere(\n", + " base_url=azure_cohere_command_endpoint,\n", + " cohere_api_key=azure_cohere_command_key,\n", + " )\n", + "\n", + " response = chatbot(messages=[HumanMessage(content=question)], documents=documents)\n", + " return response\n", + "\n", + "\n", + "ask = RunnableLambda(ask)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Create the Chain\n", + "We can now create the chain by chaining the retriever and the chat function together" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# create the answer chain using our search function and chat function\n", + "answer_chain = {\"documents\": retriever, \"question\": RunnablePassthrough()} | ask" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Run our chain\n", + "You can now run the full chain and see the response as an `AIMessage` object. This will contain the models answer along with citations from the documents retrieved from `Azure AI Search`" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "answer_chain.invoke(\"luxury hotel\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Aditional resources\n", + "\n", + "Here are some additional reference:\n", + "\n", + "* [Plan and manage costs (marketplace)](https://learn.microsoft.com/azure/ai-studio/how-to/costs-plan-manage#monitor-costs-for-models-offered-through-the-azure-marketplace)\n", + " \n", + "* [Cohere examples with LangChain](https://docs.cohere.com/docs/cohere-and-langchain) " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "base", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.13" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/sdk/python/foundation-models/cohere/cohere-aisearch-rag.ipynb b/sdk/python/foundation-models/cohere/cohere-aisearch-rag.ipynb new file mode 100644 index 00000000000..56974da0137 --- /dev/null +++ b/sdk/python/foundation-models/cohere/cohere-aisearch-rag.ipynb @@ -0,0 +1,508 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Use Cohere Command R or Cohere Command R+ with Azure AI Search to Answer Questions Using Your Data\n", + "\n", + "You can use Cohere `Command R` or Cohere `Command R+` models deployed in `Azure AI`and Azure ML to create advanced retrieval augmented generation (RAG) pipelines.\n", + "\n", + "> Review the [documentation](https://learn.microsoft.com/en-us/azure/ai-studio/how-to/deploy-models-cohere-command) for the Cohere family of models at for AI Studio and for ML Studio for details on how to provision inference endpoints, regional availability, pricing and inference schema reference." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Prerequisites\n", + "\n", + "Before we start, there are certain steps we need to take to deploy the models:\n", + "\n", + "* Register for a valid Azure account with subscription \n", + "* Make sure you have access to [Azure AI Studio](https://learn.microsoft.com/en-us/azure/ai-studio/what-is-ai-studio?tabs=home)\n", + "* Create a project and resource group\n", + "* Select `Cohere Command R` or `Cohere Command R+`.\n", + "\n", + " > Notice that some models may not be available in all the regions in Azure AI and Azure Machine Learning. On those cases, you can create a workspace or project in the region where the models are available and then consume it with a connection from a different one. To learn more about using connections see [Consume models with connections](https://learn.microsoft.com/en-us/azure/ai-studio/how-to/deployments-connections)\n", + "\n", + "* Deploy with \"Pay-as-you-go\"\n", + "* Follow the same steps for `Cohere-embed-v3-english`\n", + "\n", + "Once deployed successfully, you should be assigned for an API endpoint and a security key for inference.\n", + "\n", + "For more information, you should consult Azure's official documentation [here](https://learn.microsoft.com/en-us/azure/ai-studio/how-to/deploy-models-cohere-command) for model deployment and inference.\n", + "\n", + "### Services\n", + "\n", + "You will need to ensure the following services have been created in your Azure environment:\n", + "* Ensure you have created a search service. This can be done in the `Azure Portal` and more instructions can be found here: https://learn.microsoft.com/en-us/azure/search/search-what-is-azure-search **NOTE: You do not need to create the index, this will be done below**\n", + "\n", + "* Create a Cohere `Embed` and `Command` endpoint in the `Azure AI Studio`. Instructions can be found here: https://learn.microsoft.com/en-us/azure/ai-studio/how-to/deploy-models-cohere-command " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Example\n", + "\n", + "The following example demonstrate how to create a RAG workflow that uses a `Cohere Command R` or `Cohere Command R+` model deployed in Azure AI and Azure ML. We will also leverage Azure AI Search to store our documents along with LangChain to orchestrate the process" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Install Dependencies" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "! pip install --quiet cohere azure-search-documents azure-identity" + ] + }, + { + "cell_type": "code", + "execution_count": 91, + "metadata": {}, + "outputs": [], + "source": [ + "import cohere\n", + "from azure.core.credentials import AzureKeyCredential\n", + "from azure.search.documents.indexes import SearchIndexClient\n", + "from azure.search.documents import SearchClient\n", + "from azure.search.documents.models import VectorizedQuery\n", + "from azure.search.documents.indexes.models import (\n", + " SimpleField,\n", + " SearchFieldDataType,\n", + " SearchableField,\n", + " SearchField,\n", + " VectorSearch,\n", + " HnswAlgorithmConfiguration,\n", + " VectorSearchProfile,\n", + " SearchIndex,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "## Constants\n", + "We will set the values for the keys and models that we will use in our RAG pipeline" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's create instances of our Chat and Embed models deployed in Azure AI or Azure ML. Use the `langchain_cohere` package and configure it as follows:\n", + "\n", + "- `embed_endpoint` and `command_endpoint`: Use the endpoint URL from your deployment. Include `/v1` at the end of the endpoint URL." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# set constants\n", + "search_service_endpoint = \"https://.search.windows.net\"\n", + "key_credential = AzureKeyCredential(\"AZURE_SEARCH_ADMIN_KEY\")\n", + "index_name = \"AZURE_SEARCH_INDEX\" # this index does not have to be created yet\n", + "\n", + "# Embed\n", + "azure_cohere_embed_endpoint = \"https://..inference.ai.azure.com/v1\"\n", + "azure_cohere_embed_key = \"\"\n", + "\n", + "# Command\n", + "azure_cohere_command_endpoint = \"https://..inference.ai.azure.com/v1\"\n", + "azure_cohere_command_key = \"\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## AI Search Index\n", + "\n", + "We need to set up the index in our `Azure AI Search`\n", + "\n", + "We will use the `Cohere` package as well and set the URL to be our Azure AI endpoint, which will serve as the embedding model for our index" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Set up the embedding model to be used in the vector index\n", + "co_embed = cohere.Client(\n", + " base_url=azure_cohere_embed_endpoint, api_key=azure_cohere_embed_key\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Lets create some sample data to add to our index" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "docs_to_index = [\n", + " {\n", + " \"hotelId\": \"1\",\n", + " \"hotelName\": \"Fancy Stay\",\n", + " \"description\": \"Best hotel in town if you like luxury hotels.\",\n", + " \"category\": \"Luxury\",\n", + " },\n", + " {\n", + " \"hotelId\": \"2\",\n", + " \"hotelName\": \"Roach Motel\",\n", + " \"description\": \"Cheapest hotel in town. Infact, a motel.\",\n", + " \"category\": \"Budget\",\n", + " },\n", + " {\n", + " \"hotelId\": \"3\",\n", + " \"hotelName\": \"EconoStay\",\n", + " \"description\": \"Very popular hotel in town.\",\n", + " \"category\": \"Budget\",\n", + " },\n", + " {\n", + " \"hotelId\": \"4\",\n", + " \"hotelName\": \"Modern Stay\",\n", + " \"description\": \"Modern architecture, very polite staff and very clean. Also very affordable.\",\n", + " \"category\": \"Luxury\",\n", + " },\n", + " {\n", + " \"hotelId\": \"5\",\n", + " \"hotelName\": \"Secret Point\",\n", + " \"description\": \"One of the best hotel in town. The hotel is ideally located on the main commercial artery of the city in the heart of New York.\",\n", + " \"category\": \"Boutique\",\n", + " },\n", + "]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Create the embeddings\n", + "We will create a vector field for each of the hotel descriptions" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# embed each of the descriptions\n", + "# you will notice that Cohere has a field called \"input_type\" which can be set to \"search_document\", \"search_query\", \"classification\", or \"clustering\" depedning on the text you are embedding\n", + "for doc in docs_to_index:\n", + " doc[\"descriptionVector\"] = co_embed.embed(\n", + " texts=[doc[\"description\"]], input_type=\"search_document\"\n", + " ).embeddings[0]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Create the index\n", + "Next we will create the index using the Azure SDK" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# create the search index in azure\n", + "index_client = SearchIndexClient(\n", + " endpoint=search_service_endpoint, credential=key_credential\n", + ")\n", + "fields = [\n", + " SimpleField(\n", + " name=\"hotelId\",\n", + " type=SearchFieldDataType.String,\n", + " key=True,\n", + " sortable=True,\n", + " filterable=True,\n", + " facetable=True,\n", + " ),\n", + " SearchableField(name=\"hotelName\", type=SearchFieldDataType.String),\n", + " SearchableField(name=\"description\", type=SearchFieldDataType.String),\n", + " SearchableField(name=\"category\", type=SearchFieldDataType.String, filterable=True),\n", + " SearchField(\n", + " name=\"descriptionVector\",\n", + " type=SearchFieldDataType.Collection(SearchFieldDataType.Single),\n", + " searchable=True,\n", + " vector_search_dimensions=1024,\n", + " vector_search_profile_name=\"myHnswProfile\",\n", + " ),\n", + "]\n", + "\n", + "# Configure the vector search configuration\n", + "vector_search = VectorSearch(\n", + " algorithms=[HnswAlgorithmConfiguration(name=\"myHnsw\")],\n", + " profiles=[\n", + " VectorSearchProfile(\n", + " name=\"myHnswProfile\",\n", + " algorithm_configuration_name=\"myHnsw\",\n", + " )\n", + " ],\n", + ")\n", + "\n", + "# Create the search index with the semantic settings\n", + "index = SearchIndex(name=index_name, fields=fields, vector_search=vector_search)\n", + "result = index_client.create_or_update_index(index)\n", + "print(f\" {result.name} created\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Insert the embeddings\n", + "Finally, we will add the data into our vector index" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "search_client = SearchClient(\n", + " endpoint=search_service_endpoint, index_name=index_name, credential=key_credential\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "search_client.upload_documents(documents=docs_to_index)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Vector Search\n", + "We will create a small wrapper function to embed our query and search using Azure AI similarity search" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Search function to encode the query and search the index\n", + "def search(query):\n", + " \"\"\"\n", + " Searches for documents based on the given query.\n", + "\n", + " Args:\n", + " query (str): The search query.\n", + "\n", + " Returns:\n", + " list: A list of search results.\n", + " \"\"\"\n", + " query_embedding = co_embed.embed(\n", + " texts=[query], input_type=\"search_query\"\n", + " ).embeddings[0]\n", + "\n", + " # Azure AI search requires a vector query\n", + " vector_query = VectorizedQuery(\n", + " vector=query_embedding, k_nearest_neighbors=3, fields=\"descriptionVector\"\n", + " )\n", + "\n", + " search_results = search_client.search(\n", + " search_text=None,\n", + " vector_queries=[vector_query],\n", + " select=[\"hotelName\", \"description\", \"category\"],\n", + " )\n", + "\n", + " return search_results" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Test our search function\n", + "for r in search(\"luxury hotel\"):\n", + " print(r)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Create our Chat function\n", + "Next, we will create a basic chat function using the `Chat` Cohere endpoint \n", + "\n", + "Cohere's `chat endpoint` can accept documents directly and will return a grounded answer that includes citations against those documents.\n", + "\n", + "No prompt is needed since the `Cohere` model will automatically use a RAG prompt when documents are passed in." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "co_chat = cohere.Client(\n", + " base_url=azure_cohere_command_endpoint, api_key=azure_cohere_command_key\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "def ask(question):\n", + " \"\"\"\n", + " Ask a question to the chatbot, expecting a dictionary with 'question' and 'documents'.\n", + "\n", + " Args:\n", + " inputs (dict): A dictionary containing 'question' and 'documents'.\n", + "\n", + " Returns:\n", + " str: The response from the chatbot.\n", + " \"\"\"\n", + " search_results = search(question)\n", + "\n", + " # select category, description, and hotelName from the search results\n", + " documents = [\n", + " {\n", + " \"category\": result[\"category\"],\n", + " \"description\": result[\"description\"],\n", + " \"hotelName\": result[\"hotelName\"],\n", + " }\n", + " for result in search_results\n", + " ]\n", + "\n", + " response = co_chat.chat(message=question, documents=documents)\n", + "\n", + " return response" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "res = ask(\"Whats a good hotel close to city center?\")\n", + "print(res)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Clean the results\n", + "We can also pull the citations and text response from the response" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "def pretty_text(text, citations):\n", + " # Sort citations by start position to avoid issues when altering text indices\n", + " sorted_citations = sorted(citations, key=lambda x: x.start, reverse=True)\n", + "\n", + " # Process each citation in reverse order to prevent index shifting\n", + " for citation in sorted_citations:\n", + " doc_ids_str = \", \".join(citation.document_ids)\n", + " citation_text = text[citation.start : citation.end]\n", + " # Bold the citation text and add document ids as superscript\n", + " new_text = f\"**{citation_text}**^({doc_ids_str})\"\n", + " text = text[: citation.start] + new_text + text[citation.end :]\n", + "\n", + " return text" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "pretty_text_output = pretty_text(res.text, res.citations)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "print(pretty_text_output)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Aditional resources\n", + "\n", + "Here are some additional reference:\n", + "\n", + "* [Plan and manage costs](https://learn.microsoft.com/azure/ai-studio/how-to/costs-plan-manage#monitor-costs-for-models-offered-through-the-azure-marketplace)\n", + "\n", + "* [Learn more about the Cohere SDK](https://learn.microsoft.com/en-us/azure/ai-studio/how-to/deploy-models-cohere-command)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "base", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.13" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/sdk/python/foundation-models/cohere/cohere_faiss_langchain_embed.ipynb b/sdk/python/foundation-models/cohere/cohere_faiss_langchain_embed.ipynb new file mode 100644 index 00000000000..bcb9198f36a --- /dev/null +++ b/sdk/python/foundation-models/cohere/cohere_faiss_langchain_embed.ipynb @@ -0,0 +1,189 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Create a local (FAISS) vector index using Cohere embeddings - Langchain\t\n", + "\n", + "You can use Cohere Embed v3 - English and Cohere Embed v3 - Multilingual models deployed in Azure AI and Azure ML with `langchain` to create more sophisticated intelligent applications.\n", + "\n", + "> Review the [documentation](https://learn.microsoft.com/en-us/azure/ai-studio/how-to/deploy-models-cohere-embed) for the Cohere family of models at for AI Studio and for ML Studio for details on how to provision inference endpoints, regional availability, pricing and inference schema reference." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Prerequisites\n", + "\n", + "Before we start, there are certain steps we need to take to deploy the models:\n", + "\n", + "* Register for a valid Azure account with subscription \n", + "* Make sure you have access to [Azure AI Studio](https://learn.microsoft.com/en-us/azure/ai-studio/what-is-ai-studio?tabs=home)\n", + "* Create a project and resource group\n", + "* Select `Cohere Embed`.\n", + "\n", + " > Notice that some models may not be available in all the regions in Azure AI and Azure Machine Learning. On those cases, you can create a workspace or project in the region where the models are available and then consume it with a connection from a different one. To learn more about using connections see [Consume models with connections](https://learn.microsoft.com/en-us/azure/ai-studio/how-to/deployments-connections)\n", + "\n", + "* Deploy with \"Pay-as-you-go\"\n", + "\n", + "Once deployed successfully, you should be assigned for an API endpoint and a security key for inference.\n", + "\n", + "For more information, you should consult Azure's official documentation [here](https://learn.microsoft.com/en-us/azure/ai-studio/how-to/deploy-models-cohere) for model deployment and inference." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Example\n", + "\n", + "The following example demonstrate how to create a Facebook AI Similarity Search (Faiss) that uses a Cohere Embed v3 - English and Cohere Embed v3 - Multilingual models deployed in Azure AI and Azure ML. The chain has been configured with a `Faiss`. This example has been adapted from [LangChain official documentation](https://python.langchain.com/docs/integrations/vectorstores/faiss/)." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Install Dependencies" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "! pip install --quiet langchain langchain_cohere faiss-cpu" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_cohere import CohereEmbeddings\n", + "from langchain.vectorstores import FAISS\n", + "from langchain_core.documents.base import Document" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Prepare the data and format it for vectorstore" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "project_descriptions = [\n", + " \"Project Alpha: Implementing a new CRM system to improve customer relationships.\",\n", + " \"Project Beta: Developing an AI-based forecasting tool for sales trends.\",\n", + " \"Project Gamma: Overhauling the company website for better user engagement.\",\n", + " \"Project Delta: Integrating blockchain technology for enhanced supply chain transparency.\",\n", + " \"Project Epsilon: Launching a cloud migration strategy to enhance data accessibility and security.\",\n", + " \"Project Zeta: Implementing machine learning algorithms to optimize inventory management systems.\",\n", + " \"Project Eta: Developing a mobile application to improve field service operations and customer reporting.\",\n", + " \"Project Theta: Upgrading cybersecurity measures to protect against emerging digital threats.\",\n", + " \"Project Iota: Creating a virtual reality training program to enhance employee skill sets and engagement.\",\n", + " \"Project Kappa: Automating financial processes using AI to increase efficiency and reduce errors.\",\n", + "]\n", + "\n", + "project_docs = []\n", + "for doc in project_descriptions:\n", + " project_docs.append(Document(page_content=doc, metadata={\"source\": \"local\"}))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's create an instance of our model deployed in Azure AI or Azure ML. Use the `langchain_cohere` package and configure it as follows:\n", + "\n", + "- `embed_endpoint`: Use the endpoint URL from your deployment. Include `/v1` at the end of the endpoint URL." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "azure_embed_url = \"https://..inference.ai.azure.com/v1\"\n", + "azure_embed_key = \"\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "embed_model = CohereEmbeddings(base_url=azure_embed_url, cohere_api_key=azure_embed_key)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Ingest documents with Embed endpoint" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "db = FAISS.from_documents(project_docs, embed_model)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now, we can query the vectorstore" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "query = \"what are AI based projects?\"\n", + "results = db.similarity_search(query, k=3)\n", + "\n", + "for result in results:\n", + " print(result.page_content)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "rag-app", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.13" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/sdk/python/foundation-models/cohere/command_faiss_langchain.ipynb b/sdk/python/foundation-models/cohere/command_faiss_langchain.ipynb new file mode 100644 index 00000000000..6f69bd5fdd2 --- /dev/null +++ b/sdk/python/foundation-models/cohere/command_faiss_langchain.ipynb @@ -0,0 +1,279 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Use Cohere Command R/R+ to answer questions from data in local (FAISS) vector index - Langchain\n", + "\n", + "You can use Cohere Command R or Cohere Command R+ models deployed in Azure AI and Azure ML with `langchain` to create more sophisticated intelligent applications.\n", + "\n", + "> Review the [documentation](https://learn.microsoft.com/en-us/azure/ai-studio/how-to/deploy-models-cohere-command) for the Cohere family of models at for AI Studio and for ML Studio for details on how to provision inference endpoints, regional availability, pricing and inference schema reference." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Prerequisites\n", + "\n", + "Before we start, there are certain steps we need to take to deploy the models:\n", + "\n", + "* Register for a valid Azure account with subscription \n", + "* Make sure you have access to [Azure AI Studio](https://learn.microsoft.com/en-us/azure/ai-studio/what-is-ai-studio?tabs=home)\n", + "* Create a project and resource group\n", + "* Select `Cohere Command R` or `Cohere Command R+` and `Cohere-embed-v3-english` or `Cohere-embed-v3-multilingual`.\n", + "* Deploy with \"Pay-as-you-go\"\n", + "\n", + "Once deployed successfully, you should be assigned for an API endpoint and a security key for inference." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Install Dependencies\n", + "\n", + "The following example demonstrate how to create a Facebook AI Similarity Search (Faiss) that uses a Cohere Embed v3 - English and Cohere Embed v3 - Multilingual models deployed in Azure AI and Azure ML. The chain has been configured with a `CohereChat` and `CohereEmbeddings`. This example has been adapted from [LangChain official documentation](https://python.langchain.com/docs/integrations/chat/cohere/)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "! pip install --quiet langchain langchain_cohere faiss-cpu" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_cohere.chat_models import ChatCohere\n", + "from langchain_cohere import CohereEmbeddings\n", + "from langchain_core.documents.base import Document\n", + "from langchain.vectorstores import FAISS\n", + "from langchain.schema.runnable import RunnablePassthrough, RunnableLambda\n", + "from langchain.schema.output_parser import StrOutputParser\n", + "from langchain.prompts import ChatPromptTemplate\n", + "from langchain.schema import HumanMessage" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Constants\n", + "We will set the values for the keys and models that we will use in our RAG pipeline" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's create instances of our Chat and Embed models deployed in Azure AI or Azure ML. Use the `langchain_cohere` package and configure it as follows:\n", + "\n", + "- `embed_endpoint` and `command_endpoint`: Use the endpoint URL from your deployment. Include `/v1` at the end of the endpoint URL." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Embed\n", + "azure_cohere_embed_endpoint = \"https://..inference.ai.azure.com/v1\"\n", + "azure_cohere_embed_key = \"\"\n", + "\n", + "# Command\n", + "azure_cohere_command_endpoint = \"https://..inference.ai.azure.com/v1\"\n", + "azure_cohere_command_key = \"\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Prepare the data and format it for vectorstore FAISS" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Set up the embedding model to be used in the vector index\n", + "embed_model = CohereEmbeddings(\n", + " base_url=azure_cohere_embed_endpoint, cohere_api_key=azure_cohere_embed_key\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "project_descriptions = [\n", + " \"Project Alpha: Implementing a new CRM system to improve customer relationships.\",\n", + " \"Project Beta: Developing an AI-based forecasting tool for sales trends.\",\n", + " \"Project Gamma: Overhauling the company website for better user engagement.\",\n", + " \"Project Delta: Integrating blockchain technology for enhanced supply chain transparency.\",\n", + " \"Project Epsilon: Launching a cloud migration strategy to enhance data accessibility and security.\",\n", + " \"Project Zeta: Implementing machine learning algorithms to optimize inventory management systems.\",\n", + " \"Project Eta: Developing a mobile application to improve field service operations and customer reporting.\",\n", + " \"Project Theta: Upgrading cybersecurity measures to protect against emerging digital threats.\",\n", + " \"Project Iota: Creating a virtual reality training program to enhance employee skill sets and engagement.\",\n", + " \"Project Kappa: Automating financial processes using AI to increase efficiency and reduce errors.\",\n", + "]\n", + "\n", + "project_docs = []\n", + "for doc in project_descriptions:\n", + " project_docs.append(Document(page_content=doc, metadata={\"source\": \"local\"}))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Ingest documents with Embed endpoint" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# create the vector store using FAISS\n", + "db = FAISS.from_documents(project_docs, embed_model)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We can also convert the vectorstore into a Retriever class. This allows us to easily use it in other LangChain methods, which largely work with retrievers" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "retriever = db.as_retriever(search_kwargs={\"k\": 2})" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Create our Chat function\n", + "Next, we will create a basic chat function using the `ChatCohere` class\n", + "\n", + "Cohere's `chat endpoint` can accept documents directly and will return a grounded answer that includes citations against those documents.\n", + "\n", + "No prompt is needed since the `Cohere` model will automatically use a RAG prompt when documents are passed in.\n", + "\n", + "Because we are using `LangChain's expression language (LCEL)`, we will also wrap our function at the end with a `RunableLambda` function. Learn more about LCEL here: https://python.langchain.com/docs/expression_language/get_started/" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "def ask(inputs):\n", + " \"\"\"\n", + " Ask a question to the chatbot, expecting a dictionary with 'question' and 'documents'.\n", + "\n", + " Args:\n", + " inputs (dict): A dictionary containing 'question' and 'documents'.\n", + "\n", + " Returns:\n", + " str: The response from the chatbot.\n", + " \"\"\"\n", + " question = inputs[\"question\"]\n", + " documents = inputs[\"documents\"]\n", + "\n", + " chatbot = ChatCohere(\n", + " base_url=azure_cohere_command_endpoint,\n", + " cohere_api_key=azure_cohere_command_key,\n", + " )\n", + "\n", + " response = chatbot(messages=[HumanMessage(content=question)], documents=documents)\n", + " return response\n", + "\n", + "\n", + "ask = RunnableLambda(ask)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Create the Chain\n", + "We can now create the chain by chaining the retriever and the chat function together" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# create the answer chain using our search function and chat function\n", + "answer_chain = (\n", + " {\"documents\": retriever, \"question\": RunnablePassthrough()}\n", + " | ask\n", + " | StrOutputParser()\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Run our chain\n", + "You can now run the full chain and see the response as an `AIMessage` object. This will contain the models answer along with citations from the documents retrieved from `FAISS`" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "answer_chain.invoke(\"What AI based projects do we have?\")" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "rag-app", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.19" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/sdk/python/foundation-models/cohere/command_tools-langchain.ipynb b/sdk/python/foundation-models/cohere/command_tools-langchain.ipynb new file mode 100644 index 00000000000..69b0faf1d74 --- /dev/null +++ b/sdk/python/foundation-models/cohere/command_tools-langchain.ipynb @@ -0,0 +1,357 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Command R+ tool/function calling using LangChain\t\n", + "\n", + "You can use Cohere Command R or Cohere Command R+ models deployed in Azure AI and Azure ML with `langchain` to create more sophisticated intelligent applications.\n", + "\n", + "> Review the [documentation](https://learn.microsoft.com/en-us/azure/ai-studio/how-to/deploy-models-cohere-command) for the Cohere family of models at for AI Studio and for ML Studio for details on how to provision inference endpoints, regional availability, pricing and inference schema reference." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Prerequisites\n", + "\n", + "Before we start, there are certain steps we need to take to deploy the models:\n", + "\n", + "* Register for a valid Azure account with subscription \n", + "* Make sure you have access to [Azure AI Studio](https://learn.microsoft.com/en-us/azure/ai-studio/what-is-ai-studio?tabs=home)\n", + "* Create a project and resource group\n", + "* Select `Cohere Command R` or `Cohere Command R+` and `Cohere-embed-v3-english` or `Cohere-embed-v3-multilingual`.\n", + "* Deploy with \"Pay-as-you-go\"\n", + "\n", + "Once deployed successfully, you should be assigned for an API endpoint and a security key for inference." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Example\n", + "\n", + "Tool use allows developers to connect Cohere's models to external tools like search engines, APIs, functions, databases, etc.\n", + "\n", + "Multi-step tool use is an extension of this basic idea, and allows the model to call more than one tool in a sequence of steps, using the results from one tool call in a subsequent step. This process allows the language model to reason, perform dynamic actions, and quickly adapt on the basis of information coming from external sources.\n", + "\n", + "The recommended way to achieve [multi-step tool use with Cohere](https://docs.cohere.com/docs/multi-step-tool-use) is by leveraging the [Langchain framework](https://python.langchain.com/docs/integrations/providers/cohere/#react-agent) in Python.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Install Dependencies" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "! pip install --quiet langchain langchain_cohere" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from langchain_cohere.chat_models import ChatCohere\n", + "from langchain_cohere import CohereEmbeddings\n", + "from langchain_core.documents.base import Document\n", + "from langchain.vectorstores import FAISS\n", + "from langchain.agents import AgentExecutor\n", + "from langchain_cohere.react_multi_hop.agent import create_cohere_react_agent\n", + "from langchain.prompts import ChatPromptTemplate\n", + "from langchain_community.tools.tavily_search import TavilySearchResults\n", + "from langchain_core.pydantic_v1 import BaseModel, Field\n", + "import os" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Constants\n", + "We will set the values for the keys and models that we will use in our RAG pipeline" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's create instances of our Chat and Embed models deployed in Azure AI or Azure ML. Use the `langchain_cohere` package and configure it as follows:\n", + "\n", + "- `embed_endpoint` and `command_endpoint`: Use the endpoint URL from your deployment. Include `/v1` at the end of the endpoint URL." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Embed\n", + "azure_cohere_embed_endpoint = \"https://..inference.ai.azure.com/v1\"\n", + "azure_cohere_embed_key = \"\"\n", + "\n", + "# Command\n", + "azure_cohere_command_endpoint = \"https://..inference.ai.azure.com/v1\"\n", + "azure_cohere_command_key = \"\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Set up the command model to be used in a chat\n", + "chat_model = ChatCohere(\n", + " base_url=azure_cohere_command_endpoint, cohere_api_key=azure_cohere_command_key\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Set up the embedding model to be used in the vector index\n", + "embed_model = CohereEmbeddings(\n", + " base_url=azure_cohere_embed_endpoint, cohere_api_key=azure_cohere_embed_key\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Define tools\n", + "Your agent will be equipped with the following tools. The model can pick between\n", + "\n", + "* doing a search on the web\n", + "* and/or retrieving data from a vector store\n", + "* and/or directly answering [ this tool comes out of the box! ]\n", + "\n", + "Plus the model can self-reflect." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Vector search\n", + "\n", + "You can easily equip your agent with a vector store!" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "project_descriptions = [\n", + " \"Project Alpha: Implementing a new CRM system to improve customer relationships.\",\n", + " \"Project Beta: Developing an AI-based forecasting tool for sales trends.\",\n", + " \"Project Gamma: Overhauling the company website for better user engagement.\",\n", + " \"Project Delta: Integrating blockchain technology for enhanced supply chain transparency.\",\n", + " \"Project Epsilon: Launching a cloud migration strategy to enhance data accessibility and security.\",\n", + " \"Project Zeta: Implementing machine learning algorithms to optimize inventory management systems.\",\n", + " \"Project Eta: Developing a mobile application to improve field service operations and customer reporting.\",\n", + " \"Project Theta: Upgrading cybersecurity measures to protect against emerging digital threats.\",\n", + " \"Project Iota: Creating a virtual reality training program to enhance employee skill sets and engagement.\",\n", + " \"Project Kappa: Automating financial processes using AI to increase efficiency and reduce errors.\",\n", + "]\n", + "\n", + "project_docs = []\n", + "for doc in project_descriptions:\n", + " project_docs.append(Document(page_content=doc, metadata={\"source\": \"local\"}))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Ingest documents with Embed endpoint" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "db = FAISS.from_documents(project_docs, embed_model)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We can also convert the vectorstore into a Retriever class. This allows us to easily use it in other LangChain methods, which largely work with retrievers" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "db_retriever = db.as_retriever(search_kwargs={\"k\": 2})" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from langchain.tools.retriever import create_retriever_tool\n", + "\n", + "vectorstore_search = create_retriever_tool(\n", + " retriever=db_retriever,\n", + " name=\"vectorstore_search\",\n", + " description=\"Retrieve relevant info from a vectorstore that contains information about Projects.\",\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Web search\n", + "You can easily equip your agent with web search!" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "os.environ[\n", + " \"TAVILY_API_KEY\"\n", + "] = \"API_KEY\" # you can create an API key for free on Tavily's website\n", + "\n", + "internet_search = TavilySearchResults()\n", + "internet_search.name = \"internet_search\"\n", + "internet_search.description = \"Returns a list of relevant document snippets for a textual query retrieved from the internet.\"\n", + "\n", + "\n", + "class TavilySearchInput(BaseModel):\n", + " query: str = Field(description=\"Query to search the internet with\")\n", + "\n", + "\n", + "internet_search.args_schema = TavilySearchInput" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Create ReAct Agent\n", + "\n", + "The model can smartly pick the right tool(s) for the user query, call them in any sequence, analyze the results and self-reflect.\n", + "Once the model considers it has enough information to answer the user question, it generates the final answer." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Preamble\n", + "preamble = \"\"\"\n", + "You are an expert who answers the user's question with the most relevant datasource. You are equipped with an internet search tool and a special vectorstore of information about Projects.\n", + "Use internet search to define strategies.\n", + "\"\"\"\n", + "\n", + "# Prompt template\n", + "prompt = ChatPromptTemplate.from_template(\"{input}\")\n", + "\n", + "# Create the ReAct agent\n", + "agent = create_cohere_react_agent(\n", + " llm=chat_model,\n", + " tools=[internet_search, vectorstore_search],\n", + " prompt=prompt,\n", + ")\n", + "\n", + "agent_executor = AgentExecutor(\n", + " agent=agent, tools=[internet_search, vectorstore_search], verbose=True\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Ask a standalone question to the ReAct agent\n", + "A question that requires using a predefined tool from Langchain" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "response = agent_executor.invoke(\n", + " {\n", + " \"input\": \"Summarize the AI based projects we are currently working on\",\n", + " \"preamble\": preamble,\n", + " }\n", + ")\n", + "\n", + "response[\"output\"]" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "response = agent_executor.invoke(\n", + " {\n", + " \"input\": \"How can we automate financial processes on Project Kappa? Then define the strategy to execite that process.\",\n", + " \"preamble\": preamble,\n", + " }\n", + ")\n", + "\n", + "response[\"output\"]" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "rag-app", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.19" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +}