Skip to content

Add AI-powered search features #827

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 29 commits into
base: main
Choose a base branch
from
Open

Conversation

Strift
Copy link
Contributor

@Strift Strift commented Mar 12, 2025

Pull Request

Related issue

Fixes #817

What does this PR do?

Update settings to handle embedders

Docs: https://www.meilisearch.com/docs/reference/api/settings#embedders

Update the methods getEmbedders, updateEmbedders, resetEmbedders. Also, the method updateSettings should be able to accept the new embedders parameter.

Here is the list of fields in the embedders object:

  • source sub field is available and accepts: ollama, rest, openAI, huggingFace and userProvided
  • apiKey sub field is available (string) - optional because not compatible with all sources. Only for openAi, ollama, rest.
  • model sub field is available (string) - optional because not compatible with all sources. Only for ollama, openAI, huggingFace
  • documentTemplate sub field is available (string) - optional
  • dimensions - optional because not compatible with all sources. Only for openAi, huggingFace, ollama, and rest
  • distribution - optional
  • request - mandatory only if using rest embedder
  • response - mandatory only if using rest embedder
  • documentTemplateMaxBytes - optional
  • revision - optional, only for huggingFace
  • headers - optional, only for rest
  • binaryQuantized - optional

Update search to handle vector search and hybrid search

Docs: https://www.meilisearch.com/docs/reference/api/search

Update the search method::

  • hybrid search parameter, with sub fields semanticRatio and embedder. embedder is mandatory if hybrid is set.
  • vector parameter is available
  • retrieveVectors parameter available
  • semanticHitCount in search response
  • Accept _semanticScore in the search response (optional)
  • vector should be returned in the search response, but optional (because depends on search parameters)
  • _vectors present in the search response, but optional

Add similar documents endpoint

Docs: https://www.meilisearch.com/docs/reference/api/similar

  • Implement searchSimilarDocuments associated with the POST /indexes/:uid/similar. Do NOT implement with GET.

PR checklist

Please check if your PR fulfills the following requirements:

  • Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
  • Have you read the contributing guidelines?
  • Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!

@Strift Strift marked this pull request as draft March 12, 2025 06:48
@Strift Strift force-pushed the feat/add-ai-powered-search branch 2 times, most recently from cb480aa to c8647ec Compare March 18, 2025 02:07
@Strift Strift force-pushed the feat/add-ai-powered-search branch from 5467c2f to 4306308 Compare April 8, 2025 08:27
@Strift Strift requested a review from Copilot April 24, 2025 06:10
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds AI-powered search capabilities to Meilisearch by introducing new embedders settings, hybrid search parameters, and vector search support.

  • Updated the settings API to include embedders configuration.
  • Enhanced search and similar document endpoints with hybrid and vector search features.
  • Added and updated tests for embedders, hybrid search, and vector search.

Reviewed Changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
src/test/java/com/meilisearch/sdk/SimilarDocumentRequestTest.java Added tests for SimilarDocumentRequest with various parameters including embedder and vector retrieval.
src/test/java/com/meilisearch/sdk/SearchRequestTest.java Introduced hybrid search and retrieveVectors tests.
src/test/java/com/meilisearch/integration/TasksTest.java Improved assertions for tasks retrieval.
src/test/java/com/meilisearch/integration/SettingsTest.java Added tests verifying get, update, and reset embedders settings.
src/test/java/com/meilisearch/integration/SearchTest.java Added integration tests for vector search and retrieveVectors feature.
src/main/java/com/meilisearch/sdk/model/Settings.java Updated embedders type from Embedders to Embedder.
src/main/java/com/meilisearch/sdk/model/SearchResult.java Added _vectors field to support vector search responses.
src/main/java/com/meilisearch/sdk/model/Hybrid.java Introduced a new Hybrid class for hybrid search configuration.
src/main/java/com/meilisearch/sdk/model/Embedder.java New file defining Embedder configurations with embedders related fields.
src/main/java/com/meilisearch/sdk/SimilarDocumentRequest.java Refactored JSON conversion using null-safe putOpt.
src/main/java/com/meilisearch/sdk/SettingsHandler.java Added methods to get, update, and reset embedders settings.
src/main/java/com/meilisearch/sdk/SearchRequest.java Extended to support hybrid search, vector search, and retrieveVectors.
src/main/java/com/meilisearch/sdk/IndexSearchRequest.java Enhanced to include retrieveVectors parameter in search requests.
src/main/java/com/meilisearch/sdk/Index.java Added wrapper methods for embedders settings management.
.github/workflows/tests.yml Minor update to enable more verbose Gradle output during tests.

@Strift Strift marked this pull request as ready for review April 24, 2025 06:42
@Strift Strift requested a review from curquiza April 24, 2025 06:43
@curquiza curquiza requested review from brunoocasali and removed request for curquiza April 24, 2025 07:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[v1.13] Stabilize AI-powered search
1 participant