Skip to content

Commit

Permalink
fix Similarity scorer args names
Browse files Browse the repository at this point in the history
  • Loading branch information
tcapelle committed Feb 7, 2025
1 parent 2c26d3e commit 5514d46
Showing 1 changed file with 4 additions and 5 deletions.
9 changes: 4 additions & 5 deletions docs/docs/guides/evaluation/builtin_scorers.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -186,14 +186,14 @@ import TabItem from '@theme/TabItem';

similarity_scorer = EmbeddingSimilarityScorer(
model_id="openai/text-embedding-3-small", # or any other model supported by litellm
target_column="reference_text", # the dataset column to compare the output against
threshold=0.4 # the cosine similarity threshold
)
```

Note: You can use `column_map` to map the `target` column to a different name.

**Parameters:**

- `target`: This scorer expects a `target` column in your dataset. It calculates the cosine similarity of the embeddings of the `target` text to the AI system output. Use the `column_map` attribute if the column has a different name.
- `threshold` (float): The minimum cosine similarity score (between -1 and 1) needed to consider the two texts similar (defaults to `0.5`).

Here is an example usage in the context of an evaluation:
Expand All @@ -206,18 +206,17 @@ import TabItem from '@theme/TabItem';
# Initialize scorer
similarity_scorer = EmbeddingSimilarityScorer(
model_id="openai/text-embedding-3-small", # or any other model supported by litellm
target_column="reference_text",
threshold=0.7
)
# Create dataset
dataset = [
{
"input": "He's name is John",
"reference": "John likes various types of cheese.",
"target": "John likes various types of cheese.",
},
{
"input": "He's name is Pepe.",
"reference": "Pepe likes various types of cheese.",
"target": "Pepe likes various types of cheese.",
},
]
# Define model
Expand Down

0 comments on commit 5514d46

Please sign in to comment.