Skip to content

Commit

Permalink
added SPLADE example
Browse files Browse the repository at this point in the history
  • Loading branch information
cmacdonald authored Jul 14, 2024
1 parent 1efb056 commit 40e5d02
Showing 1 changed file with 19 additions and 0 deletions.
19 changes: 19 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,25 @@ splade_retr = splade.query() >> pt.BatchRetrieve('./msmarco_psg', wmodel='Tf')

```

# PISA

For faster retrieval with SPLADE, you can use the fast PISA retrieval backend provided by [PyTerrier_PISA](https://github.com/terrierteam/pyterrier_pisa):

```python
import pyt_splade
splade = pyt_splade.Splade()
dataset = pt.get_dataset('irds:msmarco-passage')
index = PisaIndex('./msmarco-passage-splade', stemmer='none')

# indexing
idx_pipeline = splade.doc_encoder() >> index.toks_indexer()
idx_pipeline.index(dataset.get_corpus_iter())

# retrieval

retr_pipeline = splade.query_encoder() >> index.quantized()
```

# Demo

We have a demo of PyTerrier_SPLADE at https://huggingface.co/spaces/terrierteam/splade
Expand Down

0 comments on commit 40e5d02

Please sign in to comment.