Blazing Fast SetFit Inference with 🤗 Optimum Intel on Xeon

[tomaarsen]

Hello!

Pull Request overview

• Add "Blazing Fast SetFit Inference on Intel Xeon" blogpost

Details

This blogpost follows the notebook contributed by Intel showcasing the performance optimizations possible for SetFit models using optimum-intel.

TODOs:

• Twice we refer to the "notebook", which is currently https://github.com/danielkorat/setfit/blob/faster-optimum-intel-nb/notebooks/setfit-optimum-intel.ipynb. After Switch to differentiable head, larger input sequence setfit#489 is merged, we should link to https://github.com/huggingface/setfit/blob/main/notebooks/setfit-optimum-intel.ipynb instead.
• Merge Switch to differentiable head, larger input sequence setfit#489 (await final changes by @danielkorat)

@echarlaix Feel free to assist me with reviewing.
cc @danielkorat @MosheWasserb @orenpereg

• Tom Aarsen

[tomaarsen]

Note for reviewers: The blogpost is still being edited in a Google Docs environment outside of this PR, so expect fairly notable changes to be merged into this PR later.

[echarlaix]

Looks great !

[tomaarsen]

Thanks for the review @echarlaix! Your suggestions look very sensible.

@danielkorat @orenpereg @MosheWasserb What is your ideal release date?

[orenpereg]

Hi @tomaarsen, we are just waiting for Intel approval to publish performance numbers on Intel CPU. hope to get it in the next few days.

[tomaarsen]

All good! Thanks for the heads up.

[danielkorat]

@tomaarsen
Please replace with this updated version:

[danielkorat]

[aprabhak2]

@danielkorat, Do we want to draw the arrow from 119 to 935 to show the 7.8x speedup?

[danielkorat]

@aprabhak2 The arrow was initially diagonal, from 119.6 to 935.2, but we got a comment from Gadi saying that a vertical arrow will look better in publication.
WDYT @orenpereg ?

[orenpereg]

I would recommend to stay with the the vertical arrow. the idea is the the green line in general provides speedup over the blue line. Maybe just change "throughput speedup" to "max throughput speedup 7.8x"

[tomaarsen]

Fixed via fbc92fc, feel free to comment again if you prefer another figure.

[aprabhak2]

Looks good from my side.

[aprabhak2]

@danielkorat, Do we want to draw the arrow from 119 to 935 to show the 7.8x speedup?