feat: ort v2 BREAKING

[joshniemela]

Migrated the code to ort to 2.0.0 based on the breaking API changes
Changed padding strategy and wrote a sequential version of query_embed, this gives a speedup of 95%

[github-actions]

PR Compliance Checks

Thank you for your Pull Request! We have run several checks on this pull request in order to make sure it's suitable for merging into this project. The results are listed in the following section.

Issue Reference

In order to be considered for merging, the pull request description must refer to a specific issue number. This is described in our Contributing Guide.
This check is looking for a phrase similar to: "Fixes #XYZ" or "Resolves #XYZ" where XYZ is the issue number that this PR is meant to address.

Protected Branch

In order to be considered for merging, the pull request changes must not be implemented on the "main" branch. This is described in our Contributing Guide. We would suggest that you close this PR and implement your changes as described in our Contributing Guide and open a new pull request.

[Anush008]

Maybe we can dynamic linking mentioned here.
https://ort.pyke.io/setup/linking#load-dynamic-a-glimmer-of-hope

Reference from FastEmbed-go https://github.com/Anush008/fastembed-go/blob/7fe735ed91f0d510ce07b58c6d5dd5ae31fb0218/.github/workflows/release.yml#L30-L34

For ORT_DYLIB_PATH.

[decahedron1]

Libraries should only enable features they need.

[dependencies]
ort = { version = "2.0.0-alpha.4", default-features = false, features = [ "ndarray" ] }

[dev-dependencies]
ort = "2.0.0-alpha.4"

[decahedron1]

@Anush008 btw, 7e0526a has a critical performance fix. @joshniemela and I found that PaddingStrategy::Fixed caused fastembed-rs to be almost 15x slower than a Python equivalent on shorter sequences. Like the comment says, a new field should be added to InitOptions to change the padding strategy if the user wishes (InitOptions should also be marked #[non_exhaustive] imo).

[joshniemela]

#9 is probably also related to this PR

[decahedron1]

Indeed it significantly improves #9.

master

init: 138.47ms
passage_embed: 579.84ms
query_embed: 147.26ms

v2

init: 133.73ms
passage_embed: 39.97ms
query_embed: 4.50ms

[joshniemela]

Indeed it significantly improves #9.

master

init: 138.47ms
passage_embed: 579.84ms
query_embed: 147.26ms

v2

init: 133.73ms
passage_embed: 39.97ms
query_embed: 4.50ms

Is it possible to do a test on python as well on the same machine?

[decahedron1]

Is it possible to do a test on python as well on the same machine?

init: 118.33ms
passage_embed: 17.55ms
query_embed: 3.51ms

[github-actions]

🎉 This PR is included in version 1.11.0 🎉

The release is available on:

• GitHub release
• v1.11.0

Your semantic-release bot 📦🚀

[Anush008]

Bummer. Should've been a major release.

[Anush008]

@joshniemela, @decahedron1 Thanks for the efforts. Super grateful. This is amazing.

[Anush008]

After some commit slogging, it is finally up.
https://github.com/Anush008/fastembed-rs/releases/tag/v2.0.0

I've yanked 1.11.0.

[prattcmp]

Thank you for the perf enhancements, but the ONNX runtime upgrade needs to be notated.

This should be released as v2.0.0-alpha or with some other indication that it is not a stable release since it depends on an alpha version of ONNX runtime.

[decahedron1]

This should be released as v2.0.0-alpha or with some other indication that it is not a stable release since it depends on an alpha version of ONNX runtime.

ort v2.0.0-alpha.4 uses ONNX Runtime v1.16.3, so only a minor upgrade as ort 1.16 uses 1.16.0. It does not use an alpha version of ONNX Runtime.

[prattcmp]

This should be released as v2.0.0-alpha or with some other indication that it is not a stable release since it depends on an alpha version of ONNX runtime.

ort v2.0.0-alpha.4 uses ONNX Runtime v1.16.3, so only a minor upgrade as ort 1.16 uses 1.16.0. It does not use an alpha version of ONNX Runtime.

Thank you for correcting that the crate is alpha, not the actual runtime. Nevertheless, this release is not stable and should indicate such in its versioning.

P.S. I think its really awesome that you are maintaining this crate for the community! I'm just trying to help :).