Why List<Double> instead of float[] for embeddings ? #632
michaelsembwever
started this conversation in
General
Replies: 1 comment
-
This was addressed in version M2, where the type of embeddings has been changed from |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Curious to why
List<Double>
was used, considering the performance benefits a primitive array gives, particularly with SIMD operations newer Java versions can do.A number of the vector stores have already a
toFloatArray(List<Double>)
to workaround this.Bonus question :)
What about using the JVector library and its
VectorFloat
class …?It's heavily optimised against the Panama Vector API(SIMD), and detects and takes advantage of JDK20+ when available.
While spring-ai might not be doing much with the embeddings, it's quite likely consumers of the framework will be, e.g. re-ranking large similarity result sets.
Beta Was this translation helpful? Give feedback.
All reactions