ggml : automatic selection of best CPU backend

[slaren]

This the way it works:

• Backends can export a function called ggml_backend_score
• When loading a backend, all the available variants are checked and the highest score one is loaded
• A score of 0 means that the backend cannot be used in the current system
• The available variants are discovered automatically based on the file name, for example, when loading the CPU backend, all files that match libggml-cpu-*.so (or ggml-cpu-*.dll on windows) are checked.

The CPU backend implements this functionality for x86-64 and returns a score depending on the features included in the build that are supported on the running system.

The llama-server docker image has been updated to include variants for AVX, AVX2, AVX512 and AMX.

Caveat: the AVX and AVX2 variants still require FMA and F16C, which will limit the number of processors supported. More variants may be needed to fully support some microarchitectures.

[giladgd]

I think it may be worth having a cmake flag to build all the common CPU backend variations in a single build rather than having to build multiple times and combining the backend libraries.
Having this would make it easier to maintain a centralized list of the common configurations that would be supported by projects that use llama.cpp.

[slaren]

Yes, I agree that would be better. TBH selecting the variants is a headache because there are so many options and each microarchitecture supports a different subset of them, so I didn't want to think too much about it. It might make more sense to build a variant for each microarchitecture, but there is going to be a lot of them.