Add Qualcomm mobile SoC native backend for GGML

[zhouwg]

1. Apple's dedicated machine learning acceleration library(ANE) is very important for GGML/llama.cpp's performance on iOS/Mac

we should/might study something about Qualcomm's dedicated/proprietary machine learning acceleration library accordingly.

2. background:

• simple backend
• Opinions about template based GEMM kernels

  Opinions about template based GEMM kernels llama.cpp#3965

  

• Add GPU support to ggml

  Add GPU support to ggml llama.cpp#915

3. A small R&D team from Intel China branch in Shanghai has provided ggml with the sycl native backend based on Intel chips.

• Adding Native Support of SYCL for Intel GPUs

  Adding Native Support of SYCL for Intel GPUs llama.cpp#4749

• SYCL backend support Multi-card

  SYCL backend support Multi-card llama.cpp#5282

[zhouwg]

It seems done(although lack of implementation of other GGML OPs using QNN API ) and now it's PR stage.

Implementation of GGML_OP_MUL_MAT using QNN API and GGML_OP_ADD using QNN API and GGML_OP_MUL using QNN API could be found in ggml-qnn.cpp. other implementation of GGML OPs using QNN API is very similar to GGML_OP_ADD / GGML_OP_MUL / GGML_OP_MULMAT in ggml-qnn.cpp. hope community could help to do it in upstream whiserp.cpp after the PR is accepted by upstream whisper.cpp

so I'd like to close this opening issue accordingly.

[ZodiacFRA]

@zhouwg Thanks for you work :) Has the PR been merged? I cannot find QNN in the llama.cpp nor whisper.cpp supported backend list

[zhouwg]

@zhouwg Thanks for you work :) Has the PR been merged? I cannot find QNN in the llama.cpp nor whisper.cpp supported backend list

not yet. pls refer to my third formal PR:ggml-org/llama.cpp#12326 and hope it could get some active supports from the great llama.cpp community and it will be a good starting point of ggml-qnn.