Do not include arm_neon.h when running under nvcc #1028
Conversation
src/ggml-impl.h
Outdated
| @@ -14,7 +14,7 @@ | |||
| #include <arm_sve.h> | |||
| #endif // __ARM_FEATURE_SVE | |||
|
|
|||
| #if defined(__ARM_NEON) | |||
| #if defined(__ARM_NEON) && !(defined(__NVCC__) && defined(__CUDACC__)) | |||
There was a problem hiding this comment.
Is there any case where __CUDACC__ is defined and __NVCC__ is not?
There was a problem hiding this comment.
This is following the recommendation of abseil/abseil-cpp#1665 (comment)
I guess per https://llvm.org/docs/CompileCudaWithLLVM.html#detecting-clang-vs-nvcc-from-code we will have defined(__CUDACC__) && !defined(__NVCC__) when compiling with Clang. I'm not really sure whether we want to avoid including the header or not in this case. Do we ever want the header when compiling CUDA code? Probably not? Shall we just go with !defined(__CUDACC__) then?
There was a problem hiding this comment.
Okay, I've change it so it won't include the header when compiling CUDA code under either GCC or Clang.
There was a problem hiding this comment.
I am not sure if it necessary for clang, but in any case this code is not actually used in the CUDA backend, so disabling it entirely for all CUDA compilation is ok. In the future I will probably move this to the CPU backend.
frankier
force-pushed
the
no-neon-header-nvcc
branch
from
88b306f
to
808733f
Compare
I'm having trouble building llama.cpp with ARM and CUDA. The reason seems to be that the build process ends up trying to compile something with
nvccwhich includes thearm_neon.hheader which uses a bunch of NEON intrinsics thatnvccdoesn't know about.See here: https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=1090459&view=logs&jobId=d93976e3-6ed1-588b-ebf8-e8a19c2becc4&j=d93976e3-6ed1-588b-ebf8-e8a19c2becc4&t=9243320e-a173-5427-7ff1-0cd57d39b91c
Following abseil/abseil-cpp#1665 I've changed the code to conditionally include the header.