[SWDEV-492880] Add ROCm path for hipRTC

[huanrwan-amd]

This fix is for #3314 .

As in the issue: "hipRTC/Comgr automatically added -I /opt/rocm/include to kernel compilations. However, this behavior is being removed, as hipRTC has no guarantees about a system ROCm installation during kernel compilation/execution."

[huanrwan-amd]

Open for suggestions: For dealing with no symbolic link to the rocm directory cases

[cderb]

I'll approve because this will be correct for the case with the symbolic link present and doesn't present complications otherwise.
In the case where rocm is installed somewhere else, this change would not be sufficient. That path might be part of the CMAKE_PREFIX_PATH.

[huanrwan-amd]

I'll approve because this will be correct for the case with the symbolic link present and doesn't present complications otherwise. In the case where rocm is installed somewhere else, this change would not be sufficient. That path might be part of the CMAKE_PREFIX_PATH.

Thanks for the input. For my understanding, the CMAKE_PREFIX_PATH is for building the MIOpendriver binary and *.so. This function BuildHip() is to using hipRTC for an online compilation.
One possibility here is to check ROCM_PATH and ROCM_HOME env:

        const char* rocm_path = std::getenv("ROCM_PATH");
        if(rocm_path == nullptr)
        {
            rocm_path = "/opt/rocm";
        }
        const char* rocm_home = std::getenv("ROCM_HOME");
        if(rocm_home == nullptr)
        {
            rocm_home = rocm_path;
        }
        opts.push_back(std::string("-I") + rocm_home + "/include");

I see people using both ROCM_PATH and ROCM_HOME at: cupy/cupy#4493, But ROCM_PATH is recommended. Any suggestions?

[lamb-j]

I've seen ROCM_PATH used as well, not familiar with ROCM_HOME

[huanrwan-amd]

Hi @lamb-j and @cderb updated as we discussed. Using ROCM_PATH for a user specified ROCm path and log the build options for debugging.

[cderb]

LGTM

[BrianHarrisonAMD]

Greetings @huanrwan-amd

Heads up, there was a formatting error on these changes.
I fixed it to allow CI to run, but here is the script I run in case you need it in the future:

#!/bin/bash
find . -iname '*.h' \
    -o -iname '*.hpp' \
    -o -iname '*.cpp' \
    -o -iname '*.h.in' \
    -o -iname '*.hpp.in' \
    -o -iname '*.cpp.in' \
    -o -iname '*.cl' \
| grep -v -E '(build/)|(install/)' \
| xargs -n 1 -P $(nproc) -I{} -t clang-format-12 -style=file {} -i 2>/dev/null

[BrianHarrisonAMD]

There is an error on this line for the checker, I think we need to use the MIOpen Macros instead.

Ex:

// at top of file
MIOPEN_DECLARE_ENV_VAR_STR(MIOPEN_CUSTOM_CACHE_DIR)

// etc.

// usage example
const auto& custom = env::value(MIOPEN_CUSTOM_CACHE_DIR);
if(!custom.empty())
{
    p = ExpandUser(custom);
}

It will also be a string type, and you can avoid some of the below manual conversions to std::string.

[CAHEK7]

And the most important part, it caches the results, that all the consequence compilations with get that path much faster.

Also it supports default values, so path.empty() check is not required - it either returns user-provided path or default /opt/rocm and that result is cached.

[huanrwan-amd]

Hi @CAHEK7 , thank you very much for providing comments and instructions. There would be some cases where ROCM_PATH is not there at all. I checked with a clean installation of ROCm6.2.4. No ROCM_PATH...
So I think give a default value here is useful here to use hipRTC out of box, although we can capture this built options strings through logs printed below and add ROCM_PATH manually.

[CAHEK7]

Greetings @huanrwan-amd

Heads up, there was a formatting error on these changes. I fixed it to allow CI to run, but here is the script I run in case you need it in the future:

#!/bin/bash
find . -iname '*.h' \
    -o -iname '*.hpp' \
    -o -iname '*.cpp' \
    -o -iname '*.h.in' \
    -o -iname '*.hpp.in' \
    -o -iname '*.cpp.in' \
    -o -iname '*.cl' \
| grep -v -E '(build/)|(install/)' \
| xargs -n 1 -P $(nproc) -I{} -t clang-format-12 -style=file {} -i 2>/dev/null

It's also here https://github.com/ROCm/MIOpen/wiki/How-to-format-code

[BrianHarrisonAMD]

@huanrwan-amd Looks like some new formatting errors on this PR.
Likely need to run the clang script.

[huanrwan-amd]

@huanrwan-amd Looks like some new formatting errors on this PR. Likely need to run the clang script.

Thank you for the reminder, ran scripts and pushed the changes.