[L0 v2] Multiple Graph E2E tests (flaky?) fail on Intel Devices

[Maetveis]

Describe the bug

There are multiple tests that seem to be failing in multiple PRs, that have no relevant
changes:

Some examples of the failures

• https://github.com/intel/llvm/actions/runs/15717366388/job/44292174201?pr=19030
• https://github.com/intel/llvm/actions/runs/15713348923/job/44278344460?pr=19028
• https://github.com/intel/llvm/actions/runs/15726048870/job/44317302676?pr=18995

PVC:

********************
Failed Tests (12):
  SYCL :: Graph/Explicit/interop-level-zero-launch-kernel.cpp
  SYCL :: Graph/Explicit/memadvise.cpp
  SYCL :: Graph/Explicit/opencl_local_acc.cpp
  SYCL :: Graph/Explicit/prefetch.cpp
  SYCL :: Graph/Explicit/spec_constants_handler_api.cpp
  SYCL :: Graph/Explicit/spec_constants_kernel_bundle_api.cpp
  SYCL :: Graph/RecordReplay/interop-level-zero-launch-kernel.cpp
  SYCL :: Graph/RecordReplay/memadvise.cpp
  SYCL :: Graph/RecordReplay/opencl_local_acc.cpp
  SYCL :: Graph/RecordReplay/prefetch.cpp
  SYCL :: Graph/RecordReplay/raw_kernel_arg.cpp
  SYCL :: Graph/RecordReplay/spec_constants_handler_api.cpp

BMG:

********************
Failed Tests (11):
  SYCL :: Graph/Explicit/interop-level-zero-launch-kernel.cpp
  SYCL :: Graph/Explicit/memadvise.cpp
  SYCL :: Graph/Explicit/opencl_local_acc.cpp
  SYCL :: Graph/Explicit/prefetch.cpp
  SYCL :: Graph/Explicit/raw_kernel_arg.cpp
  SYCL :: Graph/RecordReplay/interop-level-zero-launch-kernel.cpp
  SYCL :: Graph/RecordReplay/memadvise.cpp
  SYCL :: Graph/RecordReplay/opencl_local_acc.cpp
  SYCL :: Graph/RecordReplay/prefetch.cpp
  SYCL :: Graph/RecordReplay/raw_kernel_arg.cpp
  SYCL :: Graph/RecordReplay/spec_constants_kernel_bundle_api.cpp

To reproduce

1. Include a code snippet that is as short as possible
2. Specify the command which should be used to compile the program
3. Specify the command which should be used to launch the program
4. Indicate what is wrong and what was expected

Environment

• OS: Linux
• Target device and vendor: Intel GPU
• DPC++ version: [e.g. commit hash or output of clang++ --version]
• Dependencies version: [e.g. the output of sycl-ls --verbose]

Additional context

No response

[EwanC]

Looking at the logs, this is V2 adapter related, e.g the first run seems to complete but UR_LOADER_USE_LEVEL_ZERO_V2=1 fails. cc @pbalcer

Looking at the preftch output It looks like this command fails UR ---> commandListLocked->appendUSMPrefetch( pMemory, size, flags, numSyncPointsInWaitList, eventsWaitList, hCommandBuffer->createEventIfRequested(pSyncPoint)) and for the memadvise fails commandListLocked->appendUSMAdvise( pMemory, size, advice, numSyncPointsInWaitList, eventsWaitList, hCommandBuffer->createEventIfRequested(pSyncPoint))

The other fails have less obbious error output though

# note: command had no output on stdout or stderr
# error: command failed with exit status: -11

I'm not sure why these would be flaky

[EwanC]

Created a PR to disable these tests to get CI stable #19036

Also spotted this additional failing test in https://github.com/intel/llvm/actions/runs/15712376199/job/44320223882?pr=18792

********************
FAIL: SYCL :: Graph/RecordReplay/barrier_multi_graph.cpp (773 of 1823)
******************** TEST 'SYCL :: Graph/RecordReplay/barrier_multi_graph.cpp' FAILED ********************
Exit Code: -11

Command Output (stdout):
--
# RUN: at line 2
env ONEAPI_DEVICE_SELECTOR=level_zero:gpu  /__w/llvm/llvm/build-e2e/Graph/RecordReplay/Output/barrier_multi_graph.cpp.tmp.out
# executed command: env ONEAPI_DEVICE_SELECTOR=level_zero:gpu /__w/llvm/llvm/build-e2e/Graph/RecordReplay/Output/barrier_multi_graph.cpp.tmp.out
# note: command had no output on stdout or stderr
# RUN: at line 2
env env UR_LOADER_USE_LEVEL_ZERO_V2=1 ONEAPI_DEVICE_SELECTOR=level_zero:gpu  /__w/llvm/llvm/build-e2e/Graph/RecordReplay/Output/barrier_multi_graph.cpp.tmp.out
# executed command: env env UR_LOADER_USE_LEVEL_ZERO_V2=1 ONEAPI_DEVICE_SELECTOR=level_zero:gpu /__w/llvm/llvm/build-e2e/Graph/RecordReplay/Output/barrier_multi_graph.cpp.tmp.out
# note: command had no output on stdout or stderr
# error: command failed with exit status: -11

[pbalcer]

Might be worth checking if this is related to copy offload. UR_L0_V2_FORCE_DISABLE_COPY_OFFLOAD=1. Many of these tests involve data copies...