Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Windows replace / by \ in absolute path for remote build #25443

Open
sacha-123 opened this issue Mar 4, 2025 · 10 comments
Open

Windows replace / by \ in absolute path for remote build #25443

sacha-123 opened this issue Mar 4, 2025 · 10 comments
Labels
P2 We'll consider working on this in future. (Assignee optional) team-Remote-Exec Issues and PRs for the Execution (Remote) team type: bug

Comments

@sacha-123
Copy link

sacha-123 commented Mar 4, 2025

Description of the bug:

I have a buildfarm Worker running on Linux. For a licensed C++ compiler that is installed at the /usr folder. The bug appear with any absolute path, so I will put /usr/bin/gcc examples bellow, which does the same thing. I have to use absolute path here cause I can't really move the compiler around since it is licenced.

We can interact with this worker from linux and it works fine: The worker receive the correct cmd line with the right /usr/bin/gcc

The issue is that most of the teams is on windows, so we need to interact with that worker from Windows. The cmd line used is the same. And all config file are the same too. But in that case, the worker receive backslash in the command line for the tool path (other cmd args doesn't seems to be affected). The worker get \usr\bin\gcc which make linux mad and I receive the error: No such file or directory from the execvp of the worker.

After speaking with the buildfarm teams on slack, they told me to try to set a --host_platform to a linux one (on the Windows cmd line), I have tried this:

platform(
    name = "remote",
    constraint_values = [
        "@platforms//os:linux",
        "@platforms//cpu:x86_64",
    ],
)

And it didn't change anything, the worker still receive backslash for the tool path.

Just as a precision, I do not have any backslash in the config file for my compiler repository rule, all of them are /.
My config is almost the same as the ubuntu_cc_toolchain_config from the buidbuddy toolchain: http://github.com/buildbuddy-io/buildbuddy-toolchain

Thanks for your help

Which category does this issue belong to?

Remote Execution

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

No response

Which operating system are you running Bazel on?

MSYS_NT-10.0-26100 -- Windows 11

What is the output of bazel info release?

I am not using bazel but bazelisk is currently set to USE_BAZEL_VERSION=8.0.0, without this line I get: release 8.1.1

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse HEAD ?

Sorry this is a private project on a local network

If this is a regression, please try to identify the Bazel commit where the bug was introduced with bazelisk --bisect.

This is not a regression.

Have you found anything relevant by searching the web?

No, I have contacted the buildfarm teams on slack. And they have redirected me to setup an issue, cause that doesn't seems to be an issue form buildfarm

Any other information, logs, or outputs that you want to share?

No response

@fmeum
Copy link
Collaborator

fmeum commented Mar 4, 2025

Could you check whether #22218 fixes the issue for you?

@sacha-123
Copy link
Author

Yes, can I set that directly in the bazeliskrc ?

@sgowroji sgowroji added the team-Remote-Exec Issues and PRs for the Execution (Remote) team label Mar 4, 2025
@fmeum
Copy link
Collaborator

fmeum commented Mar 4, 2025

It hasn't been merged yet, so it's unfortunately not quite that easy. You can check out the PR locally and then build a standalone Bazel binary with bazel build -c opt //src:bazel.

@tjgq tjgq added P2 We'll consider working on this in future. (Assignee optional) and removed untriaged labels Mar 4, 2025
@sacha-123
Copy link
Author

sacha-123 commented Mar 4, 2025

PS C:\Users\sbellier\Local\bazel_fmeum> bazelisk build -c opt //src:bazel
2025/03/04 11:28:14 Downloading https://releases.bazel.build/7.2.0/release/bazel-7.2.0-windows-x86_64.exe...
Downloading: 63 MB out of 63 MB (100%) 
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
DEBUG: C:/users/sbellier/_bazel_sbellier/umeaak7n/external/rules_jvm_external~/coursier.bzl:721:18: Found duplicate artifact versions
    com.google.code.gson:gson has multiple versions 2.10.1, 2.8.9
    com.google.errorprone:error_prone_annotations has multiple versions 2.23.0, 2.3.2
    com.google.guava:guava has multiple versions 33.0.0-jre, 31.1-jre
    com.google.guava:guava-testlib has multiple versions 33.0.0-jre, 31.1-jre
    com.google.truth:truth has multiple versions 1.4.0, 1.1.2
    org.mockito:mockito-core has multiple versions 5.4.0, 4.3.1
Please remove duplicate artifacts from the artifact list so you do not get unexpected artifact versions
INFO: Repository bazel_tools~cc_configure_extension~local_config_cc instantiated at:
  <builtin>: in <toplevel>
Repository rule cc_autoconf defined at:
  C:/users/sbellier/_bazel_sbellier/umeaak7n/external/bazel_tools/tools/cpp/cc_configure.bzl:109:30: in <toplevel>
ERROR: An error occurred during the fetch of repository 'bazel_tools~cc_configure_extension~local_config_cc':
   Traceback (most recent call last):
        File "C:/users/sbellier/_bazel_sbellier/umeaak7n/external/bazel_tools/tools/cpp/cc_configure.bzl", line 89, column 36, in cc_autoconf_impl
                configure_windows_toolchain(repository_ctx)
        File "C:/users/sbellier/_bazel_sbellier/umeaak7n/external/bazel_tools/tools/cpp/windows_cc_configure.bzl", line 830, column 35, in configure_windows_toolchain
                msvc_vars_x64 = _get_msvc_vars(repository_ctx, paths, "x64")
        File "C:/users/sbellier/_bazel_sbellier/umeaak7n/external/bazel_tools/tools/cpp/windows_cc_configure.bzl", line 667, column 32, in _get_msvc_vars
                env = setup_vc_env_vars(repository_ctx, vc_path)
        File "C:/users/sbellier/_bazel_sbellier/umeaak7n/external/bazel_tools/tools/cpp/windows_cc_configure.bzl", line 383, column 24, in setup_vc_env_vars
                _check_env_vars(env_map, cmd, expected = envvars)
        File "C:/users/sbellier/_bazel_sbellier/umeaak7n/external/bazel_tools/tools/cpp/windows_cc_configure.bzl", line 389, column 32, in _check_env_vars
                auto_configure_fail(
        File "C:/users/sbellier/_bazel_sbellier/umeaak7n/external/bazel_tools/tools/cpp/lib_cc_configure.bzl", line 112, column 9, in auto_configure_fail
                fail("\n%sAuto-Configuration Error:%s %s\n" % (red, no_color, msg))
Error in fail:
Auto-Configuration Error: Setting up VC environment variables failed, WINDOWSSDKDIR is not set by the following command:
    "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\Build\VCVARSALL.BAT" amd64  -vcvars_ver=14.41.34120
ERROR: no such package '@@bazel_tools~cc_configure_extension~local_config_cc//':
Auto-Configuration Error: Setting up VC environment variables failed, WINDOWSSDKDIR is not set by the following command:
    "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\Build\VCVARSALL.BAT" amd64  -vcvars_ver=14.41.34120
ERROR: C:/users/sbellier/local/bazel_fmeum/tools/test/BUILD:46:10: //tools/test:xml depends on @@bazel_tools~cc_configure_extension~local_config_cc//:cc-compiler-x64_windows in repository @@bazel_tools~cc_configure_extension~local_config_cc which failed to fetch. no such package '@@bazel_tools~cc_configure_extension~local_config_cc//':
Auto-Configuration Error: Setting up VC environment variables failed, WINDOWSSDKDIR is not set by the following command:
    "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\Build\VCVARSALL.BAT" amd64  -vcvars_ver=14.41.34120
ERROR: Analysis of target '//src:bazel' failed; build aborted: Analysis failed
INFO: Elapsed time: 23.832s, Critical Path: 0.07s
INFO: 1 process: 1 internal.
ERROR: Build did NOT complete successfully
FAILED:
    Fetching repository @@rules_java~~toolchains~remotejdk21_win; starting 8s
    Fetching repository @@abseil-cpp~; starting 8s
    Fetching repository @@grpc~; starting 8s
    Fetching repository @@protobuf~; starting 8s
    Fetching https://mirror.bazel.build/cdn.azul.com/zulu/bin/zulu21.32.17-ca-jdk21.0.2-win_x64.zip; 188.5 MiB (94.2%) 7s
    Fetching C:/users/sbellier/_bazel_sbellier/umeaak7n/external/abseil-cpp~; Extracting abseil-cpp-20240116.2.tar.gz 6s
    Fetching C:/users/sbellier/_bazel_sbellier/umeaak7n/external/grpc~; Extracting v1.48.1.tar.gz 6s
    Fetching C:/users/sbellier/_bazel_sbellier/umeaak7n/external/protobuf~; Extracting protobuf-all-21.7.zip 5s

From that issue #13261, I do have in fact the Windows SDK installed from Build Tools. Maybe I need to install the Windows SDK from the VS community ? (but I need admin right for that, so I can't try that now... :,( )

@sacha-123
Copy link
Author

Also is your fork a fork form Bazel 8 ? (or at least 7.4.0 ?) cause we are using inject_repo

@fmeum
Copy link
Collaborator

fmeum commented Mar 4, 2025

My PR is based on current master, which has inject_repo but may contain incompatible changes compared to Bazel 8. You probably do need VS Community + the SDK installed through it.

@sacha-123
Copy link
Author

I got the exact same issue, but this time with all SDK from Visual Studio installed through Build Tools and VS Community...

Image

I do have the right value for BAZEL_VS / BAZEL_VS and my bash is from Msys not from WSL
I don't know what I am doing wrong here to compile it

@fmeum
Copy link
Collaborator

fmeum commented Mar 4, 2025

@meteorcloudy Do you happen to see what the issue with the setup could be?

@meteorcloudy
Copy link
Member

No idea, does manually run "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\Build\VCVARSALL.BAT" amd64 -vcvars_ver=14.41.34120 set the env var correctly?

@sacha-123
Copy link
Author

sacha-123 commented Mar 5, 2025

Sorry, I had forgotten to reply :)

No it doesn't seems to have set any env variable.

Image

Have any of you made this work with the 2022 version of build tools or community ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P2 We'll consider working on this in future. (Assignee optional) team-Remote-Exec Issues and PRs for the Execution (Remote) team type: bug
Projects
None yet
Development

No branches or pull requests

7 participants