libstd.so removal breaks external proc_macro with -Cprefer-dynamic

[10ne1]

Hello,

I hope I'm explaining this correctly, please let me know if more info or clarifications are necessary.

In ChromeOS we are building external proc_macro crates with -Cprefer-dynamic, like this one. By this I mean non rustc_driver and outside the scope of the rustc_private feature, which modifies the linking behavior such that libstd.so is not dynalically linked if its already statically linked.

This results in binaries which dynamically link against libstd.so, however libstd.so got removed in 2cf1559, so now the binaries fail:

$ ldd /build/amd64-generic/.../spdm_proc_macros-6621385a8bfa4491
        linux-vdso.so.1 (0x00007ffe9954c000)
        libstd-024445ba29c0123d.so => not found
        libm.so.6 => /lib64/libm.so.6 (0x00007fc7c5c65000)
        libc.so.6 => /lib64/libc.so.6 (0x00007fc7c5a85000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fc7c5dda000)

$ readelf -d /build/amd64-generic/.../spdm_proc_macros-6621385a8bfa4491 | grep NEEDED
 0x0000000000000001 (NEEDED)             Shared library: [libstd-024445ba29c0123d.so]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [ld-linux-x86-64.so.2]

Perhaps commit 2cf1559 was too eager in removing libstd.so from the sysroot? AFAIU it incorrectly assumes libstd is always statically linked?

Code

If I remove the can_be_rustc_dynamic_dep check added in commit 2cf1559, the binaries start working again because libstd.so is not gated behind link_std_into_rustc_driver(target_compiler.host) anymore.

Any ideas how to proceed? I can send a PR undoing the can_be_rustc_dynamic_dep check, however maybe the check can somehow be improved? Or is it just an incorrect assumption that libstd is always statically linked?

Version it worked on

Worked fine in 1.82 or earlier, before commit 2cf1559 landed.

If I undo or revert that commit like I mention in the Code section above, binaries work fine again on all versions 1.83 and later.

@rustbot modify labels: +regression-from-stable-to-stable -regression-untriaged

[jieyouxu]

Maybe this is #132202 (cf. relnotes #133352)?

cc @Kobzol

[bjorn3]

You can add the output of rustc --print target-libdir to LD_LIBRARY_PATH. This is what cargo also does: https://github.com/rust-lang/cargo/blob/f6bebc3abbd9b273a457f77c31f8e93bc77f93fc/src/cargo/core/compiler/compilation.rs#L322

Perhaps commit 2cf1559 was too eager in removing libstd.so from the sysroot? AFAIU it incorrectly assumes libstd is always statically linked?

The copy of libstd.so in lib/ was only meant to be used by rustc. The one to be used by user code is the directory shown by rustc --print target-libdir.

[Kobzol]

Yeah, this is an instance of Hyrum's Law, this libstd.so shouldn't have ever been used by anything else than rustc, but since it was in the sysroot, some other libraries might have linked to it "by accident".

[10ne1]

Thank you @bjorn3 and @Kobzol for the clarifications

After the initial breakage, ChromeOS applied this workaround in the rust-host ebuild which installed the libstd.so from rustc --print target-libdir in place of the one which you removed (yes, it hardcoded the path and it worked):

doins "${obj}/lib64/rustlib/x86_64-pc-linux-gnu/lib/libstd-"*".so"

I'll replace that with adding rustc --print target-libdir to LD_LIBRARY_PATH.

If it's good enough for cargo, then it's good enough for us as well. 😂