Support raw-dylib link kind on ELF #135695

Noratrieb · 2025-01-18T17:33:17Z

raw-dylib is a link kind that allows rustc to link against a library without having any library files present.
This currently only exists on Windows. rustc will take all the symbols from raw-dylib link blocks and put them in an import library, where they can then be resolved by the linker.

While import libraries don't exist on ELF, it would still be convenient to have this same functionality. Not having the libraries present at build-time can be convenient for several reasons, especially cross-compilation. With raw-dylib, code linking against a library can be cross-compiled without needing to have these libraries available on the build machine. If the libc crate makes use of this, it would allow cross-compilation without having any libc available on the build machine. This is not yet possible with this implementation, at least against libc's like glibc that use symbol versioning. The raw-dylib kind could be extended with support for symbol versioning in the future.

This implementation is very experimental and I have not tested it very well. I have tested it for a toy example and the lz4-sys crate, where it was able to successfully link a binary despite not having a corresponding library at build-time.

I was inspired by Björn's comments in https://internals.rust-lang.org/t/bundle-zig-cc-in-rustup-by-default/22096/27
Tracking issue: #135694

r? bjorn3

try-job: aarch64-apple
try-job: x86_64-msvc-1
try-job: x86_64-msvc-2
try-job: test-various

rustbot · 2025-01-18T17:33:27Z

These commits modify compiler targets.
(See the Target Tier Policy.)

Some changes occurred in src/tools/compiletest

cc @jieyouxu

compiler/rustc_codegen_ssa/src/back/link.rs

compiler/rustc_target/src/spec/mod.rs

tests/ui/linkage-attr/raw-dylib/elf/basic-symbol-links-with.rs

compiler/rustc_codegen_ssa/src/back/link.rs

jieyouxu

compiletest changes LGTM, would be great if you could add the two new directives to the directives listing in r-d-g

compiler/rustc_target/src/spec/mod.rs

src/tools/compiletest/src/directive-list.rs

Noratrieb · 2025-02-13T06:45:28Z

oh I should probably
@bors rollup=iffy
because while all 3 platforms are tested now i still don't trust it

Support raw-dylib link kind on ELF raw-dylib is a link kind that allows rustc to link against a library without having any library files present. This currently only exists on Windows. rustc will take all the symbols from raw-dylib link blocks and put them in an import library, where they can then be resolved by the linker. While import libraries don't exist on ELF, it would still be convenient to have this same functionality. Not having the libraries present at build-time can be convenient for several reasons, especially cross-compilation. With raw-dylib, code linking against a library can be cross-compiled without needing to have these libraries available on the build machine. If the libc crate makes use of this, it would allow cross-compilation without having any libc available on the build machine. This is not yet possible with this implementation, at least against libc's like glibc that use symbol versioning. The raw-dylib kind could be extended with support for symbol versioning in the future. This implementation is very experimental and I have not tested it very well. I have tested it for a toy example and the lz4-sys crate, where it was able to successfully link a binary despite not having a corresponding library at build-time. I was inspired by Björn's comments in https://internals.rust-lang.org/t/bundle-zig-cc-in-rustup-by-default/22096/27 Tracking issue: rust-lang#135694 r? bjorn3 try-job: aarch64-apple try-job: x86_64-msvc-1 try-job: x86_64-msvc-2

bors · 2025-02-13T14:39:41Z

⌛ Testing commit cc8fea2 with merge 01d3937...

bors · 2025-02-13T15:26:23Z

💔 Test failed - checks-actions

Noratrieb · 2025-02-13T18:43:47Z

Hello, WebAssembly! Nice to see you here my friend.
@bors try

bors · 2025-02-13T18:45:00Z

⌛ Trying commit ea6cbd2 with merge f95b8da...

Support raw-dylib link kind on ELF raw-dylib is a link kind that allows rustc to link against a library without having any library files present. This currently only exists on Windows. rustc will take all the symbols from raw-dylib link blocks and put them in an import library, where they can then be resolved by the linker. While import libraries don't exist on ELF, it would still be convenient to have this same functionality. Not having the libraries present at build-time can be convenient for several reasons, especially cross-compilation. With raw-dylib, code linking against a library can be cross-compiled without needing to have these libraries available on the build machine. If the libc crate makes use of this, it would allow cross-compilation without having any libc available on the build machine. This is not yet possible with this implementation, at least against libc's like glibc that use symbol versioning. The raw-dylib kind could be extended with support for symbol versioning in the future. This implementation is very experimental and I have not tested it very well. I have tested it for a toy example and the lz4-sys crate, where it was able to successfully link a binary despite not having a corresponding library at build-time. I was inspired by Björn's comments in https://internals.rust-lang.org/t/bundle-zig-cc-in-rustup-by-default/22096/27 Tracking issue: rust-lang#135694 r? bjorn3 try-job: aarch64-apple try-job: x86_64-msvc-1 try-job: x86_64-msvc-2 try-job: test-various

bors · 2025-02-13T19:39:18Z

💔 Test failed - checks-actions

Noratrieb · 2025-02-14T20:06:49Z

yes, wasm is wasm, sure.
@bors try

Support raw-dylib link kind on ELF raw-dylib is a link kind that allows rustc to link against a library without having any library files present. This currently only exists on Windows. rustc will take all the symbols from raw-dylib link blocks and put them in an import library, where they can then be resolved by the linker. While import libraries don't exist on ELF, it would still be convenient to have this same functionality. Not having the libraries present at build-time can be convenient for several reasons, especially cross-compilation. With raw-dylib, code linking against a library can be cross-compiled without needing to have these libraries available on the build machine. If the libc crate makes use of this, it would allow cross-compilation without having any libc available on the build machine. This is not yet possible with this implementation, at least against libc's like glibc that use symbol versioning. The raw-dylib kind could be extended with support for symbol versioning in the future. This implementation is very experimental and I have not tested it very well. I have tested it for a toy example and the lz4-sys crate, where it was able to successfully link a binary despite not having a corresponding library at build-time. I was inspired by Björn's comments in https://internals.rust-lang.org/t/bundle-zig-cc-in-rustup-by-default/22096/27 Tracking issue: rust-lang#135694 r? bjorn3 try-job: aarch64-apple try-job: x86_64-msvc-1 try-job: x86_64-msvc-2 try-job: test-various

bors · 2025-02-14T20:08:02Z

⌛ Trying commit 2846589 with merge c4642e7...

bors · 2025-02-14T21:27:41Z

💔 Test failed - checks-actions

tests/run-make/raw-dylib-elf-verbatim-absolute/rmake.rs

raw-dylib is a link kind that allows rustc to link against a library without having any library files present. This currently only exists on Windows. rustc will take all the symbols from raw-dylib link blocks and put them in an import library, where they can then be resolved by the linker. While import libraries don't exist on ELF, it would still be convenient to have this same functionality. Not having the libraries present at build-time can be convenient for several reasons, especially cross-compilation. With raw-dylib, code linking against a library can be cross-compiled without needing to have these libraries available on the build machine. If the libc crate makes use of this, it would allow cross-compilation without having any libc available on the build machine. This is not yet possible with this implementation, at least against libc's like glibc that use symbol versioning. The raw-dylib kind could be extended with support for symbol versioning in the future. This implementation is very experimental and I have not tested it very well. I have tested it for a toy example and the lz4-sys crate, where it was able to successfully link a binary despite not having a corresponding library at build-time.

Noratrieb · 2025-02-15T16:48:09Z

@bors try

bors · 2025-02-15T16:49:21Z

⌛ Trying commit a815898 with merge cc05381ad31b5da4d81a26cb0728f72cfa869085...

Support raw-dylib link kind on ELF raw-dylib is a link kind that allows rustc to link against a library without having any library files present. This currently only exists on Windows. rustc will take all the symbols from raw-dylib link blocks and put them in an import library, where they can then be resolved by the linker. While import libraries don't exist on ELF, it would still be convenient to have this same functionality. Not having the libraries present at build-time can be convenient for several reasons, especially cross-compilation. With raw-dylib, code linking against a library can be cross-compiled without needing to have these libraries available on the build machine. If the libc crate makes use of this, it would allow cross-compilation without having any libc available on the build machine. This is not yet possible with this implementation, at least against libc's like glibc that use symbol versioning. The raw-dylib kind could be extended with support for symbol versioning in the future. This implementation is very experimental and I have not tested it very well. I have tested it for a toy example and the lz4-sys crate, where it was able to successfully link a binary despite not having a corresponding library at build-time. I was inspired by Björn's comments in https://internals.rust-lang.org/t/bundle-zig-cc-in-rustup-by-default/22096/27 Tracking issue: rust-lang#135694 r? bjorn3 try-job: aarch64-apple try-job: x86_64-msvc-1 try-job: x86_64-msvc-2 try-job: test-various

bors · 2025-02-15T18:03:31Z

💔 Test failed - checks-actions

rust-log-analyzer · 2025-02-15T18:05:40Z

The job test-various failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

file:.git/config remote.origin.url=https://github.com/rust-lang-ci/rust
file:.git/config remote.origin.fetch=+refs/heads/*:refs/remotes/origin/*
file:.git/config gc.auto=0
file:.git/config http.https://github.com/.extraheader=AUTHORIZATION: basic ***
file:.git/config branch.try.remote=origin
file:.git/config branch.try.merge=refs/heads/try
file:.git/config remote.upstream.fetch=+refs/heads/*:refs/remotes/upstream/*
file:.git/config submodule.library/backtrace.active=true
file:.git/config submodule.library/backtrace.url=https://github.com/rust-lang/backtrace-rs.git
file:.git/config submodule.library/stdarch.active=true
---
---- [run-make] tests/run-make/raw-dylib-elf-verbatim stdout ----

error: rmake recipe failed to complete
status: exit status: 1
command: cd "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/raw-dylib-elf-verbatim/rmake_out" && env -u RUSTFLAGS AR="ar" BUILD_ROOT="/checkout/obj/build/x86_64-unknown-linux-gnu" CARGO="/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-tools-bin/cargo" CC="x86_64-linux-musl-gcc" CC_DEFAULT_FLAGS="-ffunction-sections -fdata-sections -fPIC -m64 -static" CXX="x86_64-linux-musl-g++" CXX_DEFAULT_FLAGS="-ffunction-sections -fdata-sections -fPIC -m64 -static" HOST_RPATH_DIR="/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" LD_LIBRARY_PATH="/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-bootstrap-tools/x86_64-unknown-linux-gnu/release/deps:/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/lib:/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-tools-bin:/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" LD_LIB_PATH_ENVVAR="LD_LIBRARY_PATH" LLVM_BIN_DIR="/checkout/obj/build/x86_64-unknown-linux-gnu/ci-llvm/bin" LLVM_COMPONENTS="aarch64 aarch64asmparser aarch64codegen aarch64desc aarch64disassembler aarch64info aarch64utils aggressiveinstcombine all all-targets amdgpu amdgpuasmparser amdgpucodegen amdgpudesc amdgpudisassembler amdgpuinfo amdgputargetmca amdgpuutils analysis arm armasmparser armcodegen armdesc armdisassembler arminfo armutils asmparser asmprinter avr avrasmparser avrcodegen avrdesc avrdisassembler avrinfo binaryformat bitreader bitstreamreader bitwriter bpf bpfasmparser bpfcodegen bpfdesc bpfdisassembler bpfinfo cfguard codegen codegendata codegentypes core coroutines coverage csky cskyasmparser cskycodegen cskydesc cskydisassembler cskyinfo debuginfobtf debuginfocodeview debuginfodwarf debuginfogsym debuginfologicalview debuginfomsf debuginfopdb demangle dlltooldriver dwarflinker dwarflinkerclassic dwarflinkerparallel dwp engine executionengine extensions filecheck frontenddriver frontendhlsl frontendoffloading frontendopenacc frontendopenmp fuzzercli fuzzmutate globalisel hexagon hexagonasmparser hexagoncodegen hexagondesc hexagondisassembler hexagoninfo hipstdpar instcombine instrumentation interfacestub interpreter ipo irprinter irreader jitlink libdriver lineeditor linker loongarch loongarchasmparser loongarchcodegen loongarchdesc loongarchdisassembler loongarchinfo lto m68k m68kasmparser m68kcodegen m68kdesc m68kdisassembler m68kinfo mc mca mcdisassembler mcjit mcparser mips mipsasmparser mipscodegen mipsdesc mipsdisassembler mipsinfo mirparser msp430 msp430asmparser msp430codegen msp430desc msp430disassembler msp430info native nativecodegen nvptx nvptxcodegen nvptxdesc nvptxinfo objcarcopts objcopy object objectyaml option orcdebugging orcjit orcshared orctargetprocess passes powerpc powerpcasmparser powerpccodegen powerpcdesc powerpcdisassembler powerpcinfo profiledata remarks riscv riscvasmparser riscvcodegen riscvdesc riscvdisassembler riscvinfo riscvtargetmca runtimedyld sandboxir scalaropts selectiondag sparc sparcasmparser sparccodegen sparcdesc sparcdisassembler sparcinfo support symbolize systemz systemzasmparser systemzcodegen systemzdesc systemzdisassembler systemzinfo tablegen target targetparser textapi textapibinaryreader transformutils vectorize webassembly webassemblyasmparser webassemblycodegen webassemblydesc webassemblydisassembler webassemblyinfo webassemblyutils windowsdriver windowsmanifest x86 x86asmparser x86codegen x86desc x86disassembler x86info x86targetmca xray xtensa xtensaasmparser xtensacodegen xtensadesc xtensadisassembler xtensainfo" LLVM_FILECHECK="/checkout/obj/build/x86_64-unknown-linux-gnu/ci-llvm/bin/FileCheck" NODE="/node/bin/node" PYTHON="/usr/bin/python3" RUSTC="/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" RUSTC_LINKER="x86_64-linux-musl-gcc" RUSTDOC="/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustdoc" SOURCE_ROOT="/checkout" TARGET="x86_64-unknown-linux-musl" TARGET_RPATH_DIR="/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-musl/lib" TARGET_RPATH_ENV="/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/raw-dylib-elf-verbatim/rmake_out:/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-bootstrap-tools/x86_64-unknown-linux-gnu/release/deps:/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/lib" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/raw-dylib-elf-verbatim/rmake"
--- stderr -------------------------------
command failed at line 24
command failed at line 24
Command { cmd: LC_ALL="C" LD_LIBRARY_PATH="/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/raw-dylib-elf-verbatim/rmake_out:/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/raw-dylib-elf-verbatim/rmake_out:/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-bootstrap-tools/x86_64-unknown-linux-gnu/release/deps:/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/lib:/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-bootstrap-tools/x86_64-unknown-linux-gnu/release/deps:/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/lib:/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-tools-bin:/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/raw-dylib-elf-verbatim/rmake_out/main", stdin_buf: None, stdin: None, stdout: None, stderr: None, drop_bomb: DropBomb { command: "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/raw-dylib-elf-verbatim/rmake_out/main", defused: true, armed_location: Location { file: "/checkout/tests/run-make/raw-dylib-elf-verbatim/rmake.rs", line: 24, col: 18 } }, already_executed: true }
output status: `exit status: 127`
=== STDOUT ===


=== STDERR ===
=== STDERR ===
/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/raw-dylib-elf-verbatim/rmake_out/main: error while loading shared libraries: /lib/x86_64-linux-gnu/libc.so: invalid ELF header



failures:

rustbot assigned bjorn3 Jan 18, 2025

Noratrieb mentioned this pull request Jan 18, 2025

Tracking Issue for raw-dylib on ELF #135694

Open

5 tasks

This comment has been minimized.

Sign in to view