[DO NOT MERGE] `./x test rust-analyzer` #136779

jieyouxu · 2025-02-09T17:33:33Z

I somehow made ./x test rust-analyzer work on my machine¹, but at what cost?

Not intended for merge but only as a reference. If we do want to land this, I'll need to tidy this up.

Notes:

I abused a bunch of cargo features in-rust-tree. It probably doesn't need to be, and simply --cfg might work. I was trying to get the main rust-analyzer tests to build at all. Anything building is already a miracle.
I had to slap a bunch of the following to all the r-a crates to get the tests to build at all. I don't 100% understand why, but otherwise I get a whole ton of expected `rustc_lexer` to be available in rlib format build failures.

#![cfg_attr(feature = "in-rust-tree", feature(rustc_private))]
#[cfg(all(feature = "in-rust-tree", test))]
extern crate rustc_driver as _;

Skipped one config test that was fixed on r-a master but not synced here in r-l/r yet.

try-job: aarch64-gnu
try-job: x86_64-apple-1
try-job: aarch64-apple
try-job: i686-mingw
try-job: x86_64-mingw-1
try-job: i686-msvc-1
try-job: x86_64-msvc-1

x86_64-unknown-linux-gnu, haven't bothered trying this on msvc yet. ↩

jieyouxu · 2025-02-09T17:34:24Z

Sorry Albert, this PR is purely for reference purposes and is highly experimental.

src/tools/rust-analyzer/crates/base-db/src/lib.rs

jieyouxu · 2025-02-09T17:46:45Z

src/tools/rust-analyzer/crates/base-db/Cargo.toml

+[features]
+default = []
+in-rust-tree = []


Maybe could be --cfg instead but idk

jieyouxu · 2025-02-09T17:47:09Z

src/bootstrap/src/core/build_steps/test.rs

+        // RA's test suite tries to write to the source directory, that can't work in Rust CI.
+        cargo.env("SKIP_SLOW_TESTS", "1");


Note that this will mean not all r-a tests would be run via ./x test rust-analyzer

src/bootstrap/src/core/build_steps/test.rs

jieyouxu · 2025-02-09T17:55:48Z

src/bootstrap/src/core/build_steps/compile.rs

+    /// When using `download-rustc`, we need to use a new build of `std` for running unit tests of
+    /// std itself, but we need to use the downloaded copy of std for linking to rustdoc. Allow this
+    /// to be overridden by `builder.ensure` from other steps.


Unrelated comment reflow diffs, was annoying when reading these locally, I'll drop these formatting diffs if we want to merge the actual changes.

jieyouxu · 2025-02-09T17:56:43Z

src/bootstrap/src/core/build_steps/compile.rs

+        trace!(
+            ?force_recompile,
+            is_git_managed = builder.rust_info().is_managed_git_subrepository(),
+            download_rustc = builder.download_rustc(),
+            last_modified_commit =
+                ?builder.config.last_modified_commit(&["library"], "download-rustc", true),
+            "checking std force-recompile"
+        );


Unrelated tracing diff, was mostly to help local debugging, I'd like to upstream some of these tracing bits separately (unrelated to r-a tests).

jieyouxu · 2025-02-09T17:58:17Z

src/bootstrap/src/core/build_steps/compile.rs

-        if let Some(enzyme_install) = enzyme_install {
+        if builder.config.llvm_enzyme {
+            let enzyme_install = builder.ensure(llvm::Enzyme { target: build_compiler.host });
            let lib_ext = std::env::consts::DLL_EXTENSION;
            let src_lib = enzyme_install.join("build/Enzyme/libEnzyme-19").with_extension(lib_ext);
            let libdir = builder.sysroot_target_libdir(build_compiler, build_compiler.host);
            let target_libdir =
-                builder.sysroot_target_libdir(target_compiler, target_compiler.host);
+                builder.sysroot_target_libdir(output_compiler, output_compiler.host);
            let dst_lib = libdir.join("libEnzyme-19").with_extension(lib_ext);
            let target_dst_lib = target_libdir.join("libEnzyme-19").with_extension(lib_ext);
            builder.copy_link(&src_lib, &dst_lib);
            builder.copy_link(&src_lib, &target_dst_lib);
        }


Unrelated enzyme change

jieyouxu · 2025-02-09T18:00:41Z

src/bootstrap/src/core/build_steps/compile.rs

+            skip_all,
+            fields(target = ?self.target, compiler = ?self.compiler),
+        ),
+    )]
    fn run(self, builder: &Builder<'_>) -> u32 {
        let compiler = self.compiler;


On Rustc there's a comment on self.compiler that says

/// The **previous** compiler used to compile this compiler. pub compiler: Compiler,

I'm not 100% sure if that's actually true under all scenarios.

jieyouxu · 2025-02-09T18:02:49Z

src/bootstrap/src/core/build_steps/test.rs

+    #[cfg_attr(
+        feature = "tracing",
+        instrument(
+            level = "debug",
+            name = "RustAnalyzer::run",
+            skip_all,
+            fields(stage = self.stage, host = ?self.host),
+        ),
+    )]


Unrelated, but you can look at the tracing logs locally via

$ BOOTSTRAP_TRACING=bootstrap=trace ./x test rust-analyzer

(or =debug)

bors · 2025-02-09T19:34:05Z

☔ The latest upstream changes (presumably #136751) made this pull request unmergeable. Please resolve the merge conflicts.

jieyouxu · 2025-02-10T14:22:54Z

(Uh sorry Onur, I have no idea how I managed to click that button, did not mean to do that)

jieyouxu · 2025-02-10T15:59:12Z

src/bootstrap/src/core/build_steps/test.rs

+                // FIXME: may need a fix from https://github.com/rust-lang/rust-analyzer/pull/19124.
+                "--skip=config::tests::cargo_target_dir_subdir",


Other tests pass locally, except for the test fixed in rust-lang/rust-analyzer#19124 (which can be re-eval'd after the next subtree sync)

[DO NOT MERGE] `./x test rust-analyzer` I somehow made `./x test rust-analyzer` work on my machine[^machine], **but at what cost?** Not intended for merge but only as a reference. If we do want to land this, I'll need to tidy this up. Notes: - I abused a bunch of cargo features `in-rust-tree`. It probably doesn't need to be, and simply `--cfg` might work. I was trying to get the main rust-analyzer tests to build *at all*. Anything building is already a miracle. - I had to slap a bunch of the following to all the r-a crates to get the tests to build at all. I don't 100% understand why, but otherwise I get a whole ton of ``expected `rustc_lexer` to be available in rlib format`` build failures. ```rs #![cfg_attr(feature = "in-rust-tree", feature(rustc_private))] #[cfg(all(feature = "in-rust-tree", test))] extern crate rustc_driver as _; ``` - Skipped one config test that was fixed on r-a master but not synced here in r-l/r yet. [^machine]: `x86_64-unknown-linux-gnu`, haven't bothered trying this on msvc yet. try-job: aarch64-gnu try-job: x86_64-apple-1 try-job: aarch64-apple try-job: i686-mingw-1 try-job: x86_64-mingw-1 try-job: i686-msvc-1 try-job: x86_64-msvc-1

jieyouxu · 2025-02-10T16:13:32Z

@bors try

bors · 2025-02-10T16:14:43Z

⌛ Trying commit 5ae6a16 with merge 74d2e0b...

jieyouxu · 2025-02-14T11:16:47Z

@bors try

[DO NOT MERGE] `./x test rust-analyzer` I somehow made `./x test rust-analyzer` work on my machine[^machine], **but at what cost?** Not intended for merge but only as a reference. If we do want to land this, I'll need to tidy this up. Notes: - I abused a bunch of cargo features `in-rust-tree`. It probably doesn't need to be, and simply `--cfg` might work. I was trying to get the main rust-analyzer tests to build *at all*. Anything building is already a miracle. - I had to slap a bunch of the following to all the r-a crates to get the tests to build at all. I don't 100% understand why, but otherwise I get a whole ton of ``expected `rustc_lexer` to be available in rlib format`` build failures. ```rs #![cfg_attr(feature = "in-rust-tree", feature(rustc_private))] #[cfg(all(feature = "in-rust-tree", test))] extern crate rustc_driver as _; ``` - Skipped one config test that was fixed on r-a master but not synced here in r-l/r yet. [^machine]: `x86_64-unknown-linux-gnu`, haven't bothered trying this on msvc yet. try-job: aarch64-gnu try-job: x86_64-apple-1 try-job: aarch64-apple try-job: i686-mingw try-job: x86_64-mingw-1 try-job: i686-msvc-1 try-job: x86_64-msvc-1

bors · 2025-02-14T11:17:59Z

⌛ Trying commit b1738c9 with merge 7388bdea11d7e5065c7352cc6a1c9c6eb81a91e8...

bors · 2025-02-14T13:13:55Z

💔 Test failed - checks-actions

jieyouxu · 2025-02-14T14:10:50Z

@bors try

bors · 2025-02-14T14:12:10Z

⌛ Trying commit 5322e1c with merge 1105fba3228149721a170beb101c38f9fd83b35b...

[DO NOT MERGE] `./x test rust-analyzer` I somehow made `./x test rust-analyzer` work on my machine[^machine], **but at what cost?** Not intended for merge but only as a reference. If we do want to land this, I'll need to tidy this up. Notes: - I abused a bunch of cargo features `in-rust-tree`. It probably doesn't need to be, and simply `--cfg` might work. I was trying to get the main rust-analyzer tests to build *at all*. Anything building is already a miracle. - I had to slap a bunch of the following to all the r-a crates to get the tests to build at all. I don't 100% understand why, but otherwise I get a whole ton of ``expected `rustc_lexer` to be available in rlib format`` build failures. ```rs #![cfg_attr(feature = "in-rust-tree", feature(rustc_private))] #[cfg(all(feature = "in-rust-tree", test))] extern crate rustc_driver as _; ``` - Skipped one config test that was fixed on r-a master but not synced here in r-l/r yet. [^machine]: `x86_64-unknown-linux-gnu`, haven't bothered trying this on msvc yet. try-job: aarch64-gnu try-job: x86_64-apple-1 try-job: aarch64-apple try-job: i686-mingw try-job: x86_64-mingw-1 try-job: i686-msvc-1 try-job: x86_64-msvc-1

bors · 2025-02-14T16:21:52Z

💔 Test failed - checks-actions

jieyouxu · 2025-02-14T16:30:24Z

@bors try

[DO NOT MERGE] `./x test rust-analyzer` I somehow made `./x test rust-analyzer` work on my machine[^machine], **but at what cost?** Not intended for merge but only as a reference. If we do want to land this, I'll need to tidy this up. Notes: - I abused a bunch of cargo features `in-rust-tree`. It probably doesn't need to be, and simply `--cfg` might work. I was trying to get the main rust-analyzer tests to build *at all*. Anything building is already a miracle. - I had to slap a bunch of the following to all the r-a crates to get the tests to build at all. I don't 100% understand why, but otherwise I get a whole ton of ``expected `rustc_lexer` to be available in rlib format`` build failures. ```rs #![cfg_attr(feature = "in-rust-tree", feature(rustc_private))] #[cfg(all(feature = "in-rust-tree", test))] extern crate rustc_driver as _; ``` - Skipped one config test that was fixed on r-a master but not synced here in r-l/r yet. [^machine]: `x86_64-unknown-linux-gnu`, haven't bothered trying this on msvc yet. try-job: aarch64-gnu try-job: x86_64-apple-1 try-job: aarch64-apple try-job: i686-mingw try-job: x86_64-mingw-1 try-job: i686-msvc-1 try-job: x86_64-msvc-1

bors · 2025-02-14T16:31:36Z

⌛ Trying commit 34ff523 with merge 249d236...

rust-log-analyzer · 2025-02-14T18:05:09Z

A job failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

bors · 2025-02-14T18:05:47Z

💔 Test failed - checks-actions

jieyouxu · 2025-02-14T18:24:42Z

@bors try

bors · 2025-02-14T18:25:52Z

⌛ Trying commit 34ff523 with merge c91d3c9...

[DO NOT MERGE] `./x test rust-analyzer` I somehow made `./x test rust-analyzer` work on my machine[^machine], **but at what cost?** Not intended for merge but only as a reference. If we do want to land this, I'll need to tidy this up. Notes: - I abused a bunch of cargo features `in-rust-tree`. It probably doesn't need to be, and simply `--cfg` might work. I was trying to get the main rust-analyzer tests to build *at all*. Anything building is already a miracle. - I had to slap a bunch of the following to all the r-a crates to get the tests to build at all. I don't 100% understand why, but otherwise I get a whole ton of ``expected `rustc_lexer` to be available in rlib format`` build failures. ```rs #![cfg_attr(feature = "in-rust-tree", feature(rustc_private))] #[cfg(all(feature = "in-rust-tree", test))] extern crate rustc_driver as _; ``` - Skipped one config test that was fixed on r-a master but not synced here in r-l/r yet. [^machine]: `x86_64-unknown-linux-gnu`, haven't bothered trying this on msvc yet. try-job: aarch64-gnu try-job: x86_64-apple-1 try-job: aarch64-apple try-job: i686-mingw try-job: x86_64-mingw-1 try-job: i686-msvc-1 try-job: x86_64-msvc-1

rust-log-analyzer · 2025-02-14T19:35:08Z

A job failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

bors · 2025-02-14T19:35:46Z

💔 Test failed - checks-actions

jieyouxu · 2025-02-14T19:50:40Z

Please?
@bors try

[DO NOT MERGE] `./x test rust-analyzer` I somehow made `./x test rust-analyzer` work on my machine[^machine], **but at what cost?** Not intended for merge but only as a reference. If we do want to land this, I'll need to tidy this up. Notes: - I abused a bunch of cargo features `in-rust-tree`. It probably doesn't need to be, and simply `--cfg` might work. I was trying to get the main rust-analyzer tests to build *at all*. Anything building is already a miracle. - I had to slap a bunch of the following to all the r-a crates to get the tests to build at all. I don't 100% understand why, but otherwise I get a whole ton of ``expected `rustc_lexer` to be available in rlib format`` build failures. ```rs #![cfg_attr(feature = "in-rust-tree", feature(rustc_private))] #[cfg(all(feature = "in-rust-tree", test))] extern crate rustc_driver as _; ``` - Skipped one config test that was fixed on r-a master but not synced here in r-l/r yet. [^machine]: `x86_64-unknown-linux-gnu`, haven't bothered trying this on msvc yet. try-job: aarch64-gnu try-job: x86_64-apple-1 try-job: aarch64-apple try-job: i686-mingw try-job: x86_64-mingw-1 try-job: i686-msvc-1 try-job: x86_64-msvc-1

bors · 2025-02-14T19:51:53Z

⌛ Trying commit 34ff523 with merge 91f9263...

rust-log-analyzer · 2025-02-14T19:52:45Z

A job failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

/dev/sdb1       147G  4.1G  136G   3% /mnt
tmpfs           1.6G  4.0K  1.6G   1% /run/user/1001
================================================================================

Failed to stop mysql.service: Unit mysql.service not loaded.
##[error]Process completed with exit code 5.

bors · 2025-02-14T19:53:17Z

💔 Test failed - checks-actions

rustbot assigned albertlarsan68 Feb 9, 2025

This comment was marked as off-topic.

Sign in to view

rustbot added A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Feb 9, 2025

jieyouxu unassigned albertlarsan68 Feb 9, 2025

jieyouxu added S-experimental Status: Ongoing experiment that does not require reviewing and won't be merged in its current state. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Feb 9, 2025

jieyouxu added the T-rust-analyzer Relevant to the rust-analyzer team, which will review and decide on the PR/issue. label Feb 9, 2025

jieyouxu commented Feb 9, 2025

View reviewed changes

jieyouxu force-pushed the rahhh branch from 3b7ca5e to 07646dd Compare February 9, 2025 18:12

bors added the S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. label Feb 9, 2025

jieyouxu requested a review from onur-ozkan February 10, 2025 14:21

jieyouxu removed the request for review from onur-ozkan February 10, 2025 14:33

jieyouxu force-pushed the rahhh branch from 07646dd to 5ae6a16 Compare February 10, 2025 15:43

jieyouxu commented Feb 10, 2025

View reviewed changes

This comment was marked as resolved.

Sign in to view

This comment was marked as off-topic.

Sign in to view

[DO NOT MERGE] bootstrap: skip r-a xtask tests

78fd6b5

jieyouxu force-pushed the rahhh branch from 2e4ea0b to b1738c9 Compare February 14, 2025 11:16

This comment has been minimized.

Sign in to view

jieyouxu force-pushed the rahhh branch from b1738c9 to 5322e1c Compare February 14, 2025 14:05

This comment has been minimized.

Sign in to view

[DO NOT MERGE] bootstrap: skip some tests that fail on specific jobs

34ff523

jieyouxu force-pushed the rahhh branch from 5322e1c to 34ff523 Compare February 14, 2025 16:30

		// RA's test suite tries to write to the source directory, that can't work in Rust CI.
		cargo.env("SKIP_SLOW_TESTS", "1");

		// FIXME: may need a fix from https://github.com/rust-lang/rust-analyzer/pull/19124.
		"--skip=config::tests::cargo_target_dir_subdir",

[DO NOT MERGE] ./x test rust-analyzer #136779

Are you sure you want to change the base?

[DO NOT MERGE] ./x test rust-analyzer #136779

Conversation

jieyouxu commented Feb 9, 2025 • edited Loading

Footnotes

This comment was marked as off-topic.

jieyouxu commented Feb 9, 2025

jieyouxu Feb 9, 2025

Choose a reason for hiding this comment

jieyouxu Feb 9, 2025

Choose a reason for hiding this comment

jieyouxu Feb 9, 2025 • edited Loading

Choose a reason for hiding this comment

jieyouxu Feb 9, 2025

Choose a reason for hiding this comment

jieyouxu Feb 9, 2025

Choose a reason for hiding this comment

jieyouxu Feb 9, 2025

Choose a reason for hiding this comment

jieyouxu Feb 9, 2025

Choose a reason for hiding this comment

bors commented Feb 9, 2025

jieyouxu commented Feb 10, 2025

jieyouxu Feb 10, 2025

Choose a reason for hiding this comment

This comment was marked as resolved.

This comment was marked as resolved.

This comment was marked as off-topic.

jieyouxu commented Feb 10, 2025

bors commented Feb 10, 2025

jieyouxu commented Feb 14, 2025

bors commented Feb 14, 2025

This comment has been minimized.

bors commented Feb 14, 2025

jieyouxu commented Feb 14, 2025

bors commented Feb 14, 2025

This comment has been minimized.

bors commented Feb 14, 2025

jieyouxu commented Feb 14, 2025

bors commented Feb 14, 2025

rust-log-analyzer commented Feb 14, 2025

bors commented Feb 14, 2025

jieyouxu commented Feb 14, 2025

bors commented Feb 14, 2025

rust-log-analyzer commented Feb 14, 2025

bors commented Feb 14, 2025

jieyouxu commented Feb 14, 2025

bors commented Feb 14, 2025

rust-log-analyzer commented Feb 14, 2025

bors commented Feb 14, 2025

[DO NOT MERGE] `./x test rust-analyzer` #136779

[DO NOT MERGE] `./x test rust-analyzer` #136779

jieyouxu commented Feb 9, 2025 •

edited

Loading

jieyouxu Feb 9, 2025 •

edited

Loading