introduce parallel salsa

[davidbarsky]

Initial pass; very much not ready. Putting it up for initial feedback; I'm still figuring out how to wrap &Db reasonably.

[netlify]

✅ Deploy Preview for salsa-rs canceled.

Name	Link
🔨 Latest commit	db7274e
🔍 Latest deploy log	https://app.netlify.com/sites/salsa-rs/deploys/6744aa4736364c00083d9667

[MichaReiser]

Nit: I kind of preferred the old salsa's ParallelDatabase trait rather than relying on Clone. It makes the contract more explicit.

[davidbarsky]

It's certainly option, but I'm not sure ParallelDatabase works well with when an end-user of Salsa only has &dyn HirDatabase and it should be safe and reasonable to use Rayon with queries that use &dyn HirDatabase. Like I said in my other comment, I'm going off of this document, but I'd be happy to change this design! I mostly care about making the example work with any database.

[nikomatsakis]

This is what I was debating about in those variations-- there might be a hybrid here. That said, I think that ALL databases will effectively be parallel databases under this plan (that is ~required to allow queries to use parallelism internally, since they are written against a dyn Database).

I guess we could still make a ParallelDatabase trait and have them work against a dyn ParallelDatabase, but I'm not convinced there's a use case for "non-parallel databases".

[davidbarsky]

At least in rust-analyzer's case, I can't really think of a situation where it wouldn't make sense to have every Database be a ParallelDatabase. I don't think we'd want to make everything parallel, but having the option to do so without extensive refactors is really valuable, in my view.

[codspeed-hq]

CodSpeed Performance Report

Merging #568 will not alter performance

_{Comparing davidbarsky/push-lqummkkxvkzn (db7274e) with master (32f8ee9)}

Summary

✅ 8 untouched benchmarks

[davidbarsky]

i've been doing a bunch of digging to figure out why this panics when used with tests/parallel/parallel_map.rs test. i initially saw that the underling Zalza::views was empty, so the test would naturally panic—there is nothing to downcast to! I've since added calls to tracked_fn in the test, and now, I'm seeing a downcast-able database:

[src/par_map.rs:16:5] db.zalsa().views() = DynDowncasts {
    vec: [
        DynDowncast(
            "dyn parallel_map::common::LogDatabase",
        ),
    ],

This makes sense! databases are discovered upon first use, but for par_map to work, I think the list of databases to cast to need to be reflexive: that is, they need to include themselves. I'll try making this change.

[nikomatsakis]

Ah, yeah, that's easy enough to fix. I was wondering if we'd encounter an issue where the correct view wasn't registered but I think we can fix that too if needed easily enough.

[nikomatsakis]

Ah, yeah, that's easy enough to fix. I was wondering if we'd encounter an issue where the correct view wasn't registered but I think we can fix that too if needed easily enough.

[davidbarsky]

The Miri errors appear to be coming from crossbeam-epoch, but I'm not sure how bad of an error that is.

[Veykril]

The Miri errors appear to be coming from crossbeam-epoch, but I'm not sure how bad of an error that is.

Judging from the issue tracker that crate seems to have a couple stacked borrows violations

[nikomatsakis]

Does miri have a way to add 'known failures'...

[nikomatsakis]

@davidbarsky I'd be ok moving the miri tests to "warn only" for the time being and then trying to investigate the use of crossbeam-epoch more closely.

[davidbarsky]

@davidbarsky I'd be ok moving the miri tests to "warn only" for the time being and then trying to investigate the use of crossbeam-epoch more closely.

Lemme see about doing that. I think I still need to port over a test or two over as well.

[davidbarsky]

Added a cancellation test.

[MichaReiser]

@davidbarsky what's the status of this PR? We now have a use case where the hacky hand rolled concurrency no longer works 😆

The requested fixed has been implemented, but Niko didn't have the time to re-stamp it.

[MichaReiser]

I tried to use the salsa::par_map function in astral-sh/ruff#14760 but I always received an empty list of accumulated values. I haven't yet looked into what goes wrong when using par_map but I suspect that some query dependencies aren't correctly recorded?