Wake up threads more aggressively if there is known work to do

[james7132]

Upon investigating bevyengine/bevy#6941, we've found that thread wakeup times have been a significant bottleneck when spinning up large batches of tasks, like those mentioned in #91.

Would it be possible to more aggressively wake up threads (i.e. 2+ at a time), if and only if we know there is sufficient work for all of them? Whenever a thread is stealing work from another, it has an approximation of how much remaining work is in both threads, so it may be reasonable to weigh the potential contention against the cost of sequentially context switches as threads wake up.

[notgull]

async-executor is generally intended to be a simple reference executor that shows how to use async-task. While I welcome performance optimizations, in this crate I want to avoid chasing diminishing returns as it would significantly complicate the implementation.

I've been working on an alternative executor implementation that implements some of the crazier strategies, however. If Bevy wants this I can prioritize the work

[james7132]

I'd be interested in this alternative executor. I'm currently experimenting with using async-task and rayon together to try to improve task start latency and overall compute throughput, while also supporting async tasks on the same thread pool (see bevyengine/bevy#11995). It does seem like async_executor's idle CPU usage is lower, owing to the use of parking, and the contention on the executor as a whole seems higher with rayon. I'm wondering if there's potentially a sweet spot somewhere in between.

[notgull]

I've had more time to think about it, and I've changed my mind. I'm willing to do this work for async-executor to make it faster, as many projects use it an a rising tide raises all boats. So if we want to work on integrating new features into this executor I would support it.