Co-authored-by: krynju <[email protected]>

…-topmost-bug

Add missing check to walk_data

…-no-capture

signature: Don't capture input arguments

Implement WeakChunk like WeakThunk
Swap Chunk for WeakChunk in eager thunk submission

chunks: Allow weak Chunk references in Thunk args

…ux-1.0

DaggerWebDash: Add Mux 1.x to compat

…ache-bug

Fix incorrect assertion in schedule!

…t-invokelatest

checkpoint: Use at-invokelatest

…broadcast

at-spawn: Add support for broadcasting

…z-docs

Update scheduler visualization docs

Adds ProcessorTypeScope(T), which matches processors that are a subtype
of T. In the process, also expands the scope system to support
lazily-evaluated scoping behavior, such as doing a subtype check or
checking for `default_enabled`, via "taints".

Also changes behavior such that proclist and single override scope when
set, to prevent issues with mixing proclist/single with scope.

…-type-scope

Add ProcessorTypeScope, deprecate proclist and single

…ntations

The worker scheduler would previously assume that it was fine to
schedule infinite amounts of work onto the same processor at once, which
is only efficient when tasks do lots of `yield`ing. Because most tasks
do not actually exhibit low occupancy, we want to teach at least the
worker scheduler to limit its eagerness when executing high-occupancy
tasks.

This commit teaches `@spawn` and the worker scheduler about a new
`occupancy` task option, which (on the user side) is a value between 0
and 1 which approximates how fully the task occupies the processor. If
the occupancy is 0.2, then 5 such tasks can execute concurrently and
fully occupy the processor.

Processors now operate primarily from a single controlling task per
processor, and work is executed in a lowest-occupancy-first manner to
attempt to maximize throughput.

With processors using occupancy estimates to limit oversubscription,
it's now quite easy for tasks to become starved for work. This commit
also adds work-stealing logic to each processor, allowing a starved
processor to steal scope-compatible tasks from other busy processors.
Processors will be able to steal so long as they are not fully occupied.

Implement work stealing

APIs like `delayed` and `spawn` assumed that passed kwargs were to be
treated as options to the scheduler, which is both somewhat confusing
for users, and precludes passing kwargs to user functions.

This commit changes those APIs, as well as `@spawn`, to instead pass
kwargs directly to the user's function. Options are now passed in an
`Options` struct to `delayed` and `spawn` as the second argument (the
first being the function), while `@spawn` still keeps them before the
call (which is generally more convenient).

Internally, `Thunk`'s `inputs` field is now a
`Vector{Pair{Union{Symbol,Nothing},Any}}`, where the second element of
each pair is the argument, while the first element is a position; if
`nothing`, it's a positional argument, and if a `Symbol`, then it's a
kwarg.

Add keyword argument support

…le directive

…ppers

…DArray type

…DArray type

…bute function, revising testing forthe darray

…bute function, revising testing forthe darray

Co-authored-by: Julian Samaroo <[email protected]>