async/await pattern

[mahrud]

This adds a very rudimentary, top-level version of the async/await parallelization pattern. Here is a demo:

i1 : f = x -> (sleep 1; 2*x);

i2 : elapsedTime apply(3, f)
 -- 3.0006s elapsed

o2 = {0, 2, 4}

o2 : List

i3 : elapsedTime await apply(3, async f)
 -- 1.0005s elapsed

o3 = {0, 2, 4}

o3 : List

The main benefit of this pattern is simplicity, requiring adding as little as two words to make a computation run in parallel. Specifically, the advantage over parallelApply is that you don't need a host of new functions, instead you can turn existing functions parallel. More examples and good practices are here.

Here are some more examples of instantly turning all/any/position parallel:

i4 : g = async f;

i5 : elapsedTime all({ f 2, f 3, f 4 }, even)
 -- 3.00085s elapsed

o5 = true

i6 : elapsedTime all({ g 2, g 3, g 4 }, even @@ await)
 -- 1.00104s elapsed

o6 = true

i7 : elapsedTime any({ f 2, f 3, f 4 }, i -> i % 3 == 0)
 -- 3.00053s elapsed

o7 = true

i8 : elapsedTime any({ g 2, g 3, g 4 }, i -> (await i) % 3 == 0)
 -- 1.00048s elapsed

o8 = true

i9 : elapsedTime position({ 2, 3, 4 }, i -> (f i) % 3 == 0)
 -- 2.00041s elapsed

o9 = 1

i10 : elapsedTime position({ g 2, g 3, g 4 }, i -> (await i) % 3 == 0)
 -- 1.00038s elapsed

o10 = 1

@d-torrance: I'd like to know whether parallelApply, which is more complicated, has specific benefits or guardrails that are missing from the first example above. I imagine if I had a massive example this might crash and burn, but short of examples like the typo in #3439, I haven't seen any drawbacks.

The next steps are:

• make sure this is stable when scaled up
• expand await to also handle other container types
• move async/await to keywords in the interpreter
• unreachable tasks should be cancelled (Killing unreachable threads #3454), because currently in the all/any/position examples above, all branches are still working in the background even if very first one returned in such a way that the any/all command was done.

[d-torrance]

This is great!

Is this ready to merge now, or should the TODO checklist be tacked first?

[mahrud]

At the very least unreachable tasks scheduled with async should be killed. The others as much as possible.