-
-
Notifications
You must be signed in to change notification settings - Fork 68
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make Dagger.finish_stream()
propagate downstream
#579
Commits on Nov 15, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 90a974f - Browse repository at this point
Copy the full SHA 90a974fView commit details -
Configuration menu - View commit details
-
Copy full SHA for cbac605 - Browse repository at this point
Copy the full SHA cbac605View commit details -
Configuration menu - View commit details
-
Copy full SHA for e441bd0 - Browse repository at this point
Copy the full SHA e441bd0View commit details -
Configuration menu - View commit details
-
Copy full SHA for 17096fa - Browse repository at this point
Copy the full SHA 17096faView commit details -
Use Base.promote_op() instead of Base._return_type()
return_type() is kinda broken in v1.10, see: JuliaLang/julia#52385 In any case Base.promote_op() is the official public API for this operation so we should use it anyway.
Configuration menu - View commit details
-
Copy full SHA for 563f646 - Browse repository at this point
Copy the full SHA 563f646View commit details -
Special-case StreamingFunction in EagerThunkMetadata() constructor
This always us to handle all the other kinds of task specs.
Configuration menu - View commit details
-
Copy full SHA for 2f29be7 - Browse repository at this point
Copy the full SHA 2f29be7View commit details -
Fix reference to task-queues.md in the docs
This should get the docs building again.
Configuration menu - View commit details
-
Copy full SHA for d25d6c1 - Browse repository at this point
Copy the full SHA d25d6c1View commit details -
Because it doesn't actually do anything now.
Configuration menu - View commit details
-
Copy full SHA for aa07cc9 - Browse repository at this point
Copy the full SHA aa07cc9View commit details -
Configuration menu - View commit details
-
Copy full SHA for f8d0b8b - Browse repository at this point
Copy the full SHA f8d0b8bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 1be1a41 - Browse repository at this point
Copy the full SHA 1be1a41View commit details -
Use procs() when initializing EAGER_CONTEXT
Using `myid()` with `workers()` meant that when the context was initialized with a single worker the processor list would be: `[OSProc(1), OSProc(1)]`. `procs()` will always include PID 1 and any other workers, which is what we want.
Configuration menu - View commit details
-
Copy full SHA for f58404a - Browse repository at this point
Copy the full SHA f58404aView commit details -
Configuration menu - View commit details
-
Copy full SHA for ee11f3f - Browse repository at this point
Copy the full SHA ee11f3fView commit details -
Add a --verbose option to runtests.jl
This is a bit nicer than commenting/uncommenting a line in the code.
Configuration menu - View commit details
-
Copy full SHA for 0d70835 - Browse repository at this point
Copy the full SHA 0d70835View commit details -
Ensure that stream_fetch_values!() yields in its loop
Otherwise it may spin (see comments for details). Also refactored it into a while-loop instead of using a @goto.
Configuration menu - View commit details
-
Copy full SHA for 27392f0 - Browse repository at this point
Copy the full SHA 27392f0View commit details -
Add support for limiting the evaluations of a streaming DAG
This is useful for testing and benchmarking.
Configuration menu - View commit details
-
Copy full SHA for 717ceb7 - Browse repository at this point
Copy the full SHA 717ceb7View commit details -
Dev the migration-helper branch of MemPool.jl
This is currently necessary for the streaming branch, we'll have to change this later but it's good to have CI working for now.
Configuration menu - View commit details
-
Copy full SHA for a0c0805 - Browse repository at this point
Copy the full SHA a0c0805View commit details -
Configuration menu - View commit details
-
Copy full SHA for f4d709c - Browse repository at this point
Copy the full SHA f4d709cView commit details -
Configuration menu - View commit details
-
Copy full SHA for a7bdfdb - Browse repository at this point
Copy the full SHA a7bdfdbView commit details -
Initial support for robustly migrating streaming tasks
This works by converting the output buffers into a safely-serializeable container and sending that to the new node.
Configuration menu - View commit details
-
Copy full SHA for 770a241 - Browse repository at this point
Copy the full SHA 770a241View commit details -
Inherit the top-level testsets in the streaming tests
This makes them be displayed as if they were running under the original task.
Configuration menu - View commit details
-
Copy full SHA for 0b968d6 - Browse repository at this point
Copy the full SHA 0b968d6View commit details -
Replace
rand_finite()
with a deterministicProducer
functorThis makes the tests a little easier to control.
Configuration menu - View commit details
-
Copy full SHA for 0268b7e - Browse repository at this point
Copy the full SHA 0268b7eView commit details -
Configuration menu - View commit details
-
Copy full SHA for 0dbdab3 - Browse repository at this point
Copy the full SHA 0dbdab3View commit details -
Configuration menu - View commit details
-
Copy full SHA for 1cf99b8 - Browse repository at this point
Copy the full SHA 1cf99b8View commit details -
Configuration menu - View commit details
-
Copy full SHA for 71ee854 - Browse repository at this point
Copy the full SHA 71ee854View commit details -
Configuration menu - View commit details
-
Copy full SHA for 79ee021 - Browse repository at this point
Copy the full SHA 79ee021View commit details -
Configuration menu - View commit details
-
Copy full SHA for 09e5826 - Browse repository at this point
Copy the full SHA 09e5826View commit details -
Configuration menu - View commit details
-
Copy full SHA for 3911a73 - Browse repository at this point
Copy the full SHA 3911a73View commit details -
Configuration menu - View commit details
-
Copy full SHA for f71f604 - Browse repository at this point
Copy the full SHA f71f604View commit details -
Configuration menu - View commit details
-
Copy full SHA for b930a42 - Browse repository at this point
Copy the full SHA b930a42View commit details -
Configuration menu - View commit details
-
Copy full SHA for 16d73c9 - Browse repository at this point
Copy the full SHA 16d73c9View commit details -
Configuration menu - View commit details
-
Copy full SHA for 2b2da8e - Browse repository at this point
Copy the full SHA 2b2da8eView commit details -
Configuration menu - View commit details
-
Copy full SHA for 5be724f - Browse repository at this point
Copy the full SHA 5be724fView commit details -
streaming: Buffers and tasks per input/output
Instead of taking/putting values sequentially (which may block), runs "pull" and "push" tasks for each input and output, respectively. Uses buffers to communicate values between pullers/pushers and the streaming task, instead of only using one buffer per task-to-task connection.
Configuration menu - View commit details
-
Copy full SHA for d545637 - Browse repository at this point
Copy the full SHA d545637View commit details -
Configuration menu - View commit details
-
Copy full SHA for 61ab9c1 - Browse repository at this point
Copy the full SHA 61ab9c1View commit details -
Configuration menu - View commit details
-
Copy full SHA for a51cbf9 - Browse repository at this point
Copy the full SHA a51cbf9View commit details -
Configuration menu - View commit details
-
Copy full SHA for 31944af - Browse repository at this point
Copy the full SHA 31944afView commit details -
Configuration menu - View commit details
-
Copy full SHA for d5c27ab - Browse repository at this point
Copy the full SHA d5c27abView commit details -
Configuration menu - View commit details
-
Copy full SHA for fbae73f - Browse repository at this point
Copy the full SHA fbae73fView commit details -
Configuration menu - View commit details
-
Copy full SHA for bf53117 - Browse repository at this point
Copy the full SHA bf53117View commit details -
Configuration menu - View commit details
-
Copy full SHA for b9e3c70 - Browse repository at this point
Copy the full SHA b9e3c70View commit details -
Configuration menu - View commit details
-
Copy full SHA for 8908478 - Browse repository at this point
Copy the full SHA 8908478View commit details -
Configuration menu - View commit details
-
Copy full SHA for 1f21693 - Browse repository at this point
Copy the full SHA 1f21693View commit details -
Configuration menu - View commit details
-
Copy full SHA for c4bc7b2 - Browse repository at this point
Copy the full SHA c4bc7b2View commit details -
Configuration menu - View commit details
-
Copy full SHA for 51e1606 - Browse repository at this point
Copy the full SHA 51e1606View commit details -
Configuration menu - View commit details
-
Copy full SHA for 4ea09c4 - Browse repository at this point
Copy the full SHA 4ea09c4View commit details -
Configuration menu - View commit details
-
Copy full SHA for 8bf5fbf - Browse repository at this point
Copy the full SHA 8bf5fbfView commit details -
Configuration menu - View commit details
-
Copy full SHA for 07ba8b1 - Browse repository at this point
Copy the full SHA 07ba8b1View commit details -
Configuration menu - View commit details
-
Copy full SHA for 3aba122 - Browse repository at this point
Copy the full SHA 3aba122View commit details -
Configuration menu - View commit details
-
Copy full SHA for 6ac140c - Browse repository at this point
Copy the full SHA 6ac140cView commit details -
Configuration menu - View commit details
-
Copy full SHA for f60cb77 - Browse repository at this point
Copy the full SHA f60cb77View commit details -
Switch from RemoteFetcher to RemoteChannelFetcher Pass object rather than type to `stream_{push,pull}_values!` ProcessRingBuffer: Don't exit on graceful interrupt when non-empty
Configuration menu - View commit details
-
Copy full SHA for b3b70e1 - Browse repository at this point
Copy the full SHA b3b70e1View commit details
Commits on Nov 16, 2024
-
Configuration menu - View commit details
-
Copy full SHA for efc80be - Browse repository at this point
Copy the full SHA efc80beView commit details -
Configuration menu - View commit details
-
Copy full SHA for e6a504d - Browse repository at this point
Copy the full SHA e6a504dView commit details -
Streaming tests cleanup and fixes
- Added some whitespace. - Deleted the unused `rand_finite()` methods. - Allow passing the `timeout` to `test_finishes()` - Fix bug in one of the tests where we weren't waiting for all the tasks to finish, which would occasionally cause test failures because of the race condition.
Configuration menu - View commit details
-
Copy full SHA for 149adb5 - Browse repository at this point
Copy the full SHA 149adb5View commit details
Commits on Nov 17, 2024
-
Make Dagger.finish_stream() propagate downstream
Previously a streaming task calling `Dagger.finish_stream()` would only stop the caller, but now it will also stop all downstream tasks. This is done by: - Getting the output handler tasks to close their `RemoteChannel` when exiting. - Making the input handler tasks close their buffers when the `RemoteChannel` is closed. - Exiting `stream!()` when an input buffer is closed.
Configuration menu - View commit details
-
Copy full SHA for 079e9fa - Browse repository at this point
Copy the full SHA 079e9faView commit details -
`unwrap_nested_exception()` now supports `DTaskFailedException` so we can match against the real exceptions thrown.
Configuration menu - View commit details
-
Copy full SHA for 999bdd7 - Browse repository at this point
Copy the full SHA 999bdd7View commit details