Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Killing ongoing rayon task #1014

Closed
DarkFenX opened this issue Jan 31, 2023 · 2 comments
Closed

Killing ongoing rayon task #1014

DarkFenX opened this issue Jan 31, 2023 · 2 comments

Comments

@DarkFenX
Copy link

DarkFenX commented Jan 31, 2023

I am writing a synchronous computational library (which does not use rayon by itself), with an option to have HTTP interface on top of it. HTTP interface uses axum/tokio, but as soon as there is any CPU-heavy request to handle, it spawns a rayon task (using tokio-rayon crate so far, as far as I understood from #679, there is no built-in functionality for that so far).

The load depends on what comes in a request, and it's hard to predict by just looking at arguments. My HTTP server is configured with tower timeouts, so that too heavy tasks are killed.

The issue is, so far all i can do in an error handler is to return HTTP error code, while rayon will keep processing it, sometimes until the bitter kill signal. Are there workarounds, which do not involve me spamming my library with channel arguments throughout all its functions, and adding checkpoints where the lib checks for messages and decide if it should stop processing and return an error?

@cuviper
Copy link
Member

cuviper commented Jan 31, 2023

There's no arbitrary way to kill a task -- it's basically the same problem as thread cancellation. Every call has some expected return value that we can't just ignore, and threads may be blocked on the result of other threads as well.

Are there workarounds, which do not involve me spamming my library with channel arguments throughout all its functions, and adding checkpoints where the lib checks for messages and decide if it should stop processing and return an error?

A channel or a simple atomic flag will do, and I think that's probably the best possible. You can decide from that how to handle your cancellation, either stuff like try_fold returning an error, or just panic and maybe catch that at a higher level (assuming you don't have panic=abort).

@DarkFenX
Copy link
Author

DarkFenX commented Feb 1, 2023

Yes, I just finished reading up on issues with thread cancellation (dangers of it were and probably still are unfamiliar to me). I will try using atomic flag for termination, thanks.

@DarkFenX DarkFenX closed this as completed Feb 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants