Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: improved signal handling #79

Merged
merged 12 commits into from
Feb 19, 2023
Merged

Conversation

hderms
Copy link
Contributor

@hderms hderms commented Feb 15, 2023

fixes: #74

  • adds ctrlc dependency to handle SIGINT
  • registers a SIGINT handler in Process initialization method spawn() which sends Signal::Shutdown to the admin thread of the process

In my experience locally the workers seem to get the shutdown Signal and stop processing requests. However, it's desirable to ensure that the child threads actually stop. I believe that is occurring because we call Process.wait()

pub fn wait(self) {
indirectly through the server initialization process here:
Ok(segcache) => segcache.wait(),

Indeed, when I add println! statements and mess around with thread::sleep(), it seems like we are in fact waiting for all the worker threads to finish when I send SIGINT to pelikan_segcache_rs

I could see there being an issue with adding/using ctrlc. It spawns a thread to listen to signals, which seems like a potentially worthwhile tradeoff to make handling SIGINT easier, as the other libraries I found like https://docs.rs/signal-hook/latest/signal_hook/ are considerably lower level. If we want to use one of those we could potentially avoid having a dedicated thread by having some currently existing thread check for signals, but I think it's likely to complicate the implementation.

@brayniac
Copy link
Collaborator

I think we're going to need a way to disambiguate between different signals. As our behaviors might be different depending on what signal we receive. Is this possible with ctrlc or do we need to use a different crate?

.dockerignore Outdated
@@ -0,0 +1,2 @@
.git
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this file added intentionally?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ugh woops I had that in a separate PR and checked it in accidentally because it was slowing down building docker containers a lot on my local. I'll get rid of that

@hderms
Copy link
Contributor Author

hderms commented Feb 16, 2023

I think we're going to need a way to disambiguate between different signals. As our behaviors might be different depending on what signal we receive. Is this possible with ctrlc or do we need to use a different crate?

ctrlc crate, by default, only handles SIGINT on *nix platforms:

set_handler() allows setting a handler closure which is executed on Ctrl+C. On Unix, this corresponds to a SIGINT signal. On windows, Ctrl+C corresponds to CTRL_C_EVENT or CTRL_BREAK_EVENT.

I think if we wanted to handle other signals in specific ways we'd probably want to use the signal-hook crate. There is a Tokio integration which would probably make it easier.

@swlynch99
Copy link
Contributor

@brayniac Are there any signals we want to handle besides SIGINT and SIGTERM? We're just going to be gracefully shutting down for both. It's not like it's a huge issue to swap things out in the future if we want to handle, for example, SIGUSR1, and need to have different behaviour then.

@hderms
Copy link
Contributor Author

hderms commented Feb 16, 2023

@swlynch99 yeah I would vote in favor of getting SIGINT working properly (and potentially turning on termination feature of ctrlc crate which would make SIGTERM and SIGHUP handled the same way as SIGINT is being handled currently) so that people who want to run segcache in docker containers have predictable behavior and we can just remove it later in favor of signal-hook when we get the time

Copy link
Collaborator

@brayniac brayniac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@brayniac brayniac changed the title fix: explicitly handle SIGINT by attempting to signal shutdown (fix: #74) feat: shutdown after receiving SIGINT Feb 19, 2023
@brayniac brayniac changed the title feat: shutdown after receiving SIGINT feat: improved signal handling Feb 19, 2023
@brayniac brayniac merged commit b27af4a into pelikan-io:main Feb 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Handle SIGINT in segcache
4 participants