Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Safely server reload #4577

Open
maxkain opened this issue Jan 18, 2022 · 6 comments
Open

Safely server reload #4577

maxkain opened this issue Jan 18, 2022 · 6 comments

Comments

@maxkain
Copy link

maxkain commented Jan 18, 2022

Server reload function does not work as needed, I can not update workers from new sources in Symfony app using k911/swoole-bundle. Sometimes it works in controllers, sometimes not. And never works in other services.
I use full server restart, but it can to interrupt current worker processes.
Is it possible to send every worker stop signal, if worker handles request wait it, else restart it fully with new source code and start handling new requests while other workers finishing current requests?

@NathanFreeman
Copy link
Member

Hi, you can set max_wait_time and reload_async to reload or stop worker softly.

@maxkain
Copy link
Author

maxkain commented Jan 21, 2022

Hi, you can set max_wait_time and reload_async to reload or stop worker softly.

With max_wait_time server waits requests when reloading, but it does not accept new requests, so they need to wait too.
And reloading is not works as needed, it does not load all project files.
When shutdown occurs it waits 5-10 seconds, and setting more max_wait_time not helps. But it is not needed if new clients will be waiting such a lot time.

@maxkain
Copy link
Author

maxkain commented Jan 21, 2022

It would be great, if server, when restarting, could wait while current requests will be finished, and at the same time accept new requests.
But in now solution is to move any long processes someway to background or doing deploy rarely at the certain time.

@NathanFreeman
Copy link
Member

NathanFreeman commented Jan 27, 2022

I think it is hard to achieve it.
This is an endless loop.

@maxkain
Copy link
Author

maxkain commented Feb 18, 2022

I see another way to implement zero downtime deploy using nginx, like this https://syshero.org/2016-06-09-zero-downtime-deployments-using-nginx/

Steps:

  1. Deploy second instance of application with another port opened and optionally warmup it.
  2. Switch nginx proxy to the second instance, so new requests are going to it.
  3. Wait for finishing requests of the first instance using max_wait_time and shutdown it (kill -15 master_pid)

@cesurapp
Copy link

It would be great, if server, when restarting, could wait while current requests will be finished, and at the same time accept new requests. But in now solution is to move any long processes someway to background or doing deploy rarely at the certain time.

It may seem logical at the application level. But for the database layer, you must shut down the system. The new application may fail before the database migrations are complete.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants