Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Options to block startup unless RabbitMQ Transport / Event Handler are connected #3381

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

chriswiggins
Copy link
Contributor

We have been doing some failure scenario planning inside our staging Kubernetes environment and have noticed that if our Janus instances come up before RabbitMQ, they will happily start even though an error is logged to the console.

These changes bring in new options to both RabbitMQ locations: block_startup_until_connected. Setting this to true in either or both of the respective configuration files will cause the server to infinitely loop until RabbitMQ becomes available.

As a side note - it doesn't appear we can get these Transport / Event plugins status information externally. Is that something thats possible?

Copy link
Member

@lminiero lminiero left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, added a few notes inline.

@@ -35,6 +35,7 @@ general: {
#queue_autodelete = false # Whether or not incoming queue should autodelete after janus disconnects from RabbitMQ
#queue_exclusive = false # Whether or not incoming queue should only allow one subscriber
#heartbeat = 60 # Defines the seconds without communication that should pass before considering the TCP connection unreachable.
#block_startup_until_connected = true # Whether to block the server from starting until a connection to the RabbitMQ server is established.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here.

uint8_t retry_interval = 5; // Retry interval in seconds

while (!g_atomic_int_get(&stopping)) {
if (janus_rabbitmqevh_tryconnect() == 0) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code style: we prefer no space between the check (while, if etc.) and what comes next. As such, this should be

while(...

and

if(...

There's a few other occurrences of the same, so please change them there too.

src/events/janus_rabbitmqevh.c Show resolved Hide resolved
@@ -455,7 +461,22 @@ int janus_rabbitmq_init(janus_transport_callbacks *callback, const char *config_
rmq_client = g_malloc0(sizeof(janus_rabbitmq_client));

/* Connect */
int result = janus_rabbitmq_connect();
uint8_t retry_interval = 5; // Retry interval in seconds
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here.

if (!block_startup_until_connected) {
break;
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All these empty lines are unneeded: I think you can make this block more compact without them.

break;
}

JANUS_LOG(LOG_ERR, "RabbitMQ: Connection failed, retrying in %d seconds\n", retry_interval);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if this should be a LOG_WARN instead, since you're retrying.

@@ -87,6 +87,7 @@ static GThread *handler_thread;
static GThread *in_thread;
static void *jns_rmqevh_hdlr(void *data);
static void *jns_rmqevh_hrtbt(void *data);
int janus_rabbitmqevh_tryconnect(void);
int janus_rabbitmqevh_connect(void);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure why there's two functions here now. You made it inline in the transport, why not do the same in the event handler?

@lminiero
Copy link
Member

@chriswiggins any update on my comments?

@chriswiggins
Copy link
Contributor Author

@chriswiggins any update on my comments?

Apologies I have been away on vacation. Will have a look next week 😀

@lminiero
Copy link
Member

lminiero commented Sep 6, 2024

@chriswiggins ping 😁

@chriswiggins
Copy link
Contributor Author

Hey @lminiero - apologies I keep doing little bits and pieces on this and then getting pulled to other tasks.

Will endeavour to get this done this week

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants