You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This is more of a question about the current design of the ReplicaSet functions, which don't seem to play well with multi-threading.
Background
Connection.hs supports mongo replica sets by storing the known members in a ReplicaSet object. If the user desires e.g. the replica set primary, the function primary first queries the known members for the address of the primary host; then it returns a Pipe to the primary, if available. However, that function does not actually return a new connection to the primary, but instead it possibly re-uses a previously open connection (see connection function).
Question
My scenario is a multi-threaded application, using a connection pool to distribute connections to server threads. It would be great if I could simply use primary as the resource-creating function of the pool, but this is not possible since subsequent connections taken from the pool would then be exactly the same connection (because of re-use).
This is a blocker, since I can't use any of the ReplicaSet functions. A trivial solution would be to expose an additional function that simply returns the Host, instead of a Pipe, like:
primary::ReplicaSet->IOHost
But I was wondering: is there any particular reason for re-using connections in the primary/secondaryOk/routedHost functions? I may be missing somthing, but it looks like this behaviour is good for simple single-threaded apps, but no-good for more realistic multi-threaded servers.
The text was updated successfully, but these errors were encountered:
Hi!
If there are no objections I will open a PR solving this issue. Since sharing the connections as it is done now is not thread-safe, I propose to not reuse connections at all, making connection always return a new connection.
Another solution (see here): allow reusing connections, but only by a thread at a time. In the linked commit, I've moved the code calling fetchReplicaInfo inside the scope of the modifyMVar, so that only one thread at a time can update the members of the replica set. The downside is that updating the members then blocks the other threads asking for a connection, so I'm not sure this is a good idea.
acondolu
pushed a commit
to acondolu/mongodb
that referenced
this issue
Dec 16, 2020
This commit addresses mongodb-haskell#119 i.e. the fact that pipes to replica set
members are reused. That choice is not thread-safe. For instance:
- thread 1 asks for a pipe to some host of the replica set; a new pipe is
opened and returned.
- after that, thread 2 asks for another pipe; updateMembers may reuse the pipe
returned to thread 1 to fetch the replica info, which is bad.
This commit disables previous pipes to be stored into a ReplicaSet object.
Subsequent calls to updateMembers will open a new pipe to fetch the
replica info. However, that pipe is not closed right away, but
temporarily kept in case the host we want to connect to is the same
as the one that has been used to fetch the replica info. If it isn't,
the pipe is closed and not stored as it was before.
Hello,
This is more of a question about the current design of the
ReplicaSet
functions, which don't seem to play well with multi-threading.Background
Connection.hs
supports mongo replica sets by storing the known members in aReplicaSet
object. If the user desires e.g. the replica set primary, the functionprimary
first queries the known members for the address of the primary host; then it returns aPipe
to the primary, if available. However, that function does not actually return a new connection to the primary, but instead it possibly re-uses a previously open connection (seeconnection
function).Question
My scenario is a multi-threaded application, using a connection pool to distribute connections to server threads. It would be great if I could simply use
primary
as the resource-creating function of the pool, but this is not possible since subsequent connections taken from the pool would then be exactly the same connection (because of re-use).This is a blocker, since I can't use any of the
ReplicaSet
functions. A trivial solution would be to expose an additional function that simply returns theHost
, instead of aPipe
, like:But I was wondering: is there any particular reason for re-using connections in the
primary
/secondaryOk
/routedHost
functions? I may be missing somthing, but it looks like this behaviour is good for simple single-threaded apps, but no-good for more realistic multi-threaded servers.The text was updated successfully, but these errors were encountered: