Static reader node replication #113
Conversation
|
Going to simplify |
noria-server/src/controller/inner.rs
Outdated
| // Typically, we are looking for a single egress node that connects the leaf to | ||
| // readers in other domains. | ||
| // TODO: if each reader (including the 1st) definitely gets its own domain, this | ||
| // code can be a lot simpler. |
There was a problem hiding this comment.
How much is "a lot"? I think we should favor simplicity then.
|
Where did we end up with this? I vaguely remember discussing the outstanding issues with @fintelia before the holidays. |
|
Last I recall query removal was being corrected because it didn't properly handle deletion of non-reader nodes. I'm not sure what the current status is |
|
@fintelia I don't have any more changes to make. Is there a case you're concerned about in particular? It looks like the old code removes non-reader nodes in reverse topological order before it reaches my code for removing leaf/reader nodes, so it shouldn't be affected. |
noria-server/src/controller/inner.rs
Outdated
| ); | ||
| // The nodes we remove first do not have children any more | ||
| for node in &nodes { | ||
| assert_eq!( |
There was a problem hiding this comment.
Couldn't this check have been done when node is added to the collection?
noria-server/src/controller/inner.rs
Outdated
| .next() | ||
| .unwrap(); | ||
| assert!(self.ingredients[child].is_egress()); | ||
| Some(child) |
There was a problem hiding this comment.
This is the only branch that returns Some(). Why not just place the if statement from below right here instead?
Place replicas in separate domains, including when there is only 1 replica. Place these domains round robin on separate workers. (
migrate/mod.rs)Support for removing all reader replicas along with a view. (
controller/inner.rs)Obtain views to replicas round robin. CLI options to set replicas and reader threads.
Benchmarks: 16 client threads and cores, 4 Noria servers with 1 core per server, 1 write every 100 on the vote benchmark. As number of replicas increases, Noria can handle a greater number of ops/s before falling behind.
Tests: Same as before, with updated node counts to account for additional replicas. One new test to check that view actually returns different replicas, and that writes are reflected in these replicas.