Skip to content

Commit 47e883c

Browse files
committed
add tvu port binding info
1 parent 2a1441f commit 47e883c

File tree

1 file changed

+50
-0
lines changed

1 file changed

+50
-0
lines changed

docs/src/validator/tvu.md

+50
Original file line numberDiff line numberDiff line change
@@ -14,3 +14,53 @@ those blocks' transactions through the runtime.
1414
## Retransmit Stage
1515

1616
![Retransmit Block Diagram](/img/retransmit_stage.svg)
17+
18+
19+
## TVU sockets
20+
21+
Externally, TVU appears to bind to one port, typically 8002 UDP. Internally, TVU is actually bound with multiple sockets
22+
to improve kernel's handling of the packet queues. It is setup so that a node can advertise one external ip/port for TVU/TPU. We're binding multiple sockets to the same port using SO_REUSEPORT:
23+
24+
```rust
25+
let (tvu_port, tvu_sockets) = multi_bind_in_range_with_config(
26+
bind_ip_addr,
27+
port_range,
28+
socket_config_reuseport,
29+
num_tvu_sockets.get(),
30+
)
31+
.expect("tvu multi_bind");
32+
```
33+
34+
`multi_bind_in_range_with_config` sets `SO_REUSEPORT`. This means that other nodes only need to know about the one ip/port pair for TVU (similar principle applies in the case of TPU UDP sockets). The kernel distributes the incoming packets to all sockets bound to that port, and each socket can be serviced by a different thread.
35+
36+
37+
The TVU socket information is published via Gossip and is available in `ContactInfo` struct.
38+
To set a tvu socket, the node can call set_tvu() which is created by the macro `set_socket!`. For example:
39+
40+
```rust
41+
info.set_tvu(QUIC, (addr, tvu_quic_port)).unwrap();
42+
```
43+
44+
`set_tvu()` will call `set_socket()`
45+
46+
under the hood in the actual `ContactInfo` CRDS all sockets are identified by a tag/key like these:
47+
48+
```rust
49+
const SOCKET_TAG_TVU: u8 = 10; // For UDP
50+
const SOCKET_TAG_TVU_QUIC: u8 = 11; // For QUIC
51+
```
52+
53+
* `set_socket()` will create a `SocketEntry` and store that into `ContactInfo::sockets`
54+
* `set_socket()` will also update the `ContactInfo::cache`
55+
56+
```rust
57+
cache: [SocketAddr; SOCKET_CACHE_SIZE]
58+
```
59+
60+
the cache is purely for quick lookups and optimization. it is not serialized and sent to peer nodes.
61+
But, `SocketEntry` will be serialized and sent to peer nodes in the `ContactInfo` CRDS. On the receiving end, the `get_socket!` macro will return the TVU port.
62+
so you can call:
63+
```rust
64+
get_socket!(tvu, SOCKET_TAG_TVU, SOCKET_TAG_TVU_QUIC);
65+
```
66+
to retrieve the TVU ports of the remote node.

0 commit comments

Comments
 (0)