Skip to content

Commit 14432a4

Browse files
alexpyattaevalexpyattaev
authored
add tvu port binding info to docs (#4948)
* add tvu port binding info * add notes about TPU and quic Co-authored-by: Greg Cusack <[email protected]>
1 parent 163dc65 commit 14432a4

File tree

2 files changed

+70
-15
lines changed

2 files changed

+70
-15
lines changed

‎docs/art/tvu.bob

+13-13
Original file line numberDiff line numberDiff line change
@@ -1,28 +1,28 @@
11
+------------+
22
| Gossip |
33
| Service |
4-
+-------------+ | |
5-
|Child | +------------+ +-+-----+----+
6-
|Validators | |Neighborhood| | ^
7-
| | |Validators | | |
8-
+-------+-----+ | | | |
9-
^ +---------+--+ Peer| |Votes
10-
| ^ List| |
11-
+-----------------+ | | |
12-
shreds(forward=true)| | | |
4+
+-------------+ | |
5+
|Child | +------------+ +-+-----+----+
6+
|Validators | |Neighborhood| | ^
7+
| | |Validators | | |
8+
+-------+-----+ | | | |
9+
^ +---------+--+ Peer| |Votes
10+
| ^ List| |
11+
+------------------------+ | | |
12+
shreds(forward = true) | | | |
1313
+---------------------------------------------------------------------------------------------+
1414
| | | | | |
1515
| TVU | | +--------+ | |
1616
| | | v | |
1717
+------------+ | +-------+ +------------+ +-+--+---+---+ +---+--------------+ +-------------+ |
18-
| | Repair | | | | | | Retransmit | | Replay | | Transaction | |
18+
| | Repair | | | | | | Retransmit | | Replay | | Transaction | |
1919
| Upstream +------------->+ +->+ Shred +--->+ Stage +----->+ Stage +----->+ Status | |
20-
| Validators | TVU | | Shred | | Verify | | | | +--------------+ | | Service | |
20+
| Validators | Turbine | | Shred | | Verify | | | | +--------------+ | | Service | |
2121
| +------------->+ Fetch | | Leader Sig | +------+-----+ | | PoH Verify | | | (optional) | |
2222
| | | | Stage | | Stage | ^ | | TX Sig Verify| | +-------------+ |
2323
| +------------->+ | | | | | | | | |
24-
| | TVU | | | +--+---------+ | | +-+------------+ | |
25-
+------------+ Forwards| +-------+ ^ | | | | |
24+
| | Turbine | | | +--+---------+ | | +-+------------+ | |
25+
+------------+ Forwards | +-------+ ^ | | | | |
2626
| | | +------------------+ |
2727
| | | | |
2828
| | | | |

‎docs/src/validator/tvu.md

+57-2
Original file line numberDiff line numberDiff line change
@@ -6,11 +6,66 @@ pagination_label: Validator's Transaction Validation Unit (TVU)
66
---
77

88
TVU (Transaction Validation Unit) is the logic of the validator
9-
responsible for validating and propagating blocks and processing
10-
those blocks' transactions through the runtime.
9+
responsible for propagating blocks between validators and ensuring that
10+
those blocks' transactions reach the replay stage. Its principal external
11+
interface is the turbine protocol.
1112

1213
![TVU Block Diagram](/img/tvu.svg)
1314

1415
## Retransmit Stage
1516

1617
![Retransmit Block Diagram](/img/retransmit_stage.svg)
18+
19+
## TVU sockets
20+
21+
Externally, TVU UDP receiver appears to bind to one port, typically 8002 UDP.
22+
Internally, TVU is actually bound with multiple sockets to improve kernel's handling of the packet queues.
23+
24+
> **NOTE:** TPU sockets use similar logic
25+
26+
A node advertises one external ip/port for TVU while binding multiple sockets to that same port using SO_REUSEPORT:
27+
28+
```rust
29+
let (tvu_port, tvu_sockets) = multi_bind_in_range_with_config(
30+
bind_ip_addr,
31+
port_range,
32+
socket_config_reuseport,
33+
num_tvu_sockets.get(),
34+
)
35+
.expect("tvu multi_bind");
36+
```
37+
38+
`multi_bind_in_range_with_config` sets `SO_REUSEPORT`. This means that other nodes only need to know about the one ip/port pair for TVU (similar principle applies in the case of TPU UDP sockets). The kernel distributes the incoming packets to all sockets bound to that port, and each socket can be serviced by a different thread.
39+
40+
> **NOTE:** TVU QUIC socket does not use `SO_REUSEPORT`, but otherwise works similarly to UDP
41+
42+
The TVU socket information is published via Gossip and is available in the `ContactInfo` struct.
43+
To set a TVU socket, the node calls `set_tvu(...)`. The `set_tvu()` method is created by the macro `set_socket!`. For example:
44+
45+
```rust
46+
info.set_tvu(QUIC, (addr, tvu_quic_port)).unwrap();
47+
info.set_tvu(UDP, (addr, tvu_udp_port)).unwrap();
48+
```
49+
50+
Under the hood, `set_tvu()` calls `set_socket()`.
51+
52+
In the `ContactInfo` struct, all sockets are identified by a tag/key, e.g.:
53+
54+
```rust
55+
const SOCKET_TAG_TVU: u8 = 10; // For UDP
56+
const SOCKET_TAG_TVU_QUIC: u8 = 11; // For QUIC
57+
```
58+
59+
* `set_socket()` creates a `SocketEntry` and stores that into `ContactInfo::sockets`
60+
* `set_socket()` updates `ContactInfo::cache`
61+
62+
```rust
63+
cache: [SocketAddr; SOCKET_CACHE_SIZE]
64+
```
65+
66+
`cache` is purely for quick lookups and optimization; it is not serialized and sent to peer nodes.
67+
`SocketEntry` is serialized and sent to peer nodes within the gossip message type `CrdsData::ContactInfo`. Upon receiving the `ContactInfo`, the peer node calls the `get_socket!` macro to retrieve the TVU port associated with the node.
68+
For example, to retrieve the TVU ports of the remote node, the peer node calls:
69+
```rust
70+
get_socket!(tvu, SOCKET_TAG_TVU, SOCKET_TAG_TVU_QUIC);
71+
```

0 commit comments

Comments
 (0)