networking/litep2p: Node running litep2p seems to be leaking memory

[alexggh]

Looking over the dashboards on our kusama validators the memory on node that is running litep2p seems to be constantly increasing it is now at 12GiB, all other nodes are around 3-4 GiB and constant.

https://grafana.teleport.parity.io/goto/Uoh4CQmHg?orgId=1

cc: @paritytech/networking

[lexnv]

Thanks Alex for raising this! 🙏

We also had this issue with addresses, although the memory consumed here is in the order of GiB (#5998).

There are a few places that come to mind where to look at next:

• Transport manager peers: we don't seem to ever remove old/disconnected entries
• Address store: this is intertwined at the moment with the above peer state, which might grow indefinely as well. There's already work in this area to have an address store eviction algorithm to bound the growth of internal structures:
  • address_store: Improve address tracking and add eviction algorithm litep2p#250

I would start by looking at litep2p and then move to substrate code

[lexnv]

Initial testing with debug logs built on top of branch lexnv/holistic-litep2p-test-dhtandpeerset shows memory leaks in litep2p:

• transport manager peer state (from above)
• TCP / WebSocket state tracking

Will let my node running over night and followup with patches

[lexnv]

We need metrics to filter out potential leaks (ie monotonically increasing state tracking is a concern).

We have around 3 separate leaks:

• transport-manager: Fix memory leak by bounding the number of tracked peers litep2p#268
• TCP/WebSocket: Fix memory leak of pending dials litep2p#269
• TCP/WebSocket: Fix memory leak of canceled state and canceled futures litep2p#270 (grouped together the cancellation state by connection ID with the AbortHandler since I believe they are similar)

1. Transport Manager State Leak

• transport-manager: Fix memory leak by bounding the number of tracked peers litep2p#268
• we are increasing state tracking of peers in an unbounded way
• state tracking can be a bit smarter and similar to a garbage collector removing the state of disconnected peers once every N minutes

2. TCP/WebSocket Pending Dials Leak

• TCP/WebSocket: Fix memory leak of pending dials litep2p#269
• we are leaking pending dials state tracking, i expect we are not removing this state properly

3. TCP/WebSocket Cancellation Logic Leak

• TCP/WebSocket: Fix memory leak of canceled state and canceled futures litep2p#270
• cancel dials leak connection IDs
• cancel dials leak AbortableHandlers for dialing futures

[lexnv]

Litep2p PRs

For more details and explained edge-cases when the leaks happen see:

• transport: Fix pending dials memory leak litep2p#271
• tcp/websocket/quic: Fix cancel memory leak litep2p#272

[lexnv]

Lower severity memory leaks in the ping and identify protocols:

• ping: Fix memory leak of unremoved pending_opens litep2p#274
• identify: Fix memory leak of unused pending_opens litep2p#273