docs: Add test reporting doc to benchmarks dir

[stubbsta]

Description

This PR is a first pass at adding a nwaku test summary page which aims to provide a quick reference for anyone implementing the waku protocol using nwaku to see the expected performance as well as have quick access to test reports.

Changes

• Added file docs/benchmarks/test-results-summary.md

How to test

1. Pull https://github.com/waku-org/docs.waku.org
2. Edit url to use this branch: https://github.com/waku-org/docs.waku.org/blob/develop/fetch-content.js#L64
3. yarn build
4. yarn serve

[stubbsta]

This is a draft to get inputs on the formatting and content.
The Quick ref section definitely needs work. I would appreciate some info on how to relate the data back to Status App.

[jm-clius]

Thanks for this! I've added a couple of comments and suggestions on what other sections to add.

[stubbsta]

@fryorcraken I wonder if the TL;DR section is not too wordy? Is the requirement not to have it be something very short that can be read quickly and easily remembered such as:

• Relay network average bandwidth usage: x KB/s
• Disv5 average bandwidth usage: y KB/s
• etc.

and then if the reader wants more info (such as the network size and message rate for the simulations where the above values were obtained, they can look at the Insights section and if they want even more info they can go look at the reports on notion?

[jm-clius]

A few more comments. :)

[jm-clius]

Thanks! LGTM now. :) I know there's still an outstanding section on relevance to Status, but happy to see this merged once that is done.

[stubbsta]

Hi @plopezlpz, could you please have a look and maybe provide some inputs?
One of the main objectives of the doc is to indicate how these values might be relevant/helpful to Status, but my knowledge of the Status waku config is limited.
For example, it would be good to be able to compare the overall Status bandwidth usage to the average Waku bandwidth usage.
Does the Status app have its own performance benchmarks that we could use to compare?

[fryorcraken]

This looks really good 👍

[fryorcraken]

Maybe more interesting to read the bandwidth overhead.

I would not exclude discv5

So something like:

for 1KB message per second, sent by 2000 publisher nodes (all nodes in network are publisher). The total bandwidth is 3MBps. Knowing that data injected to the network is 2MBs (1KB * 2000 per second). There is an overheard of 1MBps (gossipsub amplification + peer discovery discv5).

[stubbsta]

@fryorcraken I don't think the regression tests currently measure total bandwidth usage per node (@AlbertoSoutullo could you please confirm this), we can only approximate using the libp2p and discv5 data we have and the payload size and msg rate data:

e.g.
Knowing that data injected to the network is 2MBs (1KB * 2000 per second) and knowing that
libp2p in and out is 2x 8 KB/s and discv5 in and out is 10.2 + 1.6 KB/s.
We have a total of bandwidth usage of approx. 2.1 MBs

Is this sufficient for now?

[AlbertoSoutullo]

We don't report the total usage but the median one. But still, the total libp2p and discv5 usage can be measured and reported. Actually, in the v0.35 report we found that the bandwidth usage of discv5 in some nodes is extremely high compared with the libp2p one.

[fryorcraken]

The aim is to produce a messaging API, whose default values should be the one used by Status and perform benchmarks using the messaging API so it's all relevant for Status or any app using said API.

[stubbsta]

Would it make sense to add something here like:

Future Improvements: A messaging API is currently in development to streamline interactions with the Waku protocol suite.
Once completed, Waku will be able to provide benchmarks at the messaging API level of the software stack, enabling applications using the API to more easily compare their own benchmark results.

or rather just leave it out for now?

[fryorcraken]

Would be good to state the obvious of application data injected (1msg/s *1000 nodes) and data measured so we understand the overhead.