How can I config node to get good performance of mir

[609127400]

I runned benchmark in 3 physical machines, every machine run 3 mir node, etc. 9 mir nodes totally.
the config of node like as follow:

{
  "Trantor": {
    "Mempool": {
      "MaxTransactionsInBatch": "1024",
      "MaxPayloadInBatch": "1048576",
      "BatchTimeout": "100000000"
    },
    "Iss": {
      "InitialMembership": {
        "Nodes": {
          "0": {
            "Id": "0",
            "Addr": "/ip4/192.168.253.128/tcp/10000",
            "Key": null,
            "Weight": "1"
          },
          "1": {
            "Id": "1",
            "Addr": "/ip4/192.168.253.128/tcp/10001",
            "Key": null,
            "Weight": "1"
          },
          "2": {
            "Id": "2",
            "Addr": "/ip4/192.168.253.128/tcp/10002",
            "Key": null,
            "Weight": "1"
          },
          "3": {
            "Id": "3",
            "Addr": "/ip4/192.168.253.129/tcp/10000",
            "Key": null,
            "Weight": "1"
          },
          "4": {
            "Id": "4",
            "Addr": "/ip4/192.168.253.129/tcp/10001",
            "Key": null,
            "Weight": "1"
          },
          "5": {
            "Id": "5",
            "Addr": "/ip4/192.168.253.129/tcp/10002",
            "Key": null,
            "Weight": "1"
          },
          "6": {
            "Id": "6",
            "Addr": "/ip4/192.168.253.130/tcp/10000",
            "Key": null,
            "Weight": "1"
          },
          "7": {
            "Id": "7",
            "Addr": "/ip4/192.168.253.130/tcp/10001",
            "Key": null,
            "Weight": "1"
          },
          "8": {
            "Id": "8",
            "Addr": "/ip4/192.168.253.130/tcp/10002",
            "Key": null,
            "Weight": "1"
          }
        }
      },
      "ConfigOffset": "2",
      "SegmentLength": "4",
      "EpochLength": "0",
      "MaxProposeDelay": "0",
      "MsgBufCapacity": "33554432",
      "RetainedEpochs": "1",
      "CatchUpTimerPeriod": "1000000000",
      "CheckpointResendPeriod": "1000000000",
      "LeaderSelectionPolicy": "simple",
      "PBFTDoneResendPeriod": "1000000000",
      "PBFTCatchUpDelay": "1000000000",
      "PBFTViewChangeSNTimeout": "4000000000",
      "PBFTViewChangeSegmentTimeout": "8000000000",
      "PBFTViewChangeResendPeriod": "1000000000"
    },
    "Net": {
      "ConnectionBufferSize": "512",
      "ReconnectionPeriod": "3000000000",
      "MinComplainPeriod": "1000000000"
    },
    "Availability": {
      "Limit": "1",
      "MaxRequests": "9223372036854775807"
    }
  },
  "TxGen": {
    "ClientID": "2",
    "PayloadSize": "512",
    "NumClients": "100"
  },
  "Duration": "600000000000"
}

finally, I got result of mir network as follow:

<style> </style>

experiment-id	batch-timeout	clients-node	clients-total	duration-target	max-batch-bytes	max-batch-tx	num-nodes	segment-length	duration-actual	latency-95p	latency-avg	latency-max	latency-median	net-down-avg	net-loss-avg	net-up-avg	throughput
0	0.1	100	900	600	1048576	1024	9	4	602	20.161	5.3315	37.785	4.344	0.0922	0	0.1205	18.5738
1	0.1	100	900	600	1048576	1024	9	4	601	20.98	6.3899	37.621	4.359	0.0975	0	0.1137	15.2841
2	0.1	100	900	600	1048576	1024	9	4	602	17.142	6.2795	36.752	4.41	0.0943	0	0.1073	15.7645
3	0.1	100	900	600	1048576	1024	9	4	602	20.844	7.0576	33.588	4.402	0.0938	0	0.0973	14.0332
4	0.1	100	900	600	1048576	1024	9	4	601	24.423	7.4515	29.417	4.412	0.1021	0	0.0926	13.289
5	0.1	100	900	600	1048576	1024	9	4	600	24.694	8.6646	41.267	4.966	0.0998	0	0.0899	11.4825
6	0.1	100	900	600	1048576	1024	9	4	602	21.712	8.6545	24.569	8.314	0.0984	0	0.0909	11.4428
7	0.1	100	900	600	1048576	1024	9	4	601	24.901	9.0406	49.875	4.958	0.1045	0	0.0835	10.7973
8	0.1	100	900	600	1048576	1024	9	4	602	24.498	8.7784	32.59	8.413	0.098	0	0.0852	11.2769

I wanna known how can I modify the config to improve the performance of mir network, I readly need it!
of course, I wanna run mir network with different node number like 16/32/64, but I need known what's the best config for node's performance firstly.

Thanks!

[matejpavlovic]

Hi, looking at the table, it looks like the nodes might have been overloaded. The latency is on the order of seconds, the tail latency even tens of seconds. What were the hardware specs of the machines? Running 3 nodes on a single machine is generally a lot, unless the machine is huge. The experiments I ran for the Trantor evaluation used a separate machine for every node:

For each replica, we use an Amazon EC2 virtual machine of type t4g.xlarge with 4 virtual Arm-based AWS Graviton2 CPUs and 16GB of memory (even though most of the memory is not used)