v1.3.1 Performance Testing

[EnriqueL8]

Run the performance tests using the perf-cli, similar to #1470 (comment)

[EnriqueL8]

Run against 1.3.1 RC 1

Started: 19/07/24
Duration: ~32 hours
Git commit: 3a2d40c

Node Configuration
2 FireFly nodes on one virtual server (EC2 m4.xlarge)
Entire FireFly stack is local to the server (ie both blockchains, Postgres databases, etc)
Single geth node with 2 instances of ethconnect
Maximum time to confirm before considering failure = 1 minute

stackJSONPath: /home/ubuntu/.firefly/stacks/newstack/stack.json

wsConfig:
  wsPath: /ws
  readBufferSize: 16000
  writeBufferSize: 16000
  initialDelay: 250ms
  maximumDelay: 30s
  initialConnectAttempts: 5
  heartbeatInterval: 5s

instances:
  - name: long-run
    tests: [{"name": "msg_broadcast", "workers":50},{"name": "msg_private", "workers":50},{"name": "blob_broadcast", "workers":30},{"name": "blob_private", "workers":30},{"name": "custom_ethereum_contract", "workers":20},{"name": "token_mint", "workers":10}]
    length: 500h
    sender: 0
    recipient: 1
    messageOptions:
      longMessage: false
    tokenOptions:
      tokenType: fungible
    contractOptions: {"address": "0x2715cc29d158e2d5261cb33a2eec0c6789e7eb6d"}

Shutdown summary

INFO[2024-07-20T18:09:05.280] Shutdown summary:
INFO[2024-07-20T18:09:05.282]  - Prometheus metric sent_mints_total        = 119006.000000
INFO[2024-07-20T18:09:05.282]  - Prometheus metric sent_mint_errors_total  = 0.000000
INFO[2024-07-20T18:09:05.282]  - Prometheus metric mint_token_balance      = 0.000000
INFO[2024-07-20T18:09:05.282]  - Prometheus metric received_events_total   = 7299508.000000
INFO[2024-07-20T18:09:05.282]  - Prometheus metric incomplete_events_total = 0.000000
INFO[2024-07-20T18:09:05.282]  - Prometheus metric delinquent_msgs_total    = 0.000000
INFO[2024-07-20T18:09:05.282]  - Prometheus metric actions_submitted_total = 3530924.000000
INFO[2024-07-20T18:09:05.282]  - Test duration: 31h42m59.398456728s
INFO[2024-07-20T18:09:05.282]  - Measured actions: 14599012
INFO[2024-07-20T18:09:05.282]  - Measured send TPS: 127.927970
INFO[2024-07-20T18:09:05.282]  - Measured throughput: 127.860299
INFO[2024-07-20T18:09:05.282]  - Measured send duration: min: 7.136251ms, max: 2.688845933s, avg: 132ms
INFO[2024-07-20T18:09:05.283]  - Measured event receiving duration: min: 2.007772501s, max: 18.380322296s, avg: 6.138s
INFO[2024-07-20T18:09:05.283]  - Measured total duration: min: 2.007772501s, max: 18.380322296s, avg: 6.138s

Grafana Dashboard

[EnriqueL8]

Happy with the above results and performance tests, no degradation compared to v1.3.0 - closing!