Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve rxStream performance using bufio reader #188

Merged
merged 1 commit into from
Mar 10, 2023
Merged

Improve rxStream performance using bufio reader #188

merged 1 commit into from
Mar 10, 2023

Conversation

balajiv113
Copy link
Contributor

New report with bufio reader

[test@localhost ~]$ LD_LIBRARY_PATH=. ./iperf3 -c host.crc.testing
Connecting to host host.crc.testing, port 5201
[  4] local 192.168.127.2 port 41688 connected to 192.168.127.254 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec   193 MBytes  1.62 Gbits/sec    0    529 KBytes
[  4]   1.00-2.00   sec   205 MBytes  1.72 Gbits/sec    0    529 KBytes
[  4]   2.00-3.00   sec   206 MBytes  1.73 Gbits/sec    0    529 KBytes
[  4]   3.00-4.00   sec   208 MBytes  1.74 Gbits/sec    0    529 KBytes
[  4]   4.00-5.00   sec   208 MBytes  1.75 Gbits/sec    0    529 KBytes
[  4]   5.00-6.00   sec   203 MBytes  1.70 Gbits/sec    0    529 KBytes
[  4]   6.00-7.00   sec   196 MBytes  1.65 Gbits/sec    0    529 KBytes
[  4]   7.00-8.00   sec   152 MBytes  1.28 Gbits/sec    0    529 KBytes
[  4]   8.00-9.00   sec   198 MBytes  1.66 Gbits/sec    0    529 KBytes
[  4]   9.00-10.00  sec   192 MBytes  1.61 Gbits/sec    1    399 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  1.92 GBytes  1.65 Gbits/sec    1             sender
[  4]   0.00-10.00  sec  1.91 GBytes  1.64 Gbits/sec                  receiver

iperf Done.
[test@localhost ~]$ LD_LIBRARY_PATH=. ./iperf3 -c host.crc.testing -R
Connecting to host host.crc.testing, port 5201
Reverse mode, remote host host.crc.testing is sending
[  4] local 192.168.127.2 port 43040 connected to 192.168.127.254 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec   142 MBytes  1.19 Gbits/sec
[  4]   1.00-2.00   sec   142 MBytes  1.19 Gbits/sec
[  4]   2.00-3.00   sec   139 MBytes  1.17 Gbits/sec
[  4]   3.00-4.00   sec   139 MBytes  1.16 Gbits/sec
[  4]   4.00-5.00   sec   138 MBytes  1.16 Gbits/sec
[  4]   5.00-6.00   sec   132 MBytes  1.11 Gbits/sec
[  4]   6.00-7.00   sec   136 MBytes  1.14 Gbits/sec
[  4]   7.00-8.00   sec   137 MBytes  1.15 Gbits/sec
[  4]   8.00-9.00   sec   136 MBytes  1.14 Gbits/sec
[  4]   9.00-10.00  sec   136 MBytes  1.14 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  1.35 GBytes  1.16 Gbits/sec    0             sender
[  4]   0.00-10.00  sec  1.34 GBytes  1.15 Gbits/sec                  receiver

iperf Done.
[test@localhost ~]$ LD_LIBRARY_PATH=. ./iperf3 -c host.crc.testing -u
Connecting to host host.crc.testing, port 5201
[  4] local 192.168.127.2 port 42362 connected to 192.168.127.254 port 5201
[ ID] Interval           Transfer     Bandwidth       Total Datagrams
[  4]   0.00-1.00   sec   128 KBytes  1.05 Mbits/sec  16
[  4]   1.00-2.00   sec   128 KBytes  1.05 Mbits/sec  16
[  4]   2.00-3.00   sec   128 KBytes  1.05 Mbits/sec  16
[  4]   3.00-4.00   sec   128 KBytes  1.05 Mbits/sec  16
[  4]   4.00-5.00   sec   128 KBytes  1.05 Mbits/sec  16
[  4]   5.00-6.00   sec   128 KBytes  1.05 Mbits/sec  16
[  4]   6.00-7.00   sec   128 KBytes  1.05 Mbits/sec  16
[  4]   7.00-8.00   sec   128 KBytes  1.05 Mbits/sec  16
[  4]   8.00-9.00   sec   128 KBytes  1.05 Mbits/sec  16
[  4]   9.00-10.00  sec   128 KBytes  1.05 Mbits/sec  16
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Jitter    Lost/Total Datagrams
[  4]   0.00-10.00  sec  1.25 MBytes  1.05 Mbits/sec  0.167 ms  0/159 (0%)
[  4] Sent 159 datagrams

iperf Done.
[test@localhost ~]$ LD_LIBRARY_PATH=. ./iperf3 -c host.crc.testing -R -u
Connecting to host host.crc.testing, port 5201
Reverse mode, remote host host.crc.testing is sending
[  4] local 192.168.127.2 port 40664 connected to 192.168.127.254 port 5201
[ ID] Interval           Transfer     Bandwidth       Jitter    Lost/Total Datagrams
[  4]   0.00-1.00   sec   136 KBytes  1.11 Mbits/sec  37299069969.359 ms  0/17 (0%)
[  4]   1.00-2.00   sec   128 KBytes  1.05 Mbits/sec  13281233906.164 ms  0/16 (0%)
[  4]   2.00-3.00   sec   128 KBytes  1.05 Mbits/sec  4729103814.604 ms  0/16 (0%)
[  4]   3.00-4.00   sec   128 KBytes  1.05 Mbits/sec  1683911528.736 ms  0/16 (0%)
[  4]   4.00-5.00   sec   128 KBytes  1.05 Mbits/sec  599597333.517 ms  0/16 (0%)
[  4]   5.00-6.00   sec   128 KBytes  1.05 Mbits/sec  213501099.256 ms  0/16 (0%)
[  4]   6.00-7.00   sec   128 KBytes  1.05 Mbits/sec  76022218.521 ms  0/16 (0%)
[  4]   7.00-8.00   sec   128 KBytes  1.05 Mbits/sec  27069545.487 ms  0/16 (0%)
[  4]   8.00-9.00   sec   128 KBytes  1.05 Mbits/sec  9638765.001 ms  0/16 (0%)
[  4]   9.00-10.00  sec   128 KBytes  1.05 Mbits/sec  3432114.984 ms  0/16 (0%)
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Jitter    Lost/Total Datagrams
[  4]   0.00-10.00  sec  1.26 MBytes  1.05 Mbits/sec  3432114.984 ms  0/161 (0%)
[  4] Sent 161 datagrams

iperf Done.

Old report

[test@localhost ~]$ LD_LIBRARY_PATH=. ./iperf3 -c host.crc.testing
Connecting to host host.crc.testing, port 5201
[  4] local 192.168.127.2 port 34950 connected to 192.168.127.254 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec   145 MBytes  1.22 Gbits/sec    0    428 KBytes
[  4]   1.00-2.00   sec   152 MBytes  1.28 Gbits/sec    0    495 KBytes
[  4]   2.00-3.00   sec   147 MBytes  1.23 Gbits/sec    0    519 KBytes
[  4]   3.00-4.00   sec   143 MBytes  1.20 Gbits/sec    0    519 KBytes
[  4]   4.00-5.00   sec   139 MBytes  1.17 Gbits/sec    0    519 KBytes
[  4]   5.00-6.00   sec   149 MBytes  1.25 Gbits/sec    0    519 KBytes
[  4]   6.00-7.00   sec   151 MBytes  1.26 Gbits/sec    0    519 KBytes
[  4]   7.00-8.00   sec   142 MBytes  1.19 Gbits/sec    0    519 KBytes
[  4]   8.00-9.00   sec   135 MBytes  1.13 Gbits/sec    0    519 KBytes
[  4]   9.00-10.00  sec   126 MBytes  1.06 Gbits/sec    0    519 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  1.40 GBytes  1.20 Gbits/sec    0             sender
[  4]   0.00-10.00  sec  1.39 GBytes  1.20 Gbits/sec                  receiver

iperf Done.
[test@localhost ~]$ LD_LIBRARY_PATH=. ./iperf3 -c host.crc.testing -R
Connecting to host host.crc.testing, port 5201
Reverse mode, remote host host.crc.testing is sending
[  4] local 192.168.127.2 port 51686 connected to 192.168.127.254 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec   140 MBytes  1.17 Gbits/sec
[  4]   1.00-2.00   sec   142 MBytes  1.19 Gbits/sec
[  4]   2.00-3.00   sec   139 MBytes  1.17 Gbits/sec
[  4]   3.00-4.00   sec   137 MBytes  1.15 Gbits/sec
[  4]   4.00-5.00   sec   140 MBytes  1.17 Gbits/sec
[  4]   5.00-6.00   sec   118 MBytes   988 Mbits/sec
[  4]   6.00-7.00   sec   135 MBytes  1.13 Gbits/sec
[  4]   7.00-8.00   sec   135 MBytes  1.13 Gbits/sec
[  4]   8.00-9.00   sec   133 MBytes  1.12 Gbits/sec
[  4]   9.00-10.00  sec   130 MBytes  1.09 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  1.32 GBytes  1.13 Gbits/sec    0             sender
[  4]   0.00-10.00  sec  1.32 GBytes  1.13 Gbits/sec                  receiver

iperf Done.
[test@localhost ~]$ LD_LIBRARY_PATH=. ./iperf3 -c host.crc.testing -u
Connecting to host host.crc.testing, port 5201
[  4] local 192.168.127.2 port 53445 connected to 192.168.127.254 port 5201
[ ID] Interval           Transfer     Bandwidth       Total Datagrams
[  4]   0.00-1.00   sec   128 KBytes  1.05 Mbits/sec  16
[  4]   1.00-2.00   sec   128 KBytes  1.05 Mbits/sec  16
[  4]   2.00-3.01   sec   128 KBytes  1.04 Mbits/sec  16
[  4]   3.01-4.00   sec   128 KBytes  1.05 Mbits/sec  16
[  4]   4.00-5.00   sec   128 KBytes  1.05 Mbits/sec  16
[  4]   5.00-6.01   sec   128 KBytes  1.05 Mbits/sec  16
[  4]   6.01-7.01   sec   128 KBytes  1.05 Mbits/sec  16
[  4]   7.01-8.01   sec   128 KBytes  1.05 Mbits/sec  16
[  4]   8.01-9.00   sec   128 KBytes  1.05 Mbits/sec  16
[  4]   9.00-10.00  sec   128 KBytes  1.05 Mbits/sec  16
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Jitter    Lost/Total Datagrams
[  4]   0.00-10.00  sec  1.25 MBytes  1.05 Mbits/sec  0.151 ms  0/159 (0%)
[  4] Sent 159 datagrams

iperf Done.
[test@localhost ~]$ LD_LIBRARY_PATH=. ./iperf3 -c host.crc.testing -R -u
Connecting to host host.crc.testing, port 5201
Reverse mode, remote host host.crc.testing is sending
[  4] local 192.168.127.2 port 42615 connected to 192.168.127.254 port 5201
[ ID] Interval           Transfer     Bandwidth       Jitter    Lost/Total Datagrams
[  4]   0.00-1.00   sec   136 KBytes  1.11 Mbits/sec  37299069969.779 ms  0/17 (0%)
[  4]   1.00-2.00   sec   128 KBytes  1.05 Mbits/sec  13281233906.246 ms  0/16 (0%)
[  4]   2.00-3.00   sec   128 KBytes  1.05 Mbits/sec  4729103814.593 ms  0/16 (0%)
[  4]   3.00-4.00   sec   128 KBytes  1.05 Mbits/sec  1683911528.705 ms  0/16 (0%)
[  4]   4.00-5.00   sec   128 KBytes  1.05 Mbits/sec  599597333.429 ms  0/16 (0%)
[  4]   5.00-6.00   sec   128 KBytes  1.05 Mbits/sec  213501099.189 ms  0/16 (0%)
[  4]   6.00-7.00   sec   128 KBytes  1.05 Mbits/sec  76022218.312 ms  0/16 (0%)
[  4]   7.00-8.00   sec   128 KBytes  1.05 Mbits/sec  27069545.394 ms  0/16 (0%)
[  4]   8.00-9.00   sec   128 KBytes  1.05 Mbits/sec  9638764.920 ms  0/16 (0%)
[  4]   9.00-10.00  sec   128 KBytes  1.05 Mbits/sec  3432114.971 ms  0/16 (0%)
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Jitter    Lost/Total Datagrams
[  4]   0.00-10.00  sec  1.26 MBytes  1.06 Mbits/sec  3432114.971 ms  0/161 (0%)
[  4] Sent 161 datagrams

iperf Done.

@balajiv113
Improve rxStream performance using bufio reader
ed36a56 
Signed-off-by: Balaji Vijayakumar <kuttibalaji.v6@gmail.com>
This was referenced Mar 1, 2023
Closed
Merged
@cfergeau
Copy link
Contributor

cfergeau commented Mar 6, 2023

Do you have any theory why bufio improves performance? Just curious, apart from my curiosity, the change looks good to me :)

@balajiv113
Copy link
Contributor Author

@cfergeau
bufio supports buffering (reads in bulks. I believe here default size is 4096)
So this will eventually reduces the number of raw calls and for few the data will be returned directly from buffer cache

Since we were doing multiple size based reads during stream protocol only here we will get a performance gain, in other case of non-stream this will degrade the performance

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Mar 10, 2023

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: balajiv113, gbraad

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@cfergeau cfergeau merged commit 7bb4a5d into containers:main Mar 10, 2023
@balajiv113 balajiv113 deleted the perf-improvement branch March 10, 2023 10:06
@cfergeau cfergeau mentioned this pull request Aug 28, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gbraad gbraad gbraad approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@balajiv113 @cfergeau @gbraad