Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Performance regression in JsonSerializer.Deserialize(Stream) for very small types like Int32 #59103

Closed
adamsitnik opened this issue Sep 14, 2021 · 2 comments
Closed

Performance regression in JsonSerializer.Deserialize(Stream) for very small types like Int32 #59103

adamsitnik opened this issue Sep 14, 2021 · 2 comments
Labels
area-System.Text.Json tenet-performance Performance related issue
Milestone
7.0.0

Comments

@adamsitnik
Copy link
Member

Please keep in mind that this particular scenario (deserializing a single integer from a stream) is very niche and we are most likely going to accept it. But since I don't own this area I would prefer the owners to make the call (cc @eiriktsarpalis, as it has been most likely caused by #50778)

System.Text.Json.Serialization.Tests.ReadJson.DeserializeFromStream

Result Base Diff Ratio Alloc Delta Modality Operating System Bit Processor Name Base V Diff V
Slower 279.18 311.98 0.89 +0 Windows 10.0.19043.1165 X64 AMD Ryzen Threadripper PRO 3945WX 12-Cores 5.0.921.35908 6.0.21.41701
Slower 368.39 412.24 0.89 +0 Windows 10.0.20348 X64 AMD EPYC 7452 5.0.921.35908 6.0.21.41701
Same 366.21 407.11 0.90 +0 Windows 10.0.20348 X64 AMD EPYC 7452 5.0.921.35908 6.0.21.41701
Slower 313.55 349.29 0.90 +0 Windows 10.0.18363.1621 X64 Intel Xeon CPU E5-1650 v4 3.60GHz 5.0.921.35908 6.0.21.41701
Slower 382.35 439.94 0.87 +0 Windows 8.1 X64 Intel Core i7-3610QM CPU 2.30GHz (Ivy Bridge) 5.0.921.35908 6.0.21.45401
Same 358.35 398.44 0.90 +0 Windows 10.0.19042.685 X64 Intel Core i7-5557U CPU 3.10GHz (Broadwell) 5.0.921.35908 6.0.21.41701
Same 317.12 326.48 0.97 +0 several? Windows 10.0.19043.1165 X64 Intel Core i7-6700 CPU 3.40GHz (Skylake) 5.0.921.35908 6.0.21.41701
Slower 371.66 420.87 0.88 +0 Windows 10.0.22454 X64 Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R) 5.0.921.35908 6.0.21.41701
Same 243.79 271.99 0.90 +0 Windows 10.0.22451 X64 Intel Core i7-8700 CPU 3.20GHz (Coffee Lake) 5.0.921.35908 6.0.21.41701
Slower 257.96 290.70 0.89 +0 Windows 10.0.19042.1165 X64 Intel Core i9-9900T CPU 2.10GHz 5.0.921.35908 6.0.21.41701
Slower 574.14 813.87 0.71 -72 Windows 7 SP1 X64 Intel Core2 Duo CPU T9600 2.80GHz 5.0.721.25508 6.0.21.41701
Slower 425.50 476.70 0.89 +0 centos 8 X64 AMD EPYC 7452 5.0.921.35908 6.0.21.41701
Slower 421.77 477.18 0.88 +0 debian 10 X64 AMD EPYC 7452 5.0.921.35908 6.0.21.41701
Same 449.48 481.64 0.93 +0 rhel 7 X64 AMD EPYC 7452 5.0.921.35908 6.0.21.41701
Slower 418.96 487.80 0.86 +0 sles 15 X64 AMD EPYC 7452 5.0.921.35908 6.0.21.41701
Slower 425.31 478.95 0.89 +0 opensuse-leap 15.3 X64 AMD EPYC 7452 5.0.921.35908 6.0.21.41701
Slower 323.45 382.79 0.84 +0 ubuntu 18.04 X64 Intel Xeon CPU E5-1650 v4 3.60GHz 5.0.921.35908 6.0.21.41701
Slower 438.55 501.86 0.87 +0 ubuntu 18.04 X64 Intel Core i7-2720QM CPU 2.20GHz (Sandy Bridge) 5.0.921.35908 6.0.21.41701
Same 315.30 343.17 0.92 +0 alpine 3.13 X64 Intel Core i7-7700 CPU 3.60GHz (Kaby Lake) 5.0.921.35908 6.0.21.41701
Same 1074.21 1003.96 1.07 +0 ubuntu 16.04 Arm64 Unknown processor 5.0.421.11614 6.0.21.41701
Same 414.84 447.50 0.93 +0 Windows 10.0.19043.1165 Arm64 Microsoft SQ1 3.0 GHz 5.0.921.35908 6.0.21.41701
Same 458.85 494.14 0.93 +0 Windows 10.0.22000 Arm64 Microsoft SQ1 3.0 GHz 5.0.921.35908 6.0.21.41701
Slower 290.57 342.19 0.85 +0 Windows 10.0.19043.1165 X86 AMD Ryzen Threadripper PRO 3945WX 12-Cores 5.0.921.35908 6.0.21.41701
Slower 325.89 420.65 0.77 +0 Windows 10.0.18363.1621 X86 Intel Xeon CPU E5-1650 v4 3.60GHz 5.0.921.35908 6.0.21.41701
Slower 419.02 524.61 0.80 +0 macOS Big Sur 11.5.2 X64 Intel Core i5-4278U CPU 2.60GHz (Haswell) 5.0.921.35908 6.0.21.41701
Slower 380.41 428.66 0.89 +0 macOS Big Sur 11.5.2 X64 Intel Core i7-4870HQ CPU 2.50GHz (Haswell) 5.0.921.35908 6.0.21.41701
Same 400.40 445.21 0.90 +0 macOS Big Sur 11.4 X64 Intel Core i7-5557U CPU 3.10GHz (Broadwell) 5.0.921.35908 6.0.21.41701

Repro:

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net5.0 net6.0 --filter "System.Text.Json.Serialization.Tests.ReadJson<Int32>.DeserializeFromStream"

https://pvscmdupload.blob.core.windows.net/reports/allTestHistory%2frefs%2fheads%2fmain_x64_Windows%2010.0.18362%2fSystem.Text.Json.Serialization.Tests.ReadJson(Int32).DeserializeFromStream.html

image
@ghost
Copy link

ghost commented Sep 14, 2021

Tagging subscribers to this area: @eiriktsarpalis, @layomia
See info in area-owners.md if you want to be subscribed.

Issue Details

Please keep in mind that this particular scenario (deserializing a single integer from a stream) is very niche and we are most likely going to accept it. But since I don't own this area I would prefer the owners to make the call (cc @eiriktsarpalis, as it has been most likely caused by #50778)

System.Text.Json.Serialization.Tests.ReadJson.DeserializeFromStream

Result Base Diff Ratio Alloc Delta Modality Operating System Bit Processor Name Base V Diff V
Slower 279.18 311.98 0.89 +0 Windows 10.0.19043.1165 X64 AMD Ryzen Threadripper PRO 3945WX 12-Cores 5.0.921.35908 6.0.21.41701
Slower 368.39 412.24 0.89 +0 Windows 10.0.20348 X64 AMD EPYC 7452 5.0.921.35908 6.0.21.41701
Same 366.21 407.11 0.90 +0 Windows 10.0.20348 X64 AMD EPYC 7452 5.0.921.35908 6.0.21.41701
Slower 313.55 349.29 0.90 +0 Windows 10.0.18363.1621 X64 Intel Xeon CPU E5-1650 v4 3.60GHz 5.0.921.35908 6.0.21.41701
Slower 382.35 439.94 0.87 +0 Windows 8.1 X64 Intel Core i7-3610QM CPU 2.30GHz (Ivy Bridge) 5.0.921.35908 6.0.21.45401
Same 358.35 398.44 0.90 +0 Windows 10.0.19042.685 X64 Intel Core i7-5557U CPU 3.10GHz (Broadwell) 5.0.921.35908 6.0.21.41701
Same 317.12 326.48 0.97 +0 several? Windows 10.0.19043.1165 X64 Intel Core i7-6700 CPU 3.40GHz (Skylake) 5.0.921.35908 6.0.21.41701
Slower 371.66 420.87 0.88 +0 Windows 10.0.22454 X64 Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R) 5.0.921.35908 6.0.21.41701
Same 243.79 271.99 0.90 +0 Windows 10.0.22451 X64 Intel Core i7-8700 CPU 3.20GHz (Coffee Lake) 5.0.921.35908 6.0.21.41701
Slower 257.96 290.70 0.89 +0 Windows 10.0.19042.1165 X64 Intel Core i9-9900T CPU 2.10GHz 5.0.921.35908 6.0.21.41701
Slower 574.14 813.87 0.71 -72 Windows 7 SP1 X64 Intel Core2 Duo CPU T9600 2.80GHz 5.0.721.25508 6.0.21.41701
Slower 425.50 476.70 0.89 +0 centos 8 X64 AMD EPYC 7452 5.0.921.35908 6.0.21.41701
Slower 421.77 477.18 0.88 +0 debian 10 X64 AMD EPYC 7452 5.0.921.35908 6.0.21.41701
Same 449.48 481.64 0.93 +0 rhel 7 X64 AMD EPYC 7452 5.0.921.35908 6.0.21.41701
Slower 418.96 487.80 0.86 +0 sles 15 X64 AMD EPYC 7452 5.0.921.35908 6.0.21.41701
Slower 425.31 478.95 0.89 +0 opensuse-leap 15.3 X64 AMD EPYC 7452 5.0.921.35908 6.0.21.41701
Slower 323.45 382.79 0.84 +0 ubuntu 18.04 X64 Intel Xeon CPU E5-1650 v4 3.60GHz 5.0.921.35908 6.0.21.41701
Slower 438.55 501.86 0.87 +0 ubuntu 18.04 X64 Intel Core i7-2720QM CPU 2.20GHz (Sandy Bridge) 5.0.921.35908 6.0.21.41701
Same 315.30 343.17 0.92 +0 alpine 3.13 X64 Intel Core i7-7700 CPU 3.60GHz (Kaby Lake) 5.0.921.35908 6.0.21.41701
Same 1074.21 1003.96 1.07 +0 ubuntu 16.04 Arm64 Unknown processor 5.0.421.11614 6.0.21.41701
Same 414.84 447.50 0.93 +0 Windows 10.0.19043.1165 Arm64 Microsoft SQ1 3.0 GHz 5.0.921.35908 6.0.21.41701
Same 458.85 494.14 0.93 +0 Windows 10.0.22000 Arm64 Microsoft SQ1 3.0 GHz 5.0.921.35908 6.0.21.41701
Slower 290.57 342.19 0.85 +0 Windows 10.0.19043.1165 X86 AMD Ryzen Threadripper PRO 3945WX 12-Cores 5.0.921.35908 6.0.21.41701
Slower 325.89 420.65 0.77 +0 Windows 10.0.18363.1621 X86 Intel Xeon CPU E5-1650 v4 3.60GHz 5.0.921.35908 6.0.21.41701
Slower 419.02 524.61 0.80 +0 macOS Big Sur 11.5.2 X64 Intel Core i5-4278U CPU 2.60GHz (Haswell) 5.0.921.35908 6.0.21.41701
Slower 380.41 428.66 0.89 +0 macOS Big Sur 11.5.2 X64 Intel Core i7-4870HQ CPU 2.50GHz (Haswell) 5.0.921.35908 6.0.21.41701
Same 400.40 445.21 0.90 +0 macOS Big Sur 11.4 X64 Intel Core i7-5557U CPU 3.10GHz (Broadwell) 5.0.921.35908 6.0.21.41701

Repro:

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net5.0 net6.0 --filter "System.Text.Json.Serialization.Tests.ReadJson<Int32>.DeserializeFromStream"

https://pvscmdupload.blob.core.windows.net/reports/allTestHistory%2frefs%2fheads%2fmain_x64_Windows%2010.0.18362%2fSystem.Text.Json.Serialization.Tests.ReadJson(Int32).DeserializeFromStream.html

image

Author: adamsitnik
Assignees: -
Labels:

area-System.Text.Json, tenet-performance
Milestone: -

@dotnet-issue-labeler dotnet-issue-labeler bot added the untriaged New issue has not been triaged by the area owner label Sep 14, 2021
@eiriktsarpalis
Copy link
Member

As discussed, a constant-factor regression in async serialization/deserialization is expected due to the infrastructure now incorporating support for IAsyncEnumerable. It would thus be reasonable that small value deserialization would indicate substantial regressions. We'd still need to profile this, just in case an unrelated performance bug has slipped through.

@eiriktsarpalis eiriktsarpalis removed the untriaged New issue has not been triaged by the area owner label Sep 15, 2021
@eiriktsarpalis eiriktsarpalis added this to the 7.0.0 milestone Sep 15, 2021
@adamsitnik adamsitnik mentioned this issue Sep 17, 2021
Closed
19 tasks
@jeffhandley jeffhandley closed this as not planned Won't fix, can't repro, duplicate, stale Jul 9, 2022
@ghost ghost locked as resolved and limited conversation to collaborators Aug 8, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
area-System.Text.Json tenet-performance Performance related issue
Projects
None yet
Milestone
7.0.0
Development

No branches or pull requests
3 participants
@jeffhandley @eiriktsarpalis @adamsitnik