-
Notifications
You must be signed in to change notification settings - Fork 143
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[IAST] Removed Activator calls in SourceGenerator for performance reasons #5052
Conversation
Datadog ReportBranch report: ✅ 0 Failed, 303986 Passed, 1016 Skipped, 31m 34.2s Wall Time New Flaky Tests (1)
|
Execution-Time Benchmarks Report ⏱️Execution-time results for samples comparing the following branches/commits: Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:
Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard. Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph). gantt
title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (5052) - mean (73ms) : 62, 83
. : milestone, 73,
master - mean (69ms) : 66, 73
. : milestone, 69,
section CallTarget+Inlining+NGEN
This PR (5052) - mean (1,028ms) : 1010, 1046
. : milestone, 1028,
master - mean (1,029ms) : 1008, 1051
. : milestone, 1029,
gantt
title Execution time (ms) FakeDbCommand (.NET Core 3.1)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (5052) - mean (106ms) : 103, 109
. : milestone, 106,
master - mean (105ms) : 101, 109
. : milestone, 105,
section CallTarget+Inlining+NGEN
This PR (5052) - mean (730ms) : 706, 754
. : milestone, 730,
master - mean (730ms) : 709, 751
. : milestone, 730,
gantt
title Execution time (ms) FakeDbCommand (.NET 6)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (5052) - mean (90ms) : 86, 93
. : milestone, 90,
master - mean (90ms) : 87, 93
. : milestone, 90,
section CallTarget+Inlining+NGEN
This PR (5052) - mean (694ms) : 670, 719
. : milestone, 694,
master - mean (696ms) : 672, 721
. : milestone, 696,
gantt
title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (5052) - mean (187ms) : 186, 189
. : milestone, 187,
master - mean (188ms) : 185, 191
. : milestone, 188,
section CallTarget+Inlining+NGEN
This PR (5052) - mean (1,135ms) : 1112, 1158
. : milestone, 1135,
master - mean (1,137ms) : 1116, 1158
. : milestone, 1137,
gantt
title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (5052) - mean (271ms) : 267, 276
. : milestone, 271,
master - mean (272ms) : 268, 275
. : milestone, 272,
section CallTarget+Inlining+NGEN
This PR (5052) - mean (1,092ms) : 1066, 1118
. : milestone, 1092,
master - mean (1,093ms) : 1073, 1114
. : milestone, 1093,
gantt
title Execution time (ms) HttpMessageHandler (.NET 6)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (5052) - mean (261ms) : 257, 264
. : milestone, 261,
master - mean (262ms) : 258, 265
. : milestone, 262,
section CallTarget+Inlining+NGEN
This PR (5052) - mean (1,061ms) : 1035, 1086
. : milestone, 1061,
master - mean (1,058ms) : 1026, 1090
. : milestone, 1058,
|
Throughput/Crank Report:zap:Throughput results for AspNetCoreSimpleController comparing the following branches/commits: Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red. Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards! gantt
title Throughput Linux x64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (5052) (11.824M) : 0, 11823983
master (11.893M) : 0, 11892622
benchmarks/2.9.0 (11.642M) : 0, 11641842
section Automatic
This PR (5052) (8.085M) : 0, 8084516
master (8.099M) : 0, 8098638
benchmarks/2.9.0 (8.382M) : 0, 8382125
section Trace stats
This PR (5052) (8.432M) : 0, 8431542
master (5.895M) : 0, 5895492
section Manual
This PR (5052) (10.257M) : 0, 10257047
master (10.361M) : 0, 10361124
section Manual + Automatic
This PR (5052) (7.714M) : 0, 7713794
master (7.670M) : 0, 7670088
section Version Conflict
This PR (5052) (6.925M) : 0, 6925165
master (6.859M) : 0, 6859229
gantt
title Throughput Linux arm64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (5052) (9.683M) : 0, 9682513
master (9.617M) : 0, 9616856
benchmarks/2.9.0 (9.719M) : 0, 9718800
section Automatic
This PR (5052) (6.565M) : 0, 6565323
master (6.615M) : 0, 6615394
section Trace stats
This PR (5052) (6.927M) : 0, 6927407
master (6.934M) : 0, 6934299
section Manual
This PR (5052) (8.285M) : 0, 8285117
master (8.540M) : 0, 8539968
section Manual + Automatic
This PR (5052) (6.115M) : 0, 6115393
master (6.148M) : 0, 6147793
section Version Conflict
This PR (5052) (5.677M) : 0, 5677012
master (5.693M) : 0, 5693193
gantt
title Throughput Windows x64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (5052) (10.193M) : 0, 10192711
benchmarks/2.9.0 (9.943M) : 0, 9943294
section Automatic
This PR (5052) (6.915M) : 0, 6914785
benchmarks/2.9.0 (7.260M) : 0, 7260428
section Trace stats
This PR (5052) (7.368M) : 0, 7368213
section Manual
This PR (5052) (8.607M) : 0, 8606855
section Manual + Automatic
This PR (5052) (6.833M) : 0, 6832702
section Version Conflict
This PR (5052) (6.135M) : 0, 6134900
gantt
title Throughput Linux x64 (ASM) (Total requests)
dateFormat X
axisFormat %s
section Baseline
master (7.496M) : 0, 7496114
benchmarks/2.9.0 (7.836M) : 0, 7835831
section No attack
master (1.764M) : 0, 1763502
benchmarks/2.9.0 (3.226M) : 0, 3226182
section Attack
master (1.399M) : 0, 1399328
benchmarks/2.9.0 (2.474M) : 0, 2473872
section Blocking
master (3.098M) : 0, 3097902
section IAST default
master (6.459M) : 0, 6458516
section IAST full
master (5.769M) : 0, 5769139
section Base vuln
master (0.971M) : 0, 970670
section IAST vuln
master (0.901M) : 0, 900962
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why not 😄
Summary of changes
Removed all the
Activator.CreateInstance
calls in the AspectsGenerator source code gen.Reason for change
Improve the performance of the code generator
Implementation details
Added a helper to directly call the correcponding attribute constructor based on the class name and parameters kinds