Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[IAST] Removed Activator calls in SourceGenerator for performance reasons #5052

Merged
merged 1 commit into from
Jan 12, 2024

Conversation

daniel-romano-DD
Copy link
Contributor

@daniel-romano-DD daniel-romano-DD commented Jan 12, 2024

Summary of changes

Removed all the Activator.CreateInstance calls in the AspectsGenerator source code gen.

Reason for change

Improve the performance of the code generator

Implementation details

Added a helper to directly call the correcponding attribute constructor based on the class name and parameters kinds

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
@datadog-ddstaging
Copy link

datadog-ddstaging bot commented Jan 12, 2024

Datadog Report

Branch report: dani/asm/source_generator_refactor
Commit report: e2713fc
Test service: dd-trace-dotnet

✅ 0 Failed, 303986 Passed, 1016 Skipped, 31m 34.2s Wall Time
❄️ 1 New Flaky

New Flaky Tests (1)

  • SubmitsTraces - Datadog.Trace.ClrProfiler.IntegrationTests.AdoNet.SystemDataSqlClientTests - Last Failure

    Expand for error
     Expected exit code: 0, actual exit code: -1073741819.
    

@andrewlock
Copy link
Member

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

Loading
gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5052) - mean (73ms)  : 62, 83
     .   : milestone, 73,
    master - mean (69ms)  : 66, 73
     .   : milestone, 69,

    section CallTarget+Inlining+NGEN
    This PR (5052) - mean (1,028ms)  : 1010, 1046
     .   : milestone, 1028,
    master - mean (1,029ms)  : 1008, 1051
     .   : milestone, 1029,

Loading
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5052) - mean (106ms)  : 103, 109
     .   : milestone, 106,
    master - mean (105ms)  : 101, 109
     .   : milestone, 105,

    section CallTarget+Inlining+NGEN
    This PR (5052) - mean (730ms)  : 706, 754
     .   : milestone, 730,
    master - mean (730ms)  : 709, 751
     .   : milestone, 730,

Loading
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5052) - mean (90ms)  : 86, 93
     .   : milestone, 90,
    master - mean (90ms)  : 87, 93
     .   : milestone, 90,

    section CallTarget+Inlining+NGEN
    This PR (5052) - mean (694ms)  : 670, 719
     .   : milestone, 694,
    master - mean (696ms)  : 672, 721
     .   : milestone, 696,

Loading
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5052) - mean (187ms)  : 186, 189
     .   : milestone, 187,
    master - mean (188ms)  : 185, 191
     .   : milestone, 188,

    section CallTarget+Inlining+NGEN
    This PR (5052) - mean (1,135ms)  : 1112, 1158
     .   : milestone, 1135,
    master - mean (1,137ms)  : 1116, 1158
     .   : milestone, 1137,

Loading
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5052) - mean (271ms)  : 267, 276
     .   : milestone, 271,
    master - mean (272ms)  : 268, 275
     .   : milestone, 272,

    section CallTarget+Inlining+NGEN
    This PR (5052) - mean (1,092ms)  : 1066, 1118
     .   : milestone, 1092,
    master - mean (1,093ms)  : 1073, 1114
     .   : milestone, 1093,

Loading
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5052) - mean (261ms)  : 257, 264
     .   : milestone, 261,
    master - mean (262ms)  : 258, 265
     .   : milestone, 262,

    section CallTarget+Inlining+NGEN
    This PR (5052) - mean (1,061ms)  : 1035, 1086
     .   : milestone, 1061,
    master - mean (1,058ms)  : 1026, 1090
     .   : milestone, 1058,

@daniel-romano-DD daniel-romano-DD marked this pull request as ready for review January 12, 2024 11:53
@daniel-romano-DD daniel-romano-DD requested a review from a team as a code owner January 12, 2024 11:53
@andrewlock
Copy link
Member

Throughput/Crank Report:zap:

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

Loading
gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5052) (11.824M)   : 0, 11823983
    master (11.893M)   : 0, 11892622
    benchmarks/2.9.0 (11.642M)   : 0, 11641842

    section Automatic
    This PR (5052) (8.085M)   : 0, 8084516
    master (8.099M)   : 0, 8098638
    benchmarks/2.9.0 (8.382M)   : 0, 8382125

    section Trace stats
    This PR (5052) (8.432M)   : 0, 8431542
    master (5.895M)   : 0, 5895492

    section Manual
    This PR (5052) (10.257M)   : 0, 10257047
    master (10.361M)   : 0, 10361124

    section Manual + Automatic
    This PR (5052) (7.714M)   : 0, 7713794
    master (7.670M)   : 0, 7670088

    section Version Conflict
    This PR (5052) (6.925M)   : 0, 6925165
    master (6.859M)   : 0, 6859229

Loading
gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5052) (9.683M)   : 0, 9682513
    master (9.617M)   : 0, 9616856
    benchmarks/2.9.0 (9.719M)   : 0, 9718800

    section Automatic
    This PR (5052) (6.565M)   : 0, 6565323
    master (6.615M)   : 0, 6615394

    section Trace stats
    This PR (5052) (6.927M)   : 0, 6927407
    master (6.934M)   : 0, 6934299

    section Manual
    This PR (5052) (8.285M)   : 0, 8285117
    master (8.540M)   : 0, 8539968

    section Manual + Automatic
    This PR (5052) (6.115M)   : 0, 6115393
    master (6.148M)   : 0, 6147793

    section Version Conflict
    This PR (5052) (5.677M)   : 0, 5677012
    master (5.693M)   : 0, 5693193

Loading
gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5052) (10.193M)   : 0, 10192711
    benchmarks/2.9.0 (9.943M)   : 0, 9943294

    section Automatic
    This PR (5052) (6.915M)   : 0, 6914785
    benchmarks/2.9.0 (7.260M)   : 0, 7260428

    section Trace stats
    This PR (5052) (7.368M)   : 0, 7368213

    section Manual
    This PR (5052) (8.607M)   : 0, 8606855

    section Manual + Automatic
    This PR (5052) (6.833M)   : 0, 6832702

    section Version Conflict
    This PR (5052) (6.135M)   : 0, 6134900

Loading
gantt
    title Throughput Linux x64 (ASM) (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    master (7.496M)   : 0, 7496114
    benchmarks/2.9.0 (7.836M)   : 0, 7835831

    section No attack
    master (1.764M)   : 0, 1763502
    benchmarks/2.9.0 (3.226M)   : 0, 3226182

    section Attack
    master (1.399M)   : 0, 1399328
    benchmarks/2.9.0 (2.474M)   : 0, 2473872

    section Blocking
    master (3.098M)   : 0, 3097902

    section IAST default
    master (6.459M)   : 0, 6458516

    section IAST full
    master (5.769M)   : 0, 5769139

    section Base vuln
    master (0.971M)   : 0, 970670

    section IAST vuln
    master (0.901M)   : 0, 900962

Copy link
Member

@andrewlock andrewlock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not 😄

@daniel-romano-DD daniel-romano-DD merged commit 02a6860 into master Jan 12, 2024
53 of 55 checks passed
@daniel-romano-DD daniel-romano-DD deleted the dani/asm/source_generator_refactor branch January 12, 2024 15:43
@github-actions github-actions bot added this to the vNext milestone Jan 12, 2024
@andrewlock andrewlock added the area:builds project files, build scripts, pipelines, versioning, releases, packages label Jan 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:builds project files, build scripts, pipelines, versioning, releases, packages
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants