Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

AspectsGenerator refactor #5078

Merged
merged 6 commits into from
Jan 23, 2024
Merged

AspectsGenerator refactor #5078

merged 6 commits into from
Jan 23, 2024

Conversation

daniel-romano-DD
Copy link
Contributor

@daniel-romano-DD daniel-romano-DD commented Jan 18, 2024 •
edited
Loading

Summary of changes

Refactor AspectsGenerator to implement IIncrementalGenerator best practices in order to improve IDE performance

Reason for change

Improve performance of the generator in IDEs

Implementation details

Followed this guideline (thank you @andrewlock)

Test coverage

Other details

andrewlock
andrewlock reviewed Jan 18, 2024
View reviewed changes
tracer/src/Datadog.Trace.SourceGenerators/AspectsDefinitions/AspectsDefinitionsGenerator.cs Outdated Show resolved Hide resolved
tracer/src/Datadog.Trace.SourceGenerators/AspectsDefinitions/AspectsDefinitionsGenerator.cs
// Process InstrumentMethodAttribute first
foreach (AttributeData attributeData in boundAttributes)
if (attribute.AttributeClass is null) { continue; }
if (attribute.AttributeClass.Name == "AspectClassAttribute")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You may want to use the full type name to be safe (though in reality, it almost certainly won't be a problem!)

dd-trace-dotnet/tracer/src/Datadog.Trace.SourceGenerators/EnumExtensions/EnumExtensionsGenerator.cs

Lines 76 to 78 in 40a49fb

if ((attributeData.AttributeClass?.Name == "FlagsAttribute" ||
attributeData.AttributeClass?.Name == "Flags") &&
attributeData.AttributeClass.ToDisplayString() == FlagsAttribute)

tracer/src/Datadog.Trace.SourceGenerators/AspectsDefinitions/AspectsDefinitionsGenerator.cs Outdated Show resolved Hide resolved
tracer/src/Datadog.Trace/Iast/Aspects/System.DirectoryServices/DirectoryEntryAspect.cs Outdated Show resolved Hide resolved
@daniel-romano-DD daniel-romano-DD force-pushed the dani/asm/source_generator_refactor branch from 90f6d7a to cb41312 Compare January 18, 2024 15:28
@daniel-romano-DD daniel-romano-DD marked this pull request as ready for review January 18, 2024 15:30
@daniel-romano-DD daniel-romano-DD requested review from a team as code owners January 18, 2024 15:30
@datadog-ddstaging
Copy link

datadog-ddstaging bot commented Jan 18, 2024 •
edited
Loading

Datadog Report

Branch report: dani/asm/source_generator_refactor
Commit report: aaf8ea4
Test service: dd-trace-dotnet

✅ 0 Failed, 306530 Passed, 1506 Skipped, 57m 22.05s Wall Time

andrewlock
andrewlock approved these changes Jan 18, 2024
View reviewed changes
Copy link
Member

@andrewlock andrewlock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work! 👍

tracer/src/Datadog.Trace.SourceGenerators/AspectsDefinitions/AspectsDefinitionsGenerator.cs
new DiagnosticDescriptor(
"AG1",
"AspectsGenerator Error",
"Error message",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is "Error Message" really the correct string 🤔

Copy link
Contributor Author

@daniel-romano-DD daniel-romano-DD Jan 18, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, I'm not issuing diagnostics by now. It's a placeholder for later

tracer/src/Datadog.Trace.SourceGenerators/Datadog.Trace.SourceGenerators.csproj
Comment on lines +287 to +289
<ItemGroup>
<Folder Include="AspectsDefinitions\Sources\" />
</ItemGroup>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't needed I think

Suggested change
<ItemGroup>
<Folder Include="AspectsDefinitions\Sources\" />
</ItemGroup>

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is If I want to add the sources to compilation on the fly. Something left for next improvement

...andard2.0/Datadog.Trace.SourceGenerators/AspectsDefinitionsGenerator/AspectsDefinitions.g.cs
" [AspectMethodInsertBefore(\"System.Net.WebClient::UploadValues(System.String,System.String,System.Collections.Specialized.NameValueCollection)\",\"\",[2],[False],[None],Propagation,[])] Review(System.String)",
" [AspectMethodInsertBefore(\"System.Net.WebClient::UploadValuesTaskAsync(System.String,System.Collections.Specialized.NameValueCollection)\",\"\",[1],[False],[None],Propagation,[])] Review(System.String)",
" [AspectMethodInsertBefore(\"System.Net.WebClient::UploadValuesTaskAsync(System.String,System.String,System.Collections.Specialized.NameValueCollection)\",\"\",[2],[False],[None],Propagation,[])] Review(System.String)",
" [AspectMethodInsertBefore(\"System.Net.WebClient::set_BaseAddress(System.String)\",\"\",[0],[False],[None],Propagation,[])] Review(System.String)",
Copy link
Member

@andrewlock andrewlock Jan 18, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume all the Propagation -> Default is intentional, right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it's Default. It was a pending bug we had to fix (it is usage metadata)

@andrewlock
Copy link
Member

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5078) - mean (72ms)  : 64, 81
     .   : milestone, 72,
    master - mean (73ms)  : 63, 84
     .   : milestone, 73,

    section CallTarget+Inlining+NGEN
    This PR (5078) - mean (961ms)  : 933, 990
     .   : milestone, 961,
    master - mean (1,028ms)  : 1003, 1053
     .   : milestone, 1028,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5078) - mean (107ms)  : 103, 111
     .   : milestone, 107,
    master - mean (106ms)  : 102, 110
     .   : milestone, 106,

    section CallTarget+Inlining+NGEN
    This PR (5078) - mean (702ms)  : 683, 720
     .   : milestone, 702,
    master - mean (731ms)  : 714, 748
     .   : milestone, 731,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5078) - mean (91ms)  : 88, 94
     .   : milestone, 91,
    master - mean (90ms)  : 87, 93
     .   : milestone, 90,

    section CallTarget+Inlining+NGEN
    This PR (5078) - mean (662ms)  : 643, 681
     .   : milestone, 662,
    master - mean (693ms)  : 669, 716
     .   : milestone, 693,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5078) - mean (189ms)  : 186, 191
     .   : milestone, 189,
    master - mean (189ms)  : 185, 192
     .   : milestone, 189,

    section CallTarget+Inlining+NGEN
    This PR (5078) - mean (1,072ms)  : 1045, 1098
     .   : milestone, 1072,
    master - mean (1,135ms)  : 1117, 1154
     .   : milestone, 1135,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5078) - mean (273ms)  : 269, 277
     .   : milestone, 273,
    master - mean (272ms)  : 267, 277
     .   : milestone, 272,

    section CallTarget+Inlining+NGEN
    This PR (5078) - mean (1,051ms)  : 1027, 1075
     .   : milestone, 1051,
    master - mean (1,086ms)  : 1064, 1109
     .   : milestone, 1086,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5078) - mean (261ms)  : 257, 265
     .   : milestone, 261,
    master - mean (261ms)  : 258, 265
     .   : milestone, 261,

    section CallTarget+Inlining+NGEN
    This PR (5078) - mean (1,013ms)  : 992, 1033
     .   : milestone, 1013,
    master - mean (1,053ms)  : 1033, 1073
     .   : milestone, 1053,
Loading

@andrewlock
Copy link
Member

Throughput/Crank Report:zap:

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5078) (11.689M)   : 0, 11689433
    master (11.793M)   : 0, 11792846
    benchmarks/2.9.0 (11.757M)   : 0, 11756801

    section Automatic
    This PR (5078) (8.011M)   : 0, 8010946
    master (8.076M)   : 0, 8076448
    benchmarks/2.9.0 (8.573M)   : 0, 8573327

    section Trace stats
    This PR (5078) (8.374M)   : 0, 8373553
    master (8.407M)   : 0, 8406782

    section Manual
    This PR (5078) (10.293M)   : 0, 10293117
    master (10.353M)   : 0, 10352820

    section Manual + Automatic
    This PR (5078) (7.527M)   : 0, 7527013
    master (7.610M)   : 0, 7610023

    section Version Conflict
    This PR (5078) (6.846M)   : 0, 6845758
    master (6.845M)   : 0, 6845162
Loading 
gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5078) (9.641M)   : 0, 9641178
    master (9.498M)   : 0, 9497633
    benchmarks/2.9.0 (9.605M)   : 0, 9605360

    section Automatic
    This PR (5078) (6.663M)   : 0, 6662812
    master (6.538M)   : 0, 6538176

    section Trace stats
    This PR (5078) (7.008M)   : 0, 7008071
    master (6.967M)   : 0, 6966769

    section Manual
    This PR (5078) (8.103M)   : 0, 8102983
    master (8.183M)   : 0, 8183082

    section Manual + Automatic
    This PR (5078) (6.165M)   : 0, 6165256
    master (6.155M)   : 0, 6155194

    section Version Conflict
    This PR (5078) (5.698M)   : 0, 5698037
    master (5.582M)   : 0, 5581839
Loading 
gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5078) (10.125M)   : 0, 10125368
    master (10.290M)   : 0, 10289786
    benchmarks/2.9.0 (9.956M)   : 0, 9956027

    section Automatic
    This PR (5078) (7.116M)   : 0, 7115993
    master (7.052M)   : 0, 7051599
    benchmarks/2.9.0 (7.438M)   : 0, 7437608

    section Trace stats
    This PR (5078) (7.530M)   : 0, 7529843
    master (7.398M)   : 0, 7397958

    section Manual
    This PR (5078) (8.803M)   : 0, 8802741
    master (8.955M)   : 0, 8954877

    section Manual + Automatic
    This PR (5078) (6.842M)   : 0, 6841876
    master (6.754M)   : 0, 6754356

    section Version Conflict
    This PR (5078) (6.282M)   : 0, 6282007
    master (6.121M)   : 0, 6120614
Loading 
gantt
    title Throughput Linux x64 (ASM) (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5078) (7.507M)   : 0, 7506914
    master (7.304M)   : 0, 7303586
    benchmarks/2.9.0 (7.813M)   : 0, 7813475

    section No attack
    This PR (5078) (1.762M)   : 0, 1762427
    master (1.762M)   : 0, 1762048
    benchmarks/2.9.0 (3.251M)   : 0, 3251456

    section Attack
    This PR (5078) (1.421M)   : 0, 1421175
    master (1.413M)   : 0, 1413289
    benchmarks/2.9.0 (2.577M)   : 0, 2577462

    section Blocking
    This PR (5078) (3.146M)   : 0, 3145994
    master (3.108M)   : 0, 3107757

    section IAST default
    This PR (5078) (6.556M)   : 0, 6555696
    master (6.434M)   : 0, 6433540

    section IAST full
    This PR (5078) (5.543M)   : 0, 5543276
    master (5.532M)   : 0, 5532224

    section Base vuln
    This PR (5078) (0.939M)   : 0, 938653
    master (0.929M)   : 0, 928748

    section IAST vuln
    This PR (5078) (0.845M)   : crit ,0, 845416
    master (0.890M)   : 0, 890135
Loading

@andrewlock
Copy link
Member

Benchmarks Report 🐌

Benchmarks for #5078 compared to master:

  • 1 benchmarks are faster, with geometric mean 1.209
  • 2 benchmarks are slower, with geometric mean 1.187
  • 2 benchmarks have fewer allocations
  • 1 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 8.77μs 48.5ns 314ns 0.0225 0.00898 0 7.48 KB
master StartStopWithChild netcoreapp3.1 10.9μs 61ns 381ns 0.0326 0.0163 0 7.58 KB
master StartStopWithChild net472 17.2μs 70.3ns 272ns 1.34 0.342 0.111 7.95 KB
#5078 StartStopWithChild net6.0 8.62μs 47.4ns 268ns 0.0206 0.00825 0 7.49 KB
#5078 StartStopWithChild netcoreapp3.1 10.8μs 60.3ns 377ns 0.0329 0.0164 0 7.57 KB
#5078 StartStopWithChild net472 17.2μs 72.3ns 280ns 1.32 0.335 0.112 7.95 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 455μs 242ns 938ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 629μs 149ns 556ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 815μs 312ns 1.21μs 0.403 0 0 3.3 KB
#5078 WriteAndFlushEnrichedTraces net6.0 474μs 232ns 900ns 0 0 0 2.7 KB
#5078 WriteAndFlushEnrichedTraces netcoreapp3.1 627μs 193ns 723ns 0 0 0 2.7 KB
#5078 WriteAndFlushEnrichedTraces net472 807μs 367ns 1.37μs 0.401 0 0 3.3 KB
Benchmarks.Trace.Asm.AppSecBodyBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net6.0 38.8μs 30.9ns 120ns 0.0195 0 0 1.77 KB
master AllCycleSimpleBody netcoreapp3.1 41.3μs 38.2ns 148ns 0.0207 0 0 1.74 KB
master AllCycleSimpleBody net472 44.8μs 18.7ns 72.3ns 0.288 0 0 1.81 KB
master AllCycleMoreComplexBody net6.0 200μs 85.1ns 330ns 0.0995 0 0 9.25 KB
master AllCycleMoreComplexBody netcoreapp3.1 211μs 241ns 932ns 0.106 0 0 9.14 KB
master AllCycleMoreComplexBody net472 225μs 152ns 588ns 1.46 0 0 9.32 KB
master ObjectExtractorSimpleBody net6.0 146ns 0.19ns 0.711ns 0.00396 0 0 280 B
master ObjectExtractorSimpleBody netcoreapp3.1 215ns 0.133ns 0.513ns 0.00379 0 0 272 B
master ObjectExtractorSimpleBody net472 165ns 0.316ns 1.22ns 0.0446 0 0 281 B
master ObjectExtractorMoreComplexBody net6.0 3.05μs 1.79ns 6.44ns 0.0533 0 0 3.78 KB
master ObjectExtractorMoreComplexBody netcoreapp3.1 4.22μs 1.33ns 4.81ns 0.0506 0 0 3.69 KB
master ObjectExtractorMoreComplexBody net472 3.81μs 5.91ns 22.1ns 0.602 0.00568 0 3.8 KB
#5078 AllCycleSimpleBody net6.0 38μs 32.2ns 125ns 0.0191 0 0 1.77 KB
#5078 AllCycleSimpleBody netcoreapp3.1 41.9μs 18.7ns 67.3ns 0.0207 0 0 1.74 KB
#5078 AllCycleSimpleBody net472 44μs 19ns 73.6ns 0.286 0 0 1.81 KB
#5078 AllCycleMoreComplexBody net6.0 198μs 62.9ns 235ns 0.1 0 0 9.25 KB
#5078 AllCycleMoreComplexBody netcoreapp3.1 212μs 97.5ns 352ns 0.106 0 0 9.14 KB
#5078 AllCycleMoreComplexBody net472 226μs 108ns 418ns 1.46 0 0 9.32 KB
#5078 ObjectExtractorSimpleBody net6.0 146ns 0.0686ns 0.247ns 0.00391 0 0 280 B
#5078 ObjectExtractorSimpleBody netcoreapp3.1 202ns 0.136ns 0.528ns 0.00368 0 0 272 B
#5078 ObjectExtractorSimpleBody net472 169ns 0.287ns 1.11ns 0.0446 0 0 281 B
#5078 ObjectExtractorMoreComplexBody net6.0 3.04μs 1.98ns 7.66ns 0.0532 0 0 3.78 KB
#5078 ObjectExtractorMoreComplexBody netcoreapp3.1 4.02μs 1.41ns 5.45ns 0.0502 0 0 3.69 KB
#5078 ObjectExtractorMoreComplexBody net472 3.84μs 3.07ns 11.9ns 0.602 0.00579 0 3.8 KB
Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunWaf(args=NestedMap (10)) net6.0 51.9μs 20.3ns 73.1ns 0.208 0 0 16.06 KB
master RunWaf(args=NestedMap (10)) netcoreapp3.1 70.2μs 389ns 2.4μs 0.196 0 0 16.06 KB
master RunWaf(args=NestedMap (10)) net472 95.8μs 29.1ns 113ns 2.55 0.0962 0 16.14 KB
master RunWafTwice(args=NestedMap (10)) net6.0 57.5μs 11.2ns 42.1ns 0.218 0 0 16.6 KB
master RunWafTwice(args=NestedMap (10)) netcoreapp3.1 70.5μs 342ns 1.41μs 0.21 0 0 16.58 KB
master RunWafTwice(args=NestedMap (10)) net472 109μs 309ns 1.16μs 2.65 0.104 0 16.69 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 114μs 232ns 897ns 0.286 0 0 22.41 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 130μs 652ns 2.99μs 0.254 0 0 22.36 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net472 165μs 562ns 2.1μs 3.56 0.162 0 22.7 KB
master RunWaf(args=NestedMap (100)) net6.0 98.1μs 56.8ns 220ns 0.441 0 0 32.76 KB
master RunWaf(args=NestedMap (100)) netcoreapp3.1 131μs 734ns 5.09μs 0.439 0 0 33.33 KB
master RunWaf(args=NestedMap (100)) net472 192μs 847ns 3.28μs 5.34 0.375 0 33.67 KB
master RunWafTwice(args=NestedMap (100)) net6.0 105μs 56.3ns 195ns 0.47 0 0 33.3 KB
master RunWafTwice(args=NestedMap (100)) netcoreapp3.1 137μs 750ns 4.18μs 0.456 0 0 33.86 KB
master RunWafTwice(args=NestedMap (100)) net472 196μs 56.9ns 213ns 5.37 0.391 0 34.23 KB
master RunWafWithAttack(args=Neste(...)tack) [23]) net6.0 160μs 41.4ns 155ns 0.498 0 0 39.1 KB
master RunWafWithAttack(args=Neste(...)tack) [23]) netcoreapp3.1 197μs 99.9ns 346ns 0.507 0 0 39.63 KB
master RunWafWithAttack(args=Neste(...)tack) [23]) net472 257μs 1.12μs 4.33μs 6.31 0.505 0 40.23 KB
master RunWaf(args=NestedMap (20)) net6.0 104μs 596ns 4.42μs 0.443 0 0 32.18 KB
master RunWaf(args=NestedMap (20)) netcoreapp3.1 133μs 724ns 4.22μs 0.413 0 0 32.3 KB
master RunWaf(args=NestedMap (20)) net472 188μs 106ns 412ns 5.19 0.37 0 32.63 KB
master RunWafTwice(args=NestedMap (20)) net6.0 112μs 344ns 1.33μs 0.449 0 0 32.72 KB
master RunWafTwice(args=NestedMap (20)) netcoreapp3.1 133μs 82.5ns 309ns 0.398 0 0 32.82 KB
master RunWafTwice(args=NestedMap (20)) net472 194μs 136ns 527ns 5.25 0.389 0 33.19 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 168μs 63.2ns 228ns 0.559 0 0 38.53 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 203μs 597ns 2.31μs 0.484 0 0 38.6 KB
master RunWafWithAttack(args=Neste(...)tack) [22]) net472 255μs 537ns 2.08μs 6.12 0.5 0 39.2 KB
#5078 RunWaf(args=NestedMap (10)) net6.0 54.1μs 186ns 721ns 0.217 0 0 16.06 KB
#5078 RunWaf(args=NestedMap (10)) netcoreapp3.1 69.6μs 331ns 1.37μs 0.215 0 0 16.06 KB
#5078 RunWaf(args=NestedMap (10)) net472 101μs 93.1ns 361ns 2.54 0.0959 0 16.14 KB
#5078 RunWafTwice(args=NestedMap (10)) net6.0 52.6μs 13.5ns 48.8ns 0.226 0 0 16.6 KB
#5078 RunWafTwice(args=NestedMap (10)) netcoreapp3.1 71.7μs 344ns 1.5μs 0.221 0 0 16.58 KB
#5078 RunWafTwice(args=NestedMap (10)) net472 107μs 521ns 2.15μs 2.64 0.106 0 16.69 KB
#5078 RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 110μs 59.9ns 232ns 0.274 0 0 22.41 KB
#5078 RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 128μs 606ns 2.35μs 0.267 0 0 22.36 KB
#5078 RunWafWithAttack(args=Neste(...)tack) [22]) net472 159μs 187ns 723ns 3.61 0.16 0 22.7 KB
#5078 RunWaf(args=NestedMap (100)) net6.0 108μs 402ns 1.56μs 0.428 0 0 32.76 KB
#5078 RunWaf(args=NestedMap (100)) netcoreapp3.1 130μs 51.7ns 200ns 0.454 0 0 33.33 KB
#5078 RunWaf(args=NestedMap (100)) net472 192μs 123ns 459ns 5.32 0.373 0 33.67 KB
#5078 RunWafTwice(args=NestedMap (100)) net6.0 101μs 29.4ns 106ns 0.487 0 0 33.3 KB
#5078 RunWafTwice(args=NestedMap (100)) netcoreapp3.1 138μs 789ns 5.69μs 0.457 0 0 33.86 KB
#5078 RunWafTwice(args=NestedMap (100)) net472 199μs 933ns 3.73μs 5.36 0.39 0 34.23 KB
#5078 RunWafWithAttack(args=Neste(...)tack) [23]) net6.0 169μs 64.6ns 250ns 0.504 0 0 39.1 KB
#5078 RunWafWithAttack(args=Neste(...)tack) [23]) netcoreapp3.1 197μs 901ns 3.49μs 0.504 0 0 39.63 KB
#5078 RunWafWithAttack(args=Neste(...)tack) [23]) net472 261μs 965ns 3.61μs 6.35 0.508 0 40.23 KB
#5078 RunWaf(args=NestedMap (20)) net6.0 99.3μs 21.2ns 92.5ns 0.447 0 0 32.18 KB
#5078 RunWaf(args=NestedMap (20)) netcoreapp3.1 139μs 614ns 2.3μs 0.427 0 0 32.3 KB
#5078 RunWaf(args=NestedMap (20)) net472 186μs 144ns 557ns 5.13 0.373 0 32.63 KB
#5078 RunWafTwice(args=NestedMap (20)) net6.0 103μs 33.4ns 129ns 0.462 0 0 32.72 KB
#5078 RunWafTwice(args=NestedMap (20)) netcoreapp3.1 140μs 221ns 854ns 0.408 0 0 32.82 KB
#5078 RunWafTwice(args=NestedMap (20)) net472 194μs 114ns 442ns 5.22 0.386 0 33.19 KB
#5078 RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 165μs 109ns 421ns 0.548 0 0 38.53 KB
#5078 RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 200μs 619ns 2.4μs 0.51 0 0 38.6 KB
#5078 RunWafWithAttack(args=Neste(...)tack) [22]) net472 254μs 302ns 1.17μs 6.22 0.508 0 39.2 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 173μs 187ns 723ns 0.26 0 0 18.25 KB
master SendRequest netcoreapp3.1 197μs 1.09μs 6.52μs 0.193 0 0 20.41 KB
master SendRequest net472 0.000465ns 0.000189ns 0.000733ns 0 0 0 0 b
#5078 SendRequest net6.0 173μs 172ns 664ns 0.172 0 0 18.25 KB
#5078 SendRequest netcoreapp3.1 195μs 224ns 868ns 0.195 0 0 20.41 KB
#5078 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #5078

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1 41.71 KB 41.96 KB 247 B 0.59%

Fewer allocations 🎉 in #5078

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net6.0 41.85 KB 41.46 KB -391 B -0.93%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 547μs 1μs 3.88μs 0.534 0 0 41.85 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 657μs 951ns 3.68μs 0.334 0 0 41.71 KB
master WriteAndFlushEnrichedTraces net472 839μs 2.91μs 11.3μs 8.28 2.48 0.414 53.25 KB
#5078 WriteAndFlushEnrichedTraces net6.0 558μs 177ns 615ns 0.563 0 0 41.46 KB
#5078 WriteAndFlushEnrichedTraces netcoreapp3.1 652μs 1.51μs 5.83μs 0.329 0 0 41.96 KB
#5078 WriteAndFlushEnrichedTraces net472 854μs 4.05μs 17.2μs 8.13 2.57 0.428 53.24 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.05μs 0.274ns 0.988ns 0.0106 0 0 768 B
master ExecuteNonQuery netcoreapp3.1 1.47μs 0.732ns 2.83ns 0.0103 0 0 768 B
master ExecuteNonQuery net472 1.8μs 0.71ns 2.56ns 0.116 0 0 730 B
#5078 ExecuteNonQuery net6.0 1.09μs 0.29ns 1.08ns 0.0104 0 0 768 B
#5078 ExecuteNonQuery netcoreapp3.1 1.45μs 0.716ns 2.77ns 0.0101 0 0 768 B
#5078 ExecuteNonQuery net472 1.81μs 1.72ns 6.43ns 0.115 0 0 730 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.21μs 1.18ns 4.56ns 0.013 0 0 936 B
master CallElasticsearch netcoreapp3.1 1.56μs 1.72ns 6.45ns 0.0126 0 0 936 B
master CallElasticsearch net472 2.5μs 0.735ns 2.85ns 0.151 0 0 955 B
master CallElasticsearchAsync net6.0 1.3μs 0.558ns 2.16ns 0.0124 0 0 912 B
master CallElasticsearchAsync netcoreapp3.1 1.54μs 0.486ns 1.88ns 0.0131 0 0 984 B
master CallElasticsearchAsync net472 2.77μs 1.15ns 4.44ns 0.16 0 0 1.01 KB
#5078 CallElasticsearch net6.0 1.25μs 2.74ns 10.6ns 0.013 0 0 936 B
#5078 CallElasticsearch netcoreapp3.1 1.53μs 1.62ns 6.08ns 0.0129 0 0 936 B
#5078 CallElasticsearch net472 2.49μs 1.3ns 5.02ns 0.151 0 0 955 B
#5078 CallElasticsearchAsync net6.0 1.29μs 0.691ns 2.58ns 0.0129 0 0 912 B
#5078 CallElasticsearchAsync netcoreapp3.1 1.69μs 0.593ns 2.22ns 0.0128 0 0 984 B
#5078 CallElasticsearchAsync net472 2.56μs 0.862ns 3.34ns 0.16 0 0 1.01 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.48μs 0.646ns 2.24ns 0.0126 0 0 912 B
master ExecuteAsync netcoreapp3.1 1.66μs 1.43ns 5.35ns 0.0124 0 0 912 B
master ExecuteAsync net472 1.83μs 0.483ns 1.81ns 0.139 0 0 875 B
#5078 ExecuteAsync net6.0 1.36μs 0.892ns 3.45ns 0.0129 0 0 912 B
#5078 ExecuteAsync netcoreapp3.1 1.63μs 1.37ns 4.95ns 0.0123 0 0 912 B
#5078 ExecuteAsync net472 1.88μs 3.08ns 11.9ns 0.138 0 0 875 B
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 4.29μs 7.03ns 25.3ns 0.0299 0 0 2.1 KB
master SendAsync netcoreapp3.1 5.06μs 2.05ns 7.68ns 0.0354 0 0 2.63 KB
master SendAsync net472 7.8μs 3ns 11.6ns 0.523 0 0 3.31 KB
#5078 SendAsync net6.0 4.3μs 3.44ns 12.9ns 0.0298 0 0 2.1 KB
#5078 SendAsync netcoreapp3.1 5.15μs 13.2ns 49.5ns 0.0347 0 0 2.63 KB
#5078 SendAsync net472 7.71μs 4.81ns 18.6ns 0.522 0 0 3.31 KB
Benchmarks.Trace.Iast.StringAspectsBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #5078

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1 204.27 KB 202.62 KB -1.65 KB -0.81%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StringConcatBenchmark net6.0 59μs 1.16μs 11.3μs 0 0 0 43.44 KB
master StringConcatBenchmark netcoreapp3.1 54μs 244ns 1.17μs 0 0 0 42.64 KB
master StringConcatBenchmark net472 37.6μs 77.7ns 269ns 0 0 0 61.93 KB
master StringConcatAspectBenchmark net6.0 276μs 6.83μs 66.5μs 0 0 0 212.85 KB
master StringConcatAspectBenchmark netcoreapp3.1 327μs 6.37μs 59.4μs 0 0 0 204.27 KB
master StringConcatAspectBenchmark net472 223μs 962ns 3.47μs 0 0 0 221.18 KB
#5078 StringConcatBenchmark net6.0 52.3μs 280ns 1.46μs 0 0 0 43.44 KB
#5078 StringConcatBenchmark netcoreapp3.1 63.6μs 835ns 8.09μs 0 0 0 42.64 KB
#5078 StringConcatBenchmark net472 38.1μs 93.1ns 336ns 0 0 0 61.9 KB
#5078 StringConcatAspectBenchmark net6.0 289μs 7.98μs 78.1μs 0 0 0 213.57 KB
#5078 StringConcatAspectBenchmark netcoreapp3.1 293μs 3.55μs 32.1μs 0 0 0 202.62 KB
#5078 StringConcatAspectBenchmark net472 237μs 2.97μs 28μs 0 0 0 221.18 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 1.59μs 0.62ns 2.32ns 0.0223 0 0 1.57 KB
master EnrichedLog netcoreapp3.1 2.2μs 1.64ns 6.14ns 0.0208 0 0 1.57 KB
master EnrichedLog net472 2.58μs 1.36ns 5.26ns 0.238 0 0 1.5 KB
#5078 EnrichedLog net6.0 1.45μs 0.525ns 2.03ns 0.0224 0 0 1.57 KB
#5078 EnrichedLog netcoreapp3.1 2.21μs 0.752ns 2.81ns 0.0209 0 0 1.57 KB
#5078 EnrichedLog net472 2.65μs 4.16ns 16.1ns 0.237 0 0 1.5 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 113μs 130ns 487ns 0.0567 0 0 4.21 KB
master EnrichedLog netcoreapp3.1 119μs 108ns 418ns 0.059 0 0 4.21 KB
master EnrichedLog net472 149μs 119ns 447ns 0.668 0.223 0 4.39 KB
#5078 EnrichedLog net6.0 113μs 111ns 431ns 0.0563 0 0 4.21 KB
#5078 EnrichedLog netcoreapp3.1 119μs 188ns 726ns 0.0585 0 0 4.21 KB
#5078 EnrichedLog net472 148μs 93.3ns 349ns 0.668 0.223 0 4.39 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.99μs 0.867ns 3.36ns 0.0299 0 0 2.13 KB
master EnrichedLog netcoreapp3.1 4.28μs 3.46ns 13.4ns 0.0277 0 0 2.13 KB
master EnrichedLog net472 4.86μs 2.06ns 7.98ns 0.308 0 0 1.95 KB
#5078 EnrichedLog net6.0 3.05μs 2.08ns 8.06ns 0.029 0 0 2.13 KB
#5078 EnrichedLog netcoreapp3.1 4.2μs 9.22ns 34.5ns 0.027 0 0 2.13 KB
#5078 EnrichedLog net472 4.82μs 2.56ns 9.59ns 0.307 0 0 1.95 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 1.33μs 0.546ns 2.04ns 0.0153 0 0 1.1 KB
master SendReceive netcoreapp3.1 1.69μs 0.803ns 3.01ns 0.0147 0 0 1.1 KB
master SendReceive net472 2.2μs 3.74ns 14.5ns 0.177 0 0 1.12 KB
#5078 SendReceive net6.0 1.37μs 1.61ns 6.23ns 0.0157 0 0 1.1 KB
#5078 SendReceive netcoreapp3.1 1.72μs 1.07ns 4.13ns 0.0145 0 0 1.1 KB
#5078 SendReceive net472 2.12μs 2.75ns 10.6ns 0.176 0 0 1.12 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.72μs 0.774ns 2.9ns 0.0217 0 0 1.53 KB
master EnrichedLog netcoreapp3.1 3.84μs 1.8ns 6.47ns 0.0213 0 0 1.58 KB
master EnrichedLog net472 4.31μs 1.05ns 4.05ns 0.311 0 0 1.97 KB
#5078 EnrichedLog net6.0 2.76μs 1.01ns 3.9ns 0.0209 0 0 1.53 KB
#5078 EnrichedLog netcoreapp3.1 3.87μs 1.36ns 5.27ns 0.0212 0 0 1.58 KB
#5078 EnrichedLog net472 4.37μs 2.28ns 8.81ns 0.311 0 0 1.97 KB
Benchmarks.Trace.SpanBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #5078

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net6.0 1.192 537.83 641.17
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net6.0 1.181 462.49 546.23

Faster 🎉 in #5078

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑netcoreapp3.1 1.209 753.54 623.27

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 462ns 0.13ns 0.504ns 0.00742 0 0 536 B
master StartFinishSpan netcoreapp3.1 753ns 0.697ns 2.61ns 0.00719 0 0 536 B
master StartFinishSpan net472 751ns 0.758ns 2.93ns 0.0854 0 0 538 B
master StartFinishScope net6.0 538ns 0.121ns 0.467ns 0.00905 0 0 656 B
master StartFinishScope netcoreapp3.1 926ns 0.686ns 2.66ns 0.00853 0 0 656 B
master StartFinishScope net472 949ns 0.37ns 1.28ns 0.098 0 0 618 B
#5078 StartFinishSpan net6.0 546ns 0.153ns 0.592ns 0.00738 0 0 536 B
#5078 StartFinishSpan netcoreapp3.1 623ns 0.512ns 1.92ns 0.00729 0 0 536 B
#5078 StartFinishSpan net472 821ns 3.94ns 15.3ns 0.0852 0 0 538 B
#5078 StartFinishScope net6.0 641ns 0.171ns 0.642ns 0.00934 0 0 656 B
#5078 StartFinishScope netcoreapp3.1 890ns 0.245ns 0.918ns 0.00867 0 0 656 B
#5078 StartFinishScope net472 980ns 0.247ns 0.957ns 0.0981 0 0 618 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 726ns 0.26ns 1.01ns 0.00903 0 0 656 B
master RunOnMethodBegin netcoreapp3.1 1.01μs 0.786ns 2.84ns 0.00901 0 0 656 B
master RunOnMethodBegin net472 1.11μs 0.388ns 1.5ns 0.098 0 0 618 B
#5078 RunOnMethodBegin net6.0 659ns 0.183ns 0.683ns 0.00922 0 0 656 B
#5078 RunOnMethodBegin netcoreapp3.1 934ns 0.453ns 1.75ns 0.00891 0 0 656 B
#5078 RunOnMethodBegin net472 1.1μs 0.481ns 1.8ns 0.0982 0 0 618 B

robertpi
robertpi approved these changes Jan 23, 2024
View reviewed changes
Copy link
Member

@robertpi robertpi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed with @andrewlock , I reviewed the changes in Datadog.Trace and these seem good. Didn't attempt to review Datadog.Trace.SourceGenerator.

@andrewlock andrewlock merged commit 98f8abb into master Jan 23, 2024
55 checks passed
@andrewlock andrewlock deleted the dani/asm/source_generator_refactor branch January 23, 2024 10:53
@github-actions github-actions bot added this to the vNext milestone Jan 23, 2024
@andrewlock andrewlock added the area:builds project files, build scripts, pipelines, versioning, releases, packages label Jan 24, 2024
@daniel-romano-DD daniel-romano-DD changed the title Dani/asm/source generator refactor AspectsGenerator refactor May 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@andrewlock andrewlock andrewlock approved these changes

@robertpi robertpi robertpi approved these changes

Assignees
No one assigned
Labels
area:builds project files, build scripts, pipelines, versioning, releases, packages
Projects
None yet
Milestone
2.46.0
Development

Successfully merging this pull request may close these issues.
3 participants
@daniel-romano-DD @andrewlock @robertpi