Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Collect system-tests logs on test failure #8500

Merged
merged 2 commits into from
Mar 5, 2025
Merged

Collect system-tests logs on test failure #8500

merged 2 commits into from
Mar 5, 2025

Conversation

smola
Copy link
Member

@smola smola commented Mar 4, 2025 •
edited
Loading

What Does This Do

Collect system-tests logs on test failures, which is when we need them the most.

Motivation

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@smola smola added comp: testing Testing tag: no release notes Changes to exclude from release notes labels Mar 4, 2025
@smola smola changed the title Collect system-test logs on test failure Collect system-tests logs on test failure Mar 4, 2025
@smola smola marked this pull request as ready for review March 4, 2025 12:32
@smola smola requested a review from a team as a code owner March 4, 2025 12:32
@smola smola requested a review from dougqh March 4, 2025 12:32
@smola smola enabled auto-merge (squash) March 4, 2025 12:40
@pr-commenter
Copy link

pr-commenter bot commented Mar 4, 2025 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/collect-system-test-error-logs
git_commit_date 1741108009 1741108666
git_commit_sha 9966995 f618cc6
release_version 1.47.0-SNAPSHOT~9966995dd4 1.47.0-SNAPSHOT~f618cc6762
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1741111202 1741111202
ci_job_id 832313306 832313306
ci_pipeline_id 57678823 57678823
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-wxnla8-s-project-304-concurrent-3-5ljebolr 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-wxnla8-s-project-304-concurrent-3-5ljebolr 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 64 metrics, 7 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.47.0-SNAPSHOT~f618cc6762, baseline=1.47.0-SNAPSHOT~9966995dd4

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.142 s) : 0, 1141725
Total [baseline] (9.272 s) : 0, 9271536
Agent [candidate] (1.134 s) : 0, 1133736
Total [candidate] (9.224 s) : 0, 9223763
section iast
Agent [baseline] (1.265 s) : 0, 1264916
Total [baseline] (9.889 s) : 0, 9888974
Agent [candidate] (1.26 s) : 0, 1260109
Total [candidate] (9.873 s) : 0, 9872776
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.259 s) : 0, 1259181
Total [baseline] (9.841 s) : 0, 9840662
Agent [candidate] (1.259 s) : 0, 1258512
Total [candidate] (9.835 s) : 0, 9835361
section iast_TELEMETRY_OFF
Agent [baseline] (1.251 s) : 0, 1251327
Total [baseline] (9.89 s) : 0, 9890407
Agent [candidate] (1.26 s) : 0, 1259926
Total [candidate] (9.904 s) : 0, 9904428
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.142 s -
Agent iast 1.265 s 123.191 ms (10.8%)
Agent iast_HARDCODED_SECRET_DISABLED 1.259 s 117.456 ms (10.3%)
Agent iast_TELEMETRY_OFF 1.251 s 109.602 ms (9.6%)
Total tracing 9.272 s -
Total iast 9.889 s 617.438 ms (6.7%)
Total iast_HARDCODED_SECRET_DISABLED 9.841 s 569.126 ms (6.1%)
Total iast_TELEMETRY_OFF 9.89 s 618.871 ms (6.7%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.134 s -
Agent iast 1.26 s 126.373 ms (11.1%)
Agent iast_HARDCODED_SECRET_DISABLED 1.259 s 124.775 ms (11.0%)
Agent iast_TELEMETRY_OFF 1.26 s 126.19 ms (11.1%)
Total tracing 9.224 s -
Total iast 9.873 s 649.013 ms (7.0%)
Total iast_HARDCODED_SECRET_DISABLED 9.835 s 611.598 ms (6.6%)
Total iast_TELEMETRY_OFF 9.904 s 680.665 ms (7.4%) 
gantt
    title insecure-bank - break down per module: candidate=1.47.0-SNAPSHOT~f618cc6762, baseline=1.47.0-SNAPSHOT~9966995dd4

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (721.097 ms) : 0, 721097
BytebuddyAgent [candidate] (715.742 ms) : 0, 715742
GlobalTracer [baseline] (241.077 ms) : 0, 241077
GlobalTracer [candidate] (239.254 ms) : 0, 239254
AppSec [baseline] (56.064 ms) : 0, 56064
AppSec [candidate] (55.626 ms) : 0, 55626
Debugger [baseline] (95.49 ms) : 0, 95490
Debugger [candidate] (95.086 ms) : 0, 95086
Remote Config [baseline] (598.764 µs) : 0, 599
Remote Config [candidate] (584.867 µs) : 0, 585
Telemetry [baseline] (12.372 ms) : 0, 12372
Telemetry [candidate] (12.493 ms) : 0, 12493
section iast
BytebuddyAgent [baseline] (840.229 ms) : 0, 840229
BytebuddyAgent [candidate] (836.697 ms) : 0, 836697
GlobalTracer [baseline] (231.399 ms) : 0, 231399
GlobalTracer [candidate] (230.626 ms) : 0, 230626
IAST [baseline] (22.876 ms) : 0, 22876
IAST [candidate] (22.704 ms) : 0, 22704
AppSec [baseline] (57.225 ms) : 0, 57225
AppSec [candidate] (57.119 ms) : 0, 57119
Debugger [baseline] (83.972 ms) : 0, 83972
Debugger [candidate] (83.304 ms) : 0, 83304
Remote Config [baseline] (520.249 µs) : 0, 520
Remote Config [candidate] (517.053 µs) : 0, 517
Telemetry [baseline] (13.601 ms) : 0, 13601
Telemetry [candidate] (14.162 ms) : 0, 14162
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (834.007 ms) : 0, 834007
BytebuddyAgent [candidate] (836.126 ms) : 0, 836126
GlobalTracer [baseline] (230.376 ms) : 0, 230376
GlobalTracer [candidate] (230.847 ms) : 0, 230847
IAST [baseline] (23.731 ms) : 0, 23731
IAST [candidate] (22.881 ms) : 0, 22881
AppSec [baseline] (56.429 ms) : 0, 56429
AppSec [candidate] (56.762 ms) : 0, 56762
Debugger [baseline] (84.199 ms) : 0, 84199
Debugger [candidate] (83.527 ms) : 0, 83527
Remote Config [baseline] (509.883 µs) : 0, 510
Remote Config [candidate] (519.247 µs) : 0, 519
Telemetry [baseline] (14.964 ms) : 0, 14964
Telemetry [candidate] (12.883 ms) : 0, 12883
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (831.076 ms) : 0, 831076
BytebuddyAgent [candidate] (838.472 ms) : 0, 838472
GlobalTracer [baseline] (229.954 ms) : 0, 229954
GlobalTracer [candidate] (231.011 ms) : 0, 231011
IAST [baseline] (23.733 ms) : 0, 23733
IAST [candidate] (23.974 ms) : 0, 23974
AppSec [baseline] (55.281 ms) : 0, 55281
AppSec [candidate] (55.236 ms) : 0, 55236
Debugger [baseline] (83.792 ms) : 0, 83792
Debugger [candidate] (83.524 ms) : 0, 83524
Remote Config [baseline] (512.962 µs) : 0, 513
Remote Config [candidate] (527.623 µs) : 0, 528
Telemetry [baseline] (12.024 ms) : 0, 12024
Telemetry [candidate] (12.056 ms) : 0, 12056
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.47.0-SNAPSHOT~f618cc6762, baseline=1.47.0-SNAPSHOT~9966995dd4

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.138 s) : 0, 1137968
Total [baseline] (11.473 s) : 0, 11473318
Agent [candidate] (1.149 s) : 0, 1148972
Total [candidate] (11.464 s) : 0, 11463550
section appsec
Agent [baseline] (1.272 s) : 0, 1271712
Total [baseline] (11.668 s) : 0, 11668412
Agent [candidate] (1.273 s) : 0, 1273327
Total [candidate] (11.64 s) : 0, 11639673
section iast
Agent [baseline] (1.26 s) : 0, 1259961
Total [baseline] (11.925 s) : 0, 11925329
Agent [candidate] (1.261 s) : 0, 1260972
Total [candidate] (11.963 s) : 0, 11962567
section profiling
Agent [baseline] (1.376 s) : 0, 1376017
Total [baseline] (11.757 s) : 0, 11756703
Agent [candidate] (1.39 s) : 0, 1390279
Total [candidate] (11.901 s) : 0, 11900901
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.138 s -
Agent appsec 1.272 s 133.744 ms (11.8%)
Agent iast 1.26 s 121.993 ms (10.7%)
Agent profiling 1.376 s 238.049 ms (20.9%)
Total tracing 11.473 s -
Total appsec 11.668 s 195.095 ms (1.7%)
Total iast 11.925 s 452.012 ms (3.9%)
Total profiling 11.757 s 283.385 ms (2.5%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.149 s -
Agent appsec 1.273 s 124.356 ms (10.8%)
Agent iast 1.261 s 112.001 ms (9.7%)
Agent profiling 1.39 s 241.307 ms (21.0%)
Total tracing 11.464 s -
Total appsec 11.64 s 176.124 ms (1.5%)
Total iast 11.963 s 499.018 ms (4.4%)
Total profiling 11.901 s 437.351 ms (3.8%) 
gantt
    title petclinic - break down per module: candidate=1.47.0-SNAPSHOT~f618cc6762, baseline=1.47.0-SNAPSHOT~9966995dd4

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (718.519 ms) : 0, 718519
BytebuddyAgent [candidate] (726.013 ms) : 0, 726013
GlobalTracer [baseline] (240.113 ms) : 0, 240113
GlobalTracer [candidate] (243.137 ms) : 0, 243137
AppSec [baseline] (55.574 ms) : 0, 55574
AppSec [candidate] (55.929 ms) : 0, 55929
Debugger [baseline] (95.752 ms) : 0, 95752
Debugger [candidate] (95.688 ms) : 0, 95688
Remote Config [baseline] (592.401 µs) : 0, 592
Remote Config [candidate] (601.276 µs) : 0, 601
Telemetry [baseline] (12.433 ms) : 0, 12433
Telemetry [candidate] (12.562 ms) : 0, 12562
section appsec
BytebuddyAgent [baseline] (734.827 ms) : 0, 734827
BytebuddyAgent [candidate] (735.525 ms) : 0, 735525
GlobalTracer [baseline] (236.656 ms) : 0, 236656
GlobalTracer [candidate] (237.023 ms) : 0, 237023
IAST [baseline] (21.315 ms) : 0, 21315
IAST [candidate] (21.646 ms) : 0, 21646
AppSec [baseline] (177.403 ms) : 0, 177403
AppSec [candidate] (177.763 ms) : 0, 177763
Debugger [baseline] (84.341 ms) : 0, 84341
Debugger [candidate] (84.276 ms) : 0, 84276
Remote Config [baseline] (647.119 µs) : 0, 647
Remote Config [candidate] (631.07 µs) : 0, 631
Telemetry [baseline] (12.276 ms) : 0, 12276
Telemetry [candidate] (12.327 ms) : 0, 12327
section iast
BytebuddyAgent [baseline] (836.675 ms) : 0, 836675
BytebuddyAgent [candidate] (836.293 ms) : 0, 836293
GlobalTracer [baseline] (230.651 ms) : 0, 230651
GlobalTracer [candidate] (230.702 ms) : 0, 230702
IAST [baseline] (22.812 ms) : 0, 22812
IAST [candidate] (22.843 ms) : 0, 22843
AppSec [baseline] (57.335 ms) : 0, 57335
AppSec [candidate] (57.186 ms) : 0, 57186
Debugger [baseline] (83.305 ms) : 0, 83305
Debugger [candidate] (83.504 ms) : 0, 83504
Remote Config [baseline] (523.128 µs) : 0, 523
Remote Config [candidate] (517.197 µs) : 0, 517
Telemetry [baseline] (13.666 ms) : 0, 13666
Telemetry [candidate] (14.922 ms) : 0, 14922
section profiling
BytebuddyAgent [baseline] (706.79 ms) : 0, 706790
BytebuddyAgent [candidate] (713.727 ms) : 0, 713727
GlobalTracer [baseline] (349.423 ms) : 0, 349423
GlobalTracer [candidate] (353.174 ms) : 0, 353174
AppSec [baseline] (55.246 ms) : 0, 55246
AppSec [candidate] (55.024 ms) : 0, 55024
Debugger [baseline] (88.195 ms) : 0, 88195
Debugger [candidate] (88.959 ms) : 0, 88959
Remote Config [baseline] (619.499 µs) : 0, 619
Remote Config [candidate] (637.259 µs) : 0, 637
Telemetry [baseline] (12.489 ms) : 0, 12489
Telemetry [candidate] (12.72 ms) : 0, 12720
ProfilingAgent [baseline] (123.033 ms) : 0, 123033
ProfilingAgent [candidate] (125.354 ms) : 0, 125354
Profiling [baseline] (123.053 ms) : 0, 123053
Profiling [candidate] (125.375 ms) : 0, 125375
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2025-03-04T17:27:57 2025-03-04T17:35:51
git_branch master smola/collect-system-test-error-logs
git_commit_date 1741108009 1741108666
git_commit_sha 9966995 f618cc6
release_version 1.47.0-SNAPSHOT~9966995dd4 1.47.0-SNAPSHOT~f618cc6762
start_time 2025-03-04T17:27:43 2025-03-04T17:35:37
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1741110158 1741110158
ci_job_id 832313307 832313307
ci_pipeline_id 57678823 57678823
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-awbh27yp-project-304-concurrent-2-yjvzedpe 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-awbh27yp-project-304-concurrent-2-yjvzedpe 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 13 metrics, 17 unstable metrics.

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.47.0-SNAPSHOT~f618cc6762, baseline=1.47.0-SNAPSHOT~9966995dd4
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.345 ms) : 1326, 1364
.   : milestone, 1345,
appsec (1.737 ms) : 1714, 1761
.   : milestone, 1737,
appsec_no_iast (1.734 ms) : 1711, 1757
.   : milestone, 1734,
code_origins (1.681 ms) : 1647, 1715
.   : milestone, 1681,
iast (1.523 ms) : 1498, 1549
.   : milestone, 1523,
profiling (1.512 ms) : 1486, 1537
.   : milestone, 1512,
tracing (1.486 ms) : 1462, 1510
.   : milestone, 1486,
section candidate
no_agent (1.355 ms) : 1336, 1374
.   : milestone, 1355,
appsec (1.727 ms) : 1703, 1752
.   : milestone, 1727,
appsec_no_iast (1.721 ms) : 1698, 1744
.   : milestone, 1721,
code_origins (1.687 ms) : 1655, 1720
.   : milestone, 1687,
iast (1.521 ms) : 1497, 1546
.   : milestone, 1521,
profiling (1.531 ms) : 1505, 1557
.   : milestone, 1531,
tracing (1.483 ms) : 1459, 1507
.   : milestone, 1483,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.345 ms [1.326 ms, 1.364 ms] -
appsec 1.737 ms [1.714 ms, 1.761 ms] 392.421 µs (29.2%)
appsec_no_iast 1.734 ms [1.711 ms, 1.757 ms] 389.195 µs (28.9%)
code_origins 1.681 ms [1.647 ms, 1.715 ms] 336.01 µs (25.0%)
iast 1.523 ms [1.498 ms, 1.549 ms] 178.392 µs (13.3%)
profiling 1.512 ms [1.486 ms, 1.537 ms] 166.534 µs (12.4%)
tracing 1.486 ms [1.462 ms, 1.51 ms] 140.63 µs (10.5%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.355 ms [1.336 ms, 1.374 ms] -
appsec 1.727 ms [1.703 ms, 1.752 ms] 372.235 µs (27.5%)
appsec_no_iast 1.721 ms [1.698 ms, 1.744 ms] 366.152 µs (27.0%)
code_origins 1.687 ms [1.655 ms, 1.72 ms] 332.09 µs (24.5%)
iast 1.521 ms [1.497 ms, 1.546 ms] 166.203 µs (12.3%)
profiling 1.531 ms [1.505 ms, 1.557 ms] 176.007 µs (13.0%)
tracing 1.483 ms [1.459 ms, 1.507 ms] 128.054 µs (9.4%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.47.0-SNAPSHOT~f618cc6762, baseline=1.47.0-SNAPSHOT~9966995dd4
    dateFormat X
    axisFormat %s
section baseline
no_agent (382.401 µs) : 362, 403
.   : milestone, 382,
iast (515.907 µs) : 493, 539
.   : milestone, 516,
iast_FULL (732.221 µs) : 710, 754
.   : milestone, 732,
iast_GLOBAL (565.179 µs) : 542, 588
.   : milestone, 565,
iast_HARDCODED_SECRET_DISABLED (511.816 µs) : 490, 533
.   : milestone, 512,
iast_INACTIVE (461.669 µs) : 440, 483
.   : milestone, 462,
iast_TELEMETRY_OFF (502.983 µs) : 481, 525
.   : milestone, 503,
tracing (457.779 µs) : 437, 479
.   : milestone, 458,
section candidate
no_agent (385.663 µs) : 366, 405
.   : milestone, 386,
iast (513.922 µs) : 492, 536
.   : milestone, 514,
iast_FULL (735.683 µs) : 714, 758
.   : milestone, 736,
iast_GLOBAL (559.837 µs) : 538, 582
.   : milestone, 560,
iast_HARDCODED_SECRET_DISABLED (509.734 µs) : 488, 531
.   : milestone, 510,
iast_INACTIVE (465.6 µs) : 444, 487
.   : milestone, 466,
iast_TELEMETRY_OFF (504.429 µs) : 482, 527
.   : milestone, 504,
tracing (463.484 µs) : 441, 486
.   : milestone, 463,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 382.401 µs [362.163 µs, 402.639 µs] -
iast 515.907 µs [493.29 µs, 538.523 µs] 133.506 µs (34.9%)
iast_FULL 732.221 µs [710.072 µs, 754.37 µs] 349.82 µs (91.5%)
iast_GLOBAL 565.179 µs [542.365 µs, 587.992 µs] 182.778 µs (47.8%)
iast_HARDCODED_SECRET_DISABLED 511.816 µs [490.133 µs, 533.499 µs] 129.415 µs (33.8%)
iast_INACTIVE 461.669 µs [440.355 µs, 482.982 µs] 79.268 µs (20.7%)
iast_TELEMETRY_OFF 502.983 µs [480.753 µs, 525.214 µs] 120.583 µs (31.5%)
tracing 457.779 µs [436.839 µs, 478.72 µs] 75.378 µs (19.7%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 385.663 µs [365.969 µs, 405.358 µs] -
iast 513.922 µs [491.53 µs, 536.315 µs] 128.259 µs (33.3%)
iast_FULL 735.683 µs [713.64 µs, 757.725 µs] 350.019 µs (90.8%)
iast_GLOBAL 559.837 µs [537.514 µs, 582.159 µs] 174.173 µs (45.2%)
iast_HARDCODED_SECRET_DISABLED 509.734 µs [488.163 µs, 531.306 µs] 124.071 µs (32.2%)
iast_INACTIVE 465.6 µs [444.052 µs, 487.148 µs] 79.937 µs (20.7%)
iast_TELEMETRY_OFF 504.429 µs [481.918 µs, 526.94 µs] 118.766 µs (30.8%)
tracing 463.484 µs [441.175 µs, 485.793 µs] 77.821 µs (20.2%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/collect-system-test-error-logs
git_commit_date 1741108009 1741108666
git_commit_sha 9966995 f618cc6
release_version 1.47.0-SNAPSHOT~9966995dd4 1.47.0-SNAPSHOT~f618cc6762
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1741110675 1741110675
ci_job_id 832313309 832313309
ci_pipeline_id 57678823 57678823
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-wxnla8-s-project-304-concurrent-4-562ih9se 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-wxnla8-s-project-304-concurrent-4-562ih9se 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.47.0-SNAPSHOT~f618cc6762, baseline=1.47.0-SNAPSHOT~9966995dd4
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.265 s) : 15265000, 15265000
.   : milestone, 15265000,
appsec (14.86 s) : 14860000, 14860000
.   : milestone, 14860000,
iast (18.702 s) : 18702000, 18702000
.   : milestone, 18702000,
iast_GLOBAL (17.737 s) : 17737000, 17737000
.   : milestone, 17737000,
profiling (15.094 s) : 15094000, 15094000
.   : milestone, 15094000,
tracing (15.249 s) : 15249000, 15249000
.   : milestone, 15249000,
section candidate
no_agent (15.23 s) : 15230000, 15230000
.   : milestone, 15230000,
appsec (15.143 s) : 15143000, 15143000
.   : milestone, 15143000,
iast (19.092 s) : 19092000, 19092000
.   : milestone, 19092000,
iast_GLOBAL (17.975 s) : 17975000, 17975000
.   : milestone, 17975000,
profiling (15.605 s) : 15605000, 15605000
.   : milestone, 15605000,
tracing (14.921 s) : 14921000, 14921000
.   : milestone, 14921000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.265 s [15.265 s, 15.265 s] -
appsec 14.86 s [14.86 s, 14.86 s] -405.0 ms (-2.7%)
iast 18.702 s [18.702 s, 18.702 s] 3.437 s (22.5%)
iast_GLOBAL 17.737 s [17.737 s, 17.737 s] 2.472 s (16.2%)
profiling 15.094 s [15.094 s, 15.094 s] -171.0 ms (-1.1%)
tracing 15.249 s [15.249 s, 15.249 s] -16.0 ms (-0.1%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.23 s [15.23 s, 15.23 s] -
appsec 15.143 s [15.143 s, 15.143 s] -87.0 ms (-0.6%)
iast 19.092 s [19.092 s, 19.092 s] 3.862 s (25.4%)
iast_GLOBAL 17.975 s [17.975 s, 17.975 s] 2.745 s (18.0%)
profiling 15.605 s [15.605 s, 15.605 s] 375.0 ms (2.5%)
tracing 14.921 s [14.921 s, 14.921 s] -309.0 ms (-2.0%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.47.0-SNAPSHOT~f618cc6762, baseline=1.47.0-SNAPSHOT~9966995dd4
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.466 ms) : 1455, 1477
.   : milestone, 1466,
appsec (2.334 ms) : 2290, 2377
.   : milestone, 2334,
iast (2.109 ms) : 2054, 2164
.   : milestone, 2109,
iast_GLOBAL (2.15 ms) : 2095, 2205
.   : milestone, 2150,
profiling (1.95 ms) : 1907, 1994
.   : milestone, 1950,
tracing (1.953 ms) : 1911, 1995
.   : milestone, 1953,
section candidate
no_agent (1.471 ms) : 1459, 1482
.   : milestone, 1471,
appsec (2.327 ms) : 2284, 2370
.   : milestone, 2327,
iast (2.101 ms) : 2046, 2156
.   : milestone, 2101,
iast_GLOBAL (2.154 ms) : 2098, 2210
.   : milestone, 2154,
profiling (2.424 ms) : 2184, 2663
.   : milestone, 2424,
tracing (1.939 ms) : 1897, 1981
.   : milestone, 1939,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.466 ms [1.455 ms, 1.477 ms] -
appsec 2.334 ms [2.29 ms, 2.377 ms] 867.967 µs (59.2%)
iast 2.109 ms [2.054 ms, 2.164 ms] 643.299 µs (43.9%)
iast_GLOBAL 2.15 ms [2.095 ms, 2.205 ms] 683.892 µs (46.7%)
profiling 1.95 ms [1.907 ms, 1.994 ms] 484.469 µs (33.0%)
tracing 1.953 ms [1.911 ms, 1.995 ms] 486.969 µs (33.2%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.471 ms [1.459 ms, 1.482 ms] -
appsec 2.327 ms [2.284 ms, 2.37 ms] 856.412 µs (58.2%)
iast 2.101 ms [2.046 ms, 2.156 ms] 630.3 µs (42.9%)
iast_GLOBAL 2.154 ms [2.098 ms, 2.21 ms] 683.131 µs (46.5%)
profiling 2.424 ms [2.184 ms, 2.663 ms] 953.05 µs (64.8%)
tracing 1.939 ms [1.897 ms, 1.981 ms] 468.263 µs (31.8%)

@smola smola merged commit d8451ff into master Mar 5, 2025
209 checks passed
@smola smola deleted the smola/collect-system-test-error-logs branch March 5, 2025 08:19
@github-actions github-actions bot added this to the 1.47.0 milestone Mar 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sezen-datadog sezen-datadog sezen-datadog approved these changes

@dougqh dougqh Awaiting requested review from dougqh dougqh is a code owner automatically assigned from DataDog/apm-java

Assignees
No one assigned
Labels
comp: testing Testing tag: no release notes Changes to exclude from release notes
Projects
None yet
Milestone
1.47.0
Development

Successfully merging this pull request may close these issues.
2 participants
@smola @sezen-datadog