Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update system-tests to ffc1cfbe944dcf466606a84b4a442b990f675979 #8502

Merged
merged 2 commits into from
Mar 4, 2025
Merged

Update system-tests to ffc1cfbe944dcf466606a84b4a442b990f675979 #8502

merged 2 commits into from
Mar 4, 2025

Conversation

smola
Copy link
Member

@smola smola commented Mar 4, 2025

What Does This Do

Including:

Motivation

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@smola smola added comp: testing Testing tag: no release notes Changes to exclude from release notes labels Mar 4, 2025
@pr-commenter
Copy link

pr-commenter bot commented Mar 4, 2025 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/system-tests-update
git_commit_date 1741106404 1741107367
git_commit_sha a5047d5 9546a54
release_version 1.47.0-SNAPSHOT~a5047d58bf 1.47.0-SNAPSHOT~9546a54f5d
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1741110114 1741110114
ci_job_id 832253793 832253793
ci_pipeline_id 57674102 57674102
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-njn2trh-project-304-concurrent-1-w5oyw8fn 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-njn2trh-project-304-concurrent-1-w5oyw8fn 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 65 metrics, 6 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.47.0-SNAPSHOT~9546a54f5d, baseline=1.47.0-SNAPSHOT~a5047d58bf

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.132 s) : 0, 1132255
Total [baseline] (11.48 s) : 0, 11480141
Agent [candidate] (1.133 s) : 0, 1132850
Total [candidate] (11.473 s) : 0, 11473433
section appsec
Agent [baseline] (1.28 s) : 0, 1279624
Total [baseline] (11.701 s) : 0, 11700866
Agent [candidate] (1.273 s) : 0, 1272819
Total [candidate] (11.653 s) : 0, 11653495
section iast
Agent [baseline] (1.265 s) : 0, 1265193
Total [baseline] (11.95 s) : 0, 11949958
Agent [candidate] (1.275 s) : 0, 1274767
Total [candidate] (11.952 s) : 0, 11951844
section profiling
Agent [baseline] (1.382 s) : 0, 1381714
Total [baseline] (11.789 s) : 0, 11789140
Agent [candidate] (1.388 s) : 0, 1387620
Total [candidate] (11.762 s) : 0, 11761782
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.132 s -
Agent appsec 1.28 s 147.37 ms (13.0%)
Agent iast 1.265 s 132.938 ms (11.7%)
Agent profiling 1.382 s 249.459 ms (22.0%)
Total tracing 11.48 s -
Total appsec 11.701 s 220.725 ms (1.9%)
Total iast 11.95 s 469.817 ms (4.1%)
Total profiling 11.789 s 308.999 ms (2.7%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.133 s -
Agent appsec 1.273 s 139.969 ms (12.4%)
Agent iast 1.275 s 141.916 ms (12.5%)
Agent profiling 1.388 s 254.77 ms (22.5%)
Total tracing 11.473 s -
Total appsec 11.653 s 180.062 ms (1.6%)
Total iast 11.952 s 478.411 ms (4.2%)
Total profiling 11.762 s 288.349 ms (2.5%) 
gantt
    title petclinic - break down per module: candidate=1.47.0-SNAPSHOT~9546a54f5d, baseline=1.47.0-SNAPSHOT~a5047d58bf

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (714.575 ms) : 0, 714575
BytebuddyAgent [candidate] (715.177 ms) : 0, 715177
GlobalTracer [baseline] (239.001 ms) : 0, 239001
GlobalTracer [candidate] (239.19 ms) : 0, 239190
AppSec [baseline] (55.693 ms) : 0, 55693
AppSec [candidate] (55.616 ms) : 0, 55616
Debugger [baseline] (95.052 ms) : 0, 95052
Debugger [candidate] (94.889 ms) : 0, 94889
Remote Config [baseline] (587.637 µs) : 0, 588
Remote Config [candidate] (571.067 µs) : 0, 571
Telemetry [baseline] (12.402 ms) : 0, 12402
Telemetry [candidate] (12.395 ms) : 0, 12395
section appsec
BytebuddyAgent [baseline] (740.261 ms) : 0, 740261
BytebuddyAgent [candidate] (734.771 ms) : 0, 734771
GlobalTracer [baseline] (237.831 ms) : 0, 237831
GlobalTracer [candidate] (236.987 ms) : 0, 236987
IAST [baseline] (21.32 ms) : 0, 21320
IAST [candidate] (21.408 ms) : 0, 21408
AppSec [baseline] (178.284 ms) : 0, 178284
AppSec [candidate] (177.549 ms) : 0, 177549
Debugger [baseline] (84.543 ms) : 0, 84543
Debugger [candidate] (84.911 ms) : 0, 84911
Remote Config [baseline] (635.447 µs) : 0, 635
Remote Config [candidate] (630.622 µs) : 0, 631
Telemetry [baseline] (12.294 ms) : 0, 12294
Telemetry [candidate] (12.288 ms) : 0, 12288
section iast
BytebuddyAgent [baseline] (840.864 ms) : 0, 840864
BytebuddyAgent [candidate] (847.729 ms) : 0, 847729
GlobalTracer [baseline] (232.049 ms) : 0, 232049
GlobalTracer [candidate] (232.341 ms) : 0, 232341
IAST [baseline] (22.877 ms) : 0, 22877
IAST [candidate] (23.122 ms) : 0, 23122
AppSec [baseline] (57.097 ms) : 0, 57097
AppSec [candidate] (57.373 ms) : 0, 57373
Debugger [baseline] (83.613 ms) : 0, 83613
Debugger [candidate] (84.017 ms) : 0, 84017
Remote Config [baseline] (520.93 µs) : 0, 521
Remote Config [candidate] (534.85 µs) : 0, 535
Telemetry [baseline] (12.981 ms) : 0, 12981
Telemetry [candidate] (14.411 ms) : 0, 14411
section profiling
BytebuddyAgent [baseline] (709.243 ms) : 0, 709243
BytebuddyAgent [candidate] (712.941 ms) : 0, 712941
GlobalTracer [baseline] (350.96 ms) : 0, 350960
GlobalTracer [candidate] (352.809 ms) : 0, 352809
AppSec [baseline] (55.527 ms) : 0, 55527
AppSec [candidate] (54.81 ms) : 0, 54810
Debugger [baseline] (88.515 ms) : 0, 88515
Debugger [candidate] (88.694 ms) : 0, 88694
Remote Config [baseline] (604.12 µs) : 0, 604
Remote Config [candidate] (621.793 µs) : 0, 622
Telemetry [baseline] (12.492 ms) : 0, 12492
Telemetry [candidate] (12.81 ms) : 0, 12810
ProfilingAgent [baseline] (123.983 ms) : 0, 123983
ProfilingAgent [candidate] (124.197 ms) : 0, 124197
Profiling [baseline] (124.004 ms) : 0, 124004
Profiling [candidate] (124.218 ms) : 0, 124218
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.47.0-SNAPSHOT~9546a54f5d, baseline=1.47.0-SNAPSHOT~a5047d58bf

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.154 s) : 0, 1154224
Total [baseline] (9.284 s) : 0, 9284118
Agent [candidate] (1.132 s) : 0, 1131754
Total [candidate] (9.246 s) : 0, 9246038
section iast
Agent [baseline] (1.26 s) : 0, 1260238
Total [baseline] (9.908 s) : 0, 9907682
Agent [candidate] (1.256 s) : 0, 1255857
Total [candidate] (9.895 s) : 0, 9894686
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.256 s) : 0, 1256100
Total [baseline] (9.872 s) : 0, 9871939
Agent [candidate] (1.26 s) : 0, 1259719
Total [candidate] (9.841 s) : 0, 9841199
section iast_TELEMETRY_OFF
Agent [baseline] (1.263 s) : 0, 1263287
Total [baseline] (9.882 s) : 0, 9881518
Agent [candidate] (1.264 s) : 0, 1263699
Total [candidate] (9.877 s) : 0, 9877415
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.154 s -
Agent iast 1.26 s 106.014 ms (9.2%)
Agent iast_HARDCODED_SECRET_DISABLED 1.256 s 101.876 ms (8.8%)
Agent iast_TELEMETRY_OFF 1.263 s 109.063 ms (9.4%)
Total tracing 9.284 s -
Total iast 9.908 s 623.564 ms (6.7%)
Total iast_HARDCODED_SECRET_DISABLED 9.872 s 587.822 ms (6.3%)
Total iast_TELEMETRY_OFF 9.882 s 597.4 ms (6.4%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.132 s -
Agent iast 1.256 s 124.102 ms (11.0%)
Agent iast_HARDCODED_SECRET_DISABLED 1.26 s 127.965 ms (11.3%)
Agent iast_TELEMETRY_OFF 1.264 s 131.944 ms (11.7%)
Total tracing 9.246 s -
Total iast 9.895 s 648.648 ms (7.0%)
Total iast_HARDCODED_SECRET_DISABLED 9.841 s 595.161 ms (6.4%)
Total iast_TELEMETRY_OFF 9.877 s 631.377 ms (6.8%) 
gantt
    title insecure-bank - break down per module: candidate=1.47.0-SNAPSHOT~9546a54f5d, baseline=1.47.0-SNAPSHOT~a5047d58bf

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (730.708 ms) : 0, 730708
BytebuddyAgent [candidate] (714.248 ms) : 0, 714248
GlobalTracer [baseline] (243.097 ms) : 0, 243097
GlobalTracer [candidate] (239.299 ms) : 0, 239299
AppSec [baseline] (55.975 ms) : 0, 55975
AppSec [candidate] (55.54 ms) : 0, 55540
Debugger [baseline] (96.001 ms) : 0, 96001
Debugger [candidate] (94.78 ms) : 0, 94780
Remote Config [baseline] (617.905 µs) : 0, 618
Remote Config [candidate] (571.744 µs) : 0, 572
Telemetry [baseline] (12.488 ms) : 0, 12488
Telemetry [candidate] (12.306 ms) : 0, 12306
section iast
BytebuddyAgent [baseline] (836.303 ms) : 0, 836303
BytebuddyAgent [candidate] (834.52 ms) : 0, 834520
GlobalTracer [baseline] (230.572 ms) : 0, 230572
GlobalTracer [candidate] (229.852 ms) : 0, 229852
IAST [baseline] (22.783 ms) : 0, 22783
IAST [candidate] (22.626 ms) : 0, 22626
AppSec [baseline] (57.262 ms) : 0, 57262
AppSec [candidate] (57.057 ms) : 0, 57057
Debugger [baseline] (83.487 ms) : 0, 83487
Debugger [candidate] (83.311 ms) : 0, 83311
Remote Config [baseline] (519.577 µs) : 0, 520
Remote Config [candidate] (525.722 µs) : 0, 526
Telemetry [baseline] (14.236 ms) : 0, 14236
Telemetry [candidate] (12.951 ms) : 0, 12951
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (833.689 ms) : 0, 833689
BytebuddyAgent [candidate] (834.459 ms) : 0, 834459
GlobalTracer [baseline] (230.232 ms) : 0, 230232
GlobalTracer [candidate] (230.95 ms) : 0, 230950
IAST [baseline] (22.782 ms) : 0, 22782
IAST [candidate] (23.231 ms) : 0, 23231
AppSec [baseline] (56.88 ms) : 0, 56880
AppSec [candidate] (57.268 ms) : 0, 57268
Debugger [baseline] (83.566 ms) : 0, 83566
Debugger [candidate] (84.147 ms) : 0, 84147
Remote Config [baseline] (517.701 µs) : 0, 518
Remote Config [candidate] (515.027 µs) : 0, 515
Telemetry [baseline] (13.45 ms) : 0, 13450
Telemetry [candidate] (14.154 ms) : 0, 14154
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (839.849 ms) : 0, 839849
BytebuddyAgent [candidate] (840.323 ms) : 0, 840323
GlobalTracer [baseline] (231.573 ms) : 0, 231573
GlobalTracer [candidate] (231.883 ms) : 0, 231883
IAST [baseline] (23.971 ms) : 0, 23971
IAST [candidate] (22.547 ms) : 0, 22547
AppSec [baseline] (55.946 ms) : 0, 55946
AppSec [candidate] (57.411 ms) : 0, 57411
Debugger [baseline] (84.058 ms) : 0, 84058
Debugger [candidate] (83.625 ms) : 0, 83625
Remote Config [baseline] (533.121 µs) : 0, 533
Remote Config [candidate] (523.325 µs) : 0, 523
Telemetry [baseline] (12.254 ms) : 0, 12254
Telemetry [candidate] (12.119 ms) : 0, 12119
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2025-03-04T17:11:42 2025-03-04T17:19:36
git_branch master smola/system-tests-update
git_commit_date 1741106404 1741107367
git_commit_sha a5047d5 9546a54
release_version 1.47.0-SNAPSHOT~a5047d58bf 1.47.0-SNAPSHOT~9546a54f5d
start_time 2025-03-04T17:11:28 2025-03-04T17:19:22
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1741109183 1741109183
ci_job_id 832253794 832253794
ci_pipeline_id 57674102 57674102
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-njn2trh-project-304-concurrent-2-ivnkxq9k 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-njn2trh-project-304-concurrent-2-ivnkxq9k 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 14 metrics, 16 unstable metrics.

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.47.0-SNAPSHOT~9546a54f5d, baseline=1.47.0-SNAPSHOT~a5047d58bf
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.363 ms) : 1343, 1384
.   : milestone, 1363,
appsec (1.718 ms) : 1692, 1743
.   : milestone, 1718,
appsec_no_iast (1.722 ms) : 1698, 1746
.   : milestone, 1722,
code_origins (1.695 ms) : 1661, 1729
.   : milestone, 1695,
iast (1.501 ms) : 1477, 1526
.   : milestone, 1501,
profiling (1.521 ms) : 1497, 1546
.   : milestone, 1521,
tracing (1.491 ms) : 1467, 1515
.   : milestone, 1491,
section candidate
no_agent (1.351 ms) : 1331, 1370
.   : milestone, 1351,
appsec (1.728 ms) : 1703, 1754
.   : milestone, 1728,
appsec_no_iast (1.714 ms) : 1689, 1738
.   : milestone, 1714,
code_origins (1.698 ms) : 1665, 1730
.   : milestone, 1698,
iast (1.508 ms) : 1482, 1534
.   : milestone, 1508,
profiling (1.566 ms) : 1541, 1591
.   : milestone, 1566,
tracing (1.49 ms) : 1466, 1514
.   : milestone, 1490,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.363 ms [1.343 ms, 1.384 ms] -
appsec 1.718 ms [1.692 ms, 1.743 ms] 354.409 µs (26.0%)
appsec_no_iast 1.722 ms [1.698 ms, 1.746 ms] 358.826 µs (26.3%)
code_origins 1.695 ms [1.661 ms, 1.729 ms] 331.918 µs (24.4%)
iast 1.501 ms [1.477 ms, 1.526 ms] 138.326 µs (10.1%)
profiling 1.521 ms [1.497 ms, 1.546 ms] 158.105 µs (11.6%)
tracing 1.491 ms [1.467 ms, 1.515 ms] 127.687 µs (9.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.351 ms [1.331 ms, 1.37 ms] -
appsec 1.728 ms [1.703 ms, 1.754 ms] 377.867 µs (28.0%)
appsec_no_iast 1.714 ms [1.689 ms, 1.738 ms] 363.194 µs (26.9%)
code_origins 1.698 ms [1.665 ms, 1.73 ms] 347.234 µs (25.7%)
iast 1.508 ms [1.482 ms, 1.534 ms] 157.481 µs (11.7%)
profiling 1.566 ms [1.541 ms, 1.591 ms] 215.475 µs (16.0%)
tracing 1.49 ms [1.466 ms, 1.514 ms] 139.413 µs (10.3%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.47.0-SNAPSHOT~9546a54f5d, baseline=1.47.0-SNAPSHOT~a5047d58bf
    dateFormat X
    axisFormat %s
section baseline
no_agent (381.266 µs) : 361, 401
.   : milestone, 381,
iast (514.284 µs) : 492, 537
.   : milestone, 514,
iast_FULL (734.112 µs) : 712, 756
.   : milestone, 734,
iast_GLOBAL (559.934 µs) : 538, 582
.   : milestone, 560,
iast_HARDCODED_SECRET_DISABLED (512.487 µs) : 491, 534
.   : milestone, 512,
iast_INACTIVE (462.637 µs) : 441, 484
.   : milestone, 463,
iast_TELEMETRY_OFF (496.924 µs) : 475, 519
.   : milestone, 497,
tracing (458.843 µs) : 438, 480
.   : milestone, 459,
section candidate
no_agent (381.098 µs) : 361, 401
.   : milestone, 381,
iast (515.526 µs) : 493, 538
.   : milestone, 516,
iast_FULL (728.489 µs) : 706, 751
.   : milestone, 728,
iast_GLOBAL (563.751 µs) : 541, 586
.   : milestone, 564,
iast_HARDCODED_SECRET_DISABLED (509.021 µs) : 487, 531
.   : milestone, 509,
iast_INACTIVE (463.034 µs) : 442, 484
.   : milestone, 463,
iast_TELEMETRY_OFF (501.113 µs) : 479, 523
.   : milestone, 501,
tracing (455.976 µs) : 435, 477
.   : milestone, 456,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 381.266 µs [361.239 µs, 401.292 µs] -
iast 514.284 µs [491.761 µs, 536.808 µs] 133.019 µs (34.9%)
iast_FULL 734.112 µs [712.18 µs, 756.044 µs] 352.846 µs (92.5%)
iast_GLOBAL 559.934 µs [537.833 µs, 582.035 µs] 178.668 µs (46.9%)
iast_HARDCODED_SECRET_DISABLED 512.487 µs [491.047 µs, 533.927 µs] 131.221 µs (34.4%)
iast_INACTIVE 462.637 µs [441.266 µs, 484.008 µs] 81.372 µs (21.3%)
iast_TELEMETRY_OFF 496.924 µs [475.146 µs, 518.701 µs] 115.658 µs (30.3%)
tracing 458.843 µs [437.84 µs, 479.846 µs] 77.577 µs (20.3%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 381.098 µs [361.395 µs, 400.801 µs] -
iast 515.526 µs [493.468 µs, 537.583 µs] 134.428 µs (35.3%)
iast_FULL 728.489 µs [706.45 µs, 750.529 µs] 347.391 µs (91.2%)
iast_GLOBAL 563.751 µs [541.455 µs, 586.046 µs] 182.653 µs (47.9%)
iast_HARDCODED_SECRET_DISABLED 509.021 µs [487.246 µs, 530.797 µs] 127.923 µs (33.6%)
iast_INACTIVE 463.034 µs [441.894 µs, 484.175 µs] 81.936 µs (21.5%)
iast_TELEMETRY_OFF 501.113 µs [478.824 µs, 523.403 µs] 120.015 µs (31.5%)
tracing 455.976 µs [434.552 µs, 477.399 µs] 74.878 µs (19.6%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/system-tests-update
git_commit_date 1741106404 1741107367
git_commit_sha a5047d5 9546a54
release_version 1.47.0-SNAPSHOT~a5047d58bf 1.47.0-SNAPSHOT~9546a54f5d
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1741109730 1741109730
ci_job_id 832253795 832253795
ci_pipeline_id 57674102 57674102
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-njn2trh-project-304-concurrent-3-jpag182k 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-njn2trh-project-304-concurrent-3-jpag182k 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.47.0-SNAPSHOT~9546a54f5d, baseline=1.47.0-SNAPSHOT~a5047d58bf
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.474 ms) : 1463, 1486
.   : milestone, 1474,
appsec (2.348 ms) : 2305, 2392
.   : milestone, 2348,
iast (2.121 ms) : 2065, 2176
.   : milestone, 2121,
iast_GLOBAL (2.162 ms) : 2107, 2218
.   : milestone, 2162,
profiling (1.978 ms) : 1934, 2022
.   : milestone, 1978,
tracing (1.957 ms) : 1915, 2000
.   : milestone, 1957,
section candidate
no_agent (1.478 ms) : 1467, 1490
.   : milestone, 1478,
appsec (2.352 ms) : 2309, 2396
.   : milestone, 2352,
iast (2.118 ms) : 2063, 2173
.   : milestone, 2118,
iast_GLOBAL (2.165 ms) : 2109, 2220
.   : milestone, 2165,
profiling (1.971 ms) : 1927, 2015
.   : milestone, 1971,
tracing (1.965 ms) : 1923, 2008
.   : milestone, 1965,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.474 ms [1.463 ms, 1.486 ms] -
appsec 2.348 ms [2.305 ms, 2.392 ms] 874.35 µs (59.3%)
iast 2.121 ms [2.065 ms, 2.176 ms] 646.443 µs (43.9%)
iast_GLOBAL 2.162 ms [2.107 ms, 2.218 ms] 688.201 µs (46.7%)
profiling 1.978 ms [1.934 ms, 2.022 ms] 504.009 µs (34.2%)
tracing 1.957 ms [1.915 ms, 2.0 ms] 483.315 µs (32.8%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.478 ms [1.467 ms, 1.49 ms] -
appsec 2.352 ms [2.309 ms, 2.396 ms] 873.939 µs (59.1%)
iast 2.118 ms [2.063 ms, 2.173 ms] 639.272 µs (43.2%)
iast_GLOBAL 2.165 ms [2.109 ms, 2.22 ms] 686.236 µs (46.4%)
profiling 1.971 ms [1.927 ms, 2.015 ms] 492.561 µs (33.3%)
tracing 1.965 ms [1.923 ms, 2.008 ms] 486.851 µs (32.9%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.47.0-SNAPSHOT~9546a54f5d, baseline=1.47.0-SNAPSHOT~a5047d58bf
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.465 s) : 15465000, 15465000
.   : milestone, 15465000,
appsec (14.991 s) : 14991000, 14991000
.   : milestone, 14991000,
iast (18.752 s) : 18752000, 18752000
.   : milestone, 18752000,
iast_GLOBAL (18.261 s) : 18261000, 18261000
.   : milestone, 18261000,
profiling (15.038 s) : 15038000, 15038000
.   : milestone, 15038000,
tracing (14.808 s) : 14808000, 14808000
.   : milestone, 14808000,
section candidate
no_agent (15.307 s) : 15307000, 15307000
.   : milestone, 15307000,
appsec (14.942 s) : 14942000, 14942000
.   : milestone, 14942000,
iast (18.916 s) : 18916000, 18916000
.   : milestone, 18916000,
iast_GLOBAL (17.862 s) : 17862000, 17862000
.   : milestone, 17862000,
profiling (14.965 s) : 14965000, 14965000
.   : milestone, 14965000,
tracing (14.719 s) : 14719000, 14719000
.   : milestone, 14719000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.465 s [15.465 s, 15.465 s] -
appsec 14.991 s [14.991 s, 14.991 s] -474.0 ms (-3.1%)
iast 18.752 s [18.752 s, 18.752 s] 3.287 s (21.3%)
iast_GLOBAL 18.261 s [18.261 s, 18.261 s] 2.796 s (18.1%)
profiling 15.038 s [15.038 s, 15.038 s] -427.0 ms (-2.8%)
tracing 14.808 s [14.808 s, 14.808 s] -657.0 ms (-4.2%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.307 s [15.307 s, 15.307 s] -
appsec 14.942 s [14.942 s, 14.942 s] -365.0 ms (-2.4%)
iast 18.916 s [18.916 s, 18.916 s] 3.609 s (23.6%)
iast_GLOBAL 17.862 s [17.862 s, 17.862 s] 2.555 s (16.7%)
profiling 14.965 s [14.965 s, 14.965 s] -342.0 ms (-2.2%)
tracing 14.719 s [14.719 s, 14.719 s] -588.0 ms (-3.8%)

@smola smola marked this pull request as ready for review March 4, 2025 13:38
@smola smola requested a review from a team as a code owner March 4, 2025 13:38
@smola smola enabled auto-merge (squash) March 4, 2025 16:56
@smola smola merged commit 951960b into master Mar 4, 2025
209 checks passed
@smola smola deleted the smola/system-tests-update branch March 4, 2025 18:14
@github-actions github-actions bot added this to the 1.47.0 milestone Mar 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@PerfectSlayer PerfectSlayer PerfectSlayer approved these changes

@manuel-alvarez-alvarez manuel-alvarez-alvarez Awaiting requested review from manuel-alvarez-alvarez manuel-alvarez-alvarez is a code owner automatically assigned from DataDog/apm-java

Assignees
No one assigned
Labels
comp: testing Testing tag: no release notes Changes to exclude from release notes
Projects
None yet
Milestone
1.47.0
Development

Successfully merging this pull request may close these issues.
2 participants
@smola @PerfectSlayer