Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Hostname tests #8413

Draft
wants to merge 3 commits into
base: master
Choose a base branch
from
Draft

Hostname tests #8413

wants to merge 3 commits into from

Conversation

smola
Copy link
Member

@smola smola commented Feb 18, 2025

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@pr-commenter
Copy link

pr-commenter bot commented Feb 18, 2025 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/host-tests
git_commit_date 1741216058 1741251472
git_commit_sha 4eaa53b 59e159b
release_version 1.48.0-SNAPSHOT~4eaa53be19 1.48.0-SNAPSHOT~59e159b1d3
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1741254085 1741254085
ci_job_id 835212890 835212890
ci_pipeline_id 57901107 57901107
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-642whzhj-project-304-concurrent-4-hxsa5js3 6.8.0-1023-aws #25~22.04.1-Ubuntu SMP Tue Jan 28 12:51:22 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-642whzhj-project-304-concurrent-4-hxsa5js3 6.8.0-1023-aws #25~22.04.1-Ubuntu SMP Tue Jan 28 12:51:22 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None
variant iast iast

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 57 metrics, 5 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:startup:insecure-bank:iast:Remote Config better
[-60.761µs; -14.832µs] or [-9.596%; -2.342%]		 595.417µs 633.214µs
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.48.0-SNAPSHOT~59e159b1d3, baseline=1.48.0-SNAPSHOT~4eaa53be19

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.048 s) : 0, 1048042
Total [baseline] (10.515 s) : 0, 10515171
Agent [candidate] (1.047 s) : 0, 1046759
Total [candidate] (10.498 s) : 0, 10498447
section appsec
Agent [baseline] (1.182 s) : 0, 1181998
Total [baseline] (10.699 s) : 0, 10699054
Agent [candidate] (1.192 s) : 0, 1192377
Total [candidate] (10.83 s) : 0, 10830490
section iast
Agent [baseline] (1.176 s) : 0, 1175515
Total [baseline] (11.032 s) : 0, 11032066
Agent [candidate] (1.179 s) : 0, 1178988
Total [candidate] (11.046 s) : 0, 11045868
section profiling
Agent [baseline] (1.26 s) : 0, 1260139
Total [baseline] (10.899 s) : 0, 10899486
Agent [candidate] (1.257 s) : 0, 1257408
Total [candidate] (10.83 s) : 0, 10830482
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.048 s -
Agent appsec 1.182 s 133.957 ms (12.8%)
Agent iast 1.176 s 127.473 ms (12.2%)
Agent profiling 1.26 s 212.098 ms (20.2%)
Total tracing 10.515 s -
Total appsec 10.699 s 183.882 ms (1.7%)
Total iast 11.032 s 516.894 ms (4.9%)
Total profiling 10.899 s 384.314 ms (3.7%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.047 s -
Agent appsec 1.192 s 145.619 ms (13.9%)
Agent iast 1.179 s 132.23 ms (12.6%)
Agent profiling 1.257 s 210.65 ms (20.1%)
Total tracing 10.498 s -
Total appsec 10.83 s 332.043 ms (3.2%)
Total iast 11.046 s 547.421 ms (5.2%)
Total profiling 10.83 s 332.036 ms (3.2%) 
gantt
    title petclinic - break down per module: candidate=1.48.0-SNAPSHOT~59e159b1d3, baseline=1.48.0-SNAPSHOT~4eaa53be19

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (722.138 ms) : 0, 722138
BytebuddyAgent [candidate] (722.521 ms) : 0, 722521
GlobalTracer [baseline] (240.805 ms) : 0, 240805
GlobalTracer [candidate] (241.267 ms) : 0, 241267
AppSec [baseline] (55.679 ms) : 0, 55679
AppSec [candidate] (55.653 ms) : 0, 55653
Remote Config [baseline] (700.242 µs) : 0, 700
Remote Config [candidate] (697.326 µs) : 0, 697
Telemetry [baseline] (13.684 ms) : 0, 13684
Telemetry [candidate] (11.472 ms) : 0, 11472
section appsec
BytebuddyAgent [baseline] (734.115 ms) : 0, 734115
BytebuddyAgent [candidate] (740.573 ms) : 0, 740573
GlobalTracer [baseline] (236.027 ms) : 0, 236027
GlobalTracer [candidate] (238.704 ms) : 0, 238704
IAST [baseline] (21.366 ms) : 0, 21366
IAST [candidate] (21.861 ms) : 0, 21861
AppSec [baseline] (177.417 ms) : 0, 177417
AppSec [candidate] (178.085 ms) : 0, 178085
Remote Config [baseline] (661.372 µs) : 0, 661
Remote Config [candidate] (668.227 µs) : 0, 668
Telemetry [baseline] (8.261 ms) : 0, 8261
Telemetry [candidate] (8.412 ms) : 0, 8412
section iast
BytebuddyAgent [baseline] (837.647 ms) : 0, 837647
BytebuddyAgent [candidate] (842.765 ms) : 0, 842765
GlobalTracer [baseline] (232.758 ms) : 0, 232758
GlobalTracer [candidate] (232.18 ms) : 0, 232180
IAST [baseline] (23.354 ms) : 0, 23354
IAST [candidate] (23.022 ms) : 0, 23022
AppSec [baseline] (57.343 ms) : 0, 57343
AppSec [candidate] (56.826 ms) : 0, 56826
Remote Config [baseline] (628.355 µs) : 0, 628
Remote Config [candidate] (615.191 µs) : 0, 615
Telemetry [baseline] (8.828 ms) : 0, 8828
Telemetry [candidate] (8.668 ms) : 0, 8668
section profiling
BytebuddyAgent [baseline] (710.247 ms) : 0, 710247
BytebuddyAgent [candidate] (708.485 ms) : 0, 708485
GlobalTracer [baseline] (349.189 ms) : 0, 349189
GlobalTracer [candidate] (349.448 ms) : 0, 349448
AppSec [baseline] (54.916 ms) : 0, 54916
AppSec [candidate] (53.998 ms) : 0, 53998
Remote Config [baseline] (665.836 µs) : 0, 666
Remote Config [candidate] (664.539 µs) : 0, 665
Telemetry [baseline] (8.945 ms) : 0, 8945
Telemetry [candidate] (8.826 ms) : 0, 8826
ProfilingAgent [baseline] (95.658 ms) : 0, 95658
ProfilingAgent [candidate] (95.707 ms) : 0, 95707
Profiling [baseline] (95.684 ms) : 0, 95684
Profiling [candidate] (95.731 ms) : 0, 95731
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.48.0-SNAPSHOT~59e159b1d3, baseline=1.48.0-SNAPSHOT~4eaa53be19

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.043 s) : 0, 1042751
Total [baseline] (8.672 s) : 0, 8672271
Agent [candidate] (1.052 s) : 0, 1051952
Total [candidate] (8.694 s) : 0, 8694495
section iast
Agent [baseline] (1.189 s) : 0, 1189178
Total [baseline] (9.256 s) : 0, 9255927
Agent [candidate] (1.169 s) : 0, 1168683
Total [candidate] (9.272 s) : 0, 9271638
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.17 s) : 0, 1169508
Total [baseline] (9.182 s) : 0, 9182454
Agent [candidate] (1.172 s) : 0, 1171797
Total [candidate] (9.179 s) : 0, 9179028
section iast_TELEMETRY_OFF
Agent [baseline] (1.169 s) : 0, 1168870
Total [baseline] (9.271 s) : 0, 9270576
Agent [candidate] (1.176 s) : 0, 1176465
Total [candidate] (9.235 s) : 0, 9234930
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.043 s -
Agent iast 1.189 s 146.427 ms (14.0%)
Agent iast_HARDCODED_SECRET_DISABLED 1.17 s 126.757 ms (12.2%)
Agent iast_TELEMETRY_OFF 1.169 s 126.119 ms (12.1%)
Total tracing 8.672 s -
Total iast 9.256 s 583.656 ms (6.7%)
Total iast_HARDCODED_SECRET_DISABLED 9.182 s 510.184 ms (5.9%)
Total iast_TELEMETRY_OFF 9.271 s 598.305 ms (6.9%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.052 s -
Agent iast 1.169 s 116.731 ms (11.1%)
Agent iast_HARDCODED_SECRET_DISABLED 1.172 s 119.846 ms (11.4%)
Agent iast_TELEMETRY_OFF 1.176 s 124.514 ms (11.8%)
Total tracing 8.694 s -
Total iast 9.272 s 577.143 ms (6.6%)
Total iast_HARDCODED_SECRET_DISABLED 9.179 s 484.533 ms (5.6%)
Total iast_TELEMETRY_OFF 9.235 s 540.436 ms (6.2%) 
gantt
    title insecure-bank - break down per module: candidate=1.48.0-SNAPSHOT~59e159b1d3, baseline=1.48.0-SNAPSHOT~4eaa53be19

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (718.647 ms) : 0, 718647
BytebuddyAgent [candidate] (727.276 ms) : 0, 727276
GlobalTracer [baseline] (239.78 ms) : 0, 239780
GlobalTracer [candidate] (242.134 ms) : 0, 242134
AppSec [baseline] (55.204 ms) : 0, 55204
AppSec [candidate] (55.899 ms) : 0, 55899
Remote Config [baseline] (688.666 µs) : 0, 689
Remote Config [candidate] (704.618 µs) : 0, 705
Telemetry [baseline] (13.44 ms) : 0, 13440
Telemetry [candidate] (10.83 ms) : 0, 10830
section iast
BytebuddyAgent [baseline] (852.75 ms) : 0, 852750
BytebuddyAgent [candidate] (835.546 ms) : 0, 835546
GlobalTracer [baseline] (231.153 ms) : 0, 231153
GlobalTracer [candidate] (230.093 ms) : 0, 230093
AppSec [baseline] (57.403 ms) : 0, 57403
AppSec [candidate] (55.435 ms) : 0, 55435
Remote Config [baseline] (633.214 µs) : 0, 633
Remote Config [candidate] (595.417 µs) : 0, 595
Telemetry [baseline] (8.828 ms) : 0, 8828
Telemetry [candidate] (8.58 ms) : 0, 8580
IAST [baseline] (23.152 ms) : 0, 23152
IAST [candidate] (23.467 ms) : 0, 23467
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (835.732 ms) : 0, 835732
BytebuddyAgent [candidate] (837.129 ms) : 0, 837129
GlobalTracer [baseline] (230.109 ms) : 0, 230109
GlobalTracer [candidate] (230.963 ms) : 0, 230963
AppSec [baseline] (55.745 ms) : 0, 55745
AppSec [candidate] (56.581 ms) : 0, 56581
Remote Config [baseline] (607.913 µs) : 0, 608
Remote Config [candidate] (597.898 µs) : 0, 598
Telemetry [baseline] (8.724 ms) : 0, 8724
Telemetry [candidate] (8.715 ms) : 0, 8715
IAST [baseline] (23.677 ms) : 0, 23677
IAST [candidate] (22.848 ms) : 0, 22848
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (834.931 ms) : 0, 834931
BytebuddyAgent [candidate] (841.612 ms) : 0, 841612
GlobalTracer [baseline] (230.317 ms) : 0, 230317
GlobalTracer [candidate] (231.124 ms) : 0, 231124
AppSec [baseline] (56.836 ms) : 0, 56836
AppSec [candidate] (56.058 ms) : 0, 56058
Remote Config [baseline] (611.35 µs) : 0, 611
Remote Config [candidate] (606.196 µs) : 0, 606
Telemetry [baseline] (8.695 ms) : 0, 8695
Telemetry [candidate] (8.696 ms) : 0, 8696
IAST [baseline] (22.528 ms) : 0, 22528
IAST [candidate] (23.254 ms) : 0, 23254
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2025-03-06T09:11:37 2025-03-06T09:19:23
git_branch master smola/host-tests
git_commit_date 1741216058 1741251472
git_commit_sha 4eaa53b 59e159b
release_version 1.48.0-SNAPSHOT~4eaa53be19 1.48.0-SNAPSHOT~59e159b1d3
start_time 2025-03-06T09:11:23 2025-03-06T09:19:09
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1741253163 1741253163
ci_job_id 835212891 835212891
ci_pipeline_id 57901107 57901107
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-3ysmm6qi-project-304-concurrent-2-f8pmnewh 6.8.0-1023-aws #25~22.04.1-Ubuntu SMP Tue Jan 28 12:51:22 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-3ysmm6qi-project-304-concurrent-2-f8pmnewh 6.8.0-1023-aws #25~22.04.1-Ubuntu SMP Tue Jan 28 12:51:22 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 14 metrics, 16 unstable metrics.

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.48.0-SNAPSHOT~59e159b1d3, baseline=1.48.0-SNAPSHOT~4eaa53be19
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.367 ms) : 1346, 1387
.   : milestone, 1367,
appsec (1.744 ms) : 1720, 1767
.   : milestone, 1744,
appsec_no_iast (1.734 ms) : 1708, 1759
.   : milestone, 1734,
code_origins (1.707 ms) : 1673, 1740
.   : milestone, 1707,
iast (1.523 ms) : 1499, 1547
.   : milestone, 1523,
profiling (1.512 ms) : 1488, 1536
.   : milestone, 1512,
tracing (1.488 ms) : 1463, 1513
.   : milestone, 1488,
section candidate
no_agent (1.35 ms) : 1331, 1370
.   : milestone, 1350,
appsec (1.739 ms) : 1715, 1763
.   : milestone, 1739,
appsec_no_iast (1.747 ms) : 1722, 1771
.   : milestone, 1747,
code_origins (1.708 ms) : 1675, 1742
.   : milestone, 1708,
iast (1.535 ms) : 1511, 1559
.   : milestone, 1535,
profiling (1.496 ms) : 1473, 1520
.   : milestone, 1496,
tracing (1.502 ms) : 1477, 1527
.   : milestone, 1502,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.367 ms [1.346 ms, 1.387 ms] -
appsec 1.744 ms [1.72 ms, 1.767 ms] 376.844 µs (27.6%)
appsec_no_iast 1.734 ms [1.708 ms, 1.759 ms] 367.154 µs (26.9%)
code_origins 1.707 ms [1.673 ms, 1.74 ms] 339.872 µs (24.9%)
iast 1.523 ms [1.499 ms, 1.547 ms] 156.221 µs (11.4%)
profiling 1.512 ms [1.488 ms, 1.536 ms] 145.175 µs (10.6%)
tracing 1.488 ms [1.463 ms, 1.513 ms] 121.202 µs (8.9%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.35 ms [1.331 ms, 1.37 ms] -
appsec 1.739 ms [1.715 ms, 1.763 ms] 388.892 µs (28.8%)
appsec_no_iast 1.747 ms [1.722 ms, 1.771 ms] 396.386 µs (29.4%)
code_origins 1.708 ms [1.675 ms, 1.742 ms] 358.11 µs (26.5%)
iast 1.535 ms [1.511 ms, 1.559 ms] 184.878 µs (13.7%)
profiling 1.496 ms [1.473 ms, 1.52 ms] 146.225 µs (10.8%)
tracing 1.502 ms [1.477 ms, 1.527 ms] 151.726 µs (11.2%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.48.0-SNAPSHOT~59e159b1d3, baseline=1.48.0-SNAPSHOT~4eaa53be19
    dateFormat X
    axisFormat %s
section baseline
no_agent (385.027 µs) : 365, 405
.   : milestone, 385,
iast (516.07 µs) : 494, 538
.   : milestone, 516,
iast_FULL (737.318 µs) : 715, 759
.   : milestone, 737,
iast_GLOBAL (561.405 µs) : 539, 584
.   : milestone, 561,
iast_HARDCODED_SECRET_DISABLED (517.829 µs) : 496, 540
.   : milestone, 518,
iast_INACTIVE (465.881 µs) : 445, 487
.   : milestone, 466,
iast_TELEMETRY_OFF (505.626 µs) : 483, 529
.   : milestone, 506,
tracing (464.993 µs) : 444, 486
.   : milestone, 465,
section candidate
no_agent (381.14 µs) : 361, 401
.   : milestone, 381,
iast (517.27 µs) : 496, 539
.   : milestone, 517,
iast_FULL (734.831 µs) : 713, 757
.   : milestone, 735,
iast_GLOBAL (567.891 µs) : 546, 590
.   : milestone, 568,
iast_HARDCODED_SECRET_DISABLED (518.166 µs) : 496, 540
.   : milestone, 518,
iast_INACTIVE (460.603 µs) : 440, 482
.   : milestone, 461,
iast_TELEMETRY_OFF (506.093 µs) : 483, 529
.   : milestone, 506,
tracing (460.846 µs) : 440, 482
.   : milestone, 461,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 385.027 µs [365.268 µs, 404.786 µs] -
iast 516.07 µs [494.102 µs, 538.038 µs] 131.043 µs (34.0%)
iast_FULL 737.318 µs [715.138 µs, 759.497 µs] 352.291 µs (91.5%)
iast_GLOBAL 561.405 µs [539.062 µs, 583.748 µs] 176.378 µs (45.8%)
iast_HARDCODED_SECRET_DISABLED 517.829 µs [495.843 µs, 539.816 µs] 132.802 µs (34.5%)
iast_INACTIVE 465.881 µs [444.556 µs, 487.206 µs] 80.854 µs (21.0%)
iast_TELEMETRY_OFF 505.626 µs [482.731 µs, 528.521 µs] 120.599 µs (31.3%)
tracing 464.993 µs [444.063 µs, 485.923 µs] 79.966 µs (20.8%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 381.14 µs [361.455 µs, 400.825 µs] -
iast 517.27 µs [495.599 µs, 538.942 µs] 136.13 µs (35.7%)
iast_FULL 734.831 µs [712.586 µs, 757.077 µs] 353.691 µs (92.8%)
iast_GLOBAL 567.891 µs [545.907 µs, 589.875 µs] 186.751 µs (49.0%)
iast_HARDCODED_SECRET_DISABLED 518.166 µs [496.422 µs, 539.911 µs] 137.026 µs (36.0%)
iast_INACTIVE 460.603 µs [439.661 µs, 481.544 µs] 79.463 µs (20.8%)
iast_TELEMETRY_OFF 506.093 µs [482.909 µs, 529.277 µs] 124.953 µs (32.8%)
tracing 460.846 µs [439.503 µs, 482.188 µs] 79.705 µs (20.9%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/host-tests
git_commit_date 1741216058 1741251472
git_commit_sha 4eaa53b 59e159b
release_version 1.48.0-SNAPSHOT~4eaa53be19 1.48.0-SNAPSHOT~59e159b1d3
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1741253659 1741253659
ci_job_id 835212892 835212892
ci_pipeline_id 57901107 57901107
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-642whzhj-project-304-concurrent-5-6e3jzbhl 6.8.0-1023-aws #25~22.04.1-Ubuntu SMP Tue Jan 28 12:51:22 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-642whzhj-project-304-concurrent-5-6e3jzbhl 6.8.0-1023-aws #25~22.04.1-Ubuntu SMP Tue Jan 28 12:51:22 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.48.0-SNAPSHOT~59e159b1d3, baseline=1.48.0-SNAPSHOT~4eaa53be19
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.291 s) : 15291000, 15291000
.   : milestone, 15291000,
appsec (15.172 s) : 15172000, 15172000
.   : milestone, 15172000,
iast (19.083 s) : 19083000, 19083000
.   : milestone, 19083000,
iast_GLOBAL (18.024 s) : 18024000, 18024000
.   : milestone, 18024000,
profiling (14.904 s) : 14904000, 14904000
.   : milestone, 14904000,
tracing (14.922 s) : 14922000, 14922000
.   : milestone, 14922000,
section candidate
no_agent (15.35 s) : 15350000, 15350000
.   : milestone, 15350000,
appsec (15.184 s) : 15184000, 15184000
.   : milestone, 15184000,
iast (18.772 s) : 18772000, 18772000
.   : milestone, 18772000,
iast_GLOBAL (18.261 s) : 18261000, 18261000
.   : milestone, 18261000,
profiling (15.427 s) : 15427000, 15427000
.   : milestone, 15427000,
tracing (15.015 s) : 15015000, 15015000
.   : milestone, 15015000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.291 s [15.291 s, 15.291 s] -
appsec 15.172 s [15.172 s, 15.172 s] -119.0 ms (-0.8%)
iast 19.083 s [19.083 s, 19.083 s] 3.792 s (24.8%)
iast_GLOBAL 18.024 s [18.024 s, 18.024 s] 2.733 s (17.9%)
profiling 14.904 s [14.904 s, 14.904 s] -387.0 ms (-2.5%)
tracing 14.922 s [14.922 s, 14.922 s] -369.0 ms (-2.4%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.35 s [15.35 s, 15.35 s] -
appsec 15.184 s [15.184 s, 15.184 s] -166.0 ms (-1.1%)
iast 18.772 s [18.772 s, 18.772 s] 3.422 s (22.3%)
iast_GLOBAL 18.261 s [18.261 s, 18.261 s] 2.911 s (19.0%)
profiling 15.427 s [15.427 s, 15.427 s] 77.0 ms (0.5%)
tracing 15.015 s [15.015 s, 15.015 s] -335.0 ms (-2.2%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.48.0-SNAPSHOT~59e159b1d3, baseline=1.48.0-SNAPSHOT~4eaa53be19
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.476 ms) : 1465, 1488
.   : milestone, 1476,
appsec (2.328 ms) : 2285, 2371
.   : milestone, 2328,
iast (2.11 ms) : 2055, 2164
.   : milestone, 2110,
iast_GLOBAL (2.155 ms) : 2100, 2210
.   : milestone, 2155,
profiling (1.981 ms) : 1937, 2026
.   : milestone, 1981,
tracing (1.947 ms) : 1905, 1989
.   : milestone, 1947,
section candidate
no_agent (1.473 ms) : 1462, 1485
.   : milestone, 1473,
appsec (2.334 ms) : 2291, 2377
.   : milestone, 2334,
iast (2.103 ms) : 2049, 2158
.   : milestone, 2103,
iast_GLOBAL (2.153 ms) : 2098, 2208
.   : milestone, 2153,
profiling (1.972 ms) : 1928, 2015
.   : milestone, 1972,
tracing (1.951 ms) : 1909, 1993
.   : milestone, 1951,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.476 ms [1.465 ms, 1.488 ms] -
appsec 2.328 ms [2.285 ms, 2.371 ms] 852.328 µs (57.7%)
iast 2.11 ms [2.055 ms, 2.164 ms] 633.617 µs (42.9%)
iast_GLOBAL 2.155 ms [2.1 ms, 2.21 ms] 678.824 µs (46.0%)
profiling 1.981 ms [1.937 ms, 2.026 ms] 505.054 µs (34.2%)
tracing 1.947 ms [1.905 ms, 1.989 ms] 470.757 µs (31.9%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.473 ms [1.462 ms, 1.485 ms] -
appsec 2.334 ms [2.291 ms, 2.377 ms] 860.733 µs (58.4%)
iast 2.103 ms [2.049 ms, 2.158 ms] 629.833 µs (42.7%)
iast_GLOBAL 2.153 ms [2.098 ms, 2.208 ms] 679.562 µs (46.1%)
profiling 1.972 ms [1.928 ms, 2.015 ms] 498.354 µs (33.8%)
tracing 1.951 ms [1.909 ms, 1.993 ms] 477.87 µs (32.4%)

@PerfectSlayer
Copy link
Contributor

Can we close this one then?

@smola
Copy link
Member Author

smola commented Feb 20, 2025

@PerfectSlayer I'm considering actually merging these changes, to always test all fallbacks and ensure they all give the same result. If most of our test runs are always stopping at HOSTNAME, we won't be validating that the fallback mechanisms work as we expected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@smola @PerfectSlayer