Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enable Java 21 in CI #6483

Merged
merged 6 commits into from
Jan 22, 2024
Merged

Enable Java 21 in CI #6483

merged 6 commits into from
Jan 22, 2024

Conversation

bantonsson
Copy link
Contributor

@bantonsson bantonsson commented Jan 12, 2024 •
edited
Loading

What Does This Do

Enables Java 21 as one of the required JVM versions in CircleCI

Motivation

We need to test on Java 21

Additional Notes

  • Corrects a number of smaller issues that failed some tests
  • Does not add support for virtual threads executors

Jira ticket: APMJAVA-1202

@pr-commenter
Copy link

pr-commenter bot commented Jan 12, 2024 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master ban/test-out-java-21
git_commit_date 1705744776 1705911480
git_commit_sha 4adcb07 60fb621
release_version 1.29.0-SNAPSHOT~4adcb07e5a 1.29.0-SNAPSHOT~60fb621b8d
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1705914160 1705914160
ci_job_id 414221197 414221197
ci_pipeline_id 26892480 26892480
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 44 metrics, 10 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.29.0-SNAPSHOT~60fb621b8d, baseline=1.29.0-SNAPSHOT~4adcb07e5a

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.06 s) : 0, 1060246
Total [baseline] (9.362 s) : 0, 9361563
Agent [candidate] (1.054 s) : 0, 1053799
Total [candidate] (9.34 s) : 0, 9339856
section appsec
Agent [baseline] (1.151 s) : 0, 1150589
Total [baseline] (9.476 s) : 0, 9475566
Agent [candidate] (1.157 s) : 0, 1157275
Total [candidate] (9.549 s) : 0, 9548630
section iast
Agent [baseline] (1.177 s) : 0, 1177010
Total [baseline] (9.669 s) : 0, 9669198
Agent [candidate] (1.185 s) : 0, 1185398
Total [candidate] (9.598 s) : 0, 9598031
section profiling
Agent [baseline] (1.281 s) : 0, 1281438
Total [baseline] (9.591 s) : 0, 9590602
Agent [candidate] (1.28 s) : 0, 1279824
Total [candidate] (9.592 s) : 0, 9592273
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.06 s -
Agent appsec 1.151 s 90.343 ms (8.5%)
Agent iast 1.177 s 116.764 ms (11.0%)
Agent profiling 1.281 s 221.192 ms (20.9%)
Total tracing 9.362 s -
Total appsec 9.476 s 114.003 ms (1.2%)
Total iast 9.669 s 307.635 ms (3.3%)
Total profiling 9.591 s 229.039 ms (2.4%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.054 s -
Agent appsec 1.157 s 103.476 ms (9.8%)
Agent iast 1.185 s 131.6 ms (12.5%)
Agent profiling 1.28 s 226.026 ms (21.4%)
Total tracing 9.34 s -
Total appsec 9.549 s 208.774 ms (2.2%)
Total iast 9.598 s 258.175 ms (2.8%)
Total profiling 9.592 s 252.417 ms (2.7%) 
gantt
    title petclinic - break down per module: candidate=1.29.0-SNAPSHOT~60fb621b8d, baseline=1.29.0-SNAPSHOT~4adcb07e5a

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (670.027 ms) : 0, 670027
BytebuddyAgent [candidate] (665.773 ms) : 0, 665773
GlobalTracer [baseline] (296.946 ms) : 0, 296946
GlobalTracer [candidate] (295.315 ms) : 0, 295315
AppSec [baseline] (50.994 ms) : 0, 50994
AppSec [candidate] (50.474 ms) : 0, 50474
Remote Config [baseline] (657.447 µs) : 0, 657
Remote Config [candidate] (653.802 µs) : 0, 654
Telemetry [baseline] (7.274 ms) : 0, 7274
Telemetry [candidate] (7.275 ms) : 0, 7275
section appsec
BytebuddyAgent [baseline] (665.396 ms) : 0, 665396
BytebuddyAgent [candidate] (667.734 ms) : 0, 667734
GlobalTracer [baseline] (294.828 ms) : 0, 294828
GlobalTracer [candidate] (298.001 ms) : 0, 298001
AppSec [baseline] (148.595 ms) : 0, 148595
AppSec [candidate] (149.504 ms) : 0, 149504
Remote Config [baseline] (636.367 µs) : 0, 636
Remote Config [candidate] (654.132 µs) : 0, 654
Telemetry [baseline] (6.874 ms) : 0, 6874
Telemetry [candidate] (6.982 ms) : 0, 6982
section iast
BytebuddyAgent [baseline] (774.749 ms) : 0, 774749
BytebuddyAgent [candidate] (781.566 ms) : 0, 781566
GlobalTracer [baseline] (286.946 ms) : 0, 286946
GlobalTracer [candidate] (288.495 ms) : 0, 288495
AppSec [baseline] (51.881 ms) : 0, 51881
AppSec [candidate] (54.327 ms) : 0, 54327
Remote Config [baseline] (634.836 µs) : 0, 635
Remote Config [candidate] (601.139 µs) : 0, 601
Telemetry [baseline] (7.455 ms) : 0, 7455
Telemetry [candidate] (7.303 ms) : 0, 7303
IAST [baseline] (21.046 ms) : 0, 21046
IAST [candidate] (18.534 ms) : 0, 18534
section profiling
BytebuddyAgent [baseline] (664.793 ms) : 0, 664793
BytebuddyAgent [candidate] (664.637 ms) : 0, 664637
GlobalTracer [baseline] (377.92 ms) : 0, 377920
GlobalTracer [candidate] (377.172 ms) : 0, 377172
AppSec [baseline] (51.026 ms) : 0, 51026
AppSec [candidate] (50.794 ms) : 0, 50794
Remote Config [baseline] (983.263 µs) : 0, 983
Remote Config [candidate] (977.759 µs) : 0, 978
Telemetry [baseline] (7.345 ms) : 0, 7345
Telemetry [candidate] (7.315 ms) : 0, 7315
ProfilingAgent [baseline] (124.933 ms) : 0, 124933
ProfilingAgent [candidate] (124.496 ms) : 0, 124496
Profiling [baseline] (124.958 ms) : 0, 124958
Profiling [candidate] (124.52 ms) : 0, 124520
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.29.0-SNAPSHOT~60fb621b8d, baseline=1.29.0-SNAPSHOT~4adcb07e5a

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.057 s) : 0, 1056785
Total [baseline] (8.742 s) : 0, 8741752
Agent [candidate] (1.07 s) : 0, 1069986
Total [candidate] (8.772 s) : 0, 8771892
section iast
Agent [baseline] (1.18 s) : 0, 1180100
Total [baseline] (9.29 s) : 0, 9290332
Agent [candidate] (1.184 s) : 0, 1184460
Total [candidate] (9.284 s) : 0, 9283572
section iast_TELEMETRY_OFF
Agent [baseline] (1.168 s) : 0, 1168477
Total [baseline] (9.275 s) : 0, 9274747
Agent [candidate] (1.168 s) : 0, 1167560
Total [candidate] (9.288 s) : 0, 9287778
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.057 s -
Agent iast 1.18 s 123.315 ms (11.7%)
Agent iast_TELEMETRY_OFF 1.168 s 111.692 ms (10.6%)
Total tracing 8.742 s -
Total iast 9.29 s 548.579 ms (6.3%)
Total iast_TELEMETRY_OFF 9.275 s 532.995 ms (6.1%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.07 s -
Agent iast 1.184 s 114.474 ms (10.7%)
Agent iast_TELEMETRY_OFF 1.168 s 97.574 ms (9.1%)
Total tracing 8.772 s -
Total iast 9.284 s 511.68 ms (5.8%)
Total iast_TELEMETRY_OFF 9.288 s 515.886 ms (5.9%) 
gantt
    title insecure-bank - break down per module: candidate=1.29.0-SNAPSHOT~60fb621b8d, baseline=1.29.0-SNAPSHOT~4adcb07e5a

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (667.43 ms) : 0, 667430
BytebuddyAgent [candidate] (676.403 ms) : 0, 676403
GlobalTracer [baseline] (296.313 ms) : 0, 296313
GlobalTracer [candidate] (299.622 ms) : 0, 299622
AppSec [baseline] (50.866 ms) : 0, 50866
AppSec [candidate] (51.158 ms) : 0, 51158
Remote Config [baseline] (655.765 µs) : 0, 656
Remote Config [candidate] (668.094 µs) : 0, 668
Telemetry [baseline] (7.234 ms) : 0, 7234
Telemetry [candidate] (7.286 ms) : 0, 7286
section iast
BytebuddyAgent [baseline] (776.656 ms) : 0, 776656
BytebuddyAgent [candidate] (781.523 ms) : 0, 781523
GlobalTracer [baseline] (287.309 ms) : 0, 287309
GlobalTracer [candidate] (287.935 ms) : 0, 287935
AppSec [baseline] (53.36 ms) : 0, 53360
AppSec [candidate] (52.935 ms) : 0, 52935
IAST [baseline] (20.431 ms) : 0, 20431
IAST [candidate] (20.284 ms) : 0, 20284
Remote Config [baseline] (606.785 µs) : 0, 607
Remote Config [candidate] (592.407 µs) : 0, 592
Telemetry [baseline] (7.356 ms) : 0, 7356
Telemetry [candidate] (6.591 ms) : 0, 6591
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (767.739 ms) : 0, 767739
BytebuddyAgent [candidate] (768.463 ms) : 0, 768463
GlobalTracer [baseline] (286.447 ms) : 0, 286447
GlobalTracer [candidate] (285.987 ms) : 0, 285987
AppSec [baseline] (55.137 ms) : 0, 55137
AppSec [candidate] (52.948 ms) : 0, 52948
IAST [baseline] (17.837 ms) : 0, 17837
IAST [candidate] (18.849 ms) : 0, 18849
Remote Config [baseline] (605.423 µs) : 0, 605
Remote Config [candidate] (615.007 µs) : 0, 615
Telemetry [baseline] (6.386 ms) : 0, 6386
Telemetry [candidate] (6.359 ms) : 0, 6359
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-01-22T08:41:57 2024-01-22T08:58:25
git_branch master ban/test-out-java-21
git_commit_date 1705744776 1705911480
git_commit_sha 4adcb07 60fb621
release_version 1.29.0-SNAPSHOT~4adcb07e5a 1.29.0-SNAPSHOT~60fb621b8d
start_time 2024-01-22T08:41:43 2024-01-22T08:58:12
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1705914160 1705914160
ci_job_id 414221197 414221197
ci_pipeline_id 26892480 26892480
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 8 metrics, 14 unstable metrics.

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.29.0-SNAPSHOT~60fb621b8d, baseline=1.29.0-SNAPSHOT~4adcb07e5a
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.343 ms) : 1323, 1362
.   : milestone, 1343,
appsec (1.749 ms) : 1724, 1775
.   : milestone, 1749,
iast (1.544 ms) : 1520, 1569
.   : milestone, 1544,
profiling (1.517 ms) : 1492, 1542
.   : milestone, 1517,
tracing (1.495 ms) : 1471, 1520
.   : milestone, 1495,
section candidate
no_agent (1.37 ms) : 1351, 1389
.   : milestone, 1370,
appsec (1.783 ms) : 1759, 1808
.   : milestone, 1783,
iast (1.529 ms) : 1505, 1554
.   : milestone, 1529,
profiling (1.53 ms) : 1503, 1557
.   : milestone, 1530,
tracing (1.504 ms) : 1479, 1529
.   : milestone, 1504,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.343 ms [1.323 ms, 1.362 ms] -
appsec 1.749 ms [1.724 ms, 1.775 ms] 406.848 µs (30.3%)
iast 1.544 ms [1.52 ms, 1.569 ms] 201.771 µs (15.0%)
profiling 1.517 ms [1.492 ms, 1.542 ms] 174.252 µs (13.0%)
tracing 1.495 ms [1.471 ms, 1.52 ms] 152.864 µs (11.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.37 ms [1.351 ms, 1.389 ms] -
appsec 1.783 ms [1.759 ms, 1.808 ms] 413.188 µs (30.2%)
iast 1.529 ms [1.505 ms, 1.554 ms] 158.98 µs (11.6%)
profiling 1.53 ms [1.503 ms, 1.557 ms] 159.699 µs (11.7%)
tracing 1.504 ms [1.479 ms, 1.529 ms] 133.699 µs (9.8%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.29.0-SNAPSHOT~60fb621b8d, baseline=1.29.0-SNAPSHOT~4adcb07e5a
    dateFormat X
    axisFormat %s
section baseline
no_agent (369.777 µs) : 349, 390
.   : milestone, 370,
iast (483.68 µs) : 463, 505
.   : milestone, 484,
iast_FULL (543.291 µs) : 523, 564
.   : milestone, 543,
iast_INACTIVE (450.33 µs) : 429, 472
.   : milestone, 450,
iast_TELEMETRY_OFF (474.23 µs) : 454, 495
.   : milestone, 474,
tracing (439.39 µs) : 419, 460
.   : milestone, 439,
section candidate
no_agent (365.511 µs) : 346, 385
.   : milestone, 366,
iast (474.977 µs) : 455, 495
.   : milestone, 475,
iast_FULL (541.052 µs) : 521, 562
.   : milestone, 541,
iast_INACTIVE (455.327 µs) : 434, 477
.   : milestone, 455,
iast_TELEMETRY_OFF (468.259 µs) : 447, 489
.   : milestone, 468,
tracing (447.074 µs) : 426, 468
.   : milestone, 447,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 369.777 µs [349.217 µs, 390.338 µs] -
iast 483.68 µs [462.666 µs, 504.694 µs] 113.903 µs (30.8%)
iast_FULL 543.291 µs [522.627 µs, 563.955 µs] 173.514 µs (46.9%)
iast_INACTIVE 450.33 µs [428.936 µs, 471.725 µs] 80.553 µs (21.8%)
iast_TELEMETRY_OFF 474.23 µs [453.568 µs, 494.892 µs] 104.453 µs (28.2%)
tracing 439.39 µs [419.009 µs, 459.772 µs] 69.613 µs (18.8%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 365.511 µs [345.819 µs, 385.202 µs] -
iast 474.977 µs [454.678 µs, 495.276 µs] 109.466 µs (29.9%)
iast_FULL 541.052 µs [520.558 µs, 561.545 µs] 175.541 µs (48.0%)
iast_INACTIVE 455.327 µs [433.863 µs, 476.79 µs] 89.816 µs (24.6%)
iast_TELEMETRY_OFF 468.259 µs [447.485 µs, 489.032 µs] 102.748 µs (28.1%)
tracing 447.074 µs [425.897 µs, 468.251 µs] 81.563 µs (22.3%)

@bantonsson bantonsson force-pushed the ban/test-out-java-21 branch 9 times, most recently from 9dc39bc to eeb256a Compare January 19, 2024 14:54
@bantonsson bantonsson force-pushed the ban/test-out-java-21 branch from 708b897 to 60fb621 Compare January 22, 2024 08:18
@bantonsson bantonsson marked this pull request as ready for review January 22, 2024 08:56
@bantonsson bantonsson requested review from a team as code owners January 22, 2024 08:56
@bantonsson bantonsson requested review from shatzi, mcculls and am312 and removed request for a team January 22, 2024 08:56
mcculls
mcculls approved these changes Jan 22, 2024
View reviewed changes
Copy link
Contributor

@mcculls mcculls left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@PerfectSlayer PerfectSlayer added the comp: tooling Build & Tooling label Jan 22, 2024
@bantonsson bantonsson added the tag: no release notes Changes to exclude from release notes label Jan 22, 2024
@bantonsson bantonsson enabled auto-merge January 22, 2024 09:04
@bantonsson bantonsson merged commit 0dc091a into master Jan 22, 2024
11 checks passed
@bantonsson bantonsson deleted the ban/test-out-java-21 branch January 22, 2024 09:06
@github-actions github-actions bot added this to the 1.28.0 milestone Jan 22, 2024
@bm1549 bm1549 mentioned this pull request Jan 29, 2024
Merged
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mcculls mcculls mcculls approved these changes

@PerfectSlayer PerfectSlayer PerfectSlayer approved these changes

@jpbempel jpbempel jpbempel approved these changes

@shatzi shatzi Awaiting requested review from shatzi shatzi is a code owner automatically assigned from DataDog/debugger-java

@am312 am312 Awaiting requested review from am312 am312 is a code owner automatically assigned from DataDog/apm-java

Assignees
No one assigned
Labels
comp: tooling Build & Tooling tag: no release notes Changes to exclude from release notes
Projects
None yet
Milestone
1.28.0
Development

Successfully merging this pull request may close these issues.
4 participants
@bantonsson @mcculls @PerfectSlayer @jpbempel