Implement attempt to fix tests #8393

daniel-mohedano · 2025-02-14T15:08:15Z

What Does This Do

Implements attempt to fix tests as part of Flaky Test Management. A test detected as in "Attempt to fix" will be retried a fixed amount of times to determine if its flakiness has been fixed or not.

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the public documentation in case of new configuration flag or behavior

Jira ticket: SDTEST-1636

nikita-tkachenko-datadog · 2025-02-14T15:25:19Z

...t/agent-ci-visibility/src/main/java/datadog/trace/civisibility/execution/RunNFixedTimes.java

I think we can generalise and reuse the existing RunNTimes policy. We can move the datadog.trace.civisibility.config.EarlyFlakeDetectionSettings#getExecutions method inside the policy and store the List<ExecutionsByDuration> in the instance as well. For EFD we will get the list from EFD settings, and for attempt to fix it'll be a singleton list of one element with duration Long.MAX_VALUE and the desired number of executions.

pr-commenter · 2025-02-14T15:45:48Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	daniel.mohedano/attempt-to-fix-tests
git_commit_date	1739796033	1739804743
git_commit_sha	`5b9a331`	`5548303`
release_version	1.47.0-SNAPSHOT~5b9a331fd8	1.47.0-SNAPSHOT~55483034fa

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1739807788	1739807788
ci_job_id	810864774	810864774
ci_pipeline_id	56083009	56083009
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-g-bssrbz-project-304-concurrent-4-9169moeb 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-g-bssrbz-project-304-concurrent-4-9169moeb 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 57 metrics, 5 unstable metrics.

scenario	Δ mean execution_time	candidate mean execution_time	baseline mean execution_time
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:Remote Config	better [-40.448µs; -12.506µs] or [-6.512%; -2.014%]	594.607µs	621.084µs

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.47.0-SNAPSHOT~55483034fa, baseline=1.47.0-SNAPSHOT~5b9a331fd8

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.043 s) : 0, 1043047
Total [baseline] (8.65 s) : 0, 8649963
Agent [candidate] (1.046 s) : 0, 1046166
Total [candidate] (8.643 s) : 0, 8643352
section iast
Agent [baseline] (1.172 s) : 0, 1172134
Total [baseline] (9.242 s) : 0, 9241763
Agent [candidate] (1.173 s) : 0, 1172585
Total [candidate] (9.239 s) : 0, 9238663
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.17 s) : 0, 1170493
Total [baseline] (9.226 s) : 0, 9225939
Agent [candidate] (1.17 s) : 0, 1170148
Total [candidate] (9.196 s) : 0, 9195770
section iast_TELEMETRY_OFF
Agent [baseline] (1.169 s) : 0, 1169401
Total [baseline] (9.234 s) : 0, 9234434
Agent [candidate] (1.167 s) : 0, 1166917
Total [candidate] (9.249 s) : 0, 9248959

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.043 s	-
Agent	iast	1.172 s	129.088 ms (12.4%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.17 s	127.446 ms (12.2%)
Agent	iast_TELEMETRY_OFF	1.169 s	126.355 ms (12.1%)
Total	tracing	8.65 s	-
Total	iast	9.242 s	591.8 ms (6.8%)
Total	iast_HARDCODED_SECRET_DISABLED	9.226 s	575.976 ms (6.7%)
Total	iast_TELEMETRY_OFF	9.234 s	584.472 ms (6.8%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.046 s	-
Agent	iast	1.173 s	126.419 ms (12.1%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.17 s	123.982 ms (11.9%)
Agent	iast_TELEMETRY_OFF	1.167 s	120.751 ms (11.5%)
Total	tracing	8.643 s	-
Total	iast	9.239 s	595.311 ms (6.9%)
Total	iast_HARDCODED_SECRET_DISABLED	9.196 s	552.418 ms (6.4%)
Total	iast_TELEMETRY_OFF	9.249 s	605.607 ms (7.0%)

gantt
    title insecure-bank - break down per module: candidate=1.47.0-SNAPSHOT~55483034fa, baseline=1.47.0-SNAPSHOT~5b9a331fd8

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (719.379 ms) : 0, 719379
BytebuddyAgent [candidate] (721.867 ms) : 0, 721867
GlobalTracer [baseline] (240.225 ms) : 0, 240225
GlobalTracer [candidate] (241.106 ms) : 0, 241106
AppSec [baseline] (55.483 ms) : 0, 55483
AppSec [candidate] (55.792 ms) : 0, 55792
Remote Config [baseline] (703.139 µs) : 0, 703
Remote Config [candidate] (694.571 µs) : 0, 695
Telemetry [baseline] (12.044 ms) : 0, 12044
Telemetry [candidate] (11.479 ms) : 0, 11479
section iast
BytebuddyAgent [baseline] (837.141 ms) : 0, 837141
BytebuddyAgent [candidate] (836.722 ms) : 0, 836722
GlobalTracer [baseline] (230.429 ms) : 0, 230429
GlobalTracer [candidate] (230.577 ms) : 0, 230577
AppSec [baseline] (57.367 ms) : 0, 57367
AppSec [candidate] (57.511 ms) : 0, 57511
Remote Config [baseline] (626.828 µs) : 0, 627
Remote Config [candidate] (630.484 µs) : 0, 630
Telemetry [baseline] (8.653 ms) : 0, 8653
Telemetry [candidate] (8.774 ms) : 0, 8774
IAST [baseline] (22.723 ms) : 0, 22723
IAST [candidate] (23.121 ms) : 0, 23121
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (835.247 ms) : 0, 835247
BytebuddyAgent [candidate] (834.731 ms) : 0, 834731
GlobalTracer [baseline] (230.36 ms) : 0, 230360
GlobalTracer [candidate] (230.915 ms) : 0, 230915
AppSec [baseline] (57.24 ms) : 0, 57240
AppSec [candidate] (57.081 ms) : 0, 57081
Remote Config [baseline] (626.761 µs) : 0, 627
Remote Config [candidate] (614.652 µs) : 0, 615
Telemetry [baseline] (8.795 ms) : 0, 8795
Telemetry [candidate] (8.673 ms) : 0, 8673
IAST [baseline] (22.972 ms) : 0, 22972
IAST [candidate] (22.837 ms) : 0, 22837
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (834.521 ms) : 0, 834521
BytebuddyAgent [candidate] (832.805 ms) : 0, 832805
GlobalTracer [baseline] (230.557 ms) : 0, 230557
GlobalTracer [candidate] (230.816 ms) : 0, 230816
AppSec [baseline] (56.519 ms) : 0, 56519
AppSec [candidate] (52.864 ms) : 0, 52864
Remote Config [baseline] (621.084 µs) : 0, 621
Remote Config [candidate] (594.607 µs) : 0, 595
Telemetry [baseline] (8.636 ms) : 0, 8636
Telemetry [candidate] (8.502 ms) : 0, 8502
IAST [baseline] (23.215 ms) : 0, 23215
IAST [candidate] (26.122 ms) : 0, 26122

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.47.0-SNAPSHOT~55483034fa, baseline=1.47.0-SNAPSHOT~5b9a331fd8

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.041 s) : 0, 1040649
Total [baseline] (10.483 s) : 0, 10483208
Agent [candidate] (1.046 s) : 0, 1045919
Total [candidate] (10.527 s) : 0, 10527395
section appsec
Agent [baseline] (1.181 s) : 0, 1181481
Total [baseline] (10.72 s) : 0, 10720044
Agent [candidate] (1.184 s) : 0, 1183810
Total [candidate] (10.735 s) : 0, 10735471
section iast
Agent [baseline] (1.172 s) : 0, 1172418
Total [baseline] (11.014 s) : 0, 11013910
Agent [candidate] (1.171 s) : 0, 1170999
Total [candidate] (10.975 s) : 0, 10975190
section profiling
Agent [baseline] (1.26 s) : 0, 1260390
Total [baseline] (10.859 s) : 0, 10858632
Agent [candidate] (1.269 s) : 0, 1269347
Total [candidate] (10.863 s) : 0, 10862920

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.041 s	-
Agent	appsec	1.181 s	140.832 ms (13.5%)
Agent	iast	1.172 s	131.769 ms (12.7%)
Agent	profiling	1.26 s	219.741 ms (21.1%)
Total	tracing	10.483 s	-
Total	appsec	10.72 s	236.836 ms (2.3%)
Total	iast	11.014 s	530.701 ms (5.1%)
Total	profiling	10.859 s	375.423 ms (3.6%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.046 s	-
Agent	appsec	1.184 s	137.891 ms (13.2%)
Agent	iast	1.171 s	125.08 ms (12.0%)
Agent	profiling	1.269 s	223.428 ms (21.4%)
Total	tracing	10.527 s	-
Total	appsec	10.735 s	208.076 ms (2.0%)
Total	iast	10.975 s	447.795 ms (4.3%)
Total	profiling	10.863 s	335.525 ms (3.2%)

gantt
    title petclinic - break down per module: candidate=1.47.0-SNAPSHOT~55483034fa, baseline=1.47.0-SNAPSHOT~5b9a331fd8

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (716.682 ms) : 0, 716682
BytebuddyAgent [candidate] (722.338 ms) : 0, 722338
GlobalTracer [baseline] (239.177 ms) : 0, 239177
GlobalTracer [candidate] (240.594 ms) : 0, 240594
AppSec [baseline] (55.428 ms) : 0, 55428
AppSec [candidate] (55.525 ms) : 0, 55525
Remote Config [baseline] (694.498 µs) : 0, 694
Remote Config [candidate] (686.851 µs) : 0, 687
Telemetry [baseline] (13.502 ms) : 0, 13502
Telemetry [candidate] (11.538 ms) : 0, 11538
section appsec
BytebuddyAgent [baseline] (733.608 ms) : 0, 733608
BytebuddyAgent [candidate] (736.792 ms) : 0, 736792
GlobalTracer [baseline] (236.683 ms) : 0, 236683
GlobalTracer [candidate] (236.553 ms) : 0, 236553
AppSec [baseline] (176.353 ms) : 0, 176353
AppSec [candidate] (175.544 ms) : 0, 175544
Remote Config [baseline] (671.698 µs) : 0, 672
Remote Config [candidate] (668.701 µs) : 0, 669
Telemetry [baseline] (8.234 ms) : 0, 8234
Telemetry [candidate] (8.272 ms) : 0, 8272
IAST [baseline] (21.446 ms) : 0, 21446
IAST [candidate] (21.406 ms) : 0, 21406
section iast
BytebuddyAgent [baseline] (837.153 ms) : 0, 837153
BytebuddyAgent [candidate] (836.117 ms) : 0, 836117
GlobalTracer [baseline] (230.891 ms) : 0, 230891
GlobalTracer [candidate] (230.526 ms) : 0, 230526
AppSec [baseline] (57.097 ms) : 0, 57097
AppSec [candidate] (57.069 ms) : 0, 57069
Remote Config [baseline] (626.291 µs) : 0, 626
Remote Config [candidate] (626.522 µs) : 0, 627
Telemetry [baseline] (8.618 ms) : 0, 8618
Telemetry [candidate] (8.721 ms) : 0, 8721
IAST [baseline] (22.771 ms) : 0, 22771
IAST [candidate] (22.75 ms) : 0, 22750
section profiling
BytebuddyAgent [baseline] (707.603 ms) : 0, 707603
BytebuddyAgent [candidate] (712.224 ms) : 0, 712224
GlobalTracer [baseline] (349.655 ms) : 0, 349655
GlobalTracer [candidate] (352.696 ms) : 0, 352696
AppSec [baseline] (55.127 ms) : 0, 55127
AppSec [candidate] (55.103 ms) : 0, 55103
Remote Config [baseline] (673.479 µs) : 0, 673
Remote Config [candidate] (686.637 µs) : 0, 687
Telemetry [baseline] (8.989 ms) : 0, 8989
Telemetry [candidate] (9.07 ms) : 0, 9070
ProfilingAgent [baseline] (96.042 ms) : 0, 96042
ProfilingAgent [candidate] (96.919 ms) : 0, 96919
Profiling [baseline] (96.067 ms) : 0, 96067
Profiling [candidate] (96.945 ms) : 0, 96945

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2025-02-17T15:26:17	2025-02-17T15:33:22
git_branch	master	daniel.mohedano/attempt-to-fix-tests
git_commit_date	1739796033	1739804743
git_commit_sha	`5b9a331`	`5548303`
release_version	1.47.0-SNAPSHOT~5b9a331fd8	1.47.0-SNAPSHOT~55483034fa
start_time	2025-02-17T15:26:03	2025-02-17T15:33:08

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1739806760	1739806760
ci_job_id	810864775	810864775
ci_pipeline_id	56083009	56083009
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-emwfzakh-project-304-concurrent-3-ab6p22f9 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-emwfzakh-project-304-concurrent-3-ab6p22f9 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 16 unstable metrics.

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.47.0-SNAPSHOT~55483034fa, baseline=1.47.0-SNAPSHOT~5b9a331fd8
    dateFormat X
    axisFormat %s
section baseline
no_agent (387.203 µs) : 367, 407
.   : milestone, 387,
iast (520.875 µs) : 499, 543
.   : milestone, 521,
iast_FULL (742.939 µs) : 721, 765
.   : milestone, 743,
iast_GLOBAL (554.579 µs) : 533, 577
.   : milestone, 555,
iast_HARDCODED_SECRET_DISABLED (520.867 µs) : 498, 543
.   : milestone, 521,
iast_INACTIVE (458.96 µs) : 438, 480
.   : milestone, 459,
iast_TELEMETRY_OFF (493.636 µs) : 472, 515
.   : milestone, 494,
tracing (457.498 µs) : 436, 479
.   : milestone, 457,
section candidate
no_agent (385.557 µs) : 366, 405
.   : milestone, 386,
iast (505.633 µs) : 483, 528
.   : milestone, 506,
iast_FULL (742.623 µs) : 721, 764
.   : milestone, 743,
iast_GLOBAL (554.255 µs) : 533, 576
.   : milestone, 554,
iast_HARDCODED_SECRET_DISABLED (509.624 µs) : 487, 533
.   : milestone, 510,
iast_INACTIVE (462.077 µs) : 441, 483
.   : milestone, 462,
iast_TELEMETRY_OFF (497.34 µs) : 474, 521
.   : milestone, 497,
tracing (453.718 µs) : 433, 474
.   : milestone, 454,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	387.203 µs [367.282 µs, 407.125 µs]	-
iast	520.875 µs [498.758 µs, 542.993 µs]	133.672 µs (34.5%)
iast_FULL	742.939 µs [721.022 µs, 764.855 µs]	355.735 µs (91.9%)
iast_GLOBAL	554.579 µs [532.536 µs, 576.621 µs]	167.375 µs (43.2%)
iast_HARDCODED_SECRET_DISABLED	520.867 µs [498.4 µs, 543.335 µs]	133.664 µs (34.5%)
iast_INACTIVE	458.96 µs [437.586 µs, 480.335 µs]	71.757 µs (18.5%)
iast_TELEMETRY_OFF	493.636 µs [472.192 µs, 515.079 µs]	106.432 µs (27.5%)
tracing	457.498 µs [436.465 µs, 478.532 µs]	70.295 µs (18.2%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	385.557 µs [366.033 µs, 405.08 µs]	-
iast	505.633 µs [482.937 µs, 528.328 µs]	120.076 µs (31.1%)
iast_FULL	742.623 µs [720.797 µs, 764.45 µs]	357.067 µs (92.6%)
iast_GLOBAL	554.255 µs [532.837 µs, 575.672 µs]	168.698 µs (43.8%)
iast_HARDCODED_SECRET_DISABLED	509.624 µs [486.504 µs, 532.744 µs]	124.067 µs (32.2%)
iast_INACTIVE	462.077 µs [440.718 µs, 483.437 µs]	76.521 µs (19.8%)
iast_TELEMETRY_OFF	497.34 µs [473.729 µs, 520.95 µs]	111.783 µs (29.0%)
tracing	453.718 µs [433.098 µs, 474.339 µs]	68.162 µs (17.7%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.47.0-SNAPSHOT~55483034fa, baseline=1.47.0-SNAPSHOT~5b9a331fd8
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.368 ms) : 1348, 1388
.   : milestone, 1368,
appsec (1.747 ms) : 1723, 1772
.   : milestone, 1747,
appsec_no_iast (1.757 ms) : 1732, 1783
.   : milestone, 1757,
iast (1.516 ms) : 1491, 1540
.   : milestone, 1516,
profiling (1.529 ms) : 1506, 1552
.   : milestone, 1529,
tracing (1.475 ms) : 1450, 1500
.   : milestone, 1475,
section candidate
no_agent (1.374 ms) : 1354, 1395
.   : milestone, 1374,
appsec (1.732 ms) : 1708, 1755
.   : milestone, 1732,
appsec_no_iast (1.766 ms) : 1740, 1791
.   : milestone, 1766,
iast (1.523 ms) : 1499, 1547
.   : milestone, 1523,
profiling (1.514 ms) : 1491, 1538
.   : milestone, 1514,
tracing (1.466 ms) : 1441, 1491
.   : milestone, 1466,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.368 ms [1.348 ms, 1.388 ms]	-
appsec	1.747 ms [1.723 ms, 1.772 ms]	379.425 µs (27.7%)
appsec_no_iast	1.757 ms [1.732 ms, 1.783 ms]	389.133 µs (28.4%)
iast	1.516 ms [1.491 ms, 1.54 ms]	147.654 µs (10.8%)
profiling	1.529 ms [1.506 ms, 1.552 ms]	161.23 µs (11.8%)
tracing	1.475 ms [1.45 ms, 1.5 ms]	106.744 µs (7.8%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.374 ms [1.354 ms, 1.395 ms]	-
appsec	1.732 ms [1.708 ms, 1.755 ms]	357.592 µs (26.0%)
appsec_no_iast	1.766 ms [1.74 ms, 1.791 ms]	391.555 µs (28.5%)
iast	1.523 ms [1.499 ms, 1.547 ms]	148.653 µs (10.8%)
profiling	1.514 ms [1.491 ms, 1.538 ms]	139.9 µs (10.2%)
tracing	1.466 ms [1.441 ms, 1.491 ms]	91.567 µs (6.7%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	daniel.mohedano/attempt-to-fix-tests
git_commit_date	1739796033	1739804743
git_commit_sha	`5b9a331`	`5548303`
release_version	1.47.0-SNAPSHOT~5b9a331fd8	1.47.0-SNAPSHOT~55483034fa

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1739807309	1739807309
ci_job_id	810864776	810864776
ci_pipeline_id	56083009	56083009
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-bqrkrfyj-project-304-concurrent-3-712y2prd 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-bqrkrfyj-project-304-concurrent-3-712y2prd 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant	appsec	appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.47.0-SNAPSHOT~55483034fa, baseline=1.47.0-SNAPSHOT~5b9a331fd8
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.469 ms) : 1458, 1481
.   : milestone, 1469,
appsec (2.366 ms) : 2323, 2410
.   : milestone, 2366,
iast (2.112 ms) : 2056, 2168
.   : milestone, 2112,
iast_GLOBAL (2.154 ms) : 2098, 2209
.   : milestone, 2154,
profiling (1.965 ms) : 1921, 2008
.   : milestone, 1965,
tracing (1.941 ms) : 1898, 1983
.   : milestone, 1941,
section candidate
no_agent (1.472 ms) : 1460, 1483
.   : milestone, 1472,
appsec (2.365 ms) : 2321, 2409
.   : milestone, 2365,
iast (2.113 ms) : 2057, 2169
.   : milestone, 2113,
iast_GLOBAL (2.157 ms) : 2102, 2213
.   : milestone, 2157,
profiling (1.99 ms) : 1945, 2035
.   : milestone, 1990,
tracing (1.946 ms) : 1903, 1989
.   : milestone, 1946,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.469 ms [1.458 ms, 1.481 ms]	-
appsec	2.366 ms [2.323 ms, 2.41 ms]	897.167 µs (61.1%)
iast	2.112 ms [2.056 ms, 2.168 ms]	642.321 µs (43.7%)
iast_GLOBAL	2.154 ms [2.098 ms, 2.209 ms]	684.509 µs (46.6%)
profiling	1.965 ms [1.921 ms, 2.008 ms]	495.371 µs (33.7%)
tracing	1.941 ms [1.898 ms, 1.983 ms]	471.55 µs (32.1%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.472 ms [1.46 ms, 1.483 ms]	-
appsec	2.365 ms [2.321 ms, 2.409 ms]	893.112 µs (60.7%)
iast	2.113 ms [2.057 ms, 2.169 ms]	641.266 µs (43.6%)
iast_GLOBAL	2.157 ms [2.102 ms, 2.213 ms]	685.832 µs (46.6%)
profiling	1.99 ms [1.945 ms, 2.035 ms]	518.137 µs (35.2%)
tracing	1.946 ms [1.903 ms, 1.989 ms]	474.436 µs (32.2%)

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.47.0-SNAPSHOT~55483034fa, baseline=1.47.0-SNAPSHOT~5b9a331fd8
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.937 s) : 14937000, 14937000
.   : milestone, 14937000,
appsec (14.892 s) : 14892000, 14892000
.   : milestone, 14892000,
iast (18.62 s) : 18620000, 18620000
.   : milestone, 18620000,
iast_GLOBAL (17.936 s) : 17936000, 17936000
.   : milestone, 17936000,
profiling (15.621 s) : 15621000, 15621000
.   : milestone, 15621000,
tracing (14.802 s) : 14802000, 14802000
.   : milestone, 14802000,
section candidate
no_agent (14.946 s) : 14946000, 14946000
.   : milestone, 14946000,
appsec (15.27 s) : 15270000, 15270000
.   : milestone, 15270000,
iast (18.755 s) : 18755000, 18755000
.   : milestone, 18755000,
iast_GLOBAL (18.082 s) : 18082000, 18082000
.   : milestone, 18082000,
profiling (15.13 s) : 15130000, 15130000
.   : milestone, 15130000,
tracing (14.996 s) : 14996000, 14996000
.   : milestone, 14996000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.937 s [14.937 s, 14.937 s]	-
appsec	14.892 s [14.892 s, 14.892 s]	-45.0 ms (-0.3%)
iast	18.62 s [18.62 s, 18.62 s]	3.683 s (24.7%)
iast_GLOBAL	17.936 s [17.936 s, 17.936 s]	2.999 s (20.1%)
profiling	15.621 s [15.621 s, 15.621 s]	684.0 ms (4.6%)
tracing	14.802 s [14.802 s, 14.802 s]	-135.0 ms (-0.9%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.946 s [14.946 s, 14.946 s]	-
appsec	15.27 s [15.27 s, 15.27 s]	324.0 ms (2.2%)
iast	18.755 s [18.755 s, 18.755 s]	3.809 s (25.5%)
iast_GLOBAL	18.082 s [18.082 s, 18.082 s]	3.136 s (21.0%)
profiling	15.13 s [15.13 s, 15.13 s]	184.0 ms (1.2%)
tracing	14.996 s [14.996 s, 14.996 s]	50.0 ms (0.3%)

…Test

nikita-tkachenko-datadog

Left a few minor comments, other than that LGTM

nikita-tkachenko-datadog · 2025-02-17T11:28:44Z

...nt-ci-visibility/src/main/java/datadog/trace/civisibility/config/TestManagementSettings.java

@@ -22,6 +24,10 @@ public int getAttemptToFixRetries() {
    return attemptToFixRetries;
  }

+  public List<ExecutionsByDuration> getExecutionsByDuration() {


I'd rename this to something like getAttemptToFixExecutions to emphasise that this is relevant only for attempt to fix tests

Then, optionally, to promote encapsulation, we could make TestManagementSettingsSerializer a nested class in this one, and remove the getAttemptToFixRetries method

nikita-tkachenko-datadog · 2025-02-17T11:40:56Z

...-agent/agent-ci-visibility/src/main/java/datadog/trace/civisibility/execution/RunNTimes.java

+
+  @Override
+  public boolean hasSucceededAllRetries() {
+    return currentExecutionIsLast() && successfulExecutionsSeen == maxExecutions - 1;


I don't think successfulExecutionsSeen == maxExecutions - 1 will work in all possible cases.

Imagine we have an initial value of maxExecutions as 10.
We run the test case 4 or 5 times, all of the executions are successful, but then the last one takes a while to complete (for some arbitrary reason, e.g. the network is very slow all of a sudden).
We re-evaluate max executions based on the last duration, and now the value is 2.
Given that we have already ran the test 4 times, successfulExecutionsSeen == maxExecutions - 1 is never going to be true, even if all the executions really are successful.

Yes, for now this method is only relevant for "attempted to fix" test cases, and for now the number of executions for such tests is constant. But having these "hidden assumptions" in our code makes it fragile: if things change a year from now we might not remember that there's an invariant that no longer holds.

I'd add a totalExecutionsSeen field and compare successfulExecutionsSeen with it (there are, of course, many other ways of implementing this: two boolean flags, list of execution statuses, etc - choose whichever you like the most).

nikita-tkachenko-datadog · 2025-02-17T11:44:14Z

...a-agent/agent-ci-visibility/src/main/java/datadog/trace/civisibility/config/TestSetting.java

@@ -34,7 +34,7 @@ public static boolean isSet(int mask, TestSetting setting) {
    return (mask & setting.flag) != 0;
  }

-  public static class TestSettingsSerializer {
+  public static class TestSettingSerializer {


Given that we refer to this class as TestSetting.TestSettingSerializer, should we rename it for brevity? TestSetting.Serializer looks nicer to me.
Same goes for ExecutionsByDuration.ExecutionsByDurationSerializer

nikita-tkachenko-datadog · 2025-02-17T11:50:59Z

internal-api/src/main/java/datadog/trace/api/civisibility/telemetry/tag/RetryReason.java

@@ -4,7 +4,8 @@

 public enum RetryReason implements TagValue {
  atr("Auto Test Retries"),
-  efd("Early Flakiness Detection");
+  efd("Early Flakiness Detection"),
+  attemptToFix("Attempt to Fix");


Couldn't find attemptToFix reason mentioned explicitly in the RFC. Let's talk to Andrey to get it mentioned there to ensure the value is consistent across tracers

Just checked in the confluence specs page and it is indicated to use attempt_to_fix as a retry reason, so will change it to that

| Package | Type | Package file | Manager | Update | Change | |---|---|---|---|---|---| | [com.datadoghq:dd-trace-api](https://github.com/datadog/dd-trace-java) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `1.46.1` -> `1.47.0` | | [com.datadoghq:dd-trace-ot](https://github.com/datadog/dd-trace-java) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `1.46.1` -> `1.47.0` | | [software.amazon.awssdk:sdk-core](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.30.33` -> `2.30.34` | | [software.amazon.awssdk:sqs](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.30.33` -> `2.30.34` | | [software.amazon.awssdk:dynamodb-enhanced](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.30.33` -> `2.30.34` | | [software.amazon.awssdk:dynamodb](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.30.33` -> `2.30.34` | | [software.amazon.awssdk:aws-core](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.30.33` -> `2.30.34` | | [software.amazon.awssdk:bom](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.30.33` -> `2.30.34` | | [software.amazon.awssdk:auth](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.30.33` -> `2.30.34` | --- ### Release Notes <details> <summary>datadog/dd-trace-java (com.datadoghq:dd-trace-api)</summary> ### [`v1.47.0`](https://github.com/DataDog/dd-trace-java/releases/tag/v1.47.0): 1.47.0 ##### Components ##### Application Security Management (IAST) - 🐛 Exclude com.stripe.net.HttpURLConnectionClient to solve IAST SSRF vulnerability false positives ([#8483](DataDog/dd-trace-java#8483) - [@jandro996](https://github.com/jandro996)) - 🐛 Add exclusion to solve IAST weak randomness vulnerability false positives ([#8462](DataDog/dd-trace-java#8462) - [@jandro996](https://github.com/jandro996)) - ✨ Fix weak randomness false positive in Kafka client ([#8408](DataDog/dd-trace-java#8408) - [@smola](https://github.com/smola)) - ✨ Fix location for SSRF with Kong Unirest ([#8407](DataDog/dd-trace-java#8407) - [@smola](https://github.com/smola)) - ✨ Exclude IBM Instana from IAST ([#8406](DataDog/dd-trace-java#8406) - [@smola](https://github.com/smola)) - 🐛 Fix org.json iast instrumentation test for latest dependency ([#8347](DataDog/dd-trace-java#8347) - [@jandro996](https://github.com/jandro996)) - ✨ Configuration to Disable APM Tracing ([#8219](DataDog/dd-trace-java#8219) - [@jandro996](https://github.com/jandro996)) - ✨ Address cookie vulnerability cardinality issues ([#8210](DataDog/dd-trace-java#8210) - [@jandro996](https://github.com/jandro996)) - ✨ Email HTML Injection detection in IAST ([#8205](DataDog/dd-trace-java#8205) - [@sezen-datadog](https://github.com/sezen-datadog)) ##### Application Security Management (WAF) - 🐛✨ Ensure usr.exists tag is not overridden when UsernameNotFoundException is thrown ([#8376](DataDog/dd-trace-java#8376) - [@manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) - 🐛✨ Ensure usr.exists tag is not overridden by auto instrumentation ([#8374](DataDog/dd-trace-java#8374) - [@manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) - ✨ Update appsec metrics with event_rules_version tag ([#8354](DataDog/dd-trace-java#8354) - [@sezen-datadog](https://github.com/sezen-datadog)) - ✨ Update metrics: appsec.waf.requests ([#8353](DataDog/dd-trace-java#8353) - [@Mariovido](https://github.com/Mariovido)) - ✨ Improve ASM support in vert.x 5.0 ([#8285](DataDog/dd-trace-java#8285) - [@manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) - ✨ Update metrics: appsec.waf.updates and appsec.waf.init ([#8280](DataDog/dd-trace-java#8280) - [@Mariovido](https://github.com/Mariovido)) - ✨ Configuration to Disable APM Tracing ([#8219](DataDog/dd-trace-java#8219) - [@jandro996](https://github.com/jandro996)) ##### Build & Tooling - 🐛 Do not generate Muzzle references for primitive arrays in method body ([#8361](DataDog/dd-trace-java#8361) - [@amarziali](https://github.com/amarziali)) - 📖 Improve dev env setup documentation for Windows ([#8180](DataDog/dd-trace-java#8180) - [@lucaspimentel](https://github.com/lucaspimentel)) ##### Continuous Integration Visibility - ✨ Add support for skip-EFD tagging ([#8487](DataDog/dd-trace-java#8487) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - 🐛 Fix an NPE in Gradle Android instrumentation ([#8484](DataDog/dd-trace-java#8484) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - ✨ Consider modified tests when applying fail-fast tests ordering ([#8474](DataDog/dd-trace-java#8474) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - ✨ Implement tests reordering for TestNG ([#8467](DataDog/dd-trace-java#8467) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - 🐛 Fix Gradle Launcher instrumentation to not interfere with Gradle Test Kit ([#8465](DataDog/dd-trace-java#8465) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - 🧹 Use separate TestEventHandlers per framework in CI Vis instrumentations ([#8451](DataDog/dd-trace-java#8451) - [@daniel-mohedano](https://github.com/daniel-mohedano)) - ✨ Remove warning log when JUnit 4 test method cannot be retrieved ([#8445](DataDog/dd-trace-java#8445) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - 🐛 Fix Scalatest tracing for tests that are reported asynchronously ([#8444](DataDog/dd-trace-java#8444) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - ✨ Implement attempt to fix tests ([#8393](DataDog/dd-trace-java#8393) - [@daniel-mohedano](https://github.com/daniel-mohedano)) - ✨ Implement test disabling ([#8377](DataDog/dd-trace-java#8377) - [@daniel-mohedano](https://github.com/daniel-mohedano)) - ✨ Update CODEOWNERS parser to not log errors on comments with leading whitespace ([#8349](DataDog/dd-trace-java#8349) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - ✨ Request Test Management tests list ([#8345](DataDog/dd-trace-java#8345) - [@daniel-mohedano](https://github.com/daniel-mohedano)) - ✨ Receive test management settings from CIVis settings request ([#8331](DataDog/dd-trace-java#8331) - [@daniel-mohedano](https://github.com/daniel-mohedano)) - ✨ Implement quarantined tests tagging ([#8326](DataDog/dd-trace-java#8326) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - ✨ Implement tests quarantining ([#8320](DataDog/dd-trace-java#8320) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - ✨ Add tag to specify if the user is setting DD_SERVICE ([#8318](DataDog/dd-trace-java#8318) - [@daniel-mohedano](https://github.com/daniel-mohedano)) ##### Crash tracking - ✨ Only fork jps when required ([#8419](DataDog/dd-trace-java#8419) - [@mcculls](https://github.com/mcculls)) - 🐛 Use Java home of the crashed process to launch crash uploader ([#8348](DataDog/dd-trace-java#8348) - [@jbachorik](https://github.com/jbachorik)) ##### Data Streams Monitoring - 🐛 Fix error happening when sqs message attributes are readonly ([#8473](DataDog/dd-trace-java#8473) - [@vandonr](https://github.com/vandonr)) - 🐛 Fix bug on proto schema extraction ([#8403](DataDog/dd-trace-java#8403) - [@vandonr](https://github.com/vandonr)) - 🐛 Fix service name overrides in consumers ([#8387](DataDog/dd-trace-java#8387) - [@piochelepiotr](https://github.com/piochelepiotr)) ##### Database Monitoring - ✨ Add DBMTracePreparedStatements to tracer configuration log ([#8508](DataDog/dd-trace-java#8508) - [@cecile75](https://github.com/cecile75)) ##### Dynamic Instrumentation - ✨ Look in another location for grpc service methods ([#8468](DataDog/dd-trace-java#8468) - [@evanchooly](https://github.com/evanchooly)) - 🐛 Fix Exception Replay with Lambda proxy classes ([#8452](DataDog/dd-trace-java#8452) - [@jpbempel](https://github.com/jpbempel)) - ✨ Add code origin support for spring-webmvc ([#8416](DataDog/dd-trace-java#8416) - [@evanchooly](https://github.com/evanchooly)) - ✨ Add support for scanning jar from loaded class ([#8370](DataDog/dd-trace-java#8370) - [@jpbempel](https://github.com/jpbempel)) - 🐛 Disable capture of entry values ([#8369](DataDog/dd-trace-java#8369) - [@jpbempel](https://github.com/jpbempel)) - 🐛 Fix CodeOrigin for `@Trace` annotation ([#8344](DataDog/dd-trace-java#8344) - [@jpbempel](https://github.com/jpbempel)) - 🐛 Fix equals/hashCode for CodeOrigin probe ([#8319](DataDog/dd-trace-java#8319) - [@jpbempel](https://github.com/jpbempel)) - ✨ Add code origin support to kafka message listeners ([#8301](DataDog/dd-trace-java#8301) - [@evanchooly](https://github.com/evanchooly)) ##### Metrics - ✨ Create metric: appsec.waf.error ([#8381](DataDog/dd-trace-java#8381) - [@sezen-datadog](https://github.com/sezen-datadog)) - ✨ Create metric: appsec.rasp.error ([#8364](DataDog/dd-trace-java#8364) - [@sezen-datadog](https://github.com/sezen-datadog)) ##### Profiling - ✨ Bump ddprof library to 1.22.0 ([#8463](DataDog/dd-trace-java#8463) - [@jbachorik](https://github.com/jbachorik)) - IBM J9 8u361 corresponds to OpenJDK 8u362 by [@jbachorik](https://github.com/jbachorik) in DataDog/java-profiler#187 - Fix compatibility with musl libc 1.2.4 by [@jbachorik](https://github.com/jbachorik) in DataDog/java-profiler#189 - Modify version extraction by [@jbachorik](https://github.com/jbachorik) in DataDog/java-profiler#179 - Do not write null values to jvminfo event by [@jbachorik](https://github.com/jbachorik) in DataDog/java-profiler#184 - Productize VMStructs-based stack walker by [@jbachorik](https://github.com/jbachorik) in DataDog/java-profiler#177 - A few minor downport issues by [@jbachorik](https://github.com/jbachorik) in DataDog/java-profiler#180 - Enable ASGCT by default on fairly safe J9 JDK versions by [@jbachorik](https://github.com/jbachorik) in DataDog/java-profiler#181 - 🐛 Exclude OrderedThreadPoolExecutor from queue-time measurements ([#8456](DataDog/dd-trace-java#8456) - [@jbachorik](https://github.com/jbachorik)) - ✨ Record JVM info on JVMs without JFR ([#8431](DataDog/dd-trace-java#8431) - [@jbachorik](https://github.com/jbachorik)) - 🐛 Actually use CleanupTask in TempLocationManager ([#8420](DataDog/dd-trace-java#8420) - [@mcculls](https://github.com/mcculls)) - ✨ Only fork jps when required ([#8419](DataDog/dd-trace-java#8419) - [@mcculls](https://github.com/mcculls)) - 🐛 Adjust JFR checks for J9 ([#8405](DataDog/dd-trace-java#8405) - [@jbachorik](https://github.com/jbachorik)) - 🧹 Disable smap RSS parsing by default ([#8342](DataDog/dd-trace-java#8342) - [@MattAlp](https://github.com/MattAlp)) ##### Telemetry - 🐛 Add support for JBoss jar:file format to DependencyResolver ([#8428](DataDog/dd-trace-java#8428) - [@jandro996](https://github.com/jandro996)) - ✨ Update metrics: appsec.waf.requests ([#8353](DataDog/dd-trace-java#8353) - [@Mariovido](https://github.com/Mariovido)) ##### Trace context propagation - ✨ Introduce tracing propagator ([#8313](DataDog/dd-trace-java#8313) - [@PerfectSlayer](https://github.com/PerfectSlayer)) ##### Tracer core - 🐛 Fix Stable Config telemetry source names ([#8460](DataDog/dd-trace-java#8460) - [@BaptisteFoy](https://github.com/BaptisteFoy)) - ✨ Probe trace endpoints with a valid payload of empty arrays ([#8414](DataDog/dd-trace-java#8414) - [@mcculls](https://github.com/mcculls)) - ✨ Add 1 minute fail-safe to JUL/JMX class-loading callback ([#8399](DataDog/dd-trace-java#8399) - [@mcculls](https://github.com/mcculls)) - ✨ Migrate DSM injection calls to context-first APIs ([#8383](DataDog/dd-trace-java#8383) - [@PerfectSlayer](https://github.com/PerfectSlayer)) - 🧹 Move continuation capture methods from scope to tracer ([#8371](DataDog/dd-trace-java#8371) - [@mcculls](https://github.com/mcculls)) - ✨ Migrate context extraction calls to context-first APIs ([#8368](DataDog/dd-trace-java#8368) - [@PerfectSlayer](https://github.com/PerfectSlayer)) - 🧹 Migrate context injection calls to context-first APIs ([#8358](DataDog/dd-trace-java#8358) - [@PerfectSlayer](https://github.com/PerfectSlayer)) - 💡 Support reading configurations from files ([#8338](DataDog/dd-trace-java#8338) - [@mtoffl01](https://github.com/mtoffl01)) - 💡 Implementation of BaggagePropagator and BaggageContext ([#8330](DataDog/dd-trace-java#8330) - [@mhlidd](https://github.com/mhlidd)) - 🧹 Combine continuation implementations into one which supports multiple activations ([#8324](DataDog/dd-trace-java#8324) - [@mcculls](https://github.com/mcculls)) - ✨ Introduce tracing propagator ([#8313](DataDog/dd-trace-java#8313) - [@PerfectSlayer](https://github.com/PerfectSlayer)) - ✨ Remove old context propagation API ([#8271](DataDog/dd-trace-java#8271) - [@PerfectSlayer](https://github.com/PerfectSlayer)) ##### Instrumentations ##### AWS Lambda instrumentation - 🐛 Send error message and stack to Lambda extension ([#8417](DataDog/dd-trace-java#8417) - [@nhulston](https://github.com/nhulston)) ##### AWS SDK instrumentation - 🐛 Fix error happening when sqs message attributes are readonly ([#8473](DataDog/dd-trace-java#8473) - [@vandonr](https://github.com/vandonr)) - 💡 Inject trace context into AWS Step Functions input ([#7585](DataDog/dd-trace-java#7585) - [@DylanLovesCoffee](https://github.com/DylanLovesCoffee)) ##### Core Java language instrumentation - ✨ Look in another location for grpc service methods ([#8468](DataDog/dd-trace-java#8468) - [@evanchooly](https://github.com/evanchooly)) - ✨ Add code origin support for spring-webmvc ([#8416](DataDog/dd-trace-java#8416) - [@evanchooly](https://github.com/evanchooly)) - 💡 Implementation of BaggagePropagator and BaggageContext ([#8330](DataDog/dd-trace-java#8330) - [@mhlidd](https://github.com/mhlidd)) - ✨ Add code origin support to kafka message listeners ([#8301](DataDog/dd-trace-java#8301) - [@evanchooly](https://github.com/evanchooly)) ##### gRPC instrumentation - ✨ Look in another location for grpc service methods ([#8468](DataDog/dd-trace-java#8468) - [@evanchooly](https://github.com/evanchooly)) ##### Kafka instrumentation - ✨ Add messaging.destination.name tag to kafka integrations ([#8366](DataDog/dd-trace-java#8366) - [@rarguelloF](https://github.com/rarguelloF)) ##### Protocol Buffer instrumentation - 🐛 Fix bug on proto schema extraction ([#8403](DataDog/dd-trace-java#8403) - [@vandonr](https://github.com/vandonr)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - "after 6pm every weekday,before 2am every weekday" in timezone Australia/Melbourne, Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Never, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://github.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). GitOrigin-RevId: 108a0f86aa59ab4c938cbac0688dd4c19cb301fa

daniel-mohedano added 4 commits February 14, 2025 15:17

implement attempt to fix

d264232

update integration tests

1743577

update integration test fixtures

981851f

add attempt to fix to maven test management smoke test

d4c1cc1

daniel-mohedano added type: enhancement comp: ci visibility Continuous Integration Visibility labels Feb 14, 2025

daniel-mohedano requested a review from nikita-tkachenko-datadog February 14, 2025 15:11

nikita-tkachenko-datadog reviewed Feb 14, 2025

View reviewed changes

daniel-mohedano added 6 commits February 17, 2025 10:28

fix typos

e30023c

generalize RunNTimes to be used with EFD and AttemptToFix

4c8d511

move execution policy tests from instrumentation to ExecutionStrategy…

189e97a

…Test

another typo

237029a

spotless

4e93e3c

fix visibility of methods in RunNTimes

99d9749

nikita-tkachenko-datadog approved these changes Feb 17, 2025

View reviewed changes

daniel-mohedano added 4 commits February 17, 2025 13:36

simplified test management maven smoke test

9693e42

encapsulated serializers and jsonadapters inside their classes

52c3eff

update retry reason tag value for attempt to fix

9ecb70d

update RunNTimes logic for success on all retries

5548303

daniel-mohedano marked this pull request as ready for review February 17, 2025 15:15

daniel-mohedano requested review from a team as code owners February 17, 2025 15:15

daniel-mohedano requested review from juan-fernandez and amarziali February 17, 2025 15:15

daniel-mohedano merged commit 69e0706 into master Feb 18, 2025
204 checks passed

daniel-mohedano deleted the daniel.mohedano/attempt-to-fix-tests branch February 18, 2025 08:44

github-actions bot added this to the 1.47.0 milestone Feb 18, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement attempt to fix tests #8393

Implement attempt to fix tests #8393

daniel-mohedano commented Feb 14, 2025 •

edited by jira bot

Loading

nikita-tkachenko-datadog Feb 14, 2025

pr-commenter bot commented Feb 14, 2025 •

edited

Loading

nikita-tkachenko-datadog left a comment

nikita-tkachenko-datadog Feb 17, 2025

nikita-tkachenko-datadog Feb 17, 2025

nikita-tkachenko-datadog Feb 17, 2025

nikita-tkachenko-datadog Feb 17, 2025

nikita-tkachenko-datadog Feb 17, 2025

daniel-mohedano Feb 17, 2025

Implement attempt to fix tests #8393

Implement attempt to fix tests #8393

Conversation

daniel-mohedano commented Feb 14, 2025 • edited by jira bot Loading

What Does This Do

Contributor Checklist

nikita-tkachenko-datadog Feb 14, 2025

Choose a reason for hiding this comment

pr-commenter bot commented Feb 14, 2025 • edited Loading

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

nikita-tkachenko-datadog left a comment

Choose a reason for hiding this comment

nikita-tkachenko-datadog Feb 17, 2025

Choose a reason for hiding this comment

nikita-tkachenko-datadog Feb 17, 2025

Choose a reason for hiding this comment

nikita-tkachenko-datadog Feb 17, 2025

Choose a reason for hiding this comment

nikita-tkachenko-datadog Feb 17, 2025

Choose a reason for hiding this comment

nikita-tkachenko-datadog Feb 17, 2025

Choose a reason for hiding this comment

daniel-mohedano Feb 17, 2025

Choose a reason for hiding this comment

daniel-mohedano commented Feb 14, 2025 •

edited by jira bot

Loading

pr-commenter bot commented Feb 14, 2025 •

edited

Loading