Add API to capture logs in unit tests #8513

smola · 2025-03-05T12:11:22Z

What Does This Do

Add an API to capture logs during unit test.
To enable it, TestLogbackAppender must be setup in src/test/resources/logback-test.xml for the module. There is an integration example in this PR.
TestLogCollector can be used during tests. enable() is used to start capturing, drainCapturedLogs() to retrieve captured logs, and disable() to stop capturing.
enable() will fail if it was not disabled in the previous step. This should help avoiding tests keeping capture enabled for the rest of the test run.
This will NOT work with parallel tests.

Motivation

Asserting logs, especially in unit test modules where telemetry is not setup and there are no better alternatives.

Additional Notes

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

PerfectSlayer

Dumb question: how is this going to work with parallel test execution? -- not sure we use them though 🤔

dd-java-agent/agent-iast/build.gradle

utils/test-utils/build.gradle

utils/test-utils/src/main/java/datadog/trace/test/logging/TestLogCollector.java

smola · 2025-03-05T12:49:32Z

@PerfectSlayer This will definitely not work with parallel tests. Best case, the second test that starts, will fail on enable() (this is intended, to catch these issues early). AFAIK we do not parallelize any test within a module, but if we did, any test relying on this would need to be not parallelized.

pr-commenter · 2025-03-05T12:51:29Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	smola/capture-test-logs
git_commit_date	1741171359	1741179470
git_commit_sha	`a951656`	`e8145ff`
release_version	1.47.0-SNAPSHOT~a951656fc4	1.48.0-SNAPSHOT~e8145ffd98

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1741181785	1741181785
ci_job_id	833525099	833525099
ci_pipeline_id	57776923	57776923
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-snmtyxn8-project-304-concurrent-2-u4epouzq 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-snmtyxn8-project-304-concurrent-2-u4epouzq 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 58 metrics, 5 unstable metrics.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.48.0-SNAPSHOT~e8145ffd98, baseline=1.47.0-SNAPSHOT~a951656fc4

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.044 s) : 0, 1043934
Total [baseline] (8.682 s) : 0, 8682256
Agent [candidate] (1.042 s) : 0, 1041705
Total [candidate] (8.664 s) : 0, 8664144
section iast
Agent [baseline] (1.169 s) : 0, 1169226
Total [baseline] (9.205 s) : 0, 9205384
Agent [candidate] (1.172 s) : 0, 1171545
Total [candidate] (9.228 s) : 0, 9228412
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.174 s) : 0, 1174497
Total [baseline] (9.203 s) : 0, 9202971
Agent [candidate] (1.17 s) : 0, 1169982
Total [candidate] (9.152 s) : 0, 9151919
section iast_TELEMETRY_OFF
Agent [baseline] (1.164 s) : 0, 1164269
Total [baseline] (9.254 s) : 0, 9254214
Agent [candidate] (1.173 s) : 0, 1172842
Total [candidate] (9.252 s) : 0, 9251702

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.044 s	-
Agent	iast	1.169 s	125.292 ms (12.0%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.174 s	130.563 ms (12.5%)
Agent	iast_TELEMETRY_OFF	1.164 s	120.335 ms (11.5%)
Total	tracing	8.682 s	-
Total	iast	9.205 s	523.128 ms (6.0%)
Total	iast_HARDCODED_SECRET_DISABLED	9.203 s	520.715 ms (6.0%)
Total	iast_TELEMETRY_OFF	9.254 s	571.958 ms (6.6%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.042 s	-
Agent	iast	1.172 s	129.841 ms (12.5%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.17 s	128.277 ms (12.3%)
Agent	iast_TELEMETRY_OFF	1.173 s	131.137 ms (12.6%)
Total	tracing	8.664 s	-
Total	iast	9.228 s	564.268 ms (6.5%)
Total	iast_HARDCODED_SECRET_DISABLED	9.152 s	487.775 ms (5.6%)
Total	iast_TELEMETRY_OFF	9.252 s	587.558 ms (6.8%)

gantt
    title insecure-bank - break down per module: candidate=1.48.0-SNAPSHOT~e8145ffd98, baseline=1.47.0-SNAPSHOT~a951656fc4

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (720.237 ms) : 0, 720237
BytebuddyAgent [candidate] (718.161 ms) : 0, 718161
GlobalTracer [baseline] (240.495 ms) : 0, 240495
GlobalTracer [candidate] (239.512 ms) : 0, 239512
AppSec [baseline] (55.911 ms) : 0, 55911
AppSec [candidate] (55.556 ms) : 0, 55556
Remote Config [baseline] (694.788 µs) : 0, 695
Remote Config [candidate] (684.734 µs) : 0, 685
Telemetry [baseline] (11.668 ms) : 0, 11668
Telemetry [candidate] (12.829 ms) : 0, 12829
section iast
BytebuddyAgent [baseline] (835.728 ms) : 0, 835728
BytebuddyAgent [candidate] (836.39 ms) : 0, 836390
GlobalTracer [baseline] (229.863 ms) : 0, 229863
GlobalTracer [candidate] (230.746 ms) : 0, 230746
IAST [baseline] (22.787 ms) : 0, 22787
IAST [candidate] (22.95 ms) : 0, 22950
AppSec [baseline] (56.698 ms) : 0, 56698
AppSec [candidate] (57.159 ms) : 0, 57159
Remote Config [baseline] (609.798 µs) : 0, 610
Remote Config [candidate] (631.977 µs) : 0, 632
Telemetry [baseline] (8.636 ms) : 0, 8636
Telemetry [candidate] (8.785 ms) : 0, 8785
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (840.298 ms) : 0, 840298
BytebuddyAgent [candidate] (835.849 ms) : 0, 835849
GlobalTracer [baseline] (230.055 ms) : 0, 230055
GlobalTracer [candidate] (230.509 ms) : 0, 230509
IAST [baseline] (22.903 ms) : 0, 22903
IAST [candidate] (22.621 ms) : 0, 22621
AppSec [baseline] (56.844 ms) : 0, 56844
AppSec [candidate] (56.906 ms) : 0, 56906
Remote Config [baseline] (616.842 µs) : 0, 617
Remote Config [candidate] (612.099 µs) : 0, 612
Telemetry [baseline] (8.734 ms) : 0, 8734
Telemetry [candidate] (8.583 ms) : 0, 8583
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (831.584 ms) : 0, 831584
BytebuddyAgent [candidate] (838.089 ms) : 0, 838089
GlobalTracer [baseline] (229.656 ms) : 0, 229656
GlobalTracer [candidate] (230.983 ms) : 0, 230983
IAST [baseline] (24.439 ms) : 0, 24439
IAST [candidate] (24.154 ms) : 0, 24154
AppSec [baseline] (54.543 ms) : 0, 54543
AppSec [candidate] (55.389 ms) : 0, 55389
Remote Config [baseline] (608.517 µs) : 0, 609
Remote Config [candidate] (617.608 µs) : 0, 618
Telemetry [baseline] (8.547 ms) : 0, 8547
Telemetry [candidate] (8.615 ms) : 0, 8615

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.48.0-SNAPSHOT~e8145ffd98, baseline=1.47.0-SNAPSHOT~a951656fc4

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.042 s) : 0, 1041852
Total [baseline] (10.458 s) : 0, 10458167
Agent [candidate] (1.045 s) : 0, 1044638
Total [candidate] (10.482 s) : 0, 10481993
section appsec
Agent [baseline] (1.183 s) : 0, 1182654
Total [baseline] (10.724 s) : 0, 10723790
Agent [candidate] (1.182 s) : 0, 1181543
Total [candidate] (10.759 s) : 0, 10759029
section iast
Agent [baseline] (1.171 s) : 0, 1171147
Total [baseline] (10.93 s) : 0, 10930378
Agent [candidate] (1.172 s) : 0, 1171600
Total [candidate] (10.989 s) : 0, 10988615
section profiling
Agent [baseline] (1.26 s) : 0, 1259634
Total [baseline] (10.808 s) : 0, 10808269
Agent [candidate] (1.257 s) : 0, 1257191
Total [candidate] (10.875 s) : 0, 10874512

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.042 s	-
Agent	appsec	1.183 s	140.802 ms (13.5%)
Agent	iast	1.171 s	129.295 ms (12.4%)
Agent	profiling	1.26 s	217.783 ms (20.9%)
Total	tracing	10.458 s	-
Total	appsec	10.724 s	265.622 ms (2.5%)
Total	iast	10.93 s	472.211 ms (4.5%)
Total	profiling	10.808 s	350.102 ms (3.3%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.045 s	-
Agent	appsec	1.182 s	136.905 ms (13.1%)
Agent	iast	1.172 s	126.962 ms (12.2%)
Agent	profiling	1.257 s	212.554 ms (20.3%)
Total	tracing	10.482 s	-
Total	appsec	10.759 s	277.036 ms (2.6%)
Total	iast	10.989 s	506.622 ms (4.8%)
Total	profiling	10.875 s	392.519 ms (3.7%)

gantt
    title petclinic - break down per module: candidate=1.48.0-SNAPSHOT~e8145ffd98, baseline=1.47.0-SNAPSHOT~a951656fc4

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (720.198 ms) : 0, 720198
BytebuddyAgent [candidate] (722.279 ms) : 0, 722279
GlobalTracer [baseline] (240.242 ms) : 0, 240242
GlobalTracer [candidate] (239.738 ms) : 0, 239738
AppSec [baseline] (55.732 ms) : 0, 55732
AppSec [candidate] (55.418 ms) : 0, 55418
Remote Config [baseline] (685.968 µs) : 0, 686
Remote Config [candidate] (682.57 µs) : 0, 683
Telemetry [baseline] (10.067 ms) : 0, 10067
Telemetry [candidate] (11.426 ms) : 0, 11426
section appsec
BytebuddyAgent [baseline] (734.337 ms) : 0, 734337
BytebuddyAgent [candidate] (734.147 ms) : 0, 734147
GlobalTracer [baseline] (236.429 ms) : 0, 236429
GlobalTracer [candidate] (235.509 ms) : 0, 235509
IAST [baseline] (21.742 ms) : 0, 21742
IAST [candidate] (21.565 ms) : 0, 21565
AppSec [baseline] (177.233 ms) : 0, 177233
AppSec [candidate] (176.952 ms) : 0, 176952
Remote Config [baseline] (660.602 µs) : 0, 661
Remote Config [candidate] (658.308 µs) : 0, 658
Telemetry [baseline] (8.312 ms) : 0, 8312
Telemetry [candidate] (8.603 ms) : 0, 8603
section iast
BytebuddyAgent [baseline] (836.964 ms) : 0, 836964
BytebuddyAgent [candidate] (837.09 ms) : 0, 837090
GlobalTracer [baseline] (230.765 ms) : 0, 230765
GlobalTracer [candidate] (230.126 ms) : 0, 230126
IAST [baseline] (22.63 ms) : 0, 22630
IAST [candidate] (23.018 ms) : 0, 23018
AppSec [baseline] (56.62 ms) : 0, 56620
AppSec [candidate] (57.144 ms) : 0, 57144
Remote Config [baseline] (609.123 µs) : 0, 609
Remote Config [candidate] (620.351 µs) : 0, 620
Telemetry [baseline] (8.616 ms) : 0, 8616
Telemetry [candidate] (8.74 ms) : 0, 8740
section profiling
BytebuddyAgent [baseline] (708.792 ms) : 0, 708792
BytebuddyAgent [candidate] (707.672 ms) : 0, 707672
GlobalTracer [baseline] (349.788 ms) : 0, 349788
GlobalTracer [candidate] (349.658 ms) : 0, 349658
AppSec [baseline] (55.199 ms) : 0, 55199
AppSec [candidate] (54.446 ms) : 0, 54446
Remote Config [baseline] (676.935 µs) : 0, 677
Remote Config [candidate] (665.867 µs) : 0, 666
Telemetry [baseline] (9.03 ms) : 0, 9030
Telemetry [candidate] (9.013 ms) : 0, 9013
ProfilingAgent [baseline] (95.702 ms) : 0, 95702
ProfilingAgent [candidate] (95.524 ms) : 0, 95524
Profiling [baseline] (95.726 ms) : 0, 95726
Profiling [candidate] (95.548 ms) : 0, 95548

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2025-03-05T13:06:45	2025-03-05T13:14:32
git_branch	master	smola/capture-test-logs
git_commit_date	1741171359	1741179470
git_commit_sha	`a951656`	`e8145ff`
release_version	1.47.0-SNAPSHOT~a951656fc4	1.48.0-SNAPSHOT~e8145ffd98
start_time	2025-03-05T13:06:31	2025-03-05T13:14:17

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1741180871	1741180871
ci_job_id	833525100	833525100
ci_pipeline_id	57776923	57776923
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-1pycrazx-project-304-concurrent-1-37eog7sv 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-1pycrazx-project-304-concurrent-1-37eog7sv 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 14 metrics, 16 unstable metrics.

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.48.0-SNAPSHOT~e8145ffd98, baseline=1.47.0-SNAPSHOT~a951656fc4
    dateFormat X
    axisFormat %s
section baseline
no_agent (383.622 µs) : 364, 403
.   : milestone, 384,
iast (512.396 µs) : 491, 534
.   : milestone, 512,
iast_FULL (731.986 µs) : 710, 754
.   : milestone, 732,
iast_GLOBAL (568.045 µs) : 545, 591
.   : milestone, 568,
iast_HARDCODED_SECRET_DISABLED (517.856 µs) : 496, 540
.   : milestone, 518,
iast_INACTIVE (464.119 µs) : 443, 485
.   : milestone, 464,
iast_TELEMETRY_OFF (503.803 µs) : 481, 527
.   : milestone, 504,
tracing (463.061 µs) : 442, 484
.   : milestone, 463,
section candidate
no_agent (381.379 µs) : 361, 402
.   : milestone, 381,
iast (515.002 µs) : 493, 537
.   : milestone, 515,
iast_FULL (734.283 µs) : 712, 756
.   : milestone, 734,
iast_GLOBAL (565.555 µs) : 543, 588
.   : milestone, 566,
iast_HARDCODED_SECRET_DISABLED (522.322 µs) : 500, 544
.   : milestone, 522,
iast_INACTIVE (463.039 µs) : 442, 484
.   : milestone, 463,
iast_TELEMETRY_OFF (502.71 µs) : 480, 525
.   : milestone, 503,
tracing (463.156 µs) : 442, 485
.   : milestone, 463,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	383.622 µs [363.962 µs, 403.282 µs]	-
iast	512.396 µs [490.584 µs, 534.208 µs]	128.774 µs (33.6%)
iast_FULL	731.986 µs [710.125 µs, 753.848 µs]	348.364 µs (90.8%)
iast_GLOBAL	568.045 µs [545.21 µs, 590.88 µs]	184.423 µs (48.1%)
iast_HARDCODED_SECRET_DISABLED	517.856 µs [496.193 µs, 539.519 µs]	134.234 µs (35.0%)
iast_INACTIVE	464.119 µs [442.837 µs, 485.401 µs]	80.497 µs (21.0%)
iast_TELEMETRY_OFF	503.803 µs [480.954 µs, 526.652 µs]	120.181 µs (31.3%)
tracing	463.061 µs [442.154 µs, 483.968 µs]	79.439 µs (20.7%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	381.379 µs [360.932 µs, 401.825 µs]	-
iast	515.002 µs [492.761 µs, 537.242 µs]	133.623 µs (35.0%)
iast_FULL	734.283 µs [712.17 µs, 756.396 µs]	352.904 µs (92.5%)
iast_GLOBAL	565.555 µs [543.113 µs, 587.997 µs]	184.176 µs (48.3%)
iast_HARDCODED_SECRET_DISABLED	522.322 µs [500.462 µs, 544.181 µs]	140.943 µs (37.0%)
iast_INACTIVE	463.039 µs [441.869 µs, 484.209 µs]	81.66 µs (21.4%)
iast_TELEMETRY_OFF	502.71 µs [480.119 µs, 525.302 µs]	121.332 µs (31.8%)
tracing	463.156 µs [441.539 µs, 484.773 µs]	81.777 µs (21.4%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.48.0-SNAPSHOT~e8145ffd98, baseline=1.47.0-SNAPSHOT~a951656fc4
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.364 ms) : 1345, 1384
.   : milestone, 1364,
appsec (1.724 ms) : 1699, 1748
.   : milestone, 1724,
appsec_no_iast (1.734 ms) : 1711, 1758
.   : milestone, 1734,
code_origins (1.672 ms) : 1638, 1706
.   : milestone, 1672,
iast (1.519 ms) : 1494, 1544
.   : milestone, 1519,
profiling (1.568 ms) : 1543, 1592
.   : milestone, 1568,
tracing (1.506 ms) : 1482, 1531
.   : milestone, 1506,
section candidate
no_agent (1.35 ms) : 1332, 1369
.   : milestone, 1350,
appsec (1.738 ms) : 1714, 1762
.   : milestone, 1738,
appsec_no_iast (1.75 ms) : 1727, 1773
.   : milestone, 1750,
code_origins (1.688 ms) : 1654, 1721
.   : milestone, 1688,
iast (1.507 ms) : 1483, 1531
.   : milestone, 1507,
profiling (1.534 ms) : 1509, 1560
.   : milestone, 1534,
tracing (1.513 ms) : 1488, 1537
.   : milestone, 1513,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.364 ms [1.345 ms, 1.384 ms]	-
appsec	1.724 ms [1.699 ms, 1.748 ms]	359.591 µs (26.4%)
appsec_no_iast	1.734 ms [1.711 ms, 1.758 ms]	370.222 µs (27.1%)
code_origins	1.672 ms [1.638 ms, 1.706 ms]	307.888 µs (22.6%)
iast	1.519 ms [1.494 ms, 1.544 ms]	154.997 µs (11.4%)
profiling	1.568 ms [1.543 ms, 1.592 ms]	203.563 µs (14.9%)
tracing	1.506 ms [1.482 ms, 1.531 ms]	142.067 µs (10.4%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.35 ms [1.332 ms, 1.369 ms]	-
appsec	1.738 ms [1.714 ms, 1.762 ms]	387.7 µs (28.7%)
appsec_no_iast	1.75 ms [1.727 ms, 1.773 ms]	399.992 µs (29.6%)
code_origins	1.688 ms [1.654 ms, 1.721 ms]	337.115 µs (25.0%)
iast	1.507 ms [1.483 ms, 1.531 ms]	156.792 µs (11.6%)
profiling	1.534 ms [1.509 ms, 1.56 ms]	184.099 µs (13.6%)
tracing	1.513 ms [1.488 ms, 1.537 ms]	162.124 µs (12.0%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	smola/capture-test-logs
git_commit_date	1741171359	1741179470
git_commit_sha	`a951656`	`e8145ff`
release_version	1.47.0-SNAPSHOT~a951656fc4	1.48.0-SNAPSHOT~e8145ffd98

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1741181403	1741181403
ci_job_id	833525101	833525101
ci_pipeline_id	57776923	57776923
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-snmtyxn8-project-304-concurrent-3-pqnkl0sf 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-snmtyxn8-project-304-concurrent-3-pqnkl0sf 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant	appsec	appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.48.0-SNAPSHOT~e8145ffd98, baseline=1.47.0-SNAPSHOT~a951656fc4
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.355 s) : 15355000, 15355000
.   : milestone, 15355000,
appsec (14.892 s) : 14892000, 14892000
.   : milestone, 14892000,
iast (18.85 s) : 18850000, 18850000
.   : milestone, 18850000,
iast_GLOBAL (18.121 s) : 18121000, 18121000
.   : milestone, 18121000,
profiling (14.971 s) : 14971000, 14971000
.   : milestone, 14971000,
tracing (15.159 s) : 15159000, 15159000
.   : milestone, 15159000,
section candidate
no_agent (15.448 s) : 15448000, 15448000
.   : milestone, 15448000,
appsec (15.167 s) : 15167000, 15167000
.   : milestone, 15167000,
iast (18.169 s) : 18169000, 18169000
.   : milestone, 18169000,
iast_GLOBAL (18.095 s) : 18095000, 18095000
.   : milestone, 18095000,
profiling (15.241 s) : 15241000, 15241000
.   : milestone, 15241000,
tracing (15.196 s) : 15196000, 15196000
.   : milestone, 15196000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.355 s [15.355 s, 15.355 s]	-
appsec	14.892 s [14.892 s, 14.892 s]	-463.0 ms (-3.0%)
iast	18.85 s [18.85 s, 18.85 s]	3.495 s (22.8%)
iast_GLOBAL	18.121 s [18.121 s, 18.121 s]	2.766 s (18.0%)
profiling	14.971 s [14.971 s, 14.971 s]	-384.0 ms (-2.5%)
tracing	15.159 s [15.159 s, 15.159 s]	-196.0 ms (-1.3%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.448 s [15.448 s, 15.448 s]	-
appsec	15.167 s [15.167 s, 15.167 s]	-281.0 ms (-1.8%)
iast	18.169 s [18.169 s, 18.169 s]	2.721 s (17.6%)
iast_GLOBAL	18.095 s [18.095 s, 18.095 s]	2.647 s (17.1%)
profiling	15.241 s [15.241 s, 15.241 s]	-207.0 ms (-1.3%)
tracing	15.196 s [15.196 s, 15.196 s]	-252.0 ms (-1.6%)

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.48.0-SNAPSHOT~e8145ffd98, baseline=1.47.0-SNAPSHOT~a951656fc4
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.467 ms) : 1456, 1478
.   : milestone, 1467,
appsec (2.332 ms) : 2289, 2376
.   : milestone, 2332,
iast (2.107 ms) : 2052, 2162
.   : milestone, 2107,
iast_GLOBAL (2.148 ms) : 2093, 2203
.   : milestone, 2148,
profiling (1.999 ms) : 1954, 2044
.   : milestone, 1999,
tracing (1.955 ms) : 1912, 1997
.   : milestone, 1955,
section candidate
no_agent (1.471 ms) : 1460, 1483
.   : milestone, 1471,
appsec (2.343 ms) : 2299, 2386
.   : milestone, 2343,
iast (2.118 ms) : 2063, 2173
.   : milestone, 2118,
iast_GLOBAL (2.16 ms) : 2105, 2215
.   : milestone, 2160,
profiling (1.972 ms) : 1927, 2016
.   : milestone, 1972,
tracing (1.959 ms) : 1917, 2002
.   : milestone, 1959,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.467 ms [1.456 ms, 1.478 ms]	-
appsec	2.332 ms [2.289 ms, 2.376 ms]	865.465 µs (59.0%)
iast	2.107 ms [2.052 ms, 2.162 ms]	639.929 µs (43.6%)
iast_GLOBAL	2.148 ms [2.093 ms, 2.203 ms]	681.119 µs (46.4%)
profiling	1.999 ms [1.954 ms, 2.044 ms]	531.765 µs (36.2%)
tracing	1.955 ms [1.912 ms, 1.997 ms]	487.656 µs (33.2%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.471 ms [1.46 ms, 1.483 ms]	-
appsec	2.343 ms [2.299 ms, 2.386 ms]	871.467 µs (59.2%)
iast	2.118 ms [2.063 ms, 2.173 ms]	646.807 µs (44.0%)
iast_GLOBAL	2.16 ms [2.105 ms, 2.215 ms]	688.655 µs (46.8%)
profiling	1.972 ms [1.927 ms, 2.016 ms]	500.335 µs (34.0%)
tracing	1.959 ms [1.917 ms, 2.002 ms]	488.145 µs (33.2%)

jandro996

LGTM! just one thing, IMHO it's weird to have those same changes in #8510

smola · 2025-03-05T17:43:55Z

@jandro996 Yep, I'll rebase that one!

smola · 2025-03-05T17:44:12Z

@jandro996 Yep, I'll rebase that one!

smola added comp: testing Testing tag: no release notes Changes to exclude from release notes labels Mar 5, 2025

smola marked this pull request as ready for review March 5, 2025 12:32

smola requested review from a team as code owners March 5, 2025 12:32

smola requested review from manuel-alvarez-alvarez and sezen-datadog March 5, 2025 12:32

PerfectSlayer reviewed Mar 5, 2025

View reviewed changes

dd-java-agent/agent-iast/build.gradle Outdated Show resolved Hide resolved

utils/test-utils/build.gradle Outdated Show resolved Hide resolved

utils/test-utils/src/main/java/datadog/trace/test/logging/TestLogCollector.java Outdated Show resolved Hide resolved

Add API to capture logs in unit tests

e8145ff

smola force-pushed the smola/capture-test-logs branch from 16b9858 to e8145ff Compare March 5, 2025 12:58

smola requested a review from PerfectSlayer March 5, 2025 12:58

smola mentioned this pull request Mar 5, 2025

Improve detection of missing request end events #8510

Merged

jandro996 approved these changes Mar 5, 2025

View reviewed changes

smola merged commit c779bc9 into master Mar 5, 2025
210 checks passed

smola deleted the smola/capture-test-logs branch March 5, 2025 17:44

github-actions bot added this to the 1.48.0 milestone Mar 5, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add API to capture logs in unit tests #8513

Add API to capture logs in unit tests #8513

smola commented Mar 5, 2025 •

edited

Loading

PerfectSlayer left a comment

smola commented Mar 5, 2025

pr-commenter bot commented Mar 5, 2025 •

edited

Loading

jandro996 left a comment

smola commented Mar 5, 2025

smola commented Mar 5, 2025

Add API to capture logs in unit tests #8513

Add API to capture logs in unit tests #8513

Conversation

smola commented Mar 5, 2025 • edited Loading

What Does This Do

Motivation

Additional Notes

Contributor Checklist

PerfectSlayer left a comment

Choose a reason for hiding this comment

smola commented Mar 5, 2025

pr-commenter bot commented Mar 5, 2025 • edited Loading

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

jandro996 left a comment

Choose a reason for hiding this comment

smola commented Mar 5, 2025

smola commented Mar 5, 2025

smola commented Mar 5, 2025 •

edited

Loading

pr-commenter bot commented Mar 5, 2025 •

edited

Loading