v3.28.0 proposal #3383

Qard · 2023-07-11T21:15:21Z

Features

profiling: Adapt to new pprof-nodejs API ([profiling] Adapt to new pprof-nodejs API #3368)
iast: Report IAST metrics via telemetry (Report IAST metrics via telemetry #2805)
ci-visibility: Change test.type in browser tests ([ci-visibility] Change test.type in browser tests #3358)

Bug Fixes

ci-visibility: Intelligent test runner: only report skipped flag if tests were actually skipped ([ci-visibility] Intelligent test runner: only report skipped flag if tests were actually skipped #3369)
tracer: add dc store binding polyfill and migrate http2 (add dc store binding polyfill and migrate http2 #3284)
tracer: grpc: Convert grpc to use tracing channel (Convert grpc to use tracing channel #3366)
ci-visibility: validate CI tags before sending them (validate CI tags before sending them #3310)
tracer: express: recognize regex with flags (express: recognize regex with flags #3301)

github-actions · 2023-07-11T21:16:24Z

Overall package size

Self size: 4.87 MB
Deduped: 57.86 MB
No deduping: 57.95 MB

Dependency sizes

name	version	self size	total size
@datadog/native-iast-taint-tracking	1.5.0	14.86 MB	14.86 MB
@datadog/native-appsec	3.2.0	13.38 MB	13.39 MB
@datadog/pprof	3.0.0	10.54 MB	11.39 MB
protobufjs	7.2.4	2.74 MB	6.52 MB
@datadog/native-iast-rewriter	2.0.1	2.09 MB	2.1 MB
@opentelemetry/core	1.14.0	872.87 kB	1.47 MB
@datadog/native-metrics	2.0.0	898.77 kB	1.3 MB
@opentelemetry/api	1.4.1	780.32 kB	780.32 kB
msgpack-lite	0.1.26	201.16 kB	281.59 kB
opentracing	0.14.7	194.81 kB	194.81 kB
semver	7.5.3	93.39 kB	123.79 kB
@datadog/sketches-js	2.1.0	109.9 kB	109.9 kB
lodash.sortby	4.7.0	75.76 kB	75.76 kB
lru-cache	7.14.0	74.95 kB	74.95 kB
ipaddr.js	2.0.1	59.52 kB	59.52 kB
int64-buffer	0.1.10	49.18 kB	49.18 kB
ignore	5.2.0	48.87 kB	48.87 kB
import-in-the-middle	1.3.5	34.34 kB	38.81 kB
istanbul-lib-coverage	3.2.0	29.34 kB	29.34 kB
retry	0.10.1	27.44 kB	27.44 kB
lodash.uniq	4.5.0	25.01 kB	25.01 kB
limiter	1.1.5	23.17 kB	23.17 kB
lodash.kebabcase	4.1.1	17.75 kB	17.75 kB
lodash.pick	4.4.0	16.33 kB	16.33 kB
node-abort-controller	3.0.1	14.33 kB	14.33 kB
crypto-randomuuid	1.0.0	11.18 kB	11.18 kB
diagnostics_channel	1.1.0	7.07 kB	7.07 kB
path-to-regexp	0.1.7	6.78 kB	6.78 kB
koalas	1.0.2	6.47 kB	6.47 kB
methods	1.1.2	5.29 kB	5.29 kB
module-details-from-path	1.0.3	4.47 kB	4.47 kB

_{🤖 This report was automatically generated by heaviest-objects-in-the-universe}

codecov · 2023-07-11T21:18:06Z

Codecov Report

❗ No coverage uploaded for pull request base (v3.x@ba80cbb). Click here to learn what that means.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             v3.x    #3383   +/-   ##
=======================================
  Coverage        ?   84.77%           
=======================================
  Files           ?      211           
  Lines           ?     8256           
  Branches        ?       33           
=======================================
  Hits            ?     6999           
  Misses          ?     1257           
  Partials        ?        0

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

* add dc store binding polyfill and migrate http2 * fix error handling for synchronous exceptions * OutboundPlugin#tagPeerService() --------- Co-authored-by: Thomas Hunter II <tlhunter@datadog.com>

* ciapp-3571: add url validator * ciapp-3571: add url validator and change tests from ci-spec * fix variables to follow camelcase * Remove valid import Co-authored-by: Juan Antonio Fernández de Alba <juan.fernandezdealba@datadoghq.com> * Remove unnecessary check Co-authored-by: Juan Antonio Fernández de Alba <juan.fernandezdealba@datadoghq.com> * Remove unnecessary check for CI_PIPELINE_URL Co-authored-by: Juan Antonio Fernández de Alba <juan.fernandezdealba@datadoghq.com> * Format typo Co-authored-by: Juan Antonio Fernández de Alba <juan.fernandezdealba@datadoghq.com> * Use validateMetadata return directly Co-authored-by: Juan Antonio Fernández de Alba <juan.fernandezdealba@datadoghq.com> * use new ci-spec JSONs * fix spacing and sort imports * add test to validateMetadata * Update JSONs and refactor URL checking * Update JSONs and refactor URL checking * Add URL validation tests * Apply suggestions from code review Co-authored-by: Juan Antonio Fernández de Alba <juan.fernandezdealba@datadoghq.com> * Add tag to test suite --------- Co-authored-by: Juan Antonio Fernández de Alba <juan.fernandezdealba@datadoghq.com>

* Add framework type checking * Improve code readability Co-authored-by: Juan Antonio Fernández de Alba <juan.fernandezdealba@datadoghq.com> * Fix imports * Fix styling Co-authored-by: Juan Antonio Fernández de Alba <juan.fernandezdealba@datadoghq.com> * Add cypress and playwright tests * Change browser conditions to detect all playwright instances * Fix include string * Fix some suites still being reported as test type --------- Co-authored-by: Juan Antonio Fernández de Alba <juan.fernandezdealba@datadoghq.com>

* Remove cpu-experimental profiler * Adapt to API changes in pprof-nodejs

…tests were actually skipped (#3369)

Co-authored-by: Ugaitz Urien <ugaitz.urien@datadoghq.com>

* Convert grpc to use tracing channel * Allow addError to accept explicit span rather than delegating to activeSpan

juan-fernandez

looks good from ci visibility's perspective. Only feedback is to change "civis" to ci-visibility in the release notes

Supplying a `DD_TRACE_PEER_SERVICE_MAPPING` allows users to modify the `peer.service` tag in the same manner that `DD_SERVICE_MAPPING` does for service name. If the `peer.service` value was matched by the mapping, the original value of the tag before remapping is reported in `_dd.peer.service.remapped_from`.

pr-commenter · 2023-07-13T23:24:02Z

Benchmarks

Benchmark execution time: 2023-07-14 14:07:23

Comparing candidate commit 550519b in PR branch v3.28.0-proposal with baseline commit ba80cbb in branch v3.x.

Found 56 performance improvements and 0 performance regressions! Performance is the same for 653 metrics, 29 unstable metrics.

scenario:appsec-iast-no-vulnerability-control-14

🟩 cpu_user_time [-155.846ms; -137.515ms] or [-24.552%; -21.664%]
🟩 execution_time [-207.561ms; -180.602ms] or [-25.982%; -22.607%]
🟩 instructions [-281.4M instructions; -277.3M instructions] or [-18.501%; -18.231%]
🟩 max_rss_usage [-6.111KB; -5.710KB] or [-9.549%; -8.922%]

scenario:appsec-iast-no-vulnerability-control-16

🟩 cpu_user_time [-143.933ms; -120.719ms] or [-23.598%; -19.792%]
🟩 execution_time [-185.731ms; -153.238ms] or [-24.227%; -19.988%]
🟩 instructions [-244.1M instructions; -239.2M instructions] or [-17.003%; -16.661%]
🟩 max_rss_usage [-7.497KB; -6.820KB] or [-10.726%; -9.758%]

scenario:appsec-iast-no-vulnerability-control-18

🟩 cpu_user_time [-135.814ms; -116.688ms] or [-22.214%; -19.086%]
🟩 execution_time [-175.474ms; -150.923ms] or [-23.738%; -20.417%]
🟩 instructions [-214.5M instructions; -210.7M instructions] or [-15.135%; -14.867%]
🟩 max_rss_usage [-7.679KB; -6.986KB] or [-9.412%; -8.562%]

scenario:appsec-iast-no-vulnerability-iast-enabled-always-active-14

🟩 cpu_user_time [-222.494ms; -176.378ms] or [-6.853%; -5.432%]
🟩 execution_time [-249.044ms; -186.524ms] or [-7.336%; -5.494%]
🟩 instructions [-543.7M instructions; -528.0M instructions] or [-21.718%; -21.089%]

scenario:appsec-iast-no-vulnerability-iast-enabled-always-active-16

🟩 instructions [-572.6M instructions; -557.1M instructions] or [-23.750%; -23.107%]

scenario:appsec-iast-no-vulnerability-iast-enabled-always-active-18

🟩 cpu_user_time [-204.887ms; -178.200ms] or [-17.867%; -15.540%]
🟩 execution_time [-227.758ms; -181.336ms] or [-17.673%; -14.071%]
🟩 instructions [-441.3M instructions; -431.0M instructions] or [-19.971%; -19.503%]

scenario:appsec-iast-no-vulnerability-iast-enabled-default-config-14

🟩 instructions [-368.9M instructions; -345.8M instructions] or [-17.044%; -15.977%]

scenario:appsec-iast-no-vulnerability-iast-enabled-default-config-16

🟩 instructions [-419.7M instructions; -396.9M instructions] or [-20.121%; -19.031%]

scenario:appsec-iast-no-vulnerability-iast-enabled-default-config-18

🟩 cpu_user_time [-166.961ms; -140.806ms] or [-15.404%; -12.991%]
🟩 execution_time [-195.049ms; -155.843ms] or [-15.789%; -12.615%]
🟩 instructions [-282.9M instructions; -261.7M instructions] or [-15.055%; -13.928%]

scenario:appsec-iast-with-vulnerability-control-14

🟩 cpu_user_time [-398.558ms; -359.944ms] or [-34.484%; -31.143%]
🟩 execution_time [-587.366ms; -522.237ms] or [-35.798%; -31.829%]
🟩 instructions [-929.9M instructions; -918.1M instructions] or [-32.231%; -31.821%]
🟩 max_rss_usage [-3.761KB; -3.442KB] or [-5.581%; -5.108%]

scenario:appsec-iast-with-vulnerability-control-16

🟩 cpu_user_time [-371.400ms; -332.830ms] or [-32.674%; -29.281%]
🟩 execution_time [-590.201ms; -514.272ms] or [-35.291%; -30.751%]
🟩 instructions [-909.0M instructions; -898.5M instructions] or [-31.963%; -31.593%]
🟩 max_rss_usage [-4.495KB; -4.061KB] or [-5.983%; -5.406%]

scenario:appsec-iast-with-vulnerability-control-18

🟩 cpu_user_time [-368.231ms; -321.871ms] or [-32.372%; -28.297%]
🟩 execution_time [-624.037ms; -523.142ms] or [-37.228%; -31.209%]
🟩 instructions [-919.9M instructions; -910.1M instructions] or [-32.242%; -31.898%]

scenario:appsec-iast-with-vulnerability-iast-enabled-always-active-14

🟩 cpu_user_time [-529.281ms; -405.560ms] or [-12.878%; -9.868%]
🟩 execution_time [-922.330ms; -700.819ms] or [-18.443%; -14.014%]
🟩 instructions [-1.8G instructions; -1.7G instructions] or [-34.742%; -34.154%]
🟩 max_rss_usage [-12.899KB; -9.124KB] or [-11.337%; -8.019%]

scenario:appsec-iast-with-vulnerability-iast-enabled-always-active-16

🟩 cpu_user_time [-497.806ms; -339.679ms] or [-12.017%; -8.200%]
🟩 execution_time [-888.876ms; -657.052ms] or [-17.695%; -13.080%]
🟩 instructions [-1.7G instructions; -1.6G instructions] or [-34.392%; -33.751%]
🟩 max_rss_usage [-11.033KB; -7.124KB] or [-9.129%; -5.895%]

scenario:appsec-iast-with-vulnerability-iast-enabled-always-active-18

🟩 cpu_user_time [-538.162ms; -435.190ms] or [-28.429%; -22.990%]
🟩 execution_time [-957.651ms; -692.766ms] or [-34.623%; -25.046%]
🟩 instructions [-1.7G instructions; -1.6G instructions] or [-35.447%; -34.981%]
🟩 max_rss_usage [-12.103KB; -10.036KB] or [-9.658%; -8.008%]

scenario:appsec-iast-with-vulnerability-iast-enabled-default-config-14

🟩 cpu_user_time [-487.882ms; -404.742ms] or [-12.311%; -10.213%]
🟩 execution_time [-805.234ms; -618.745ms] or [-16.972%; -13.041%]
🟩 instructions [-1.3G instructions; -1.3G instructions] or [-32.232%; -30.689%]

scenario:appsec-iast-with-vulnerability-iast-enabled-default-config-16

🟩 cpu_user_time [-506.167ms; -434.332ms] or [-12.578%; -10.793%]
🟩 execution_time [-861.597ms; -645.839ms] or [-17.907%; -13.422%]
🟩 instructions [-1.3G instructions; -1.3G instructions] or [-32.672%; -31.463%]

scenario:appsec-iast-with-vulnerability-iast-enabled-default-config-18

🟩 cpu_user_time [-459.274ms; -372.809ms] or [-26.907%; -21.841%]
🟩 execution_time [-830.180ms; -614.278ms] or [-33.507%; -24.793%]
🟩 instructions [-1.3G instructions; -1.2G instructions] or [-32.865%; -31.474%]

Drarig29 · 2023-07-24T12:56:12Z

For the record: this version released #3374

Qard added the semver-minor label Jul 11, 2023

Qard requested review from a team as code owners July 11, 2023 21:15

rochdev and others added 9 commits July 11, 2023 14:40

add dc store binding polyfill and migrate http2 (#3284)

d9312b6

* add dc store binding polyfill and migrate http2 * fix error handling for synchronous exceptions * OutboundPlugin#tagPeerService() --------- Co-authored-by: Thomas Hunter II <tlhunter@datadog.com>

express: recognize regex with flags (#3301)

8639ffb

dep: bump protobufjs to 7.2.4 (#3374)

308bf18

[profiling] Adapt to new pprof-nodejs API (#3368)

9557b9c

* Remove cpu-experimental profiler * Adapt to API changes in pprof-nodejs

[ci-visibility] Intelligent test runner: only report skipped flag if …

c93382e

…tests were actually skipped (#3369)

Report IAST metrics via telemetry (#2805)

d670be8

Co-authored-by: Ugaitz Urien <ugaitz.urien@datadoghq.com>

Convert grpc to use tracing channel (#3366)

516d6e8

* Convert grpc to use tracing channel * Allow addError to accept explicit span rather than delegating to activeSpan

Qard force-pushed the v3.28.0-proposal branch from 52f63ab to 470f81a Compare July 11, 2023 21:40

juan-fernandez previously approved these changes Jul 12, 2023

View reviewed changes

uurien previously approved these changes Jul 12, 2023

View reviewed changes

Qard force-pushed the v3.28.0-proposal branch from 470f81a to 1bab648 Compare July 12, 2023 20:37

tlhunter previously approved these changes Jul 13, 2023

View reviewed changes

tlhunter and others added 4 commits July 13, 2023 15:14

docs: add link to openai docs (#3381)

b860cfa

Reduce times in appsec-iast tests (#3354)

f3dba64

benchmarks: add node 12, 14, 20, upgrade node 14, 16 (#3388)

7e41dca

Qard dismissed stale reviews from tlhunter, juan-fernandez, and uurien via 887a106 July 13, 2023 22:14

Qard force-pushed the v3.28.0-proposal branch from 1bab648 to 887a106 Compare July 13, 2023 22:14

v3.28.0

550519b

tlhunter force-pushed the v3.28.0-proposal branch from 887a106 to 550519b Compare July 13, 2023 23:16

tlhunter approved these changes Jul 14, 2023

View reviewed changes

uurien approved these changes Jul 14, 2023

View reviewed changes

Qard merged commit 51eabba into v3.x Jul 14, 2023

Qard deleted the v3.28.0-proposal branch July 14, 2023 16:04

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

v3.28.0 proposal #3383

v3.28.0 proposal #3383

Qard commented Jul 11, 2023 •

edited

Loading

github-actions bot commented Jul 11, 2023 •

edited

Loading

codecov bot commented Jul 11, 2023 •

edited

Loading

juan-fernandez left a comment

pr-commenter bot commented Jul 13, 2023 •

edited

Loading

Drarig29 commented Jul 24, 2023 •

edited

Loading

v3.28.0 proposal #3383

v3.28.0 proposal #3383

Conversation

Qard commented Jul 11, 2023 • edited Loading

Features

Bug Fixes

github-actions bot commented Jul 11, 2023 • edited Loading

Overall package size

Dependency sizes

codecov bot commented Jul 11, 2023 • edited Loading

Codecov Report

juan-fernandez left a comment

Choose a reason for hiding this comment

pr-commenter bot commented Jul 13, 2023 • edited Loading

Benchmarks

scenario:appsec-iast-no-vulnerability-control-14

scenario:appsec-iast-no-vulnerability-control-16

scenario:appsec-iast-no-vulnerability-control-18

scenario:appsec-iast-no-vulnerability-iast-enabled-always-active-14

scenario:appsec-iast-no-vulnerability-iast-enabled-always-active-16

scenario:appsec-iast-no-vulnerability-iast-enabled-always-active-18

scenario:appsec-iast-no-vulnerability-iast-enabled-default-config-14

scenario:appsec-iast-no-vulnerability-iast-enabled-default-config-16

scenario:appsec-iast-no-vulnerability-iast-enabled-default-config-18

scenario:appsec-iast-with-vulnerability-control-14

scenario:appsec-iast-with-vulnerability-control-16

scenario:appsec-iast-with-vulnerability-control-18

scenario:appsec-iast-with-vulnerability-iast-enabled-always-active-14

scenario:appsec-iast-with-vulnerability-iast-enabled-always-active-16

scenario:appsec-iast-with-vulnerability-iast-enabled-always-active-18

scenario:appsec-iast-with-vulnerability-iast-enabled-default-config-14

scenario:appsec-iast-with-vulnerability-iast-enabled-default-config-16

scenario:appsec-iast-with-vulnerability-iast-enabled-default-config-18

Drarig29 commented Jul 24, 2023 • edited Loading

Qard commented Jul 11, 2023 •

edited

Loading

github-actions bot commented Jul 11, 2023 •

edited

Loading

codecov bot commented Jul 11, 2023 •

edited

Loading

pr-commenter bot commented Jul 13, 2023 •

edited

Loading

Drarig29 commented Jul 24, 2023 •

edited

Loading