Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

appsec: support for SSRF Exploit Prevention #2707

Merged
merged 22 commits into from
Jun 3, 2024
Merged

appsec: support for SSRF Exploit Prevention #2707

merged 22 commits into from
Jun 3, 2024

Conversation

Hellzy
Copy link
Contributor

@Hellzy Hellzy commented May 23, 2024 •
edited
Loading

JIRA: APPSEC-52492

This change adds support for SSRF exploit prevention.

What does this PR do?

  • Add appsec to net/http/roundtripper
  • Add a new dyngo roundtripper operation
  • Add a new listener for stacktrace actions
  • Add stacktraces to meta_struct, if generated
  • Add RC capability for RASP SSRF and register it for appsec

Motivation

Reviewer's Checklist

  • Changed code has unit tests for its functionality at or near 100% coverage.
  • System-Tests covering this feature have been added and enabled with the va.b.c-dev version tag.
  • There is a benchmark for any new code, or changes to existing code.
  • If this interacts with the agent in a new way, a system test has been added.
  • Add an appropriate team label so this PR gets put in the right place for the release notes.
  • Non-trivial go.mod changes, e.g. adding new modules, are reviewed by @DataDog/dd-trace-go-guild.

Unsure? Have a question? Request a review!

@github-actions github-actions bot added the apm:ecosystem contrib/* related feature requests or bugs label May 23, 2024
@Hellzy Hellzy force-pushed the francois.mazeau/ssrf-plug branch from b437853 to ad0dc14 Compare May 23, 2024 15:50
@pr-commenter
Copy link

pr-commenter bot commented May 23, 2024 •
edited
Loading

Benchmarks

Benchmark execution time: 2024-06-03 13:13:27

Comparing candidate commit 85a9bfd in PR branch francois.mazeau/ssrf-plug with baseline commit 1b338db in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 44 metrics, 0 unstable metrics.

@Hellzy Hellzy force-pushed the francois.mazeau/ssrf-plug branch 3 times, most recently from de2e464 to f347322 Compare May 24, 2024 12:21
Hellzy
Hellzy commented May 24, 2024
View reviewed changes
contrib/net/http/roundtripper.go Outdated Show resolved Hide resolved
internal/appsec/emitter/httpsec/http.go Outdated Show resolved Hide resolved
internal/appsec/testdata/rasp.json Show resolved Hide resolved
@eliottness eliottness mentioned this pull request May 27, 2024
Closed
8 tasks
@Hellzy Hellzy changed the title [FORK] appsec: support for SSRF Exploit Prevention appsec: support for SSRF Exploit Prevention May 27, 2024
@Hellzy Hellzy marked this pull request as ready for review May 27, 2024 15:32
@Hellzy Hellzy requested review from a team as code owners May 27, 2024 15:32
Julio-Guerra
Julio-Guerra requested changes May 28, 2024
View reviewed changes
internal/appsec/emitter/httpsec/roundtripper.go Outdated Show resolved Hide resolved
internal/appsec/emitter/httpsec/roundtripper.go Outdated Show resolved Hide resolved
internal/appsec/emitter/httpsec/roundtripper.go Outdated Show resolved Hide resolved
internal/appsec/listener/sharedsec/shared.go Outdated Show resolved Hide resolved
internal/appsec/listener/sharedsec/shared.go Outdated Show resolved Hide resolved
internal/appsec/listener/sharedsec/shared.go Outdated Show resolved Hide resolved
internal/appsec/waf_test.go Outdated Show resolved Hide resolved
internal/appsec/config/config.go Outdated Show resolved Hide resolved
internal/appsec/emitter/httpsec/http.go Show resolved Hide resolved
@eliottness eliottness force-pushed the francois.mazeau/ssrf-plug branch 3 times, most recently from d4b42d9 to b15a3fa Compare May 29, 2024 16:09
Julio-Guerra
Julio-Guerra requested changes May 30, 2024
View reviewed changes
Copy link
Contributor

@Julio-Guerra Julio-Guerra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really happy about the end result.

Most of my comments are just about extra comments in the code and similar stuffs.

The one "major" thing left is aligning {dyngo,httpsec,graphqlsec,grpces}.StartOperation APIs and I will be good

appsec/events/block.go Outdated Show resolved Hide resolved
appsec/events/block.go Outdated Show resolved Hide resolved
appsec/events/block.go Outdated Show resolved Hide resolved
internal/appsec/emitter/httpsec/http.go
@@ -34,7 +36,7 @@ import (
// This function should not be called when AppSec is disabled in order to
// get preciser error logs.
func MonitorParsedBody(ctx context.Context, body any) error {
parent := fromContext(ctx)
parent, _ := ctx.Value(listener.ContextKey{}).(*types.Operation)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fromContext was good/better - I suggest to revert

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reverted it because it was not sync across packages even though it could have and was repeated multiple times. Anyway its a one-liner so it only makes operation more opaque IMHO

internal/appsec/emitter/httpsec/roundtripper.go Outdated

var badInputContextOnce sync.Once

func RoundTrip(ctx context.Context, request *http.Request, rt http.RoundTripper) (*http.Response, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

side note: Sqreen's implementation of auto SSRF was hooked into http.do (the private function). It seems to fit into you operation args/res, where the request URL passed to http.do can be passed put in the arg field URL.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I refactored this. Please take a look in this the emitter and the contrib

internal/appsec/emitter/httpsec/types/types.go Show resolved Hide resolved
internal/appsec/emitter/sharedsec/shared.go Outdated
Comment on lines 43 to 44
dyngo.OnData(op, func(e *events.SecurityBlockingEvent) { err = e })
dyngo.StartOperation(op, args)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's minor but really is a mistake not to have the OnData listener under the user id operation scope.
Same for all other SecurityBlockingEvent listeners of this PR. It's just that StartOperation doesn't have the right API everywhere. We should align them all with setup like it was done in httpsec IIRC.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Definitively but the PR already has 10 parts inside so let's not add an 11th

internal/appsec/listener/httpsec/roundtripper.go Outdated
"gopkg.in/DataDog/dd-trace-go.v1/internal/log"
)

// RegisterRoundTripperListener registers a listener on outgoing requests to run the WAF.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// RegisterRoundTripperListener registers a listener on outgoing requests to run the WAF.
// RegisterRoundTripperListener registers a listener on outgoing HTTP client requests to run the WAF.

internal/appsec/emitter/httpsec/roundtripper.go
URL: url,
}

parent, _ := ctx.Value(listener.ContextKey{}).(dyngo.Operation)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
parent, _ := ctx.Value(listener.ContextKey{}).(dyngo.Operation)
parent = listener.FromContext(ctx)

even though listener doesn't sound like the right place for it, it would already by better to make it public and use it from here too.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Without the cast this would be easier to work with 🤔

internal/appsec/emitter/httpsec/roundtripper.go Outdated Show resolved Hide resolved
Julio-Guerra
Julio-Guerra requested changes May 30, 2024
View reviewed changes
Copy link
Contributor

@Julio-Guerra Julio-Guerra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really happy about the end result.

Most of my comments are just about extra comments in the code and similar stuffs.

The one "major" thing left is aligning {dyngo,httpsec,graphqlsec,grpces}.StartOperation APIs and I will be good

darccio
darccio reviewed Jun 3, 2024
View reviewed changes
Copy link
Member

@darccio darccio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that a test is timing out consistently:

=== RUN   TestUserBlocking/stream-block
2024/06/03 10:39:39 Datadog Tracer v1.65.0-dev WARN: No global service name was detected. GRPC Server may have been created before calling tracer.Start(). Will dynamically fetch service name for every span. Note this may have a slight performance cost, it is always recommended to start the tracer before initializing any traced packages.
2024/06/03 10:39:39 Datadog Tracer v1.65.0-dev WARN: No global service name was detected. GRPC Server may have been created before calling tracer.Start(). Will dynamically fetch service name for every span. Note this may have a slight performance cost, it is always recommended to start the tracer before initializing any traced packages.
panic: test timed out after 10m0s
running tests:
	TestUserBlocking (10m0s)
	TestUserBlocking/stream-block (10m0s)

goroutine 142 [running]:
testing.(*M).startAlarm.func1()
	/opt/hostedtoolcache/go/1.22.3/x64/src/testing/testing.go:2366 +0x385
created by time.goFunc
	/opt/hostedtoolcache/go/1.22.3/x64/src/time/sleep.go:177 +0x2d

goroutine 1 [chan receive, 9 minutes]:
testing.(*T).Run(0xc000165520, {0xc0ef34?, 0x0?}, 0xca08b8)
	/opt/hostedtoolcache/go/1.22.3/x64/src/testing/testing.go:1750 +0x3ab
testing.runTests.func1(0xc000165520)
	/opt/hostedtoolcache/go/1.22.3/x64/src/testing/testing.go:2161 +0x37
testing.tRunner(0xc000165520, 0xc0001fdc70)
	/opt/hostedtoolcache/go/1.22.3/x64/src/testing/testing.go:1689 +0xfb
testing.runTests(0xc000120cf0, {0x11fcb20, 0x16, 0x16}, {0x1?, 0x515ace?, 0x14760a0?})
	/opt/hostedtoolcache/go/1.22.3/x64/src/testing/testing.go:2159 +0x445
testing.(*M).Run(0xc00024a0a0)
	/opt/hostedtoolcache/go/1.22.3/x64/src/testing/testing.go:2027 +0x68b
main.main()
	_testmain.go:93 +0x16c

goroutine 275 [chan receive, 9 minutes]:
testing.(*T).Run(0xc000164820, {0xc0bb57?, 0x120f5a0?}, 0xc0003d0460)
	/opt/hostedtoolcache/go/1.22.3/x64/src/testing/testing.go:1750 +0x3ab
gopkg.in/DataDog/dd-trace-go.v1/contrib/google.golang.org/grpc.TestUserBlocking(0xc000164820)
	/home/runner/work/dd-trace-go/dd-trace-go/contrib/google.golang.org/grpc/appsec_test.go:297 +0x22a
testing.tRunner(0xc000164820, 0xca08b8)
	/opt/hostedtoolcache/go/1.22.3/x64/src/testing/testing.go:1689 +0xfb
created by testing.(*T).Run in goroutine 1
	/opt/hostedtoolcache/go/1.22.3/x64/src/testing/testing.go:1742 +0x390

goroutine 132 [select, 9 minutes]:
google.golang.org/grpc.newClientStreamWithParams.func4()
	/home/runner/work/dd-trace-go/dd-trace-go/go_pkg_mod_cache/google.golang.org/grpc@v1.57.1/stream.go:383 +0x8c
created by google.golang.org/grpc.newClientStreamWithParams in goroutine 250
	/home/runner/work/dd-trace-go/dd-trace-go/go_pkg_mod_cache/google.golang.org/grpc@v1.57.1/stream.go:382 +0xe08

goroutine 216 [sleep]:
time.Sleep(0x989680)
	/opt/hostedtoolcache/go/1.22.3/x64/src/runtime/time.go:195 +0x115
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).run(0xc0003ac2d0, 0xc00055ca20)
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:332 +0x19f
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start.func1()
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:354 +0x98
created by gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start in goroutine 210
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:350 +0x125

goroutine 253 [select, 9 minutes]:
google.golang.org/grpc/internal/grpcsync.(*CallbackSerializer).run(0xc0004c23e0, {0xd69d90, 0xc0003a60a0})
	/home/runner/work/dd-trace-go/dd-trace-go/go_pkg_mod_cache/google.golang.org/grpc@v1.57.1/internal/grpcsync/callback_serializer.go:83 +0x112
created by google.golang.org/grpc/internal/grpcsync.NewCallbackSerializer in goroutine 250
	/home/runner/work/dd-trace-go/dd-trace-go/go_pkg_mod_cache/google.golang.org/grpc@v1.57.1/internal/grpcsync/callback_serializer.go:55 +0x11a

goroutine 229 [IO wait, 9 minutes]:
internal/poll.runtime_pollWait(0x7fea652a1b88, 0x72)
	/opt/hostedtoolcache/go/1.22.3/x64/src/runtime/netpoll.go:345 +0x85
internal/poll.(*pollDesc).wait(0xc0001b4980?, 0xc00044a000?, 0x0)
	/opt/hostedtoolcache/go/1.22.3/x64/src/internal/poll/fd_poll_runtime.go:84 +0x27
internal/poll.(*pollDesc).waitRead(...)
	/opt/hostedtoolcache/go/1.22.3/x64/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc0001b4980, {0xc00044a000, 0x8000, 0x8000})
	/opt/hostedtoolcache/go/1.22.3/x64/src/internal/poll/fd_unix.go:164 +0x27a
net.(*netFD).Read(0xc0001b4980, {0xc00044a000?, 0x1040100000000?, 0x0?})
	/opt/hostedtoolcache/go/1.22.3/x64/src/net/fd_posix.go:55 +0x25
net.(*conn).Read(0xc000880000, {0xc00044a000?, 0x10401?, 0xc000000000?})
	/opt/hostedtoolcache/go/1.22.3/x64/src/net/net.go:179 +0x45
bufio.(*Reader).Read(0xc0005d63c0, {0xc000282040, 0x9, 0xc0004560c0?})
	/opt/hostedtoolcache/go/1.22.3/x64/src/bufio/bufio.go:241 +0x197
io.ReadAtLeast({0xd636a0, 0xc0005d63c0}, {0xc000282040, 0x9, 0x9}, 0x9)
	/opt/hostedtoolcache/go/1.22.3/x64/src/io/io.go:335 +0x90
io.ReadFull(...)
	/opt/hostedtoolcache/go/1.22.3/x64/src/io/io.go:354
golang.org/x/net/http2.readFrameHeader({0xc000282040, 0x9, 0xc0004560c0?}, {0xd636a0?, 0xc0005d63c0?})
	/home/runner/work/dd-trace-go/dd-trace-go/go_pkg_mod_cache/golang.org/x/net@v0.23.0/http2/frame.go:237 +0x65
golang.org/x/net/http2.(*Framer).ReadFrame(0xc000282000)
	/home/runner/work/dd-trace-go/dd-trace-go/go_pkg_mod_cache/golang.org/x/net@v0.23.0/http2/frame.go:498 +0x85
google.golang.org/grpc/internal/transport.(*http2Client).reader(0xc00017b208, 0xc0005d6420)
	/home/runner/work/dd-trace-go/dd-trace-go/go_pkg_mod_cache/google.golang.org/grpc@v1.57.1/internal/transport/http2_client.go:1595 +0x22d
created by google.golang.org/grpc/internal/transport.newHTTP2Client in goroutine 254
	/home/runner/work/dd-trace-go/dd-trace-go/go_pkg_mod_cache/google.golang.org/grpc@v1.57.1/internal/transport/http2_client.go:397 +0x1d53

goroutine 215 [chan receive]:
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).reportStats(0xc0003ac2d0)
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:386 +0x192
created by gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start in goroutine 210
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:349 +0xe6

goroutine 66 [chan receive]:
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).reportStats(0xc0002d8000)
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:386 +0x192
created by gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start in goroutine 39
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:349 +0xe6

goroutine 67 [sleep]:
time.Sleep(0x989680)
	/opt/hostedtoolcache/go/1.22.3/x64/src/runtime/time.go:195 +0x115
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).run(0xc0002d8000, 0xc000132600)
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:332 +0x19f
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start.func1()
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:354 +0x98
created by gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start in goroutine 39
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:350 +0x125

goroutine 68 [select]:
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).runLoadAgentFeatures(0xc0002d8000, 0xc000132420)
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:514 +0x79
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start.func2()
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:360 +0x98
created by gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start in goroutine 39
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:356 +0x165

goroutine 306 [chan receive]:
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).reportStats(0xc0003ac5a0)
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:386 +0x192
created by gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start in goroutine 294
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:349 +0xe6

goroutine 282 [chan receive]:
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).reportStats(0xc0003ac4b0)
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:386 +0x192
created by gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start in goroutine 277
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:349 +0xe6

goroutine 102 [chan receive]:
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).reportStats(0xc0003ac000)
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:386 +0x192
created by gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start in goroutine 49
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:349 +0xe6

goroutine 308 [select]:
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).runLoadAgentFeatures(0xc0003ac5a0, 0xc000132660)
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:514 +0x79
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start.func2()
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:360 +0x98
created by gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start in goroutine 294
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:356 +0x165

goroutine 171 [select]:
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).runLoadAgentFeatures(0xc0003ac1e0, 0xc00036e3c0)
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:514 +0x79
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start.func2()
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:360 +0x98
created by gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start in goroutine 156
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:356 +0x165

goroutine 217 [select]:
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).runLoadAgentFeatures(0xc0003ac2d0, 0xc00036e360)
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:514 +0x79
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start.func2()
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:360 +0x98
created by gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start in goroutine 210
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:356 +0x165

goroutine 103 [sleep]:
time.Sleep(0x989680)
	/opt/hostedtoolcache/go/1.22.3/x64/src/runtime/time.go:195 +0x115
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).run(0xc0003ac000, 0xc00055cd20)
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:332 +0x19f
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start.func1()
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:354 +0x98
created by gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start in goroutine 49
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:350 +0x125

goroutine 104 [select]:
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).runLoadAgentFeatures(0xc0003ac000, 0xc00055ccc0)
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:514 +0x79
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start.func2()
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:360 +0x98
created by gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start in goroutine 49
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:356 +0x165

goroutine 252 [select, 9 minutes]:
google.golang.org/grpc/internal/grpcsync.(*CallbackSerializer).run(0xc0004c2320, {0xd69d90, 0xc0003a6050})
	/home/runner/work/dd-trace-go/dd-trace-go/go_pkg_mod_cache/google.golang.org/grpc@v1.57.1/internal/grpcsync/callback_serializer.go:83 +0x112
created by google.golang.org/grpc/internal/grpcsync.NewCallbackSerializer in goroutine 250
	/home/runner/work/dd-trace-go/dd-trace-go/go_pkg_mod_cache/google.golang.org/grpc@v1.57.1/internal/grpcsync/callback_serializer.go:55 +0x11a

goroutine 133 [select, 9 minutes]:
google.golang.org/grpc/internal/transport.(*recvBufferReader).read(0xc0004c0370, {0xc00055a070, 0x5, 0x5})
	/home/runner/work/dd-trace-go/dd-trace-go/go_pkg_mod_cache/google.golang.org/grpc@v1.57.1/internal/transport/transport.go:183 +0x8a
google.golang.org/grpc/internal/transport.(*recvBufferReader).Read(0xc0004c0370, {0xc00055a070?, 0xc0004ba018?, 0xc0002a71e8?})
	/home/runner/work/dd-trace-go/dd-trace-go/go_pkg_mod_cache/google.golang.org/grpc@v1.57.1/internal/transport/transport.go:177 +0x15b
google.golang.org/grpc/internal/transport.(*transportReader).Read(0xc0004b4a20, {0xc00055a070?, 0xc0002a7260?, 0x929ee5?})
	/home/runner/work/dd-trace-go/dd-trace-go/go_pkg_mod_cache/google.golang.org/grpc@v1.57.1/internal/transport/transport.go:514 +0x2c
io.ReadAtLeast({0xd64cc0, 0xc0004b4a20}, {0xc00055a070, 0x5, 0x5}, 0x5)
	/opt/hostedtoolcache/go/1.22.3/x64/src/io/io.go:335 +0x90
io.ReadFull(...)
	/opt/hostedtoolcache/go/1.22.3/x64/src/io/io.go:354
google.golang.org/grpc/internal/transport.(*Stream).Read(0xc000693200, {0xc00055a070, 0x5, 0x5})
	/home/runner/work/dd-trace-go/dd-trace-go/go_pkg_mod_cache/google.golang.org/grpc@v1.57.1/internal/transport/transport.go:498 +0x96
google.golang.org/grpc.(*parser).recvMsg(0xc00055a060, 0x400000)
	/home/runner/work/dd-trace-go/dd-trace-go/go_pkg_mod_cache/google.golang.org/grpc@v1.57.1/rpc_util.go:600 +0x46
google.golang.org/grpc.recvAndDecompress(0x238?, 0xc000693200, {0x0, 0x0}, 0x400000, 0x0, {0x0, 0x0})
	/home/runner/work/dd-trace-go/dd-trace-go/go_pkg_mod_cache/google.golang.org/grpc@v1.57.1/rpc_util.go:730 +0x5a
google.golang.org/grpc.recv(0xc00055a060, {0x7fea1c422460, 0x14e2ee0}, 0xc000693200, {0x0, 0x0}, {0xb76ac0, 0xc00024e7c0}, 0x400000, 0x0, ...)
	/home/runner/work/dd-trace-go/dd-trace-go/go_pkg_mod_cache/google.golang.org/grpc@v1.57.1/rpc_util.go:796 +0xa7
google.golang.org/grpc.(*serverStream).RecvMsg(0xc0002d81e0, {0xb76ac0, 0xc00024e7c0})
	/home/runner/work/dd-trace-go/dd-trace-go/go_pkg_mod_cache/google.golang.org/grpc@v1.57.1/stream.go:1712 +0x16a
gopkg.in/DataDog/dd-trace-go.v1/contrib/google.golang.org/grpc.(*serverStream).RecvMsg(0xc00069a380, {0xb76ac0, 0xc00024e7c0})
	/home/runner/work/dd-trace-go/dd-trace-go/contrib/google.golang.org/grpc/server.go:60 +0x255
gopkg.in/DataDog/dd-trace-go.v1/contrib/google.golang.org/grpc.appsecServerStream.RecvMsg({{0xd6bd90, 0xc00069a380}, 0xc00064a150, {0xd69d58, 0xc0004564b0}}, {0xb76ac0, 0xc00024e7c0})
	/home/runner/work/dd-trace-go/dd-trace-go/contrib/google.golang.org/grpc/appsec.go:137 +0xcb
gopkg.in/DataDog/dd-trace-go.v1/contrib/google.golang.org/grpc.(*fixtureStreamPingServer).Recv(0xc000398220)
	/home/runner/work/dd-trace-go/dd-trace-go/contrib/google.golang.org/grpc/fixtures_test_grpc.pb.go:157 +0x46
gopkg.in/DataDog/dd-trace-go.v1/contrib/google.golang.org/grpc.(*fixtureServer).StreamPing(0xc0003d06c0, {0xd6dbf0, 0xc000398220})
	/home/runner/work/dd-trace-go/dd-trace-go/contrib/google.golang.org/grpc/grpc_test.go:547 +0x36
gopkg.in/DataDog/dd-trace-go.v1/contrib/google.golang.org/grpc.(*appsecFixtureServer).StreamPing(0xc0003d06c0, {0xd6dbf0, 0xc000398220})
	/home/runner/work/dd-trace-go/dd-trace-go/contrib/google.golang.org/grpc/appsec_test.go:507 +0xb0
gopkg.in/DataDog/dd-trace-go.v1/contrib/google.golang.org/grpc._Fixture_StreamPing_Handler({0xb40140?, 0xc0003d06c0}, {0xd6be20, 0xc000456750})
	/home/runner/work/dd-trace-go/dd-trace-go/contrib/google.golang.org/grpc/fixtures_test_grpc.pb.go:138 +0xd8
gopkg.in/DataDog/dd-trace-go.v1/contrib/google.golang.org/grpc.StreamServerInterceptor.func1.appsecStreamHandlerMiddleware.4({0xb40140, 0xc0003d06c0}, {0xd6bd90, 0xc00069a380})
	/home/runner/work/dd-trace-go/dd-trace-go/contrib/google.golang.org/grpc/appsec.go:116 +0x3bc
gopkg.in/DataDog/dd-trace-go.v1/contrib/google.golang.org/grpc.StreamServerInterceptor.func1({0xb40140, 0xc0003d06c0}, {0xd6bf88, 0xc0002d81e0}, 0xc0005cc000, 0xca0a70)
	/home/runner/work/dd-trace-go/dd-trace-go/contrib/google.golang.org/grpc/server.go:122 +0x502
google.golang.org/grpc.(*Server).processStreamingRPC(0xc0001421e0, {0xd6e4c0, 0xc000164680}, 0xc000693200, 0xc00068a240, 0x11f2280, 0x0)
	/home/runner/work/dd-trace-go/dd-trace-go/go_pkg_mod_cache/google.golang.org/grpc@v1.57.1/server.go:1659 +0x1285
google.golang.org/grpc.(*Server).handleStream(0xc0001421e0, {0xd6e4c0, 0xc000164680}, 0xc000693200, 0x0)
	/home/runner/work/dd-trace-go/dd-trace-go/go_pkg_mod_cache/google.golang.org/grpc@v1.57.1/server.go:1739 +0x999
google.golang.org/grpc.(*Server).serveStreams.func1.1()
	/home/runner/work/dd-trace-go/dd-trace-go/go_pkg_mod_cache/google.golang.org/grpc@v1.57.1/server.go:970 +0xbb
created by google.golang.org/grpc.(*Server).serveStreams.func1 in goroutine 322
	/home/runner/work/dd-trace-go/dd-trace-go/go_pkg_mod_cache/google.golang.org/grpc@v1.57.1/server.go:981 +0x136

goroutine 276 [select]:
github.com/DataDog/appsec-internal-go/limiter.(*TokenTicker).updateBucket(0xc0003544a0, {0xc0004b2790?, 0xc0004b2780?, 0x14760a0?}, 0x53d185?, 0xc0008ba660, 0x0)
	/home/runner/work/dd-trace-go/dd-trace-go/go_pkg_mod_cache/github.com/!data!dog/appsec-internal-go@v1.6.0/limiter/limiter.go:52 +0x105
created by github.com/DataDog/appsec-internal-go/limiter.(*TokenTicker).start in goroutine 275
	/home/runner/work/dd-trace-go/dd-trace-go/go_pkg_mod_cache/github.com/!data!dog/appsec-internal-go@v1.6.0/limiter/limiter.go:118 +0x105

goroutine 169 [chan receive]:
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).reportStats(0xc0003ac1e0)
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:386 +0x192
created by gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start in goroutine 156
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:349 +0xe6

goroutine 170 [sleep]:
time.Sleep(0x989680)
	/opt/hostedtoolcache/go/1.22.3/x64/src/runtime/time.go:195 +0x115
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).run(0xc0003ac1e0, 0xc00036e5a0)
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:332 +0x19f
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start.func1()
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:354 +0x98
created by gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start in goroutine 156
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:350 +0x125

goroutine 113 [chan receive]:
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).reportStats(0xc0003ac0f0)
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:386 +0x192
created by gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start in goroutine 108
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:349 +0xe6

goroutine 146 [sleep]:
time.Sleep(0x989680)
	/opt/hostedtoolcache/go/1.22.3/x64/src/runtime/time.go:195 +0x115
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).run(0xc0003ac0f0, 0xc00036e000)
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:332 +0x19f
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start.func1()
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:354 +0x98
created by gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start in goroutine 108
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:350 +0x125

goroutine 147 [select]:
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).runLoadAgentFeatures(0xc0003ac0f0, 0xc00055cc60)
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:514 +0x79
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start.func2()
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:360 +0x98
created by gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start in goroutine 108
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:356 +0x165

goroutine 250 [select, 9 minutes]:
google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader(0xc0002c0ea0)
	/home/runner/work/dd-trace-go/dd-trace-go/go_pkg_mod_cache/google.golang.org/grpc@v1.57.1/internal/transport/transport.go:331 +0x7c
google.golang.org/grpc/internal/transport.(*Stream).RecvCompress(...)
	/home/runner/work/dd-trace-go/dd-trace-go/go_pkg_mod_cache/google.golang.org/grpc@v1.57.1/internal/transport/transport.go:346
google.golang.org/grpc.(*csAttempt).recvMsg(0xc0008a2820, {0xb76b80, 0xc00040a040}, 0xc000541d18?)
	/home/runner/work/dd-trace-go/dd-trace-go/go_pkg_mod_cache/google.golang.org/grpc@v1.57.1/stream.go:1066 +0xc9
google.golang.org/grpc.(*clientStream).RecvMsg.func1(0x6461c0?)
	/home/runner/work/dd-trace-go/dd-trace-go/go_pkg_mod_cache/google.golang.org/grpc@v1.57.1/stream.go:917 +0x1f
google.golang.org/grpc.(*clientStream).withRetry(0xc0002c0900, 0xc000541e00, 0xc000541df0)
	/home/runner/work/dd-trace-go/dd-trace-go/go_pkg_mod_cache/google.golang.org/grpc@v1.57.1/stream.go:768 +0x13a
google.golang.org/grpc.(*clientStream).RecvMsg(0xc0002c0900, {0xb76b80?, 0xc00040a040?})
	/home/runner/work/dd-trace-go/dd-trace-go/go_pkg_mod_cache/google.golang.org/grpc@v1.57.1/stream.go:916 +0x11c
gopkg.in/DataDog/dd-trace-go.v1/contrib/google.golang.org/grpc.(*fixtureStreamPingClient).Recv(0xc0006461c0)
	/home/runner/work/dd-trace-go/dd-trace-go/contrib/google.golang.org/grpc/fixtures_test_grpc.pb.go:81 +0x46
gopkg.in/DataDog/dd-trace-go.v1/contrib/google.golang.org/grpc.TestUserBlocking.func5(0xc0003ec000)
	/home/runner/work/dd-trace-go/dd-trace-go/contrib/google.golang.org/grpc/appsec_test.go:304 +0x17d
testing.tRunner(0xc0003ec000, 0xc0003d0460)
	/opt/hostedtoolcache/go/1.22.3/x64/src/testing/testing.go:1689 +0xfb
created by testing.(*T).Run in goroutine 275
	/opt/hostedtoolcache/go/1.22.3/x64/src/testing/testing.go:1742 +0x390

goroutine 64 [chan receive]:
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).reportStats(0xc0002d80f0)
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:386 +0x192
created by gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start in goroutine 59
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:349 +0xe6

goroutine 65 [sleep]:
time.Sleep(0x989680)
	/opt/hostedtoolcache/go/1.22.3/x64/src/runtime/time.go:195 +0x115
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).run(0xc0002d80f0, 0xc0000b6c60)
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:332 +0x19f
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start.func1()
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:354 +0x98
created by gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start in goroutine 59
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:350 +0x125

goroutine 242 [select]:
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).runLoadAgentFeatures(0xc0002d80f0, 0xc0000b69c0)
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:514 +0x79
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start.func2()
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:360 +0x98
created by gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start in goroutine 59
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:356 +0x165

goroutine 307 [sleep]:
time.Sleep(0x989680)
	/opt/hostedtoolcache/go/1.22.3/x64/src/runtime/time.go:195 +0x115
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).run(0xc0003ac5a0, 0xc000132c00)
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:332 +0x19f
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start.func1()
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:354 +0x98
created by gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start in goroutine 294
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:350 +0x125

goroutine 230 [select, 9 minutes]:
google.golang.org/grpc/internal/transport.(*controlBuffer).get(0xc0003304b0, 0x1)
	/home/runner/work/dd-trace-go/dd-trace-go/go_pkg_mod_cache/google.golang.org/grpc@v1.57.1/internal/transport/controlbuf.go:418 +0x113
google.golang.org/grpc/internal/transport.(*loopyWriter).run(0xc000216150)
	/home/runner/work/dd-trace-go/dd-trace-go/go_pkg_mod_cache/google.golang.org/grpc@v1.57.1/internal/transport/controlbuf.go:552 +0x86
google.golang.org/grpc/internal/transport.newHTTP2Client.func6()
	/home/runner/work/dd-trace-go/dd-trace-go/go_pkg_mod_cache/google.golang.org/grpc@v1.57.1/internal/transport/http2_client.go:451 +0x85
created by google.golang.org/grpc/internal/transport.newHTTP2Client in goroutine 254
	/home/runner/work/dd-trace-go/dd-trace-go/go_pkg_mod_cache/google.golang.org/grpc@v1.57.1/internal/transport/http2_client.go:449 +0x22fb

goroutine 283 [sleep]:
time.Sleep(0x989680)
	/opt/hostedtoolcache/go/1.22.3/x64/src/runtime/time.go:195 +0x115
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).run(0xc0003ac4b0, 0xc000[3820](https://github.com/DataDog/dd-trace-go/actions/runs/9318622002/job/25728313367#step:7:3821)60)
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:332 +0x19f
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start.func1()
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:354 +0x98
created by gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start in goroutine 277
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:350 +0x125

goroutine 284 [select]:
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).runLoadAgentFeatures(0xc0003ac4b0, 0xc00036ed80)
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:514 +0x79
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start.func2()
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:360 +0x98
created by gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start in goroutine 277
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:356 +0x165

goroutine 187 [chan receive]:
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).reportStats(0xc0003ac3c0)
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:386 +0x192
created by gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start in goroutine 182
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:349 +0xe6

goroutine 188 [sleep]:
time.Sleep(0x989680)
	/opt/hostedtoolcache/go/1.22.3/x64/src/runtime/time.go:195 +0x115
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).run(0xc0003ac3c0, 0xc000[3833](https://github.com/DataDog/dd-trace-go/actions/runs/9318622002/job/25728313367#step:7:3834)e0)
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:332 +0x19f
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start.func1()
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:354 +0x98
created by gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start in goroutine 182
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:350 +0x125

goroutine 189 [select]:
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).runLoadAgentFeatures(0xc0003ac3c0, 0xc000383320)
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:514 +0x79
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start.func2()
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:360 +0x98
created by gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start in goroutine 182
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:356 +0x165

goroutine 251 [IO wait, 9 minutes]:
internal/poll.runtime_pollWait(0x7fea652a1a90, 0x72)
	/opt/hostedtoolcache/go/1.22.3/x64/src/runtime/netpoll.go:345 +0x85
internal/poll.(*pollDesc).wait(0x7?, 0x1?, 0x0)
	/opt/hostedtoolcache/go/1.22.3/x64/src/internal/poll/fd_poll_runtime.go:84 +0x27
internal/poll.(*pollDesc).waitRead(...)
	/opt/hostedtoolcache/go/1.22.3/x64/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Accept(0xc0004b6300)
	/opt/hostedtoolcache/go/1.22.3/x64/src/internal/poll/fd_unix.go:611 +0x2ac
net.(*netFD).accept(0xc0004b6300)
	/opt/hostedtoolcache/go/1.22.3/x64/src/net/fd_unix.go:172 +0x29
net.(*TCPListener).accept(0xc0004c2180)
	/opt/hostedtoolcache/go/1.22.3/x64/src/net/tcpsock_posix.go:159 +0x1e
net.(*TCPListener).Accept(0xc0004c2180)
	/opt/hostedtoolcache/go/1.22.3/x64/src/net/tcpsock.go:327 +0x30
google.golang.org/grpc.(*Server).Serve(0xc0001421e0, {0xd684f8, 0xc0004c2180})
	/home/runner/work/dd-trace-go/dd-trace-go/go_pkg_mod_cache/google.golang.org/grpc@v1.57.1/server.go:837 +0x469
created by gopkg.in/DataDog/dd-trace-go.v1/contrib/google.golang.org/grpc.newAppsecRig in goroutine 250
	/home/runner/work/dd-trace-go/dd-trace-go/contrib/google.golang.org/grpc/appsec_test.go:456 +0x386

goroutine 255 [chan receive]:
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).reportStats(0xc0003ac690)
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:386 +0x192
created by gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start in goroutine 250
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:349 +0xe6

goroutine 256 [sleep]:
time.Sleep(0x989680)
	/opt/hostedtoolcache/go/1.22.3/x64/src/runtime/time.go:195 +0x115
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).run(0xc0003ac690, 0xc000418780)
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:332 +0x19f
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start.func1()
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:354 +0x98
created by gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start in goroutine 250
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:350 +0x125

goroutine 257 [select]:
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).runLoadAgentFeatures(0xc0003ac690, 0xc0004186c0)
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:514 +0x79
gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start.func2()
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:360 +0x98
created by gopkg.in/DataDog/dd-trace-go.v1/internal/datastreams.(*Processor).Start in goroutine 250
	/home/runner/work/dd-trace-go/dd-trace-go/internal/datastreams/processor.go:356 +0x165

goroutine 192 [select, 9 minutes]:
google.golang.org/grpc/internal/transport.(*controlBuffer).get(0xc0008105f0, 0x1)
	/home/runner/work/dd-trace-go/dd-trace-go/go_pkg_mod_cache/google.golang.org/grpc@v1.57.1/internal/transport/controlbuf.go:418 +0x113
google.golang.org/grpc/internal/transport.(*loopyWriter).run(0xc00064a000)
	/home/runner/work/dd-trace-go/dd-trace-go/go_pkg_mod_cache/google.golang.org/grpc@v1.57.1/internal/transport/controlbuf.go:552 +0x86
google.golang.org/grpc/internal/transport.NewServerTransport.func2()
	/home/runner/work/dd-trace-go/dd-trace-go/go_pkg_mod_cache/google.golang.org/grpc@v1.57.1/internal/transport/http2_server.go:336 +0xd5
created by google.golang.org/grpc/internal/transport.NewServerTransport in goroutine 191
	/home/runner/work/dd-trace-go/dd-trace-go/go_pkg_mod_cache/google.golang.org/grpc@v1.57.1/internal/transport/http2_server.go:333 +0x1a6c

goroutine 193 [select, 9 minutes]:
google.golang.org/grpc/internal/transport.(*http2Server).keepalive(0xc000164680)
	/home/runner/work/dd-trace-go/dd-trace-go/go_pkg_mod_cache/google.golang.org/grpc@v1.57.1/internal/transport/http2_server.go:1150 +0x205
created by google.golang.org/grpc/internal/transport.NewServerTransport in goroutine 191
	/home/runner/work/dd-trace-go/dd-trace-go/go_pkg_mod_cache/google.golang.org/grpc@v1.57.1/internal/transport/http2_server.go:339 +0x1aae

goroutine 322 [IO wait, 9 minutes]:
internal/poll.runtime_pollWait(0x7fea652a1e70, 0x72)
	/opt/hostedtoolcache/go/1.22.3/x64/src/runtime/netpoll.go:345 +0x85
internal/poll.(*pollDesc).wait(0xc00046e000?, 0xc0004e8000?, 0x0)
	/opt/hostedtoolcache/go/1.22.3/x64/src/internal/poll/fd_poll_runtime.go:84 +0x27
internal/poll.(*pollDesc).waitRead(...)
	/opt/hostedtoolcache/go/1.22.3/x64/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc00046e000, {0xc0004e8000, 0x8000, 0x8000})
	/opt/hostedtoolcache/go/1.22.3/x64/src/internal/poll/fd_unix.go:164 +0x27a
net.(*netFD).Read(0xc00046e000, {0xc0004e8000?, 0xc0488f?, 0xc00030c000?})
	/opt/hostedtoolcache/go/1.22.3/x64/src/net/fd_posix.go:55 +0x25
net.(*conn).Read(0xc000814010, {0xc0004e8000?, 0x0?, 0xc00049c060?})
	/opt/hostedtoolcache/go/1.22.3/x64/src/net/net.go:179 +0x45
bufio.(*Reader).Read(0xc00055c540, {0xc000306040, 0x9, 0xca2bb0?})
	/opt/hostedtoolcache/go/1.22.3/x64/src/bufio/bufio.go:241 +0x197
io.ReadAtLeast({0xd636a0, 0xc00055c540}, {0xc000306040, 0x9, 0x9}, 0x9)
	/opt/hostedtoolcache/go/1.22.3/x64/src/io/io.go:335 +0x90
io.ReadFull(...)
	/opt/hostedtoolcache/go/1.22.3/x64/src/io/io.go:354
golang.org/x/net/http2.readFrameHeader({0xc000306040, 0x9, 0xc00049c030?}, {0xd636a0?, 0xc00055c540?})
	/home/runner/work/dd-trace-go/dd-trace-go/go_pkg_mod_cache/golang.org/x/net@v0.23.0/http2/frame.go:237 +0x65
golang.org/x/net/http2.(*Framer).ReadFrame(0xc000306000)
	/home/runner/work/dd-trace-go/dd-trace-go/go_pkg_mod_cache/golang.org/x/net@v0.23.0/http2/frame.go:498 +0x85
google.golang.org/grpc/internal/transport.(*http2Server).HandleStreams(0xc000164680, 0xc0003be5a0, 0xca1220)
	/home/runner/work/dd-trace-go/dd-trace-go/go_pkg_mod_cache/google.golang.org/grpc@v1.57.1/internal/transport/http2_server.go:637 +0x14e
google.golang.org/grpc.(*Server).serveStreams(0xc0001421e0, {0xd6e4c0, 0xc000164680})
	/home/runner/work/dd-trace-go/dd-trace-go/go_pkg_mod_cache/google.golang.org/grpc@v1.57.1/server.go:963 +0x1bf
google.golang.org/grpc.(*Server).handleRawConn.func1()
	/home/runner/work/dd-trace-go/dd-trace-go/go_pkg_mod_cache/google.golang.org/grpc@v1.57.1/server.go:905 +0x45
created by google.golang.org/grpc.(*Server).handleRawConn in goroutine 191
	/home/runner/work/dd-trace-go/dd-trace-go/go_pkg_mod_cache/google.golang.org/grpc@v1.57.1/server.go:904 +0x15b
FAIL	gopkg.in/DataDog/dd-trace-go.v1/contrib/google.golang.org/grpc	600.015s

contrib/net/http/roundtripper.go Outdated Show resolved Hide resolved
@eliottness eliottness force-pushed the francois.mazeau/ssrf-plug branch 2 times, most recently from 42d91b4 to 85a9bfd Compare June 3, 2024 12:48
eliottness and others added 22 commits June 3, 2024 16:29
@eliottness
appsec: support for SSRF Exploit Prevention
8a6bac3 
Signed-off-by: Eliott Bouhana <eliott.bouhana@datadoghq.com>
@eliottness
remove spanID and nested event span_id
95f8872 
Signed-off-by: Eliott Bouhana <eliott.bouhana@datadoghq.com>
@eliottness
TEMP!: hardcode RASP SSRF rule & span tag for staging
ab7887e 
Signed-off-by: Eliott Bouhana <eliott.bouhana@datadoghq.com>
@Hellzy @eliottness
listeners: remove timeout arg for roundtriper
81aabd8
@Hellzy @eliottness
waf_test: used sharedsec.HTTPClientIPAddre
314c6e2
@Hellzy @eliottness
waf.go: remove hardcoded rasp ssrf rule
55e8fec
@Hellzy @eliottness
http: add stacktraces to span
c63c77e
@Hellzy @eliottness
Register RC capa and check RASP enablement
171c9a6
@Hellzy @eliottness
imports police
e8a34d3
@Hellzy @eliottness
net/http: remove rasp test tag
b93e0dc
@Hellzy @eliottness
emitter: simplify to 1 generated stacktrace only
709b466
@Hellzy @eliottness
Don't add nil stacktraces to spans
f045f29
@eliottness
internal/appsec: refactor roundtripper and add new events.SecurityBlo…
5619fa4 
…ckingEvent type to the public API

Signed-off-by: Eliott Bouhana <eliott.bouhana@datadoghq.com>
@eliottness
rename stack_trace action in generate_stack as per libddwaf upgrading…
cd8164a 
… guide

Signed-off-by: Eliott Bouhana <eliott.bouhana@datadoghq.com>
@eliottness
fix multiple issues:
985b0aa 
* wrong action name: stack_trace -> generate_stack
* stacktrace msgp serialization error: wait for new version for better support
* dyngo.EmitData on parent operation instead of child operation because of variable shadowing
* update contrib roundtripper to ignore events.SecurityBlockingEvent

Signed-off-by: Eliott Bouhana <eliott.bouhana@datadoghq.com>
@eliottness
rename appsec/events.SecurityBlockingEvent -> BlockingSecurityEvent
a24d23a 
Signed-off-by: Eliott Bouhana <eliott.bouhana@datadoghq.com>
@eliottness
misc suggestions from @Julio-Guerra
b2f4edf 
Signed-off-by: Eliott Bouhana <eliott.bouhana@datadoghq.com>
@eliottness
appsec: grpc: remove MonitoringError and fix user blocking
463d661 
Signed-off-by: Eliott Bouhana <eliott.bouhana@datadoghq.com>
@eliottness
run go mod tidy
dc164df 
Signed-off-by: Eliott Bouhana <eliott.bouhana@datadoghq.com>
@eliottness
internal/appsec: add appsec.RASPEnabled() and use it to disable RASP …
8d6f944 
…when needed

Signed-off-by: Eliott Bouhana <eliott.bouhana@datadoghq.com>
@eliottness @Julio-Guerra
Update internal/appsec/emitter/httpsec/roundtripper.go
c835a08 
Co-authored-by: Julio Guerra <julio.guerra@datadoghq.com>
@eliottness
order imports
aef04d2 
Signed-off-by: Eliott Bouhana <eliott.bouhana@datadoghq.com>
@eliottness eliottness merged commit 6126fb6 into main Jun 3, 2024
199 of 200 checks passed
@eliottness eliottness deleted the francois.mazeau/ssrf-plug branch June 3, 2024 14:50
@eliottness eliottness mentioned this pull request Jun 20, 2024
Merged
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@github-actions github-actions[bot] github-actions[bot] left review comments

@darccio darccio darccio approved these changes

@Julio-Guerra Julio-Guerra Julio-Guerra approved these changes

@eliottness eliottness eliottness approved these changes

Assignees
No one assigned
Labels
apm:ecosystem contrib/* related feature requests or bugs appsec
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@Hellzy @darccio @Julio-Guerra @eliottness