Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[receiver/windowseventlog] Panic when buffer size is 0 #17878

Closed
cpheps opened this issue Jan 20, 2023 · 2 comments · Fixed by #17881
Closed

[receiver/windowseventlog] Panic when buffer size is 0 #17878

cpheps opened this issue Jan 20, 2023 · 2 comments · Fixed by #17881
Assignees
Labels
bug Something isn't working needs triage New item requiring triage receiver/windowseventlog

Comments

@cpheps
Copy link
Contributor

cpheps commented Jan 20, 2023

Component(s)

Windows Event Log Receiver

What happened?

Description

An index out of bounds panic can occasionally occur in the windowseventlog receiver here at buffer.FirstByte().

We believe this happens in due to the bufferUsed being passed in as a pointer to be used as an unsafe pointer inside the Windows system call in evtFormatMessage. In some cases the unsafe pointer likely loses it's reference in the stack frame shift out of the function and this sets bufferUsed to it's zero value. When this error case is triggered and bufferUsed has a value of 0 it resizes the buffer to 0 then the function makes a recursive call to itself. On the next call to buffer.FirstByte() a panic occurs due to indexing into a 0 sized buffer.

When developing this receiver my team hit a similar issue with unsafe pointers in evtRender and changed bufferUsed to be a return value rather than being passed in.

We didn't apply the same logic to evtFormatMessage at the time because it never hit an issue. I propose we copy the same strategy as evtRender with how we use the bufferUsed value.

It's also probably worth doing a safety check on bufferUsed in the case it returns a ErrorInsufficientBuffer and return an error if bufferUsed is 0.

Steps to Reproduce

We were not able to reliably reproduce this it just occasionally would occur if we left it running for several days.

Collector version

v0.66.0

Environment information

Environment

OS: Windows Sever 2022

OpenTelemetry Collector configuration

No response

Log output

No response

Additional context

We have tested the proposed fix and left it run in the same environment for two weeks that was seeing the issue and have not seen any panics.

@cpheps cpheps added bug Something isn't working needs triage New item requiring triage labels Jan 20, 2023
@github-actions
Copy link
Contributor

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@cpheps
Copy link
Contributor Author

cpheps commented Jan 20, 2023

I'd like to be assigned to this as we already have a fix that we would like to submit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working needs triage New item requiring triage receiver/windowseventlog
Projects
None yet
1 participant