Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Filebeat copy_fields processor can recurse, leading to crash #19206

Closed
fearful-symmetry opened this issue Jun 15, 2020 · 2 comments
Closed

Filebeat copy_fields processor can recurse, leading to crash #19206

fearful-symmetry opened this issue Jun 15, 2020 · 2 comments
Assignees
Labels
bug :Processors Team:Integrations Label for the Integrations team

Comments

@fearful-symmetry
Copy link
Contributor

It's possible for the copy_fields processor to fall into infinite recursion if we point at two fields with the same root:

                    - copy_fields:
                        fields:
                          - from: message
                            to: message.original

The from and to fields will point to the same root, leading an eventual crash. It appears copy_fields doesn't do an actual copy and just moves references. @urso discovered the root cause here.

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jun 15, 2020
@fearful-symmetry fearful-symmetry added the Team:Integrations Label for the Integrations team label Jun 15, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations (Team:Integrations)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Jun 15, 2020
@urso
Copy link

urso commented Jun 15, 2020

copy source: https://github.com/elastic/beats/blob/master/libbeat/processors/actions/copy_fields.go#L99

slices and maps are stored by reference. The code does not handle these types correctly, which leads to a child-document pointing to it's root. Full copy semantics are required.

Another bug we see due to these reference semantics copy_fields => delete field in the original document will also remove the deleted field from the copied sub-document. The copy_fields processor should provide copy semantics always.

@kvch kvch closed this as completed in 0940e25 Aug 19, 2020
@zube zube bot added [zube]: Done and removed [zube]: Ready labels Aug 19, 2020
melchiormoulin pushed a commit to melchiormoulin/beats that referenced this issue Oct 14, 2020
@zube zube bot removed the [zube]: Done label Nov 18, 2020
jsoriano pushed a commit to jsoriano/beats that referenced this issue Jun 21, 2021
jsoriano pushed a commit to jsoriano/beats that referenced this issue Jun 21, 2021
jsoriano added a commit that referenced this issue Jun 22, 2021
…26382)

Closes #19206

(cherry picked from commit 0940e25)

Co-authored-by: ian woolf <btw515wolf2@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug :Processors Team:Integrations Label for the Integrations team
Projects
None yet
Development

No branches or pull requests

4 participants