[WIP] Fuzzing hackathon

[belimawr]

What does this PR do?

Add fuzzy tests

Why is it important?

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

• [ ]

How to test this PR locally

Related issues

• It seems there is a bug with go fuzz that breaks one of our tests: testing: fuzzer aborts with "fuzzing process hung or terminated unexpectedly while minimizing: EOF" golang/go#52569 (comment)

Use cases

Screenshots

Logs

[botelastic]

This pull request doesn't have a Team:<team> label.

[mergify]

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @belimawr? 🙏.
For such, you'll need to label your PR with:

• The upcoming major version of the Elastic Stack
• The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

[elasticmachine]

💔 Tests Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2023-07-03T13:25:17.790+0000

• Duration: 55 min 9 sec

Test stats 🧪

Test	Results
Failed	2
Passed	8037
Skipped	757
Total	8796

Test errors

Expand to view the tests failures

Build&Test / filebeat-goIntegTest / FuzzFilestreamID/493ec6c1b7733f03 – github.com/elastic/beats/v7/filebeat/input/filestream

Expand to view the error details

 Failed 

Expand to view the stacktrace

 === RUN   FuzzFilestreamID/493ec6c1b7733f03
    environment_test.go:367: keys in store: [filestream::$::native::3951338-2049]
    environment_test.go:231: could not get state for 'filestream::$$::native::3951338-2049' from registry, err: error when getting expected key 'filestream::$$::native::3951338-2049' from store: failed in store/get operation on store 'filebeat': unknown key
    input_integration_test.go:1107: 
        	Error Trace:	/var/lib/jenkins/workspace/PR-35962-3-f677346b-ec57-4445-ae23-df3fd206a7f9/src/github.com/elastic/beats/filebeat/input/filestream/environment_test.go:226
        	            				/var/lib/jenkins/workspace/PR-35962-3-f677346b-ec57-4445-ae23-df3fd206a7f9/src/github.com/elastic/beats/filebeat/input/filestream/input_integration_test.go:1107
        	            				/var/lib/jenkins/workspace/PR-35962-3-f677346b-ec57-4445-ae23-df3fd206a7f9/.gvm/versions/go1.19.10.linux.amd64/src/reflect/value.go:584
        	            				/var/lib/jenkins/workspace/PR-35962-3-f677346b-ec57-4445-ae23-df3fd206a7f9/.gvm/versions/go1.19.10.linux.amd64/src/reflect/value.go:368
        	            				/var/lib/jenkins/workspace/PR-35962-3-f677346b-ec57-4445-ae23-df3fd206a7f9/.gvm/versions/go1.19.10.linux.amd64/src/testing/fuzz.go:337
        	Error:      	Condition never satisfied
        	Test:       	FuzzFilestreamID/493ec6c1b7733f03
        	Messages:   	expected offset: '15', cursor offset: ''
    environment_test.go:75: Debug Logs:
    environment_test.go:81: {"Level":"debug","Time":"2023-07-03T13:49:29.729443303Z","LoggerName":"scanner","Message":"recursive glob enabled","Caller":{"Defined":true,"PC":11727406,"File":"/var/lib/jenkins/workspace/PR-35962-3-f677346b-ec57-4445-ae23-df3fd206a7f9/src/github.com/elastic/beats/filebeat/input/filestream/fswatch.go","Line":286,"Function":"github.com/elastic/beats/v7/filebeat/input/filestream.(*fileScanner).resolveRecursiveGlobs"},"Stack":"","Context":[]}
    environment_test.go:81: {"Level":"info","Time":"2023-07-03T13:49:29.729594741Z","LoggerName":"metric_registry","Message":"registering","Caller":{"Defined":true,"PC":10788686,"File":"/var/lib/jenkins/workspace/PR-35962-3-f677346b-ec57-4445-ae23-df3fd206a7f9/src/github.com/elastic/beats/libbeat/monitoring/inputmon/input.go","Line":63,"Function":"github.com/elastic/beats/v7/libbeat/monitoring/inputmon.NewInputRegistry"},"Stack":"","Context":[{"Key":"input_type","Type":15,"Integer":0,"String":"filestream","Interface":null},{"Key":"id","Type":15,"Integer":0,"String":"fake-ID","Interface":null},{"Key":"key","Type":15,"Integer":0,"String":"fake-ID","Interface":null},{"Key":"uuid","Type":15,"Integer":0,"String":"7ebe558d-d716-428c-9b7a-224cce968774","Interface":null}]}
    environment_test.go:81: {"Level":"debug","Time":"2023-07-03T13:49:29.729655699Z","LoggerName":"","Message":"Starting prospector","Caller":{"Defined":true,"PC":11758890,"File":"/var/lib/jenkins/workspace/PR-35962-3-f677346b-ec57-4445-ae23-df3fd206a7f9/src/github.com/elastic/beats/filebeat/input/filestream/prospector.go","Line":133,"Function":"github.com/elastic/beats/v7/filebeat/input/filestream.(*fileProspector).Run"},"Stack":"","Context":[{"Key":"prospector","Type":15,"Integer":0,"String":"file_prospector","Interface":null}]}
    environment_test.go:81: {"Level":"debug","Time":"2023-07-03T13:49:29.729679455Z","LoggerName":"file_watcher","Message":"Start next scan","Caller":{"Defined":true,"PC":11722290,"File":"/var/lib/jenkins/workspace/PR-35962-3-f677346b-ec57-4445-ae23-df3fd206a7f9/src/github.com/elastic/beats/filebeat/input/filestream/fswatch.go","Line":139,"Function":"github.com/elastic/beats/v7/filebeat/input/filestream.(*fileWatcher).watch"},"Stack":"","Context":[]}
    environment_test.go:81: {"Level":"debug","Time":"2023-07-03T13:49:29.729742758Z","LoggerName":"file_watcher","Message":"Found 1 paths","Caller":{"Defined":true,"PC":11725710,"File":"/var/lib/jenkins/workspace/PR-35962-3-f677346b-ec57-4445-ae23-df3fd206a7f9/src/github.com/elastic/beats/filebeat/input/filestream/fswatch.go","Line":213,"Function":"github.com/elastic/beats/v7/filebeat/input/filestream.(*fileWatcher).watch"},"Stack":"","Context":[]}
    environment_test.go:81: {"Level":"debug","Time":"2023-07-03T13:49:29.729775259Z","LoggerName":"","Message":"A new file /tmp/FuzzFilestreamID493ec6c1b7733f034290762578/001/test.log has been found","Caller":{"Defined":true,"PC":11762101,"File":"/var/lib/jenkins/workspace/PR-35962-3-f677346b-ec57-4445-ae23-df3fd206a7f9/src/github.com/elastic/beats/filebeat/input/filestream/prospector.go","Line":179,"Function":"github.com/elastic/beats/v7/filebeat/input/filestream.(*fileProspector).onFSEvent"},"Stack":"","Context":[{"Key":"prospector","Type":15,"Integer":0,"String":"file_prospector","Interface":null},{"Key":"operation","Type":15,"Integer":0,"String":"create","Interface":null},{"Key":"source_name","Type":15,"Integer":0,"String":"native::3951338-2049","Interface":null},{"Key":"os_id","Type":25,"Integer":0,"String":"","Interface":{"inode":3951338,"device":2049}},{"Key":"new_path","Type":15,"Integer":0,"String":"/tmp/FuzzFilestreamID493ec6c1b7733f034290762578/001/test.log","Interface":null}]}
    environment_test.go:81: {"Level":"debug","Time":"2023-07-03T13:49:29.729800262Z","LoggerName":"","Message":"Starting harvester for file","Caller":{"Defined":true,"PC":11346438,"File":"/var/lib/jenkins/workspace/PR-35962-3-f677346b-ec57-4445-ae23-df3fd206a7f9/src/github.com/elastic/beats/filebeat/input/filestream/internal/input-logfile/harvester.go","Line":139,"Function":"github.com/elastic/beats/v7/filebeat/input/filestream/internal/input-logfile.(*defaultHarvesterGroup).Start"},"Stack":"","Context":[{"Key":"source_file","Type":15,"Integer":0,"String":"filestream::$::native::3951338-2049","Interface":null}]}
    environment_test.go:81: {"Level":"debug","Time":"2023-07-03T13:49:29.729874205Z","LoggerName":"","Message":"newLogFileReader with config.MaxBytes:10485760","Caller":{"Defined":true,"PC":11745393,"File":"/var/lib/jenkins/workspace/PR-35962-3-f677346b-ec57-4445-ae23-df3fd206a7f9/src/github.com/elastic/beats/filebeat/input/filestream/input.go","Line":183,"Function":"github.com/elastic/beats/v7/filebeat/input/filestream.(*filestream).open"},"Stack":"","Context":[{"Key":"source_file","Type":15,"Integer":0,"String":"filestream::$::native::3951338-2049","Interface":null},{"Key":"path","Type":15,"Integer":0,"String":"/tmp/FuzzFilestreamID493ec6c1b7733f034290762578/001/test.log","Interface":null},{"Key":"state-id","Type":15,"Integer":0,"String":"native::3951338-2049","Interface":null}]}
    environment_test.go:81: {"Level":"debug","Time":"2023-07-03T13:49:29.72993365Z","LoggerName":"","Message":"End of file reached: /tmp/FuzzFilestreamID493ec6c1b7733f034290762578/001/test.log; Backoff now.","Caller":{"Defined":true,"PC":11714623,"File":"/var/lib/jenkins/workspace/PR-35962-3-f677346b-ec57-4445-ae23-df3fd206a7f9/src/github.com/elastic/beats/filebeat/input/filestream/filestream.go","Line":131,"Function":"github.com/elastic/beats/v7/filebeat/input/filestream.(*logFile).Read"},"Stack":"","Context":[{"Key":"source_file","Type":15,"Integer":0,"String":"filestream::$::native::3951338-2049","Interface":null},{"Key":"path","Type":15,"Integer":0,"String":"/tmp/FuzzFilestreamID493ec6c1b7733f034290762578/001/test.log","Interface":null},{"Key":"state-id","Type":15,"Integer":0,"String":"native::3951338-2049","Interface":null}]}
    --- FAIL: FuzzFilestreamID/493ec6c1b7733f03 (1.01s)
 

Build&Test / filebeat-goIntegTest / FuzzFilestreamID – github.com/elastic/beats/v7/filebeat/input/filestream

Expand to view the error details

 Failed 

Expand to view the stacktrace

 === RUN   FuzzFilestreamID
--- FAIL: FuzzFilestreamID (15.76s)
 

Steps errors

Expand to view the steps failures

filebeat-goIntegTest - mage goIntegTest

• Took 6 min 42 sec . View more details here
• Description: mage goIntegTest

filebeat-goIntegTest - mage goIntegTest

• Took 3 min 59 sec . View more details here
• Description: mage goIntegTest

filebeat-goIntegTest - mage goIntegTest

• Took 3 min 57 sec . View more details here
• Description: mage goIntegTest

Error signal

• Took 0 min 0 sec . View more details here
• Description: Error 'hudson.AbortException: script returned exit code 1'

🐛 Flaky test report

❕ There are test failures but not known flaky tests.

Expand to view the summary

Genuine test errors

💔 There are test failures but not known flaky tests, most likely a genuine test failure.

• Name: Build&Test / filebeat-goIntegTest / FuzzFilestreamID/493ec6c1b7733f03 – github.com/elastic/beats/v7/filebeat/input/filestream
• Name: Build&Test / filebeat-goIntegTest / FuzzFilestreamID – github.com/elastic/beats/v7/filebeat/input/filestream

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

• /package : Generate the packages and run the E2E tests.

• /beats-tester : Run the installation tests with beats-tester.

• run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

[mergify]

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @belimawr? 🙏.
For such, you'll need to label your PR with:

• The upcoming major version of the Elastic Stack
• The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

[mergify]

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b fuzzing-hackathon upstream/fuzzing-hackathon
git merge upstream/main
git push upstream fuzzing-hackathon