[Filebeat][module][elastisearch] exclude gc and audit log from server fileset

[alioug]

What does this PR do?

Fix filebeat sending gc.log even if gc fileset is disabled

Why is it important?

gc.log is verbose and unnecessarily fills filebeat indices/data streams

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

• Check gc.log is sended when gc fileset is enabled

How to test this PR locally

Install filebeat and elasticsearch locally, only enable server fileset (disable gc fileset) and check in filebeat indices there is no more gc logs.

Related issues

• Closes Filebeat elasticsearch module sending gc.log even if only server submodule is enabled #30995

Use cases

Screenshots

Logs

[mergify]

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b fix-filebeat-es-module upstream/fix-filebeat-es-module
git merge upstream/main
git push upstream fix-filebeat-es-module

[mergify]

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @alioug? 🙏.
For such, you'll need to label your PR with:

• The upcoming major version of the Elastic Stack
• The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2022-09-13T02:50:54.506+0000

• Duration: 12 min 7 sec

❕ Flaky test report

No test was executed to be analysed.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

• /package : Generate the packages and run the E2E tests.

• /beats-tester : Run the installation tests with beats-tester.

• run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

[mergify]

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b fix-filebeat-es-module upstream/fix-filebeat-es-module
git merge upstream/main
git push upstream fix-filebeat-es-module

[klacabane]

Another workaround would be specifying more precise path names in the filebeat configuration to avoid pulling unnecessary files, but it's great to add guards around the suggested defaults. Going that route we'd also have to filter out audit files right ? Yes it's mentioned in #30995 (comment)

[klacabane]

@alioug are you still interested in merging this change ? We should also filter out audit logs to be exhaustive

[alioug]

yes of course, i will add the audit log file.

[klacabane]

/test

[matschaffer]

At first I thought we should add rotated GC as well based on https://github.com/elastic/beats/blob/main/filebeat/module/elasticsearch/gc/manifest.yml#L6

But since the manifest default is *.log I think only the initial gc log will get picked up. LGTM!

[matschaffer]

/test

[matschaffer]

I've assigned myself to this and the associated issue to make sure we merge it for @alioug once it's green.

[matschaffer]

All green! Merging. Thanks for the fix @alioug !