[Filebeat] Check expand_event_list_from_field before checking content-type

[kaiyan-sheng]

What does this PR do?

This PR is to fix s3 input reading json format logs. When expand_event_list_from_field config is given, such as cloudtrail fileset, s3 input should use json.NewDecoder first. If this expand_event_list_from_field is not given, then check content-type and use gzip.NewReader.

Why is it important?

This will fix s3 input when using cloudtrail fileset.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works

Related issues

• Closes [Filebeat] S3 input issue for cloudtrail: createEventsFromS3Info failed #16374

Screenshots

When Metadata is set as below:

I was able to read the cloudtrail json.gz file as shown below in Kibana:

[ChrsMark]

the code change lgtm!

I have a question if these config options and how all of them are combined and prioritised are documented for the users. Would this be useful? 🤔

[kaiyan-sheng]

the code change lgtm!

I have a question if these config options and how all of them are combined and prioritized are documented for the users. Would this be useful? 🤔

Ahh good point! I don't think we have how these options combined and prioritized documented! Let me add some under expand_event_list_from_field. Thanks!