Update filebeat httpjson input to support pagination via Header and Okta module

[alakahakai]

Update filebeat httpjson input to support:

• Pagination via HTTP Header fields and regexp based matching
• Rate limit via HTTP Header fields
• Okta module

[elasticmachine]

Pinging @elastic/siem (Team:SIEM)

[andrewkroh]

I think it would be good to try to split some of these larger functions into some smaller ones that can be more easily unit tested. processHTTPRequest is getting kind of long and doesn't have high test coverage.

• coverage-report.html.txt (added txt extension to be able to attach)

The new config parameters need to be added to the docs.

[andrewkroh]

I see no issues with the changes 👍 . Can you please add some tests for the new features. The rate limiting, pagination, and config validation code doesn't get much exercise 🚲 in the tests. It might be easier to test some of cases if you can split the code into pieces that can be synchronously unit tested.

[andrewkroh]

Thank you for adding config validation test cases. Are you able to add some test coverage for the applyRateLimit and the pagination block in processHTTPRequest?

[andrewkroh]

Thank you for adding more tests. LGTM. Just one minor comment.