Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Filebeat] Document netflow internal_networks and set default #24110

Merged
merged 3 commits into from
Feb 20, 2021

Conversation

andrewkroh
Copy link
Member

What does this PR do?

Documentation for the internal_networks option of the Netflow input and module was missing.
Also the module's manifest did not declare the option so if it was not set in the module config
it would cause an error.

I did not see where a default was set for the netflow input's internal_networks option so I set that
to private to keep the old behavior before this was configurable.

Fixes #24094

Why is it important?

Existing users of the netflow module are broken after upgrade unless they change their config.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Related issues

@elasticmachine
Copy link
Collaborator

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Feb 18, 2021
Documentation for the `internal_networks` option of the Netflow input and module was missing.
Also the module's manifest did not declare the option so if it was not set in the module config
it would cause an error.

I did not see where a default was set for the netflow input's internal_networks option so I set that
to `private` to keep the old behavior before this was configurable.

Fixes elastic#24094
@andrewstucki
Copy link
Contributor

andrewstucki commented Feb 18, 2021

@andrewkroh wow, not sure how we didn't catch this previously, but I checked over the previous changes again and we also look like we have an issue with the configuration. In filebeat/module/netflow/log/config/netflow.yml:

{{if .internal_networks}}
internal_hosts:
{{range .internal_networks}}
- '{{ . }}'
{{end}}
{{end}}

notice the internal_hosts v. internal_networks. When the input has:

 InternalNetworks []string `config:"internal_networks"`

Any chance you could add a fix for that here too? Otherwise it's not configurable at all 😞

Copy link
Contributor

@andrewstucki andrewstucki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@elasticmachine
Copy link
Collaborator

elasticmachine commented Feb 18, 2021

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: Started by user Andrew Stucki

  • Start Time: 2021-02-19T22:19:43.973+0000

  • Duration: 48 min 34 sec

  • Commit: d10a383

Test stats 🧪

Test Results
Failed 0
Passed 13056
Skipped 2215
Total 15271

Trends 🧪

Image of Build Times

Image of Tests

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test Results
Failed 0
Passed 13056
Skipped 2215
Total 15271

@andrewkroh andrewkroh merged commit 3ca53aa into elastic:master Feb 20, 2021
v1v added a commit to v1v/beats that referenced this pull request Feb 22, 2021
* upstream/master:
  [Elastic Agent] Fix docker entrypoint for elastic-agent. (elastic#24155)
  [PACKAGING] Push docker images with the architecture in the version (elastic#24121)
  [Agent] Add agent standalone manifests for system module & Pod's log collection (elastic#23938)
  indicator type url is in upper case (elastic#24152)
  [Filebeat] Document netflow internal_networks and set default (elastic#24110)
  [Filebeat] Adding fixes to the TI module (elastic#24133)
  [Enhancement] Add RotateOnStartup feature flag for file output (elastic#19347)
  [Ingest Manager] Fix: Successfully installed and enrolled agent running standalone (elastic#24128)
  Set Elastic licence type for APM server Beats update job (elastic#24122)
  Add logrotation section on Running Filebeat on k8s (elastic#24120)
  [CI] Run if manual UI (elastic#24116)
  [CI] enable x-pack/heartbeat in the CI (elastic#23873)
v1v added a commit to v1v/beats that referenced this pull request Feb 23, 2021
…dows-7

* upstream/master:
  Remove OSS reference for kibana and elasticsearch (elastic#24164)
  Skip flaky TestActions on MacOSx (elastic#23966)
  [Filebeat][AWS] Fix vpcflow pipeline exception: Cannot invoke "Object.getClass()" because "receiver" is null (elastic#24167)
  [Elastic Agent] Fix docker entrypoint for elastic-agent. (elastic#24155)
  [PACKAGING] Push docker images with the architecture in the version (elastic#24121)
  [Agent] Add agent standalone manifests for system module & Pod's log collection (elastic#23938)
  indicator type url is in upper case (elastic#24152)
  [Filebeat] Document netflow internal_networks and set default (elastic#24110)
  [Filebeat] Adding fixes to the TI module (elastic#24133)
  [Enhancement] Add RotateOnStartup feature flag for file output (elastic#19347)
  [Ingest Manager] Fix: Successfully installed and enrolled agent running standalone (elastic#24128)
  Set Elastic licence type for APM server Beats update job (elastic#24122)
  Add logrotation section on Running Filebeat on k8s (elastic#24120)
  [CI] Run if manual UI (elastic#24116)
  [CI] enable x-pack/heartbeat in the CI (elastic#23873)
  chore: comment out the E2E (elastic#24109)
  chore: add-backport-next (elastic#24098)
  Adjust the position of the architecture name in Dockerlogbeat tarball (elastic#24095)
  Update dependencies for M1 support in System (elastic#24019)
v1v added a commit to v1v/beats that referenced this pull request Feb 23, 2021
…-arm

* upstream/master: (24 commits)
  Add example input autodsicover config (elastic#24157)
  Empty configuration options generate `<no value>` string for azure-eventhub input (elastic#24156)
  Remove OSS reference for kibana and elasticsearch (elastic#24164)
  Skip flaky TestActions on MacOSx (elastic#23966)
  [Filebeat][AWS] Fix vpcflow pipeline exception: Cannot invoke "Object.getClass()" because "receiver" is null (elastic#24167)
  [Elastic Agent] Fix docker entrypoint for elastic-agent. (elastic#24155)
  [PACKAGING] Push docker images with the architecture in the version (elastic#24121)
  [Agent] Add agent standalone manifests for system module & Pod's log collection (elastic#23938)
  indicator type url is in upper case (elastic#24152)
  [Filebeat] Document netflow internal_networks and set default (elastic#24110)
  [Filebeat] Adding fixes to the TI module (elastic#24133)
  [Enhancement] Add RotateOnStartup feature flag for file output (elastic#19347)
  [Ingest Manager] Fix: Successfully installed and enrolled agent running standalone (elastic#24128)
  Set Elastic licence type for APM server Beats update job (elastic#24122)
  Add logrotation section on Running Filebeat on k8s (elastic#24120)
  [CI] Run if manual UI (elastic#24116)
  [CI] enable x-pack/heartbeat in the CI (elastic#23873)
  chore: comment out the E2E (elastic#24109)
  chore: add-backport-next (elastic#24098)
  Adjust the position of the architecture name in Dockerlogbeat tarball (elastic#24095)
  ...
andrewkroh added a commit to andrewkroh/beats that referenced this pull request Mar 1, 2021
…c#24110)

Documentation for the `internal_networks` option of the Netflow input and module was missing.
Also the module's manifest did not declare the option so if it was not set in the module config
it would cause an error.

I did not see where a default was set for the netflow input's internal_networks option so I set that
to `private` to keep the old behavior before this was configurable.

Fixes elastic#24094

(cherry picked from commit 3ca53aa)
@andrewkroh andrewkroh added v7.13.0 and removed needs_backport PR is waiting to be backported to other branches. labels Mar 1, 2021
andrewkroh added a commit to andrewkroh/beats that referenced this pull request Mar 1, 2021
…c#24110)

Documentation for the `internal_networks` option of the Netflow input and module was missing.
Also the module's manifest did not declare the option so if it was not set in the module config
it would cause an error.

I did not see where a default was set for the netflow input's internal_networks option so I set that
to `private` to keep the old behavior before this was configurable.

Fixes elastic#24094

(cherry picked from commit 3ca53aa)
andrewkroh added a commit that referenced this pull request Mar 4, 2021
#24278)

Documentation for the `internal_networks` option of the Netflow input and module was missing.
Also the module's manifest did not declare the option so if it was not set in the module config
it would cause an error.

I did not see where a default was set for the netflow input's internal_networks option so I set that
to `private` to keep the old behavior before this was configurable.

Fixes #24094

(cherry picked from commit 3ca53aa)
andrewkroh added a commit to andrewkroh/beats that referenced this pull request Mar 4, 2021
…c#24110)

Documentation for the `internal_networks` option of the Netflow input and module was missing.
Also the module's manifest did not declare the option so if it was not set in the module config
it would cause an error.

I did not see where a default was set for the netflow input's internal_networks option so I set that
to `private` to keep the old behavior before this was configurable.

Fixes elastic#24094

(cherry picked from commit 3ca53aa)
andrewkroh added a commit that referenced this pull request Mar 17, 2021
#24279)

Documentation for the `internal_networks` option of the Netflow input and module was missing.
Also the module's manifest did not declare the option so if it was not set in the module config
it would cause an error.

I did not see where a default was set for the netflow input's internal_networks option so I set that
to `private` to keep the old behavior before this was configurable.

Fixes #24094

(cherry picked from commit 3ca53aa)
andrewkroh added a commit that referenced this pull request Mar 17, 2021
#24280)

Documentation for the `internal_networks` option of the Netflow input and module was missing.
Also the module's manifest did not declare the option so if it was not set in the module config
it would cause an error.

I did not see where a default was set for the netflow input's internal_networks option so I set that
to `private` to keep the old behavior before this was configurable.

Fixes #24094

(cherry picked from commit 3ca53aa)
leweafan pushed a commit to leweafan/beats that referenced this pull request Apr 28, 2023
…c#24110) (elastic#24280)

Documentation for the `internal_networks` option of the Netflow input and module was missing.
Also the module's manifest did not declare the option so if it was not set in the module config
it would cause an error.

I did not see where a default was set for the netflow input's internal_networks option so I set that
to `private` to keep the old behavior before this was configurable.

Fixes elastic#24094

(cherry picked from commit 563e964)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[filebeat][netflow] filebeat does't start when internal_networks variable isn't defined in config file
3 participants