[filebeat][netflow] filebeat does't start when internal_networks variable isn't defined in config file #24094

abraxxa · 2021-02-18T09:28:23Z

Version: 7.11.1
Operating System: Ubuntu 18.04
Discuss Forum URL: https://discuss.elastic.co/t/filebeat-netflow-module-do-not-start-after-update-to-7-11/264037/3
Steps to Reproduce: enable the netflow module without the internal_networks variable defined in its yaml config file

The new variable 'internal_networks' is also neither documented in https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-netflow.html nor https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-netflow.html.

Error messages from journalctl -u filebeat --no-pager:

Feb 18 09:38:03 tsa-tc-tsses-1 filebeat[2411]: 2021-02-18T09:38:03.352+0100        INFO        instance/beat.go:474        filebeat stopped.
Feb 18 09:38:03 tsa-tc-tsses-1 filebeat[2411]: 2021-02-18T09:38:03.352+0100        ERROR        instance/beat.go:971        Exiting: Failed to start crawler: creating module reloader failed: could not create module registry for filesets: Error getting config for fileset netflow/log: Error interpreting the template of the input: template: text:8:5: executing "text" at <.internal_networks>: map has no entry for key "internal_networks"
Feb 18 09:38:03 tsa-tc-tsses-1 filebeat[2411]: Exiting: Failed to start crawler: creating module reloader failed: could not create module registry for filesets: Error getting config for fileset netflow/log: Error interpreting the template of the input: template: text:8:5: executing "text" at <.internal_networks>: map has no entry for key "internal_networks"

The text was updated successfully, but these errors were encountered:

elasticmachine · 2021-02-18T09:32:13Z

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

Documentation for the `internal_networks` option of the Netflow input and module was missing. Also the module's manifest did not declare the option so if it was not set in the module config it would cause an error. I did not see where a default was set for the netflow input's internal_networks option so I set that to `private` to keep the old behavior before this was configurable. Fixes elastic#24094

andrewkroh · 2021-02-18T15:03:39Z

I opened a fix at #24110.

To workaround the issue you'll have to update the module config to add the var.private_networks like at

beats/x-pack/filebeat/modules.d/netflow.yml.disabled

Lines 4 to 14 in dc2d849

    
           - module: netflow 
        
             log: 
        
               enabled: true 
        
               var: 
        
                 netflow_host: localhost 
        
                 netflow_port: 2055 
        
                 # internal_networks specifies which networks are considered internal or private 
        
                 # you can specify either a CIDR block or any of the special named ranges listed 
        
                 # at: https://www.elastic.co/guide/en/beats/filebeat/current/defining-processors.html#condition-network 
        
                 internal_networks: 
        
                   - private

Documentation for the `internal_networks` option of the Netflow input and module was missing. Also the module's manifest did not declare the option so if it was not set in the module config it would cause an error. I did not see where a default was set for the netflow input's internal_networks option so I set that to `private` to keep the old behavior before this was configurable. Fixes #24094

…c#24110) Documentation for the `internal_networks` option of the Netflow input and module was missing. Also the module's manifest did not declare the option so if it was not set in the module config it would cause an error. I did not see where a default was set for the netflow input's internal_networks option so I set that to `private` to keep the old behavior before this was configurable. Fixes elastic#24094 (cherry picked from commit 3ca53aa)

#24278) Documentation for the `internal_networks` option of the Netflow input and module was missing. Also the module's manifest did not declare the option so if it was not set in the module config it would cause an error. I did not see where a default was set for the netflow input's internal_networks option so I set that to `private` to keep the old behavior before this was configurable. Fixes #24094 (cherry picked from commit 3ca53aa)

…c#24110) Documentation for the `internal_networks` option of the Netflow input and module was missing. Also the module's manifest did not declare the option so if it was not set in the module config it would cause an error. I did not see where a default was set for the netflow input's internal_networks option so I set that to `private` to keep the old behavior before this was configurable. Fixes elastic#24094 (cherry picked from commit 3ca53aa)

#24279) Documentation for the `internal_networks` option of the Netflow input and module was missing. Also the module's manifest did not declare the option so if it was not set in the module config it would cause an error. I did not see where a default was set for the netflow input's internal_networks option so I set that to `private` to keep the old behavior before this was configurable. Fixes #24094 (cherry picked from commit 3ca53aa)

#24280) Documentation for the `internal_networks` option of the Netflow input and module was missing. Also the module's manifest did not declare the option so if it was not set in the module config it would cause an error. I did not see where a default was set for the netflow input's internal_networks option so I set that to `private` to keep the old behavior before this was configurable. Fixes #24094 (cherry picked from commit 3ca53aa)

…c#24110) (elastic#24280) Documentation for the `internal_networks` option of the Netflow input and module was missing. Also the module's manifest did not declare the option so if it was not set in the module config it would cause an error. I did not see where a default was set for the netflow input's internal_networks option so I set that to `private` to keep the old behavior before this was configurable. Fixes elastic#24094 (cherry picked from commit 563e964)

botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Feb 18, 2021

andresrc added the Team:Security-External Integrations label Feb 18, 2021

botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Feb 18, 2021

andrewkroh added bug Filebeat Filebeat labels Feb 18, 2021

andrewkroh mentioned this issue Feb 18, 2021

[Filebeat] Document netflow internal_networks and set default #24110

Merged

6 tasks

andrewkroh closed this as completed in #24110 Feb 20, 2021

andrewkroh mentioned this issue Mar 1, 2021

Cherry-pick #24110 to 7.x: [Filebeat] Document netflow internal_networks and set default #24278

Merged

6 tasks

andrewkroh mentioned this issue Mar 1, 2021

Cherry-pick #24110 to 7.11: [Filebeat] Document netflow internal_networks and set default #24279

Merged

6 tasks

andrewkroh mentioned this issue Mar 1, 2021

Cherry-pick #24110 to 7.12: [Filebeat] Document netflow internal_networks and set default #24280

Merged

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[filebeat][netflow] filebeat does't start when internal_networks variable isn't defined in config file #24094

[filebeat][netflow] filebeat does't start when internal_networks variable isn't defined in config file #24094

abraxxa commented Feb 18, 2021

elasticmachine commented Feb 18, 2021

andrewkroh commented Feb 18, 2021 •

edited

Loading

[filebeat][netflow] filebeat does't start when internal_networks variable isn't defined in config file #24094

[filebeat][netflow] filebeat does't start when internal_networks variable isn't defined in config file #24094

Comments

abraxxa commented Feb 18, 2021

elasticmachine commented Feb 18, 2021

andrewkroh commented Feb 18, 2021 • edited Loading

andrewkroh commented Feb 18, 2021 •

edited

Loading