Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

netflow: Use internal and external for locality fields #24295

Merged
merged 2 commits into from
Mar 10, 2021

Conversation

adriansr
Copy link
Contributor

@adriansr adriansr commented Mar 2, 2021

What does this PR do?

Changes netflow input to use internal and external for locality fields:

  • source.locality
  • destination.locality
  • flow.locality

Previously it was using public and private.

Why is it important?

There was a mismatch between the output values and the values expected by the ingest pipeline used by the Netflow module. Also the new values make more sense from a locality perspective.

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • [ ] I have made corresponding changes to the documentation
  • [ ] I have made corresponding change to the default configuration files
  • [ ] I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Related issues

Closes #24272

@adriansr adriansr added enhancement needs_backport PR is waiting to be backported to other branches. Team:Security-External Integrations labels Mar 2, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Mar 2, 2021
@adriansr adriansr added the review label Mar 2, 2021
@elasticmachine
Copy link
Collaborator

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: Pull request #24295 opened

  • Start Time: 2021-03-02T16:20:50.265+0000

  • Duration: 51 min 28 sec

  • Commit: 3aeeaa4

Test stats 🧪

Test Results
Failed 0
Passed 6841
Skipped 1153
Total 7994

Trends 🧪

Image of Build Times

Image of Tests

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test Results
Failed 0
Passed 6841
Skipped 1153
Total 7994

@adriansr adriansr merged commit 0c6acc9 into elastic:master Mar 10, 2021
adriansr added a commit to adriansr/beats that referenced this pull request Mar 10, 2021
Changes netflow input to use internal and external for locality fields:
- source.locality
- destination.locality
- flow.locality

Previously it was using public and private.
Fixes elastic#24272

(cherry picked from commit 0c6acc9)
@adriansr adriansr added v7.13.0 and removed needs_backport PR is waiting to be backported to other branches. labels Mar 10, 2021
adriansr added a commit to adriansr/beats that referenced this pull request Mar 10, 2021
Changes netflow input to use internal and external for locality fields:
- source.locality
- destination.locality
- flow.locality

Previously it was using public and private.
Fixes elastic#24272

(cherry picked from commit 0c6acc9)
@elasticmachine
Copy link
Collaborator

💔 Build Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: Pull request #24295 updated

  • Start Time: 2021-03-10T08:17:53.377+0000

  • Duration: 51 min 22 sec

  • Commit: b7f2137

Test stats 🧪

Test Results
Failed 0
Passed 5375
Skipped 738
Total 6113

Trends 🧪

Image of Build Times

Image of Tests

Steps errors 6

Expand to view the steps failures

x-pack/filebeat-windows-windows-2019 - mage build unitTest
  • Took 4 min 39 sec . View more details on here
  • Description: mage build unitTest
x-pack/filebeat-windows-2016-windows-2016 - Install Go/Mage/Python 1.15.8
  • Took 0 min 2 sec . View more details on here
  • Description: .ci/scripts/install-tools.bat
x-pack/filebeat-windows-2016-windows-2016 - Install Go/Mage/Python 1.15.8
  • Took 0 min 2 sec . View more details on here
  • Description: .ci/scripts/install-tools.bat
x-pack/filebeat-windows-2012-windows-2012-r2 - Install Go/Mage/Python 1.15.8
  • Took 0 min 3 sec . View more details on here
  • Description: .ci/scripts/install-tools.bat
x-pack/filebeat-windows-2012-windows-2012-r2 - Install Go/Mage/Python 1.15.8
  • Took 0 min 2 sec . View more details on here
  • Description: .ci/scripts/install-tools.bat
Error signal
  • Took 0 min 0 sec . View more details on here
  • Description: Error 'hudson.AbortException: script returned exit code 1'

Log output

Expand to view the last 100 lines of log output

[2021-03-10T09:08:49.303Z]  containerd:
[2021-03-10T09:08:49.303Z]   Version:          1.3.7
[2021-03-10T09:08:49.303Z]   GitCommit:        8fba4e9a7d01810a393d5d25a3621dc101981175
[2021-03-10T09:08:49.303Z]  runc:
[2021-03-10T09:08:49.303Z]   Version:          1.0.0-rc10
[2021-03-10T09:08:49.303Z]   GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
[2021-03-10T09:08:49.303Z]  docker-init:
[2021-03-10T09:08:49.303Z]   Version:          0.18.0
[2021-03-10T09:08:49.303Z]   GitCommit:        fec3683
[2021-03-10T09:08:49.303Z] Change ownership of all files inside the specific folder from root/root to current user/group
[2021-03-10T09:08:50.892Z] Cleaning up /var/lib/jenkins/workspace/PR-24295-2-9ab3145c-e026-478f-b09c-4aae6b728d53
[2021-03-10T09:08:50.893Z] Client: Docker Engine - Community
[2021-03-10T09:08:50.893Z]  Version:           20.10.3
[2021-03-10T09:08:50.893Z]  API version:       1.41
[2021-03-10T09:08:50.893Z]  Go version:        go1.13.15
[2021-03-10T09:08:50.893Z]  Git commit:        48d30b5
[2021-03-10T09:08:50.893Z]  Built:             Fri Jan 29 14:33:13 2021
[2021-03-10T09:08:50.893Z]  OS/Arch:           linux/amd64
[2021-03-10T09:08:50.893Z]  Context:           default
[2021-03-10T09:08:50.893Z]  Experimental:      true
[2021-03-10T09:08:50.893Z] 
[2021-03-10T09:08:50.893Z] Server: Docker Engine - Community
[2021-03-10T09:08:50.893Z]  Engine:
[2021-03-10T09:08:50.893Z]   Version:          20.10.3
[2021-03-10T09:08:50.893Z]   API version:      1.41 (minimum version 1.12)
[2021-03-10T09:08:50.893Z]   Go version:       go1.13.15
[2021-03-10T09:08:50.893Z]   Git commit:       46229ca
[2021-03-10T09:08:50.893Z]   Built:            Fri Jan 29 14:31:25 2021
[2021-03-10T09:08:50.893Z]   OS/Arch:          linux/amd64
[2021-03-10T09:08:50.893Z]   Experimental:     false
[2021-03-10T09:08:50.893Z]  containerd:
[2021-03-10T09:08:50.893Z]   Version:          1.4.4
[2021-03-10T09:08:50.893Z]   GitCommit:        05f951a3781f4f2c1911b05e61c160e9c30eaa8e
[2021-03-10T09:08:50.893Z]  runc:
[2021-03-10T09:08:50.893Z]   Version:          1.0.0-rc93
[2021-03-10T09:08:50.893Z]   GitCommit:        12644e614e25b05da6fd08a38ffa0cfe1903fdec
[2021-03-10T09:08:50.893Z]  docker-init:
[2021-03-10T09:08:50.893Z]   Version:          0.19.0
[2021-03-10T09:08:50.893Z]   GitCommit:        de40ad0
[2021-03-10T09:08:50.893Z] Change ownership of all files inside the specific folder from root/root to current user/group
[2021-03-10T09:08:50.893Z] Unable to find image 'alpine:3.4' locally
[2021-03-10T09:08:51.459Z] 3.4: Pulling from library/alpine
[2021-03-10T09:08:51.717Z] c1e54eec4b57: Pulling fs layer
[2021-03-10T09:08:51.979Z] c1e54eec4b57: Verifying Checksum
[2021-03-10T09:08:51.980Z] c1e54eec4b57: Download complete
[2021-03-10T09:08:52.258Z] c1e54eec4b57: Pull complete
[2021-03-10T09:08:52.258Z] Digest: sha256:b733d4a32c4da6a00a84df2ca32791bb03df95400243648d8c539e7b4cce329c
[2021-03-10T09:08:52.258Z] Status: Downloaded newer image for alpine:3.4
[2021-03-10T09:08:53.407Z] Change permissions with write access of all files inside the specific folder
[2021-03-10T09:08:53.514Z] Running in /var/lib/jenkins/workspace/PR-24295-2-5a2bb2ac-985f-4f3d-aaea-d7e64245e154
[2021-03-10T09:08:54.199Z] Change permissions with write access of all files inside the specific folder
[2021-03-10T09:08:55.863Z] + python .ci/scripts/pre_archive_test.py
[2021-03-10T09:08:57.765Z] Copy ./x-pack/filebeat/build into build/x-pack/filebeat/build
[2021-03-10T09:08:57.776Z] Running in /var/lib/jenkins/workspace/PR-24295-2-9ab3145c-e026-478f-b09c-4aae6b728d53/src/github.com/elastic/beats/build
[2021-03-10T09:08:58.079Z] + rm -rf ve
[2021-03-10T09:08:58.079Z] + find . -type d -name vendor -exec rm -r {} ;
[2021-03-10T09:08:58.091Z] Recording test results
[2021-03-10T09:08:59.064Z] [Checks API] No suitable checks publisher found.
[2021-03-10T09:08:59.408Z] + go clean -modcache
[2021-03-10T09:09:03.000Z] Cleaning up /var/lib/jenkins/workspace/PR-24295-2-9ab3145c-e026-478f-b09c-4aae6b728d53
[2021-03-10T09:09:03.000Z] Client: Docker Engine - Community
[2021-03-10T09:09:03.000Z]  Version:           20.10.3
[2021-03-10T09:09:03.000Z]  API version:       1.41
[2021-03-10T09:09:03.000Z]  Go version:        go1.13.15
[2021-03-10T09:09:03.000Z]  Git commit:        48d30b5
[2021-03-10T09:09:03.000Z]  Built:             Fri Jan 29 14:33:13 2021
[2021-03-10T09:09:03.000Z]  OS/Arch:           linux/amd64
[2021-03-10T09:09:03.000Z]  Context:           default
[2021-03-10T09:09:03.000Z]  Experimental:      true
[2021-03-10T09:09:03.000Z] 
[2021-03-10T09:09:03.000Z] Server: Docker Engine - Community
[2021-03-10T09:09:03.000Z]  Engine:
[2021-03-10T09:09:03.000Z]   Version:          20.10.3
[2021-03-10T09:09:03.000Z]   API version:      1.41 (minimum version 1.12)
[2021-03-10T09:09:03.000Z]   Go version:       go1.13.15
[2021-03-10T09:09:03.000Z]   Git commit:       46229ca
[2021-03-10T09:09:03.000Z]   Built:            Fri Jan 29 14:31:25 2021
[2021-03-10T09:09:03.000Z]   OS/Arch:          linux/amd64
[2021-03-10T09:09:03.000Z]   Experimental:     false
[2021-03-10T09:09:03.000Z]  containerd:
[2021-03-10T09:09:03.000Z]   Version:          1.4.4
[2021-03-10T09:09:03.000Z]   GitCommit:        05f951a3781f4f2c1911b05e61c160e9c30eaa8e
[2021-03-10T09:09:03.000Z]  runc:
[2021-03-10T09:09:03.000Z]   Version:          1.0.0-rc93
[2021-03-10T09:09:03.000Z]   GitCommit:        12644e614e25b05da6fd08a38ffa0cfe1903fdec
[2021-03-10T09:09:03.000Z]  docker-init:
[2021-03-10T09:09:03.000Z]   Version:          0.19.0
[2021-03-10T09:09:03.000Z]   GitCommit:        de40ad0
[2021-03-10T09:09:03.000Z] Change ownership of all files inside the specific folder from root/root to current user/group
[2021-03-10T09:09:09.557Z] Change permissions with write access of all files inside the specific folder
[2021-03-10T09:09:09.833Z] Running in /var/lib/jenkins/workspace/PR-24295-2-9ab3145c-e026-478f-b09c-4aae6b728d53
[2021-03-10T09:09:14.740Z] Stage "Packaging" skipped due to earlier failure(s)
[2021-03-10T09:09:14.793Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-24295/src/github.com/elastic/beats
[2021-03-10T09:09:15.290Z] Running on Jenkins in /var/lib/jenkins/workspace/Beats_beats_PR-24295
[2021-03-10T09:09:15.337Z] [INFO] getVaultSecret: Getting secrets
[2021-03-10T09:09:15.441Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2021-03-10T09:09:16.099Z] + chmod 755 generate-build-data.sh
[2021-03-10T09:09:16.099Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-24295/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-24295/runs/2 FAILURE 3082459
[2021-03-10T09:09:16.350Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-24295/runs/2/steps/?limit=10000 -o steps-info.json
[2021-03-10T09:09:17.261Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-24295/runs/2/tests/?status=FAILED -o tests-errors.json

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test Results
Failed 0
Passed 5375
Skipped 738
Total 6113

adriansr added a commit that referenced this pull request Mar 18, 2021
Changes netflow input to use internal and external for locality fields:
- source.locality
- destination.locality
- flow.locality

Previously it was using public and private.
Fixes #24272

(cherry picked from commit 0c6acc9)
adriansr added a commit that referenced this pull request Mar 22, 2021
Changes netflow input to use internal and external for locality fields:
- source.locality
- destination.locality
- flow.locality

Previously it was using public and private.
Fixes #24272

(cherry picked from commit 0c6acc9)
leweafan pushed a commit to leweafan/beats that referenced this pull request Apr 28, 2023
elastic#24461)

Changes netflow input to use internal and external for locality fields:
- source.locality
- destination.locality
- flow.locality

Previously it was using public and private.
Fixes elastic#24272

(cherry picked from commit a1a8d7a)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Locality values mismatch between Netflow input and ingest pipeline
3 participants