Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Winlogbeat] Add IP validation to Security module #21325

Merged
merged 1 commit into from
Sep 29, 2020

Conversation

andrewkroh
Copy link
Member

@andrewkroh andrewkroh commented Sep 24, 2020

What does this PR do?

For event 4778 (A session was reconnected to a Window Station) the winlog.event_data.ClientAddress
could be "LOCAL" which is obviosuly not a valid IP so we don't want to copy it into source.ip in that case.

Why is it important?

This bug can causes mapping exceptions.

Checklist

  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Related issues

@elasticmachine
Copy link
Collaborator

Pinging @elastic/siem (Team:SIEM)

@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Sep 24, 2020
@andrewkroh andrewkroh changed the title Add IP validation to Security module [Winlogbeat] Add IP validation to Security module Sep 24, 2020
For event 4778 (A session was reconnected to a Window Station) the `winlog.event_data.ClientAddress`
could be "LOCAL" which is obviosuly not a valid IP so we don't want to copy it into `source.ip` in that case.

Fixes elastic#19627
@elasticmachine
Copy link
Collaborator

elasticmachine commented Sep 24, 2020

💔 Build Failed

Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [andrewkroh commented: run tests]

  • Start Time: 2020-09-28T14:30:54.619+0000

  • Duration: 86 min 34 sec

Test stats 🧪

Test Results
Failed 0
Passed 20287
Skipped 1858
Total 22145

Steps errors

Expand to view the steps failures

  • Name: make -C generator/_templates/metricbeat test

    • Description: make -C generator/_templates/metricbeat test

    • Duration: 7 min 33 sec

    • Start Time: 2020-09-28T15:04:58.359+0000

    • log

  • Name: Notifies GitHub of the status of a Pull Request

    • Description: script returned exit code 2

    • Duration: 0 min 1 sec

    • Start Time: 2020-09-28T15:12:12.448+0000

    • log

  • Name: Process JUnit reports with runbld

    • Description:

    • Duration: 0 min 17 sec

    • Start Time: 2020-09-28T15:56:11.114+0000

    • log

Log output

Expand to view the last 100 lines of log output

[2020-09-28T15:56:05.392Z]  Git commit:        4484c46d9d
[2020-09-28T15:56:05.392Z]  Built:             Wed Sep 16 17:02:36 2020
[2020-09-28T15:56:05.392Z]  OS/Arch:           linux/amd64
[2020-09-28T15:56:05.392Z]  Experimental:      false
[2020-09-28T15:56:05.392Z] 
[2020-09-28T15:56:05.392Z] Server: Docker Engine - Community
[2020-09-28T15:56:05.392Z]  Engine:
[2020-09-28T15:56:05.392Z]   Version:          19.03.13
[2020-09-28T15:56:05.392Z]   API version:      1.40 (minimum version 1.12)
[2020-09-28T15:56:05.392Z]   Go version:       go1.13.15
[2020-09-28T15:56:05.392Z]   Git commit:       4484c46d9d
[2020-09-28T15:56:05.392Z]   Built:            Wed Sep 16 17:01:06 2020
[2020-09-28T15:56:05.392Z]   OS/Arch:          linux/amd64
[2020-09-28T15:56:05.392Z]   Experimental:     false
[2020-09-28T15:56:05.392Z]  containerd:
[2020-09-28T15:56:05.392Z]   Version:          1.3.7
[2020-09-28T15:56:05.392Z]   GitCommit:        8fba4e9a7d01810a393d5d25a3621dc101981175
[2020-09-28T15:56:05.392Z]  runc:
[2020-09-28T15:56:05.392Z]   Version:          1.0.0-rc10
[2020-09-28T15:56:05.392Z]   GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
[2020-09-28T15:56:05.392Z]  docker-init:
[2020-09-28T15:56:05.392Z]   Version:          0.18.0
[2020-09-28T15:56:05.392Z]   GitCommit:        fec3683
[2020-09-28T15:56:09.000Z] Post stage
[2020-09-28T15:56:09.016Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats
[2020-09-28T15:56:09.032Z] Archiving artifacts
[2020-09-28T15:56:09.201Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats
[2020-09-28T15:56:09.214Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/uncategorized-1601304759352
[2020-09-28T15:56:09.254Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/libbeat-stress-tests-1601305009075
[2020-09-28T15:56:09.292Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/winlogbeat-crosscompile-1601305082214
[2020-09-28T15:56:09.327Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/auditbeat-macos-macosx-1601305086792
[2020-09-28T15:56:09.366Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/x-pack-elastic-agent-build-1601305087553
[2020-09-28T15:56:09.407Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/auditbeat-crosscompile-1601305114751
[2020-09-28T15:56:09.440Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/x-pack-dockerlogbeat-build-1601305121255
[2020-09-28T15:56:09.476Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/journalbeat-unitTest-1601305125636
[2020-09-28T15:56:09.510Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/x-pack-functionbeat-build-1601305224072
[2020-09-28T15:56:09.542Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/heartbeat-macos-macosx-1601305260830
[2020-09-28T15:56:09.579Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/libbeat-crosscompile-1601305286590
[2020-09-28T15:56:09.703Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/packetbeat-build-1601305290084
[2020-09-28T15:56:09.738Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/metricbeat-unitTest-1601305357491
[2020-09-28T15:56:09.783Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/filebeat-macos-macosx-1601305387400
[2020-09-28T15:56:09.825Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/x-pack-elastic-agent-windows-windows-2019-1601305420826
[2020-09-28T15:56:09.871Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/metricbeat-macos-macosx-1601305421702
[2020-09-28T15:56:09.909Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/x-pack-auditbeat-macos-macosx-1601305448743
[2020-09-28T15:56:09.947Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/heartbeat-build-1601305457176
[2020-09-28T15:56:09.983Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/x-pack-auditbeat-build-1601305472689
[2020-09-28T15:56:10.016Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/packetbeat-macos-macosx-1601305496696
[2020-09-28T15:56:10.050Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/x-pack-auditbeat-windows-windows-2019-1601305515641
[2020-09-28T15:56:10.084Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/generator-macos-beat-macosx-1601305515988
[2020-09-28T15:56:10.121Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/heartbeat-windows-windows-2019-1601305533270
[2020-09-28T15:56:10.156Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/winlogbeat-windows-windows-2019-1601305544195
[2020-09-28T15:56:10.189Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/auditbeat-windows-windows-2019-1601305546304
[2020-09-28T15:56:10.222Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/auditbeat-build-1601305602171
[2020-09-28T15:56:10.256Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/x-pack-libbeat-build-1601305606497
[2020-09-28T15:56:10.292Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/metricbeat-crosscompile-1601305623729
[2020-09-28T15:56:10.324Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/x-pack-filebeat-macos-macosx-1601305628384
[2020-09-28T15:56:10.363Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/generator-metricbeat-test-1601305632434
[2020-09-28T15:56:10.402Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/x-pack-functionbeat-macos-macosx-1601305670489
[2020-09-28T15:56:10.434Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/filebeat-windows-windows-2019-1601305671536
[2020-09-28T15:56:10.477Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/x-pack-elastic-agent-macos-macosx-1601305689054
[2020-09-28T15:56:10.514Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/generator-beat-test-1601305689840
[2020-09-28T15:56:10.549Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/x-pack-filebeat-windows-windows-2019-1601305696988
[2020-09-28T15:56:10.582Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/x-pack-functionbeat-windows-windows-2019-1601305720012
[2020-09-28T15:56:10.617Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/x-pack-metricbeat-macos-macosx-1601305762544
[2020-09-28T15:56:10.655Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/packetbeat-windows-windows-2019-1601305785900
[2020-09-28T15:56:10.692Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/generator-macos-metricbeat-macosx-1601305897168
[2020-09-28T15:56:10.730Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/metricbeat-windows-windows-2019-1601305966161
[2020-09-28T15:56:10.774Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/x-pack-winlogbeat-build-windows-2019-1601306057128
[2020-09-28T15:56:10.812Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/x-pack-metricbeat-windows-windows-2019-1601306404297
[2020-09-28T15:56:10.844Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/filebeat-build-1601306537080
[2020-09-28T15:56:10.883Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/libbeat-build-1601306715966
[2020-09-28T15:56:10.928Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/x-pack-filebeat-build-1601306884666
[2020-09-28T15:56:10.977Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/metricbeat-goIntegTest-1601307077194
[2020-09-28T15:56:11.021Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/metricbeat-pythonIntegTest-1601307189369
[2020-09-28T15:56:11.061Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats/x-pack-metricbeat-build-1601308267794
[2020-09-28T15:56:11.404Z] + cat
[2020-09-28T15:56:11.404Z] + /usr/local/bin/runbld ./runbld-test-reports --job-name elastic+beats+pull-request
[2020-09-28T15:56:11.404Z] Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
[2020-09-28T15:56:18.003Z] runbld>>> runbld started
[2020-09-28T15:56:18.003Z] runbld>>> 1.6.12/f45d832f2ba0aa2722ab4ec1fda8ad140f027f8b
[2020-09-28T15:56:18.958Z] runbld>>> The following profiles matched the job 'elastic+beats+pull-request' in order of occurrence in the config (last value wins).
[2020-09-28T15:56:18.958Z] runbld>>> Matches in the system config:
[2020-09-28T15:56:18.958Z] runbld>>> - Matched ^elastic\+beats
[2020-09-28T15:56:18.958Z] runbld>>> - Matched ^elastic\+beats\+pull-request
[2020-09-28T15:56:20.343Z] runbld>>> Debug logging enabled.
[2020-09-28T15:56:20.343Z] runbld>>> Storing result
[2020-09-28T15:56:20.605Z] runbld>>> Store result: created {:total 2, :successful 2, :failed 0} 1
[2020-09-28T15:56:20.605Z] runbld>>> BUILD: https://c150076387b5421f9154dfbf536e5c60.us-west1.gcp.cloud.es.io:9243/build-1597739501209/t/20200928155620-533CC17D
[2020-09-28T15:56:20.605Z] runbld>>> Adding system facts.
[2020-09-28T15:56:21.547Z] runbld>>> Sending debug log to infra-root+runbld-debug@e***.co
[2020-09-28T15:56:22.489Z] runbld>>> Error: The source clone was not found in /var/lib/jenkins/workspace/Beats_beats_PR-21325/src/github.com/elastic/beats.  The most common cause is that Jenkins and runbld are configured with different working directories (referred to as 'basedir' in JJB and 'cwd' in runbld config).
[2020-09-28T15:56:27.798Z] ERROR: runbld post build action failed.
[2020-09-28T15:56:27.799Z] ERROR: script returned exit code 1
[2020-09-28T15:56:28.093Z] Running on worker-1244230 in /var/lib/jenkins/workspace/Beats_beats_PR-21325
[2020-09-28T15:56:28.124Z] [INFO] getVaultSecret: Getting secrets
[2020-09-28T15:56:28.194Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2020-09-28T15:56:30.101Z] + chmod 755 generate-build-data.sh
[2020-09-28T15:56:30.101Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-21325/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-21325/runs/3 FAILURE 5134081
[2020-09-28T15:56:30.101Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-21325/runs/3/steps/?limit=10000 -o steps-info.json
[2020-09-28T15:56:35.737Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-21325/runs/3/tests/?status=FAILED -o tests-errors.json

Copy link
Contributor

@leehinman leehinman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@andrewkroh
Copy link
Member Author

run tests

@andrewkroh andrewkroh merged commit 8c992c5 into elastic:master Sep 29, 2020
v1v added a commit to v1v/beats that referenced this pull request Sep 29, 2020
* upstream/master:
  feat: prepare release pipelines (elastic#21238)
  Add IP validation to Security module (elastic#21325)
  Fixes for new 7.10 rsa2elk datasets (elastic#21240)
  o365input: Restart after fatal error (elastic#21258)
  Fix panic in cgroups monitoring (elastic#21355)
  Handle multiple upstreams in ingress-controller (elastic#21215)
  [CI] Fix runbld when workspace does not exist (elastic#21350)
  [Filebeat] Fix checkpoint (elastic#21344)
  [CI] Archive build reasons (elastic#21347)
  Add dashboard for pubsub metricset in googlecloud module (elastic#21326)
  [Elastic Agent] Allow embedding of certificate (elastic#21179)
  Adds a default for failure_cache.min_ttl (elastic#21085)
  [libbeat] Disk queue implementation (elastic#21176)
v1v added a commit to v1v/beats that referenced this pull request Sep 30, 2020
…ci-build-label-support

* upstream/master:
  [JJBB] Set shallow cloning to 10 (elastic#21409)
  docs: add link to release notes for 7.9.2 (elastic#21405) (elastic#21419)
  docs: Prepare Changelog for 7.9.2 (elastic#21229) (elastic#21403)
  fix: mark flaky tests (elastic#21300)
  fix: use a fixed version of setuptools (elastic#21393)
  Move Kubernetes events metricset to its own block in reference config (elastic#21407)
  [libbeat] Enable WriteAheadLimit in the disk queue (elastic#21391)
  docs: fix apt/yum formatting (elastic#21362)
  Fix shutdown tracking in s3 input (elastic#21380)
  [libbeat] Fix position writing in the disk queue
  Add UBI 8 image to the dependencies report (elastic#21374)
  Fix debug message to show actual SQS message ID (elastic#20614)
  [Elastic Agent] Rename *ConfigChange to PolicyChange (elastic#20779)
  [Elastic Agent] Add install/uninstall sub-command (elastic#21206)
  [Filebeat][httpjson] Make httpjson use cursor input when using date cursor (elastic#20751)
  feat: prepare release pipelines (elastic#21238)
  Add IP validation to Security module (elastic#21325)
@MakoWish
Copy link
Contributor

MakoWish commented Jan 4, 2021

What version of Winlogbeat should we expect to see this fix applied to? We are currently running 7.10.1 and still seeing this issue.

andrewkroh added a commit to andrewkroh/beats that referenced this pull request Jan 5, 2021
For event 4778 (A session was reconnected to a Window Station) the `winlog.event_data.ClientAddress`
could be "LOCAL" which is obviosuly not a valid IP so we don't want to copy it into `source.ip` in that case.

Fixes elastic#19627

(cherry picked from commit 8c992c5)
andrewkroh added a commit to andrewkroh/beats that referenced this pull request Jan 5, 2021
For event 4778 (A session was reconnected to a Window Station) the `winlog.event_data.ClientAddress`
could be "LOCAL" which is obviosuly not a valid IP so we don't want to copy it into `source.ip` in that case.

Fixes elastic#19627

(cherry picked from commit 8c992c5)
@andrewkroh
Copy link
Member Author

@MakoWish Sorry I missed a backport for this. The PRs to add this to 7.11 and future 7.x releases are open.

andrewkroh added a commit that referenced this pull request Jan 6, 2021
For event 4778 (A session was reconnected to a Window Station) the `winlog.event_data.ClientAddress`
could be "LOCAL" which is obviosuly not a valid IP so we don't want to copy it into `source.ip` in that case.

Fixes #19627

(cherry picked from commit 8c992c5)
andrewkroh added a commit that referenced this pull request Jan 6, 2021
For event 4778 (A session was reconnected to a Window Station) the `winlog.event_data.ClientAddress`
could be "LOCAL" which is obviosuly not a valid IP so we don't want to copy it into `source.ip` in that case.

Fixes #19627

(cherry picked from commit 8c992c5)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Winlogbeat Could not index event - "source.ip: 'LOCAL'" Not an IP String Literal
4 participants