Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security Solution] [Detections] Fixes bug for determining when we hit max signals after filtering with lists #71768

Merged
merged 5 commits into from
Jul 15, 2020

Conversation

dhurley14
Copy link
Contributor

@dhurley14 dhurley14 commented Jul 14, 2020

Summary

Before exceptions were introduced, we were determining when we hit max signals by checking how many events were searched, as those events were being sent straight to bulk create and would be indexed as signals.

Now with exceptions, each search result from the rule query doesn't necessarily mean we are going to index it as a signal, so we need to increment our counter for max signals by the count returned from bulk create (which does the indexing into the signals index), not the count returned from the search.

Checklist

Delete any items that are not applicable to this PR.

For maintainers

@dhurley14 dhurley14 self-assigned this Jul 14, 2020
@dhurley14 dhurley14 added Feature:Detection Rules Anything related to Security Solution's Detection Rules release_note:skip Skip the PR/issue when compiling release notes review Team:SIEM v7.9.0 v8.0.0 labels Jul 14, 2020
@dhurley14 dhurley14 marked this pull request as ready for review July 14, 2020 22:05
@dhurley14 dhurley14 requested review from a team as code owners July 14, 2020 22:05
@elasticmachine
Copy link
Contributor

Pinging @elastic/siem (Team:SIEM)

Copy link
Member

@spong spong left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One nit about a potentially unnecessary null check, but other than that LGTM! Thanks @dhurley14!

@kibanamachine
Copy link
Contributor

💚 Build Succeeded

Build metrics

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@dhurley14 dhurley14 merged commit 56de45d into elastic:master Jul 15, 2020
@dhurley14 dhurley14 deleted the search-after-bugfix branch July 15, 2020 01:27
gmmorris added a commit to gmmorris/kibana that referenced this pull request Jul 15, 2020
* master: (82 commits)
  Fixed the spacing of child accordion items for policy response dialog. (elastic#71677)
  [SECURITY] Timeline bug 7.9 (elastic#71748)
  use fixed isChromeVisible method (elastic#71813)
  [SIEM][Detection Engine][Lists] Adds specific endpoint_list REST API and API for abilities to auto-create the endpoint_list if it gets deleted (elastic#71792)
  [test] Skips flaky Saved Objects Management test
  [APM] Remove watcher integration (elastic#71655)
  [APM] Increase `xpack.apm.ui.transactionGroupBucketSize` (elastic#71661)
  [test] Skips Ingest Manager test preventing ES promotion
  [test] Skips flaky detection engine tests
  Revert "re-fix navigate path for master add SAML login to login_page (elastic#71337)"
  [tests] Temporarily skipped Fleet tests
  [test] Skipped monitoring test
  [Security Solution][Detections] Associate Endpoint Exceptions List to Rule during rule creation/update (elastic#71794)
  Add endpoint exception creation API validation (elastic#71791)
  Skip jest tests that timeout waiting for react (elastic#71801)
  [Security Solution][Exceptions] - Adds filtering to endpoint index patterns by exceptional fields (elastic#71757)
  [Reporting] Re-delete a file (elastic#71730)
  [Security Solution] [Detections] Fixes bug for determining when we hit max signals after filtering with lists (elastic#71768)
  [Ingest Manager] Better display of Fleet requirements (elastic#71686)
  [tests] Temporarily skipped to promote snapshot
  ...
angorayc pushed a commit that referenced this pull request Jul 15, 2020
…t max signals after filtering with lists (#71768) (#71800)

update signal counter with filtered results, not with direct search results.

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
dhurley14 added a commit to dhurley14/kibana that referenced this pull request Jul 15, 2020
…en we hit max signals after filtering with lists (elastic#71768)"

This reverts commit 56de45d.
dhurley14 added a commit that referenced this pull request Jul 15, 2020
…ons] Fixes bug for determining when we hit max signals after filtering with lists (#71768)" (#71956)

This reverts commit 56de45d.
dhurley14 added a commit to dhurley14/kibana that referenced this pull request Jul 15, 2020
…ons] Fixes bug for determining when we hit max signals after filtering with lists (elastic#71768)" (elastic#71956)

This reverts commit 56de45d.
dhurley14 added a commit to dhurley14/kibana that referenced this pull request Jul 15, 2020
…ons] Fixes bug for determining when we hit max signals after filtering with lists (elastic#71768)" (elastic#71956)

This reverts commit 56de45d.
gmmorris added a commit to gmmorris/kibana that referenced this pull request Jul 16, 2020
* master: (37 commits)
  [Lens] Handle failing existence check (elastic#70718)
  [Security Solution]Fix in-app links and popup window text (elastic#71403)
  [esArchiver] automatically retry if alias creation fails (elastic#71910)
  Move data stream index pattern creation test to xpack (elastic#71511)
  [Maps] Improve language for mvt card (elastic#71947)
  [Security][Detections] Unskip failing modal tests (elastic#71969)
  skip flaky suite (elastic#71987)
  skip flaky suite (elastic#71979)
  [Security Solution] [Detections] Revert "[Security Solution] [Detections] Fixes bug for determining when we hit max signals after filtering with lists (elastic#71768)" (elastic#71956)
  rename ilm policy to remove -default (elastic#71952)
  Adjust ordering of Management category apps to make Ingest Manager higher (elastic#71948)
  skip flaky suite (elastic#71971)
  skip flaky suite (elastic#71951)
  [kbn/optimizer] ignore compressed files when reporting stats (elastic#71940)
  skip flaky suite (elastic#71867)
  [ML] Fix new job with must_not saved search (elastic#71831)
  [Resolver] Fix bug where process detail panel doesn't show up (elastic#71754)
  Cleanup (elastic#71849)
  [Resolver] aria-level and aria-flowto support enhancements (elastic#71887)
  skip flaky suite (elastic#71304)
  ...
gmmorris added a commit to gmmorris/kibana that referenced this pull request Jul 16, 2020
…feature-privileges

* alerting/consumer-based-rbac: (491 commits)
  [Lens] Handle failing existence check (elastic#70718)
  [Security Solution]Fix in-app links and popup window text (elastic#71403)
  [esArchiver] automatically retry if alias creation fails (elastic#71910)
  Move data stream index pattern creation test to xpack (elastic#71511)
  [Maps] Improve language for mvt card (elastic#71947)
  [Security][Detections] Unskip failing modal tests (elastic#71969)
  skip flaky suite (elastic#71987)
  skip flaky suite (elastic#71979)
  [Security Solution] [Detections] Revert "[Security Solution] [Detections] Fixes bug for determining when we hit max signals after filtering with lists (elastic#71768)" (elastic#71956)
  rename ilm policy to remove -default (elastic#71952)
  Adjust ordering of Management category apps to make Ingest Manager higher (elastic#71948)
  skip flaky suite (elastic#71971)
  skip flaky suite (elastic#71951)
  [kbn/optimizer] ignore compressed files when reporting stats (elastic#71940)
  skip flaky suite (elastic#71867)
  [ML] Fix new job with must_not saved search (elastic#71831)
  [Resolver] Fix bug where process detail panel doesn't show up (elastic#71754)
  Cleanup (elastic#71849)
  [Resolver] aria-level and aria-flowto support enhancements (elastic#71887)
  skip flaky suite (elastic#71304)
  ...
dhurley14 added a commit that referenced this pull request Jul 16, 2020
…ons] Fixes bug for determining when we hit max signals after filtering with lists (#71768)" (#71956) (#71983)

This reverts commit 56de45d.

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
dhurley14 added a commit that referenced this pull request Jul 16, 2020
…etections] Fixes bug for determining when we hit max signals after filtering with lists (#71768)" (#71956) (#71984)

This reverts commit 56de45d.
@MindyRS MindyRS added the Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. label Sep 23, 2021
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature:Detection Rules Anything related to Security Solution's Detection Rules release_note:skip Skip the PR/issue when compiling release notes review Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:SIEM v7.9.0 v8.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants