Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security Solution] [Detections] Remove file validation on import route #77770

Merged
merged 5 commits into from
Sep 17, 2020

Conversation

dhurley14
Copy link
Contributor

Summary

io-ts validation on the rule import file object was creating performance issues. We replace this validation with kbn schema.any() since the validation on the rules occurs later in the route handler.

Checklist

Delete any items that are not applicable to this PR.

For maintainers

@elasticmachine
Copy link
Contributor

Pinging @elastic/siem (Team:SIEM)


// uncomment the below test once we speed up the alerts client find api
// in another PR.
// it('should be able to import 10000 rules', async () => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am fine with these commented out lines if you are as this is a smaller more concentrated PR. Just pointing to this in case you didn't want them here.

But since this is for 7.9.x series I am 👍 good with them here if they are helpful.

Copy link
Contributor Author

@dhurley14 dhurley14 Sep 17, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought about deleting this test + comments but decided the note served as a nice reminder to me to make sure this test makes it back in to this PR #76398

Copy link
Contributor

@FrankHassanabad FrankHassanabad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for the work to fix our customers that are complaining about the slowness and this smaller PR here.

…quest, adds new functional tests and unit tests to make sure we can reach and don't go past bounds. These tests would have helped uncover performance issues io-ts gave us with validating the import rules file object
@kibanamachine
Copy link
Contributor

💚 Build Succeeded

Build metrics

✅ unchanged

History

  • 💔 Build #75227 failed 8f81a62d9b2a9952d457cd1ccb9494fe2bd67435

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@dhurley14 dhurley14 merged commit becb137 into elastic:master Sep 17, 2020
@dhurley14 dhurley14 deleted the remove-validation-import-route branch September 17, 2020 20:27
dhurley14 added a commit to dhurley14/kibana that referenced this pull request Sep 17, 2020
…te (elastic#77770)

* utlize schema.any() for validation on file in body of import rules request, adds new functional tests and unit tests to make sure we can reach and don't go past bounds. These tests would have helped uncover performance issues io-ts gave us with validating the import rules file object

* fix type check failure

* updates getSimpleRule and getSimpleRuleAsNdjson to accept an enabled param defaulted to false

* updates comments in e2e tests for import rules route

* fix tests after adding enabled boolean in test utils
dhurley14 added a commit to dhurley14/kibana that referenced this pull request Sep 17, 2020
…te (elastic#77770)

* utlize schema.any() for validation on file in body of import rules request, adds new functional tests and unit tests to make sure we can reach and don't go past bounds. These tests would have helped uncover performance issues io-ts gave us with validating the import rules file object

* fix type check failure

* updates getSimpleRule and getSimpleRuleAsNdjson to accept an enabled param defaulted to false

* updates comments in e2e tests for import rules route

* fix tests after adding enabled boolean in test utils
dhurley14 added a commit that referenced this pull request Sep 17, 2020
…rt route (#77770) (#77821)

* utlize schema.any() for validation on file in body of import rules request, adds new functional tests and unit tests to make sure we can reach and don't go past bounds. These tests would have helped uncover performance issues io-ts gave us with validating the import rules file object

* fix type check failure

* updates getSimpleRule and getSimpleRuleAsNdjson to accept an enabled param defaulted to false

* updates comments in e2e tests for import rules route

* fix tests after adding enabled boolean in test utils
dhurley14 added a commit that referenced this pull request Sep 17, 2020
…rt route (#77770) (#77822)

* utlize schema.any() for validation on file in body of import rules request, adds new functional tests and unit tests to make sure we can reach and don't go past bounds. These tests would have helped uncover performance issues io-ts gave us with validating the import rules file object

* fix type check failure

* updates getSimpleRule and getSimpleRuleAsNdjson to accept an enabled param defaulted to false

* updates comments in e2e tests for import rules route

* fix tests after adding enabled boolean in test utils
gmmorris added a commit to gmmorris/kibana that referenced this pull request Sep 22, 2020
* master: (92 commits)
  [ILM] Data tiers for 7.10 (elastic#76126)
  [ML] Transforms: Fixes styling of preview grid pagination in summary step (elastic#77789)
  [Drilldowns] Beta badge support. Mark URL Drilldown as Beta (elastic#75654)
  Re-enable session lifespan, idle timeout api integration tests and use unique names for the security test reports. (elastic#77746)
  [Alerting] renames code in alerting RBAC exemption to make it easier to maintain (elastic#77598)
  [Alerting & Actions] Overwrite SOs when updating instead of partially updating (elastic#73688)
  fixed react warning in Suspense in alert flyout (elastic#77777)
  [APM] Track usage of Gold+ features (elastic#77630)
  Visualize: Bad request when working with histogram aggregation (elastic#77684)
  remove legacy ES plugin (elastic#77703)
  [Lens] change name of custom query to filters (elastic#77725)
  skip flaky suite (elastic#76239)
  remove visual aspects of baseline job (elastic#77815)
  skip flaky suite (elastic#77835)
  Fixes typo in data recognizer text (elastic#77691)
  management/update trusted_apps jest snapshot
  [build] Use Elastic hosted UBI minimal base image (elastic#77776)
  [APM] Add transaction error rate alert (elastic#76933)
  [Security Solution] [Detections] Remove file validation on import route (elastic#77770)
  [Enterprise Search][tech debt] Add Kea logic paths for easier debugging/defaults (elastic#77698)
  ...
@MindyRS MindyRS added the Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. label Sep 23, 2021
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature:Detection Rules Anything related to Security Solution's Detection Rules release_note:fix review Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:SIEM v7.9.2 v7.10.0 v8.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants