2.4.70 #527
Merged
2.4.70 #527
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Extract additional fields
additional logging; auto-enable yara rules
enable license checks
Some updates weren't happening so I took a closer look at UpdateDetection. We want to be able to update most fields of any detection for one reason or another. Even community rules get updated versions. Some protections were getting in the way so they were moved up to the handler so they only applied to updates a user is making. This change invalidated some existing tests. Changing from ETOPEN to ETPRO (and similar changes to where community rules are pulled from) will, on next sync, remove rules that are no longer present from the new import places. The check that asks the question "Has a detection changed enough to warrant updating?" now checks if the ruleset has changed or if there are Overrides. When updating repos, any folders that existed before updating that weren't listed for an update are removed. It's assumed we always try to update every repo we're getting community rules from. Organized the util functions, added tests.
When validating a sigma rule client-side, first check if it has a publicID before checking that the ID matches the one in the rule. If the detection doesn't have a public ID then the rule is probably a duplicate in which the public ID will be extracted on next save to give the user a chance to change the publicID. Better error messaging around UPDATING a detection to have a publicID that already exists. This behavior used to be unique to CREATING detections.
All the logic necessary before a detection in elasticsearch can be updated is now done in a PrepareForSave function on the DetectionHandler. Detectionstore.UpdateDetection no longer needs to request the original detection before updating. New way to compare Overrides. Added and updated tests.
Forward and backward, this test shows that valid overrides can be properly compared for equality.
…s-rules Refactored UpdateDetection
strconv.Unquote seems to be geared towards how the Go language parses strings when parsing Go code such as only removing single quotes if they're around a single char. Updated to use a more thorough, hand-written function that also doesn't throw an error.
…a-titles Use Better Unquote Function For Suricata Titles
We were processing every non-commented line in the suricata rules file. Now we're processing every rule and using the comments to indicate if it's enabled or not (not commented = enabled). Some new complexity was added that's unique to Suricata. Because we manipulate the rules through pillars, we now take advantage of a unique opportunity that will let the ruleset determine if a detection is enabled UNTIL a user modifies the detection at which point the user's preference will forever override the ruleset. The new logic is: if a community rule's sid is not in the enabled pillar or the disabled pillar at the time of import then the sid is left out of the pillars, conversely if the sid is found in a pillar then it should be updated. When not specified in either pillar, the status of a suricata detection will be determined by whether it's commented out or not in the original rule file. De-linted a line. Readers only throw errors when unreading a non-seekable stream. The stream in use here is always seekable and will never return an error.
…d-suricata Cogburn/advanced suricata
FEATURE: Add SOC Quick Link for Elasticsearch ILM Deletion Security-Onion-Solutions/securityonion#12854
YARA files often contain multiple rules. If any of those rules need to import anything, they're imported once at the top of the file. Now, as SOC parses files it retains those imports and re-adds them to rules as it individualizes them making sure to only add the imports that are actually referenced.
…ports Smarter YARA Imports
Gridmembershandler.go had a custom implemented sanitize function, replaced it with the sanitize library of choice we use else where in the project. PublicIds are used in a few places to name files being written to disk so some extra care was taken to be sure we validate them more stringently and sanitize them before use. They are, after all, externally provided content. More test cases for validateId to ensure file paths or even the characters used in file paths aren't acceptable regardless of slash direction. This is used upstream of where we use PublicIds for file names.
Sanitize Strings
detections ui cosmetics; ensure soc logs included on grid screen pivot to hunt
…a-delete Suricata Sync Deletion
One expected modify overrides to show up in the thresholding pillar and the other was a simple oversight highlighted by the test.
…a-test-tweaks Fix 2 Tests
Compare the current content to the original value to determine if anything has changed.
The Update/Cancel buttons under the Detection Source now only appear when the source is modified. Sigma's Convert button still remains visible when appropriate. Changing the source, not saving it using the provided Update button, but causing the detection to save some other way now results in an Unsaved Changes dialog appearing asking you if you want to save the source before continuing. Yes saves, No reverts the source. Also fixed a bug where canceling your source changes linked the current model to the saved `origDetect` in a way that shouldn't have happened. If saving a detection fails, the status indicator is reverted back to the original value to give the user a better experience.
…visibility Cogburn/button visibility
…ved-changes-on-new Don't show "unsaved changes" dialog on New Detection page
fix detection summary metadata styling
prevent creation of new overrides with blank, required values
…alert have When a community detection needs to update, only update if there was a change in content, ruleset, or if there are overrides (hard to detect if there's a change so we always update them). This improvement is already present in suricata and elastalert.
…-strelka-import-speed Give Strelka a similar performance boost that both suricata and elast…
several bug fixes in overrides, cosmetic corrections, implemented validation logic
When migrating, overrides should have their CreatedAt and UpdatedAt values set to the proper time. When loading the overrides, in situations where the installation does not have any overrides there will be no file. This is not an error. Allow the migration to continue. Tests updated.
…on-fixes Migration override timestamps & sidsYaml does not exist
We should validate publicIds with the specific validatePublicId function to allow for YARA's long publicIds. Updated test.
…tbypublicid-length-fix Proper Validation During GetDetectionByPublicId
realert on all matches
dougburks
approved these changes
May 29, 2024
defensivedepth
approved these changes
May 29, 2024
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.