Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature findings api enhancements #914

Merged
merged 32 commits into from
Mar 13, 2024
Merged

Conversation

riysaxen-amzn
Copy link
Collaborator

@riysaxen-amzn riysaxen-amzn commented Mar 13, 2024

Description

This PR is related to paginations enhancements, for more info please refer to the below github issue

Issues Resolved

795]

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

riysaxen-amzn and others added 16 commits January 15, 2024 23:31
Signed-off-by: Riya Saxena <riysaxen@amazon.com>
Signed-off-by: Riya Saxena <riysaxen@amazon.com>
Signed-off-by: Riya Saxena <riysaxen@amazon.com>
Signed-off-by: Riya Saxena <riysaxen@amazon.com>
Signed-off-by: Riya Saxena <riysaxen@amazon.com>
Signed-off-by: Riya Saxena <riysaxen@amazon.com>
Signed-off-by: Riya Saxena <riysaxen@amazon.com>
… API enhancement

Signed-off-by: Riya Saxena <riysaxen@amazon.com>
Signed-off-by: Riya <69919272+riysaxen-amzn@users.noreply.github.com>
Signed-off-by: Riya Saxena <riysaxen@amazon.com>
@riysaxen-amzn riysaxen-amzn changed the title Feature findings ap ienhancements Feature findings api enhancements Mar 13, 2024
sbcd90
sbcd90 previously approved these changes Mar 13, 2024
}
searchSourceBuilder.fetchSource(true);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we dont pass size, it defaults to 10

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

won't consider more than 10 detector's findings

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, make sense. I had this earlier, i removed it in mid while developing this. Added back

riysaxen-amzn and others added 13 commits March 13, 2024 09:32
Signed-off-by: Riya Saxena <riysaxen@amazon.com>
Signed-off-by: Riya <69919272+riysaxen-amzn@users.noreply.github.com>
Signed-off-by: Subhobrata Dey <sbcd90@gmail.com>
Signed-off-by: Riya Saxena <riysaxen@amazon.com>
Signed-off-by: Riya Saxena <riysaxen@amazon.com>
Signed-off-by: Riya Saxena <riysaxen@amazon.com>
Signed-off-by: Riya Saxena <riysaxen@amazon.com>
Signed-off-by: Riya Saxena <riysaxen@amazon.com>
Signed-off-by: Riya Saxena <riysaxen@amazon.com>
Signed-off-by: Riya Saxena <riysaxen@amazon.com>
Signed-off-by: Riya Saxena <riysaxen@amazon.com>
eirsep
eirsep previously approved these changes Mar 13, 2024
@@ -22,6 +23,7 @@
import org.opensearch.commons.alerting.model.FindingWithDocs;
import org.opensearch.commons.alerting.model.Table;
import org.opensearch.core.rest.RestStatus;
import org.opensearch.index.query.*;
import org.opensearch.securityanalytics.action.FindingDto;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove * imports

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

addressed

Signed-off-by: Riya Saxena <riysaxen@amazon.com>
@riysaxen-amzn riysaxen-amzn merged commit 9b59f61 into main Mar 13, 2024
10 of 30 checks passed
@opensearch-trigger-bot
Copy link
Contributor

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.x 2.x
# Navigate to the new working tree
cd .worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-914-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 9b59f6153956aa86bb6fdf30b9858e27ccd9f849
# Push it to GitHub
git push --set-upstream origin backport/backport-914-to-2.x
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-914-to-2.x.

riysaxen-amzn added a commit to riysaxen-amzn/security-analytics that referenced this pull request Mar 14, 2024
* get all findings as part of findings API enhancement

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* findingsAPI feature enhancements (address comments to prev PR)

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* findingsAPI feature enhancements (address comments to prev PR)

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* added support for  param in Finding API

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* added detectionType as param for Findings API enhancements

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* added few tests to validate findings by params

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* added test for searchString param in FindingsAPI

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* adding addiional params findingIds, startTime and endTime as findings API enhancement

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* added params in getFindingsByDetectorId func

* changed the startTime and endTime req input format

* fix merge conflixt

* fix integ test failures in findings API

* fix integ tests

* refactored the logic

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* remove unused imports

* address the pr comments

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* address pr comments

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* SA integ tests fix

* SA integ tests fix

* fix integ tests for findings

Signed-off-by: Subhobrata Dey <sbcd90@gmail.com>

* fix conflixt errors

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* fix conflixt errors

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* fix conflixt errors

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* fix conflixt errors

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* fix integ tests

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* fix integ tests

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* fix integ tests

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* fix flaky integ tests

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* address pr comments

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

---------

Signed-off-by: Riya Saxena <riysaxen@amazon.com>
Signed-off-by: Riya <69919272+riysaxen-amzn@users.noreply.github.com>
Signed-off-by: Subhobrata Dey <sbcd90@gmail.com>
Co-authored-by: Subhobrata Dey <sbcd90@gmail.com>
(cherry picked from commit 9b59f61)
jowg-amazon pushed a commit to jowg-amazon/security-analytics that referenced this pull request Mar 14, 2024
* get all findings as part of findings API enhancement

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* findingsAPI feature enhancements (address comments to prev PR)

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* findingsAPI feature enhancements (address comments to prev PR)

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* added support for  param in Finding API

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* added detectionType as param for Findings API enhancements

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* added few tests to validate findings by params

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* added test for searchString param in FindingsAPI

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* adding addiional params findingIds, startTime and endTime as findings API enhancement

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* added params in getFindingsByDetectorId func

* changed the startTime and endTime req input format

* fix merge conflixt

* fix integ test failures in findings API

* fix integ tests

* refactored the logic

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* remove unused imports

* address the pr comments

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* address pr comments

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* SA integ tests fix

* SA integ tests fix

* fix integ tests for findings

Signed-off-by: Subhobrata Dey <sbcd90@gmail.com>

* fix conflixt errors

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* fix conflixt errors

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* fix conflixt errors

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* fix conflixt errors

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* fix integ tests

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* fix integ tests

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* fix integ tests

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* fix flaky integ tests

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* address pr comments

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

---------

Signed-off-by: Riya Saxena <riysaxen@amazon.com>
Signed-off-by: Riya <69919272+riysaxen-amzn@users.noreply.github.com>
Signed-off-by: Subhobrata Dey <sbcd90@gmail.com>
Co-authored-by: Subhobrata Dey <sbcd90@gmail.com>
jowg-amazon added a commit that referenced this pull request Mar 15, 2024
* get all findings as part of findings API enhancement (#803)

* get all findings as part of findings API enhancement

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* findingsAPI feature enhancements (address comments to prev PR)

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* findingsAPI feature enhancements (address comments to prev PR)

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* added support for  param in Finding API

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* added detectionType as param for Findings API enhancements

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* added few tests to validate findings by params

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* added test for searchString param in FindingsAPI

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* adding addiional params findingIds, startTime and endTime as findings API enhancement

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* added params in getFindingsByDetectorId func

* changed the startTime and endTime req input format

* fix merge conflixt

* fix integ test failures in findings API

* fix integ tests

* fix integ tests for findings

Signed-off-by: Subhobrata Dey <sbcd90@gmail.com>

---------

Signed-off-by: Riya Saxena <riysaxen@amazon.com>
Signed-off-by: Riya <69919272+riysaxen-amzn@users.noreply.github.com>
Signed-off-by: Subhobrata Dey <sbcd90@gmail.com>
Co-authored-by: Subhobrata Dey <sbcd90@gmail.com>

* fix integ test (#918)

Signed-off-by: Joanne Wang <jowg@amazon.com>

* Feature findings api enhancements (#914)

* get all findings as part of findings API enhancement

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* findingsAPI feature enhancements (address comments to prev PR)

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* findingsAPI feature enhancements (address comments to prev PR)

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* added support for  param in Finding API

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* added detectionType as param for Findings API enhancements

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* added few tests to validate findings by params

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* added test for searchString param in FindingsAPI

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* adding addiional params findingIds, startTime and endTime as findings API enhancement

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* added params in getFindingsByDetectorId func

* changed the startTime and endTime req input format

* fix merge conflixt

* fix integ test failures in findings API

* fix integ tests

* refactored the logic

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* remove unused imports

* address the pr comments

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* address pr comments

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* SA integ tests fix

* SA integ tests fix

* fix integ tests for findings

Signed-off-by: Subhobrata Dey <sbcd90@gmail.com>

* fix conflixt errors

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* fix conflixt errors

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* fix conflixt errors

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* fix conflixt errors

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* fix integ tests

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* fix integ tests

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* fix integ tests

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* fix flaky integ tests

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* address pr comments

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

---------

Signed-off-by: Riya Saxena <riysaxen@amazon.com>
Signed-off-by: Riya <69919272+riysaxen-amzn@users.noreply.github.com>
Signed-off-by: Subhobrata Dey <sbcd90@gmail.com>
Co-authored-by: Subhobrata Dey <sbcd90@gmail.com>

---------

Signed-off-by: Riya Saxena <riysaxen@amazon.com>
Signed-off-by: Riya <69919272+riysaxen-amzn@users.noreply.github.com>
Signed-off-by: Subhobrata Dey <sbcd90@gmail.com>
Signed-off-by: Joanne Wang <jowg@amazon.com>
Co-authored-by: Riya <69919272+riysaxen-amzn@users.noreply.github.com>
Co-authored-by: Subhobrata Dey <sbcd90@gmail.com>
jowg-amazon pushed a commit to jowg-amazon/security-analytics that referenced this pull request Mar 15, 2024
* get all findings as part of findings API enhancement

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* findingsAPI feature enhancements (address comments to prev PR)

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* findingsAPI feature enhancements (address comments to prev PR)

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* added support for  param in Finding API

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* added detectionType as param for Findings API enhancements

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* added few tests to validate findings by params

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* added test for searchString param in FindingsAPI

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* adding addiional params findingIds, startTime and endTime as findings API enhancement

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* added params in getFindingsByDetectorId func

* changed the startTime and endTime req input format

* fix merge conflixt

* fix integ test failures in findings API

* fix integ tests

* refactored the logic

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* remove unused imports

* address the pr comments

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* address pr comments

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* SA integ tests fix

* SA integ tests fix

* fix integ tests for findings

Signed-off-by: Subhobrata Dey <sbcd90@gmail.com>

* fix conflixt errors

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* fix conflixt errors

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* fix conflixt errors

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* fix conflixt errors

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* fix integ tests

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* fix integ tests

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* fix integ tests

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* fix flaky integ tests

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* address pr comments

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

---------

Signed-off-by: Riya Saxena <riysaxen@amazon.com>
Signed-off-by: Riya <69919272+riysaxen-amzn@users.noreply.github.com>
Signed-off-by: Subhobrata Dey <sbcd90@gmail.com>
Co-authored-by: Subhobrata Dey <sbcd90@gmail.com>
jowg-amazon added a commit that referenced this pull request Mar 15, 2024
* support object fields in aggregation based sigma rules (#789)

Signed-off-by: Subhobrata Dey <sbcd90@gmail.com>

* Fix duplicate ecs mappings which returns incorrect log index field in mapping view API (#786) (#788)

* field mapping changes

Signed-off-by: Joanne Wang <jowg@amazon.com>

* add integ test

Signed-off-by: Joanne Wang <jowg@amazon.com>

* turn unmappedfieldaliases as set and add integ test

Signed-off-by: Joanne Wang <jowg@amazon.com>

* add comments

Signed-off-by: Joanne Wang <jowg@amazon.com>

* fix integ tests

Signed-off-by: Joanne Wang <jowg@amazon.com>

* moved logic to method for better readability

Signed-off-by: Joanne Wang <jowg@amazon.com>

---------

Signed-off-by: Joanne Wang <jowg@amazon.com>

* Fix get mappings view API incorrectly returning ecs path (#867)

* add logic and integ tests to not add duplicate to log-types-config index

Signed-off-by: Joanne Wang <jowg@amazon.com>

* change naming for existingFieldMapping and change contains to equals

Signed-off-by: Joanne Wang <jowg@amazon.com>

---------

Signed-off-by: Joanne Wang <jowg@amazon.com>

* fix integ test (#918)

Signed-off-by: Joanne Wang <jowg@amazon.com>

* get all findings as part of findings API enhancement (#803)

* get all findings as part of findings API enhancement

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* findingsAPI feature enhancements (address comments to prev PR)

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* findingsAPI feature enhancements (address comments to prev PR)

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* added support for  param in Finding API

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* added detectionType as param for Findings API enhancements

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* added few tests to validate findings by params

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* added test for searchString param in FindingsAPI

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* adding addiional params findingIds, startTime and endTime as findings API enhancement

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* added params in getFindingsByDetectorId func

* changed the startTime and endTime req input format

* fix merge conflixt

* fix integ test failures in findings API

* fix integ tests

* fix integ tests for findings

Signed-off-by: Subhobrata Dey <sbcd90@gmail.com>

---------

Signed-off-by: Riya Saxena <riysaxen@amazon.com>
Signed-off-by: Riya <69919272+riysaxen-amzn@users.noreply.github.com>
Signed-off-by: Subhobrata Dey <sbcd90@gmail.com>
Co-authored-by: Subhobrata Dey <sbcd90@gmail.com>

* Feature findings api enhancements (#914)

* get all findings as part of findings API enhancement

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* findingsAPI feature enhancements (address comments to prev PR)

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* findingsAPI feature enhancements (address comments to prev PR)

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* added support for  param in Finding API

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* added detectionType as param for Findings API enhancements

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* added few tests to validate findings by params

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* added test for searchString param in FindingsAPI

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* adding addiional params findingIds, startTime and endTime as findings API enhancement

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* added params in getFindingsByDetectorId func

* changed the startTime and endTime req input format

* fix merge conflixt

* fix integ test failures in findings API

* fix integ tests

* refactored the logic

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* remove unused imports

* address the pr comments

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* address pr comments

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* SA integ tests fix

* SA integ tests fix

* fix integ tests for findings

Signed-off-by: Subhobrata Dey <sbcd90@gmail.com>

* fix conflixt errors

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* fix conflixt errors

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* fix conflixt errors

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* fix conflixt errors

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* fix integ tests

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* fix integ tests

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* fix integ tests

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* fix flaky integ tests

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

* address pr comments

Signed-off-by: Riya Saxena <riysaxen@amazon.com>

---------

Signed-off-by: Riya Saxena <riysaxen@amazon.com>
Signed-off-by: Riya <69919272+riysaxen-amzn@users.noreply.github.com>
Signed-off-by: Subhobrata Dey <sbcd90@gmail.com>
Co-authored-by: Subhobrata Dey <sbcd90@gmail.com>

* fix findings api integ tests

Signed-off-by: Joanne Wang <jowg@amazon.com>

---------

Signed-off-by: Subhobrata Dey <sbcd90@gmail.com>
Signed-off-by: Joanne Wang <jowg@amazon.com>
Signed-off-by: Riya Saxena <riysaxen@amazon.com>
Signed-off-by: Riya <69919272+riysaxen-amzn@users.noreply.github.com>
Co-authored-by: Subhobrata Dey <sbcd90@gmail.com>
Co-authored-by: Riya <69919272+riysaxen-amzn@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants