Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Log pattern tool improvement #474

Merged

Conversation

songkant-aws
Copy link
Contributor

@songkant-aws songkant-aws commented Jan 2, 2025

Description

Improve log pattern tool with cutting edge log pattern parser by introducing Brain algorithm mentioned in this RFC: opensearch-project/OpenSearch#16627

Related Issues

Check List

  • New functionality includes testing.
  • New functionality has been documented.
  • API changes companion pull request created.
  • Commits are signed per the DCO using --signoff.
  • Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Songkan Tang <songkant@amazon.com>
Signed-off-by: Songkan Tang <songkant@amazon.com>
Signed-off-by: Songkan Tang <songkant@amazon.com>
Signed-off-by: Songkan Tang <songkant@amazon.com>
Signed-off-by: Songkan Tang <songkant@amazon.com>
Signed-off-by: Songkan Tang <songkant@amazon.com>
Signed-off-by: Songkan Tang <songkant@amazon.com>
Copy link

codecov bot commented Jan 17, 2025

Codecov Report

Attention: Patch coverage is 87.23404% with 30 lines in your changes missing coverage. Please review.

Project coverage is 82.11%. Comparing base (2b76e3c) to head (64837d7).
Report is 38 commits behind head on main.

Files with missing lines Patch % Lines
...ava/org/opensearch/agent/tools/LogPatternTool.java 81.08% 13 Missing and 8 partials ⚠️
...g/opensearch/agent/tools/utils/BrainLogParser.java 92.62% 5 Missing and 4 partials ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main     #474      +/-   ##
============================================
+ Coverage     81.78%   82.11%   +0.32%     
- Complexity      193      333     +140     
============================================
  Files            11       16       +5     
  Lines           961     1571     +610     
  Branches        137      221      +84     
============================================
+ Hits            786     1290     +504     
- Misses          121      194      +73     
- Partials         54       87      +33     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Hailong-am
Copy link
Collaborator

build failure will handled by #489

@Hailong-am Hailong-am merged commit 8d988f4 into opensearch-project:main Jan 24, 2025
4 of 8 checks passed
@Hailong-am Hailong-am added backport 2.x v2.19.0 Issues targeting release v2.19.0 labels Jan 24, 2025
@opensearch-trigger-bot
Copy link
Contributor

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/skills/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/skills/backport-2.x
# Create a new branch
git switch --create backport/backport-474-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 8d988f48cdb2c34d3244bfa3f6669e834d6babc4
# Push it to GitHub
git push --set-upstream origin backport/backport-474-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/skills/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-474-to-2.x.

songkant-aws added a commit to songkant-aws/skills that referenced this pull request Jan 24, 2025
* Improve log pattern tool with cutting edge log pattern parser

Signed-off-by: Songkan Tang <songkant@amazon.com>

* Minor comment change

Signed-off-by: Songkan Tang <songkant@amazon.com>

* Address LogPatternTool class comments

Signed-off-by: Songkan Tang <songkant@amazon.com>

* Address BrainLogParser class comments

Signed-off-by: Songkan Tang <songkant@amazon.com>

* Address BrainLogParser class comment part02

Signed-off-by: Songkan Tang <songkant@amazon.com>

* Minor change of input validation logic

Signed-off-by: Songkan Tang <songkant@amazon.com>

* Address forbiddenAPI check issue

Signed-off-by: Songkan Tang <songkant@amazon.com>

* Address minor comments in BrainLogParser

Signed-off-by: Songkan Tang <songkant@amazon.com>

* Tune preprocessing regex and parameters and change related test result

Signed-off-by: Songkan Tang <songkant@amazon.com>

* Expose variable count threshold parameter configuration for LogPatternTool

Signed-off-by: Songkan Tang <songkant@amazon.com>

* Minorly tune preprocessing regex

Signed-off-by: Songkan Tang <songkant@amazon.com>

---------

Signed-off-by: Songkan Tang <songkant@amazon.com>
@songkant-aws songkant-aws mentioned this pull request Jan 24, 2025
5 tasks
Hailong-am pushed a commit that referenced this pull request Jan 24, 2025
* Log pattern tool improvement (#474)

* Improve log pattern tool with cutting edge log pattern parser

Signed-off-by: Songkan Tang <songkant@amazon.com>

* Minor comment change

Signed-off-by: Songkan Tang <songkant@amazon.com>

* Address LogPatternTool class comments

Signed-off-by: Songkan Tang <songkant@amazon.com>

* Address BrainLogParser class comments

Signed-off-by: Songkan Tang <songkant@amazon.com>

* Address BrainLogParser class comment part02

Signed-off-by: Songkan Tang <songkant@amazon.com>

* Minor change of input validation logic

Signed-off-by: Songkan Tang <songkant@amazon.com>

* Address forbiddenAPI check issue

Signed-off-by: Songkan Tang <songkant@amazon.com>

* Address minor comments in BrainLogParser

Signed-off-by: Songkan Tang <songkant@amazon.com>

* Tune preprocessing regex and parameters and change related test result

Signed-off-by: Songkan Tang <songkant@amazon.com>

* Expose variable count threshold parameter configuration for LogPatternTool

Signed-off-by: Songkan Tang <songkant@amazon.com>

* Minorly tune preprocessing regex

Signed-off-by: Songkan Tang <songkant@amazon.com>

---------

Signed-off-by: Songkan Tang <songkant@amazon.com>

* Fix compilation issue due to Java version conflicts

Signed-off-by: Songkan Tang <songkant@amazon.com>

* Fix compilation issue part2

Signed-off-by: Songkan Tang <songkant@amazon.com>

---------

Signed-off-by: Songkan Tang <songkant@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x failed backport v2.19.0 Issues targeting release v2.19.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants