Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Introduce indices filter #167

Merged
merged 4 commits into from
May 10, 2023
Merged

Introduce indices filter #167

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
1ca5b0c
Introduce indices filter
Apr 7, 2023
9eca40d
Fix review comments for indices filter feature
Apr 20, 2023
ee2a803
Enable index filter options for indices filter feature
Apr 21, 2023
a804046
Fixed review comments for index filter options
May 10, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,40 @@ For example to get stats for all cluster nodes from any node use settings:
prometheus.nodes.filter: "_all"
```

#### Indices filter

Prometheus exporter can be configured to filter indices statistics from selected indices.
To target all indices, use "" or * or _all or null.
Default value: `""`.

For example to filter indices statistics:
```
prometheus.indices_filter.selected_indices: "log-*,*log,*log*,log*-test"
```

Users can select which option for filtering indices statistics.
Default value: `"STRICT_EXPAND_OPEN_FORBID_CLOSED"`.

For example to select an option:
```
prometheus.indices_filter.selected_option: "LENIENT_EXPAND_OPEN"
```
Here is a list of indices options available for selection
```
STRICT_EXPAND_OPEN: indices options that requires every specified index to exist, expands wildcards only to open indices and allows that no indices are resolved from wildcard expressions (not returning an error).
STRICT_EXPAND_OPEN_HIDDEN: indices options that requires every specified index to exist, expands wildcards only to open indices, includes hidden indices, and allows that no indices are resolved from wildcard expressions (not returning an error).
STRICT_EXPAND_OPEN_FORBID_CLOSED: indices options that requires every specified index to exist, expands wildcards only to open indices, allows that no indices are resolved from wildcard expressions (not returning an error) and forbids the use of closed indices by throwing an error.
STRICT_EXPAND_OPEN_HIDDEN_FORBID_CLOSED: indices options that requires every specified index to exist, expands wildcards only to open indices, includes hidden indices, and allows that no indices are resolved from wildcard expressions (not returning an error) and forbids the use of closed indices by throwing an error.
STRICT_EXPAND_OPEN_FORBID_CLOSED_IGNORE_THROTTLED: indices options that requires every specified index to exist, expands wildcards only to open indices, allows that no indices are resolved from wildcard expressions (not returning an error), forbids the use of closed indices by throwing an error and ignores indices that are throttled.
STRICT_EXPAND_OPEN_CLOSED: indices option that requires every specified index to exist, expands wildcards to both open and closed indices and allows that no indices are resolved from wildcard expressions (not returning an error).
STRICT_EXPAND_OPEN_CLOSED_HIDDEN: indices option that requires every specified index to exist, expands wildcards to both open and closed indices, includes hidden indices, and allows that no indices are resolved from wildcard expressions (not returning an error).
STRICT_SINGLE_INDEX_NO_EXPAND_FORBID_CLOSED: indices option that requires each specified index or alias to exist, doesn't expand wildcards and throws error if any of the aliases resolves to multiple indices.
LENIENT_EXPAND_OPEN: indices options that ignores unavailable indices, expands wildcards only to open indices and allows that no indices are resolved from wildcard expressions (not returning an error).
LENIENT_EXPAND_OPEN_HIDDEN: indices options that ignores unavailable indices, expands wildcards to open and hidden indices, and allows that no indices are resolved from wildcard expressions (not returning an error).
LENIENT_EXPAND_OPEN_CLOSED: indices options that ignores unavailable indices, expands wildcards to both open and closed indices and allows that no indices are resolved from wildcard expressions (not returning an error).
LENIENT_EXPAND_OPEN_CLOSED_HIDDEN: indices options that ignores unavailable indices, expands wildcards to all open and closed indices and allows that no indices are resolved from wildcard expressions (not returning an error).
```

## Usage

Metrics are directly available at:
Expand Down
62 changes: 30 additions & 32 deletions build.gradle
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,39 +3,8 @@ import org.opensearch.gradle.test.RestIntegTestTask
import java.util.regex.Matcher
import java.util.regex.Pattern

apply plugin: 'java'
apply plugin: 'idea'
apply plugin: 'opensearch.opensearchplugin'
apply plugin: 'opensearch.yaml-rest-test'

def pluginName = pluginName
def pluginDescription = pluginDescription
//def projectPath = 'org.opensearch'
//def pathToPlugin = 'path.to.plugin'
def pluginClassName = pluginClassname

opensearchplugin {
name pluginName
description pluginDescription
// classname "${projectPath}.${pathToPlugin}.${pluginClassName}"
classname pluginClassName
licenseFile rootProject.file('LICENSE.txt')
noticeFile rootProject.file('NOTICE.txt')
}

// thirdparty audit needs can be enabled
thirdPartyAudit.enabled = false

// This requires an additional Jar not published as part of build-tools
loggerUsageCheck.enabled = false

// No need to validate pom, as we do not upload to maven/sonatype
validateNebulaPom.enabled = false

buildscript {

ext {

// Four {digit(s)} concatenated with a "." followed by -rc{digit(s)}
// Examples: 1.0.0.1-rc2, 2.10.4.0-rc5
// Group (3) refers to the last .{digit{(s)} pattern (this is what we want to remove for OpS version)
Expand All @@ -49,7 +18,6 @@ buildscript {
String opensearch_version
String plugin_version = version


Matcher rc_matcher = RC_pattern.matcher(plugin_version)
Matcher other_matcher = OTHER_pattern.matcher(plugin_version)

Expand Down Expand Up @@ -83,6 +51,36 @@ buildscript {
}
}

apply plugin: 'java'
apply plugin: 'idea'
apply plugin: 'opensearch.opensearchplugin'
apply plugin: 'opensearch.yaml-rest-test'
//apply plugin: 'checkstyle'

def pluginName = pluginName
def pluginDescription = pluginDescription
//def projectPath = 'org.opensearch'
//def pathToPlugin = 'path.to.plugin'
def pluginClassName = pluginClassname

opensearchplugin {
name pluginName
description pluginDescription
// classname "${projectPath}.${pathToPlugin}.${pluginClassName}"
classname pluginClassName
licenseFile rootProject.file('LICENSE.txt')
noticeFile rootProject.file('NOTICE.txt')
}

// thirdparty audit needs can be enabled
thirdPartyAudit.enabled = false

// This requires an additional Jar not published as part of build-tools
loggerUsageCheck.enabled = false

// No need to validate pom, as we do not upload to maven/sonatype
validateNebulaPom.enabled = false

repositories {
mavenLocal()
maven { url "https://aws.oss.sonatype.org/content/repositories/releases" }
Expand Down
Loading