Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG]Conflicting logs index templates #812

Closed
YANG-DB opened this issue Aug 3, 2023 · 0 comments · Fixed by #818
Closed

[BUG]Conflicting logs index templates #812

YANG-DB opened this issue Aug 3, 2023 · 0 comments · Fixed by #818
Assignees
Labels
bug Something isn't working integrations Used to denote items related to the Integrations project

Comments

@YANG-DB
Copy link
Member

YANG-DB commented Aug 3, 2023

What is the bug?
when attempting to add multiple integrations that contain log templates - the patterns collide and prevent correct creation of the specific integration's log index template

How can one reproduce the bug?
Setup AWS-ELB integration
Setup Nginx integration
The logs index templates are conflicting since they both share the next index pattern:

    "index_patterns": [
        "ss4o_logs-*-*"
    ],
    "data_stream": {},
    "template": {
   ....

What is the expected behavior?
Since the logs index templates are not created in advance for all types of possible log combinations,
We need a clear distinction between each log associated for each integration:

  1. Each integration will include the log mapping with a distinct name:
  • ELB will contain logs-elb where the index pattern inside the index template will be
    "index_patterns": [
        "ss4o_logs-elb-*"
    ],
    "data_stream": {},
    "template": {
   ....
  • nginx will contain logs-nginx where the index pattern inside the index template will be
    "index_patterns": [
        "ss4o_logs-nginx-*"
    ],
    "data_stream": {},
    "template": {
   ....

And so on for the rest of the common log index templates.

This will prevent the collision both in the index-pattern and both in the index mapping template name that will be very explicit for which integration it addresses - this also makes the template priority redundant ...

Simplified query access

In addition to simplify the queries access to the index - each mapping will contain a shortened version of the index name - AKA alias:

For ss4o_logs-elb-* the template will contain the next index alias:

    "aliases": {
      "logs-elb": {}
    },

This will simplify the queries and access patterns to the generated log indices and define the standard access convention in a meaningful format .

Do you have any additional context?
Avoid index pattern collisions

@YANG-DB YANG-DB added bug Something isn't working integrations Used to denote items related to the Integrations project labels Aug 3, 2023
@derek-ho derek-ho removed the untriaged label Aug 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working integrations Used to denote items related to the Integrations project
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants