Skip to content

Latest commit

 

History

History
83 lines (66 loc) · 2.85 KB

log_parsers.md

File metadata and controls

83 lines (66 loc) · 2.85 KB
Raw

How to Create Log Parser for Container into fluent-bit

If our log message is parsed correctly, it has to be showed in Grafana like this:

  {"log":"OpenAPI AggregationController: Processing item v1beta1.metrics.k8s.io","pid":"1","severity":"INFO","source":"controller.go:107"}

Otherwise it will looks like this:

{
  "log":"{
  \"level\":\"info\",\"ts\":\"2020-06-01T11:23:26.679Z\",\"logger\":\"gardener-resource-manager.health-reconciler\",\"msg\":\"Finished ManagedResource health checks\",\"object\":\"garden/provider-aws-dsm9r\"
  }\n"
  }
}

Create a Custom Parser

  • First of all, we need to know how the log for the specific container looks like (for example, lets take a log from the alertmanager : level=info ts=2019-01-28T12:33:49.362015626Z caller=main.go:175 build_context="(go=go1.11.2, user=root@4ecc17c53d26, date=20181109-15:40:48))

  • We can see that this log contains 4 subfields(severity=info, timestamp=2019-01-28T12:33:49.362015626Z, source=main.go:175 and the actual message). So we have to write a regex which matches this log in 4 groups(We can use https://regex101.com/ like helping tool). So, for this purpose our regex looks like this:

^level=(?<severity>\w+)\s+ts=(?<time>\d{4}-\d{2}-\d{2}[Tt].*[zZ])\s+caller=(?<source>[^\s]*+)\s+(?<log>.*)

%Y-%m-%dT%H:%M:%S.%L
  • It's time to apply our new regex into fluent-bit configuration. Go to fluent-bit-configmap.yaml and create new filter using the following template:
[FILTER]
        Name                parser
        Match               kubernetes.<< pod-name >>*<< container-name >>*
        Key_Name            log
        Parser              << parser-name >>
        Reserve_Data        True

EXAMPLE
[FILTER]
        Name                parser
        Match               kubernetes.alertmanager*alertmanager*
        Key_Name            log
        Parser              alermanagerParser
        Reserve_Data        True
  • Now lets check if there already exists parser with such a regex and time format that we need. If it doesn't, create one:
[PARSER]
        Name        << parser-name >>
        Format      regex
        Regex       << regex >>
        Time_Key    time
        Time_Format << time-format >>

EXAMPLE
[PARSER]
        Name        alermanagerParser
        Format      regex
        Regex       ^level=(?<severity>\w+)\s+ts=(?<time>\d{4}-\d{2}-\d{2}[Tt].*[zZ])\s+caller=(?<source>[^\s]*+)\s+(?<log>.*)
        Time_Key    time
        Time_Format %Y-%m-%dT%H:%M:%S.%L

Follow your development setup to validate that the parsers are working correctly.