Provide Docker target and discovery in Promtail. #4911

jeschkies · 2021-12-10T16:28:15Z

What this PR does / why we need it:
This patch adds support to fetch Docker container logs through the Docker daemon API. This should be more robust than the Loki Docker driver or scraping the logs files.

The new Docker target will also collect meta information of the scraped containers. This patch includes a specific Docker target and service discovery based on the Prometheus services discovery.

Which issue(s) this PR fixes:
Addresses #2361
Closes #4703

Special notes for your reviewer:

Checklist

Documentation added
Tests updated
Add an entry in the CHANGELOG.md about the changes.

Addresses grafana#2361 Closes grafana#4703

jeschkies · 2021-12-10T16:28:53Z

clients/pkg/promtail/targets/docker/targetmanager.go

+}
+
+func (tm *TargetManager) sync(groups []*targetgroup.Group) {
+// TODO: implement syncer that adds and removes Docker targets.


I wonder whether a syncer struct is really required.

pstibrany

I took a quick look at channels usage based on Slack discussion and added some comments.

clients/pkg/promtail/targets/docker/target.go

clients/pkg/promtail/targets/docker/target_group.go

docs/sources/clients/promtail/configuration.md

clients/pkg/promtail/targets/docker/target.go

clients/pkg/promtail/targets/docker/target_group.go

cyriltovena

LGTM

docs/sources/clients/promtail/configuration.md

docs/sources/clients/docker-driver/_index.md

docs/sources/clients/promtail/configuration.md

Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>

KMiller-Grafana

Thanks for making all the changes. LGTM.

tennox · 2022-03-04T17:07:38Z

For people looking to try this pre-release:
Docs: https://grafana.com/docs/loki/next/clients/promtail/configuration/#docker_sd_config
Docker image: grafana/promtail:main

try fix grafana/loki#2361 with grafana/loki#4911 docs: https://grafana.com/docs/loki/next/clients/promtail/configuration/#docker_sd_config

jeschkies added 3 commits December 9, 2021 17:19

Provide Docker target and discovery in Promtail.

c74a42c

Addresses grafana#2361 Closes grafana#4703

Add Docker service discovery.

0ca5f1c

Add Docker service discovery.

003b95a

jeschkies requested review from cyriltovena and owen-d December 10, 2021 16:28

pull-request-size bot added the size/L label Dec 10, 2021

jeschkies commented Dec 10, 2021

View reviewed changes

pstibrany reviewed Dec 13, 2021

View reviewed changes

clients/pkg/promtail/targets/docker/target.go Outdated Show resolved Hide resolved

clients/pkg/promtail/targets/docker/target.go Outdated Show resolved Hide resolved

clients/pkg/promtail/targets/docker/target.go Outdated Show resolved Hide resolved

Use stderr writer.

5d2c8c8

srstsavage mentioned this pull request Dec 16, 2021

if loki is not reachable and loki-docker-driver is activated, containers apps stops and cannot be stopped/killed #2361

Open

jeschkies added 2 commits December 23, 2021 18:22

Rename target manager name in error message.

05a3581

Persist position.

d5d7166

pull-request-size bot added size/XL and removed size/L labels Dec 23, 2021

jeschkies added 4 commits January 4, 2022 17:26

Run discovery manager.

ce8767e

Add target.

e19393c

Split out target creation into syncer.

d6acba3

Add targets.

2fbfd7d

trevorwhitney mentioned this pull request Jan 6, 2022

Logspout integration for logging through Docker #82

Open

jeschkies added 4 commits January 7, 2022 16:28

Document Docker target and service discovery.

b90d0b3

Rename syncer to target_group and make it threadsafe.

e874238

Format code.

165d07f

Test target manager.

7f149bb

jeschkies marked this pull request as ready for review January 7, 2022 16:43

jeschkies requested review from KMiller-Grafana and a team as code owners January 7, 2022 16:43

jeschkies added 4 commits January 7, 2022 17:49

Test target manager.

15d8121

Make linter happy.

ef4b805

Add changelog entry.

564a62e

Merge remote-tracking branch 'grafana/main' into karsten/docker-sd

01eea28