Adding inforamtion for CRI-O configuration and enhancing our manifest… #32151

gizas · 2022-06-29T11:47:12Z

… comments

What does this PR do?

This PR adds documentation reference for CRI-O paths in container input and also in our kubernetes manifests.
Comments added in our manifests to guide user to config CRI-O running environment

Why is it important?

https://github.com/elastic/sdh-beats/issues/2210

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have made corresponding change to the default configuration files
I have added tests that prove my fix is effective or that my feature works
I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

[ ]

How to test this PR locally

Related issues

Closes Add documentation for CRI-O paths #32149

Use cases

Screenshots

Logs

… comments

elasticmachine · 2022-06-29T11:58:31Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2022-07-11T12:32:36.895+0000
Duration: 77 min 24 sec

Test stats 🧪

Test	Results
Failed	0
Passed	6286
Skipped	737
Total	7023

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

ChrsMark

Thanks for adding this! Keep in mind that in pure input's configs providers' content is not available so we should structure the patterns accordingly.

Also I would consider adding a note regarding CRI-O in https://www.elastic.co/guide/en/beats/filebeat/master/running-on-kubernetes.html.

ChrsMark · 2022-06-29T12:04:43Z

deploy/kubernetes/filebeat-kubernetes.yaml

@@ -11,6 +11,8 @@ data:
    filebeat.inputs:
    - type: container
      paths:
+        #Uncomment below line if you want to enable CRI-O based logs autodiscover
+        #- /var/log/pods/${data.kubernetes.pod.uid}/${data.kubernetes.container.name}/*.log      


Content like ${data.kubernetes.*} is only available in configurations generated by the kubernetes provider, template based or hint's based. Here the configuration is the container input so the path should be sth like /var/log/pods/*/*/*.log. Haven't test it so it should be safe to make a test to ensure it's working.

is there any issue with /var/log/containers/*.log when used cri-o?
/var/log/containers/*.log is usually just a symlink of /var/log/pods/*/*/*.log

Let me test it.

If that is the case @tetianakravchenko then we dont need to do any changes at all. Not sure I was only following what we have in the code. Let me first test /var/log/pods/*/*/*.log and we can sync

@tetianakravchenko You are correct, symlink it is.

So removed it from here, I like better the /var/log/containers/*.log as have no subfolders.

@gizas do we then need this PR if the default path /var/log/containers/*.log should work for different runtimes, including cri-o?

Also /var/log/containers/*.log is symlink in containerd envs, checked it in docker.

On one hand our examples in doc show /var/log/pods/ and /var/lib/docker in each case. So we need to make sure that users have this info seperate per environment if they want to use actual links

Now the /var/log/containers/*.log is what we will propose in general, correct but I am not 100% sure that symlink is always the case in all envs. Just to be on safe side.

Does it make sense?

On one hand our examples in doc show /var/log/pods/ and /var/lib/docker in each case

you mean /var/log/containers/, instead of /var/log/pods/?

Now the /var/log/containers/*.log is what we will propose in general, correct but I am not 100% sure that symlink is always the case in all envs. Just to be on safe side.
Does it make sense?

to be on safe side - fine, but in this PR is emphasized that for cri-o runtime must be used /var/log/pods/* path, that is not correct and is misleading, no?

@tetianakravchenko updated based on our discussion. Let me know if ok now?

gizas · 2022-07-05T07:12:45Z

Also I would consider adding a note regarding CRI-O in https://www.elastic.co/guide/en/beats/filebeat/master/running-on-kubernetes.html.

It was in this PR: filebeat/docs/running-on-kubernetes.asciidoc @ChrsMark, is not it ok?

mergify · 2022-07-08T21:38:44Z

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b docs_for_crio upstream/docs_for_crio
git merge upstream/main
git push upstream docs_for_crio

…r_crio

…manifest… (elastic#32151)" This reverts commit c47ef26.

#32151) * [DOCS] Removed reference to the Stack GS (#32083) * [DOCS] Removed reference to the Stack GS (#32119) * Adding inforamtion for CRI-O configuration and enhancing our manifest comments * Adding CHANGELOG.asciidoc info * Adding CHANGELOG.next.asciidoc info * Keeping only containers path * Keeping only containers path * Keeping only containers path * Adding note * Adding note * Adding note * Adding note * Adding note * Update input-container.asciidoc * Update running-on-kubernetes.asciidoc * Fixing Conflicts * Fixing Conflicts * Removig other path references and kept only /var/log/containers/ * Removig other path references and kept only /var/log/containers/ * Adding generated files Co-authored-by: debadair <debadair@elastic.co>

debadair and others added 3 commits June 23, 2022 16:32

[DOCS] Removed reference to the Stack GS (#32083)

5a15d3c

[DOCS] Removed reference to the Stack GS (#32119)

a55ae58

Adding inforamtion for CRI-O configuration and enhancing our manifest…

e90f8fd

… comments

gizas requested review from a team as code owners June 29, 2022 11:47

gizas requested review from cmacknz and kvch and removed request for a team June 29, 2022 11:47

botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jun 29, 2022

mergify bot assigned gizas Jun 29, 2022

Adding CHANGELOG.asciidoc info

f51a450

gizas removed request for cmacknz and kvch June 29, 2022 11:50

gizas added the Team:Cloudnative-Monitoring Label for the Cloud Native Monitoring team label Jun 29, 2022

botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Jun 29, 2022

gizas added docs needs_team Indicates that the issue/PR needs a Team:* label labels Jun 29, 2022

botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Jun 29, 2022

Adding CHANGELOG.next.asciidoc info

ed0a319

gizas mentioned this pull request Jun 29, 2022

Add documentation for CRI-O paths #32149

Closed

ChrsMark reviewed Jun 29, 2022

View reviewed changes

gizas added 7 commits July 5, 2022 10:41

Keeping only containers path

330a6a3

Keeping only containers path

33ef360

Keeping only containers path

a7ba4dc

Adding note

ad3a9c8

Adding note

8e53475

Adding note

2056aed

Adding note

1e817ab

gizas added 3 commits July 5, 2022 11:16

Adding note

5f75acc

Update input-container.asciidoc

2ebb955

Update running-on-kubernetes.asciidoc

5d4ffff

ChrsMark approved these changes Jul 11, 2022

View reviewed changes

gizas added 7 commits July 11, 2022 10:43

Merge branch 'master' of github.com:elastic/beats into docs_for_crio

d464cac

Fixing Conflicts

e1ffd4e

Fixing Conflicts

e04d6d3

Merge branch 'main' into docs_for_crio

f48f0bc

Removig other path references and kept only /var/log/containers/

7a29571

Merge branch 'docs_for_crio' of github.com:elastic/beats into docs_fo…

237d660

…r_crio

Removig other path references and kept only /var/log/containers/

0094e01

tetianakravchenko approved these changes Jul 11, 2022

View reviewed changes

Adding generated files

15a62af

gizas merged commit c47ef26 into main Jul 11, 2022

gizas deleted the docs_for_crio branch July 11, 2022 14:23

bmorelli25 added a commit to bmorelli25/beats that referenced this pull request Jul 11, 2022

Revert "Adding inforamtion for CRI-O configuration and enhancing our …

aea4c74

…manifest… (elastic#32151)" This reverts commit c47ef26.

bmorelli25 mentioned this pull request Jul 11, 2022

Revert "Adding inforamtion for CRI-O configuration and enhancing our … #32306

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding inforamtion for CRI-O configuration and enhancing our manifest… #32151

Adding inforamtion for CRI-O configuration and enhancing our manifest… #32151

gizas commented Jun 29, 2022 •

edited

Loading

elasticmachine commented Jun 29, 2022 •

edited by jenkins-beats-ci bot

Loading

Build stats

Test stats 🧪

ChrsMark left a comment

ChrsMark Jun 29, 2022

tetianakravchenko Jul 4, 2022

gizas Jul 5, 2022

gizas Jul 5, 2022

tetianakravchenko Jul 7, 2022

gizas Jul 8, 2022

tetianakravchenko Jul 11, 2022

gizas Jul 11, 2022

gizas commented Jul 5, 2022

mergify bot commented Jul 8, 2022

Adding inforamtion for CRI-O configuration and enhancing our manifest… #32151

Adding inforamtion for CRI-O configuration and enhancing our manifest… #32151

Conversation

gizas commented Jun 29, 2022 • edited Loading

What does this PR do?

Why is it important?

Checklist

Author's Checklist

How to test this PR locally

Related issues

Use cases

Screenshots

Logs

elasticmachine commented Jun 29, 2022 • edited by jenkins-beats-ci bot Loading

💚 Build Succeeded

Build stats

Test stats 🧪

💚 Flaky test report

🤖 GitHub comments

ChrsMark left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gizas commented Jul 5, 2022

mergify bot commented Jul 8, 2022

gizas commented Jun 29, 2022 •

edited

Loading

elasticmachine commented Jun 29, 2022 •

edited by jenkins-beats-ci bot

Loading