Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Log by default to journald on systems with systemd #8942

Merged
merged 10 commits into from
Dec 9, 2018

Conversation

jsoriano
Copy link
Member

@jsoriano jsoriano commented Nov 6, 2018

Add -e flag to command line when running beats from systemd so beats logs are collected by journald by default in systems with systemd.

Add also some environment variables so command line flags can be more easily overriden with a drop-in unit or with systemctl edit.

Add documentation about the use of Beats with systemd.

Copy link
Member

@andrewkroh andrewkroh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this change because it makes our software behave similar to other services running on a host.

I think we'll need to add some documentation that explains how to view the logs with journalctl. One worry I have is that users will make config changes to logging.to_X or logging.files.* and expect them to take affect but those settings are pre-empted by the -e. Maybe there are some things we can do to mitigate that risk (probably just via docs).

@@ -14,6 +14,7 @@ https://github.com/elastic/beats/compare/v6.4.0...master[Check the HEAD diff]
*Affecting all Beats*

- Dissect syntax change, use * instead of ? when working with field reference. {issue}8054[8054]
- Add `-e` flag by default to systemd unit so logs are collected by journald, behaviour can be reverted by overriding BEAT_LOG_OPTS environment variable with an empty value {pull}[]
Copy link
Member

@andrewkroh andrewkroh Nov 6, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I'd word this as a more direct statement toward systemd users, like "On systems with systemd the Beat logs are now written to journald by default rather than file. ..." Or some other wording that makes the breaking change get noticed by those affected.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the suggestion, changelog rephrased, added also to developers changelog.

@jsoriano
Copy link
Member Author

jsoriano commented Nov 6, 2018

Added documentation page to include details about the use of systemd with Beats.

@jsoriano jsoriano added the docs label Nov 8, 2018
@kvch
Copy link
Contributor

kvch commented Nov 14, 2018

Could you please rebase your branch?

@jsoriano
Copy link
Member Author

@dedemorton could you please review the docs added? Thanks!

CHANGELOG-developer.asciidoc Outdated Show resolved Hide resolved
libbeat/docs/shared-systemd.asciidoc Outdated Show resolved Hide resolved
@ruflin ruflin removed Team:Integrations Label for the Integrations team labels Nov 27, 2018
@alvarolobato alvarolobato added the Team:Integrations Label for the Integrations team label Dec 3, 2018
@elasticmachine
Copy link
Collaborator

Pinging @elastic/infrastructure

@jsoriano jsoriano self-assigned this Dec 3, 2018
@jsoriano jsoriano removed the Team:Integrations Label for the Integrations team label Dec 4, 2018
Copy link
Contributor

@dedemorton dedemorton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice job! Some editorial changes, but otherwise looks good. I wonder if we should add links to this page from the getting started docs?

CHANGELOG.asciidoc Outdated Show resolved Hide resolved
libbeat/docs/shared-systemd.asciidoc Outdated Show resolved Hide resolved
libbeat/docs/shared-systemd.asciidoc Outdated Show resolved Hide resolved
libbeat/docs/shared-systemd.asciidoc Outdated Show resolved Hide resolved
libbeat/docs/shared-systemd.asciidoc Outdated Show resolved Hide resolved
libbeat/docs/shared-systemd.asciidoc Outdated Show resolved Hide resolved
libbeat/docs/shared-systemd.asciidoc Outdated Show resolved Hide resolved
libbeat/docs/shared-systemd.asciidoc Outdated Show resolved Hide resolved
libbeat/docs/shared-systemd.asciidoc Outdated Show resolved Hide resolved
libbeat/docs/shared-systemd.asciidoc Outdated Show resolved Hide resolved
@jsoriano
Copy link
Member Author

jsoriano commented Dec 5, 2018

@dedemorton thanks for your suggestions! I have applied them.

@jsoriano
Copy link
Member Author

jsoriano commented Dec 7, 2018

I am going to merge this so it gets into the next 7.0 builds.

@jsoriano jsoriano dismissed dedemorton’s stale review December 7, 2018 13:54

As we talked offline, we can open another PR for more changes in docs if needed.


NOTE: It is recommended that you use a configuration management tool to
include drop-in unit files. If you need to add a drop-in manually, use
`systemctl edit {beatname_lc}.service`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change:

`systemctl edit {beatname_lc}.service`

To

+systemctl edit {beatname_lc}.service+

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops, true, you already mentioned that, thanks!
I have wrapped a couple more command lines that were unwrapped.

Copy link
Contributor

@dedemorton dedemorton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jsoriano I missed one minor correction. Commands that contain attributes (like {beatname_lc}) need to be wrapped in plus signs, not back ticks.

Copy link
Contributor

@dedemorton dedemorton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doc changes LGTM.

@jsoriano
Copy link
Member Author

jsoriano commented Dec 8, 2018

jenkins, test this again

@jsoriano
Copy link
Member Author

jsoriano commented Dec 9, 2018

Failures in jenkins not related

@jsoriano jsoriano merged commit 07d4f7a into elastic:master Dec 9, 2018
@jsoriano jsoriano deleted the systemd-unit-logging branch December 9, 2018 19:16
andrewkroh added a commit to andrewkroh/beats-tester that referenced this pull request Dec 19, 2018
This overrides the defaults in the unit file for out services so that we get a log file. The tests currently assert that the log exists.

We could extend the tests to assert that logs are written to journald in the future.

Relates to elastic/beats#8942
andrewkroh added a commit to elastic/beats-tester that referenced this pull request Dec 19, 2018
This overrides the defaults in the unit file for out services so that we get a log file. The tests currently assert that the log exists.

We could extend the tests to assert that logs are written to journald in the future.

Relates to elastic/beats#8942
DStape pushed a commit to DStape/beats that referenced this pull request Aug 20, 2019
Add `-e` flag to command line when running beats from systemd so beats
logs are collected by journald by default in systems with systemd.

Add also some environment variables so command line flags can be
more easily overriden with a drop-in unit or with `systemctl edit`.

Add documentation about the use of Beats with systemd.
@urso urso mentioned this pull request Jan 9, 2020
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants