logfmon.conf.5

.\" $Id$
.\"
.\" Copyright (c) 2004 Nicholas Marriott <nicholas.marriott@gmail.com>
.\"
.\" Permission to use, copy, modify, and distribute this software for any
.\" purpose with or without fee is hereby granted, provided that the above
.\" copyright notice and this permission notice appear in all copies.
.\"
.\" THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
.\" WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
.\" MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
.\" ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
.\" WHATSOEVER RESULTING FROM LOSS OF MIND, USE, DATA OR PROFITS, WHETHER
.\" IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING
.\" OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
.\"
.Dd November 8, 2004
.Dt LOGFMON.CONF 5
.Os
.Sh NAME
.Nm logfmon.conf
.Nd "log file monitoring daemon configuration file"
.Sh DESCRIPTION
This manual page describes the
.Xr logfmon 8
configuration file.
It has the following format:
.Pp
Empty lines and lines beginning with the
.Sq #
character are ignored.
.Pp
Regexps and strings must be enclosed in double quotes.
Special characters in regexps and strings must be escaped.
Note that this may mean double-escaping in regexps.
.Pp
Possible commands are covered in the following sections.
.Sh OPTIONS
Options are configured using the
.Ic set
command.
It may be followed by the following options, one per command:
.Pp
.Bl -tag -width Ds
.It Ic mailtime Ar time
This is used to set the time between mails of unmatched messages.
.Pp
The time may be specified as a plain number in seconds or with a suffix of
.Ql h
for hours,
.Ql m
for minutes or
.Ql s
for seconds.
The default is 15 minutes and the minimum is 10 seconds.
.It Ic mailcmd Ar command
This sets the command to which the mail is piped.
For example:
.Bd -ragged -offset indent
set mailcmd "/usr/bin/mail -s \\"`date` log report\\" root"
.Ed
.Pp
No mail will be sent if the command is an empty string ("").
.It Ic user Ar user Li | Ar uid
This instructs
.Xr logfmon 8
to run as a different user or uid.
In common with other strings, a user name must be in double quotes.
An alternative user will only be used if
.Xr logfmon 8
is run as root.
.Pp
It is important to note that when
.Xr logfmon 8
runs as a different user or group, it will still need read access to any log files specified with a
.Ic file
command and the ability to execute any external commands (including the
.Ic mailcmd
setting) specified in the configuration file.
.It Ic group Ar group Li | Ar gid
This instructs
.Xr logfmon 8
to run as a different group or gid.
An alternative group will only be used if
.Xr logfmon 8
is run as root.
.It Ic cachefile Ar file
This informs
.Xr logfmon 8
of the path of the file used to retain a cache of the offsets and sizes of the log files to prevent duplicate processing of entries when it is restarted.
An empty string disables caching.
Note that this option may be overridden on the command line.
The default setting is /var/db/logfmon.cache.
.It Ic pidfile Ar file
This instructs
.Xr logfmon 8
to write its pid into the file specified.
If an empty string is used, the pid is not written.
If this entry does not appear in the configuration file, the default of /var/run/logfmon.pid is used.
Note that this option may be overridden on the command line.
.It Ic logregexp Ar regexp
This sets the regexp that
.Xr logfmon 8
uses to select the part of each log line which is to be compared against the ruleset.
The first submatch (enclosed in parenthesis) is used.
The default regexp is set for
.Xr syslog 3
format:
.Bd -ragged -offset indent
^[A-Z][a-z][a-z] [0-9 ][0-9] [0-9][0-9]:[0-9][0-9]:[0-9][0-9] [^ ]* (.*)$
.Ed
.It Ic maxthreads Ar number
This sets the maximum number of threads
.Xr logfmon 8
will attempt to start simultaneously.
If this number is reached,
.Xr logfmon 8
will block until some of the threads exit.
The default is 100.
.Sh MACROS
Macros may be defined using the following syntax:
.Bl -tag -width Ds
.It Ar $name Ic = Ar string
.It Ar %name Ic = Ar number
.El
.Pp
Macros are prefixed with $ to indicate a string value and % to indicate a numeric value.
Once defined, a macro may be used in any place a string or number is expected. Macros may be embedded in strings by surrounding their name (after the $ or %) with {}s, like so:
.Bd -ragged -offset indent
"abc ${mymacro} %{anothermacro} def"
.Ed
.Sh FILES
The
.Ic file
command is used to instruct
.Xr logfmon 8
to monitor a file.
The filename must be given as a parameter.
.Pp
Optionally, the file may be given a tag by specifying
.Ic tag Ar tag
following the filename.
This tag is used in
.Ic match
rules, described below, to restrict matches to a particular file.
The tag is restricted to alphanumeric characters, hyphens and underscores.
If no tag is given for a file,
.Xr logfmon 8
assigns it a tag.
.Pp
An example of a
.Ic file
command is:
.Bd -ragged -offset indent
file "/var/log/daemon" tag daemon
.Ed
.Sh RULES
Rules are described using the
.Ic match
command.
This section describes simple rules, contexts are covered in a seperate section below.
.Pp
Each
.Ic match
command gives
.Xr logfmon 8
a regexp to test each new log entry against, an action to perform and
optionally a tag or set of tags to restrict matches to a single file.
.Pp
The format of a match command is:
.Bl -tag -width Ds
.It Xo Ic match
.Li [\&  Ic in Ar tags Li ]
.Ar regexp
.Li [\&  Ic not Ar regexp Li ]
.Ar action
.Xc
.El
.Pp
If the
.Ic in Ar tags
part is not supplied, or is a single asterisk, the rule is applied in all files.
A rule may be applied to multiple tags by enclosing them in braces, for example { tag1 tag2 tag3 }.
Braces are not necessary for a single tag.
.Pp
If a
.Ic not Ar regexp
part is supplied, the rule only applies if the log entry both matches the
first regexp and
.Em doesn't
match the second, not, regexp.
.Pp
For this form of
.Ic match
command, testing of a log entry is stopped as soon as a matching rule is found.
.Pp
The possible actions are:
.Bl -tag -width Ds
.It Ic ignore
Any matched log entries in the specified files are ignored.
They are not batched into the unmatched email and are tested against no further rules.
.Pp
An example
.Ic ignore
rule is:
.Bd -ragged -offset indent
match in daemon "ntpd\\\\[[0-9]*\\\\]: adjusting local clock by [0-9.-]*s" ignore
.Ed
.It Ic exec Ar command
This executes
.Ar command
when a matching entry is seen.
$0 is replaced in the command with the entire log entry and $1 to $9 are replaced with any bracketted match expressions in the regexp.
.Pp
An example
.Ic exec
rule is:
.Bd -ragged -offset indent
match "sshd\\\\[[0-9]*\\\\]: Invalid user .* from ([0-9.]*)" exec "grep $1 /etc/pf.ignore || (echo $1 >> /etc/pf.ignore; pfctl -t ignore -T add $1; pfctl -k $1; echo $0 | mail -s 'ssh attempt' root)"
.Ed
.It Ic pipe Ar command
With this action, the entire log entry is piped to the supplied command
.It Ic write Ar file
This action writes the log entry to the specified file, creating it if it does not exist and truncating it if it does.
.It Ic write-append Ar file
This appends the entire log entry to a file.
.El
.Sh CONTEXTS
A context is a set of log entries that has been collected together.
A context is opened by an 
.Ic open
action on a
.Ic match
rule, messages may be appended to it with an
.Ic append
action, it may be closed and an action taken with a
.Ic close
rule and an action taken on the collected log entries and the context emptied without being removed by using the
.Ic clear
action.
.Pp
A context expires and is removed after a time supplied to the
.Ic open
rule.
An action may be taken when a context expires, or, with a
.Ic when
command, optionally when it reaches a certain number of entries.
If a
.Ic pipe
action is used, then $1 to $9 is replaced by any match expressions from the matched log line before the context is piped to it.
.Pp
Note that unlike the other rules,
.Xr logfmon 8
does
.Em not
stop parsing rules when a message matches a context rule.
This means that
without a matching
.Ic ignore
rule, messages that match any of the context rules described below will be included in the email of unmatched rules.
This also means, however, that messages matching an
.Ic open
can be included in the context with an
.Ic append
command, or indeed used to
.Ic exec
or
.Ic pipe
in later rules.
.Pp
The forms of
.Ic match
command relating to contexts are described below.
All the non-context actions,
.Ic ignore ,
.Ic pipe ,
.Ic exec ,
.Ic write
and
.Ic write-append ,
are permitted in rules where
.Ar action
appears below.
When
.Ic pipe
is used, the entire context is piped to the supplied command.
When
.Ic write
or
.Ic write-append
are used,
.Xr logfmon 8
attempts to write the context to the specified file, overwriting with the former and appending with the latter.
.Bl -tag -width Ds
.It Xo Ic match
.Li [\&  Ic in Ar tags Li ]
.Ar regexp
.Li [\&  Ic not Ar regexp Li ]
.Ic open Ar name
.Li [\&  Ic autoappend Li ]
.Ic expire Ar time
.Ar action
.Li [\&  Ic when Ar num
.Ar action Li ]
.Xc
This opens a context with name
.Ar name
and sets it to expire after the time specified.
.Ar time
is in the same format as for the
.Ic set mailtime
command.
$0 to $9 are replaced as normal in the context name.
.Pp
The expiry time is counted from the point at which the context is created, so a context with a time of two minutes will be expired after two minutes regardless of when the last message was appended to it.
.Pp
If the optional
.Ic when
part of the rule is supplied, the specified
.Ar action
is taken and the context is removed when the context holds
.Ar num
entries.
.Pp
The
.Ic autoappend
keyword may be included to automatically create a subsequent
.Ic append
rule with the same regexps and context name.
.It Xo Ic match
.Li [\&  Ic in Ar tags Li ]
.Ar regexp
.Li [\&  Ic not Ar regexp Li ]
.Ic append Ar name
.Xc
This appends a matching log entry to a context.
If the context does not exist, this rule is silently ignored.
.It Xo Ic match
.Li [\&  Ic in Ar tags Li ]
.Ar regexp
.Li [\&  Ic not Ar regexp Li ]
.Ic close Ar name Ar action
.Xc
This applies the specified action and removes the context.
.It Xo Ic match
.Li [\&  Ic in Ar tags Li ]
.Ar regexp
.Li [\&  Ic not Ar regexp Li ]
.Ic clear Ar name Ar action
.Xc
This applies the specified action and clears all accumulated log entries from the context.
.El
.Pp
An example set of context rules is:
.Bd -ragged -offset indent
match in auth "sshd\\\\[([0-9]*)\\\\]: input_userauth_request: invalid user .*" open "sshd-$1" expire 2m pipe "/usr/bin/mail -s \\"`date` ssh attempt (expired)\\" root"
.Ed
.Bd -ragged -offset indent
match in auth "sshd\\\\[([0-9]*)\\\\]: .*" append "sshd-$1"
.Ed
.Bd -ragged -offset indent
match in auth "sshd\\\\[([0-9]*)\\\\]: Received disconnect from .*" close "sshd-$1" pipe "/usr/bin/mail -s \\"`date` ssh attempt\\" root"
.Ed
.Pp
The first rule opens the context named with the sshd pid, the second appends all messages from the same sshd pid (including the messages matching the open and close rules) to the context and the third rule closes and mails the context when the remote client disconnects.
.Sh FILES
.Bl -tag -width "/etc/logfmon.confXXX" -compact
.It Pa /etc/logfmon.conf
default
.Xr logfmon 8
configuration file
.El
.Sh SEE ALSO
.Xr re_format 7 ,
.Xr logfmon 8
.Sh AUTHORS
.An Nicholas Marriott Aq nicholas.marriott@gmail.com