Fluent logger

A structured event logger

Examples

Simple

require 'fluent-logger'

# API: FluentLogger.new(tag_prefix, options)
log = Fluent::Logger::FluentLogger.new(nil, :host => 'localhost', :port => 24224)
unless log.post("myapp.access", {"agent" => "foo"})
  p log.last_error # You can get last error object via last_error method
end

# output: myapp.access {"agent":"foo"}

UNIX socket

require 'fluent-logger'

log = Fluent::Logger::FluentLogger.new(nil, :socket_path => "/tmp/fluent.sock")
unless log.post("myapp.access", {"agent" => "foo"})
  # Passed records are stored into logger's internal buffer so don't re-post same event.
  p log.last_error # You can get last error object via last_error method
end

# output: myapp.access {"agent":"foo"}

Tag prefix

require 'fluent-logger'

log = Fluent::Logger::FluentLogger.new('myapp', :host => 'localhost', :port => 24224)
log.post("access", {"agent" => "foo"})

# output: myapp.access {"agent":"foo"}

Nonblocking write

require 'fluent-logger'

log = Fluent::Logger::FluentLogger.new(nil, :host => 'localhost', :port => 24224, :use_nonblock => true, :wait_writeable => false)
# When wait_writeable is false
begin
  log.post("myapp.access", {"agent" => "foo"})
rescue IO::EAGAINWaitWritable => e
  # wait code for avoding "Resource temporarily unavailable"
  # Passed records are stored into logger's internal buffer so don't re-post same event.
end

# When wait_writeable is true
unless log.post("myapp.access", {"agent" => "foo"})
  # same as other example
end

# output: myapp.access {"agent":"foo"}

Singleton

require 'fluent-logger'

Fluent::Logger::FluentLogger.open(nil, :host => 'localhost', :port => 24224)
Fluent::Logger.post("myapp.access", {"agent" => "foo"})

# output: myapp.access {"agent":"foo"}

Logger options

host (String)

fluentd instance host

port (Integer)

fluentd instance port

socket_path (String)

If specified, fluentd uses unix domain socket instead of TCP.

nanosecond_precision (Bool)

Use nano second event time instead of epoch. See also "Tips" section.

use_nonblock (Bool)

Use nonblocking write(IO#write_nonblock) instead of normal write(IO#write). If Logger#post stuck on your environment, specify true.

wait_writeable (Bool)

If true, Logger#post raises an error when nonblocking write gets EAGAIN.

buffer_overflow_handler (Proc)

Pass callback for handling buffer overflow with pending data. See "Buffer overflow" section.

Standard ::Logger compatible interface

Example1

require 'fluent-logger'
f = Fluent::Logger::LevelFluentLogger.new('fluent')

f.info("some application running.")
# output: fluent.info: {"level":"INFO","message":"some application running."}

f.warn("some application running.")
# output: fluent.warn: {"level":"WARN","message":"some application running."}

Example2(add progname)

require 'fluent-logger'
f = Fluent::Logger::LevelFluentLogger.new('fluent')
f.info("some_application") {"some application running."}
# output: fluent.info: {"level":"INFO","message":"some application running.","progname":"some_application"}

Example3(set log level)

require 'fluent-logger'
f = Fluent::Logger::LevelFluentLogger.new('fluent')
f.level = Logger::WARN
f.info("some_application") {"some application running."}

Log level is ERROR so no output.

default log level is debug.

Example4(customize format for Rails)

require 'fluent-logger'
f = Fluent::Logger::LevelFluentLogger.new('fluent')

f.formatter = proc do |severity, datetime, progname, message|
  map = { level: severity }
  map[:message] = message if message
  map[:progname] = progname if progname
  map[:stage] = ENV['RAILS_ENV']
  map[:service_name] = "SomeApp"
  map
end

f.info("some_application"){"some application running."}
# output: fluent.info: {"level":"INFO","message":"some application running.","progname":"some_application","stage":"production","service_name":"SomeApp"}

Loggers

Fluent

Fluent::Logger::FluentLogger.open('tag_prefix', :host => 'localhost', :port => 24224)

Console

Fluent::Logger::ConsoleLogger.open(io)

Null

Fluent::Logger::NullLogger.open

Tips

Use nanosecond-precision time

To send events with nanosecond-precision time (Fluent 0.14 and up), specify nanosecond_precision to FluentLogger constructor.

log = Fluent::Logger::FluentLogger.new(nil, :host => 'localhost', :port => 24224, :nanosecond_precision => true)
# Use nanosecond time instead
log.post("myapp.access", {"agent" => "foo"})
log.post_with_time("myapp.access", {"agent" => "foo"}, Time.now) # Need Time object for post_with_time

Buffer overflow

You can inject your own custom proc to handle buffer overflow in the event of connection failure. This will mitigate the loss of data instead of simply throwing data away.

Your proc must accept a single argument, which will be the internal buffer of messages from the logger. A typical use-case for this would be writing to disk or possibly writing to Redis.

Example

class BufferOverflowHandler
  attr_accessor :buffer

  def flush(messages)
    @buffer ||= []
    MessagePack::Unpacker.new.feed_each(messages) do |msg|
      @buffer << msg
    end
  end
end

handler = Proc.new { |messages| BufferOverflowHandler.new.flush(messages) }

Fluent::Logger::FluentLogger.new(nil,
  :host => 'localhost', :port => 24224,
  :buffer_overflow_handler => handler)

Information

name	description
Web site	http://fluentd.org/
Documents	http://docs.fluentd.org/
Source repository	https://github.com/fluent/fluent-logger-ruby
Author	Sadayuki Furuhashi
Copyright	(c) 2011 FURUHASHI Sadayuki
License	Apache License, Version 2.0