Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

A small cleanup turned into a sizeable one... #26

Closed
wants to merge 6 commits into from
Closed

A small cleanup turned into a sizeable one... #26

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
de0b22e
Clean up whitespace & tabs mess
May 9, 2018
c096243
Remove code comment that accidentally got into the public docs.
May 9, 2018
dda85ba
Fix typo in 'Event sent to Loggly'
May 9, 2018
b6ed8a5
Don't puts on every successful event sent to Loggly
May 9, 2018
0e2495e
Log a warning instead of puts, when retrying
May 9, 2018
abdcbd6
Replace outdated instructions wrt to finding the proper token.
May 9, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 4 additions & 9 deletions docs/index.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@ include::{include_path}/plugin_header.asciidoc[]

==== Description

Ugly monkey patch to get around http://jira.codehaus.org/browse/JRUBY-5529
Got a loggly account? Use logstash to ship logs to Loggly!

This is most useful so you can use logstash to parse and structure
Expand Down Expand Up @@ -82,13 +81,9 @@ https://www.loggly.com/docs/http-endpoint/
* Value type is <<string,string>>
* There is no default value for this setting.

The loggly http input key to send to.
This is usually visible in the Loggly 'Inputs' page as something like this:
....
https://logs-01.loggly.net/inputs/abcdef12-3456-7890-abcd-ef0123456789
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
\----------> key <-------------/
....
The loggly http customer token to use for sending.
You can find yours in "Source Setup", under "Customer Tokens".

You can use `%{foo}` field lookups here if you need to pull the api key from
the event. This is mainly aimed at multitenant hosting providers who want
to offer shipping a customer's logs to that customer's loggly account.
Expand Down Expand Up @@ -164,4 +159,4 @@ https://www.loggly.com/docs/source-groups/
[id="plugins-{type}s-{plugin}-common-options"]
include::{include_path}/{type}.asciidoc[]

:default_codec!:
:default_codec!:
84 changes: 40 additions & 44 deletions lib/logstash/outputs/loggly.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,13 +35,9 @@ class LogStash::Outputs::Loggly < LogStash::Outputs::Base
# https://www.loggly.com/docs/http-endpoint/
config :host, :validate => :string, :default => "logs-01.loggly.com"

# The loggly http input key to send to.
# This is usually visible in the Loggly 'Inputs' page as something like this:
# ....
# https://logs-01.loggly.net/inputs/abcdef12-3456-7890-abcd-ef0123456789
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
# \----------> key <-------------/
# ....
# The loggly http customer token to use for sending.
# You can find yours in "Source Setup", under "Customer Tokens".
#
# You can use `%{foo}` field lookups here if you need to pull the api key from
# the event. This is mainly aimed at multitenant hosting providers who want
# to offer shipping a customer's logs to that customer's loggly account.
Expand Down Expand Up @@ -116,7 +112,7 @@ def format_message(event)
private
def send_event(url, message)
url = URI.parse(url)
@logger.info("Loggly URL", :url => url)
@logger.debug("Loggly URL", :url => url)

http = Net::HTTP::Proxy(@proxy_host,
@proxy_port,
Expand All @@ -128,7 +124,7 @@ def send_event(url, message)
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
end

request = Net::HTTP::Post.new(url.path)
request = Net::HTTP::Post.new(url.path, {'Content-Type' =>'application/json'})
request.body = message

# Variable for count total retries
Expand All @@ -139,47 +135,47 @@ def send_event(url, message)
@retry_count = 1
end


@retry_count.times do
begin
response = http.request(request)
case response.code
# HTTP_SUCCESS :Code 2xx
when HTTP_SUCCESS
puts "Event send to Loggly"
# HTTP_FORBIDDEN :Code 403
when HTTP_FORBIDDEN
@logger.warn("User does not have privileges to execute the action.")
# HTTP_NOT_FOUND :Code 404
when HTTP_NOT_FOUND
@logger.warn("Invalid URL. Please check URL should be http://logs-01.loggly.com/inputs/CUSTOMER_TOKEN/tag/logstash")
# HTTP_INTERNAL_SERVER_ERROR :Code 500
when HTTP_INTERNAL_SERVER_ERROR
@logger.warn("Internal Server Error")
# HTTP_GATEWAY_TIMEOUT :Code 504
when HTTP_GATEWAY_TIMEOUT
@logger.warn("Gateway Time Out")
else
@logger.error("Unexpected response code", :code => response.code)
end # case

if [HTTP_SUCCESS,HTTP_FORBIDDEN,HTTP_NOT_FOUND].include?(response.code) # break the retries loop for the specified response code
break
end
rescue StandardError => e
begin
response = http.request(request)
case response.code

# HTTP_SUCCESS :Code 2xx
when HTTP_SUCCESS
@logger.debug("Event sent to Loggly")

# HTTP_FORBIDDEN :Code 403
when HTTP_FORBIDDEN
@logger.warn("User does not have privileges to execute the action.")

# HTTP_NOT_FOUND :Code 404
when HTTP_NOT_FOUND
@logger.warn("Invalid URL. Please check URL should be http://logs-01.loggly.com/inputs/CUSTOMER_TOKEN/tag/logstash")

# HTTP_INTERNAL_SERVER_ERROR :Code 500
when HTTP_INTERNAL_SERVER_ERROR
@logger.warn("Internal Server Error")

# HTTP_GATEWAY_TIMEOUT :Code 504
when HTTP_GATEWAY_TIMEOUT
@logger.warn("Gateway Time Out")
else
@logger.error("Unexpected response code", :code => response.code)
end # case

if [HTTP_SUCCESS,HTTP_FORBIDDEN,HTTP_NOT_FOUND].include?(response.code) # break the retries loop for the specified response code
break
end
rescue StandardError => e
@logger.error("An unexpected error occurred", :exception => e.class.name, :error => e.to_s, :backtrace => e.backtrace)
end # rescue

if totalRetries < @retry_count && totalRetries > 0
puts "Waiting for five seconds before retry..."
@logger.warn "Waiting for five seconds before retry..."
sleep(5)
end

totalRetries = totalRetries + 1
end #loop
end # def send_event
Expand Down