Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use loggly bulk API #18

Closed
wants to merge 3 commits into from
Closed

Conversation

Vincent--
Copy link
Contributor

@Vincent-- Vincent-- commented Apr 15, 2016

See https://www.loggly.com/docs/http-bulk-endpoint/

With the bulk API and some tweaking on the number of output workers and the flush size, the speed increased by around 50 times on my setup.

@Vincent--
Copy link
Contributor Author

Next steps is to implement the multi_receive(events) to support logstash 2.2.x
See https://www.elastic.co/blog/logstash-2-2-0-and-2-1-2-released
and https://www.elastic.co/guide/en/logstash/current/_how_to_write_a_logstash_output_plugin.html

@torrancew
Copy link

+1 here, this is a huge design flaw with the current plugin. Connections are set up/torn down for each event in the current model. HTTPS and Proxies only exacerbate this.

colincoghill added a commit to koordinates/logstash-output-loggly that referenced this pull request Jul 6, 2016
@suyograo
Copy link
Contributor

suyograo commented Jan 8, 2017

@Vincent-- apologies for not moving on this PR. Can you please rebase? I can merge this then

@Vincent--
Copy link
Contributor Author

@suyograo, done 👍

@bangonkali
Copy link

Are there any plans to merge this?

@joshuabaird
Copy link

Is this ever going to be merged? This plugin is not usable for production in it's current state.

@webmat
Copy link
Contributor

webmat commented May 9, 2018

Sorry for the delay in getting to this, especially to @Vincent-- , who's been responsive in everything the team asked. This plugin doesn't have an active maintainer (see #20), so it's hard for us to dedicate much time to it.

I'll try to get this merged shortly.

@webmat webmat mentioned this pull request May 9, 2018
16 tasks
@webmat
Copy link
Contributor

webmat commented May 15, 2018

This has been merged and expanded upon. Published as v4.0.0.

@webmat webmat closed this May 15, 2018
@webmat
Copy link
Contributor

webmat commented May 15, 2018

Very late, but thanks @Vincent-- for your contribution!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants