Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nsq_to_http: 'http-client-request-timeout' flag has no effect #998

Closed
kamyanskiy opened this issue Feb 21, 2018 · 5 comments
Closed

nsq_to_http: 'http-client-request-timeout' flag has no effect #998

kamyanskiy opened this issue Feb 21, 2018 · 5 comments
Labels

Comments

@kamyanskiy
Copy link
Contributor

kamyanskiy commented Feb 21, 2018

I'm trying to use nsq_to_http to make POST request when message appears in NSQ to my Flask API. I've tried to increase "http-client-request-timeout" duration that defaults 20s up to 60 sec.
But seems it doesn't take effect.

  1. I added time.sleep(30) to POST request handler in my API, to check that nsq_to_http client waits more than 20s when it makes POST with message to my API.

  2. nsq_to_http starts with command line:

- /nsq_to_http --lookupd-http-address=nsqlookupd:4161 -channel ${CHANNEL:-"channel"} -topic ${TOPIC:-"topic"} -post ${API_URL:-"http://172.16.7.87:5000/ingoing"} -content-type application/json --consumer-opt=max_attempts,1000 --http-client-connect-timeout=60s --http-client-connect-timeout=30s

EXPECTED RESULT

nsq_to_http client timeout was increased upto 60 sec , so it should waits successfuly when api is not responded for 30 sec

ACTUAL RESULT

nsq_to_http HTTPPost client drops connection after 20 sec delay.
In logs I see:

nsq_to_http1_1  | 2018/02/21 15:35:27 INF    1 [topic/channel] (31a6b893668b:4150) connecting to nsqd
...
nsq_to_http1_1  | 2018/02/21 15:35:47 ERR    1 [topic/channel] Handler returned error (Post http://172.16.7.87:5000/ingoing: net/http: request canceled (Client.Timeout exceeded while awaiting headers)) for msg 09506159c6b4c000
nsq_to_http1_1  | 2018/02/21 15:35:47 WRN    1 [topic/channel] backing off for 2s (backoff level 1), setting all to RDY 0
nsq_to_http1_1  | 2018/02/21 15:35:49 WRN    1 [topic/channel] (31a6b893668b:4150) backoff timeout expired, sending RDY 1
@ploxiln
Copy link
Member

ploxiln commented Feb 21, 2018

I think this is because the flag value is used in an init() function which runs before the flags are parsed:

func init() {
	httpclient = &http.Client{Transport: http_api.NewDeadlineTransport(*httpConnectTimeout, *httpRequestTimeout), Timeout: *httpRequestTimeout}
	userAgent = fmt.Sprintf("nsq_to_http v%s", version.Binary)
}

@mreiferson
Copy link
Member

ahhh, good catch!

@mreiferson mreiferson added the bug label Feb 21, 2018
@mreiferson mreiferson changed the title nsq_to_http 'http-client-request-timeout' flag seems no effects nsq_to_http: 'http-client-request-timeout' flag has no effect Feb 21, 2018
@kamyanskiy
Copy link
Contributor Author

@ploxiln thank you for answer, happy to bring something to NSQ. I'm newbie in Go language and, am I correct understand - that in Go init() starts after variable initialization but before main() function. Config flags are parsed in main() function here ?

@ploxiln
Copy link
Member

ploxiln commented Feb 22, 2018

yes, you understand correctly

@mreiferson
Copy link
Member

fixed in #1000

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants