nsqadmin: doesn't handle nsqd --tls-required flag

[mreiferson]

From https://groups.google.com/d/msg/nsq-users/PkkPN9z7gPc/6EgTyHmuvg0J

nsqadmin cannot correctly negotiate TLS when nsqd in the cluster are configured with --tls-required.

Not sure yet how to fix this, some thoughts:

1. nsqadmin needs to be able to specify scheme in the --nsqd-http-address flag
2. nsqd needs to propagate TLS status to nsqlookupd, which needs to return this in its API responses
3. nsqadmin will pivot on (1) and (2) to negotiate which scheme to use

Thoughts @jehiah?

[durple]

Guys, are these screenshots indicative of this bug?
http://cl.ly/image/3Y3z3p1c2N0g
http://cl.ly/image/2A171u472S0v

If not, what am I doing wrong? And if so, are you looking to fix this anytime soon?

[mreiferson]

@durple yes, that looks like it.

I'd like to fix this after #323 lands

[durple]

Super! Thanks.

[jnicholls]

Was there a reason for waiting to fix this until after the nsqadmin refactoring, i.e. the refactoring would allow for something that wasn't possible in 0.3.x? I can't see a reason why nsqadmin couldn't have just attempted a TLS connection after receiving the 403 TLS_REQUIRED error.

In any case, it looks like #323 is coming along nicely! Which of the above (1, 2 or 3) will be the chosen solution? I would think number 1 and 2 are both required, for instances where nsqlookupd is and is not being used. But also, a simple non-TLS -> TLS "retry" logic would also work I think; nsqadmin would then remember what worked for that nsqd instance.

[mreiferson]

No particular reason, just trying to land that monster sometime this century! 😜

I thought about "negotiating" like you've suggested, but I think the problem is that nsqadmin doesn't know the HTTPS port to try 😐

[jnicholls]

No particular reason, just trying to land that monster sometime this century! 😜

Haha, fair enough.

I thought about "negotiating" like you've suggested, but I think the problem is that nsqadmin doesn't know the HTTPS port to try 😐

Oh ok, I figured it would be on the same port (4151) except it would be a socket that performs a TLS upgrade/handshake instead, i.e. it wouldn't necessarily be an https scheme'ed URL although that would make most http client libraries happier. But no worries, I appreciate the clarification!

[pctj101]

Perhaps there also needs to be some way to know that a certificate may be "self signed" and not to verify. This is useful in development environments (but not always desired in production).

[mreiferson]

I thought about "negotiating" like you've suggested, but I think the problem is that nsqadmin doesn't know the HTTPS port to try 😐

eureka, what if the HTTP 403 response included a payload that directed you towards the configured HTTPS port!!!!!!!!!!!

[mreiferson]

RFR @jehiah

This updates nsqd's 403 TLS response format to be JSON:

{
    "message": "TLS_REQUIRED",
    "https_port": 4152
}

It updates internal/http_api to retry HTTP requests that receive a 403 and exposes new nsqadmin flags to specify TLS related HTTPS client config.

[jehiah]

LGTM

[jnicholls]

When do you all expect to make another stable release, with this functionality included? Thanks in advance.

[mreiferson]

@jnicholls soon - we plan on addressing #421 and then stamping a stable release.

[jnicholls]

That's great news, thanks @mreiferson!