OkHttpClient should allow alternative protocols

[johnshajiang]

It looks only SSL/TLS protocols are supported by OkHttpClient.
Could OkHttp allow other protocols?

Say, an alternative SSLContext implementation supports a non-TLS protocol and cipher suites.
This SSLContext implementation provides its own SSLSocketFactory.
Generally, that would be ready for working.

However, it looks OkHttpClient/ConnectionSpec requires the protocols must be in TlsVersion.
Could this restriction be removed?
For example, OkHttpClient just need to receive strings as protocols, but not need to convert them to TlsVersion instances.

[yschimke]

Could you provide an example?

Generally this isn't a good fit for OkHttp. It tries to be the simple ergonomic http client for typical use cases.

But look at this example for an approach to use Unix Domain Sockets https://github.com/square/okhttp/blob/f8fd4d08decf697013008b05ad7d2be10a648358/samples/unixdomainsockets/src/main/java/okhttp3/unixdomainsockets/ClientAndServer.java

[johnshajiang]

I also tried Apache HttpClient.
It worked fine with my protocol, because it didn't use its own protocol type, like TlsVersion.

SSLContext context = ...;  // My own SSLContext implementation
SSLConnectionSocketFactory socketFactory = new SSLConnectionSocketFactory(context);
CloseableHttpClient httpClient = HttpClients.custom().setSSLSocketFactory(socketFactory).build();

The codes look simple.

[johnshajiang]

The following function blocks me currently.

fun forJavaName(javaName: String): TlsVersion {
  return when (javaName) {
    "TLSv1.3" -> TLS_1_3
    "TLSv1.2" -> TLS_1_2
    "TLSv1.1" -> TLS_1_1
    "TLSv1" -> TLS_1_0
    "SSLv3" -> SSL_3_0
    else -> throw IllegalArgumentException("Unexpected TLS version: $javaName")
  }
}

Certainly, my protocol is not in the above list. So, the IllegalArgumentException must be thrown.

[yschimke]

You can just about get this working by passing in your sslSocketFactory as the socketFactory, and then OkHttp won't know anything about your SSL types.

You need a hacky workaround #6561, but it should work ok.

We don't expose the implementation details of Platform, and discussed it again recently with the same decision #7238

[johnshajiang]

I tried to wrap my SSLSocketFactory with DelegatingSocketFactory mentioned by #6561.
This time I didn't meet IllegalArgumentException.
But my test still failed with the below exception.

javax.net.ssl.SSLException: Unsupported or unrecognized SSL message
	at java.base/sun.security.ssl.SSLSocketInputRecord.handleUnknownRecord(SSLSocketInputRecord.java:451)
	at java.base/sun.security.ssl.SSLSocketInputRecord.decode(SSLSocketInputRecord.java:175)
	at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:111)
	at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1501)
	at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1411)
	at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:451)
	at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:422)
	at okhttp3.internal.connection.RealConnection.connectTls(RealConnection.kt:379)
	at okhttp3.internal.connection.RealConnection.establishProtocol(RealConnection.kt:337)
	at okhttp3.internal.connection.RealConnection.connect(RealConnection.kt:209)
	at okhttp3.internal.connection.ExchangeFinder.findConnection(ExchangeFinder.kt:226)
	at okhttp3.internal.connection.ExchangeFinder.findHealthyConnection(ExchangeFinder.kt:106)
	at okhttp3.internal.connection.ExchangeFinder.find(ExchangeFinder.kt:74)
	at okhttp3.internal.connection.RealCall.initExchange$okhttp(RealCall.kt:255)
	at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.kt:32)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
	at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.kt:95)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
	at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.kt:83)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
	at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.kt:76)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
	at okhttp3.internal.connection.RealCall.getResponseWithInterceptorChain$okhttp(RealCall.kt:201)
	at okhttp3.internal.connection.RealCall.execute(RealCall.kt:154)

It was really unexpected.
My own SSLSocketImpl, not sun.security.SSLSocketImp, should be used.

[yschimke]

In this case you would make a "http:" request, but truth be told it's all getting a bit hacky. So maybe this was a bad idea.

[johnshajiang]

My own SSLContext and SSLSocketFactory work fine with not only Apache HttpClient, but also Eclipse Jetty server and client.
I just be wondering why does OkHttp design that restriction on the protocols.

[swankjesse]

Maybe we could take a step back and ask what your goal is? Making OkHttp extensible is possible if the use case is compelling!

[johnshajiang]

I am developing an alternative JSSE provider, which has two goals:

• implements new cipher suites for TLS.
• implements a new protocol, which certainly has different name and ID.

OkHttp works fine for my first goal. Because it recognizes (doesn't block) TLS protocols, even though it doesn't known the new cipher suites.
For the second goal, the problem just was mentioned in the above comments.

[johnshajiang]

I just modified TlsVersion.kt [1] and added my protocol to the SSL/TLS version list as workaround.
It is working for my test.

[1] https://github.com/square/okhttp/blob/parent-4.9.3/okhttp/src/main/kotlin/okhttp3/TlsVersion.kt

[yschimke]

@johnshajiang I think we could give an internal SPI for Platform with an experimental annotation.

Basically it wouldn't be guaranteed for stability across versions. But would be enough to do what you need.

In truth it looks a lot like Platform.resetForTests and extending Platform yourself. https://github.com/square/okhttp/blob/master/okhttp/src/jvmMain/kotlin/okhttp3/internal/platform/Platform.kt#L188

[yschimke]

@johnshajiang curious if you looked at the above approach? If it's possible then it's probably the best we can do for this experimental use case, until we see more mainstream requirements to allow other protocols. I'll close this in a week or so, as we probably won't make this public API.

[johnshajiang]

Sorry, I (as a newbie to OkHttp) don't understand this solution.
Do you mean I can resolve my problem without any change on OkHttp3 (I'm using 4.9.3)?

Just took a look at okhttp3.internal.platform.Platform.
It could be extended, but how can I take my Platform to be applied?
Could you please show me some sample codes?

[yschimke]

It's in the link from the comment above Platform.resetForTests. But be aware as this isn't public api, it isn't stable, so you shouldn't rely on this not changing. It would be dangerous code to rely on for any library. But fine for experimentation.

[johnshajiang]

I just figured out how to set a custom Platform, named MyPlatform.
MyPlatform overrides newSSLContext and newSslSocketFactory.

Should I need to set a ConnectionSpec like the below?

Platform.Companion.resetForTests(new MyPlatform());
ConnectionSpec spec = new ConnectionSpec.Builder(ConnectionSpec.MODERN_TLS)
    .tlsVersions("MyProtocol")
    .cipherSuites("MyCipherSuite")
    .build();
OkHttpClient httpClient = new OkHttpClient.Builder()
    .connectionSpecs(Collections.singletonList(spec))
    .build();

If do that, the below error still raises,

java.lang.IllegalArgumentException: Unexpected TLS version: MyProtocol
    at okhttp3.TlsVersion$Companion.forJavaName(TlsVersion.kt:47)
    at okhttp3.ConnectionSpec.tlsVersions(ConnectionSpec.kt:75)
    at okhttp3.ConnectionSpec.apply$okhttp(ConnectionSpec.kt:96)
    at okhttp3.internal.connection.ConnectionSpecSelector.configureSecureSocket(ConnectionSpecSelector.kt:70)

If not set a ConnectionSpec, another exception is thrown,
java.net.UnknownServiceException: Unable to find acceptable protocols. ... ...
I suppose there is a default ConnectionSpec, which certainly doesn't know my protocol.

[yschimke]

You still won't be able to set a ConnectionSpec with the values you need. I'd leave them as default and then change the Platform methods to configure your socket.

That exception comes from

okhttp/okhttp/src/jvmMain/kotlin/okhttp3/internal/connection/ConnectPlan.kt

Line 437 in 8a08fb2

internal fun planWithCurrentOrInitialConnectionSpec(

when it can't find a match between your socket and the connectionspec.

Calling isCompatible here

okhttp/okhttp/src/jvmMain/kotlin/okhttp3/ConnectionSpec.kt

Line 146 in 3ad1912

fun isCompatible(socket: SSLSocket): Boolean {

But this is what you will have to try to work through the methods in Platform, find creative ways to move it forward. Including workarounds/kludges to make it accept your connection.

This points to why it's not trivial for us to make this a new public api/feature. And apologies in advance if this turns out to be a dead end, I was hoping to give you a way forward, but haven't confirmed this will work end to end myself.

[johnshajiang]

Thanks for your info!
I'll have a try after understanding OkHttp a bit more.

[yschimke]

Going to close this, as we don't currently plan to open up for alternative protocols. With a working example (a fork?) to review we could consider reopening. But not guaranteed either.