Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

JDK11 support: FetchHTTPTest: ssl handshake_failure #268

Closed
ato opened this issue Aug 1, 2019 · 2 comments
Closed

JDK11 support: FetchHTTPTest: ssl handshake_failure #268

ato opened this issue Aug 1, 2019 · 2 comments

Comments

@ato
Copy link
Collaborator

ato commented Aug 1, 2019 

Tests run: 30, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 12.543 sec <<< FAILURE!
testSslTrustLevel(org.archive.modules.fetcher.FetchHTTPTests)  Time elapsed: 0.146 sec  <<< FAILURE!
junit.framework.AssertionFailedError
        at junit.framework.Assert.fail(Assert.java:47)
        at junit.framework.Assert.assertTrue(Assert.java:20)
        at junit.framework.Assert.assertTrue(Assert.java:27)
        at org.archive.modules.fetcher.FetchHTTPTests.runDefaultChecks(FetchHTTPTests.java:115)
        at org.archive.modules.fetcher.FetchHTTPTests.testSslTrustLevel(FetchHTTPTests.java:661)

The assertion is failing because the recorded request is from a previous test case. The actual fetch attempt is failing with an SSL handshake failure. Uncertain yet as to whether the problem is the client or the server (the embedded jetty which the tests run against).

javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure
	at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:131)
	at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:117)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:308)
	at java.base/sun.security.ssl.Alert$AlertConsumer.consume(Alert.java:285)
	at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:181)
	at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:164)
	at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1152)
	at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1063)
	at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:402)
	at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:290)
	at org.archive.modules.fetcher.FetchHTTPRequest$2.createLayeredSocket(FetchHTTPRequest.java:568)
	at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:259)
	at org.apache.http.impl.conn.HttpClientConnectionOperator.connect(HttpClientConnectionOperator.java:125)
	at org.apache.http.impl.conn.BasicHttpClientConnectionManager.connect(BasicHttpClientConnectionManager.java:318)
	at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:363)
	at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:219)
	at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:195)
	at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:86)
	at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:72)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:57)
	at org.archive.modules.fetcher.FetchHTTPRequest.execute(FetchHTTPRequest.java:745)
	at org.archive.modules.fetcher.FetchHTTP.innerProcess(FetchHTTP.java:658)
	at org.archive.modules.Processor.innerProcessResult(Processor.java:175)
	at org.archive.modules.Processor.process(Processor.java:142)
	at org.archive.modules.fetcher.FetchHTTPTests.testSslTrustLevel(FetchHTTPTests.java:664)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at junit.framework.TestCase.runTest(TestCase.java:164)
	at org.archive.state.ModuleTestBase.runTest(ModuleTestBase.java:226)
	at junit.framework.TestCase.runBare(TestCase.java:130)
	at junit.framework.TestResult$1.protect(TestResult.java:106)
	at junit.framework.TestResult.runProtected(TestResult.java:124)
	at junit.framework.TestResult.run(TestResult.java:109)
	at junit.framework.TestCase.run(TestCase.java:120)
	at junit.framework.TestSuite.runTest(TestSuite.java:230)
	at junit.framework.TestSuite.run(TestSuite.java:225)
@ato
Copy link
Collaborator Author

ato commented Aug 2, 2019

Looks like the server is not sending a certificate:

$ openssl s_client -connect localhost:7443 -showcerts
CONNECTED(00000003)
Can't use SSL_get_servername
139724177692480:error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure:ssl/record/rec_layer_s3.c:1536:SSL alert number 40
---
no peer certificate available
---
No client certificate CA names sent
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 470 bytes and written 750 bytes
Verification: OK
---
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---

failing server side with:

  javax.net.ssl.SSLHandshakeException: No available authentication scheme
  	at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:131)
  	at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:117)
  	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:308)
  	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:264)
  	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:255)
  	at java.base/sun.security.ssl.CertificateMessage$T13CertificateProducer.onProduceCertificate(CertificateMessage.java:945)
  	at java.base/sun.security.ssl.CertificateMessage$T13CertificateProducer.produce(CertificateMessage.java:934)
  	at java.base/sun.security.ssl.SSLHandshake.produce(SSLHandshake.java:436)
  	at java.base/sun.security.ssl.ClientHello$T13ClientHelloConsumer.goServerHello(ClientHello.java:1224)
  	at java.base/sun.security.ssl.ClientHello$T13ClientHelloConsumer.consume(ClientHello.java:1160)
  	at java.base/sun.security.ssl.ClientHello$ClientHelloConsumer.onClientHello(ClientHello.java:849)
  	at java.base/sun.security.ssl.ClientHello$ClientHelloConsumer.consume(ClientHello.java:810)
  	at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:392)
  	at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:448)
  	at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:425)
  	at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:178)
  	at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:164)
  	at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1152)
  	at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1063)
  	at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:402)
  	at org.mortbay.jetty.security.SslSocketConnector$SslConnection.run(SslSocketConnector.java:708)
  	at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)}

There is a JDK bug mentioning that this occurs with TLSv1.3 when only a DSA certificate is in the keystore. However it looks like we're using an RSA certificate. Applying the suggested workaround of downgrading to TLSv1.2 results in a different exception but seemingly also due to no the server having no certificate.

 javax.net.ssl.SSLHandshakeException: no cipher suites in common

@ato
Copy link
Collaborator Author

ato commented Aug 2, 2019

According to the release announcement:

Notably, Jetty 9.4.12 includes compatibility for JDK 11. Additionally, TLS 1.3 support has been implemented.

I tested upgrading the unit tests to 9.4.12 and it does indeed appear to solve this issue.

ato added a commit that referenced this issue Aug 5, 2019
@ato
jdk11 support: upgrade jetty to 9.4.19 in modules tests #268
9450c2e 
Fixes the ssl handshake_failure. Support for jdk11 was added in jetty
9.4.14 but we may as well bump to the latest stable version.

In jetty 9 it appears the request is logged after the response is sent.
Thus it was racing with the assertions that check the client IP. So to
fix this rather stashing the 'lastRequest' we just make the server
echo the client's IP in a response header.

We also have to disable some of the check for testLaxUrlEncoding() as
jetty 9 now rejects such requests with a 400 error. The actual request
line encoding is still covered though which is the important thing.
@ato ato mentioned this issue Aug 5, 2019
Merged
ato added a commit that referenced this issue Aug 5, 2019
@ato
JDK11 support: upgrade jetty to 9.4.19 in modules tests #268
3b90e75 
Fixes the ssl handshake_failure. Support for jdk11 was added in jetty
9.4.14 but we may as well bump to the latest stable version.

In Jetty 9 it appears the request is logged after the response is sent.
Thus it was racing with the assertions that check the client IP. So to
fix this rather stashing the 'lastRequest' we just make the server
echo the client's IP in a response header.

Some other minor tweaks were needed due to changes in Jetty behaviour:

- We stop checking the length of the raw response. It doesn't tell us
  anything and easily varies.

- Jetty now generates Set-Cookie with a space after the ;

- Jetty now lowercases the word "basic" in WWW-Authenticate header

- testLaxUrlEncoding(): Jetty now rejects bad paths with a 400 error so
  we disable the response checks. The actual request line is
  still checked which is the important thing.
ato added a commit that referenced this issue Aug 5, 2019
@ato
JDK11 support: upgrade jetty to 9.4.19 in modules tests #268
af5cee0 
Fixes the ssl handshake_failure. Support for jdk11 was added in jetty
9.4.14 but we may as well bump to the latest stable version.

In Jetty 9 it appears the request is logged after the response is sent.
Thus it was racing with the assertions that check the client IP. So to
fix this rather stashing the 'lastRequest' we just make the server
echo the client's IP in a response header.

Some other minor tweaks were needed due to changes in Jetty behaviour:

- We stop checking the length of the raw response. It doesn't tell us
  anything and easily varies.

- Jetty now generates Set-Cookie with a space after the ;

- Jetty now lowercases the word "basic" in WWW-Authenticate header

- testLaxUrlEncoding(): Jetty now rejects bad paths with a 400 error so
  we disable the response checks. The actual request line is
  still checked which is the important thing.
@nlevitt nlevitt closed this as completed in 9d1341c Aug 7, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@ato