Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable jib.httpTimeout system property for HTTP connection and read timeout #656

Merged
merged 9 commits into from
Jul 19, 2018
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@

import com.google.api.client.http.GenericUrl;
import com.google.api.client.http.HttpMethods;
import com.google.api.client.http.HttpRequest;
import com.google.api.client.http.HttpRequestFactory;
import com.google.api.client.http.HttpResponse;
import com.google.api.client.http.apache.ApacheHttpTransport;
Expand Down Expand Up @@ -117,11 +118,16 @@ public Response put(Request request) throws IOException {
* @throws IOException if building the HTTP request fails.
*/
public Response send(String httpMethod, Request request) throws IOException {
httpResponse =
HttpRequest httpRequest =
requestFactory
.buildRequest(httpMethod, url, request.getHttpContent())
.setHeaders(request.getHeaders())
.execute();
.setHeaders(request.getHeaders());
if (request.getHttpTimeout() != null) {
httpRequest.setConnectTimeout(request.getHttpTimeout());
httpRequest.setReadTimeout(request.getHttpTimeout());
}

httpResponse = httpRequest.execute();
return new Response(httpResponse);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -27,12 +27,16 @@ public class Request {
private final HttpHeaders headers;

/** The HTTP request body. */
@Nullable private BlobHttpContent body;
@Nullable private final BlobHttpContent body;

/** HTTP connection and read timeout. */
@Nullable private final Integer httpTimeout;

public static class Builder {

private final HttpHeaders headers = new HttpHeaders().setAccept("*/*");
@Nullable private BlobHttpContent body;
@Nullable private Integer httpTimeout;

public Request build() {
return new Request(this);
Expand Down Expand Up @@ -73,6 +77,17 @@ public Builder setUserAgent(String userAgent) {
return this;
}

/**
* Sets the HTTP connection and read timeout in milliseconds.
*
* @param httpTimeout timeout in milliseconds
* @return this
*/
public Builder setHttpTimeout(@Nullable Integer httpTimeout) {
this.httpTimeout = httpTimeout;
return this;
}

/**
* Sets the body and its corresponding {@code Content-Type} header.
*
Expand All @@ -92,6 +107,7 @@ public static Builder builder() {
private Request(Builder builder) {
this.headers = builder.headers;
this.body = builder.body;
this.httpTimeout = builder.httpTimeout;
}

HttpHeaders getHeaders() {
Expand All @@ -102,4 +118,9 @@ HttpHeaders getHeaders() {
BlobHttpContent getHttpContent() {
return body;
}

@Nullable
Integer getHttpTimeout() {
return httpTimeout;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,15 @@ public static Initializer initializer(String serverUrl, String repository) {
return new Initializer(serverUrl, repository);
}

@Nullable
private static Integer getHttpTimeout() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is using null a common way to do this? If not, I was thinking maybe use negative to indicate absence?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's not uncommon to use null to indicate a field is unset or uninitialized, or something is absent. Using a negative value works too. However, it won't be distinguishable from the user wrongly giving a negative value. That will matter if we error out on a negative timeout value as you said.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay sounds good 👍 let's add a javadoc explicitly mentioning that null means the caller should use some default.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, you weren't talking about a field here. I see what you meant. I'll think about the API-aspect of this method.

Integer httpTimeout = Integer.getInteger("jib.httpTimeout");
if (httpTimeout != null && httpTimeout >= 0) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could do just return Integer.getInteger("jib.httpTimeout", null) since in the negative case, we probably want to error (thrown by HttpRequest#setConnectTimeout)

Copy link
Member Author

@chanseokoh chanseokoh Jul 18, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking showing a warning with a negative value, instead of errorring out. Do you think it's not good?

Anyways, if we decide to error out, I'll probably detect it and fail at the frontends with better error messages, rather than just displaying up an unfriendly stack trace from Preconditions.checkArgument() with no message.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think either way is good. I lean towards the errorring since the case I could think of where negative could show up is if the user may have meant to type another timeout and accidentally put a minus sign in front - in which case, a warning may be lost in the logs to a different error when the connection fails.

Copy link
Member Author

@chanseokoh chanseokoh Jul 18, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. I think then the same argument applies to non-parsable string values. The user could accidentally put any non-digit characters, so it also makes sense to error out on non-parsable strings. Going for errorring on both cases?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, I think in the non-parsable version, this already errors, since Integer#getInteger would throw a NumberFormatException.

Copy link
Member Author

@chanseokoh chanseokoh Jul 18, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, errorring on both cases. (BTW, Integer.getInteger() doesn't throw NumberFormatException. parseInt() does, OTOH.)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah right, looks like the getInteger(String nm, Integer val) one catches and ignores the NumberFormatException

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will update the code tomorrow, but I'm thinking of doing

      Integer httpTimeout = null; // implies using the default timeout
      String timeoutValue = System.getProperty("jib.httpTimeout");
      if (timeoutValue != null) {
        httpTimeout = Integer.parseInt(timeoutValue);
        if (httpTimeout < 0) {
          throw new RegistryAuthenticationFailedException("Negative value of jib.httpTimeout");
        }
      }
      ...
      try (Connection connection = ...) {
        Request.Builder requestBuilder = Request.builder().setHttpTimeout(httpTimeout);
      ...
    } catch (NumberFormatException ex) {
      throw new RegistryAuthenticationFailedException("Non-integer value of jib.httpTimeout");
    }

Since all versions of getInteger() don't throw NumberFormatException, I guess I need to call parseInt() after System.getProperty().

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@coollog and I agreed that we make the code simple and not care about the edge cases too much here. If something goes awry, so be it. Instead, we will guard the errors at the plugin frontends.

return httpTimeout;
}
return null;
}

// TODO: Replace with a WWW-Authenticate header parser.
/**
* Instantiates from parsing a {@code WWW-Authenticate} header.
Expand Down Expand Up @@ -239,7 +248,7 @@ private Authorization authenticate(String scope) throws RegistryAuthenticationFa
URL authenticationUrl = getAuthenticationUrl(scope);

try (Connection connection = new Connection(authenticationUrl)) {
Request.Builder requestBuilder = Request.builder();
Request.Builder requestBuilder = Request.builder().setHttpTimeout(getHttpTimeout());
if (authorization != null) {
requestBuilder.setAuthorization(authorization);
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,15 @@ static class RequestState {
}
}

@Nullable
private static Integer getHttpTimeout() {
Integer httpTimeout = Integer.getInteger("jib.httpTimeout");
if (httpTimeout != null && httpTimeout >= 0) {
return httpTimeout;
}
return null;
}

/** Makes a {@link Connection} to the specified {@link URL}. */
private final Function<URL, Connection> connectionFactory;

Expand Down Expand Up @@ -162,6 +171,7 @@ T call(RequestState requestState) throws IOException, RegistryException {
Request.Builder requestBuilder =
Request.builder()
.setUserAgent(userAgent)
.setHttpTimeout(getHttpTimeout())
.setAccept(registryEndpointProvider.getAccept())
.setBody(registryEndpointProvider.getContent());
// Only sends authorization if using HTTPS.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,6 @@
import java.nio.charset.StandardCharsets;
import java.util.Arrays;
import org.junit.Assert;
import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.mockito.ArgumentCaptor;
Expand All @@ -55,47 +54,76 @@ public class ConnectionTest {

@InjectMocks private final Connection testConnection = new Connection(fakeUrl.toURL());

@Before
public void setUpMocksAndFakes() throws IOException {
fakeRequest =
Request.builder()
.setAccept(Arrays.asList("fake.accept", "another.fake.accept"))
.setUserAgent("fake user agent")
.setBody(new BlobHttpContent(Blobs.from("crepecake"), "fake.content.type"))
.setAuthorization(Authorizations.withBasicCredentials("fake-username", "fake-secret"))
.build();

Mockito.when(
mockHttpRequestFactory.buildRequest(
Mockito.any(String.class), Mockito.eq(fakeUrl), Mockito.any(BlobHttpContent.class)))
.thenReturn(mockHttpRequest);

Mockito.when(mockHttpRequest.setHeaders(Mockito.any(HttpHeaders.class)))
.thenReturn(mockHttpRequest);
Mockito.when(mockHttpRequest.execute()).thenReturn(mockHttpResponse);
}

@Test
public void testGet() throws IOException {
setUpMocksAndFakes(null);
testSend(HttpMethods.GET, Connection::get);
}

@Test
public void testPost() throws IOException {
setUpMocksAndFakes(null);
testSend(HttpMethods.POST, Connection::post);
}

@Test
public void testPut() throws IOException {
setUpMocksAndFakes(null);
testSend(HttpMethods.PUT, Connection::put);
}

@Test
public void testHttpTimeout_doNotSetByDefault() throws IOException {
setUpMocksAndFakes(null);
try (Connection connection = testConnection) {
connection.send(HttpMethods.GET, fakeRequest);
}

Mockito.verify(mockHttpRequest, Mockito.never()).setConnectTimeout(Mockito.anyInt());
Mockito.verify(mockHttpRequest, Mockito.never()).setReadTimeout(Mockito.anyInt());
}

@Test
public void testHttpTimeout() throws IOException {
setUpMocksAndFakes(5982);
try (Connection connection = testConnection) {
connection.send(HttpMethods.GET, fakeRequest);
}

Mockito.verify(mockHttpRequest).setConnectTimeout(5982);
Mockito.verify(mockHttpRequest).setReadTimeout(5982);
}

@FunctionalInterface
private interface SendFunction {

Response send(Connection connection, Request request) throws IOException;
}

private void setUpMocksAndFakes(Integer httpTimeout) throws IOException {
fakeRequest =
Request.builder()
.setAccept(Arrays.asList("fake.accept", "another.fake.accept"))
.setUserAgent("fake user agent")
.setBody(new BlobHttpContent(Blobs.from("crepecake"), "fake.content.type"))
.setAuthorization(Authorizations.withBasicCredentials("fake-username", "fake-secret"))
.setHttpTimeout(httpTimeout)
.build();

Mockito.when(
mockHttpRequestFactory.buildRequest(
Mockito.any(String.class), Mockito.eq(fakeUrl), Mockito.any(BlobHttpContent.class)))
.thenReturn(mockHttpRequest);

Mockito.when(mockHttpRequest.setHeaders(Mockito.any(HttpHeaders.class)))
.thenReturn(mockHttpRequest);
if (httpTimeout != null) {
Mockito.when(mockHttpRequest.setConnectTimeout(Mockito.anyInt())).thenReturn(mockHttpRequest);
Mockito.when(mockHttpRequest.setReadTimeout(Mockito.anyInt())).thenReturn(mockHttpRequest);
}
Mockito.when(mockHttpRequest.execute()).thenReturn(mockHttpResponse);
}

private void testSend(String httpMethod, SendFunction sendFunction) throws IOException {
try (Connection connection = testConnection) {
sendFunction.send(connection, fakeRequest);
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
/*
* Copyright 2018 Google LLC. All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License"); you may not
* use this file except in compliance with the License. You may obtain a copy of
* the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations under
* the License.
*/

package com.google.cloud.tools.jib.http;

import com.google.api.client.http.GenericUrl;
import java.io.IOException;
import java.util.function.BiFunction;

/**
* Mock {@link Connection} used for testing. Normally, you would use {@link
* org.mockito.Mockito#mock}; this class is intended to examine the {@link Request) object by
* calling its non-public package-protected methods.
*/
public class MockConnection extends Connection {

private BiFunction<String, Request, Response> responseSupplier;
private Integer httpTimeout;

public MockConnection(BiFunction<String, Request, Response> responseSupplier) {
super(new GenericUrl("ftp://non-exisiting.example.url.ever").toURL());
this.responseSupplier = responseSupplier;
}

@Override
public Response send(String httpMethod, Request request) throws IOException {
httpTimeout = request.getHttpTimeout();
return responseSupplier.apply(httpMethod, request);
}

public Integer getRequestedHttpTimeout() {
return httpTimeout;
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
/*
* Copyright 2018 Google LLC. All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License"); you may not
* use this file except in compliance with the License. You may obtain a copy of
* the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations under
* the License.
*/

package com.google.cloud.tools.jib.http;

import org.junit.Assert;
import org.junit.Test;

/** Tests for {@link Request}. */
public class RequestTest {

@Test
public void testGetHttpTimeout() {
Request request = Request.builder().build();

Assert.assertNull(request.getHttpTimeout());
}

@Test
public void testSetHttpTimeout() {
Request request = Request.builder().setHttpTimeout(3000).build();

Assert.assertEquals(Integer.valueOf(3000), request.getHttpTimeout());
}
}
Loading