Skip to content

Commit

Permalink
Increase dockerd client timeout from 30s to 60s. (#7363)
Browse files Browse the repository at this point in the history
When edged sends an image pull request to dockerd, it expects dockerd to respond with the HTTP response status code and headers within 30s. The image pull itself does not need to complete within those 30s, just the initial start of the HTTP response, after which dockerd streams JSON events in the response to indicate the pull progress.

We have one customer whose device is unable to pull module images from their remote container registry. This is because dockerd needs to check whether the image exists on the ACR before it sends the HTTP response status code, and this check itself takes ~45s or so. When edged times out and abandons the request after 30s, dockerd has different behavior depending on whether the image tag is `latest` or not:

- If the image tag is `latest`, dockerd continues to pull the image in the background even after edged cancels the request. So one would think that when the image is pulled eventually, the next request to pull the image would succeed immediately. However the semantics of the `latest` tag are that dockerd must still check the ACR to see if there's a newer image available, and this also ends up taking ~45s, so the net effect is that edged continues to time out the request.

- If the image tag is not `latest`, dockerd also stops pulling the image after edged cancels the request, instead of continuing to pull it in the background. So every attempt to pull the image is canceled and the image never gets pulled to the device.

So this change increases the timeout of requests to dockerd to 60s.

***Please replace this line with your PR description and read PR checklist below***

## Azure IoT Edge PR checklist:
  • Loading branch information
arsing authored Aug 28, 2024
1 parent 15dc810 commit e7abc49
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion edgelet/docker-rs/src/apis/client.rs
Original file line number Diff line number Diff line change
Expand Up @@ -249,7 +249,7 @@ macro_rules! api_call {
let request = api_call!(@inner build_request builder $(body $btype)?)?;

let response = ::tokio::time::timeout(
::std::time::Duration::from_secs(30),
::std::time::Duration::from_secs(60),
self.client.request(request)
)
.await??;
Expand Down

0 comments on commit e7abc49

Please sign in to comment.