Use multistage build to bring in pbf executable
#263
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
orangejulius
changed the title
pbf executablepbf executable
orangejulius
force-pushed
the
use-multistage-build
branch
from
1e3d8c4
to
6916006
Compare
|
Yeah very nice |
|
Some quick notes:
|
The polylines Docker image is a bit of a large one currently, as it includes not just Node.js and a `node_modules` directory, but a full compiler toolchain, an install of the Go language, the dependencies of the `pbf` repository from https://github.com/missinglink/pbf, and the final `pbf` executable that comes from it. All told, this brought the total image size to a whopping 950MB uncompressed. This PR makes use of multi stage builds to run the compiling of the `pbf` executable in a separate container. After this, all the toolchain and dependencies needed can be thrown away, and only the small executable copied to the final image. Using `container-diff` it looks like the image size, uncompressed, after pelias/docker-baseimage#23 as well, will be only 322MB. That's a nice 600MB savings! Before pelias/docker-baseimage#23 the image size still drops to 500MB, still a healthy reduction. Replaces #262
orangejulius
force-pushed
the
use-multistage-build
branch
from
6916006
to
017ee51
Compare
orangejulius
added a commit
to pelias/libpostal-service
that referenced
this pull request
Nov 2, 2021
…encies After pelias/docker-baseimage#23, we will no longer have a compiler toolchain in our Docker baseimage. However, due to the way Docker images work and build upon each other, the biggest wins come from ensuring we don't have a compiler toolchain _anywhere_ in our images. If you think about it, even a single image having a compiler toolchain is the same as the baseimage having it, at least when comparing the total size of all our images. Thankfully, with multistage builds we can easily remove both the C++ compiler toolchain and Golang buildtime dependencies in the libpostal service, similar to pelias/polylines#263. This alone drops the total image size for the libpostal-service from 3.2GB to 2.8GB. Further improvements are possible in the libpostal baseimage.
orangejulius
added a commit
to pelias/libpostal-service
that referenced
this pull request
Nov 2, 2021
…encies After pelias/docker-baseimage#23, we will no longer have a compiler toolchain in our Docker baseimage. However, due to the way Docker images work and build upon each other, the biggest wins come from ensuring we don't have a compiler toolchain _anywhere_ in our images. If you think about it, even a single image having a compiler toolchain is the same as the baseimage having it, at least when comparing the total size of all our images. Thankfully, with multistage builds we can easily remove both the C++ compiler toolchain and Golang buildtime dependencies in the libpostal service, similar to pelias/polylines#263. This alone drops the total image size for the libpostal-service from 3.2GB to 2.8GB. Further improvements are possible in the libpostal baseimage.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The polylines Docker image is a bit of a large one currently, as it includes not just Node.js and a
node_modulesdirectory, but a full compiler toolchain, an install of the Go language, the dependencies of thepbfrepository from https://github.com/missinglink/pbf, and the finalpbfexecutable that comes from it.All told, this brought the total image size to a whopping 950MB uncompressed.
This PR makes use of multi stage builds to run the compiling of the
pbfexecutable in a separate container. After this, all the toolchain and dependencies needed can be thrown away, and only the small executable copied to the final image.Using
container-diffit looks like the image size, uncompressed, after pelias/docker-baseimage#23 as well, will be only 322MB. That's a nice 600MB savings!Before pelias/docker-baseimage#23 the image size still drops to 500MB, also a healthy reduction.
Replaces #262