Make build-image use local environment
#553
Comments
XAMPPRocky
added
kind/cleanup
|
The removal of the debug image building should help with this as well (once #552 is merged, we can tackle that) - but sounds like you are getting dinged by the mac + docker tax (which also sounds like it's worse on a M1? if you are emulating to a x86 architecture.) It doesn't hit me as much running Docker on Linux (I feel like it doesn't take me nearly as long to build a Docker image on my machine - I should time it though). Another option from a whole new set of targets is to have dynamic commands that get changed based on a build flag. For example, in Agones (admittedly I haven't tested this in a while), we have a ifdef LOCAL_GO
# use local Go tooling, which greatly speeds up incremental rebuilds, in particular on macOS
# Note that agones.dev must be in GOPATH
GO_BUILD_LINUX_AMD64=CGO_ENABLED=0 GOOS=linux GOARCH=amd64 $(gomod_on) $(go_build_cmd)
GO_BUILD_WINDOWS_AMD64=GOOS=windows GOARCH=amd64 $(gomod_on) $(go_build_cmd)
GO_BUILD_DARWIN_AMD64=GOOS=darwin GOARCH=amd64 $(gomod_on) $(go_build_cmd)
GO_TEST=$(go_test_cmd) -v
GO_E2E_TEST_ARGS=
PERF_OUTPUT_DIR=$(build_path)/.perf
go_build_base_path=$(agones_path)
endifThis can be useful as the dynamic commands can traverse multiple targets. |
FWIW that time measurement was on a x86 Windows 10 machine. For reasons when I'm making deployments it's easier to do it on my Windows machine. Another thing that should be changed is that |
I think we don't need the debug image anymore and it can be removed entirely - we discussed this a bit in #541 but the debug version was there primarily to enable debug logging since the previous logging library looked at build level for log levels. Since we don't have that as a thing anymore, I think the debug image (and binary?) can be dropped entirely. |
Several improvements to workflows that flow through the build-image to improve build and iteration times. This includes: We don't need this anymore, so no reason to build it for each architecture. I realised that when mixing the `build-image` workflow with local rust tools, that going between them, they would invalidate each other's incremental cache, since it was fingerprinted on absolute directories - drastically expanding build times between iterations. This drastically improves both the incremental build times of images, and every other workflow through `build-image`, such as e2e testing. (`make test-agones` can be used iteratively now). We only use a few files for the Docker image, so let's ignore everything else. Took the docker context down from 1GB+ to < 50MB. On a linux machine all this brings incremental builds down from 10's of minutes to a minute or less, since only what has changed is compiled and pushed as we can now take advantage of rust incremental build and docker image caching. Work on googleforgames#553
Several improvements to workflows that flow through the build-image to improve build and iteration times. This includes: We don't need this anymore, so no reason to build it for each architecture. I realised that when mixing the `build-image` workflow with local rust tools, that going between them, they would invalidate each other's incremental cache, since it was fingerprinted on absolute directories - drastically expanding build times between iterations. This drastically improves both the incremental build times of images, and every other workflow through `build-image`, such as e2e testing. (`make test-agones` can be used iteratively now). We only use a few files for the Docker image, so let's ignore everything else. Took the docker context down from 1GB+ to < 50MB. On a linux machine all this brings incremental builds down from 10's of minutes to a minute or less, since only what has changed is compiled and pushed as we can now take advantage of rust incremental build and docker image caching. Work on googleforgames#553
* Improve build-image incremental workflow Several improvements to workflows that flow through the build-image to improve build and iteration times. This includes: We don't need this anymore, so no reason to build it for each architecture. I realised that when mixing the `build-image` workflow with local rust tools, that going between them, they would invalidate each other's incremental cache, since it was fingerprinted on absolute directories - drastically expanding build times between iterations. This drastically improves both the incremental build times of images, and every other workflow through `build-image`, such as e2e testing. (`make test-agones` can be used iteratively now). We only use a few files for the Docker image, so let's ignore everything else. Took the docker context down from 1GB+ to < 50MB. On a linux machine all this brings incremental builds down from 10's of minutes to a minute or less, since only what has changed is compiled and pushed as we can now take advantage of rust incremental build and docker image caching. Work on #553 * Fix issue with site generation.
Sometimes it's just faster and easier to build locally and create images and binaries that way, rather than use the build image. This could potentially be extended to running tests as well, but limited it to building artifacts for now. Work on googleforgames#553
Sometimes it's just faster and easier to build locally and create images and binaries that way, rather than use the build image. This could potentially be extended to running tests as well, but limited it to building artifacts for now. Work on googleforgames#553
Sometimes it's just faster and easier to build locally and create images and binaries that way, rather than use the build image. This could potentially be extended to running tests as well, but limited it to building artifacts for now. Work on #553 Co-authored-by: XAMPPRocky <4464295+XAMPPRocky@users.noreply.github.com>
* Improve build-image incremental workflow Several improvements to workflows that flow through the build-image to improve build and iteration times. This includes: We don't need this anymore, so no reason to build it for each architecture. I realised that when mixing the `build-image` workflow with local rust tools, that going between them, they would invalidate each other's incremental cache, since it was fingerprinted on absolute directories - drastically expanding build times between iterations. This drastically improves both the incremental build times of images, and every other workflow through `build-image`, such as e2e testing. (`make test-agones` can be used iteratively now). We only use a few files for the Docker image, so let's ignore everything else. Took the docker context down from 1GB+ to < 50MB. On a linux machine all this brings incremental builds down from 10's of minutes to a minute or less, since only what has changed is compiled and pushed as we can now take advantage of rust incremental build and docker image caching. Work on #553 * Fix issue with site generation.
Sometimes it's just faster and easier to build locally and create images and binaries that way, rather than use the build image. This could potentially be extended to running tests as well, but limited it to building artifacts for now. Work on #553 Co-authored-by: XAMPPRocky <4464295+XAMPPRocky@users.noreply.github.com>
Currently make's
build-imagesteps does everything required to produce a full release image for Quilkin. This process can be quite long for development work (being optimistic it can be like 20–30 minutes on a i9-9900K with 64GB of RAM, and fresh runs are much longer). I think we should splitbuild-imageintobuild-imageandbuild-release-image.build-imagewould only build the Quilkin executable and use the local environment instead of aquilkin-buildimage to reduce overhead of running through docker, andbuild-release-imagewould behave likebuild-imagecurrently does.The text was updated successfully, but these errors were encountered: