Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

change buildAndLoadNativePlatform to buildAndLoadSinglePlatform #1665

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/changelog.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
# ChangeLog
* **0.43-SNAPSHOT**:
- Make buildx build single non-native platform if requested ([1665](https://github.com/fabric8io/docker-maven-plugin/pull/1665)) @martyvona

* **0.42.1** (2023-04-06):
- Update Jnr Jffi dependency to v1.3.11 ([1660](https://github.com/fabric8io/docker-maven-plugin/issues/1660)) @rohanKanojia
Expand Down
26 changes: 17 additions & 9 deletions src/main/asciidoc/inc/build/_buildx.adoc
Original file line number Diff line number Diff line change
@@ -1,15 +1,23 @@

[[build-buildx]]

Buildx is enabled when there is a non-empty `<platform>` element inside the `<buildx>` configuration. Only the native platform
is built and saved in the local image cache by the `build` goal. All specified platforms are built and pushed into the remote
repository by the `push` goal. This behavior is to prevent non-native images from tainting the local image cache.
Buildx is enabled when there is a non-empty `<platform>` element inside the `<buildx>` configuration.

The local image cache cannot hold multi-architecture images nor can it have two platform specific images of the same name. The
recommended `<buildx>` configuration is to specify all supported platforms, including the native platform, in the `<platforms>`
element. This allows local integration testing of the build image from the local cache. During install or deploy phase, the
build machine will build and push all images to the registry. Any downstream consumers, regardless of native architecture, will
be able to use the multi-architecture image.
The local image cache cannot hold multi-architecture images nor can it have two platform specific images of the same name.
Thus the `build` goal will build and save a single-architecture image to the local image cache if possible:

* If the `<platform>` element contains a single platform, that image will be built.
* If the `<platform>` element contains more than one platform including the native platform, the native platform be used.
* If the `<platform>` element contains more than one platform not including the native platform, no image will be built.

These rules only apply to the image built and loaded into the local image cache with the `build` goal. They do not apply to the
`push` goal which will always build and push either a single-architecture or multi-architecture image with whatever platforms
are specified in the `<platform>` element.

The recommended `<buildx>` configuration is to specify all supported platforms, including the native platform, in the
`<platform>` element. This allows local integration testing of the build image from the local cache. During install or deploy
phase, the build machine will build and push a multi-architecture image containing all specified platforms to the registry.
Any downstream consumers, regardless of native architecture, will be able to use the multi-architecture image.

The `<buildx>` element within `<build>` defines how to build multi-architecture images.

Expand Down Expand Up @@ -75,4 +83,4 @@ You can now override the built platforms using a command line define:
[source,bash]
----
mvn clean deploy -Ddocker.platforms=linux/amd64,linux/arm64
----
----
30 changes: 16 additions & 14 deletions src/main/java/io/fabric8/maven/docker/service/BuildXService.java
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ public BuildXService(DockerAccess dockerAccess, DockerAssemblyManager dockerAsse
}

public void build(ProjectPaths projectPaths, ImageConfiguration imageConfig, String configuredRegistry, AuthConfig authConfig, File buildArchive) throws MojoExecutionException {
useBuilder(projectPaths, imageConfig, configuredRegistry, authConfig, buildArchive, this::buildAndLoadNativePlatform);
useBuilder(projectPaths, imageConfig, configuredRegistry, authConfig, buildArchive, this::buildAndLoadSinglePlatform);
}

public void push(ProjectPaths projectPaths, ImageConfiguration imageConfig, String configuredRegistry, AuthConfig authConfig) throws MojoExecutionException {
Expand All @@ -58,7 +58,7 @@ public void push(ProjectPaths projectPaths, ImageConfiguration imageConfig, Stri
useBuilder(projectPaths, imageConfig, configuredRegistry, authConfig, archive, this::pushMultiPlatform);
}

private <C> void useBuilder(ProjectPaths projectPaths, ImageConfiguration imageConfig, String configuredRegistry, AuthConfig authConfig, C context, Builder<C> builder) throws MojoExecutionException {
protected <C> void useBuilder(ProjectPaths projectPaths, ImageConfiguration imageConfig, String configuredRegistry, AuthConfig authConfig, C context, Builder<C> builder) throws MojoExecutionException {
BuildDirs buildDirs = new BuildDirs(projectPaths, imageConfig.getName());

Path configPath = getDockerStateDir(imageConfig.getBuildConfiguration(), buildDirs);
Expand All @@ -74,7 +74,7 @@ private <C> void useBuilder(ProjectPaths projectPaths, ImageConfiguration imageC
}
}

private void createConfigJson(Path configJson, AuthConfig authConfig) throws MojoExecutionException {
protected void createConfigJson(Path configJson, AuthConfig authConfig) throws MojoExecutionException {
try (BufferedWriter bufferedWriter = Files.newBufferedWriter(configJson, StandardCharsets.UTF_8,
StandardOpenOption.WRITE, StandardOpenOption.CREATE, StandardOpenOption.TRUNCATE_EXISTING)
) {
Expand All @@ -84,31 +84,33 @@ private void createConfigJson(Path configJson, AuthConfig authConfig) throws Moj
}
}

private void removeConfigJson(Path configJson) {
protected void removeConfigJson(Path configJson) {
try {
Files.deleteIfExists(configJson);
} catch (IOException e) {
logger.warn("Unable to delete %s", configJson);
}
}

private void buildAndLoadNativePlatform(List<String> buildX, String builderName, BuildDirs buildDirs, ImageConfiguration imageConfig, String configuredRegistry, File buildArchive) throws MojoExecutionException {
protected void buildAndLoadSinglePlatform(List<String> buildX, String builderName, BuildDirs buildDirs, ImageConfiguration imageConfig, String configuredRegistry, File buildArchive) throws MojoExecutionException {
List<String> platforms = imageConfig.getBuildConfiguration().getBuildX().getPlatforms();
// build and load the native image by re-building, image should be cached and build should be quick
// build and load the single-platform image by re-building, image should be cached and build should be quick
String nativePlatform = dockerAccess.getNativePlatform();
if (platforms.contains(nativePlatform)) {
if (platforms.size() == 1) {
buildX(buildX, builderName, buildDirs, imageConfig, configuredRegistry, platforms, buildArchive, "--load");
} else if (platforms.contains(nativePlatform)) {
buildX(buildX, builderName, buildDirs, imageConfig, configuredRegistry, Collections.singletonList(nativePlatform), buildArchive, "--load");
} else {
logger.info("Native platform not specified, no image built");
logger.info("More than one platform specified not including native %s, no image built", nativePlatform);
}
}

private void pushMultiPlatform(List<String> buildX, String builderName, BuildDirs buildDirs, ImageConfiguration imageConfig, String configuredRegistry, File buildArchive) throws MojoExecutionException {
protected void pushMultiPlatform(List<String> buildX, String builderName, BuildDirs buildDirs, ImageConfiguration imageConfig, String configuredRegistry, File buildArchive) throws MojoExecutionException {
// build and push all images. The native platform may be re-built, image should be cached and build should be quick
buildX(buildX, builderName, buildDirs, imageConfig, configuredRegistry, imageConfig.getBuildConfiguration().getBuildX().getPlatforms(), buildArchive, "--push");
}

private void buildX(List<String> buildX, String builderName, BuildDirs buildDirs, ImageConfiguration imageConfig, String configuredRegistry, List<String> platforms, File buildArchive, String extraParam)
protected void buildX(List<String> buildX, String builderName, BuildDirs buildDirs, ImageConfiguration imageConfig, String configuredRegistry, List<String> platforms, File buildArchive, String extraParam)
throws MojoExecutionException {

BuildImageConfiguration buildConfiguration = imageConfig.getBuildConfiguration();
Expand Down Expand Up @@ -158,7 +160,7 @@ private void buildX(List<String> buildX, String builderName, BuildDirs buildDirs
}
}

private Path getContextPath(File buildArchive) throws MojoExecutionException {
protected Path getContextPath(File buildArchive) throws MojoExecutionException {
String archiveName = buildArchive.getName();
String fileName = archiveName.substring(0, archiveName.indexOf('.'));
File destinationDirectory = new File(buildArchive.getParentFile(), fileName);
Expand All @@ -172,22 +174,22 @@ private Path getContextPath(File buildArchive) throws MojoExecutionException {
return destinationPath;
}

private Path getDockerStateDir(BuildImageConfiguration buildConfiguration, BuildDirs buildDirs) {
protected Path getDockerStateDir(BuildImageConfiguration buildConfiguration, BuildDirs buildDirs) {
String stateDir = buildConfiguration.getBuildX().getDockerStateDir();
Path dockerStatePath = buildDirs.getBuildPath(stateDir != null ? EnvUtil.resolveHomeReference(stateDir) : "docker");
createDirectory(dockerStatePath);
return dockerStatePath;
}

private void createDirectory(Path cachePath) {
protected void createDirectory(Path cachePath) {
try {
Files.createDirectories(cachePath);
} catch (IOException e) {
throw new IllegalArgumentException("Cannot create " + cachePath);
}
}

private String createBuilder(Path configPath, List<String> buildX, ImageConfiguration imageConfig, BuildDirs buildDirs) throws MojoExecutionException {
protected String createBuilder(Path configPath, List<String> buildX, ImageConfiguration imageConfig, BuildDirs buildDirs) throws MojoExecutionException {
BuildXConfiguration buildXConfiguration = imageConfig.getBuildConfiguration().getBuildX();
String builderName = buildXConfiguration.getBuilderName();
if (builderName == null) {
Expand Down
132 changes: 132 additions & 0 deletions src/test/java/io/fabric8/maven/docker/service/BuildXServiceTest.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
package io.fabric8.maven.docker.service;

import java.util.Arrays;
import java.util.List;
import java.io.File;
import java.nio.file.Path;
import java.nio.file.Paths;

import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;

import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.Spy;
import org.mockito.Mockito;
import org.mockito.quality.Strictness;
import org.mockito.junit.jupiter.MockitoExtension;
import org.mockito.junit.jupiter.MockitoSettings;

import io.fabric8.maven.docker.access.DockerAccess;
import io.fabric8.maven.docker.access.AuthConfig;
import io.fabric8.maven.docker.assembly.DockerAssemblyManager;
import io.fabric8.maven.docker.util.Logger;
import io.fabric8.maven.docker.util.ProjectPaths;
import io.fabric8.maven.docker.config.BuildXConfiguration;
import io.fabric8.maven.docker.config.BuildImageConfiguration;
import io.fabric8.maven.docker.config.ImageConfiguration;

@ExtendWith(MockitoExtension.class)
@MockitoSettings(strictness = Strictness.WARN)
class BuildXServiceTest {

private static final String NATIVE = "linux/amd64";
private static final String FOREIGN1 = "linux/arm64";
private static final String FOREIGN2 = "darwin/amd64";

@Mock
private DockerAccess dockerAccess;

@Mock
private DockerAssemblyManager dockerAssemblyManager;

@Mock
private Logger logger;

@Mock
private BuildXService.Exec exec;

@InjectMocks
@Spy
private BuildXService buildx;

private ImageConfiguration imageConfig;

private final ProjectPaths projectPaths = new ProjectPaths(new File("project-base-dir"), "output-dir");
private final String configuredRegistry = "configured-registry";
private final File buildArchive = new File("build-archive");
private final AuthConfig authConfig = null;

@BeforeEach
void setup() throws Exception {

Mockito.when(dockerAccess.getNativePlatform()).thenReturn(NATIVE);

Mockito.doNothing().when(buildx).createConfigJson(Mockito.any(), Mockito.any());
Mockito.doNothing().when(buildx).removeConfigJson(Mockito.any());

Mockito.doReturn(Paths.get("docker-state-dir")).when(buildx).getDockerStateDir(Mockito.any(), Mockito.any());
Mockito.doReturn("maven").when(buildx).createBuilder(Mockito.any(), Mockito.any(), Mockito.any(), Mockito.any());
}

@Test
void testBuildNativePlatform() throws Exception {
givenAnImageConfiguration(NATIVE);
mockBuildX();
buildx.build(projectPaths, imageConfig, configuredRegistry, authConfig, buildArchive);
verifyBuildXPlatforms(NATIVE);
}

@Test
void testBuildForeignPlatform() throws Exception {
givenAnImageConfiguration(FOREIGN1);
mockBuildX();
buildx.build(projectPaths, imageConfig, configuredRegistry, authConfig, buildArchive);
verifyBuildXPlatforms(FOREIGN1);
}

@Test
void testBuildNativePlatformWithForeign() throws Exception {
givenAnImageConfiguration(NATIVE, FOREIGN1);
mockBuildX();
buildx.build(projectPaths, imageConfig, configuredRegistry, authConfig, buildArchive);
verifyBuildXPlatforms(NATIVE);
}

@Test
void testBuildForeignPlatforms() throws Exception {
givenAnImageConfiguration(FOREIGN1, FOREIGN2);
buildx.build(projectPaths, imageConfig, configuredRegistry, authConfig, buildArchive);
Mockito.verify(buildx, Mockito.times(0)).buildX(Mockito.any(), Mockito.any(), Mockito.any(), Mockito.any(),
Mockito.any(), Mockito.any(), Mockito.any(), Mockito.any());
}

private void givenAnImageConfiguration(String... platforms) {

final BuildXConfiguration buildxConfig = new BuildXConfiguration.Builder()
.platforms(Arrays.asList(platforms))
.build();

final BuildImageConfiguration buildImageConfig = new BuildImageConfiguration.Builder()
.buildx(buildxConfig)
.build();

imageConfig = new ImageConfiguration.Builder()
.name("build-image")
.buildConfig(buildImageConfig)
.build();
}

private void mockBuildX() throws Exception {
Mockito.doNothing().when(buildx).buildX(Mockito.any(), Mockito.any(), Mockito.any(), Mockito.any(),
Mockito.any(), Mockito.any(), Mockito.any(), Mockito.any());
}

private void verifyBuildXPlatforms(String... platforms) throws Exception {
final List<String> expect = Arrays.asList(platforms);
Mockito.verify(buildx).buildX(Mockito.any(), Mockito.any(), Mockito.any(), Mockito.any(), Mockito.any(),
Mockito.argThat(l -> expect.equals(l)), Mockito.any(), Mockito.any());
}
}