Jib sample #1147

dgageot · 2018-10-12T10:00:45Z

No description provided.

codecov-io · 2018-10-12T10:06:06Z

Codecov Report

Merging #1147 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #1147   +/-   ##
=======================================
  Coverage   41.45%   41.45%           
=======================================
  Files          95       95           
  Lines        4229     4229           
=======================================
  Hits         1753     1753           
  Misses       2305     2305           
  Partials      171      171

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3c27520...b8de01e. Read the comment docs.

dgageot · 2018-10-12T10:26:12Z

ping @coollog @briandealwis Do you think this is ok?

briandealwis

I assumed the integration is just duplicate?

briandealwis · 2018-10-12T12:53:54Z

examples/annotated-skaffold.yaml

@@ -61,6 +61,15 @@ build:
    # bazel:
    #  target: //:skaffold_example.tar

+    # jibMaven is an artifacts built with Maven, typically (but not only) a Java project.


nit: artifacts should be singular, or is for artifacts. Could I suggest the following?

# jibMaven builds containers using the Jib plugin for Maven

Do you see this file as being documentation? Is it worth adding some detail about the module and profile items?

briandealwis · 2018-10-12T12:54:15Z

examples/annotated-skaffold.yaml

+    #  module: modulename
+    #  profile: profilename
+
+    # jibGradle is an artifacts built with Gradle, typically (but not only) a Java project.


And ditto re: artifacts or the suggested rewrite.

I'll rewrite it

briandealwis · 2018-10-12T12:58:37Z

examples/jib/pom.xml

+                <version>0.9.11</version>
+                <configuration>
+                    <from>
+                        <image>gcr.io/distroless/java@sha256:b430543bea1d8326e767058bdab3a2482ea45f59d7af5c5c61334cd29ede88a1</image>


Why did you choose a specific tag, as opposed to :latest?

dgageot · 2018-10-12T17:41:31Z

@briandealwis I took your feedback into account. integration is a necessary duplicate

briandealwis · 2018-10-12T18:02:15Z

examples/annotated-skaffold.yaml

@@ -61,6 +61,15 @@ build:
    # bazel:
    #  target: //:skaffold_example.tar

+    # jibMaven builds containers using the Jib plugin for Maven.
+    # jibMaven:
+    #  module: modulename     # selects which maven module to build


for a multimodule project

briandealwis · 2018-10-12T18:36:27Z

examples/jib/pom.xml

+                <configuration>
+                    <container>
+                        <jvmFlags>
+                            <jvmFlag>-Djava.security.egd=file:/dev/./urandom</jvmFlag>


Why the /./?

Because of a funny bug: https://bugs.java.com/bugdatabase/view_bug.do?bug_id=6202721

briandealwis · 2018-10-12T19:18:30Z

examples/jib/README.adoc

+[Jib](https://github.com/GoogleContainerTools/jib) builds Docker and OCI images
+for your Java applications and is available as plugins for Maven and Gradle.
+
+The way you configure it in `skaffold.yaml` is the following build stanza:


I think it's worth mentioning that this example is for a standalone Maven project, where all dependencies are resolved from outside. The Jib builder requires that the projects are configured to use the Jib plugins for Maven or Gradle. Multi-module builds require a bit additional configuration.

anthonydahanne · 2018-10-14T13:54:25Z

hello ! I've tried to run the samples

git fetch dgageot
git checkout jib-sample
cd examples/jib

and then

I updated the references to "gcr.io/k8s-skaffold/skaffold-jib" with "anthonydahanne/test:skaffold"
I had minikube running
and I run skaffold dev but I ended up with :
FATA[0009] exiting dev mode because the first build failed: building [anthonydahanne/test:skaffold]: digest not found

Using debug logs, I had :

[adah@MCADAH02 jib (jib-sample)]$ skaffold dev --verbosity debug
INFO[0000] Skaffold &{Version:v0.16.0 GitVersion: GitCommit:78e443973ee7475ee66d227431596351cf5e2caf GitTreeState:clean BuildDate:2018-10-11T23:05:05Z GoVersion:go1.11.1 Compiler:gc Platform:darwin/amd64} 
DEBU[0000] Defaulting build type to local build         
DEBU[0000] Defaulting deploy type to kubectl            
INFO[0000] Using kubectl context: minikube              
DEBU[0000] Using builder: local                         
DEBU[0000] Running command: [minikube docker-env --shell none] 
DEBU[0000] Command output: stdout DOCKER_TLS_VERIFY=1
DOCKER_HOST=tcp://192.168.99.100:2376
DOCKER_CERT_PATH=/Users/adah/.minikube/certs
DOCKER_API_VERSION=1.35
, stderr:  
DEBU[0000] push value not present, defaulting to false because localCluster is true 
DEBU[0000] Running command: [mvn jib:_skaffold-files -q] 
DEBU[0003] Command output: stdout /Users/adah/workspaces/skaffold/examples/jib/pom.xml
/Users/adah/workspaces/skaffold/examples/jib/src/main/java
/Users/adah/workspaces/skaffold/examples/jib/src/main/resources
/Users/adah/workspaces/skaffold/examples/jib/src/main/resources
/Users/adah/workspaces/skaffold/examples/jib/src/main/jib
, stderr:  
DEBU[0003] could not stat dependency: stat /Users/adah/workspaces/skaffold/examples/jib/src/main/resources: no such file or directory 
DEBU[0003] could not stat dependency: stat /Users/adah/workspaces/skaffold/examples/jib/src/main/resources: no such file or directory 
DEBU[0003] could not stat dependency: stat /Users/adah/workspaces/skaffold/examples/jib/src/main/jib: no such file or directory 
DEBU[0003] Found dependencies for jibMaven artifact: [/Users/adah/workspaces/skaffold/examples/jib/pom.xml /Users/adah/workspaces/skaffold/examples/jib/src/main/java /Users/adah/workspaces/skaffold/examples/jib/src/main/java/hello /Users/adah/workspaces/skaffold/examples/jib/src/main/java/hello/Application.java /Users/adah/workspaces/skaffold/examples/jib/src/main/java/hello/HelloController.java] 
Starting build...
Found [minikube] context, using local docker daemon.
Building [anthonydahanne/test:skaffold]...
INFO[0003] Building .: /usr/local/bin/mvn, [mvn prepare-package jib:dockerBuild -Dimage=jib__087e74d83f207d1cd326fc12cd7e0110bb02da61] 
DEBU[0003] Running command: [mvn prepare-package jib:dockerBuild -Dimage=jib__087e74d83f207d1cd326fc12cd7e0110bb02da61] 
[INFO] Scanning for projects...
[INFO] 
[INFO] -------------------< org.skaffold:hello-spring-boot >-------------------
[INFO] Building hello-spring-boot 0.1.0
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-resources-plugin:3.0.2:resources (default-resources) @ hello-spring-boot ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /Users/adah/workspaces/skaffold/examples/jib/src/main/resources
[INFO] skip non existing resourceDirectory /Users/adah/workspaces/skaffold/examples/jib/src/main/resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.7.0:compile (default-compile) @ hello-spring-boot ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-resources-plugin:3.0.2:testResources (default-testResources) @ hello-spring-boot ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /Users/adah/workspaces/skaffold/examples/jib/src/test/resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.7.0:testCompile (default-testCompile) @ hello-spring-boot ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.21.0:test (default-test) @ hello-spring-boot ---
[INFO] No tests to run.
[INFO] 
[INFO] --- jib-maven-plugin:0.9.11:dockerBuild (default-cli) @ hello-spring-boot ---
[WARNING] Base image 'gcr.io/distroless/java' does not use a specific image digest - build may not be reproducible
[INFO] 
[INFO] Containerizing application to Docker daemon as jib__087e74d83f207d1cd326fc12cd7e0110bb02da61...
[INFO] 
[INFO] Getting base image gcr.io/distroless/java...
[INFO] Building dependencies layer...
[INFO] Building classes layer...
[INFO] Finalizing...
[INFO] Loading to Docker daemon...
[INFO] 
[INFO] Container entrypoint set to [java, -Djava.security.egd=file:/dev/./urandom, -cp, /app/resources:/app/classes:/app/libs/*, hello.Application]
[INFO] 
[INFO] Built image to Docker daemon as jib__087e74d83f207d1cd326fc12cd7e0110bb02da61
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4.897 s
[INFO] Finished at: 2018-10-14T09:46:07-04:00
[INFO] ------------------------------------------------------------------------
DEBU[0009] Defaulting build type to local build         
DEBU[0009] Defaulting deploy type to kubectl            
INFO[0009] Using kubectl context: minikube              
DEBU[0009] Using builder: local                         
DEBU[0009] push value not present, defaulting to false because localCluster is true 
Cleaning up...
DEBU[0009] manifests apiVersion: v1
kind: Service
metadata:
  name: web
spec:
  ports:
  - port: 8080
    name: http
  type: LoadBalancer
  selector:
    app: web
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
spec:
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      containers:
      - name: web
        image: anthonydahanne/test:skaffold 
DEBU[0009] Running command: [kubectl --context minikube delete --ignore-not-found=true -f -] 
Cleanup complete in 222.996062ms
FATA[0009] exiting dev mode because the first build failed: building [anthonydahanne/test:skaffold]: digest not found

anthonydahanne · 2018-10-14T14:08:00Z

ok, so I found and fixed my mistakes :

Skaffold and Jib were using my Docker for Mac docker daemon, instead of using my Minikube own docker daemon; hence the "digest not found"; to fix that, I had to run, just before executing skaffold :

eval $(minikube docker-env)

Then I hit a second issue :

FATA[0013] exiting dev mode because the first build failed: building [anthonydahanne/test:skaffold]: tagging: Error parsing reference: "anthonydahanne/test:skaffold:7e79003b-dirty-b43b301" is not a valid repository/tag: invalid reference format

To fix that one, I just needed to remove the tag after my image name(replacing anthonydahanne/test:skaffold with anthonydahanne/test ), since, I should have known , skaffold and jib themselves tag the image

So I got it working and I could display a brilliant "Hello Anthony" after updating the HelloController :-)

anthonydahanne · 2018-10-15T00:12:13Z

I have 1 question though : I've seen in the source code, that's it's possible to append profiles, using, for example :

    jibMaven:
      profile: "prometheus"

Can we also pass -D vm arguments as well ?
Thanks ! (let me know if I'm abusing this Github issue, I'll post my questions somewhere more adequate !)

briandealwis · 2018-10-15T00:39:17Z

Not at the moment. You can define Maven properties in a profile. And we don't actually validate the profile value so you should be able to specify multiple profile IDs by separating them with commas.

What are you trying to do?

anthonydahanne · 2018-10-15T01:37:29Z

What are you trying to do?

-DskipTests

That would not be convenient to create a profile for each VM argument or their combinations...

dgageot · 2018-10-15T04:59:30Z

Thank you @anthonydahanne for the feedback. I'll create a ticket for the image name.

briandealwis · 2018-10-15T12:59:16Z

One possible workaround (of sorts) is to use environment variables. You can also activate a profile with an environment variable.

It may make sense for Skaffold to expose some information as to the build mode. For example, that skaffold dev is in use so don't run tests.

coollog · 2018-10-15T14:26:50Z

examples/annotated-skaffold.yaml

@@ -61,6 +61,15 @@ build:
    # bazel:
    #  target: //:skaffold_example.tar

+    # jibMaven builds containers using the Jib plugin for Maven.
+    # jibMaven:
+    #  module: modulename     # selects which maven module to build


Should we leave this out for now? (Since it's currently unimplemented)

coollog · 2018-10-15T14:31:23Z

examples/jib/pom.xml

+                <configuration>
+                    <container>
+                        <jvmFlags>
+                            <jvmFlag>-Djava.security.egd=file:/dev/./urandom</jvmFlag>


Perhaps we should also just include the other container-enabling JVM flags here too:

<jvmFlag>-XX:+UnlockExperimentalVMOptions</jvmFlag> <jvmFlag>-XX:+UseCGroupMemoryLimitForHeap</jvmFlag>

dgageot · 2018-10-16T13:17:32Z

@GoogleContainerTools/java-tools should be ready for review

dgageot · 2018-10-16T17:55:39Z

Something is maybe flaky with kaniko local mode

Signed-off-by: David Gageot <david@gageot.net>

balopat

LGTM

dgageot requested review from balopat, nkubala and r2d4 as code owners October 12, 2018 10:00

dgageot force-pushed the jib-sample branch 2 times, most recently from 7c0b776 to a8197b0 Compare October 12, 2018 10:43

briandealwis reviewed Oct 12, 2018

View reviewed changes

dgageot force-pushed the jib-sample branch from a8197b0 to 7e79003 Compare October 12, 2018 17:40

briandealwis reviewed Oct 12, 2018

View reviewed changes

coollog reviewed Oct 15, 2018

View reviewed changes

dgageot force-pushed the jib-sample branch from d52600f to 275da5e Compare October 16, 2018 13:17

dgageot force-pushed the jib-sample branch 2 times, most recently from 9b5d41e to d75a792 Compare October 22, 2018 08:49

dgageot added 3 commits October 24, 2018 07:57

Add jib configuration to annotated-skaffold.yaml

1345b34

Signed-off-by: David Gageot <david@gageot.net>

Add a sample for Jib

15c0451

Signed-off-by: David Gageot <david@gageot.net>

Take feedback into account

b8de01e

Signed-off-by: David Gageot <david@gageot.net>

dgageot force-pushed the jib-sample branch from d75a792 to b8de01e Compare October 24, 2018 05:57

balopat approved these changes Oct 25, 2018

View reviewed changes

balopat merged commit 6fe5304 into GoogleContainerTools:master Oct 25, 2018

dgageot deleted the jib-sample branch December 28, 2018 07:14

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Jib sample #1147

Jib sample #1147

dgageot commented Oct 12, 2018

codecov-io commented Oct 12, 2018 •

edited

Loading

dgageot commented Oct 12, 2018

briandealwis left a comment

briandealwis Oct 12, 2018

briandealwis Oct 12, 2018

dgageot Oct 12, 2018

briandealwis Oct 12, 2018

dgageot commented Oct 12, 2018

briandealwis Oct 12, 2018

briandealwis Oct 12, 2018

dgageot Oct 12, 2018

briandealwis Oct 12, 2018

anthonydahanne commented Oct 14, 2018

anthonydahanne commented Oct 14, 2018

anthonydahanne commented Oct 15, 2018

briandealwis commented Oct 15, 2018

anthonydahanne commented Oct 15, 2018

dgageot commented Oct 15, 2018

briandealwis commented Oct 15, 2018

coollog Oct 15, 2018

coollog Oct 15, 2018

dgageot commented Oct 16, 2018

dgageot commented Oct 16, 2018

balopat left a comment

Jib sample #1147

Jib sample #1147

Conversation

dgageot commented Oct 12, 2018

codecov-io commented Oct 12, 2018 • edited Loading

Codecov Report

dgageot commented Oct 12, 2018

briandealwis left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dgageot commented Oct 12, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

anthonydahanne commented Oct 14, 2018

anthonydahanne commented Oct 14, 2018

anthonydahanne commented Oct 15, 2018

briandealwis commented Oct 15, 2018

anthonydahanne commented Oct 15, 2018

dgageot commented Oct 15, 2018

briandealwis commented Oct 15, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dgageot commented Oct 16, 2018

dgageot commented Oct 16, 2018

balopat left a comment

Choose a reason for hiding this comment

codecov-io commented Oct 12, 2018 •

edited

Loading