Initial smoke-test framework for docker testimages

/kind feature
/area ci

Added small framework that allows to define small test script that allows doing some work around the Docker context.
I've added small tests around basic functionalities and binary presence.
Script results outputs are collected to the ARTIFACTS directory and if any error occurs, the build loop is interrupted, returning non-zero exit code.
The functionality docu is provided for anyone who wants to add another smoke test for new or existing image.

images/README.md

@@ -10,3 +10,22 @@ To add additional applications into the images, open a PR with changes. Follow t
 * Always build from a source to ensure compiler vulnerabilities do not affect the resulting binary
 * Link the binary to a specific version so that it's easier to update when necessary 
 * Build binaries in a separate stage, then copy the resulting binary into the final image to ensure images are small and contain the least number of layers
+
+## Writing image tests
+
+to write simple smoke tests with your image, add an **executable** file called `test.sh`.
+The scripts should contain all steps that perform basic or advanced test operations against the image. You are allowed to use all binaries available in [E2E DinD K3d image](./e2e-dind-k3d) to test built image.
+Test script **must** exit with non-zero number, if any of the steps failed.
+
+By default, current context of a test script will always be Docker build context. Image name is passed as a variable `IMG`.
+
+### Example
+
+The example below showcases the example definition of test.sh script.
+```shell
+#!/usr/bin/env bash
+set -e
+echo "$IMG"
+docker run --rm $IMG -- some-command
+test $? -eq 0 || exit 1
+```

images/build-images.sh

@@ -1,7 +1,7 @@
 #!/usr/bin/env bash
 
 set -e
-
+ARTIFACTS="${ARTIFACTS:-/tmp}"
 # WORKAROUND
 #TODO (@Ressetkk): Use bundled image with docker-credential-gcr and docker
 if [[ $CI == "true" ]]; then
@@ -20,12 +20,20 @@ toPush=()
 for v in $(find . -type d -exec test -e '{}'/Dockerfile \; -print | cut -c3-) ; do
   name=$(echo "$v" | sed "s/\//-/g")
   echo "building $name..."
+  IMG="local/$v"
   docker buildx build \
     --load \
-    -t "local/$v" \
+    -t "$IMG" \
     -t "$REGISTRY/$name:latest" \
     -t "$REGISTRY/$name:$TAG" \
     "./$v"
+
+  if [ -x "./$v/test.sh" ]; then
+    pushd "./$v"
+    echo "Run $v/test.sh"
+    IMG=$IMG ./test.sh &> "$ARTIFACTS/$name-test.log" && echo "OK!" || exit 1
+    popd
+  fi
   toPush+=("$REGISTRY/$name")
 done
 

images/buildpack/go/test.sh

@@ -0,0 +1,20 @@
+#!/usr/bin/env bash
+
+set -e
+podman run --arch=amd64 --rm "$IMG" bash -c '
+set -e
+go version
+ko version
+kubebuilder version
+kustomize version
+jobguard -help
+
+cat<<EOF > /tmp/main.go
+package main
+import "fmt"
+func main() {
+fmt.Println("Hello World!")
+}
+EOF
+go run /tmp/main.go
+'

images/e2e-dind-k3d/test.sh

@@ -0,0 +1,42 @@
+#!/usr/bin/env bash
+
+set -e
+
+echo ">>> Basic checks"
+docker run --rm --privileged \
+  "$IMG" bash -c '
+  set -e
+  helm version
+  jobguard -h
+  env
+  '
+echo "DinD checks"
+docker run --rm --privileged \
+  -e DOCKER_IN_DOCKER_ENABLED=true \
+  "$IMG" bash -c '
+  set -e
+  cat $ARTIFACTS/docker-info.log
+  docker run --rm alpine:latest uname -a
+  '
+
+echo ">>> K3D test"
+docker run --rm --privileged \
+  -e DOCKER_IN_DOCKER_ENABLED=true \
+  -e K3D_ENABLED=true \
+  -e PROVISION_REGISTRY=true \
+  "$IMG" bash -c '
+  k3d version
+  k3d cluster get
+  docker ps
+  kubectl cluster-info
+  kubectl run nginx --image=nginx:latest
+  '
+
+docker run --rm --privileged \
+  "$IMG" bash -c '
+  kind version
+  kind  create cluster
+  docker ps
+  kubectl cluster-info
+  kubectl run nginx --image=nginx:latest
+  '

images/e2e-dind-nodejs/test.sh

@@ -0,0 +1,40 @@
+#!/usr/bin/env bash
+
+set -e
+
+echo ">>> Basic checks"
+docker run --rm --privileged \
+  "$IMG" bash -c '
+  set -e
+  helm version
+  jobguard -h
+  env
+  '
+echo "DinD checks"
+docker run --rm --privileged \
+  -e DOCKER_IN_DOCKER_ENABLED=true \
+  "$IMG" bash -c '
+  set -e
+  cat $ARTIFACTS/docker-info.log
+  docker run --rm alpine:latest uname -a
+  '
+
+echo ">>> K3D test"
+docker run --rm --privileged \
+  -e DOCKER_IN_DOCKER_ENABLED=true \
+  -e K3D_ENABLED=true \
+  -e PROVISION_REGISTRY=true \
+  "$IMG" bash -c '
+  k3d version
+  k3d cluster get
+  docker ps
+  kubectl cluster-info
+  kubectl run nginx --image=nginx:latest
+  '
+echo ">>> NodeJS test"
+docker run --rm --privileged \
+  "$IMG" bash -c '
+  node --version
+  npm --version
+  yarn version
+  '