service-catalog cross build test with dind #8734

MHBauer · 2018-07-19T17:42:15Z

I created this by guessing that I need a docker-in-docker enabled instance.

I found the variable DOCKER_IN_DOCKER_ENABLED, and searched for it.

I picked this to copy from
https://github.com/kubernetes/test-infra/blob/68c68683a066e7b9986113c73a6dcb0f48775ef9/config/jobs/kubernetes-sigs/cluster-api-provider-openstack/cluster-api-provider-openstack-presubmits.yaml
because it's in the new location for config, so I figure that means someone has looked at it.

I didn't totally copy it, because I don't know what some of the args are, such as:

    labels: 
        preset-service-account: "true"

and - "--upload=gs://kubernetes-jenkins/pr-logs"

I also don't know how much memory it needs, so I dropped the 6G requirement. I do not know what the default is, but I'm going to hope it's enough for now.

krzyzacy · 2018-07-19T17:43:55Z

/assign @BenTheElder
do we still need to mount docker-graph as an emptydir?

MHBauer · 2018-07-19T18:08:25Z

fixed the bad indent on the yaml.

MHBauer · 2018-07-19T18:36:09Z

okay, should be really real this time. where's yaml-fmt when you need it?

krzyzacy · 2018-07-19T19:44:29Z

config/jobs/kubernetes-incubator/service-catalog/service-catalog-presubmits.yaml

+    always_run: true
+    skip_report: false
+    rerun_command: "/test pull-service-catalog-xbuild"
+    trigger: "/test( all| pull-service-catalog-xbuild)"


you probably want to copy the above regex

krzyzacy · 2018-07-19T23:09:36Z

config/jobs/kubernetes-incubator/service-catalog/service-catalog-presubmits.yaml

+    always_run: true
+    skip_report: false
+    rerun_command: "/test pull-service-catalog-xbuild"
+    trigger: "(?m)^/test( all|pull-service-catalog-xbuild)\\s*"


lol, /test (all|pull-service-catalog-xbuild)

how did that space get in there, 😞

MHBauer · 2018-07-20T01:16:04Z

I have a very powerful case of "the dumb" today. How can I run this check locally?

MHBauer · 2018-07-20T01:31:00Z

yay green.

BenTheElder · 2018-07-20T16:12:27Z

config/jobs/kubernetes-incubator/service-catalog/service-catalog-presubmits.yaml

+        - "images-all"
+        - "svcat-all"
+      env:
+      - name: DOCKER_IN_DOCKER_ENABLED


I'm going to turn this into a preset soon. Initially we didn't because we wanted to discourage depending on it (I only initially built the support to help us get the rest of our existing jobs onto Prow), but so far it works just fine.

This must be paired with an emptyDir volume for the the graph storage (which is remapped /var/lib/docker for :reasons:, we should probably unset the option that changes that). Unless things have improved this is necessary to avoid issues with overlays on overlays. Existing docker-in-docker jobs should have this.

What is a preset?

I was basing off of, which has preset-service-account
https://github.com/kubernetes/test-infra/blob/68c68683a066e7b9986113c73a6dcb0f48775ef9/config/jobs/kubernetes-sigs/cluster-api-provider-openstack/cluster-api-provider-openstack-presubmits.yaml

It does not have a emptydir, so maybe it's a bad example, oops.

searched for emptyDir and found helm/charts repo
https://github.com/kubernetes/test-infra/blob/1c40d5125cd571010e6949fe0b41065f5973542c/config/jobs/helm/charts/charts.yaml

preset is something we are mimicing https://kubernetes.io/docs/concepts/workloads/pods/podpreset/

BenTheElder · 2018-07-20T16:13:17Z

config/jobs/kubernetes-incubator/service-catalog/service-catalog-presubmits.yaml

+      - args:
+        - "--repo=sigs.k8s.io/$(REPO_NAME)=$(PULL_REFS)"
+        - "--root=/go/src/github.com/kubernetes-incubator/$(REPO_NAME)"
+        - "--timeout=90"


does this take 90 minutes? if so, does it use many cores / gbs of ram? if it does, we should add a resource request. if it doesn't do any of those things, we should tighten the timeout.

it's a go build, so it'll use cores we give it. What's the default? 1/2/500millicore?

the two make targets together take about 50minutes on travis, but much less locally, so I'm not sure how to gauge.

This is pretty much the most cpu intensive workload we have. Others take long, but are filled with pauses and inefficiencies. This should be straight compiling.

I'll change it to 60, if that sounds okay. What is the default?

default is infinite I think

MHBauer

@BenTheElder thanks for help. didn't mean to do a review, oops.

MHBauer · 2018-07-20T17:02:45Z

config/jobs/kubernetes-incubator/service-catalog/service-catalog-presubmits.yaml

+        - "images-all"
+        - "svcat-all"
+      env:
+      - name: DOCKER_IN_DOCKER_ENABLED


What is a preset?

I was basing off of, which has preset-service-account
https://github.com/kubernetes/test-infra/blob/68c68683a066e7b9986113c73a6dcb0f48775ef9/config/jobs/kubernetes-sigs/cluster-api-provider-openstack/cluster-api-provider-openstack-presubmits.yaml

It does not have a emptydir, so maybe it's a bad example, oops.

MHBauer · 2018-07-20T17:06:07Z

config/jobs/kubernetes-incubator/service-catalog/service-catalog-presubmits.yaml

+      - args:
+        - "--repo=sigs.k8s.io/$(REPO_NAME)=$(PULL_REFS)"
+        - "--root=/go/src/github.com/kubernetes-incubator/$(REPO_NAME)"
+        - "--timeout=90"


it's a go build, so it'll use cores we give it. What's the default? 1/2/500millicore?

the two make targets together take about 50minutes on travis, but much less locally, so I'm not sure how to gauge.

This is pretty much the most cpu intensive workload we have. Others take long, but are filled with pauses and inefficiencies. This should be straight compiling.

krzyzacy · 2018-07-20T20:13:00Z

/lgtm
/hold

krzyzacy · 2018-07-20T20:16:14Z

/lgtm cancel
whoops, if you are using kubekins image, you'll still need preset-service-account

MHBauer · 2018-07-20T20:19:05Z

lol, there's a test for it. Why didn't that show up earlier?

I0720 20:13:22.986] --- FAIL: TestValidPresets (0.00s)
I0720 20:13:22.986] 	jobs_test.go:695: Error in presubmit "pull-service-catalog-xbuild": cannot find service account preset

Is there a way to run these tests? I'll go search for what's already failed and try to read some.

MHBauer · 2018-07-20T20:20:03Z

I understand now. I'll add that back in.

krzyzacy · 2018-07-20T20:21:28Z

from gubernator link, you can find the test command: bazel test //config/tests/jobs:go_default_test

krzyzacy · 2018-07-20T20:23:43Z

I cannot find the previous snapshot, but I assume you had it before and removed it at some point?

krzyzacy · 2018-07-20T20:24:00Z

/lgtm

k8s-ci-robot · 2018-07-20T20:24:05Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: krzyzacy, MHBauer

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~config/jobs/kubernetes-incubator/service-catalog/OWNERS~~ [MHBauer,krzyzacy]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

MHBauer · 2018-07-20T20:24:09Z

snapshot of what?

krzyzacy · 2018-07-20T20:25:03Z

how your initial commit looks like :-)

MHBauer · 2018-07-20T20:32:15Z

I never put it in there, but I mentioned it in the original comment that I saw it and didn't know what it did.

MHBauer · 2018-07-20T20:32:58Z

Does this seem sane now?

krzyzacy · 2018-07-20T20:34:53Z

/hold cancel
(feel free to /hold cancel yourself in the future, as long as you have lgtm and approve)

k8s-ci-robot · 2018-07-20T20:36:49Z

@MHBauer: Updated the job-config configmap using the following files:
- key service-catalog-presubmits.yaml using file config/jobs/kubernetes-incubator/service-catalog/service-catalog-presubmits.yaml

In response to this:

I created this by guessing that I need a docker-in-docker enabled instance.

I found the variable DOCKER_IN_DOCKER_ENABLED, and searched for it.

I picked this to copy from
https://github.com/kubernetes/test-infra/blob/68c68683a066e7b9986113c73a6dcb0f48775ef9/config/jobs/kubernetes-sigs/cluster-api-provider-openstack/cluster-api-provider-openstack-presubmits.yaml
because it's in the new location for config, so I figure that means someone has looked at it.

I didn't totally copy it, because I don't know what some of the args are, such as:
   labels: 
       preset-service-account: "true"
and - "--upload=gs://kubernetes-jenkins/pr-logs"

I also don't know how much memory it needs, so I dropped the 6G requirement. I do not know what the default is, but I'm going to hope it's enough for now.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

MHBauer · 2018-07-20T20:36:53Z

Yes, you did lgtm it again. I like a little bit more context I suppose.

krzyzacy · 2018-07-20T20:40:16Z

lol, lgtm means you are doing good :-)

MHBauer · 2018-07-20T20:53:26Z

@krzyzacy alright, I need some help kubernetes-retired/service-catalog#2212 not sure if this takes longer to take effect, or if we're having more repo permission issues, or how to debug this.

krzyzacy · 2018-07-20T20:57:17Z

config/jobs/kubernetes-incubator/service-catalog/service-catalog-presubmits.yaml

+      containers:
+      - image: gcr.io/k8s-testimages/kubekins-e2e:v20180716-9145034c9-1.10
+        args:
+        - "--repo=sigs.k8s.io/$(REPO_NAME)=$(PULL_REFS)"


this should be github.com/kubernetes-incubator instead of sigs.k8s.io

of course it should 🙃

followup fix in #8763

MHBauer requested review from BenTheElder and krzyzacy as code owners July 19, 2018 17:42

k8s-ci-robot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Jul 19, 2018

k8s-ci-robot assigned BenTheElder Jul 19, 2018

MHBauer force-pushed the svc-cat-xbuild branch from 1027f0b to 29ba355 Compare July 19, 2018 18:08

MHBauer force-pushed the svc-cat-xbuild branch from 29ba355 to 4b021ec Compare July 19, 2018 18:35

krzyzacy reviewed Jul 19, 2018

View reviewed changes

MHBauer force-pushed the svc-cat-xbuild branch from 4b021ec to 2053a5d Compare July 19, 2018 22:51

krzyzacy reviewed Jul 19, 2018

View reviewed changes

MHBauer force-pushed the svc-cat-xbuild branch 2 times, most recently from 0b79da5 to 063a59d Compare July 20, 2018 01:15

MHBauer force-pushed the svc-cat-xbuild branch from 063a59d to 0912ab8 Compare July 20, 2018 01:16

MHBauer mentioned this pull request Jul 20, 2018

bootstrapping service-catalog use of prow for CI/CD #8291

Closed

37 tasks

BenTheElder reviewed Jul 20, 2018

View reviewed changes

MHBauer commented Jul 20, 2018

View reviewed changes

k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Jul 20, 2018

k8s-ci-robot assigned krzyzacy Jul 20, 2018

k8s-ci-robot added do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. lgtm "Looks good to me", indicates that a PR is ready to be merged. labels Jul 20, 2018

k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jul 20, 2018

service-catalog cross build test with dind

67bc9f1

MHBauer force-pushed the svc-cat-xbuild branch from 5489c67 to 67bc9f1 Compare July 20, 2018 20:22

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jul 20, 2018

k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jul 20, 2018

k8s-ci-robot merged commit c95b9d3 into kubernetes:master Jul 20, 2018

MHBauer deleted the svc-cat-xbuild branch July 20, 2018 20:53

krzyzacy reviewed Jul 20, 2018

View reviewed changes

service-catalog cross build test with dind #8734

service-catalog cross build test with dind #8734

Conversation

MHBauer commented Jul 19, 2018

krzyzacy commented Jul 19, 2018

MHBauer commented Jul 19, 2018

MHBauer commented Jul 19, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MHBauer commented Jul 20, 2018

MHBauer commented Jul 20, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MHBauer left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

krzyzacy commented Jul 20, 2018

krzyzacy commented Jul 20, 2018

MHBauer commented Jul 20, 2018

MHBauer commented Jul 20, 2018

krzyzacy commented Jul 20, 2018

krzyzacy commented Jul 20, 2018

krzyzacy commented Jul 20, 2018

k8s-ci-robot commented Jul 20, 2018

MHBauer commented Jul 20, 2018

krzyzacy commented Jul 20, 2018

MHBauer commented Jul 20, 2018

MHBauer commented Jul 20, 2018

krzyzacy commented Jul 20, 2018

k8s-ci-robot commented Jul 20, 2018

MHBauer commented Jul 20, 2018

krzyzacy commented Jul 20, 2018

MHBauer commented Jul 20, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment