Start-build/env and multitag jenkins plugin tests #11252

jupierce · 2016-10-06T17:36:16Z

Adding extended test for:

Adding environment variable support to build request jenkins-plugin#68 (Jenkins start-build environment variables)
Adding support for multitag jenkins-plugin#71 (multi-tag in single Jenkins step)
Adding exec function to Jenkins OpenShift Plugin jenkins-plugin#70 (oc exec support)

gabemontero · 2016-10-06T19:44:10Z

test/extended/jenkins/plugin.go

@@ -169,6 +193,80 @@ var _ = g.Describe("[jenkins][Slow] openshift pipeline plugin", func() {

 			g.By("get build console logs and see if succeeded")
 			err = waitForJenkinsActivity(fmt.Sprintf("http://%s/job/test-plugin-job/1/console", hostPort), "Finished: SUCCESS", 200)
+			logs, _ := getJenkinsJobLogs(hostPort, "test-plugin-job")


I think our wires got crossed when we discussed this after scrum.

Rather than inserting the test here, create a new test under g.Context with something like

g.It("jenkins-plugin test case for env var builds", func() {

And then another like

g.It("jenkins-plugin test case multi tag", func() {

I do think the BeforeSuite change is good. Bring up Jenkins just once for the (now greater than 1) set of tests

Actually, upon further recollection, BeforeSuite might be too broad. I seem to remember once having that for the force pull tests, and it got called when other tests files ran as well. I might have worked with @mfojtik on that.

After you refactor the test with multiple g.It's see if keeping it BeforeEach is sufficient for the test only startup up Jenkins once, where then each g.It adds the new test job and runs it.

Not completely crossed! I did understand your desire for additional specs, so I made the BeforeSuite change. However, since env vars were just being added to the existing build trigger step, it made sense (to me at least) to simply extend the scope of the existing test (i.e. test as many features of the build step as possible in a single build).

The tagging test required image streams which were already viable because of that build... it just seemed like a waste not to use them :) . That said, I understand your objection and will refactor to a new spec/build.

gabemontero · 2016-10-06T19:45:12Z

test/extended/testdata/testjob-plugin.xml

@@ -27,6 +27,22 @@
      <authToken></authToken>
      <followLog>true</followLog>
      <checkForTriggeredDeployments>true</checkForTriggeredDeployments>
+


In conjunction with the other comment, create a new test job xml file, that just does a build and includes your new env vars

gabemontero · 2016-10-06T19:45:38Z

test/extended/testdata/testjob-plugin.xml

@@ -53,16 +69,62 @@
      <authToken></authToken>
    </com.openshift.jenkins.plugins.pipeline.OpenShiftServiceVerifier>

+    <!-- 1 to 1 mapping -->


And again, a separate test job xml file that just includes what is needed for the multi tag test

jupierce · 2016-10-06T21:25:45Z

@gabemontero I confirmed that BeforeSuite is too broad -- there can only be one in the entire test suite, so I'm interfering with the whole test framework.
That leaves BeforeEach and some investigation as to whether I can keep jenkins up and running.

bparees · 2016-10-10T00:53:36Z

examples/jenkins/application-template.json

+      "spec": {},
+      "status": {
+        "dockerImageRepository": ""
+      }


no need to define empty status blocks.
or spec for that matter.

These have been removed.

bparees · 2016-10-10T01:03:08Z

test/extended/testdata/testjob-plugin.xml

      <prodStream>origin-nodejs-sample</prodStream>
+      <prodTag>prod2,prod3,prod4</prodTag>


@gabemontero do we support both "source/destination" naming conventions as well as this "test/prod" convention? and if so, we should be stripping out usage of the test/prod syntax, not adding more.

@bparees since I unfortunately originally named the instance variables testTag, prodTag, etc. we have a backwards compatibility issue, with folks at older versions of the plugin potentially having job config's with that old naming convention.

whether that unfortunately reality needs to bleed into new tests we create is certainly a subject for debate.... I tend to agree that we should test with the preferred names.

Conversely, I'm also fine with this existing, original test being the one test that vets out the backward compatibility pain. But when @jupierce pushes the updates for the new test xml files and breaks out the new tests into separate g.It sections, they would leverage the new tag.

right i'm saying we support both names, right? so let's use the new/correct name in our tests.

i'm not super concerned about having backwards compatibility tests for the field name.

gabemontero · 2016-10-10T19:34:21Z

On Mon, Oct 10, 2016 at 3:29 PM, Ben Parees notifications@github.com
wrote:

@bparees commented on this pull request.

In test/extended/testdata/testjob-plugin.xml
#11252:
   <prodStream>origin-nodejs-sample</prodStream>
 <prodTag>prod2,prod3,prod4</prodTag>
right i'm saying we support both names, right? so let's use the
new/correct name in our tests.

i'm not super concerned about having backwards compatibility tests for the
field name.

OK. @jupierce - are you fine with updating the existing test as well? Or
would you rather I handle it
in a separate PR?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#11252, or mute the thread
https://github.com/notifications/unsubscribe-auth/ADbadLDZYI8QjANjQkXttcWyptXJDDeCks5qypI2gaJpZM4KQNtW
.

jupierce · 2016-10-11T13:37:38Z

@gabemontero I'll handle the during my refactor.

gabemontero · 2016-10-11T13:46:34Z

Cool / thx @jupierce

On Tue, Oct 11, 2016 at 9:37 AM, Justin Pierce notifications@github.com
wrote:

@gabemontero https://github.com/gabemontero I'll handle the during my
refactor.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#11252 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADbadPD9BUvFXRFzU1nXkhYoq016cYFBks5qy5EsgaJpZM4KQNtW
.

jupierce · 2016-10-11T17:29:39Z

@gabemontero At first pass, it seemed like @bparees request would be easy, but it dawned on me that we are talking about the job.xml here -- not DSL. We support both forms for DSL, but there are no allowances for job.xml to change.
The job.xml is not public, so the the tag names don't really, matter, but if we want to clean this up for consistency sake, I believe it will pre-req changes to the jenkins plugin itself.

gabemontero · 2016-10-11T17:42:15Z

Yep - let's punt on the test/src and prod/dest xml change for now.

bparees · 2016-10-11T17:47:51Z

@gabemontero @jupierce
so the field labels in the UI are correct (source/dest) but the xml tags are still "test/prod"?

i can live w/ that i guess. confusing for anyone looking at the xml or editing it by hand though.

gabemontero · 2016-10-11T17:50:40Z

yes to your question @bparees

and agreed on the unfortunate confusion originally introduced

jupierce · 2016-10-11T21:39:34Z

ptal @gabemontero @bparees

First step at splitting up the Jenkins plugin tests.
Added DSL tests for multi-tag

So far, no luck getting Jenkins to stay up between tests. I know it is possible, but wanted to get basic tests in place before spending more time on it.

gabemontero

generally looks good

If the separate g.It thing becomes to much of a rabbit hole, I'll back off that request, and we can just launch Jenkins once. I'd still like for the job defs and "tests" to be separated, with different xml files, separate jobs in jenkins, tests separated into different methods that are called from the single g.It, so that if the framework changes, we already have things factored such that it is trivial to map into separate g.It calls.

Sound good?

gabemontero · 2016-10-12T13:03:57Z

test/extended/jenkins/kubernetes_plugin.go

+				host: serviceIP,
+				port: port,
+			}
+
 			jenkinsUri := fmt.Sprintf("http://%s:%s", serviceIP, port)
 			g.By(fmt.Sprintf("wait for jenkins to come up at %q", jenkinsUri))


so is the jenkinsUri just debug at this point? If so, we don't need to bother with constructing an http url string. Just print the ip and port in the g.By

gabemontero · 2016-10-12T13:21:12Z

test/extended/jenkins/plugin.go

+		return "", 0, fmt.Errorf("Unable to build request for uri %q: %v", uri, err)
+	}
+
+	req.SetBasicAuth("admin", "password")


heads up ... assuming my PR ever gets through the merge queue, you'll need to rebase and plug in my retrieval of the password for the jenkins admin user here.

No problem.

gabemontero · 2016-10-12T13:23:20Z

test/extended/jenkins/plugin.go

 	post := exutil.ArtifactPath(filename)
 	err := exutil.VarSubOnFile(pre, post, "PROJECT_NAME", namespace)
-	o.Expect(err).NotTo(o.HaveOccurred())
+	o.ExpectWithOffset(1, err).NotTo(o.HaveOccurred())


why does this distinction re: offset matter?

Offset makes ginkgo show an offset in the call stack when reporting a failure. If you don't use it, you only know the method which failed. If you use it, you can tell exactly which test called the failing method. This is critical to get information about which test failed when a method is being used by many tests.

gabemontero · 2016-10-12T13:33:59Z

test/extended/testdata/jenkins-plugin/build-job.xml

@@ -27,6 +27,22 @@
      <authToken></authToken>
      <followLog>true</followLog>
      <checkForTriggeredDeployments>true</checkForTriggeredDeployments>
+


i had originally envisioned a vanilla build test and and then a test with the build env vars .... i'm assuming you are worried about the time already being spent and wanted to consolidate (of course correct my assumption as needed) .... i could be swayed, but i'm inclined to have separate tests, one for vanilla build, one for build with env vars

My intention was indeed to minimize time, but I've split it into two tests per your request.

ok cool - thx - and yeah, to reiterate what i said in the preface to this latest review, unless you are close or something, definitely switch to the single g.It if that is the by far easiest or only way to allow us to keep Jenkins up for the duration of the tests

bparees · 2016-10-12T15:23:03Z

examples/jenkins/application-template.json

+      "apiVersion": "v1",
+      "metadata": {
+        "name": "origin-nodejs-sample2",
+        "creationTimestamp": null


don't need creationTimestamp either

Removed in latest push.

jupierce · 2016-10-12T15:28:21Z

Adding extended tests for openshift/jenkins-plugin#70 (oc exec support)

gabemontero · 2016-10-12T15:50:03Z

On Wed, Oct 12, 2016 at 11:20 AM, Justin Pierce notifications@github.com
wrote:

@jupierce commented on this pull request.

In test/extended/jenkins/plugin.go
#11252:
post := exutil.ArtifactPath(filename)
err := exutil.VarSubOnFile(pre, post, "PROJECT_NAME", namespace)

o.Expect(err).NotTo(o.HaveOccurred())

o.ExpectWithOffset(1, err).NotTo(o.HaveOccurred())
Offset makes ginkgo show an offset in the call stack when reporting a
failure. If you don't use it, you only know the method which failed. If you
use it, you can tell exactly which test called the failing method. This is
critical to get information about which test failed when a method is being
used by many tests.

Ah - educational / good to know - thx!

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#11252, or mute the thread
https://github.com/notifications/unsubscribe-auth/ADbadEqhf27sBwviiM3OyfNylBPKzlJcks5qzPqugaJpZM4KQNtW
.

gabemontero · 2016-10-12T15:51:08Z

On Wed, Oct 12, 2016 at 11:49 AM, Gabe Montero gmontero@redhat.com wrote:

On Wed, Oct 12, 2016 at 11:20 AM, Justin Pierce notifications@github.com
wrote:

@jupierce commented on this pull request.

In test/extended/jenkins/plugin.go
#11252:

post := exutil.ArtifactPath(filename)
err := exutil.VarSubOnFile(pre, post, "PROJECT_NAME", namespace)

o.Expect(err).NotTo(o.HaveOccurred())

o.ExpectWithOffset(1, err).NotTo(o.HaveOccurred())

Offset makes ginkgo show an offset in the call stack when reporting a
failure. If you don't use it, you only know the method which failed. If you
use it, you can tell exactly which test called the failing method. This is
critical to get information about which test failed when a method is being
used by many tests.

Ah - educational / good to know - thx!

Not with this pull, but perhaps a broad find/replace in our extended tests
are in order - @bparees ?

—

You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#11252, or mute the thread
https://github.com/notifications/unsubscribe-auth/ADbadEqhf27sBwviiM3OyfNylBPKzlJcks5qzPqugaJpZM4KQNtW
.

bparees · 2016-10-12T16:03:04Z

Not with this pull, but perhaps a broad find/replace in our extended tests
are in order - @bparees ?

probably, but in general we've just avoided failing tests inside helper methods to avoid this problem, so i don't think it's pervasive.

jupierce · 2016-10-13T14:11:34Z

Latest push should be close to complete. ptal @gabemontero @bparees

Jenkins is now only started once for the main plugin test spec
Cross namespace multi-tagging is being tested
Exec is being tested

jupierce · 2016-10-13T14:12:46Z

Note that tests will not pass until openshift/jenkins-plugin#87 merges

jupierce · 2016-10-13T15:01:35Z

fyi @csrwng

gabemontero · 2016-10-13T15:21:45Z

@jupierce re: when the test will pass, it is more than just the PR merging (which I just hit the button for). I need to 1) actually cut a new version of the plugin in the official jenkins download center, 2) update our jenkins image to pull in the new version of the plugin when it is ready.

The "non PR testing" version of the plugin extended tests work off of official images/plugins.

We can play it by ear, but I was hoping to pull in the features from @oatmealraisin, as well as the image stream polling tweak @csrwng mentioned, plus my change for hitting the deploy instantiate endpoint. If all goes well, that should happen by the EOW, but I'm tracking closely, and if things get hung up, I'll expedite accordingly.

jupierce · 2016-10-13T15:34:11Z

@gabemontero I can comment out the offending tests if you prefer.

gabemontero · 2016-10-13T15:40:53Z

Let's hold on posting the merge comment for a bit and see if the cutting of
the new plugin release happens soon. If need be I'll pull in the deploy
instantiate change afterwards.

On Thursday, October 13, 2016, Justin Pierce notifications@github.com
wrote:

@gabemontero https://github.com/gabemontero I can comment out the
offending tests if you prefer.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#11252 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADbadJp6UeZc5INGgYQjTBH4lJmnj8SLks5qzk99gaJpZM4KQNtW
.

gabemontero · 2016-10-13T17:21:34Z

@jupierce my fix for the admin password got merged - leading to the merge conflict noted.

jupierce · 2016-10-13T17:26:15Z

@gabemontero rebased with your password logic.

gabemontero

looks good .... only the one item which @jupierce was already aware of and is working on.

gabemontero · 2016-10-13T18:07:29Z

test/extended/jenkins/plugin.go

-		jenkinsApplicationPath := exutil.FixturePath("..", "..", "examples", "jenkins", "application-template.json")
-		err = oc.Run("new-app").Args(jenkinsApplicationPath).Execute()
-		o.Expect(err).NotTo(o.HaveOccurred())
+			oc.Run("new-project").Args(jenkinsNamespace).Execute()


@jupierce is already working on it, but a clean up of the special namespace for jenkins when the tests are done would be best.

I've now convinced myself that there is no clean/reliable way to tear down the namespace in Ginkgo. The framework intentionally disallows discourages this behavior. There is even a comment in their repo about how it would help k8s testing: onsi/ginkgo#70 (comment)

That leaves:
(1) Using Ben's new numeric template parameter, allow template to use ephemeral/unique port for the Jenkins server and leave it up for all tests.
(2) Bring up Jenkins on each test.
(3) Put all tests into the same spec.

I'm personally a fan of (2) at this point, so we can keep the specs clean.

(1) won't be there any time soon.

gabemontero · 2016-10-13T19:31:39Z

On Thu, Oct 13, 2016 at 3:29 PM, Justin Pierce notifications@github.com
wrote:

@jupierce commented on this pull request.

In test/extended/jenkins/plugin.go
#11252:
  var jenkinsEphemeralPath string
var testingSnapshot bool
if len(hexIDs) > 0 && err == nil {
    // found an openshift pipeline plugin test image, must be testing a proposed change to the plugin
    jenkinsEphemeralPath = exutil.FixturePath("testdata", "jenkins-ephemeral-template-test-new-plugin.json")
    testingSnapshot = true
} else {
    // no test image, testing the base jenkins image with the current, supported version of the plugin
    jenkinsEphemeralPath = exutil.FixturePath("..", "..", "examples", "jenkins", "jenkins-ephemeral-template.json")
}
err = oc.Run("new-app").Args(jenkinsEphemeralPath).Execute()
o.Expect(err).NotTo(o.HaveOccurred())
jenkinsApplicationPath := exutil.FixturePath("..", "..", "examples", "jenkins", "application-template.json")
err = oc.Run("new-app").Args(jenkinsApplicationPath).Execute()
o.Expect(err).NotTo(o.HaveOccurred())
    oc.Run("new-project").Args(jenkinsNamespace).Execute()
I've now convinced myself that there is no clean/reliable way to tear down
the namespace in Ginkgo. The framework intentionally disallows discourages
this behavior. There is even a comment in their repo about how it would
help k8s testing: onsi/ginkgo#70 (comment)
onsi/ginkgo#70 (comment)

That leaves:
(1) Using Ben's new numeric template parameter, allow template to use
ephemeral/unique port for the Jenkins server and leave it up for all tests.

Not familiar with this one ... could somebody provide a pointer?

(2) Bring up Jenkins on each test.
(3) Put all tests into the same spec.

I'm personally a fan of (2) at this point, so we can keep the specs clean.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#11252, or mute the thread
https://github.com/notifications/unsubscribe-auth/ADbadOBNEGSFam6TKm8vB0QOjY5YqHsSks5qzoaAgaJpZM4KQNtW
.

bparees · 2016-10-13T19:33:39Z

@gabemontero #11068

jupierce · 2016-10-21T21:32:27Z

@bparees @gabemontero
The liveness probe for Jenkins is presently set to query every 10 seconds. That gives the system about 30 seconds of downtime before k8s kills the pod.

The normal Jenkins workflow of restarting itself whenever plugins are installed could make this problematic. Restarting the server can certainly take more than 30 seconds. This makes it likely that Jenkins will be killed in the process of coming up and installing new plugins.

Perhaps it would be worth extending the period of the liveness probe to 1 minute or so?

gabemontero · 2016-10-21T22:41:38Z

Assuming I remember the particulars of the liveness probe correctly, @jupierce 's rationale for bumping makes sense to me, pending any mitigating factors I'm presently unaware of.

jupierce · 2016-10-22T00:31:23Z

First test with 60% reserved for heap instead of 80% has passed.

bparees · 2016-10-22T02:55:30Z

Perhaps it would be worth extending the period of the liveness probe to 1 minute or so?

@jupierce i assume you're proposing that as a change to the jenkins-ephemeral/persistent templates, nothing to do w/ these extended test changes, right? If so i'm fine with the suggestion.

jupierce · 2016-10-22T03:11:11Z

@derekwaynecarr the liveness probe failures ocurred under artificial load. Given the constrained memory, the probe failures may be acceptable.

jupierce · 2016-10-24T13:50:24Z

Extended tests are now passing consistently with the 0.60 ratio of heap to available memory. This change has now been baked into the s2i jenkins image.

jupierce · 2016-10-24T14:55:26Z

Flake #11517

jupierce · 2016-10-24T14:56:43Z

[testextended][extended:core(openshift pipeline plugin)]

jupierce · 2016-10-24T16:45:57Z

@bparees @gabemontero ptal . travis-ci test is failing due to #11517 . Extended test is successful, but github states it hasn't completed yet, so you will need to visit Jenkins directly to see the complete result.

bparees · 2016-10-24T17:05:06Z

[merge]

openshift-bot · 2016-10-24T17:06:22Z

[Test]ing while waiting on the merge queue

csrwng · 2016-10-25T12:47:07Z

#11452
[merge]

openshift-bot · 2016-10-25T13:21:18Z

Evaluated for origin testextended up to 24eca1b

jupierce · 2016-10-25T13:21:28Z

Rebased.

openshift-bot · 2016-10-25T13:21:29Z

Evaluated for origin test up to 24eca1b

openshift-bot · 2016-10-25T14:37:30Z

continuous-integration/openshift-jenkins/testextended SUCCESS (https://ci.openshift.redhat.com/jenkins/job/test_pr_origin_extended/681/) (Base Commit: 1f62556) (Extended Tests: core(openshift pipeline plugin))

openshift-bot · 2016-10-25T14:45:26Z

continuous-integration/openshift-jenkins/test FAILURE (https://ci.openshift.redhat.com/jenkins/job/test_pr_origin/10611/) (Base Commit: 1f62556)

jupierce · 2016-10-25T14:56:44Z

fyi @oatmealraisin

csrwng · 2016-10-25T15:03:05Z

Flake #11560
[merge]

csrwng · 2016-10-25T16:12:58Z

@bparees can you please tag this pr with the merge tag?

bparees · 2016-10-25T16:16:38Z

[merge]

openshift-bot · 2016-10-25T16:21:27Z

Evaluated for origin merge up to 24eca1b

openshift-bot · 2016-10-25T23:37:28Z

continuous-integration/openshift-jenkins/merge SUCCESS (https://ci.openshift.redhat.com/jenkins/job/test_pr_origin/10651/) (Base Commit: 3f5cad6) (Image: devenv-rhel7_5242)

gabemontero reviewed Oct 6, 2016

View reviewed changes

bparees requested changes Oct 10, 2016

View reviewed changes

gabemontero reviewed Oct 12, 2016

View reviewed changes

bparees reviewed Oct 12, 2016

View reviewed changes

openshift-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Oct 13, 2016

gabemontero reviewed Oct 13, 2016

View reviewed changes

openshift-bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Oct 13, 2016

jupierce changed the title ~~Start-build/env and multitag jenkins plugin tests~~ WIP: Start-build/env and multitag jenkins plugin tests Oct 21, 2016

jupierce changed the title ~~WIP: Start-build/env and multitag jenkins plugin tests~~ Start-build/env and multitag jenkins plugin tests Oct 24, 2016

csrwng mentioned this pull request Oct 24, 2016

Jenkins plugin ImageStream SCM extended tests #11537

Merged

openshift-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Oct 25, 2016

Start-build/env, exec, and multitag jenkins plugin tests

24eca1b

openshift-bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Oct 25, 2016

openshift-bot merged commit ca101d7 into openshift:master Oct 26, 2016

		<prodStream>origin-nodejs-sample</prodStream>
		<prodTag>prod2,prod3,prod4</prodTag>

Start-build/env and multitag jenkins plugin tests #11252

Start-build/env and multitag jenkins plugin tests #11252

Conversation

jupierce commented Oct 6, 2016 • edited Loading

gabemontero Oct 6, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jupierce commented Oct 6, 2016

Choose a reason for hiding this comment

jupierce Oct 14, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gabemontero commented Oct 10, 2016

@bparees commented on this pull request.

jupierce commented Oct 11, 2016

gabemontero commented Oct 11, 2016

jupierce commented Oct 11, 2016

gabemontero commented Oct 11, 2016

bparees commented Oct 11, 2016

gabemontero commented Oct 11, 2016

jupierce commented Oct 11, 2016 • edited Loading

gabemontero left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jupierce commented Oct 12, 2016

gabemontero commented Oct 12, 2016

@jupierce commented on this pull request.

gabemontero commented Oct 12, 2016

@jupierce commented on this pull request.

bparees commented Oct 12, 2016

jupierce commented Oct 13, 2016

jupierce commented Oct 13, 2016

jupierce commented Oct 13, 2016

gabemontero commented Oct 13, 2016

jupierce commented Oct 13, 2016

gabemontero commented Oct 13, 2016

gabemontero commented Oct 13, 2016

jupierce commented Oct 13, 2016

gabemontero left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gabemontero commented Oct 13, 2016

@jupierce commented on this pull request.

bparees commented Oct 13, 2016

jupierce commented Oct 21, 2016

gabemontero commented Oct 21, 2016

jupierce commented Oct 22, 2016

bparees commented Oct 22, 2016

jupierce commented Oct 22, 2016

jupierce commented Oct 24, 2016

jupierce commented Oct 24, 2016

jupierce commented Oct 24, 2016

jupierce commented Oct 24, 2016

bparees commented Oct 24, 2016

openshift-bot commented Oct 24, 2016

csrwng commented Oct 25, 2016

openshift-bot commented Oct 25, 2016

jupierce commented Oct 25, 2016

openshift-bot commented Oct 25, 2016

openshift-bot commented Oct 25, 2016

openshift-bot commented Oct 25, 2016

jupierce commented Oct 25, 2016

jupierce commented Oct 6, 2016 •

edited

Loading

gabemontero Oct 6, 2016 •

edited

Loading

jupierce Oct 14, 2016 •

edited

Loading

jupierce commented Oct 11, 2016 •

edited

Loading

openshift-bot commented Oct 25, 2016 •

edited

Loading