Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Save information about child jobs in parent job #13

Closed
wants to merge 6 commits into from
Closed
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -126,5 +126,17 @@
<version>2.2</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.powermock</groupId>
<artifactId>powermock-module-junit4</artifactId>
<version>1.7.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.powermock</groupId>
<artifactId>powermock-api-mockito2</artifactId>
<version>1.7.0</version>
<scope>test</scope>
</dependency>
</dependencies>
</project>
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
package org.jenkinsci.plugins.workflow.support.steps.build;

import hudson.model.Job;
import hudson.model.Run;
import jenkins.model.Jenkins;
import hudson.model.InvisibleAction;

import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.Queue;
import java.util.concurrent.ConcurrentLinkedQueue;


public class BuildInfoAction extends InvisibleAction {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see no problem with making it visible, but maybe it's outside the scope

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can put this comment to the code ;)

private final Queue<BuildInfo> buildInfos = new ConcurrentLinkedQueue<>();

BuildInfoAction() {
}

BuildInfoAction(String projectName, int buildNumber) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Useless constructor, better deleted.

buildInfos.add(new BuildInfo(projectName, buildNumber));
}

public List<Run<?, ?>> getChildBuilds() {
if (buildInfos.isEmpty()) {
return Collections.emptyList();
}

List<Run<?, ?>> builds = new ArrayList<>(buildInfos.size());

for (BuildInfo buildInfo : buildInfos) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor optimization: Return Collections.emptyList() if the size is null before creating a new ArrayList. Maybe also makes sense to pre-initialize builds size to buildInfos.size

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if ConcurrentLinkedQueue.size() is fast enough though. Maybe it will hammer the performance

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ConcurrentLinkedQueue.isEmpty() comes for free, but in current implementation it's not possible to have empty collection of buildInfos longer than several ms (+ StopTheWorld pause :) ). Anyway, I added this check anyway - who knows how this Action will be used in future.
Pre-initialize - good idea, can decrease memory footprint a little bit.

if (buildInfo != null) {
Job job = Jenkins.getActiveInstance().getItemByFullName(buildInfo.getProjectName(), Job.class);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unclear whether this should be impersonating SYSTEM. Who is calling it? Perhaps better to return the raw data and let the caller decide how to handle missing information.

if (buildInfo.getBuildNumber() != 0) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Huh? Why would the build number be zero?

builds.add((job != null) ? job.getBuildByNumber(buildInfo.getBuildNumber()) : null);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the job is missing, do not add null.

And getBuildByNumber may well add null; do not add that.

}
}
}

return builds;
}

public void addBuildInfo(String projectName, int buildNumber) {
buildInfos.add(new BuildInfo(projectName, buildNumber));
}

private static class BuildInfo {
private final String projectName;
private final int buildNumber;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Easier just to use String via Run.externalizableId.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does not look easier for me, in this case I would need one String concatenation + String.split for every call for nothing. I now that it's tiny stuff, but even from code perspective it doesn't simplify much.


BuildInfo(String projectName, int buildNumber) {
this.projectName = projectName;
this.buildNumber = buildNumber;
}

String getProjectName() {
return projectName;
}

int getBuildNumber() {
return buildNumber;
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
import hudson.model.TaskListener;
import hudson.model.listeners.RunListener;
import java.util.logging.Level;
import org.jenkinsci.plugins.workflow.graph.FlowNode;
import org.jenkinsci.plugins.workflow.steps.StepContext;

import java.util.logging.Logger;
Expand All @@ -29,6 +30,11 @@ public void onStarted(Run<?, ?> run, TaskListener listener) {
TaskListener taskListener = stepContext.get(TaskListener.class);
// encodeTo(Run) calls getDisplayName, which does not include the project name.
taskListener.getLogger().println("Starting building: " + ModelHyperlinkNote.encodeTo("/" + run.getUrl(), run.getFullDisplayName()));
FlowNode parentNode = stepContext.get(FlowNode.class);
if (parentNode != null) {
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@oleg-nenashev, I suppose that this code is not thread-safe as well. Is it correct?
I think to put synchronized(this) for addAction/getAction

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to check for null if defined via @StepContextParameter (which is actually deprecated but fixing that requires a larger change).

BuildInfoAction buildInfoAction = ensureBuildInfoAction(parentNode);
buildInfoAction.addBuildInfo(run.getParent().getFullName(), run.getNumber());
}
} catch (Exception e) {
LOGGER.log(WARNING, null, e);
}
Expand All @@ -38,6 +44,20 @@ public void onStarted(Run<?, ?> run, TaskListener listener) {
}
}

private BuildInfoAction ensureBuildInfoAction(final FlowNode parentNode) {
BuildInfoAction buildInfoAction = parentNode.getAction(BuildInfoAction.class);
if (buildInfoAction == null) {
synchronized (this) {
buildInfoAction = parentNode.getAction(BuildInfoAction.class);
if (buildInfoAction == null) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Double-checked locking, unsafe.

buildInfoAction = new BuildInfoAction();
parentNode.addAction(buildInfoAction);
}
}
}
return buildInfoAction;
}

@Override
@SuppressWarnings("deprecation") // TODO 2.30+ use removeAction
public void onCompleted(Run<?,?> run, @Nonnull TaskListener listener) {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
package org.jenkinsci.plugins.workflow.support.steps.build;

import hudson.model.Job;
import jenkins.model.Jenkins;
import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.mockito.Mockito;

import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;

import static org.mockito.Matchers.any;
import static org.mockito.Matchers.same;
import static org.junit.Assert.assertEquals;
import static org.powermock.api.mockito.PowerMockito.when;

import org.powermock.api.mockito.PowerMockito;
import org.powermock.core.classloader.annotations.PrepareForTest;
import org.powermock.modules.junit4.PowerMockRunner;

@RunWith(PowerMockRunner.class)
@PrepareForTest(Jenkins.class)
public class BuildInfoActionTest {
private Jenkins jenkins;

@Before
public void setUp() {
PowerMockito.mockStatic(Jenkins.class);
jenkins = PowerMockito.mock(Jenkins.class);
PowerMockito.when(Jenkins.getInstance()).thenReturn(jenkins);
PowerMockito.when(Jenkins.getActiveInstance()).thenReturn(jenkins);
}

@Test
public void testConcurrentAccess() throws Exception {
Job item = Mockito.mock(Job.class);
when(jenkins.getItemByFullName(any(String.class), same(Job.class))).thenReturn(item);

BuildInfoAction buildInfoAction = new BuildInfoAction("firstOne", 1);

ExecutorService executors = Executors.newFixedThreadPool(5);
Runnable task1 = createRunnable("A", buildInfoAction);
Runnable task2 = createRunnable("B", buildInfoAction);
Runnable task3 = createRunnable("C", buildInfoAction);
Runnable task4 = createRunnable("D", buildInfoAction);
Runnable task5 = createRunnable("E", buildInfoAction);

Future future1 = executors.submit(task1);
Future future2 = executors.submit(task2);
Future future3 = executors.submit(task3);
Future future4 = executors.submit(task4);
Future future5 = executors.submit(task5);

while (true) {
if (future1.isDone() && future2.isDone() && future3.isDone() && future4.isDone() && future5.isDone() ) {
break;
}

Thread.sleep(100);
}

assertEquals(501, buildInfoAction.getChildBuilds().size());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test does not seem to be proving anything very interesting. I would just delete it. (PowerMock is a pain in the butt to maintain for Jenkins plugins anyway.)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right now I used PowerMock only for Jenkins singleton, for the rest of stuff I used Mockito. What do you prefer to use for mocking Jenkins.getInstance method?

}

private Runnable createRunnable(final String name, final BuildInfoAction action) {
return new Runnable() {
@Override
public void run() {
for (int i = 0; i < 100; i++) {
action.addBuildInfo(name, i + 1);
try {
Thread.sleep(10);
} catch (InterruptedException ignored) {
}
}
}
};
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,11 @@
import hudson.model.ParametersDefinitionProperty;
import hudson.model.Queue;
import hudson.model.Result;
import hudson.model.Run;
import hudson.model.StringParameterDefinition;
import hudson.model.TaskListener;
import hudson.model.User;
import hudson.model.queue.QueueTaskFuture;
import hudson.security.AuthorizationStrategy;
import java.io.IOException;
import java.util.Arrays;
import java.util.Collections;
Expand All @@ -40,6 +40,8 @@
import static org.hamcrest.Matchers.nullValue;
import org.jenkinsci.plugins.workflow.cps.CpsFlowDefinition;
import org.jenkinsci.plugins.workflow.cps.CpsFlowExecution;
import org.jenkinsci.plugins.workflow.graph.FlowGraphWalker;
import org.jenkinsci.plugins.workflow.graph.FlowNode;
import org.jenkinsci.plugins.workflow.job.WorkflowJob;
import org.jenkinsci.plugins.workflow.job.WorkflowRun;
import org.jenkinsci.plugins.workflow.test.steps.SemaphoreStep;
Expand Down Expand Up @@ -78,20 +80,31 @@ public class BuildTriggerStepTest {
us.setDefinition(new CpsFlowDefinition(
"def ds = build 'ds'\n" +
"echo \"ds.result=${ds.result} ds.number=${ds.number}\"", true));
j.assertLogContains("ds.result=SUCCESS ds.number=1", j.buildAndAssertSuccess(us));
WorkflowRun usRun = j.buildAndAssertSuccess(us);
j.assertLogContains("ds.result=SUCCESS ds.number=1", usRun);
assertBuildInfoAction(usRun, "ds", 1);

// TODO JENKINS-28673 assert no warnings, as in StartupTest.noWarnings
// (but first need to deal with `WARNING: Failed to instantiate optional component org.jenkinsci.plugins.workflow.steps.scm.SubversionStep$DescriptorImpl; skipping`)
ds.getBuildByNumber(1).delete();

}

@Issue("JENKINS-25851")
@Test public void failingBuild() throws Exception {
j.createFreeStyleProject("ds").getBuildersList().add(new FailureBuilder());
FreeStyleProject ds = j.createFreeStyleProject("ds");
ds.getBuildersList().add(new FailureBuilder());
WorkflowJob us = j.jenkins.createProject(WorkflowJob.class, "us");
us.setDefinition(new CpsFlowDefinition("build 'ds'", true));
j.assertBuildStatus(Result.FAILURE, us.scheduleBuild2(0));
WorkflowRun usRun = j.assertBuildStatus(Result.FAILURE, us.scheduleBuild2(0));
assertBuildInfoAction(usRun, "ds", ds.getLastBuild().getNumber());

us.setDefinition(new CpsFlowDefinition("echo \"ds.result=${build(job: 'ds', propagate: false).result}\"", true));
j.assertLogContains("ds.result=FAILURE", j.buildAndAssertSuccess(us));

usRun = j.buildAndAssertSuccess(us);
j.assertLogContains("ds.result=FAILURE", usRun);

assertBuildInfoAction(usRun, "ds", ds.getLastBuild().getNumber());
}

@SuppressWarnings("deprecation")
Expand Down Expand Up @@ -124,7 +137,10 @@ public void buildParallelTests() throws Exception {
" build('test2');\n" +
" })"), "\n"), true));

j.buildAndAssertSuccess(foo);
WorkflowRun usRun = j.buildAndAssertSuccess(foo);

assertBuildInfoAction(usRun, "test1", p1.getLastBuild().getNumber());
assertBuildInfoAction(usRun, "test2", p2.getLastBuild().getNumber());
}


Expand All @@ -150,7 +166,11 @@ public void abortBuild() throws Exception {
fb.getExecutor().interrupt();

j.assertBuildStatus(Result.ABORTED, j.waitForCompletion(fb));
j.assertBuildStatus(Result.FAILURE,q.get());

WorkflowRun usRun = q.get();
j.assertBuildStatus(Result.FAILURE, q.get());

assertBuildInfoAction(usRun, "test1", p.getLastBuild().getNumber());
}

@Test
Expand Down Expand Up @@ -430,6 +450,23 @@ public MultiBranchProjectFactory newInstance() {
}
}

private void assertBuildInfoAction(WorkflowRun workflowRun, String projectName, int buildNumber) {
FlowGraphWalker walker = new FlowGraphWalker(workflowRun.getExecution());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🐜 Deprecated, use DepthFirstScanner instead

Copy link
Author

@Jimilian Jimilian Aug 7, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It needs workflow-api:2.17, so, I updated it in pom.xml as well, but now it fails in verification: https://ci.jenkins.io/blue/organizations/jenkins/Plugins%2Fpipeline-build-step-plugin/detail/PR-13/9/tests/
Also, I think @jglick would not be happy to introduce dependency for higher version (especially in case of test code needs it -> not production).
Also I didn't find tag @Deprecated in constructor of FlowGraphWalker class in new API.

@svanoort, @jglick, @oleg-nenashev So, what should I do?


for(FlowNode step : walker) {
BuildInfoAction buildInfoAction = step.getAction(BuildInfoAction.class);
if (buildInfoAction != null) {
for (Run child : buildInfoAction.getChildBuilds()) {
if (child.getNumber() == buildNumber && child.getParent().getFullName().equals(projectName)) {
return;
}
}
}
}

throw new AssertionError("Didn't find expected buildInfoAction");
}

@Issue("SECURITY-433")
@Test public void permissions() throws Exception {
WorkflowJob us = j.jenkins.createProject(WorkflowJob.class, "us");
Expand All @@ -451,5 +488,4 @@ public MultiBranchProjectFactory newInstance() {
j.jenkins.setAuthorizationStrategy(new MockAuthorizationStrategy().grant(Jenkins.READ, Computer.BUILD).everywhere().to("dev").grant(Item.DISCOVER).onItems(ds).to("dev"));
j.assertLogContains("Please login to access job ds", j.assertBuildStatus(Result.FAILURE, us.scheduleBuild2(0)));
}

}