Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[JENKINS-49073] Parallel step should propagate the worst result when not using failFast #325

Merged
merged 20 commits into from
Nov 26, 2019
Merged
Show file tree
Hide file tree
Changes from 15 commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -167,6 +167,12 @@
<version>1.11</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.jenkins-ci.plugins</groupId>
<artifactId>pipeline-build-step</artifactId>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unnecessary I think; pending JENKINS-27092, can use non-sandboxed scripts which directly throw new FlowInterruptedException(…) for tests.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps this isn't necessary, but I think it makes the test more realistic. I saw that we're already pulling in pipeline-input-step, so pulling in pipeline-build-step as well didn't seem like a big price to pay for the benefit of more realistic integration testing. I think I'd rather keep this as-is, but let me know if you feel strongly otherwise.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No strong feeling.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personally, I'd lean towards the FlowInterruptedException approach since it would make parallelPropagatesStatusImpl much easier to read, but I think either approach is fine.

<version>2.10-rc366.8d263648aa27</version> <!-- TODO: Pending release of https://github.com/jenkinsci/pipeline-build-step-plugin/pull/24 -->
basil marked this conversation as resolved.
Show resolved Hide resolved
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.jenkins-ci.plugins</groupId>
<artifactId>pipeline-input-step</artifactId>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,23 +3,29 @@
import com.cloudbees.groovy.cps.Outcome;

import groovy.lang.Closure;
import hudson.AbortException;
import hudson.Extension;
import hudson.model.Result;
import hudson.model.TaskListener;
import java.io.IOException;

import org.jenkinsci.plugins.workflow.cps.CpsVmThreadOnly;
import org.jenkinsci.plugins.workflow.cps.persistence.PersistIn;
import org.jenkinsci.plugins.workflow.steps.BodyExecutionCallback;
import org.jenkinsci.plugins.workflow.steps.FlowInterruptedException;
import org.jenkinsci.plugins.workflow.steps.Step;
import org.jenkinsci.plugins.workflow.steps.StepContext;
import org.jenkinsci.plugins.workflow.steps.StepDescriptor;
import org.jenkinsci.plugins.workflow.steps.StepExecution;

import java.io.Serializable;
import java.util.AbstractMap.SimpleEntry;
import java.util.ArrayList;
import java.util.Collections;
import java.util.Comparator;
import java.util.HashMap;
import java.util.LinkedHashMap;
import java.util.LinkedHashSet;
import java.util.List;
import java.util.Map;
import java.util.Map.Entry;
import java.util.Set;
Expand Down Expand Up @@ -63,16 +69,20 @@ public StepExecution start(StepContext context) throws Exception {
return failFast;
}


@PersistIn(PROGRAM)
static class ResultHandler implements Serializable {
private final StepContext context;
private final ParallelStepExecution stepExecution;
private final boolean failFast;
/** Have we called stop on the StepExecution? */
private boolean stopSent = false;
/** if we failFast we need to record the first failure */
private SimpleEntry<String,Throwable> originalFailure = null;
/**
* If we fail fast, we need to record the first failure.
*
* <p>We use a set because we may be encountering the same abort being delivered across
* branches. We use a linked hash set to maintain insertion order.
*/
private final LinkedHashSet<Throwable> failures = new LinkedHashSet<>();

/**
* Collect the results of sub-workflows as they complete.
Expand Down Expand Up @@ -123,14 +133,7 @@ public void onFailure(StepContext context, Throwable t) {
} catch (IOException | InterruptedException x) {
LOGGER.log(Level.WARNING, null, x);
}
if (handler.originalFailure == null) {
handler.originalFailure = new SimpleEntry<>(name, t);
} else {
Throwable originalT = handler.originalFailure.getValue();
if (t != originalT) { // could be the same abort being delivered across branches
originalT.addSuppressed(t);
}
}
handler.failures.add(t);
checkAllDone(true);
}

Expand All @@ -153,18 +156,26 @@ private void checkAllDone(boolean stepFailed) {
return;
}
if (o.isFailure()) {
if (handler.originalFailure == null) {
if (handler.failures.isEmpty()) {
// in case the plugin is upgraded whilst a parallel step is running
handler.originalFailure = new SimpleEntry<>(e.getKey(), e.getValue().getAbnormal());
handler.failures.add(e.getValue().getAbnormal());
}
// recorded in the onFailure
} else {
success.put(e.getKey(), o.getNormal());
}
}
// all done
if (handler.originalFailure!=null) {
handler.context.onFailure(handler.originalFailure.getValue());
List<Throwable> toAttach = new ArrayList<>(handler.failures);
if (!handler.failFast) {
Collections.sort(toAttach, new ThrowableComparator(new ArrayList<>(handler.failures)));
}
if (!toAttach.isEmpty()) {
Throwable head = toAttach.get(0);
for (int i = 1; i < toAttach.size(); i++) {
head.addSuppressed(toAttach.get(i));
}
handler.context.onFailure(head);
} else {
handler.context.onSuccess(success);
}
Expand All @@ -173,6 +184,62 @@ private void checkAllDone(boolean stepFailed) {
private static final long serialVersionUID = 1L;
}

static final class ThrowableComparator implements Comparator<Throwable>, Serializable {
basil marked this conversation as resolved.
Show resolved Hide resolved

private final List<Throwable> insertionOrder;

ThrowableComparator() {
this.insertionOrder = new ArrayList<>();
}

ThrowableComparator(List<Throwable> insertionOrder) {
this.insertionOrder = insertionOrder;
}

@Override
public int compare(Throwable t1, Throwable t2) {
if (!(t1 instanceof FlowInterruptedException)
&& t2 instanceof FlowInterruptedException) {
// FlowInterruptedException is always higher than any other exception.
return -1;
} else if (t1 instanceof FlowInterruptedException
&& !(t2 instanceof FlowInterruptedException)) {
// FlowInterruptedException is always higher than any other exception.
return 1;
} else if (!(t1 instanceof AbortException) && t2 instanceof AbortException) {
// AbortException is always higher than any exception other than
// FlowInterruptedException.
return -1;
} else if (t1 instanceof AbortException && !(t2 instanceof AbortException)) {
// AbortException is always higher than any exception other than
// FlowInterruptedException.
return 1;
} else if (t1 instanceof FlowInterruptedException
&& t2 instanceof FlowInterruptedException) {
// Two FlowInterruptedExceptions are compared by their results.
FlowInterruptedException fie1 = (FlowInterruptedException) t1;
FlowInterruptedException fie2 = (FlowInterruptedException) t2;
Result r1 = fie1.getResult();
Result r2 = fie2.getResult();
if (r1.isWorseThan(r2)) {
return -1;
} else if (r1.isBetterThan(r2)) {
return 1;
}
} else if (insertionOrder.contains(t1) && insertionOrder.contains(t2)) {
// Break ties by insertion order. Earlier errors are worse.
int index1 = insertionOrder.indexOf(t1);
int index2 = insertionOrder.indexOf(t2);
if (index1 < index2) {
return -1;
} else if (index1 > index2) {
return 1;
}
}
return 0;
}
}

private static final long serialVersionUID = 1L;
}

Expand Down
Loading