SPARK-1801. expose InterruptibleIterator and TaskKilledException in deve... #764
Conversation
|
Can one of the admins verify this patch? |
|
Jenkins, test this please. This seems sensible to me. We may be able to get it into a 1.0 rc |
| /** | ||
| * An iterator that wraps around an existing iterator to provide task killing functionality. | ||
| * It works by checking the interrupted flag in [[TaskContext]]. | ||
| */ | ||
| private[spark] class InterruptibleIterator[+T](val context: TaskContext, val delegate: Iterator[T]) | ||
| @DeveloperApi |
There was a problem hiding this comment.
Due to a bug in scaladoc, we also need to add ":: DeveloperApi ::" on the line immediately preceding the first line of comments.
|
Merged build triggered. |
|
Merged build started. |
| @@ -17,7 +17,10 @@ | |||
|
|
|||
| package org.apache.spark | |||
|
|
|||
| import org.apache.spark.annotation.DeveloperApi | |||
|
|
|||
| /** | |||
| * Exception for a task getting killed. | |||
There was a problem hiding this comment.
If we're exposing this API, perhaps we could add a bit more to the comment, like "Exception thrown when a task is explicitly killed (i.e., task failure is expected)."
|
Merged build finished. All automated tests passed. |
|
All automated tests passed. |
…eve... ...loper api Author: Koert Kuipers <koert@tresata.com> Closes #764 from koertkuipers/feat-rdd-developerapi and squashes the following commits: 8516dd2 [Koert Kuipers] SPARK-1801. expose InterruptibleIterator and TaskKilledException in developer api (cherry picked from commit b22952f) Signed-off-by: Aaron Davidson <aaron@databricks.com>
|
Merged into master and branch-1.0 in order to cut the release. I updated the javadoc format during merge. |
…eve... ...loper api Author: Koert Kuipers <koert@tresata.com> Closes apache#764 from koertkuipers/feat-rdd-developerapi and squashes the following commits: 8516dd2 [Koert Kuipers] SPARK-1801. expose InterruptibleIterator and TaskKilledException in developer api
...loper api