-
Notifications
You must be signed in to change notification settings - Fork 28.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
SPARK-1801. expose InterruptibleIterator and TaskKilledException in deve... #764
SPARK-1801. expose InterruptibleIterator and TaskKilledException in deve... #764
Conversation
Can one of the admins verify this patch? |
Jenkins, test this please. This seems sensible to me. We may be able to get it into a 1.0 rc |
/** | ||
* An iterator that wraps around an existing iterator to provide task killing functionality. | ||
* It works by checking the interrupted flag in [[TaskContext]]. | ||
*/ | ||
private[spark] class InterruptibleIterator[+T](val context: TaskContext, val delegate: Iterator[T]) | ||
@DeveloperApi |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Due to a bug in scaladoc, we also need to add ":: DeveloperApi ::" on the line immediately preceding the first line of comments.
Merged build triggered. |
Merged build started. |
@@ -17,7 +17,10 @@ | |||
|
|||
package org.apache.spark | |||
|
|||
import org.apache.spark.annotation.DeveloperApi | |||
|
|||
/** | |||
* Exception for a task getting killed. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we're exposing this API, perhaps we could add a bit more to the comment, like "Exception thrown when a task is explicitly killed (i.e., task failure is expected)."
Merged build finished. All automated tests passed. |
All automated tests passed. |
…eve... ...loper api Author: Koert Kuipers <koert@tresata.com> Closes #764 from koertkuipers/feat-rdd-developerapi and squashes the following commits: 8516dd2 [Koert Kuipers] SPARK-1801. expose InterruptibleIterator and TaskKilledException in developer api (cherry picked from commit b22952f) Signed-off-by: Aaron Davidson <aaron@databricks.com>
Merged into master and branch-1.0 in order to cut the release. I updated the javadoc format during merge. |
…eve... ...loper api Author: Koert Kuipers <koert@tresata.com> Closes apache#764 from koertkuipers/feat-rdd-developerapi and squashes the following commits: 8516dd2 [Koert Kuipers] SPARK-1801. expose InterruptibleIterator and TaskKilledException in developer api
...loper api