[SPARK-24251][SQL] Add AppendData logical plan. #21305

rdblue · 2018-05-11T21:05:03Z

What changes were proposed in this pull request?

This adds a new logical plan, AppendData, that was proposed in SPARK-23521: Standardize SQL logical plans.

DataFrameWriter uses the new AppendData plan for DataSourceV2 appends
AppendData is resolved if its output columns match the incoming data frame
A new analyzer rule, ResolveOutputColumns, validates data before it is appended. This rule will add safe casts, rename columns, and checks nullability

How was this patch tested?

Existing tests for v2 appends. Will add AppendData tests to validate logical plan analysis.

SparkQA · 2018-05-11T21:19:27Z

Test build #90530 has finished for PR 21305 at commit a3bf530.

This patch fails to build.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2018-05-12T02:04:37Z

Test build #90535 has finished for PR 21305 at commit d2e4c41.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

bersprockets · 2018-05-17T13:39:59Z

...alyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala

+/**
+ * Append data to an existing DataSourceV2 table.
+ */
+case class AppendData(


How does this logical plan node map to the 8 operations outlined in your SPIP?

This is InsertInto. Wenchen wanted to call it Append because that is more specific.

I updated the SPIP doc to have both names.

jose-torres

I actually really like this. I had thought the standardization would end up being a more invasive change than this. (Although I guess AppendData is the node with the simplest mapping from writer behavior.)

jose-torres · 2018-05-24T18:56:31Z

...alyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala

+ * Append data to an existing DataSourceV2 table.
+ */
+case class AppendData(
+    table: LogicalPlan,


Do we still want to represent the table as a generic LogicalPlan rather than a separate table type? (Maybe the answer's yes; I'm not really clear on why this happens in the current code.)

Yes. These plans should not be specific to DataSourceV2Relation. In the long term, we want any other relations using AppendData to automatically have the same validation rules applied. That means that AppendData should be generic and not care what the table implementation is.

+1 on this. We will still keep data source v1 and file format(for fallback), which doesn't use v2 relation.

Then seems that above code comment may be updated?

jose-torres · 2018-05-24T19:00:04Z

sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala

+
+          } else {
+            val writer = ws.createWriter(
+              UUID.randomUUID.toString, df.logicalPlan.output.toStructType, mode,


nit: probably put the timestamp back to minimize the change

+1. The timestamp was added to reduce the possibility of conflicting job ids.

How would random UUIDs conflict?

it's very unlikely to conflict, but UUID+timestamp is even more unlikely, isn't it? I feel it's safer the keep the original logic, e.g. we may have users to look at the temporary directory name and see when the write job was started.

I see no good reason to over-complicate the unique string passed in. Here's a quote from wikipedia on the chance of a conflict (from this SO answer):

Only after generating 1 billion UUIDs every second for the next 100 years, the probability of creating just one duplicate would be about 50%. Or, to put it another way, the probability of one duplicate would be about 50% if every person on earth owned 600 million UUIDs.

Adding timestamp to a UUID to avoid collisions is unnecessary.

For the other use, why would a user go to the temp directory of some node's file system -- which may not even be used by a given source -- instead of going to the logs? What if the user wants any other piece of information besides the starting timestamp (that's in some format that has to be converted)?

In short, I don't agree with the argument that it is helpful to pass the old format. This is just a carry-over from making fake Hadoop job IDs (why it was called jobId and started with job_). It's debatable whether the write UUID itself is even useful given that there is no requirement to use it anywhere.

rdblue · 2018-07-04T22:35:22Z

@cloud-fan, can you also review this PR for DataSourceV2?

This adds the first of the logical plans proposed in SPIP: Standardize Logical Plans: AppendData. It replaces the current logical node for batch writes and adds schema validation.

This can be done in parallel with #21306 because it doesn't rely on those changes. The other logical operations, like DeleteFrom and CreateTableAsSelect will depend on it though.

SparkQA · 2018-07-05T02:15:30Z

Test build #92627 has finished for PR 21305 at commit b906ab1.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

cloud-fan · 2018-07-05T04:22:06Z

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala

@@ -172,6 +173,7 @@ class Analyzer(
      ResolveWindowOrder ::
      ResolveWindowFrame ::
      ResolveNaturalAndUsingJoin ::
+      ResolveOutputRelation ::


I feel this rule doesn't need to be in the giant resolution batch. Shall we put it in the Post-Hoc Resolution batch which run only once?

This rule may add Projection, UpCast, and Alias nodes to the plan, so there are some rules in this batch that should be run after the output is resolved. ResolveUpCast will rewrite the casts that were inserted and throw exceptions if the cast would truncate and needs to run after this rule.

I could create a batch just after resolution for output resolution. We could just run this rule and ResolveUpCast. I think the optimizer will handle collapsing Projection nodes and aliases are only resolved in this batch, so adding resolved aliases shouldn't be a problem.

Would you like a separate batch for output resolution?

ah ok then let's just keep it here

cloud-fan · 2018-07-05T04:23:44Z

...core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala


  import DataSourceV2Relation._

+  override def name: String = {
+    tableIdent.map(_.unquotedString).getOrElse("unknown")


shall we use data source short name or its full class name instead of unknown?

That's the name of the data source, not the name of the table. I'd be fine with updating this if you want to include the source name. What about s"${source.name}:unknown"?

cloud-fan · 2018-07-05T04:25:25Z

sql/core/src/test/scala/org/apache/spark/sql/sources/v2/DataSourceV2Suite.scala

@@ -203,33 +203,33 @@ class DataSourceV2Suite extends QueryTest with SharedSQLContext {
        val path = file.getCanonicalPath
        assert(spark.read.format(cls.getName).option("path", path).load().collect().isEmpty)

-        spark.range(10).select('id, -'id).write.format(cls.getName)
+        spark.range(10).select('id as 'i, -'id as 'j).write.format(cls.getName)


are these required changes?

Yes. The new resolution rule validates the dataframe that will be written to the table.

Because this uses the DataFrameWriter API, it matches columns by name because there isn't a strong expectation for ordering in the dataframe API (e.g. withColumn doesn't specify where the new column is added).

cloud-fan · 2018-07-05T04:33:58Z

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala

+
+    private def upcast(inAttr: NamedExpression, outAttr: Attribute): NamedExpression = {
+      Alias(
+        UpCast(inAttr, outAttr.dataType, Seq()), outAttr.name


Since Upcast is being used in more places, I wanna raise #21586 again.

The current Upcast has a bug that it forbids string-to-int, but allows string-to-boolean. Changing it is It's a behavior change so we should make sure the behavior is reasonable. What's our expectation of Upcast here? Are we expecting runtime error when casting string to other types?

The purpose of UpCast here is to prevent Spark from automatically inserting casts that could lose information, like long to int or string to int.

I would support the same for string to boolean to catch destructive problems from accidental column alignment (in SQL) or similar errors. The main problem here is that Spark inserts casts instead of alerting the user that there's a problem. When the write succeeds, it may be a while before the user realizes the mistake and can't recover the original data.

rdblue · 2018-07-06T16:53:52Z

@cloud-fan, I've updated this with the requested changes. Thanks for looking at it!

SparkQA · 2018-07-06T20:51:47Z

Test build #92693 has finished for PR 21305 at commit 487d864.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

cloud-fan · 2018-07-07T04:10:54Z

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala

+   */
+  object ResolveOutputRelation extends Rule[LogicalPlan] {
+    override def apply(plan: LogicalPlan): LogicalPlan = plan transform {
+      case append @ AppendData(table: NamedRelation, query, isByName)


Since we don't have an analyzer rule for ApendData(LogicalPlan ...), shall we just require AppendData.table to be a NamedRelation in class definition?

Yes, I agree.

cloud-fan · 2018-07-07T04:48:38Z

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala

+        expected.flatMap { outAttr =>
+          query.resolveQuoted(outAttr.name, resolver) match {
+            case Some(inAttr) if inAttr.nullable && !outAttr.nullable =>
+              errors += s"Cannot write nullable values to non-null column '${outAttr.name}'"


shall we add a runtime null check instead of failing at the beginning?

I would much rather have a job fail fast and give a clear error message than to fail during a write. I can see how adding such an assertion to the plan could be useful, so I'd consider it if someone wanted to add that feature later. Right now, though, I think this is good.

cloud-fan · 2018-07-07T04:49:18Z

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala

+              errors += s"Cannot write nullable values to non-null column '${outAttr.name}'"
+              None
+
+            case Some(inAttr) if !outAttr.dataType.sameType(inAttr.dataType) =>


sameType ignores nullability, shall we add null check for nested fields too?

Yes, I'll update to check nested fields.

cloud-fan · 2018-07-07T04:50:03Z

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala

+        }
+
+      } else {
+        if (expected.size > query.output.size) {


this check is already done in https://github.com/apache/spark/pull/21305/files#diff-57b3d87be744b7d79a9beacf8e5e5eb2R2152

That check is the other direction: not enough columns.

When matching by position, we need to have the same number of columns so we add this check (we already know that there aren't too few columns, so this checks for too many). When matching by name, we can call out specific columns that are missing, which is why we do the validation differently for the two cases.

cloud-fan · 2018-07-07T04:51:06Z

sql/core/src/main/java/org/apache/spark/sql/sources/v2/WriteSupport.java

-   * @param jobId A unique string for the writing job. It's possible that there are many writing
-   *              jobs running at the same time, and the returned {@link DataSourceWriter} can
-   *              use this job id to distinguish itself from other jobs.
+   * @param writeUUID A unique string for the writing job. It's possible that there are many writing


what's the rationale behind this renaming?

This is not the ID of the Spark job that is writing. I think the UUID name is more clear about what is actually passed, a unique string that identifies the write. There's also no need to make the string more complicated than a UUID since there are no guarantees about it.

This is removed in the v2 API redesign.

SparkQA · 2018-07-09T02:42:30Z

Test build #92723 has finished for PR 21305 at commit 222d097.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
trait NamedRelation extends LogicalPlan

rdblue · 2018-07-09T16:01:58Z

@cloud-fan, I've updated this and the tests are passing, so I think it is ready for another look.

I just pushed a comments-only commit to fix the Javadoc for AppendData that @viirya pointed out (thanks!). Since that's only comments, it shouldn't affect test results.

I think there's just one more point under discussion, which is the change from jobId (which isn't one) to writeUUID and dropping the timestamp. I don't think a timestamp helps avoid conflicts because it is astronomically unlikely that UUIDs will collide and, ironically, much more likely that the timestamps will.

SparkQA · 2018-07-09T19:06:48Z

Test build #92761 has finished for PR 21305 at commit 65c2670.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

rdblue · 2018-07-10T00:15:36Z

Retest this please.

rdblue · 2018-07-10T00:16:18Z

@cloud-fan, I think we can ignore that last test failure because tests are passing on the last commit that made real changes. The latest commit only changed a comment.

jose-torres · 2018-07-10T00:19:14Z

(The last test failure is a known flaky test I've been working (albeit unsuccessfully so far) to find a solution for.)

SparkQA · 2018-07-10T04:21:34Z

Test build #92780 has finished for PR 21305 at commit 65c2670.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

cloud-fan · 2018-07-11T15:35:18Z

Do we have a plan to update the data source v2 write API to support ReplaceData, DeleteData? It seems a simple WriteMode flag is not enough for the write API.

rdblue · 2018-07-11T16:05:17Z

@cloud-fan, yes. There is an open PR, #21308, that adds DeleteSupport. I'm not pushing for that just yet because I think DeleteSupport should be applied to Table after #21306 makes it in.

cloud-fan · 2018-07-11T16:15:35Z

cool! so WriteSupport will be used for AppendData, CTAS and RTAS? Do we still need the SaveMode after we standardize the DDL/DML logical plans?

rdblue · 2018-07-11T16:26:11Z

I don't think we need (or want) SaveMode passed to writers after standardizing. Uses of WriteSupport will always append data to an existing table, which makes it simpler for writers. And it will be used for all writes.

A couple other notes:

We will also need a StagedTable variant of DeleteSupport to support ReplaceData as an atomic operation, but I want to get the non-atomic variants in first; hopefully for 2.4.0.
RTAS would use dropTable and not DeleteSupport, since it is at the table level.
We may still use SaveMode in the DF writer API, which is still under discussion.

rdblue · 2018-07-17T23:57:46Z

@cloud-fan, I've rebased this so it is ready for final review when you get a chance. Thanks!

SparkQA · 2018-07-18T03:54:38Z

Test build #93208 has finished for PR 21305 at commit f041019.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Previously missing fields were allowed if the read type's corresponding field was optional.

rdblue · 2018-08-07T04:41:45Z

@cloud-fan, I've rebased and updated with the requested change to disallow missing columns, even if they're optional. Thanks for reviewing!

cloud-fan · 2018-08-07T05:03:44Z

LGTM, pending jenkins

SparkQA · 2018-08-07T07:05:02Z

Test build #94339 has finished for PR 21305 at commit e81790d.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

cloud-fan · 2018-08-07T07:46:22Z

retest this please

SparkQA · 2018-08-07T09:32:20Z

Test build #94352 has finished for PR 21305 at commit e81790d.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

cloud-fan · 2018-08-07T09:46:47Z

retest this please

SparkQA · 2018-08-07T12:55:17Z

Test build #94364 has finished for PR 21305 at commit e81790d.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

cloud-fan · 2018-08-07T13:00:22Z

retest this please

SparkQA · 2018-08-07T16:58:38Z

Test build #94373 has finished for PR 21305 at commit e81790d.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2018-08-07T20:06:20Z

Test build #94384 has finished for PR 21305 at commit 42d86e1.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
s\"its class is $
case class Uuid(randomSeed: Option[Long] = None) extends LeafExpression with Stateful
case class InSubquery(values: Seq[Expression], query: ListQuery)
trait ExpressionWithRandomSeed
case class Rand(child: Expression) extends RDG with ExpressionWithRandomSeed
case class Randn(child: Expression) extends RDG with ExpressionWithRandomSeed
case class AliasIdentifier(identifier: String, database: Option[String])

cloud-fan · 2018-08-08T01:56:57Z

thanks, merging to master!

rdblue · 2018-08-08T15:59:33Z

Thanks for reviewing, @cloud-fan!

## What changes were proposed in this pull request? This is a follow-up to #21305 that adds a test suite for AppendData analysis. This also fixes the following problems uncovered by these tests: * Incorrect order of data types passed to `canWrite` is fixed * The field check calls `canWrite` first to ensure all errors are found * `AppendData#resolved` must check resolution of the query's attributes * Column names are quoted to show empty names ## How was this patch tested? This PR adds a test suite for AppendData analysis. Closes #22043 from rdblue/SPARK-24251-add-append-data-analysis-tests. Authored-by: Ryan Blue <blue@apache.org> Signed-off-by: Wenchen Fan <wenchen@databricks.com>

This adds a new logical plan, AppendData, that was proposed in SPARK-23521: Standardize SQL logical plans. * DataFrameWriter uses the new AppendData plan for DataSourceV2 appends * AppendData is resolved if its output columns match the incoming data frame * A new analyzer rule, ResolveOutputColumns, validates data before it is appended. This rule will add safe casts, rename columns, and checks nullability Existing tests for v2 appends. Will add AppendData tests to validate logical plan analysis. Closes apache#21305 from rdblue/SPARK-24251-add-append-data. Lead-authored-by: Ryan Blue <blue@apache.org> Co-authored-by: Ryan Blue <rdblue@users.noreply.github.com> Signed-off-by: Wenchen Fan <wenchen@databricks.com> (cherry picked from commit 5fef6e3) Conflicts: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Strategy.scala sql/core/src/test/scala/org/apache/spark/sql/sources/v2/DataSourceV2Suite.scala sql/catalyst/src/main/scala/org/apache/spark/sql/types/DataType.scala sql/catalyst/src/test/scala/org/apache/spark/sql/types/DataTypeWriteCompatibilitySuite.scala

## What changes were proposed in this pull request? This is a follow-up to apache#21305 that adds a test suite for AppendData analysis. This also fixes the following problems uncovered by these tests: * Incorrect order of data types passed to `canWrite` is fixed * The field check calls `canWrite` first to ensure all errors are found * `AppendData#resolved` must check resolution of the query's attributes * Column names are quoted to show empty names ## How was this patch tested? This PR adds a test suite for AppendData analysis. Closes apache#22043 from rdblue/SPARK-24251-add-append-data-analysis-tests. Authored-by: Ryan Blue <blue@apache.org> Signed-off-by: Wenchen Fan <wenchen@databricks.com> (cherry picked from commit bdd2796)

This adds a new logical plan, AppendData, that was proposed in SPARK-23521: Standardize SQL logical plans. * DataFrameWriter uses the new AppendData plan for DataSourceV2 appends * AppendData is resolved if its output columns match the incoming data frame * A new analyzer rule, ResolveOutputColumns, validates data before it is appended. This rule will add safe casts, rename columns, and checks nullability Existing tests for v2 appends. Will add AppendData tests to validate logical plan analysis. Closes apache#21305 from rdblue/SPARK-24251-add-append-data. Lead-authored-by: Ryan Blue <blue@apache.org> Co-authored-by: Ryan Blue <rdblue@users.noreply.github.com> Signed-off-by: Wenchen Fan <wenchen@databricks.com>

This is a follow-up to apache#21305 that adds a test suite for AppendData analysis. This also fixes the following problems uncovered by these tests: * Incorrect order of data types passed to `canWrite` is fixed * The field check calls `canWrite` first to ensure all errors are found * `AppendData#resolved` must check resolution of the query's attributes * Column names are quoted to show empty names This PR adds a test suite for AppendData analysis. Closes apache#22043 from rdblue/SPARK-24251-add-append-data-analysis-tests. Authored-by: Ryan Blue <blue@apache.org> Signed-off-by: Wenchen Fan <wenchen@databricks.com>

This adds a new logical plan, AppendData, that was proposed in SPARK-23521: Standardize SQL logical plans. * DataFrameWriter uses the new AppendData plan for DataSourceV2 appends * AppendData is resolved if its output columns match the incoming data frame * A new analyzer rule, ResolveOutputColumns, validates data before it is appended. This rule will add safe casts, rename columns, and checks nullability Existing tests for v2 appends. Will add AppendData tests to validate logical plan analysis. Closes apache#21305 from rdblue/SPARK-24251-add-append-data. Lead-authored-by: Ryan Blue <blue@apache.org> Co-authored-by: Ryan Blue <rdblue@users.noreply.github.com> Signed-off-by: Wenchen Fan <wenchen@databricks.com> (cherry picked from commit 5fef6e3) Conflicts: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Strategy.scala sql/core/src/test/scala/org/apache/spark/sql/sources/v2/DataSourceV2Suite.scala sql/catalyst/src/main/scala/org/apache/spark/sql/types/DataType.scala sql/catalyst/src/test/scala/org/apache/spark/sql/types/DataTypeWriteCompatibilitySuite.scala

## What changes were proposed in this pull request? This is a follow-up to apache#21305 that adds a test suite for AppendData analysis. This also fixes the following problems uncovered by these tests: * Incorrect order of data types passed to `canWrite` is fixed * The field check calls `canWrite` first to ensure all errors are found * `AppendData#resolved` must check resolution of the query's attributes * Column names are quoted to show empty names ## How was this patch tested? This PR adds a test suite for AppendData analysis. Closes apache#22043 from rdblue/SPARK-24251-add-append-data-analysis-tests. Authored-by: Ryan Blue <blue@apache.org> Signed-off-by: Wenchen Fan <wenchen@databricks.com> (cherry picked from commit bdd2796)

rdblue mentioned this pull request May 11, 2018

[SPARK-23321][SQL]: Validate datasource v2 writes #20488

Closed

bersprockets reviewed May 17, 2018

View reviewed changes

jose-torres reviewed May 24, 2018

View reviewed changes

rdblue force-pushed the SPARK-24251-add-append-data branch from d2e4c41 to b906ab1 Compare July 4, 2018 22:22

cloud-fan reviewed Jul 5, 2018

View reviewed changes

cloud-fan reviewed Jul 7, 2018

View reviewed changes

rdblue mentioned this pull request Jul 10, 2018

[SPARK-23817][SQL]Migrate ORC file format read path to data source V2 #20933

Closed

rdblue force-pushed the SPARK-24251-add-append-data branch from 65c2670 to f041019 Compare July 17, 2018 23:55

rdblue added 4 commits August 6, 2018 21:15

SPARK-24251: Add better error message for nested fields.

4127833

SPARK-24251: Add catch-all case to DataType.canWrite.

d22ebf1

SPARK-24251: Implement strict matching for structs, add canWrite tests.

36c0237

SPARK-24251: Do not allow any missing fields.

e81790d

Previously missing fields were allowed if the read type's corresponding field was optional.

rdblue force-pushed the SPARK-24251-add-append-data branch from 618a79d to e81790d Compare August 7, 2018 04:41

Merge branch 'master' into SPARK-24251-add-append-data

42d86e1

asfgit closed this in 5fef6e3 Aug 8, 2018

rdblue mentioned this pull request Aug 8, 2018

[SPARK-24251][SQL] Add analysis tests for AppendData. #22043

Closed

[SPARK-24251][SQL] Add AppendData logical plan. #21305

[SPARK-24251][SQL] Add AppendData logical plan. #21305

Conversation

rdblue commented May 11, 2018

What changes were proposed in this pull request?

How was this patch tested?

SparkQA commented May 11, 2018

SparkQA commented May 12, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jose-torres left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

viirya Jul 8, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rdblue Jul 10, 2018 • edited Loading

Choose a reason for hiding this comment

rdblue commented Jul 4, 2018 • edited Loading

SparkQA commented Jul 5, 2018

Choose a reason for hiding this comment

rdblue Jul 5, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rdblue commented Jul 6, 2018

SparkQA commented Jul 6, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SparkQA commented Jul 9, 2018

rdblue commented Jul 9, 2018 • edited Loading

SparkQA commented Jul 9, 2018

rdblue commented Jul 10, 2018

rdblue commented Jul 10, 2018

jose-torres commented Jul 10, 2018

SparkQA commented Jul 10, 2018

cloud-fan commented Jul 11, 2018

rdblue commented Jul 11, 2018

cloud-fan commented Jul 11, 2018

rdblue commented Jul 11, 2018

rdblue commented Jul 17, 2018

SparkQA commented Jul 18, 2018

rdblue commented Aug 7, 2018

cloud-fan commented Aug 7, 2018

SparkQA commented Aug 7, 2018

cloud-fan commented Aug 7, 2018

SparkQA commented Aug 7, 2018

cloud-fan commented Aug 7, 2018

SparkQA commented Aug 7, 2018

cloud-fan commented Aug 7, 2018

SparkQA commented Aug 7, 2018

SparkQA commented Aug 7, 2018

cloud-fan commented Aug 8, 2018

rdblue commented Aug 8, 2018

viirya Jul 8, 2018 •

edited

Loading

rdblue Jul 10, 2018 •

edited

Loading

rdblue commented Jul 4, 2018 •

edited

Loading

rdblue Jul 5, 2018 •

edited

Loading

rdblue commented Jul 9, 2018 •

edited

Loading