Spark 3.5: Add tests for WHEN NOT MATCHED BY SOURCE clauses #8592
Conversation
| assertEquals( | ||
| "Should have expected rows", | ||
| ImmutableList.of( | ||
| row(1, "emp-id-1"), // updated (matched) |
There was a problem hiding this comment.
Currently this is a little confusing because the results are not in the same order as the clauses above
Clause 1
Clause 4
Clause 3
Could we add in some blanks and make it look more like
Clause 1 result row,
// Clause 2 row deleted
Clause 3 result row,
Clause 4 result row,
// Clause 5 row deleted
Minor nit here and would require rewriting this a little, it would just make the test a bit easier to read
| + "{ \"id\": 3, \"dep\": \"emp-id-3\" }\n" | ||
| + "{ \"id\": 4, \"dep\": \"emp-id-4\" }"); | ||
|
|
||
| createOrReplaceView("source", Collections.singletonList(1), Encoders.INT()); |
There was a problem hiding this comment.
It is not super clear here that this makes a table with an ID column with a single element, there are also different methods used for providing lists below.
Nit: I think we could be consistent with just always using ImmutableList.of(...)
|
Thanks, @RussellSpitzer @rdblue @Fokko! |
This PR adds tests for WHEN NOT MATCHED BY SOURCE clauses, supported in Spark 3.5.
This resolves #8276. There are more tests in Spark itself, it is just a sanity check mostly.