-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Feature][Connector-v2] improve assert sink connector #2844
Conversation
612d7bf
to
7611758
Compare
|
||
import java.io.Serializable; | ||
|
||
public class AssertRowRule implements Serializable { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This class is useless. Maybe we should remove it first.
@@ -49,7 +54,18 @@ public void write(SeaTunnelRow element) { | |||
|
|||
@Override | |||
public void close() throws IOException { | |||
// nothing | |||
assertRowRules.stream().filter(assertRule -> { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think It is easier to understand if we use anyMatch
instead of filter
and findFirst
here.
@@ -28,24 +28,24 @@ env { | |||
|
|||
source { | |||
# This is a example source plugin **only for test and demonstrate the feature source plugin** | |||
FakeSource { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this config file is no need to change in this PR.
Thx for your contribution. Could you help to update the manual doc |
Done,thank you for your code review. |
# Conflicts: # seatunnel-examples/seatunnel-flink-connector-v2-example/src/main/resources/examples/fake_to_console.conf
# Conflicts: # seatunnel-e2e/seatunnel-flink-connector-v2-e2e/connector-redis-flink-e2e/src/test/java/org/apache/seatunnel/e2e/flink/v2/redis/RedisIT.java # seatunnel-e2e/seatunnel-flink-connector-v2-e2e/connector-redis-flink-e2e/src/test/resources/redis/redis_source_and_sink.conf # seatunnel-e2e/seatunnel-spark-connector-v2-e2e/connector-redis-spark-e2e/src/test/java/org/apache/seatunnel/e2e/spark/v2/redis/RedisIT.java # seatunnel-e2e/seatunnel-spark-connector-v2-e2e/connector-redis-spark-e2e/src/test/resources/redis/redis_source_and_sink.conf
68d0da7
to
db5b79d
Compare
{ | ||
row_rules = [ | ||
{ | ||
rule_type = MAX_ROW |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If I want to test test final row number equals a expected value, how can I do.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If I want to test test final row number equals a expected value, how can I do.
add EQUAL_ROW
rule_type? any good suggestions?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
how about making max = min to express equal?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
how about making max = min to express equal?
That's a good idea.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
how about making max = min to express equal?
I agree with you. And we must judge max=min
after all data processing is completed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
how about making max = min to express equal?
I agree with you. And we must judge
max=min
after all data processing is completed.
how about remove max
and min
and add expected_num
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
how about making max = min to express equal?
I agree with you. And we must judge
max=min
after all data processing is completed.how about remove
max
andmin
and addexpected_num
Some Source Connector support AT_LEAST_ONCE, so max
and min
is needed.
please help to code review @EricJoy2048 @hailin0 |
Can this pr continue to flow? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1
Close #2796
Purpose of this pull request
improve assert sink connector
Check list
New License Guide