-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[ST-Engine] Use SeatunnelChildFirstClassLoader to loader connector plugin #2479
[ST-Engine] Use SeatunnelChildFirstClassLoader to loader connector plugin #2479
Conversation
fix style tmp tmp add e2e test
|
||
transform { | ||
# If you would like to get more information about how to configure seatunnel and see full list of transform plugins, | ||
# please go to https://seatunnel.apache.org/docs/flink/configuration/transform-plugins/Sql |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use https://seatunnel.apache.org/docs/transform/sql
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
} | ||
|
||
# If you would like to get more information about how to configure seatunnel and see full list of sink plugins, | ||
# please go to https://seatunnel.apache.org/docs/flink/configuration/sink-plugins/Console |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use https://seatunnel.apache.org/docs/connector-v2/sink/LocalFile
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
} | ||
|
||
# If you would like to get more information about how to configure seatunnel and see full list of sink plugins, | ||
# please go to https://seatunnel.apache.org/docs/flink/configuration/sink-plugins/Console |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
as above
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
@@ -65,7 +65,7 @@ public SeaTunnelServer(@NonNull Node node, @NonNull SeaTunnelConfig seaTunnelCon | |||
this.liveOperationRegistry = new LiveOperationRegistry(); | |||
this.seaTunnelConfig = seaTunnelConfig; | |||
this.executorService = | |||
Executors.newFixedThreadPool(seaTunnelConfig.getEngineConfig().getServerExecutorPoolSize()); | |||
Executors.newCachedThreadPool(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Define thread name?
explame:
Executors.newCachedThreadPool(new ThreadFactoryBuilder()
.setNameFormat("seatunnel-server-executor-%d").build());
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
@@ -40,7 +40,7 @@ public Address applyForResource(long jobId, long taskId) { | |||
Map<Long, Address> jobAddressMap = physicalVertexIdAndResourceMap.computeIfAbsent(jobId, k -> new HashMap<>()); | |||
|
|||
Address localhost = | |||
jobAddressMap.putIfAbsent(taskId, new Address("localhost", 5701)); | |||
jobAddressMap.putIfAbsent(taskId, new Address("192.168.1.10", 5801)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should not commit local ip?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
//import java.util.concurrent.atomic.AtomicBoolean; | ||
//import java.util.function.BiConsumer; | ||
// | ||
//public class TaskExecutionServiceTest { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Don't commit this file?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
# This is a example source plugin **only for test and demonstrate the feature source plugin** | ||
FakeSource { | ||
result_table_name = "fake" | ||
field_name = "name,age", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use new config style? #2406
example:
FakeSource {
result_table_name = "fake"
fields {
name = string
age = int
}
}
env { | ||
# You can set flink configuration here | ||
execution.parallelism = 1 | ||
job.mode = "STREAMING" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
will it stop automatically?
FakeSource { | ||
result_table_name = "fake" | ||
field_name = "name,age", | ||
parallelism = 3 | ||
} | ||
|
||
FakeSource { | ||
result_table_name = "fake" | ||
field_name = "name,age", | ||
parallelism = 3 | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
duplicate define?
1、Finish a main process, submit-scheduler-finish.
2、Add engine e2e.