-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Improve][DOC] Perfect the connector v2 doc #2800
Merged
Merged
Changes from 1 commit
Commits
Show all changes
11 commits
Select commit
Hold shift + click to select a range
95bc9b0
[Improve][DOC] Perfect the connector v2 doc
liugddx e15c75f
Update seatunnel-connectors-v2/README.zh.md
liugddx b8c52f1
[Improve][DOC] A little tinkering
liugddx b044748
Merge remote-tracking branch 'origin/improve-doc-connectorv2' into im…
liugddx 604aa59
[Improve][DOC] A little tinkering
liugddx 62dfa3a
[Doc][connector] add Console sink doc
liugddx e925341
[Doc][connector] add Console sink doc
liugddx ef9361b
Merge remote-tracking branch 'origin/improve-doc-connectorv2' into im…
liugddx 1884054
fix some problem
liugddx b8396bc
fix some problem
liugddx 6054abb
fine tuning
liugddx File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,11 +1,11 @@ | ||
## 目的 | ||
Because SeaTunnel design new API for connectors, 所以通过这篇文章来介绍新的接口以及新的代码结构,方便开发者快速的帮助新API和翻译层完善,以及开发出新的Connecotor. | ||
因为SeaTunnel 为connectors设计了新的API,所以通过这篇文章来介绍新的接口以及新的代码结构,方便开发者快速的帮助新API和翻译层完善,以及开发出新的Connector. | ||
## 代码结构 | ||
现阶段所有相关代码保存在`api-draft`分支上。 | ||
为了和老的代码分开,方便现阶段的并行开发,以及降低merge的难度。我们为新的执行流程定义了新的模块 | ||
### Example | ||
我们已经在`seatunnel-examples`中准备好了新版本的可本地执行Example程序,直接调用`seatunnel-flink-connector-v2-example`或`seatunnel-spark-connector-v2-example`中的`SeaTunnelApiExample`即可。这也是本地开发Connector经常会用到的调试方式。 | ||
对应的配置文件保存在同模块的`resources/examples`文件夹下,和以前一样。 | ||
对应的配置文件保存在同模块的`resources/examples`文件夹下,和以前一样。目前测试需要将connector依赖进来,如果有部分第三方包需要自行依赖请使用类加载器自行加载,如果有些 Scope为 Provided的依赖,需要在Intellij IDEAZ中单击 "Run"->"Edit configurations...",在 | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 目前在Example模块下测试需要把对应连接器的依赖添加到pom文件中,如果有部分三方依赖请自行添加至模块classpath,如果连接器的某些依赖的 |
||
"Use classpath of module"选项上选择当前工程,并勾选"Include dependencies with 'Provided Scope". | ||
### 启动类 | ||
和老的启动类分开,我们创建了两个新的启动类工程,分别是`seatunnel-core/seatunnel-flink-starter`和`seatunnel-core/seatunnel-spark-starter`. 可以在这里找到如何将配置文件解析为可以执行的Flink/Spark流程。 | ||
### SeaTunnel API | ||
|
@@ -61,4 +61,4 @@ if (Boundedness.BOUNDED.equals(context.getBoundedness())) { | |
#### 我应该实现SinkCommitter还是SinkAggregatedCommitter? | ||
当前版本推荐将实现SinkAggregatedCommitter作为首选,可以在Flink/Spark中提供较强的一致性保证,同时commit应该要实现幂等性,保存引擎重试能够正常运作。 | ||
## 实现 | ||
所有的Connector实现都应该在`seatunnel-connectors/seatunnel-connectors-seatuunel`模块下,现阶段可参考的示例均在此模块下。 | ||
所有的Connector实现都应该在`seatunnel-connectors/seatunnel-connectors-seatuunel`模块下,现阶段可参考的示例均在此模块下。 | ||
liugddx marked this conversation as resolved.
Show resolved
Hide resolved
|
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
SeaTunnel为与计算引擎进行解耦,设计了新的连接器API,通过这篇文章来介绍新的接口以及新的代码结构,方便开发者快速上手使用新版API开发连接器并理解新版API运行原理
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Who not change it?