Skip to content

zhangruipython/ETLPlatform

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

33 Commits
 
 
 
 
 
 

Repository files navigation

ETLPlatform

多数据源,大规模数据提取转换加载平台搭建

SparkML Pipeline机器学习模块+Smile机器学习应用

SparkPlatform\src\main\java\com\application\ml

具体说明博客地址: https://zhangruipython.github.io

Spark Structured Streaming结合kafka流式计算应用

SparkPlatform\src\main\java\com\application\stream

数据源:rocksdb

消息队列中间件:kafka

数据处理:structured streaming

流程图如下

SparkStructuredStream数据流处理.png

About

多数据源,大规模数据提取转换加载

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages