Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Meetup# 批流融合在唯品会的实践应用(已认领) #11

Open
fancycrabtree opened this issue Aug 14, 2020 · 0 comments
Open

Comments

@fancycrabtree
Copy link
Collaborator

作者:王新春(唯品会 数据平台实时团队高级架构师)

主要分享内容:

流式数据处理和批数据处理的体系深度融合,部分数据加工和打宽直接在流数据中处理,并作为批处理或者 OLAP 引擎(Spark SQL/Presto/ClickHouse)等的输入,以达到数据口径统一,并且降低批处理的资源消耗的目标。

具体的实践包括:使用 Flink 做流量数据实时 ETL;Flink 实时入仓 MySQL 数据;使用 Flink 加工实时宽表和实时轻度汇总层数据,并提供给离线宽表、推荐算法和数据产品等使用。

@fancycrabtree fancycrabtree changed the title 《批流融合在唯品会的实践应用》 Meetup# 批流融合在唯品会的实践应用 Nov 6, 2020
@fancycrabtree fancycrabtree changed the title Meetup# 批流融合在唯品会的实践应用 Meetup# 批流融合在唯品会的实践应用(已认领) Nov 11, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant