基于Storm技术的实时数据处理平台研究与实现

杨宇 徐万明

电脑与电信 ›› 2021, Vol. 1 ›› Issue (1) : 51.

电脑与电信 ›› 2021, Vol. 1 ›› Issue (1) : 51.
应用技术与研究

基于Storm技术的实时数据处理平台研究与实现

作者信息 +

Research and Implementation of Real-time Data Processing Platform Based on Storm Technology

Author information +
文章历史 +

摘要

针对现有的大数据处理平台实时性差、处理耗时长、资源请求慢等问题,采用Storm实时计算技术,结合Flume、 Kafka、Zookeeper等大数据处理组件,设计一个实时数据处理平台。利用tornado+WSGI+Apache技术搭建Web服务器,采用 Echarts技术对处理结果进行可视化分析。以网站访问日志作为数据源,对平台进行验证,通过测试,该平台能够完成网站的点 击率和访客数的实时计算,具有稳定可靠、操作简单、实时性强等特点。

Abstract

Aiming at the problems of poor real-time performance, long processing time, and slow resource request of the existing big data processing platform, this paper uses Storm real-time computing technology, combined with Flume, Kafka, Zookeeper and other big data processing components to design a real-time data processing platform. It uses tornado+WSGI+Apache technology to build a Web server, and uses Echarts technology to visually analyze the processing results. This article uses the website access log as the da- ta source to verify the platform. Through the test, the platform can complete the real-time calculation of the website's click-through rate and the number of visitors. It has the characteristics of stability, reliability and simple operation.

关键词

大数据 / Storm / 实时计算技术 / 数据可视化 / 点击率 / 访客数

Key words

big data / Storm / real-time computing technology / data visualization / click-through rate / number of visitors

引用本文

导出引用
杨宇 徐万明. 基于Storm技术的实时数据处理平台研究与实现[J]. 电脑与电信. 2021, 1(1): 51
YANG Yu XU Wang-ming. Research and Implementation of Real-time Data Processing Platform Based on Storm Technology[J]. Computer & Telecommunication. 2021, 1(1): 51

Accesses

Citation

Detail

段落导航
相关文章

/