摘要
针对现有的大数据处理平台实时性差、处理耗时长、资源请求慢等问题,采用Storm实时计算技术,结合Flume、
Kafka、Zookeeper等大数据处理组件,设计一个实时数据处理平台。利用tornado+WSGI+Apache技术搭建Web服务器,采用
Echarts技术对处理结果进行可视化分析。以网站访问日志作为数据源,对平台进行验证,通过测试,该平台能够完成网站的点
击率和访客数的实时计算,具有稳定可靠、操作简单、实时性强等特点。
Abstract
Aiming at the problems of poor real-time performance, long processing time, and slow resource request of the existing big
data processing platform, this paper uses Storm real-time computing technology, combined with Flume, Kafka, Zookeeper and other
big data processing components to design a real-time data processing platform. It uses tornado+WSGI+Apache technology to build a
Web server, and uses Echarts technology to visually analyze the processing results. This article uses the website access log as the da-
ta source to verify the platform. Through the test, the platform can complete the real-time calculation of the website's click-through
rate and the number of visitors. It has the characteristics of stability, reliability and simple operation.
关键词
大数据 /
Storm /
实时计算技术 /
数据可视化 /
点击率 /
访客数
Key words
big data /
Storm /
real-time computing technology /
data visualization /
click-through rate /
number of visitors
杨宇 徐万明.
基于Storm技术的实时数据处理平台研究与实现[J]. 电脑与电信. 2021, 1(1): 51
YANG Yu XU Wang-ming.
Research and Implementation of Real-time Data Processing Platform Based on Storm Technology[J]. Computer & Telecommunication. 2021, 1(1): 51
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}