Research and Implementation of Real-time Data Processing Platform Based on Storm Technology

Guizhou Vocational Technology College of Electronics & Information Qiandongnan Vocational & Technical College for Nationalities

Download:

PDF(0KB)
Export: BibTeX | EndNote (RIS)

Abstract Aiming at the problems of poor real-time performance, long processing time, and slow resource request of the existing big data processing platform, this paper uses Storm real-time computing technology, combined with Flume, Kafka, Zookeeper and other big data processing components to design a real-time data processing platform. It uses tornado+WSGI+Apache technology to build a Web server, and uses Echarts technology to visually analyze the processing results. This article uses the website access log as the da- ta source to verify the platform. Through the test, the platform can complete the real-time calculation of the website's click-through rate and the number of visitors. It has the characteristics of stability, reliability and simple operation.

Key words： big data Storm real-time computing technology data visualization click-through rate number of visitors

Published: 24 February 2021

	Service

	E-mail this article
	Add to my bookshelf
	Add to citation manager
	E-mail Alert
	RSS
	Articles by authors
	YANG Yu XU Wang-ming

Cite this article:

YANG Yu XU Wang-ming . Research and Implementation of Real-time Data Processing Platform Based on Storm Technology. Computer & Telecommunication, 2021, 1(1): 51-.

URL:

https://www.computertelecom.com.cn/EN/ OR https://www.computertelecom.com.cn/EN/Y2021/V1/I1/51

[1]	NIE Cheng WANG Jie. A Review of Research and Development in Data Analysis Methods [J]. 电脑与电信, 2024, 1(4): 200-25.
[2]	JIANG Lei HU Juan. Teaching Reform of Data Visualization Technology Course Based on OBE [J]. 电脑与电信, 2024, 1(4): 38-42.
[3]	WANG Zi-ming　XIAO Da-wei　LIU Ya-nan. Knowledge Graph Construction for Data Science and Big Data Technology[J]. 电脑与电信, 2024, 1(1): 32-34.
[4]	LI Xin JIA Mei-juan LIU Chun MAO Kai-ji. Reform and Practice of Computer Public Basic Education Model for Training Applied Talents in the Era of Big Data and Artificial Intelligence[J]. 电脑与电信, 2023, 1(11): 19-22.
[5]	ZHANG Liang ZHANG Li-mei. Design and Implementation of Comprehensive Training Platform for Big Data Specialty Based on Docker Technology [J]. 电脑与电信, 2022, 1(1-2): 46-48.
[6]	ZHU Feng. Million Enrollment Expansion in Higher Vocational Education——Research on Talent Training Strategy for Big Data Technology and Application Specialty[J]. 电脑与电信, 2021, 1(9): 85-89.
[7]	LIU Hui-fang. Research and Application of Anti-fraud Model Based on Big Data of Communication Operator[J]. 电脑与电信, 2021, 1(7): 46-52.
[8]	WANG Jia-hao. Risk Evaluation of Telecom Big Data Application Project Based on AHP-entropy Weight Method[J]. 电脑与电信, 2021, 1(7): 53-55.
[9]	YAO Jian-sheng LIU Yan-ling. Teaching Reform and Practice of Smart Tourism Data Science Course[J]. 电脑与电信, 2021, 1(4): 9-11.
[10]	SONG Man. Research on Talents Training Program of Big Data Majors in Higher Vocational Colleges ——Taking Guangzhou City Construction College as an Example[J]. 电脑与电信, 2021, 1(4): 16-22.
[11]	LI Ai-sheng. Exploration and Practice of Intelligent Training Based on CDL Architecture ——Take the Major of Information Security and Management as an Example[J]. 电脑与电信, 2021, 1(4): 81-85.
[12]	CHEN Xin-yi GE Gao-jing YUAN Ze-yuan. Prevention and Control Strategies of Telecommunications Fraud Crime under the Background of "Intelligence-led Policing" [J]. 电脑与电信, 2021, 1(12): 76-80.
[13]	CHEN Chun-yan. Research on Strategies for Optimizing Blended Teaching Mode by Using Big Data Analysis [J]. 电脑与电信, 2021, 1(11): 8-11.
[14]	FENG Chong-jun. Current Situation Analysis and Hot Research on Second-hand Housing Market Based on Python Crawler -- A Case Study in Nanjing [J]. 电脑与电信, 2021, 0(10): 65-68.
[15]	OU Wei-hong YANG Yong-qin LI Jia-hua. Efficiency Analysis and Optimization of Dig Data Processing under Cloud Computing Platform [J]. 电脑与电信, 2021, 0(10): 18-23.

Viewed

Full text

Abstract

Cited

Shared

Discussed