基于分布式框架下的中文文本特征分类

张慧芳 宗彩乐 张晓琳

电脑与电信 ›› 2019, Vol. 1 ›› Issue (5) : 1-7.

电脑与电信 ›› 2019, Vol. 1 ›› Issue (5) : 1-7.
基金项目

基于分布式框架下的中文文本特征分类

  • 张慧芳1 宗彩乐2 张晓琳1
作者信息 +

Chinese Text Feature Classification Based on Distributed Framework

Author information +
文章历史 +

摘要

研究运用复旦中文文本及搜狗中文文档作为研究对象,提高了中文文本分类精确度及召回率,分析得出特征词的最佳贡献值。应用朴素贝叶斯分类方法和改进的TFIDF关键字提取及权重计算,提出TNBIF模型分类方法,在Spark平台上并行分类实现。实验结果表明:应用TNBIF模型实行中文文本分类,精确度高达95.49%,比传统文本分类方法精确度提高5.41%,召回率提高了6.64%。本研究得出最佳贡献值为0.95。

Abstract

The study uses Fudan Chinese text and Sogou Chinese document as the research object. It improves the Chinese text classification accuracy and recall rate. And it analyzes and obtains the best contribution value of the feature words. Based on naive Bayes classification method, improved TFIDF keyword extraction and weight calculation, the TNBIF model classification method is proposed and implemented on the Spark platform. The experimental results show that the Chinese text classification is applied by the TNBIF model. The accuracy is as high as 95.49%, which is 5.41% higher than the traditional text classification method and the recall rate is increased by 6.64%. This study obtains an optimal contribution of 0.95.

关键词

TNBIF / 模型 / 海量数据集 / Spark / 特征分类 / 并行分类

Key words

TNBIF / model / massive data set / Spark / feature classification / parallel classification

引用本文

导出引用
张慧芳 宗彩乐 张晓琳. 基于分布式框架下的中文文本特征分类[J]. 电脑与电信. 2019, 1(5): 1-7
ZHANG Hui-fang ZONG Cai-le ZHANG Xiao-lin. Chinese Text Feature Classification Based on Distributed Framework[J]. Computer & Telecommunication. 2019, 1(5): 1-7
中图分类号: TP391.1   

基金

国家自然科学基金资助项目,项目编号:61562065。

Accesses

Citation

Detail

段落导航
相关文章

/