Entity relation extraction is one of the key tasks of natural language processing, and its application in the field of Tibetan medicine is of great significance for the construction of a Tibetan medicine knowledge map, intelligent assisted diagnosis and drug research and development. Aiming at the entity relation extraction task of Tibetan medical texts, this paper proposes a joint extraction method based on pre-trained model TiBERT plus multi-head attention. The TiBERT model is used to encode Tibetan medical texts and generate feature vectors containing contextual information. Multi-head attention mechanism is used to enhance feature representation and capture correlation information between different entities. The experimental results show that the F1 value of the model on the Tibetan medicine text dataset reaches 81.81%, which is significantly better than other comparison models, proving its effectiveness.
Key words
TiBERT model /
Multi-Head Attention /
entity relation extraction /
natural language processing
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
References
[1] Humphreys R G,Azzam S,Huyck C,et al.Description of the LaSIE-II System as Used for MUC7[J].1998.
[2] Zeng D,Liu K,Lai S,et al.Relation classification via convolutional deep neural network[C]//International Conference on Computational Linguistics.the 25th International Conference on Computational Linguistics,2014.
[3] Miwa M,Bansal M.End-to-End Relation Extraction using LSTMs on Sequences and Tree Structures[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers).2016.DOI:10.18653/v1/P16-1105.
[4] Hang T,Feng J,Yan L,et al.Joint extraction of entities and relations using multi-label tagging and relational alignment[J].Neural Computing and Applications,2022.DOI:10.1007/s00521-021-06685-1.
[5] 赵丹丹,张俊朋,孟佳娜,等.基于预训练模型和混合神经网络的医疗实体关系抽取[J].北京大学学报:自然科学版,2023,59(1):65-75.
[6] 耿飙,梁成全,魏炜,等.基于深度学习的非结构化医学文本知识抽取[J].计算机工程与设计,2024,45(1):177-186.
[7] 夏天赐,孙媛.基于联合模型的藏文实体关系抽取方法研究[J].中文信息学报,2018,32(12):76-83.
[8] 于韬,尼玛次仁,拥措,等.基于藏文Albert预训练语言模型的图采样与聚合实体关系抽取[J].中文信息学报,2022,36(10):63-72.
[9] 周青,拥措,拉毛东只,等.基于跨度表示的藏医药文献实体关系抽取[J/OL].北京大学学报(自然科学版),1-11[2025-03-17].https://doi.org/10.13209/j.0479-8023. 2024.125.
[10] 周青,拥措,拉毛东只,等.基于预训练的藏医药实体关系抽取[J].中文信息学报,2024,38(8):76-83.
Funding
青海民族大学2024年度校级本硕博（学生）项目“基于深度学习的藏医药命名实体识别及关系抽取技术研究”阶段性成果,项目编号：09M2024002
