Please wait a minute...
Front. Inform. Technol. Electron. Eng.  2015, Vol. 16 Issue (11): 940-956    DOI: 10.1631/FITEE.1500067
    
Automatically building large-scale named entity recognition corpora from Chinese Wikipedia
Jie Zhou, Bi-cheng Li, Gang Chen
Department of Signal Analysis and Information Processing, Zhengzhou Information Science and Technology Institute, Zhengzhou 450002, China
Download:   PDF(0KB)
Export: BibTeX | EndNote (RIS)      

Abstract  Named entity recognition (NER) is a core component in many natural language processing applications. Most NER systems rely on supervised machine learning methods, which depend on time-consuming and expensive annotations in different languages and domains. This paper presents a method for automatically building silver-standard NER corpora from Chinese Wikipedia. We refine novel and language-dependent features by exploiting the text and structure of Chinese Wikipedia. To reduce tagging errors caused by entity classification, we design four types of heuristic rules based on the characteristics of Chinese Wikipedia and train a supervised NE classifier, and a combined method is used to improve the precision and coverage. Then, we realize type identification of implicit mention by using boundary information of outgoing links. By selecting the sentences related with the domains of test data, we can train better NER models. In the experiments, large-scale NER corpora containing 2.3 million sentences are built from Chinese Wikipedia. The results show the effectiveness of automatically annotated corpora, and the trained NER models achieve the best performance when combining our silver-standard corpora with gold-standard corpora.

Key wordsNER corpora      Chinese Wikipedia      Entity classification      Domain adaptation      Corpus selection     
Received: 07 March 2015      Published: 04 November 2015
CLC:  TP391  
Cite this article:

Jie Zhou, Bi-cheng Li, Gang Chen. Automatically building large-scale named entity recognition corpora from Chinese Wikipedia. Front. Inform. Technol. Electron. Eng., 2015, 16(11): 940-956.

URL:

http://www.zjujournals.com/xueshu/fitee/10.1631/FITEE.1500067     OR     http://www.zjujournals.com/xueshu/fitee/Y2015/V16/I11/940


基于中文维基的大规模命名实体识别语料自动生成方法

目的:命名实体识别作为自然语言处理领域一项重要的基础性工作,当前主流方法是基于有监督的机器学习方法。该类方法依赖于特定语种和领域的标注语料,而语料的标注过程需耗费大量的人力、物力。本文提出一种基于中文维基的大规模命名实体识别(NER)语料自动生成方法。利用该方法能自动抽取并标记中文维基中的句子,从而为中文NER任务提供有效的语料支持。
创新点:本文根据中文维基的特点设计出四类启发式规则,并结合有监督的命名实体分类器,实现中文维基条目的命名实体类型的准确、全面识别;为避免缺失的维基链接引发的标注缺失,本文利用出链接的边界信息发现维基文档中的隐式指称项,并利用实体链接技术识别歧义指称项的实体类型;本文提出一种基于核心条目扩展的标注语料选择方法,实现测试数据的领域自适应。
方法:本文方法的整体流程如原文图2所示。该方法主要包括显式指称项的实体分类、隐式指称项的类型识别和标注语料选择三个主要步骤。在显式指称项的实体分类中,为实现准确、全面的实体类型识别,采用基于启发式规则与有监督实体分类器相结合的方法;在隐式指称项的类型识别中,提出一种新方法发现维基文档中的隐式指称项并识别歧义指称项的实体类型;在标注语料选择中,提出一种基于核心条目扩展的方法,实现测试数据的领域自适应。
结论:根据实验结果,采用本文方法能自动生成大规模的中文NER语料。此外,将生成语料与标准语料结合时,训练获得的NER模型性能更优。

关键词: NER语料,  中文维基,  实体分类,  领域自适应,  语料选择 
[1] Yuan-ping Nie, Yi Han, Jiu-ming Huang, Bo Jiao, Ai-ping Li. Attention-based encoder-decoder model for answer selection in question answering[J]. Front. Inform. Technol. Electron. Eng., 2017, 18(4): 535-544.
[2] Gopi Ram , Durbadal Mandal , Sakti Prasad Ghoshal , Rajib Kar . Optimal array factor radiation pattern synthesis for linear antenna array using cat swarm optimization: validation by an electromagnetic simulator[J]. Front. Inform. Technol. Electron. Eng., 2017, 18(4): 570-577.
[3] Lin-bo Qiao, Bo-feng Zhang, Jin-shu Su, Xi-cheng Lu. A systematic review of structured sparse learning[J]. Front. Inform. Technol. Electron. Eng., 2017, 18(4): 445-463.
[4] Rong-Feng Zhang , Ting Deng , Gui-Hong Wang , Jing-Lun Shi , Quan-Sheng Guan . A robust object tracking framework based on a reliable point assignment algorithm[J]. Front. Inform. Technol. Electron. Eng., 2017, 18(4): 545-558.
[5] . A quality requirements model and verification approach for system of systems based on description logic[J]. Front. Inform. Technol. Electron. Eng., 2017, 18(3): 346-361.
[6] Ali Darvish Falehi, Ali Mosallanejad. Dynamic stability enhancement of interconnected multi-source power systems using hierarchical ANFIS controller-TCSC based on multi-objective PSO[J]. Front. Inform. Technol. Electron. Eng., 2017, 18(3): 394-409.
[7] Wen-yan Xiao, Ming-wen Wang, Zhen Weng, Li-lin Zhang, Jia-li Zuo. Corpus-based research on English word recognition rates in primary school and word selection strategy[J]. Front. Inform. Technol. Electron. Eng., 2017, 18(3): 362-372.
[8] Hui Chen, Bao-gang Wei, Yi-ming Li, Yong-huai Liu, Wen-hao Zhu. An easy-to-use evaluation framework for benchmarking entity recognition and disambiguation systems[J]. Front. Inform. Technol. Electron. Eng., 2017, 18(2): 195-205.
[9] Jun-hong Zhang, Yu Liu. Application of complete ensemble intrinsic time-scale decomposition and least-square SVM optimized using hybrid DE and PSO to fault diagnosis of diesel engines[J]. Front. Inform. Technol. Electron. Eng., 2017, 18(2): 272-286.
[10] Li Weigang. First and Others credit-assignment schema for evaluating the academic contribution of coauthors[J]. Front. Inform. Technol. Electron. Eng., 2017, 18(2): 180-194.
[11] Yue-ting Zhuang, Fei Wu, Chun Chen, Yun-he Pan. Challenges and opportunities: from big data to knowledge in AI 2.0[J]. Front. Inform. Technol. Electron. Eng., 2017, 18(1): 3-14.
[12] Bo-hu Li, Hui-yang Qu, Ting-yu Lin, Bao-cun Hou, Xiang Zhai, Guo-qiang Shi, Jun-hua Zhou, Chao Ruan. A swarm intelligence design based on a workshop of meta-synthetic engineering[J]. Front. Inform. Technol. Electron. Eng., 2017, 18(1): 149-152.
[13] Yong-hong Tian, Xi-lin Chen, Hong-kai Xiong, Hong-liang Li, Li-rong Dai, Jing Chen, Jun-liang Xing, Jing Chen, Xi-hong Wu, Wei-min Hu, Yu Hu, Tie-jun Huang, Wen Gao. Towards human-like and transhuman perception in AI 2.0: a review[J]. Front. Inform. Technol. Electron. Eng., 2017, 18(1): 58-67.
[14] Yu-xin Peng, Wen-wu Zhu, Yao Zhao, Chang-sheng Xu, Qing-ming Huang, Han-qing Lu, Qing-hua Zheng, Tie-jun Huang, Wen Gao. Cross-media analysis and reasoning: advances and directions[J]. Front. Inform. Technol. Electron. Eng., 2017, 18(1): 44-57.
[15] Le-kui Zhou, Si-liang Tang, Jun Xiao, Fei Wu, Yue-ting Zhuang. Disambiguating named entities with deep supervised learning via crowd labels[J]. Front. Inform. Technol. Electron. Eng., 2017, 18(1): 97-106.