Please wait a minute...
FITEE
Front. Inform. Technol. Electron. Eng.  2017, Vol. 18 Issue (2): 195-205    DOI: 10.1631/FITEE.1500473
Regular Papers     
An easy-to-use evaluation framework for benchmarking entity recognition and disambiguation systems
Hui Chen, Bao-gang Wei, Yi-ming Li, Yong-huai Liu, Wen-hao Zhu
College of Computer Science and Technology, Zhejiang University, Hangzhou 310027, China; Department of Computer Science, Aberystwyth University, Ceredigion SY23 3DB, UK; School of Computer Engineering and Science, Shanghai University, Shanghai 200000, China
Download:     PDF (0 KB)     
Export: BibTeX | EndNote (RIS)      

Abstract  Entity recognition and disambiguation (ERD) is a crucial technique for knowledge base population and information extraction. In recent years, numerous papers have been published on this subject, and various ERD systems have been developed. However, there are still some confusions over the ERD field for a fair and complete comparison of these systems. Therefore, it is of emerging interest to develop a unified evaluation framework. In this paper, we present an easy-to-use evaluation framework (EUEF), which aims at facilitating the evaluation process and giving a fair comparison of ERD systems. EUEF is well designed and released to the public as an open source, and thus could be easily extended with novel ERD systems, datasets, and evaluation metrics. It is easy to discover the advantages and disadvantages of a specific ERD system and its components based on EUEF. We perform a comparison of several popular and publicly available ERD systems by using EUEF, and draw some interesting conclusions after a detailed analysis.

Key words： Entity recognition and disambiguation (ERD)      Evaluation framework      Information extraction     
Received: 26 December 2015      Published: 10 February 2017
CLC:  TP391.1  
Service
E-mail this article
Add to my bookshelf
Add to citation manager
E-mail Alert
RSS
Articles by authors
Hui Chen
Bao-gang Wei
Yi-ming Li
Yong-huai Liu
Wen-hao Zhu
Cite this article:

Hui Chen, Bao-gang Wei, Yi-ming Li, Yong-huai Liu, Wen-hao Zhu. An easy-to-use evaluation framework for benchmarking entity recognition and disambiguation systems. Front. Inform. Technol. Electron. Eng., 2017, 18(2): 195-205.

URL:

http://www.zjujournals.com/xueshu/fitee/10.1631/FITEE.1500473     OR     http://www.zjujournals.com/xueshu/fitee/Y2017/V18/I2/195

[1] Xi-ming Li, Ji-hong Ouyang, You Lu. Topic modeling for large-scale text data[J]. Front. Inform. Technol. Electron. Eng., 2015, 16(6): 457-465.
[2] Jin-song Su, Xiao-dong Shi, Yan-zhou Huang, Yang Liu, Qing-qiang Wu, Yi-dong Chen, Huai-lin Dong. Topic-aware pivot language approach for statistical machine translation[J]. Front. Inform. Technol. Electron. Eng., 2014, 15(4): 241-253.
[3] Xiao-hong Mao, Jing-hua Fu, Wei Chen, Qian You, Shiao-fen Fang, Qun-sheng Peng. Structural visualization of sequential DNA data[J]. Front. Inform. Technol. Electron. Eng., 2011, 12(4): 263-272.
[4] Yun-hua Qu, Tian-jiong Tao, Serge Sharoff, Narisong Jin, Ruo-yuan Gao, Nan Zhang, Yu-ting Yang, Cheng-zhi Xu. Using an integrated feature set to generalize and justify the Chinese-to-English transferring rule of the ‘ZHE’ aspect[J]. Front. Inform. Technol. Electron. Eng., 2010, 11(9): 663-676.