An easy-to-use evaluation framework for benchmarking entity recognition and disambiguation systems

作     者：Hui CHEN Bao-gang WEI Yi-ming LI Yong-huai LIU Wen-hao ZHU 

作者机构：College of Computer Science and Technology Zhejiang University Hangzhou 310027 China Department of Computer Science Aberystwyth University Ceredigion SY23 3DB UK School of Computer Engineering and Science Shanghai University Shanghai 200000 China 

基　　金：Project supported by the National Natural Science Foundation of China (No. 61572434)  the China Knowledge Centre for Engineering Sciences and Technology (No. CKC-EST-2015-2-5)  and the Specialized Research Fund for the Doctoral Program of Higher Education (SRFDP)  China (No. 20130101110-136) 

出 版 物：《Frontiers of Information Technology & Electronic Engineering》 (信息与电子工程前沿（英文版）)

年 卷 期：2017年第18卷第2期

页      码：195-205页

摘      要：Entity recognition and disambiguation （ERD） is a crucial technique for knowledge base population and information extraction. In recent years, numerous papers have been published on this subject, and various ERD systems have been developed. However, there are still some confusions over the ERD field for a fair and complete comparison of these systems. Therefore, it is of emerging interest to develop a unified evaluation framework. In this paper, we present an easy-to-use evaluation framework （EUEF）, which aims at facilitating the evaluation process and giving a fair comparison of ERD systems. EUEF is well designed and released to the public as an open source, and thus could be easily extended with novel ERD systems, datasets, and evaluation metrics. It is easy to discover the advantages and disadvantages of a specific ERD system and its components based on EUEF. We perform a comparison of several popular and publicly available ERD systems by using EUEF, and draw some interesting conclusions after a detailed analysis.

主 题 词：Entity recognition and disambiguation (ERD) Evaluation framework Information extraction 

学科分类：081203[081203] 08[工学] 0835[0835] 0812[工学-测绘类] 

核心收录：

D　O　I：10.1631/FITEE.1500473

馆 藏 号：203215762...