Overview of the NTCIR-6 Cross-Lingual Question Answering (CLQA) Task

Abstract

This paper describes an overview of the NTCIR-6 Cross-Lingual Question Answering (CLQA) Task, an evaluation campaign for Cross-Lingual Question Answering technology. In NTCIR-5, the first CLQA task targeting Chinese, English, and Japanese languages was carried out. Following the success of NTCIR-5 CLQA, NTCIR-6 hosted the second campaign on the CLQA task. Since the handling of Named Entities is a major issue in CLQA, we aimed to promote research on cross-lingual Question Answering technology capable of Named Entities in East Asian languages. We conducted evaluations of seven subtasks: E-J, J-J, J-E, E-C, C-C, C-E, and E E subtask, where C, E, and J stand for Chinese, English, and Japanese, respectively, and X-Y indicates that questions are given in language X and answers are extracted from documents written in language Y. For the purpose of system development, we provided participants with the sample question/answer pairs and the formal run question/answer pairs used at the previous CLQA task. The Formal Run evaluation was conducted during November 1-7, 2006 with 200 and 150 test questions for Japanese-related CLQA and Chinese related CLQA, respectively. As a result, 12 research groups world-wide participated in CLQA, and 91 runs were submitted in total.