Overview of the NTCIR-7 ACLIA IR4QA Task

Abstract

This paper presents an overview of the IR4QA (In formation Retrieval for Question Answering) Task of the NTCIR-7 ACLIA (Advanced Cross-lingual Infor mation Access) Task Cluster. IR4QA evaluates tra ditional ranked retrieval of documents using well studied metrics such as Average Precision, but the re trieval task is embedded in the context of cross-lingual question answering. That is, document retrieval is treated as a component of the entire question answer ing system. This paper concentrates on how rele vance assessments for the Simpified Chinese, Tradi tional Chinese and Japanese IR4QA test collections were obtained, and the outcome of the formal IR4QA evaluation using the three collections. For the rela tionship between IR4QA and the entire ACLIA task cluster, we refer the reader to the overview paper of ACLIA [17]. For details of the individual IR4QA sys tems, we refer the reader to the participants’ report