Overview of the Recognizing Inference in Text (RITE-2) at NTCIR-10

Abstract

This paper describes an overview of RITE-2 (Recognizing Inference in TExt) task in NTCIR-10. We evaluated sys tems that automatically recognize semantic relations be tween sentences such as paraphrase, entailment, contradic tion in Japanese, Simplified Chinese and Traditional Chi nese. The tasks in RITE-2 are Binary Classification of entailment (BC Subtask), Multi-Class Classification includ ing paraphrase and contradiction (MC Subtask), Entrance Exam Subtasks (Exam BC and Exam Search), Unit Test, and RITE4QA Subtask. We had 28 active participants, and received 215 formal runs (110 Japanese runs, 53 Traditional Chinese runs, 52 Simplified Chinese runs). This paper also describes how the datasets for RITE-2 had been developed, how the systems were evaluated, and reports RITE-2 formal run results.