Overview - Total Recall 2015

Proceedings | Runs | Participants

The primary purpose of the Total Recall Track is to evaluate, through controlled simulation, methods designed to achieve very high recall - as close as practicable to 100% - with a human assessor in the loop. Motivating applications include, among others, electronic discovery in legal proceedings, systematic review in evidencebased medicine, and the creation of fully labeled test collections for information retrieval (“IR”) evaluation. A secondary, but no less important, purpose is to develop a sandboxed virtual test environment within which IR systems may be tested, while preventing the disclosure of sensitive test data to participants. At the same time, the test environment also operates as a “black box,” affording participants confidence that their proprietary systems cannot easily be reverse engineered.

Track coordinator(s):

• Adam Roegiest, University of Waterloo
• Gordon V. Cormack, University of Waterloo
• Charles L.A. Clarke, University of Waterloo
• Maura R. Grossman, Wachtell, Lipton, Rosen & Katz

Tasks:

• full: At-Home Task
• sandbox: Sandbox Task

Track Web Page: https://plg.uwaterloo.ca/~gvcormac/trecvm/

---