Overview - Total Recall 2016

Proceedings | Runs | Participants

The primary purpose of the Total Recall Track is to evaluate, through controlled simulation, methods designed to achieve very high recall - as close as practicable to 100% - with a human assessor in the loop. Motivating applications include, among others, electronic discovery in legal proceedings, systematic review in evidencebased medicine, and the creation of fully labeled test collections for information retrieval (“IR”) evaluation. A secondary, but no less important, purpose is to develop a sandboxed virtual test environment within which IR systems may be tested, while preventing the disclosure of sensitive test data to participants. At the same time, the test environment also operates as a “black box,” affording participants confidence that their proprietary systems cannot easily be reverse engineered.

Track coordinator(s):

• Maura R. Grossman, University of Waterloo
• Gordon V. Cormack, University of Waterloo
• Adam Roegiest, University of Waterloo

Tasks:

• full: At-Home Task
• sandbox: Sandbox Task

Track Web Page: https://plg.uwaterloo.ca/~gvcormac/total-recall/

---