Overview - Tasks 2016

Proceedings | Data | Results | Runs | Participants

Standard evaluation mechanisms focus on evaluating the quality of a retrieval system in terms of the relevance of the results retrieved, completely ignoring the fact that user satisfaction mainly depends on the usefulness of the system in helping the user complete the actual task that led the user issue the query. Similar to Tasks Track 2015, Tasks Track 2016 is an attempt investigate quality of retrieval systems in terms of (1) how well they can understand the underlying task that led the user submit a query, and (2) how useful they are for helping users complete their tasks.

Track coordinator(s):

• Manisha Verma, University College London
• Emine Yilmaz, University College London
• Rishabh Mehrotra, University College London
• Evangelos Kanoulas, University of Amsterdam
• Ben Carterette, University of Delaware
• Nick Craswell, Microsoft
• Peter Bailey, Microsoft

Tasks:

• understanding: Task Understanding
• completion: Task Completion
• adhoc: Adhoc Retrieval

Track Web Page: http://www.cs.ucl.ac.uk/tasks-track-2016/

---