Text REtrieval Conference (TREC) 2009¶

Relevance Feedback¶

In the second year, the concentration shifted to finding good sets of docs to base their retrieval on. Each participant submitted one or two sets of 5 docs for each topic, and 3-5 other participants ran with those docs, thus getting a nonsystem dependent score on how good those docs were.

Track Web Page: https://groups.google.com/g/trec-relfeed

Chemical¶

TREC 2009 was the first year of the Chemical IR Track, which focuses on evaluation of search techniques for discovery of digitally stored information on chemical patents and academic journal articles. The track included two tasks: Prior Art (PA) and Technical Survey (TS) tasks.

Track coordinator(s):

M. Lupu, Information Retrieval Facility (IRF)
F. Piroi, Information Retrieval Facility (IRF)
J. Tait, Information Retrieval Facility (IRF)
J. Huang, York University
J. Zhu, University College London

Track Web Page: https://trec.nist.gov/data/chemical09.html

Legal¶

TREC 2009 was the fourth year of the Legal Track, which focuses on evaluation of search technology for “discovery” (i.e., responsive review) of electronically stored information in litigation and regulatory settings. The track included two tasks: an Interactive task (in which real users could iteratively refine their queries and/or engage in multi-pass relevance feedback) and a Batch task (two-pass search in a controlled setting with some relevant and nonrelevant documents manually marked after the first pass).

Track coordinator(s):

B. Hedin, H5
S. Tomlinson, Open Text Corporation
J. R. Baron, National Archives and Records Administration
D. W. Oard, University of Maryland, College Park

Track Web Page: http://trec-legal.umiacs.umd.edu/

Web¶

The TREC Web Track explores and evaluates Web retrieval technologies. Currently, the Web Track conducts experiments using the new billion-page ClueWeb09 collection. The TREC 2009 track is the successor to the Terabyte Retrieval Track, which ran from 2004 to 2006, and to the older Web Track, which ran from 1999 to 2003. The TREC 2009 Web Track includes both a traditional adhoc retrieval task and a new diversity task.

Track coordinator(s):

C.L.A. Clarke, University of Waterloo
N. Craswell, Microsoft
I. Soboroff, National Institute of Standards and Technology (NIST)

Track Web Page: https://plg.uwaterloo.ca/~trecweb/2009.html

Million Query¶

The Million Query Track ran for the third time in 2009. The track is designed to serve two purposes: first, it is an exploration of ad hoc retrieval over a large set of queries and a large collection of documents; second, it investigates questions of system evaluation, in particular whether it is better to evaluate using many queries judged shallowly or fewer queries judged thoroughly

Track coordinator(s):

B. Carterette, University of Delaware
H. Fang, University of Delaware
V. Pavlu, Northeastern University
E. Kanoulas, University of Sheffield

Track Web Page: https://web.archive.org/web/20100610074201/https://udel.edu/mailman/listinfo/trec-million09

Blog¶

The Blog track explores the information seeking behaviour in the blogosphere. Thus far, since its inception in 2006, the Blog track addressed two main search tasks based on the analysis of a commercial blog search engine: the opinion-finding task (i.e. “What do people think about X?”) and the blog distillation task (i.e. “Find me a blog with a principal, recurring interest in X.”)

Track coordinator(s):

C. Macdonald, University of Glasgow
I. Ounis, University of Glasgow
I. Soboroff, National Institute of Standards and Technology (NIST)

Track Web Page: https://www.dcs.gla.ac.uk/wiki/TREC-BLOG

Entity¶

The goal of the entity track is to perform entity-oriented search tasks on the World Wide Web. Many user information needs would be better answered by specific entities instead of just any type of documents.

Track coordinator(s):

K. Balog, University of Amsterdam
A. de Vries, CWI
P. Serdyukov, TU Delft
P. Thomas, CSIRO
T. Westerveld, Teezir

Track Web Page: https://web.archive.org/web/20110811014305/http://ilps.science.uva.nl/trec-entity/