Text REtrieval Conference (TREC) 2012¶

Microblog¶

The Microblog track examines search tasks and evaluation methodologies for information seeking behaviours in microblogging environments such as Twitter. It was first introduced in 2011, addressing a real-time adhoc search task, whereby the user wishes to see the most recent relevant information to the query. In 2012, the realtime adhoc task was changed slightly, and a new filtering task was added. The filtering task models a standing query where the user wishes to see relevant tweets as they are posted.

Track coordinator(s):

Ian Soboroff, National Institute of Standards and Technology (NIST)
Iadh Ounis, Craig Macdonald, University of Glasgow
Jimmy Lin, University of Maryland

Track Web Page: https://web.archive.org/web/20120604072950/http://sites.google.com/site/microblogtrack/

Web¶

The TREC Web Track explores and evaluates Web retrieval technology over large collections of Web data. In its current incarnation, the Web Track has been active since TREC 2009, where it included both a traditional adhoc retrieval task and a new diversity task. The goal of this diversity task is to return a ranked list of pages that together provide complete coverage for a query, while avoiding excessive redundancy in the result list. For TREC 2010 the track introduced a new Web spam task. For both TREC 2011 and 2012, we dropped the spam task but continued the other two tasks essentially unchanged. As we did since TREC 2009, we based our TREC 2012 experiments on the billion-page ClueWeb09 collection created by the Language Technologies Institute at Carnegie Mellon University

Track coordinator(s):

Charles L. A. Clarke, University of Waterloo
Nick Craswell, Microsoft
Ellen M. Voorhees, National Institute of Standards and Technology (NIST)

Track Web Page: https://plg.uwaterloo.ca/~trecweb/2012.html

Contextual Suggestion¶

Overview | Proceedings | Data | Runs | Participants

The contextual suggestion track investigates search techniques for complex information needs that are highly dependent on context and user interests.

Track coordinator(s):

Adriel Dean-Hall, University of Waterloo
Charles L. A. Clarke, University of Waterloo
Jaap Kamps, University of Amsterdam
Paul Thomas, CSIRO
Ellen Voorhees, National Institute of Standards and Technology (NIST)

Track Web Page: https://sites.google.com/site/treccontext/

Medical¶

The TREC Medical Records track fosters research that allows electronic health records to be retrieved based on the semantic content of free-text fields. The ability to find records by matching semantic content will enhance clinical care and support the secondary use of medical records in clinical trials and epidemiological studies. TREC 2012 is the sophomore year of the track, which attracted 24 participating research groups.

Track coordinator(s):

Ellen M. Voorhees, National Institute of Standards and Technology (NIST)
William Hersh, Oregon Health & Science University

Track Web Page: https://www.trec-cds.org/

Session¶

The TREC Session track ran for the third time in 2012. The track has the primary goal of providing test collections and evaluation measures for studying information retrieval over user sessions rather than one-time queries. These test collections are meant to be portable, reusable, statistically powerful, and open to anyone that wishes to work on the problem of retrieval over sessions

Track coordinator(s):

Evangelos Kanoulas, Google
Ben Carterette, University of Delaware
Mark Hall, University of Sheffield
Paul Clough, University of Sheffield
Mark Sanderson, Royal Melbourne Institute of Technology (RMIT University)

Track Web Page: http://ir.cis.udel.edu/sessions

Crowdsourcing¶

Overview | Proceedings | Runs | Participants

In 2012, the Crowdsourcing track had two separate tasks: a text relevance assessing task (TRAT) and an image relevance assessing task (IRAT).

Track coordinator(s):

Mark D. Smucker, University of Waterloo
Gabriella Kazai, Microsoft Research
Matthew Lease, University of Texas at Austin

Track Web Page: https://trec.nist.gov/data/crowd.html

Knowledge Base Acceleration¶

Overview | Proceedings | Data | Runs | Participants

The Knowledge Base Acceleration track in TREC 2012 focused on a single task: filter a time-ordered corpus for documents that are highly relevant to a predefined list of entities. KBA differs from previous filtering evaluations in two primary ways: the stream corpus is >100x larger than previous filtering collections, and the use of entities as topics enables systems to incorporate structured knowledge bases (KB), such as Wikipedia, as external data sources. A successful KBA system must do more than resolve the meaning of entity mentions by linking documents to the KB: it must also distinguish centrally relevant documents that are worth citing in the entity’s WP article. This combines thinking from natural language processing (NLP) and information retrieval (IR).

Track coordinator(s):

John R. Frank, Massachusetts Institute of Technology
Max Kleiman-Weiner, Massachusetts Institute of Technology
Daniel A. Roberts, Massachusetts Institute of Technology
Feng Niu, University of Wisconsin-Madison
Ce Zhang, University of Wisconsin-Madison
Christopher Re, University of Wisconsin-Madison
Ian Soboroff, National Institute of Standards and Technology (NIST)

Track Web Page: https://trec-kba.org/