Skip to content

Text REtrieval Conference (TREC) 2023

Clinical Trials

Overview | Data | Runs | Participants

The goal of the Clinical Trials track is to focus research on the clinical trials matching problem: given a free text summary of a patient health record, find suitable clinical trials for that patient.

Track coordinator(s):

  • Dina Demner-Fushman, U.S. National Library of Medicine
  • William Hersh, Oregon Health and Science University
  • Kirk Roberts, University of Texas Health Science Center

Track Web Page: http://www.trec-cds.org/

CrisisFACTs

Overview | Data | Runs | Participants

The CrisisFACTS track focuses on temporal summarization for first responders in emergency situations. These summaries differ from traditional summarization in that they order information by time and produce a series of short updates instead of a longer narrative.

Track coordinator(s):

  • Cody Buntain, University of Maryland
  • Benjamin Horne, University of Tennessee-Knoxville
  • Amanda Hughes, Brigham Young University
  • Muhammad Imran, QCRI
  • Richard McCreadie, University of Glasgow
  • Hemant Purohit, George Mason University

Track Web Page: https://crisisfacts.github.io/

Deep Learning

Overview | Data | Runs | Participants

The Deep Learning track focuses on IR tasks where a large training set is available, allowing us to compare a variety of retrieval approaches including deep neural networks and strong non-neural approaches, to see what works best in a large-data regime.

Track coordinator(s):

  • Nick Craswell, Microsoft
  • Bhaskar Mitra, Microsoft Research
  • Emine Yilmaz, University College London
  • Daniel Campos, University of Illinois at Urbana-Champaign
  • Jimmy Lin, University of Waterloo

Track Web Page: https://microsoft.github.io/msmarco/TREC-Deep-Learning

Interactive Knowledge Assistance

Overview | Data | Runs | Participants

iKAT is the successor to the Conversational Assistance Track (CAsT). The fourth year of CAST aimed to add more conversational elements to the interaction streams, by introducing mixed initiatives (clarifications, and suggestions) to create multi-path, multi-turn conversations for each topic. TREC iKAT evolves CAsT into a new track to signal this new trajectory. iKAT aims to focus on supporting multi-path, multi-turn, multi-perspective conversations. That is for a given topic, the direction and the conversation that evolves depends not only on the prior responses but also on the user.

Track coordinator(s):

  • Mohammed Aliannejadi, University of Amsterdam
  • Zahra Abbasiantaeb, University of Amsterdam
  • Shubham Chatterjee, University of Glasgow
  • Jeff Dalton, University of Glasgow
  • Leif Azzopardi, University of Strathclyde

Track Web Page: https://trecikat.com/

NeuCLIR

Overview | Data | Runs | Participants

Cross-language Information Retrieval (CLIR) has been studied at TREC and subsequent evaluation forums for more than twenty years, but recent advances in the application of deep learning to information retrieval (IR) warrant a new, large-scale effort that will enable exploration of classical and modern IR techniques for this task.

Track coordinator(s):

  • Dawn Lawrie, Johns Hopkins University
  • Sean MacAvaney, University of Glasgow
  • James Mayfield, Johns Hopkins University
  • Paul McNamee, Johns Hopkins University
  • Douglas W. Oard, University of Maryland
  • Luca Soldaini, Allen Institute for AI
  • Eugene Yang, Johns Hopkins University

Track Web Page: https://neuclir.github.io/

AToMiC

Overview | Data | Runs | Participants

The Authoring Tools for Multimedia Content (AToMiC) Track aims to build reliable benchmarks for multimedia search systems. The focus of this track is to develop and evaluate IR techniques for text-to-image and image-to-text search problems.

Track coordinator(s):

  • Jheng-Hong (Matt) Yang, University of Waterloo
  • Jimmy Lin, University of Waterloo
  • Carlos Lassance, Naver Labs Europe
  • Rafael S. Rezende, Naver Labs Europe
  • Stéphane Clinchant, Naver Labs Europe
  • Krishna Srinivasan, Google Research
  • Miriam Redi, Wikimedia Foundation

Track Web Page: https://trec-atomic.github.io/

Overview | Data | Runs | Participants

The product search track focuses on IR tasks in the world of product search and discovery. This track seeks to understand what methods work best for product search, improve evaluation methodology, and provide a reusable dataset which allows easy benchmarking in a public forum.

Track coordinator(s):

  • Daniel Campos, University of Illinois at Urbana-Champaign
  • Corby Rosset, Microsoft
  • Surya Kallumadi, Lowes
  • ChengXiang Zhai, University of Illinois at Urbana-Champaign
  • Alessandro Magnani, Walmart

Track Web Page: https://trec-product-search.github.io/

Tip-of-the-Tongue

Overview | Data | Runs | Participants

The Tip-of-the-Tongue (ToT) Track focuses on the known-item identification task where the searcher has previously experienced or consumed the item (e.g., a movie) but cannot recall a reliable identifier (i.e., It's on the tip of my tongue...). Unlike traditional ad-hoc keyword-based search, these information requests tend to be natural-language, verbose, and complex containing a wide variety of search strategies such as multi-hop reasoning, and frequently express uncertainty and suffer from false memories.

Track coordinator(s):

  • Jaime Arguello, University of North Carolina
  • Samarth Bhargav, University of Amsterdam
  • Bhaskar Mitra, Microsoft Research
  • Fernando Diaz, Google
  • Evangelos Kanoulas, University of Amsterdam

Track Web Page: https://trec-tot.github.io/