Skip to content

Overview - RAG TREC Instrument for Multilingual Evaluation (RAGTIME) 2025

Proceedings | Data | Runs | Participants

TREC RAGTIME is a TREC shared task to study and benchmark report generation from news (both English and multi-lingual). Key features of the track are its focus on multi-faceted reports (going beyond factoid QA), and a citation-based evaluation (providing supporting evidence of claims made in the report). It also benchmarks Cross-Language (CLIR) and Multi-lingual (MLIR) retrieval as supporting subtasks.

Track coordinator(s):

  • Dawn Lawrie, Johns Hopkins University, HLTCOE
  • Sean MacAvaney, University of Glasgow
  • James Mayfield, Johns Hopkins University, HLTCOE
  • Luca Soldaini, Allen Institute for AI
  • Eugene Yang, Johns Hopkins University, HLTCOE
  • Andrew Yates, Johns Hopkins University, HLTCOE

Tasks:

  • trec2025-ragtime-dryrun: Dry Run
  • trec2025-ragtime-mlir: Multilingual Retrieval
  • trec2025-ragtime-repgen: Report Generation

Track Web Page: https://trec-ragtime.github.io/