Participants - RAG TREC Instrument for Multilingual Evaluation (RAGTIME) 2025¶

hltcoe-cascade¶

• Name: Kenton Murray
• Organization: JHU HLTCOE SCALE25 gen cascade
• Runs: keep_all.mt_q3d3 | embed.mt_q3d3 | high-to-low-interview-1-rrf | high-to-low-interview-3-rrf | keep_all.src_q3d3

DFKI¶

• Name: Alonso Palomino
• Organization: Deutsche Forschungszentrum für Künstliche Intellig
• Runs: dfki-milp-base | milp-query-expanded

EvalHLTCOE¶

• Name: Will Walden
• Organization: Human Language Technology Center of Excellence - Evaluation Team
• Runs: goldenish-reports-20-most-common.jsonl | bm25-svc-claims | plaidx-svc-questions | plaidx-merged-questions | bm25-common-claims

TREMA-UNH¶

• Name: Laura Dietz
• Organization: TREMA lab at the University of New Hampshire
• Runs: chopped2000-crucible-ranking-most_common-ragtime_t+ps_lsr-listllama-ragtime.run-SupportedAnswerExtractor | chopped2000-crucible-ranking-most_common-ragtime_t+ps_lsr-listllama-ragtime.run-SupportedAnswerabilityExtractor | chopped2000-strict-crucible-nugget-references-most_common-SupportedAnswerExtractor | chopped2000-strict-crucible-nugget-references-most_common-SupportedAnswerabilityExtractor | chopped2000-extractive-crucible-nugget-references-most_common-SupportedAnswerExtractor | chopped2000-crucible-nugget-references-most_common-SupportedAnswerExtractor | chopped2000-strict-crucible-ranking-most_common-ragtime_t+ps_lsr-listllama-ragtime.run-SupportedAnswerExtractor | chopped2000-strict-crucible-ranking-most_common-ragtime_t+ps_plaidx-listllama-ragtime.run-SupportedAnswerExtractor | chopped2000-strict-crucible-ranking-most_common-ragtime_t+ps_lsr-listllama-ragtime.run-SupportedAnswerabilityExtractor | chopped2000-strict-crucible-ranking-most_common-ragtime_t+ps_plaidx-listllama-ragtime.run-SupportedAnswerabilityExtractor

hltcoe-rerank¶

• Name: Dylan Ju
• Organization: JHU HLTCOE SCALE25 rerank
• Runs: hltcoe-rerank-crux | hltcoe-rerank-listllama-fusion3 | hltcoe-rerank-knn | hltcoe-rerank-searcher | hltime-gpt5.searcher | hltime-lg.crux | hltime-lg.listllama | hltime-lg.jina | hltime-lg.qwen | hltime-lg.jina.qwen | hltime-lg.fsrrf | hltime-lg.fsrrfprf | hltime-lg.searcher | hltime-qwen | hltime-fsrrf | hltime-searcher | hltime-qwen-jina | hltime-listllama | hltime-fsrrfprf | hltime-rankk | hltime-lsr | hltime-plaidx | hltime-fsqwen

GenAIus¶

• Name: Reyyan Yeniterzi
• Organization: GenAIus Technologies
• Runs: dry_all3_eng_1000 | dry_gq_eng_1000 | genaius-llama3-3-70B | genaius-gpt-4o | genaius-gpt-oss-120b | genaius-gpt-oss-20b | genaius-question | genaius-cluster

dfki¶

• Name: Alonso Palomino
• Organization: DFKI
• Runs: dfki-milp-baseline | query_expanded+milp-KPE

AMU¶

• Name: Piotr Jablonski
• Organization: Adam Mickiewicz University
• Runs: AMU1ML | AMU1ENG

DUTH¶

• Name: Giorgos Arampatzis
• Organization: Democritus University of Thrace
• Runs: duth_mlir_eng_rrf | duth_mlir_xenc | duth-mlir-mlm6 | duth-mlir-mlm6loc | mlir-tblocal | mlir-pybm25 | mlir-mlm12 | mlir-elec | mlir-tb | mlir-fused | mlir-rrf-report | xenc-report | eng_mlm6 | eng_mlm6loc | eng_fused | tblocal | pybm25 | mlm12 | electra | tb

CSU¶

• Name: gao jinchao
• Organization: Centrl South University
• Runs: v1_qwen | v2_split_qwen | v3_surround_glm4

UvA¶

• Name: Jesse Wonnink
• Organization: University of Amsterdam
• Runs: zetaalpha

ncsu-las¶

• Name: Elizabeth Richerson
• Organization: NCSU LAS
• Runs: las_ag_sel_29 | las_ag_sel_all_4.1 | las_ag_sel_28 | las_ag_sel_new_prompt | las_ag_round_robin

WueRAG¶

• Name: Julia Wunderle
• Organization: WueRAG
• Runs: WueRAG_2025_07_08_20_05_00 | WueRAG_2025_08_22

hltcoe-retrieve¶

• Name: Andrew Yates
• Organization: JHU HLTCOE SCALE25 retrieve
• Runs: hltcoe-retrieve-plaidx_lsr_qwen | hltcoe-retrieve-expand_splade_max75 | hltcoe-retrieve-maxsimmmr | hltcoe-retrieve-plaidx-decomp

IITJ¶

• Name: Prasenjeet Roy
• Organization: Indian Institute of Technology Jodhpur
• Runs: eng-IITJ-faiss_bge_v4-run4 | eng-IITJ-faiss_bge_v5-run5 | eng-IITJ-faiss_bge_v6-run6 | eng-IITJ-faiss_bge_v7-run7 | eng-IITJ-faiss_bge_v8-run8 | eng-IITJ-faiss_bge_v9-run9

hltcoe¶

• Name: Dawn Lawrie
• Organization: Human Lang Tech COE at Johns Hopkins University
• Runs: mlir-hltcoe-METS-dt-title+desc.spacy.bm25.rm3 | hltcoe-retrieve-qwen | plaidx.title-prostat_merged | hltcoe-retrieve-lsr | rankk200.rank1.native_title_desc_spacy_rm3_merged | rankk100_plaidx | rankk100_plaidx_lsr_qwen | rankk100_lsr_qwen

HLTCOE¶

• Name: William Walden
• Organization: Human Language Technology Center of Excellence
• Runs: cru-ansR- | cru-ansR-conf- | cru-ablR- | cru-ansR-LSR- | cru-ansR-PlaidX- | cru-ansR-mostcommon- | cru-ansR-bareconf- | cru-ablR-LSR- | cru-ablR-PlaidX- | cru-ablR-conf-

coordinators¶

• Name: Ian Soboroff
• Organization: Track Coordinators
• Runs: mt-bm25-td | mt-bm25-title | extractive-rag | bm25-td-rankk | bm25-td-rank1 | bm25-t-rankk | bm25-t-rank1 | bm25-d-rankk | bm25-d-rank1

hltcoe-multiagt¶

• Name: Dawn Lawrie
• Organization: JHU HLTCOE SCALE25 gen multiagent
• Runs: lg-w-ret_plq-nt-s_cite-4q.3l.5r | lg-w-ret_lq-nt-s_cite-4q.3l.5r | lsr_qwen.concise_prompts.2loop.7docs | bulleted_list_plus_rewrite | bulleted_list | lg_nt_4q12r3l_mt_c | lg_nt_4q12r3l_natv_c | lg_e2_3q5r3l | auto_swarm_mt | gptr_nt_q4d4_mt | lg_e2_3q5r2l_mt_qw3 | gptr_nt_q3d3_mt | gptr_e2_q3d3_mt | gptr_ka_q3d3_mt | gptr_ka_q3d3_natv

IDACCS¶

• Name: John Conroy
• Organization: IDA Center for Computing Sciences
• Runs: ragtime_run_75_topics_25_docs_per_query_per_languge_1_20250707_00_08_occams_budget_4000_hybrid_raranked_concat_by_lang | ragtime_run_100_topics_25_docs_per_query_per_languge_1_rus_arb_zho_eng_20250708_14_33_occams_budget_1000_hybrid_10_reranked | ragtime_run_100_topics_25_docs_per_query_per_languge_1_rus_arb_zho_eng_20250708_14_33_occams_budget_1000_nuggets_10_reranked | IDACCS_nugget_4.1 | IDACCS_hybrid_4.1 | IDACCS_nugget_tb4.1 | IDACCS_extract_4.1 | IDACCS_hybridtb_4.1