Participants - RAG TREC Instrument for Multilingual Evaluation (RAGTIME) 2025¶
hltcoe-cascade¶
- Name: Kenton Murray
- Organization: JHU HLTCOE SCALE25 gen cascade
- Runs: keep_all.mt_q3d3 | embed.mt_q3d3 | high-to-low-interview-1-rrf | high-to-low-interview-3-rrf | keep_all.src_q3d3
IITJ¶
- Name: Prasenjeet Roy
- Organization: Indian Institute of Technology Jodhpur
- Runs: eng-IITJ-faiss_bge_v4-run4 | eng-IITJ-faiss_bge_v5-run5 | eng-IITJ-faiss_bge_v6-run6 | eng-IITJ-faiss_bge_v7-run7 | eng-IITJ-faiss_bge_v8-run8 | eng-IITJ-faiss_bge_v9-run9
GenAIus¶
- Name: Reyyan Yeniterzi
- Organization: GenAIus Technologies
- Runs: dry_all3_eng_1000 | dry_gq_eng_1000 | genaius-llama3-3-70B | genaius-gpt-4o | genaius-gpt-oss-120b | genaius-gpt-oss-20b | genaius-question | genaius-cluster
TREMA-UNH¶
- Name: Laura Dietz
- Organization: TREMA lab at the University of New Hampshire
- Runs: chopped2000-crucible-ranking-most_common-ragtime_t+ps_lsr-listllama-ragtime.run-SupportedAnswerExtractor | chopped2000-crucible-ranking-most_common-ragtime_t+ps_lsr-listllama-ragtime.run-SupportedAnswerabilityExtractor | chopped2000-strict-crucible-nugget-references-most_common-SupportedAnswerExtractor | chopped2000-strict-crucible-nugget-references-most_common-SupportedAnswerabilityExtractor | chopped2000-extractive-crucible-nugget-references-most_common-SupportedAnswerExtractor | chopped2000-crucible-nugget-references-most_common-SupportedAnswerExtractor | chopped2000-strict-crucible-ranking-most_common-ragtime_t+ps_lsr-listllama-ragtime.run-SupportedAnswerExtractor | chopped2000-strict-crucible-ranking-most_common-ragtime_t+ps_plaidx-listllama-ragtime.run-SupportedAnswerExtractor | chopped2000-strict-crucible-ranking-most_common-ragtime_t+ps_lsr-listllama-ragtime.run-SupportedAnswerabilityExtractor | chopped2000-strict-crucible-ranking-most_common-ragtime_t+ps_plaidx-listllama-ragtime.run-SupportedAnswerabilityExtractor
coordinators¶
- Name: Ian Soboroff
- Organization: Track Coordinators
- Runs: mt-bm25-td | mt-bm25-title | extractive-rag | bm25-td-rankk | bm25-td-rank1 | bm25-t-rankk | bm25-t-rank1 | bm25-d-rankk | bm25-d-rank1
UvA¶
- Name: Jesse Wonnink
- Organization: University of Amsterdam
- Runs: zetaalpha
HLTCOE¶
- Name: William Walden
- Organization: Human Language Technology Center of Excellence
- Runs: cru-ansR- | cru-ansR-conf- | cru-ablR- | cru-ansR-LSR- | cru-ansR-PlaidX- | cru-ansR-mostcommon- | cru-ansR-bareconf- | cru-ablR-LSR- | cru-ablR-PlaidX- | cru-ablR-conf-
hltcoe-retrieve¶
- Name: Andrew Yates
- Organization: JHU HLTCOE SCALE25 retrieve
- Runs: hltcoe-retrieve-plaidx_lsr_qwen | hltcoe-retrieve-expand_splade_max75 | hltcoe-retrieve-maxsimmmr | hltcoe-retrieve-plaidx-decomp
hltcoe-multiagt¶
- Name: Dawn Lawrie
- Organization: JHU HLTCOE SCALE25 gen multiagent
- Runs: lg-w-ret_plq-nt-s_cite-4q.3l.5r | lg-w-ret_lq-nt-s_cite-4q.3l.5r | lsr_qwen.concise_prompts.2loop.7docs | bulleted_list_plus_rewrite | bulleted_list | lg_nt_4q12r3l_mt_c | lg_nt_4q12r3l_natv_c | lg_e2_3q5r3l | auto_swarm_mt | gptr_nt_q4d4_mt | lg_e2_3q5r2l_mt_qw3 | gptr_nt_q3d3_mt | gptr_e2_q3d3_mt | gptr_ka_q3d3_mt | gptr_ka_q3d3_natv
hltcoe-rerank¶
- Name: Dylan Ju
- Organization: JHU HLTCOE SCALE25 rerank
- Runs: hltcoe-rerank-crux | hltcoe-rerank-listllama-fusion3 | hltcoe-rerank-knn | hltcoe-rerank-searcher | hltime-gpt5.searcher | hltime-lg.crux | hltime-lg.listllama | hltime-lg.jina | hltime-lg.qwen | hltime-lg.jina.qwen | hltime-lg.fsrrf | hltime-lg.fsrrfprf | hltime-lg.searcher | hltime-qwen | hltime-fsrrf | hltime-searcher | hltime-qwen-jina | hltime-listllama | hltime-fsrrfprf | hltime-rankk | hltime-lsr | hltime-plaidx | hltime-fsqwen
hltcoe¶
- Name: Dawn Lawrie
- Organization: Human Lang Tech COE at Johns Hopkins University
- Runs: mlir-hltcoe-METS-dt-title+desc.spacy.bm25.rm3 | hltcoe-retrieve-qwen | plaidx.title-prostat_merged | hltcoe-retrieve-lsr | rankk200.rank1.native_title_desc_spacy_rm3_merged | rankk100_plaidx | rankk100_plaidx_lsr_qwen | rankk100_lsr_qwen
DUTH¶
- Name: Giorgos Arampatzis
- Organization: Democritus University of Thrace
- Runs: duth_mlir_eng_rrf | duth_mlir_xenc | duth-mlir-mlm6 | duth-mlir-mlm6loc | mlir-tblocal | mlir-pybm25 | mlir-mlm12 | mlir-elec | mlir-tb | mlir-fused | mlir-rrf-report | xenc-report | eng_mlm6 | eng_mlm6loc | eng_fused | tblocal | pybm25 | mlm12 | electra | tb
dfki¶
- Name: Alonso Palomino
- Organization: DFKI
- Runs: dfki-milp-baseline | query_expanded+milp-KPE
IDACCS¶
- Name: John Conroy
- Organization: IDA Center for Computing Sciences
- Runs: ragtime_run_75_topics_25_docs_per_query_per_languge_1_20250707_00_08_occams_budget_4000_hybrid_raranked_concat_by_lang | ragtime_run_100_topics_25_docs_per_query_per_languge_1_rus_arb_zho_eng_20250708_14_33_occams_budget_1000_hybrid_10_reranked | ragtime_run_100_topics_25_docs_per_query_per_languge_1_rus_arb_zho_eng_20250708_14_33_occams_budget_1000_nuggets_10_reranked | IDACCS_nugget_4.1 | IDACCS_hybrid_4.1 | IDACCS_nugget_tb4.1 | IDACCS_extract_4.1 | IDACCS_hybridtb_4.1
WueRAG¶
- Name: Julia Wunderle
- Organization: WueRAG
- Runs: WueRAG_2025_07_08_20_05_00 | WueRAG_2025_08_22
ncsu-las¶
- Name: Elizabeth Richerson
- Organization: NCSU LAS
- Runs: las_ag_sel_29 | las_ag_sel_all_4.1 | las_ag_sel_28 | las_ag_sel_new_prompt | las_ag_round_robin
CSU¶
- Name: gao jinchao
- Organization: Centrl South University
- Runs: v1_qwen | v2_split_qwen | v3_surround_glm4
DFKI¶
- Name: Alonso Palomino
- Organization: Deutsche Forschungszentrum für Künstliche Intellig
- Runs: dfki-milp-base | milp-query-expanded
AMU¶
EvalHLTCOE¶
- Name: Will Walden
- Organization: Human Language Technology Center of Excellence - Evaluation Team
- Runs: goldenish-reports-20-most-common.jsonl | bm25-svc-claims | plaidx-svc-questions | plaidx-merged-questions | bm25-common-claims