Participants - Retrieval-Augmented Generation 2024¶

SGU¶

• Name: Dang Nguyen
• Organization: Saigon University
• Runs: qdrant_bge_small | qrant_bge_gemini

IIIA-UNIPD¶

• Name: Nicola Ferro
• Organization: University of Padua
• Runs: iiia_standard | iiia_dedup | iiia_dedup_p1_reverse_ag | iiia_dedup_p2_reverse_ag | iiia_dedup_p1_straight_ag | iiia_dedup_p2_straight_ag | iiia_standard_p1_reverse_ag | iiia_standard_p2_reverse_ag | iiia_standard_p1_straight_ag | iiia_standard_p2_straight_ag | iiia_standard_p1_straight_ht_ag | iiia_standard_p2_straight_ht_ag | iiia_standard_p1_reverse_ht_ag | iiia_standard_p2_reverse_ht_ag | iiia_dedup_p1_straight_ht_ag | iiia_dedup_p1_reverse_ht_ag | iiia_dedup_p2_reverse_ht_ag | iiia_dedup_p2_straight_ht_ag | iiia_dedup_p1_reverse | iiia_dedup_p2_reverse | iiia_dedup_p1_straight | iiia_dedup_p2_straight | iiia_standard_p1_reverse | iiia_standard_p2_reverse | iiia_standard_p1_straight | iiia_standard_p2_straight | iiia_dedup_p1_reverse_ht | iiia_dedup_p1_straight_ht | iiia_dedup_p2_reverse_ht | iiia_dedup_p2_straight_ht | iiia_standard_p1_reverse_ht | iiia_standard_p1_straight_ht | iiia_standard_p2_reverse_ht | iiia_standard_p2_straight_ht

TLSE3¶

• Name: Jonah Turner
• Organization: Université Toulouse 3 Paul Sabatier
• Runs: SpladeV3_only | SPLADE+Jina | SPLADE+BGEv2m3 | SPLADE+Gemini

h2oloo¶

• Name: Jimmy Lin
• Organization: University of Waterloo
• Runs: fs4+monot5_listgalore.rag24.test_top100 | fs4+monot5_rg4o.rag24.test_top100 | fs4+monot5_rl31-70b.rag24.test_top100 | listgalore_gpt4o_ragnarokv4_top20 | listgalore_l31-70b_ragnarokv4_top20 | listgalore_gpt4o_ragnarokv4nocite_top20 | listgalore_l31-70b_ragnarokv4nocite_top20

ncsu-las¶

• Name: Elizabeth Richerson
• Organization: NCSU LAS
• Runs: LAS_ENN_T5_RERANKED_MXBAI | LAS-splade-mxbai-rrf | LAS-splade-mxbai | LAS_enn_t5 | LAS_ann_t5_qdrant | LAS-splade-mxbai-rrf-mmr8 | LAS-splade-mxbai-mmr8-RAG | LAS-T5-mxbai-mmr8-RAG | LAS-splade-mxbai-rrf-mmr8-doc | LAS_splad_mxbai-rrf-occams_50_RAG

ldisnu¶

• Name: Jongyoon Kim
• Organization: Language & Data Intelligence, Seoul National Uni.
• Runs: ldilab_repllama_listt5_pass2 | ldilab_repllama_listt5_pass1 | ldilab_repllama_listt5_pass3 | ldilab_repllama_listt5_pass4 | ldilab_gpt_4o | dilab_repllama_listt5_pass1_gpt4o | dilab_repllama_listt5_pass2_gpt4o | dilab_repllama_listt5_pass3_gpt4o | dilab_repllama_listt5_pass4_gpt4o

buw¶

• Name: Cem Erturkan
• Organization: Bauhaus University Weimar
• Runs: weaviate_dense_base | dense_on_sparse | oneshot_post_sentenced | buw | buw_2 | buw_3 | buw_5

IRIT¶

• Name: Hanane Djedal
• Organization: Institut de Recherche en Informatique de Toulouse
• Runs: ISIR-IRIT-GEN | ISIR-IRIT-Vanilla | ISIR-IRIT-zephyr_p2 | ISIR-IRIT-zephyr_query_gen | ISIR-IRIT-zephyr_sprompt_3p | ISIR-IRIT-zephyr_query_gen_3p

ielab¶

• Name: Guido Zuccon
• Organization: IElab at The University of Queensland & CSIRO
• Runs: ielab-blender-llama70b-filtered | ielab-blender-llama70b | ielab-blender-llama8b | ielab-blender-llama8b-filtered | ielab-bm25-stella-hybrid | ielab-blender-llama70b-external-only | ielab-blender-llama70b-internal-only | ielab-b70bf-70bqfs-ad_hoc | ielab-b8bf-8bfs-ad_hoc | ielab-b70bf-70bqp-70bafs | ielab-b8bf-8bzs-ad_hoc | ielab-b8bf-8bp-8ba | ielab-b-8bp-8bafs | ielab-b8b-8bp-8bafs | ielab-b8bf-8bp-8bafs | ielab-b70bf-70bqp-rarr

grilllab¶

• Name: Paul Owoicho
• Organization: GRILL Lab
• Runs: grill_fine_grained_rel_2_full_doc_cohere | grill_fine_grained_rel_2_full_doc | grill_fine_grained_summaries | grill_fine_grained_rel_2_summaries_cohere | grill_fine_grained_rel_2_summaries_rrf_bm25_cohere

TREMA-UNH¶

• Name: Laura Dietz
• Organization: TREMA lab at the University of New Hampshire
• Runs: Ranked_Iterative_Fact_Extraction_and_Refinement | Enhanced_Iterative_Fact_Refinement_and_Prioritization | Ranked_Iterative_Fact_Extraction_and_Refinement_RIFER_-_bm25

WaterlooClarke¶

• Name: Charles Clarke
• Organization: University of Waterloo (Clarke)
• Runs: monster | uwc1 | uwc2 | uwc0 | uwcCQAR | uwcCQA | uwcCQR | uwcCQ | uwcBA | uwcBQ | UWCrag | UWCrag_stepbystep | UWCgarag

IITD-IRL¶

• Name: Shivani Choudhary
• Organization: Indian Institute of Technology, Delhi
• Runs: zeph_test_rag_rrf_expand_Rtask | zeph_test_rag_rrf_raw_query_Rtrack | zeph_test_rag24_doc_query_expansion+rrf_Rtask | test.rag24.rrf.expanded.BM25.MiniLM | test.rag24.no.rrf.no.expansion | test.rag24.rrf.raw.query.MiniLM+BM25 | ag_rag_gpt35_expansion_rrf_20 | ag_rag_mistral_expansion_rrf_20 | ag_rag_mistral_expansion_rrf_15 | ag_rag_mistral_expansion_rrf_7 | ag_rag_gpt35_expansion_rrf_15 | ag_rag_gpt35_expansion_rrf_7 | zeph_test_rag_rrf_expand_query | zeph_test_rag_rrf_raw_query | zeph_test_rag24_doc_query_expansion+rrf | zeph_test_rag_rrf_expand_query_mistral | zeph_test_rag_rrf_expand_mistral_top_15 | zeph_test_rag_rrf_expand_top_5 | zeph_test_rag_rrf_expand_top_10

coordinators¶

• Name: Ian Soboroff
• Organization: Track Coordinators
• Runs: fs4_bm25+rocchio_snowael_snowaem_gtel+monot5_rrf+rz_rrf.rag24.test | anserini_bm25+rocchio.rag24.test_top100 | anserini_bm25.rag24.test_top100 | fs4+monot5_rz.rag24.test_top100 | fs4+monot5.rag24.test_top100 | baseline_rag24.test_gpt-4o_top20 | baseline_rag24.test_l31_70b_instruct_top20 | baseline_rag24.test_command-r-plus_top20 | baseline_frag_rag24.test_gpt-4o_top20 | baseline_frag_rag24.test_command-r-plus_top20

citi¶

• Name: TZU-SHUO WEI
• Organization: academia sinica
• Runs: ASCITI_co_gpt | ASCITI_co_bge | ASCITI_co_co | BEST_cot_gpt3.5 | BEST_gpt3.5_another_prompt | SECOND_cot_gpt3.5 | BEST_gpt3.5_new_prompt | BEST_gpt3.5 | SECOND_gpt3.5_new_prompt | SECOND_gpt3.5

KML¶

• Name: KAMLESH KARKI
• Organization: Kumoan Machine Learning
• Runs: sim_and_rerank_v1 | sim_and_rerank_200_docs | gpt_mini | cohere+post_processing | gpt_mini_double_prompt

CIR¶

• Name: Jüri Keller
• Organization: TH Köln – University of Applied Sciences
• Runs: cir_gpt-4o-mini_Jaccard_50_0.5_100_301_p0 | cir_gpt-4o-mini_Jaccard_50_1.0_100_301_p0 | cir_gpt-4o-mini_Cosine_50_0.5_100_301_p1 | cir_gpt-4o-mini_Cosine_50_0.25_100_301_p1 | cir_gpt-4o-mini_Cosine_50_0.75_100_301_p1 | cir_gpt-4o-mini_Cosine_50_1.0_100_301_p1 | cir_gpt-4o-mini_Cosine_20_0.5_100_301_p1 | cir_gpt-4o-mini_Cosine_50_0.5_100_301_p2 | cir_gpt-4o-mini_Cosine_50_0.5_100_301_p3 | cir_gpt-4o-mini_no_reranking_50_0.5_100_301_p1

ii_research¶

• Name: Haitao Yu
• Organization: II-Lab, University of Tsukuba
• Runs: iiresearch-bm25-top10-llama3-8b-instruct

neu¶

• Name: Sen Mei
• Organization: Northeastern University
• Runs: neu | neurerank | neurag | neuragfix

uog-tht¶

• Name: Huu Tri Tran
• Organization: University of Glasgow
• Runs: FT-llama3 | PG-mistral | ICL-mistral

softbank-meisei¶

• Name: Kazuya Ueki
• Organization: SoftBank Corp. and Meisei University
• Runs: rtask-bm25-colbert_faiss | rtask-bm25-rank_zephyr | agtask-bm25-colbert_faiss-gpt4o-llama70b | rag_bm25-colbert_faiss-gpt4o-llama70b | ragtask-bm25-rank_zephyr-gpt4o-llama70b

Ruc01¶

• Name: shengjie ma
• Organization: Renmin University of China Gaoling School
• Runs: ruc001

webis¶

• Name: Alexander Bondarenko
• Organization: Webis @ Jena, Leipzig, Weimar
• Runs: webis-01 | webis-02 | webis-03 | webis-04 | webis-05 | webis-ag-run0-taskrag | webis-ag-run1-taskrag | webis-ag-run3-reuserag | webis-ag-run2-reuserag | webis-manual | webis-rag-run0-taskrag | webis-rag-run1-taskrag | webis-rag-run3-taskrag | webis-rag-run4-reuserag | webis-rag-run5-reuserag

uis-iai¶

• Name: Krisztian Balog
• Organization: University of Stavanger
• Runs: ginger_top_5 | baseline_top_5 | ginger-fluency_top_5 | ginger-fluency_top_10 | ginger-fluency_top_20

InfoLab¶

• Name: Damian Martinez
• Organization: University of Delaware
• Runs: UDInfolab.bge | UDInfolab.bge.AnsAi | UDInfolab.bge.query | UDInfolab.bge.ranker | UDInfolab.bm25.ro.tuned | UDInfolab.bm25.ro | UDInfolab.bm25 | UDInfolab.AG-v1 | UDInfolab.AG-v2 | UDInfolab.bgeV2 | UDInfolab.RAG.AnsAI | UDInfolab.RAG.Query | UDInfolab.RAG.bge.tuned | UDInfolab.RAG.bge.QueryAgm.tuned | UDInfolab.RAG.bge.QueryAnsAI.tuned