Results - Conversational Assistance 2019¶
mpi_bert¶
Metadata | Participants | Proceedings| Input| Summary| Appendix
summary (trec_eval)
map all 0.1670
bpref all 0.3304
recip_rank all 0.6001
P_10 all 0.4266
P_100 all 0.1468
P_1000 all 0.0242
recall_10 all 0.1009
recall_100 all 0.3376
recall_1000 all 0.5196
ndcg all 0.3837
ndcg_cut_10 all 0.3204
ndcg_cut_100 all 0.3148
ndcg_cut_1000 all 0.3837
mpi_base¶
Metadata | Participants | Proceedings| Input| Summary| Appendix
summary (trec_eval)
map all 0.1740
bpref all 0.3111
recip_rank all 0.5104
P_10 all 0.3595
P_100 all 0.1499
P_1000 all 0.0299
recall_10 all 0.0810
recall_100 all 0.3368
recall_1000 all 0.6257
ndcg all 0.3838
ndcg_cut_10 all 0.2440
ndcg_cut_100 all 0.2723
ndcg_cut_1000 all 0.3838
BM25_BERT_FC¶
Metadata | Participants | Proceedings| Input| Summary| Appendix
summary (trec_eval)
map all 0.1592
bpref all 0.2584
recip_rank all 0.6047
P_10 all 0.4769
P_100 all 0.1183
P_1000 all 0.0178
recall_10 all 0.1085
recall_100 all 0.2466
recall_1000 all 0.3624
ndcg all 0.3134
ndcg_cut_10 all 0.3581
ndcg_cut_100 all 0.2728
ndcg_cut_1000 all 0.3134
BM25_BERT_RANKF¶
Metadata | Participants | Proceedings| Input| Summary| Appendix
summary (trec_eval)
map all 0.1590
bpref all 0.2564
recip_rank all 0.6000
P_10 all 0.4705
P_100 all 0.1182
P_1000 all 0.0178
recall_10 all 0.1056
recall_100 all 0.2462
recall_1000 all 0.3624
ndcg all 0.3122
ndcg_cut_10 all 0.3516
ndcg_cut_100 all 0.2715
ndcg_cut_1000 all 0.3122
galago_rel_q¶
Metadata | Participants | Proceedings| Input| Summary| Appendix
summary (trec_eval)
map all 0.1051
bpref all 0.1936
recip_rank all 0.3967
P_10 all 0.3012
P_100 all 0.0982
P_1000 all 0.0173
recall_10 all 0.0634
recall_100 all 0.1885
recall_1000 all 0.3268
ndcg all 0.2314
ndcg_cut_10 all 0.1900
ndcg_cut_100 all 0.1762
ndcg_cut_1000 all 0.2314
bertrr_rel_q¶
Metadata | Participants | Proceedings| Input| Summary| Appendix
summary (trec_eval)
map all 0.1417
bpref all 0.2377
recip_rank all 0.5185
P_10 all 0.4058
P_100 all 0.1106
P_1000 all 0.0173
recall_10 all 0.0819
recall_100 all 0.2173
recall_1000 all 0.3268
ndcg all 0.2774
ndcg_cut_10 all 0.2977
ndcg_cut_100 all 0.2384
ndcg_cut_1000 all 0.2774
galago_rel_1st¶
Metadata | Participants | Proceedings| Input| Summary| Appendix
summary (trec_eval)
map all 0.1124
bpref all 0.2081
recip_rank all 0.4289
P_10 all 0.3110
P_100 all 0.1043
P_1000 all 0.0182
recall_10 all 0.0622
recall_100 all 0.1999
recall_1000 all 0.3511
ndcg all 0.2468
ndcg_cut_10 all 0.1981
ndcg_cut_100 all 0.1879
ndcg_cut_1000 all 0.2468
bertrr_rel_1st¶
Metadata | Participants | Proceedings| Input| Summary| Appendix
summary (trec_eval)
map all 0.1463
bpref all 0.2522
recip_rank all 0.5422
P_10 all 0.4046
P_100 all 0.1137
P_1000 all 0.0182
recall_10 all 0.0854
recall_100 all 0.2279
recall_1000 all 0.3511
ndcg all 0.2913
ndcg_cut_10 all 0.2994
ndcg_cut_100 all 0.2472
ndcg_cut_1000 all 0.2913
clacBase¶
Metadata | Participants | Proceedings| Input| Summary| Appendix
summary (trec_eval)
map all 0.2479
bpref all 0.4064
recip_rank all 0.6435
P_10 all 0.5052
P_100 all 0.1790
P_1000 all 0.0331
recall_10 all 0.1148
recall_100 all 0.3827
recall_1000 all 0.6946
ndcg all 0.4712
ndcg_cut_10 all 0.3561
ndcg_cut_100 all 0.3535
ndcg_cut_1000 all 0.4712
clacBaseRerank¶
Metadata | Participants | Proceedings| Input| Summary| Appendix
summary (trec_eval)
map all 0.2456
bpref all 0.4061
recip_rank all 0.6329
P_10 all 0.5040
P_100 all 0.1790
P_1000 all 0.0331
recall_10 all 0.1145
recall_100 all 0.3827
recall_1000 all 0.6946
ndcg all 0.4702
ndcg_cut_10 all 0.3565
ndcg_cut_100 all 0.3526
ndcg_cut_1000 all 0.4702
clacMagic¶
Metadata | Participants | Proceedings| Input| Summary| Appendix
summary (trec_eval)
map all 0.3037
bpref all 0.5024
recip_rank all 0.6905
P_10 all 0.5272
P_100 all 0.2072
P_1000 all 0.0365
recall_10 all 0.1385
recall_100 all 0.4862
recall_1000 all 0.7892
ndcg all 0.5489
ndcg_cut_10 all 0.4020
ndcg_cut_100 all 0.4322
ndcg_cut_1000 all 0.5489
clacMagicRerank¶
Metadata | Participants | Proceedings| Input| Summary| Appendix
summary (trec_eval)
map all 0.3028
bpref all 0.4962
recip_rank all 0.7360
P_10 all 0.5445
P_100 all 0.2072
P_1000 all 0.0365
recall_10 all 0.1355
recall_100 all 0.4862
recall_1000 all 0.7892
ndcg all 0.5489
ndcg_cut_10 all 0.4099
ndcg_cut_100 all 0.4322
ndcg_cut_1000 all 0.5489
MPmlp¶
Metadata | Participants| Input| Summary| Appendix
summary (trec_eval)
map all 0.0540
bpref all 0.2108
recip_rank all 0.2871
P_10 all 0.1786
P_100 all 0.0552
P_1000 all 0.0301
recall_10 all 0.0318
recall_100 all 0.1053
recall_1000 all 0.5878
ndcg all 0.2677
ndcg_cut_10 all 0.1017
ndcg_cut_100 all 0.0914
ndcg_cut_1000 all 0.2677
MPgate¶
Metadata | Participants| Input| Summary| Appendix
summary (trec_eval)
map all 0.0536
bpref all 0.2059
recip_rank all 0.2837
P_10 all 0.1671
P_100 all 0.0518
P_1000 all 0.0301
recall_10 all 0.0272
recall_100 all 0.0967
recall_1000 all 0.5878
ndcg all 0.2666
ndcg_cut_10 all 0.1064
ndcg_cut_100 all 0.0862
ndcg_cut_1000 all 0.2666
SMNmlp¶
Metadata | Participants| Input| Summary| Appendix
summary (trec_eval)
map all 0.0605
bpref all 0.2525
recip_rank all 0.2453
P_10 all 0.1150
P_100 all 0.0673
P_1000 all 0.0301
recall_10 all 0.0233
recall_100 all 0.1373
recall_1000 all 0.5878
ndcg all 0.2794
ndcg_cut_10 all 0.0877
ndcg_cut_100 all 0.1128
ndcg_cut_1000 all 0.2794
UDInfoC_BL¶
Metadata | Participants | Proceedings| Input| Summary| Appendix
summary (trec_eval)
map all 0.0753
bpref all 0.0911
recip_rank all 0.5991
P_10 all 0.4064
P_100 all 0.0406
P_1000 all 0.0041
recall_10 all 0.0968
recall_100 all 0.0968
recall_1000 all 0.0968
ndcg all 0.1573
ndcg_cut_10 all 0.2977
ndcg_cut_100 all 0.1579
ndcg_cut_1000 all 0.1573
ilps-lm-rm3-dt¶
Metadata | Participants | Proceedings| Input| Summary| Appendix
summary (trec_eval)
map all 0.2307
bpref all 0.3434
recip_rank all 0.5309
P_10 all 0.3954
P_100 all 0.1756
P_1000 all 0.0330
recall_10 all 0.0980
recall_100 all 0.3719
recall_1000 all 0.6503
ndcg all 0.4253
ndcg_cut_10 all 0.2746
ndcg_cut_100 all 0.3124
ndcg_cut_1000 all 0.4253
ilps-bert-feat1¶
Metadata | Participants| Input| Summary| Appendix
summary (trec_eval)
map all 0.2614
bpref all 0.3915
recip_rank all 0.6176
P_10 all 0.4711
P_100 all 0.1894
P_1000 all 0.0343
recall_10 all 0.1122
recall_100 all 0.4089
recall_1000 all 0.6808
ndcg all 0.4857
ndcg_cut_10 all 0.3697
ndcg_cut_100 all 0.3835
ndcg_cut_1000 all 0.4857
ilps-bert-feat2¶
Metadata | Participants| Input| Summary| Appendix
summary (trec_eval)
map all 0.2576
bpref all 0.3894
recip_rank all 0.6060
P_10 all 0.4491
P_100 all 0.1864
P_1000 all 0.0339
recall_10 all 0.1141
recall_100 all 0.4042
recall_1000 all 0.6741
ndcg all 0.4739
ndcg_cut_10 all 0.3488
ndcg_cut_100 all 0.3700
ndcg_cut_1000 all 0.4739
SMNgate¶
Metadata | Participants| Input| Summary| Appendix
summary (trec_eval)
map all 0.0296
bpref all 0.2364
recip_rank all 0.0726
P_10 all 0.0289
P_100 all 0.0268
P_1000 all 0.0301
recall_10 all 0.0082
recall_100 all 0.0644
recall_1000 all 0.5878
ndcg all 0.2334
ndcg_cut_10 all 0.0160
ndcg_cut_100 all 0.0390
ndcg_cut_1000 all 0.2334
UDInfoC_TS¶
Metadata | Participants | Proceedings| Input| Summary| Appendix
summary (trec_eval)
map all 0.0670
bpref all 0.0853
recip_rank all 0.5697
P_10 all 0.3931
P_100 all 0.0393
P_1000 all 0.0039
recall_10 all 0.0917
recall_100 all 0.0917
recall_1000 all 0.0917
ndcg all 0.1392
ndcg_cut_10 all 0.2705
ndcg_cut_100 all 0.1398
ndcg_cut_1000 all 0.1392
RUCIR-run3¶
Metadata | Participants | Proceedings| Input| Summary| Appendix
summary (trec_eval)
map all 0.0930
bpref all 0.1164
recip_rank all 0.5051
P_10 all 0.3832
P_100 all 0.0634
P_1000 all 0.0063
recall_10 all 0.0855
recall_100 all 0.1367
recall_1000 all 0.1367
ndcg all 0.1725
ndcg_cut_10 all 0.2632
ndcg_cut_100 all 0.1735
ndcg_cut_1000 all 0.1725
RUCIR-run1¶
Metadata | Participants | Proceedings| Input| Summary| Appendix
summary (trec_eval)
map all 0.1635
bpref all 0.1931
recip_rank all 0.7293
P_10 all 0.5428
P_100 all 0.0902
P_1000 all 0.0090
recall_10 all 0.1404
recall_100 all 0.2280
recall_1000 all 0.2280
ndcg all 0.2815
ndcg_cut_10 all 0.4085
ndcg_cut_100 all 0.2825
ndcg_cut_1000 all 0.2815
RUCIR-run2¶
Metadata | Participants | Proceedings| Input| Summary| Appendix
summary (trec_eval)
map all 0.0924
bpref all 0.1154
recip_rank all 0.4971
P_10 all 0.3769
P_100 all 0.0634
P_1000 all 0.0063
recall_10 all 0.0821
recall_100 all 0.1367
recall_1000 all 0.1367
ndcg all 0.1716
ndcg_cut_10 all 0.2582
ndcg_cut_100 all 0.1726
ndcg_cut_1000 all 0.1716
coref_shift_qe¶
Metadata | Participants | Proceedings| Input| Summary| Appendix
summary (trec_eval)
map all 0.2247
bpref all 0.3530
recip_rank all 0.5120
P_10 all 0.4133
P_100 all 0.1783
P_1000 all 0.0341
recall_10 all 0.0835
recall_100 all 0.3517
recall_1000 all 0.6451
ndcg all 0.4223
ndcg_cut_10 all 0.2862
ndcg_cut_100 all 0.3092
ndcg_cut_1000 all 0.4223
RUCIR-run4¶
Metadata | Participants | Proceedings| Input| Summary| Appendix
summary (trec_eval)
map all 0.1058
bpref all 0.1301
recip_rank all 0.5302
P_10 all 0.3803
P_100 all 0.0673
P_1000 all 0.0067
recall_10 all 0.0872
recall_100 all 0.1534
recall_1000 all 0.1534
ndcg all 0.1893
ndcg_cut_10 all 0.2714
ndcg_cut_100 all 0.1901
ndcg_cut_1000 all 0.1893
ensemble¶
Metadata | Participants | Proceedings| Input| Summary| Appendix
summary (trec_eval)
map all 0.2589
bpref all 0.3726
recip_rank all 0.5905
P_10 all 0.4364
P_100 all 0.1994
P_1000 all 0.0380
recall_10 all 0.1007
recall_100 all 0.4179
recall_1000 all 0.7734
ndcg all 0.4918
ndcg_cut_10 all 0.3079
ndcg_cut_100 all 0.3583
ndcg_cut_1000 all 0.4918
coref_cshift¶
Metadata | Participants | Proceedings| Input| Summary| Appendix
summary (trec_eval)
map all 0.2138
bpref all 0.3403
recip_rank all 0.5077
P_10 all 0.3890
P_100 all 0.1746
P_1000 all 0.0341
recall_10 all 0.0821
recall_100 all 0.3514
recall_1000 all 0.6412
ndcg all 0.4183
ndcg_cut_10 all 0.2695
ndcg_cut_100 all 0.3034
ndcg_cut_1000 all 0.4183
manual_indri¶
Metadata | Participants | Proceedings| Input| Summary| Appendix
summary (trec_eval)
map all 0.3107
bpref all 0.4299
recip_rank all 0.6636
P_10 all 0.5231
P_100 all 0.2169
P_1000 all 0.0373
recall_10 all 0.1353
recall_100 all 0.4975
recall_1000 all 0.7895
ndcg all 0.5361
ndcg_cut_10 all 0.3784
ndcg_cut_100 all 0.4237
ndcg_cut_1000 all 0.5361
ECNUICA_BERT¶
Metadata | Participants | Proceedings| Input| Summary| Appendix
summary (trec_eval)
map all 0.0078
bpref all 0.0497
recip_rank all 0.1066
P_10 all 0.0370
P_100 all 0.0124
P_1000 all 0.0038
recall_10 all 0.0064
recall_100 all 0.0213
recall_1000 all 0.0657
ndcg all 0.0348
ndcg_cut_10 all 0.0203
ndcg_cut_100 all 0.0189
ndcg_cut_1000 all 0.0348
ECNUICA_ORI¶
Metadata | Participants | Proceedings| Input| Summary| Appendix
summary (trec_eval)
map all 0.1907
bpref all 0.3527
recip_rank all 0.5219
P_10 all 0.3908
P_100 all 0.1546
P_1000 all 0.0307
recall_10 all 0.0988
recall_100 all 0.3773
recall_1000 all 0.6775
ndcg all 0.4116
ndcg_cut_10 all 0.2534
ndcg_cut_100 all 0.2923
ndcg_cut_1000 all 0.4116
ECNUICA_MIX¶
Metadata | Participants | Proceedings| Input| Summary| Appendix
summary (trec_eval)
map all 0.1718
bpref all 0.3286
recip_rank all 0.5246
P_10 all 0.3590
P_100 all 0.1429
P_1000 all 0.0281
recall_10 all 0.0915
recall_100 all 0.3440
recall_1000 all 0.6187
ndcg all 0.3774
ndcg_cut_10 all 0.2351
ndcg_cut_100 all 0.2700
ndcg_cut_1000 all 0.3774
CFDA_CLIP_RUN1¶
Metadata | Participants | Proceedings| Input| Summary| Appendix
summary (trec_eval)
map all 0.2255
bpref all 0.3335
recip_rank all 0.7762
P_10 all 0.5827
P_100 all 0.1465
P_1000 all 0.0199
recall_10 all 0.1503
recall_100 all 0.3286
recall_1000 all 0.4120
ndcg all 0.3921
ndcg_cut_10 all 0.4599
ndcg_cut_100 all 0.3618
ndcg_cut_1000 all 0.3921
CFDA_CLIP_RUN8¶
Metadata | Participants | Proceedings| Input| Summary| Appendix
summary (trec_eval)
map all 0.3630
bpref all 0.5157
recip_rank all 0.8584
P_10 all 0.7058
P_100 all 0.2160
P_1000 all 0.0334
recall_10 all 0.1958
recall_100 all 0.5011
recall_1000 all 0.6945
ndcg all 0.5883
ndcg_cut_10 all 0.5767
ndcg_cut_100 all 0.5172
ndcg_cut_1000 all 0.5883
CFDA_CLIP_RUN6¶
Metadata | Participants | Proceedings| Input| Summary| Appendix
summary (trec_eval)
map all 0.3946
bpref all 0.5693
recip_rank all 0.8662
P_10 all 0.7052
P_100 all 0.2275
P_1000 all 0.0379
recall_10 all 0.2011
recall_100 all 0.5541
recall_1000 all 0.8119
ndcg all 0.6503
ndcg_cut_10 all 0.5872
ndcg_cut_100 all 0.5570
ndcg_cut_1000 all 0.6503
CFDA_CLIP_RUN7¶
Metadata | Participants | Proceedings| Input| Summary| Appendix
summary (trec_eval)
map all 0.2686
bpref all 0.4456
recip_rank all 0.7187
P_10 all 0.5116
P_100 all 0.1839
P_1000 all 0.0293
recall_10 all 0.1324
recall_100 all 0.4169
recall_1000 all 0.6111
ndcg all 0.4863
ndcg_cut_10 all 0.4203
ndcg_cut_100 all 0.4154
ndcg_cut_1000 all 0.4863
h2oloo_RUN2¶
Metadata | Participants| Input| Summary| Appendix
summary (trec_eval)
map all 0.2742
bpref all 0.4516
recip_rank all 0.7177
P_10 all 0.5139
P_100 all 0.1894
P_1000 all 0.0310
recall_10 all 0.1300
recall_100 all 0.4249
recall_1000 all 0.6323
ndcg all 0.4959
ndcg_cut_10 all 0.4201
ndcg_cut_100 all 0.4201
ndcg_cut_1000 all 0.4959
h2oloo_RUN3¶
Metadata | Participants| Input| Summary| Appendix
summary (trec_eval)
map all 0.3239
bpref all 0.4682
recip_rank all 0.8147
P_10 all 0.6601
P_100 all 0.2043
P_1000 all 0.0310
recall_10 all 0.1730
recall_100 all 0.4660
recall_1000 all 0.6323
ndcg all 0.5407
ndcg_cut_10 all 0.5342
ndcg_cut_100 all 0.4793
ndcg_cut_1000 all 0.5407
h2oloo_RUN4¶
Metadata | Participants| Input| Summary| Appendix
summary (trec_eval)
map all 0.3207
bpref all 0.4695
recip_rank all 0.8152
P_10 all 0.6601
P_100 all 0.2013
P_1000 all 0.0312
recall_10 all 0.1729
recall_100 all 0.4601
recall_1000 all 0.6388
ndcg all 0.5411
ndcg_cut_10 all 0.5374
ndcg_cut_100 all 0.4757
ndcg_cut_1000 all 0.5411
h2oloo_RUN5¶
Metadata | Participants| Input| Summary| Appendix
summary (trec_eval)
map all 0.3536
bpref all 0.5057
recip_rank all 0.8686
P_10 all 0.6717
P_100 all 0.2135
P_1000 all 0.0335
recall_10 all 0.1846
recall_100 all 0.4918
recall_1000 all 0.6893
ndcg all 0.5801
ndcg_cut_10 all 0.5594
ndcg_cut_100 all 0.5080
ndcg_cut_1000 all 0.5801
ilps-bert-featq¶
Metadata | Participants| Input| Summary| Appendix
summary (trec_eval)
map all 0.2636
bpref all 0.3963
recip_rank all 0.6569
P_10 all 0.4815
P_100 all 0.1868
P_1000 all 0.0339
recall_10 all 0.1230
recall_100 all 0.4070
recall_1000 all 0.6735
ndcg all 0.4801
ndcg_cut_10 all 0.3651
ndcg_cut_100 all 0.3765
ndcg_cut_1000 all 0.4801
VESBERT¶
Metadata | Participants | Proceedings| Input| Summary| Appendix
summary (trec_eval)
map all 0.1251
bpref all 0.1585
recip_rank all 0.5436
P_10 all 0.3821
P_100 all 0.0883
P_1000 all 0.0088
recall_10 all 0.0873
recall_100 all 0.1817
recall_1000 all 0.1817
ndcg all 0.2144
ndcg_cut_10 all 0.2812
ndcg_cut_100 all 0.2154
ndcg_cut_1000 all 0.2144
ict_wrfml¶
Metadata | Participants | Proceedings| Input| Summary| Appendix
summary (trec_eval)
map all 0.1058
bpref all 0.2098
recip_rank all 0.3754
P_10 all 0.2549
P_100 all 0.1006
P_1000 all 0.0231
recall_10 all 0.0492
recall_100 all 0.1954
recall_1000 all 0.4058
ndcg all 0.2575
ndcg_cut_10 all 0.1645
ndcg_cut_100 all 0.1706
ndcg_cut_1000 all 0.2575
VESBERT1000¶
Metadata | Participants | Proceedings| Input| Summary| Appendix
summary (trec_eval)
map all 0.2055
bpref all 0.3311
recip_rank all 0.5586
P_10 all 0.4191
P_100 all 0.1535
P_1000 all 0.0264
recall_10 all 0.0979
recall_100 all 0.3247
recall_1000 all 0.5083
ndcg all 0.3835
ndcg_cut_10 all 0.3055
ndcg_cut_100 all 0.3171
ndcg_cut_1000 all 0.3835
ug_1stprev3_sdm¶
Metadata | Participants| Input| Summary| Appendix
summary (trec_eval)
map all 0.2541
bpref all 0.3681
recip_rank all 0.5882
P_10 all 0.4353
P_100 all 0.1975
P_1000 all 0.0354
recall_10 all 0.0986
recall_100 all 0.4089
recall_1000 all 0.7012
ndcg all 0.4693
ndcg_cut_10 all 0.3122
ndcg_cut_100 all 0.3546
ndcg_cut_1000 all 0.4693
ug_cont_lin¶
Metadata | Participants| Input| Summary| Appendix
summary (trec_eval)
map all 0.2768
bpref all 0.3914
recip_rank all 0.5873
P_10 all 0.4705
P_100 all 0.2092
P_1000 all 0.0360
recall_10 all 0.1084
recall_100 all 0.4398
recall_1000 all 0.7133
ndcg all 0.4856
ndcg_cut_10 all 0.3271
ndcg_cut_100 all 0.3767
ndcg_cut_1000 all 0.4856
ug_cur_sdm¶
Metadata | Participants| Input| Summary| Appendix
summary (trec_eval)
map all 0.3355
bpref all 0.4713
recip_rank all 0.7193
P_10 all 0.5688
P_100 all 0.2255
P_1000 all 0.0391
recall_10 all 0.1568
recall_100 all 0.5293
recall_1000 all 0.8381
ndcg all 0.5792
ndcg_cut_10 all 0.4147
ndcg_cut_100 all 0.4581
ndcg_cut_1000 all 0.5792
UNH-trema-ecn¶
Metadata | Participants | Proceedings| Input| Summary| Appendix
summary (trec_eval)
map all 0.0738
bpref all 0.1315
recip_rank all 0.5081
P_10 all 0.3416
P_100 all 0.0827
P_1000 all 0.0083
recall_10 all 0.0692
recall_100 all 0.1798
recall_1000 all 0.1798
ndcg all 0.1773
ndcg_cut_10 all 0.2230
ndcg_cut_100 all 0.1782
ndcg_cut_1000 all 0.1773
UNH-trema-ent¶
Metadata | Participants | Proceedings| Input| Summary| Appendix
summary (trec_eval)
map all 0.0763
bpref all 0.1336
recip_rank all 0.5369
P_10 all 0.3578
P_100 all 0.0828
P_1000 all 0.0083
recall_10 all 0.0719
recall_100 all 0.1800
recall_1000 all 0.1800
ndcg all 0.1812
ndcg_cut_10 all 0.2369
ndcg_cut_100 all 0.1821
ndcg_cut_1000 all 0.1812
mpi-d5_igraph¶
Metadata | Participants | Proceedings| Input| Summary| Appendix
summary (trec_eval)
map all 0.1882
bpref all 0.3466
recip_rank all 0.6008
P_10 all 0.4012
P_100 all 0.1540
P_1000 all 0.0350
recall_10 all 0.0935
recall_100 all 0.3317
recall_1000 all 0.7230
ndcg all 0.4406
ndcg_cut_10 all 0.2788
ndcg_cut_100 all 0.2875
ndcg_cut_1000 all 0.4406
mpi-d5_intu¶
Metadata | Participants | Proceedings| Input| Summary| Appendix
summary (trec_eval)
map all 0.2417
bpref all 0.3672
recip_rank all 0.5998
P_10 all 0.4480
P_100 all 0.1895
P_1000 all 0.0350
recall_10 all 0.1073
recall_100 all 0.4150
recall_1000 all 0.7230
ndcg all 0.4698
ndcg_cut_10 all 0.3058
ndcg_cut_100 all 0.3471
ndcg_cut_1000 all 0.4698
unh-trema-relco¶
Metadata | Participants | Proceedings| Input| Summary| Appendix
summary (trec_eval)
map all 0.0776
bpref all 0.1567
recip_rank all 0.5364
P_10 all 0.2913
P_100 all 0.0918
P_1000 all 0.0092
recall_10 all 0.0689
recall_100 all 0.2033
recall_1000 all 0.2033
ndcg all 0.1949
ndcg_cut_10 all 0.2161
ndcg_cut_100 all 0.1957
ndcg_cut_1000 all 0.1949
mpi-d5_union¶
Metadata | Participants | Proceedings| Input| Summary| Appendix
summary (trec_eval)
map all 0.0981
bpref all 0.3241
recip_rank all 0.2752
P_10 all 0.1572
P_100 all 0.1055
P_1000 all 0.0356
recall_10 all 0.0324
recall_100 all 0.2205
recall_1000 all 0.7478
ndcg all 0.3518
ndcg_cut_10 all 0.0869
ndcg_cut_100 all 0.1542
ndcg_cut_1000 all 0.3518
mpi-d5_cqw¶
Metadata | Participants | Proceedings| Input| Summary| Appendix
summary (trec_eval)
map all 0.1858
bpref all 0.3474
recip_rank all 0.5939
P_10 all 0.3867
P_100 all 0.1503
P_1000 all 0.0350
recall_10 all 0.0895
recall_100 all 0.3255
recall_1000 all 0.7230
ndcg all 0.4374
ndcg_cut_10 all 0.2699
ndcg_cut_100 all 0.2825
ndcg_cut_1000 all 0.4374
combination¶
Metadata | Participants | Proceedings| Input| Summary| Appendix
summary (trec_eval)
map all 0.1306
bpref all 0.2498
recip_rank all 0.5416
P_10 all 0.3121
P_100 all 0.1062
P_1000 all 0.0193
recall_10 all 0.0766
recall_100 all 0.2307
recall_1000 all 0.4119
ndcg all 0.2869
ndcg_cut_10 all 0.2321
ndcg_cut_100 all 0.2140
ndcg_cut_1000 all 0.2869
UDInfoC_TS_2¶
Metadata | Participants | Proceedings| Input| Summary| Appendix
summary (trec_eval)
map all 0.0616
bpref all 0.0806
recip_rank all 0.5446
P_10 all 0.3699
P_100 all 0.0370
P_1000 all 0.0037
recall_10 all 0.0873
recall_100 all 0.0873
recall_1000 all 0.0873
ndcg all 0.1280
ndcg_cut_10 all 0.2501
ndcg_cut_100 all 0.1286
ndcg_cut_1000 all 0.1280
UMASS_DMN_V1¶
Metadata | Participants | Proceedings| Input| Summary| Appendix
summary (trec_eval)
map all 0.0773
bpref all 0.1821
recip_rank all 0.2998
P_10 all 0.1873
P_100 all 0.0880
P_1000 all 0.0198
recall_10 all 0.0299
recall_100 all 0.1530
recall_1000 all 0.3426
ndcg all 0.2095
ndcg_cut_10 all 0.1071
ndcg_cut_100 all 0.1236
ndcg_cut_1000 all 0.2095
UMASS_DMN_V2¶
Metadata | Participants | Proceedings| Input| Summary| Appendix
summary (trec_eval)
map all 0.0828
bpref all 0.1881
recip_rank all 0.3021
P_10 all 0.2035
P_100 all 0.0939
P_1000 all 0.0198
recall_10 all 0.0328
recall_100 all 0.1676
recall_1000 all 0.3426
ndcg all 0.2143
ndcg_cut_10 all 0.1131
ndcg_cut_100 all 0.1345
ndcg_cut_1000 all 0.2143
humanbert¶
Metadata | Participants| Input| Summary| Appendix
summary (trec_eval)
map all 0.4074
bpref all 0.5759
recip_rank all 0.8841
P_10 all 0.7243
P_100 all 0.2319
P_1000 all 0.0374
recall_10 all 0.2131
recall_100 all 0.5665
recall_1000 all 0.8053
ndcg all 0.6588
ndcg_cut_10 all 0.6063
ndcg_cut_100 all 0.5719
ndcg_cut_1000 all 0.6588
pgbert¶
Metadata | Participants| Input| Summary| Appendix
summary (trec_eval)
map all 0.2705
bpref all 0.4100
recip_rank all 0.6686
P_10 all 0.5451
P_100 all 0.1777
P_1000 all 0.0296
recall_10 all 0.1343
recall_100 all 0.3935
recall_1000 all 0.5876
ndcg all 0.4687
ndcg_cut_10 all 0.4263
ndcg_cut_100 all 0.3970
ndcg_cut_1000 all 0.4687
pg2bert¶
Metadata | Participants| Input| Summary| Appendix
summary (trec_eval)
map all 0.2598
bpref all 0.3977
recip_rank all 0.6448
P_10 all 0.5231
P_100 all 0.1710
P_1000 all 0.0286
recall_10 all 0.1315
recall_100 all 0.3867
recall_1000 all 0.5794
ndcg all 0.4545
ndcg_cut_10 all 0.4055
ndcg_cut_100 all 0.3844
ndcg_cut_1000 all 0.4545
rerankingorder¶
Metadata | Participants | Proceedings| Input| Summary| Appendix
summary (trec_eval)
map all 0.1379
bpref all 0.2458
recip_rank all 0.5667
P_10 all 0.3503
P_100 all 0.1147
P_1000 all 0.0195
recall_10 all 0.0854
recall_100 all 0.2463
recall_1000 all 0.4169
ndcg all 0.2937
ndcg_cut_10 all 0.2383
ndcg_cut_100 all 0.2256
ndcg_cut_1000 all 0.2937
datasetreorder¶
Metadata | Participants | Proceedings| Input| Summary| Appendix
summary (trec_eval)
map all 0.1356
bpref all 0.2454
recip_rank all 0.5527
P_10 all 0.3422
P_100 all 0.1148
P_1000 all 0.0195
recall_10 all 0.0821
recall_100 all 0.2467
recall_1000 all 0.4168
ndcg all 0.2923
ndcg_cut_10 all 0.2351
ndcg_cut_100 all 0.2244
ndcg_cut_1000 all 0.2923
ug_cedr_rerank¶
Metadata | Participants| Input| Summary| Appendix
summary (trec_eval)
map all 0.2176
bpref all 0.2903
recip_rank all 0.6463
P_10 all 0.5023
P_100 all 0.1813
P_1000 all 0.0182
recall_10 all 0.1111
recall_100 all 0.3859
recall_1000 all 0.3867
ndcg all 0.3544
ndcg_cut_10 all 0.3560
ndcg_cut_100 all 0.3556
ndcg_cut_1000 all 0.3544
topicturnsort¶
Metadata | Participants | Proceedings| Input| Summary| Appendix
summary (trec_eval)
map all 0.1367
bpref all 0.2458
recip_rank all 0.5581
P_10 all 0.3491
P_100 all 0.1147
P_1000 all 0.0195
recall_10 all 0.0841
recall_100 all 0.2464
recall_1000 all 0.4169
ndcg all 0.2926
ndcg_cut_10 all 0.2354
ndcg_cut_100 all 0.2246
ndcg_cut_1000 all 0.2926