Results - Podcast 2021¶
baseline-BM25¶
Metadata | Participants| Input| Summary (QD) | Summary (QE) | Summary (QR) | Summary (QS)| Appendix
summary (QD)
map all 0.1166
bpref all 0.1288
recip_rank all 0.4175
P_10 all 0.1600
P_100 all 0.0690
P_1000 all 0.0135
recall_10 all 0.1145
recall_100 all 0.4172
recall_1000 all 0.7560
ndcg all 0.3693
ndcg_cut_10 all 0.1624
ndcg_cut_100 all 0.2712
ndcg_cut_1000 all 0.3693
summary (QE)
map all 0.0822
bpref all 0.0774
recip_rank all 0.2745
P_10 all 0.1050
P_100 all 0.0443
P_1000 all 0.0083
recall_10 all 0.0854
recall_100 all 0.3496
recall_1000 all 0.6099
ndcg all 0.2632
ndcg_cut_10 all 0.1076
ndcg_cut_100 all 0.1945
ndcg_cut_1000 all 0.2632
summary (QR)
map all 0.1646
bpref all 0.1947
recip_rank all 0.5400
P_10 all 0.2700
P_100 all 0.1192
P_1000 all 0.0215
recall_10 all 0.1045
recall_100 all 0.4267
recall_1000 all 0.7029
ndcg all 0.4089
ndcg_cut_10 all 0.2202
ndcg_cut_100 all 0.3117
ndcg_cut_1000 all 0.4089
summary (QS)
map all 0.1375
bpref all 0.1584
recip_rank all 0.5052
P_10 all 0.2200
P_100 all 0.0975
P_1000 all 0.0186
recall_10 all 0.1022
recall_100 all 0.4080
recall_1000 all 0.7545
ndcg all 0.3991
ndcg_cut_10 all 0.1922
ndcg_cut_100 all 0.2912
ndcg_cut_1000 all 0.3991
baseline-QL-Q¶
Metadata | Participants| Input| Summary (QD) | Summary (QE) | Summary (QR) | Summary (QS)| Appendix
summary (QD)
map all 0.1182
bpref all 0.1296
recip_rank all 0.4387
P_10 all 0.1500
P_100 all 0.0698
P_1000 all 0.0137
recall_10 all 0.1119
recall_100 all 0.4033
recall_1000 all 0.7542
ndcg all 0.3736
ndcg_cut_10 all 0.1643
ndcg_cut_100 all 0.2725
ndcg_cut_1000 all 0.3736
summary (QE)
map all 0.0836
bpref all 0.0783
recip_rank all 0.2988
P_10 all 0.1000
P_100 all 0.0458
P_1000 all 0.0083
recall_10 all 0.0865
recall_100 all 0.3565
recall_1000 all 0.6086
ndcg all 0.2657
ndcg_cut_10 all 0.1103
ndcg_cut_100 all 0.1997
ndcg_cut_1000 all 0.2657
summary (QR)
map all 0.1637
bpref all 0.1965
recip_rank all 0.5326
P_10 all 0.2520
P_100 all 0.1184
P_1000 all 0.0217
recall_10 all 0.0999
recall_100 all 0.4214
recall_1000 all 0.7029
ndcg all 0.4105
ndcg_cut_10 all 0.2144
ndcg_cut_100 all 0.3113
ndcg_cut_1000 all 0.4105
summary (QS)
map all 0.1405
bpref all 0.1614
recip_rank all 0.5169
P_10 all 0.2075
P_100 all 0.0995
P_1000 all 0.0189
recall_10 all 0.0997
recall_100 all 0.4115
recall_1000 all 0.7575
ndcg all 0.4063
ndcg_cut_10 all 0.1944
ndcg_cut_100 all 0.2985
ndcg_cut_1000 all 0.4063
baseline-BM25-D¶
Metadata | Participants| Input| Summary (QD) | Summary (QE) | Summary (QR) | Summary (QS)| Appendix
summary (QD)
map all 0.1137
bpref all 0.1292
recip_rank all 0.3414
P_10 all 0.1775
P_100 all 0.0710
P_1000 all 0.0140
recall_10 all 0.1090
recall_100 all 0.4062
recall_1000 all 0.7777
ndcg all 0.3623
ndcg_cut_10 all 0.1514
ndcg_cut_100 all 0.2528
ndcg_cut_1000 all 0.3623
summary (QE)
map all 0.0611
bpref all 0.0596
recip_rank all 0.2192
P_10 all 0.0925
P_100 all 0.0408
P_1000 all 0.0085
recall_10 all 0.0685
recall_100 all 0.2959
recall_1000 all 0.6279
ndcg all 0.2435
ndcg_cut_10 all 0.0833
ndcg_cut_100 all 0.1571
ndcg_cut_1000 all 0.2435
summary (QR)
map all 0.1714
bpref all 0.2025
recip_rank all 0.4721
P_10 all 0.2940
P_100 all 0.1188
P_1000 all 0.0229
recall_10 all 0.1215
recall_100 all 0.4209
recall_1000 all 0.7538
ndcg all 0.4239
ndcg_cut_10 all 0.2319
ndcg_cut_100 all 0.3051
ndcg_cut_1000 all 0.4239
summary (QS)
map all 0.1384
bpref all 0.1663
recip_rank all 0.4046
P_10 all 0.2425
P_100 all 0.0982
P_1000 all 0.0199
recall_10 all 0.1109
recall_100 all 0.4020
recall_1000 all 0.7879
ndcg all 0.4010
ndcg_cut_10 all 0.1821
ndcg_cut_100 all 0.2730
ndcg_cut_1000 all 0.4010
baseline-QL-D¶
Metadata | Participants| Input| Summary (QD) | Summary (QE) | Summary (QR) | Summary (QS)| Appendix
summary (QD)
map all 0.1141
bpref all 0.1296
recip_rank all 0.3500
P_10 all 0.1875
P_100 all 0.0715
P_1000 all 0.0142
recall_10 all 0.1127
recall_100 all 0.3966
recall_1000 all 0.7839
ndcg all 0.3658
ndcg_cut_10 all 0.1595
ndcg_cut_100 all 0.2545
ndcg_cut_1000 all 0.3658
summary (QE)
map all 0.0649
bpref all 0.0627
recip_rank all 0.2414
P_10 all 0.0900
P_100 all 0.0420
P_1000 all 0.0085
recall_10 all 0.0635
recall_100 all 0.3160
recall_1000 all 0.6304
ndcg all 0.2502
ndcg_cut_10 all 0.0892
ndcg_cut_100 all 0.1690
ndcg_cut_1000 all 0.2502
summary (QR)
map all 0.1751
bpref all 0.2047
recip_rank all 0.5026
P_10 all 0.3000
P_100 all 0.1174
P_1000 all 0.0230
recall_10 all 0.1217
recall_100 all 0.4116
recall_1000 all 0.7576
ndcg all 0.4311
ndcg_cut_10 all 0.2452
ndcg_cut_100 all 0.3087
ndcg_cut_1000 all 0.4311
summary (QS)
map all 0.1404
bpref all 0.1651
recip_rank all 0.4557
P_10 all 0.2450
P_100 all 0.0997
P_1000 all 0.0201
recall_10 all 0.1077
recall_100 all 0.4077
recall_1000 all 0.7949
ndcg all 0.4094
ndcg_cut_10 all 0.1964
ndcg_cut_100 all 0.2822
ndcg_cut_1000 all 0.4094
Webis_pc_bs¶
Metadata | Participants | Proceedings| Input| Summary (QD) | Summary (QE) | Summary (QR) | Summary (QS)| Appendix
summary (QD)
map all 0.0973
bpref all 0.1170
recip_rank all 0.3875
P_10 all 0.1600
P_100 all 0.0563
P_1000 all 0.0122
recall_10 all 0.1022
recall_100 all 0.3451
recall_1000 all 0.6776
ndcg all 0.3208
ndcg_cut_10 all 0.1427
ndcg_cut_100 all 0.2237
ndcg_cut_1000 all 0.3208
summary (QE)
map all 0.0660
bpref all 0.0744
recip_rank all 0.2513
P_10 all 0.0975
P_100 all 0.0368
P_1000 all 0.0075
recall_10 all 0.0773
recall_100 all 0.3072
recall_1000 all 0.5942
ndcg all 0.2330
ndcg_cut_10 all 0.0927
ndcg_cut_100 all 0.1640
ndcg_cut_1000 all 0.2330
summary (QR)
map all 0.1317
bpref all 0.1776
recip_rank all 0.4977
P_10 all 0.2520
P_100 all 0.0950
P_1000 all 0.0196
recall_10 all 0.0974
recall_100 all 0.3573
recall_1000 all 0.6447
ndcg all 0.3636
ndcg_cut_10 all 0.2011
ndcg_cut_100 all 0.2628
ndcg_cut_1000 all 0.3636
summary (QS)
map all 0.1049
bpref all 0.1387
recip_rank all 0.4253
P_10 all 0.2000
P_100 all 0.0777
P_1000 all 0.0170
recall_10 all 0.0933
recall_100 all 0.3355
recall_1000 all 0.6866
ndcg all 0.3435
ndcg_cut_10 all 0.1596
ndcg_cut_100 all 0.2324
ndcg_cut_1000 all 0.3435
Webis_pc_cola¶
Metadata | Participants | Proceedings| Input| Summary (QD) | Summary (QE) | Summary (QR) | Summary (QS)| Appendix
summary (QD)
map all 0.0352
bpref all 0.0398
recip_rank all 0.1566
P_10 all 0.0625
P_100 all 0.0263
P_1000 all 0.0122
recall_10 all 0.0265
recall_100 all 0.1483
recall_1000 all 0.6776
ndcg all 0.2289
ndcg_cut_10 all 0.0470
ndcg_cut_100 all 0.0839
ndcg_cut_1000 all 0.2289
summary (QE)
map all 0.0331
bpref all 0.0324
recip_rank all 0.1199
P_10 all 0.0450
P_100 all 0.0153
P_1000 all 0.0075
recall_10 all 0.0334
recall_100 all 0.1323
recall_1000 all 0.5942
ndcg all 0.1748
ndcg_cut_10 all 0.0386
ndcg_cut_100 all 0.0686
ndcg_cut_1000 all 0.1748
summary (QR)
map all 0.1317
bpref all 0.1776
recip_rank all 0.4977
P_10 all 0.2520
P_100 all 0.0950
P_1000 all 0.0196
recall_10 all 0.0974
recall_100 all 0.3573
recall_1000 all 0.6447
ndcg all 0.3636
ndcg_cut_10 all 0.2011
ndcg_cut_100 all 0.2628
ndcg_cut_1000 all 0.3636
summary (QS)
map all 0.0349
bpref all 0.0470
recip_rank all 0.1609
P_10 all 0.0600
P_100 all 0.0293
P_1000 all 0.0170
recall_10 all 0.0227
recall_100 all 0.1319
recall_1000 all 0.6866
ndcg all 0.2443
ndcg_cut_10 all 0.0441
ndcg_cut_100 all 0.0791
ndcg_cut_1000 all 0.2443
Webis_pc_rob¶
Metadata | Participants | Proceedings| Input| Summary (QD) | Summary (QE) | Summary (QR) | Summary (QS)| Appendix
summary (QD)
map all 0.0257
bpref all 0.0259
recip_rank all 0.1114
P_10 all 0.0400
P_100 all 0.0180
P_1000 all 0.0122
recall_10 all 0.0216
recall_100 all 0.1047
recall_1000 all 0.6776
ndcg all 0.2101
ndcg_cut_10 all 0.0264
ndcg_cut_100 all 0.0562
ndcg_cut_1000 all 0.2101
summary (QE)
map all 0.0202
bpref all 0.0118
recip_rank all 0.0631
P_10 all 0.0275
P_100 all 0.0115
P_1000 all 0.0075
recall_10 all 0.0323
recall_100 all 0.1079
recall_1000 all 0.5942
ndcg all 0.1584
ndcg_cut_10 all 0.0208
ndcg_cut_100 all 0.0471
ndcg_cut_1000 all 0.1584
summary (QR)
map all 0.1317
bpref all 0.1776
recip_rank all 0.4977
P_10 all 0.2520
P_100 all 0.0950
P_1000 all 0.0196
recall_10 all 0.0974
recall_100 all 0.3573
recall_1000 all 0.6447
ndcg all 0.3636
ndcg_cut_10 all 0.2011
ndcg_cut_100 all 0.2628
ndcg_cut_1000 all 0.3636
summary (QS)
map all 0.0245
bpref all 0.0313
recip_rank all 0.0882
P_10 all 0.0350
P_100 all 0.0205
P_1000 all 0.0170
recall_10 all 0.0163
recall_100 all 0.0936
recall_1000 all 0.6866
ndcg all 0.2250
ndcg_cut_10 all 0.0230
ndcg_cut_100 all 0.0532
ndcg_cut_1000 all 0.2250
Webis_pc_co_rob¶
Metadata | Participants | Proceedings| Input| Summary (QD) | Summary (QE) | Summary (QR) | Summary (QS)| Appendix
summary (QD)
map all 0.0304
bpref all 0.0363
recip_rank all 0.1158
P_10 all 0.0550
P_100 all 0.0278
P_1000 all 0.0122
recall_10 all 0.0270
recall_100 all 0.1510
recall_1000 all 0.6776
ndcg all 0.2193
ndcg_cut_10 all 0.0342
ndcg_cut_100 all 0.0775
ndcg_cut_1000 all 0.2193
summary (QE)
map all 0.0232
bpref all 0.0184
recip_rank all 0.0681
P_10 all 0.0275
P_100 all 0.0158
P_1000 all 0.0075
recall_10 all 0.0186
recall_100 all 0.1393
recall_1000 all 0.5942
ndcg all 0.1620
ndcg_cut_10 all 0.0185
ndcg_cut_100 all 0.0589
ndcg_cut_1000 all 0.1620
summary (QR)
map all 0.1317
bpref all 0.1776
recip_rank all 0.4977
P_10 all 0.2520
P_100 all 0.0950
P_1000 all 0.0196
recall_10 all 0.0974
recall_100 all 0.3573
recall_1000 all 0.6447
ndcg all 0.3636
ndcg_cut_10 all 0.2011
ndcg_cut_100 all 0.2628
ndcg_cut_1000 all 0.3636
summary (QS)
map all 0.0287
bpref all 0.0406
recip_rank all 0.1174
P_10 all 0.0550
P_100 all 0.0293
P_1000 all 0.0170
recall_10 all 0.0204
recall_100 all 0.1232
recall_1000 all 0.6866
ndcg all 0.2320
ndcg_cut_10 all 0.0302
ndcg_cut_100 all 0.0682
ndcg_cut_1000 all 0.2320
UCL_audio_2¶
Metadata | Participants| Input| Summary (QD) | Summary (QE) | Summary (QR) | Summary (QS)| Appendix
summary (QD)
map all 0.0948
bpref all 0.1241
recip_rank all 0.3811
P_10 all 0.1525
P_100 all 0.0645
P_1000 all 0.0065
recall_10 all 0.0981
recall_100 all 0.3838
recall_1000 all 0.3838
ndcg all 0.2458
ndcg_cut_10 all 0.1416
ndcg_cut_100 all 0.2458
ndcg_cut_1000 all 0.2458
summary (QE)
map all 0.0488
bpref all 0.0537
recip_rank all 0.2222
P_10 all 0.0800
P_100 all 0.0383
P_1000 all 0.0038
recall_10 all 0.0537
recall_100 all 0.2801
recall_1000 all 0.2801
ndcg all 0.1518
ndcg_cut_10 all 0.0768
ndcg_cut_100 all 0.1518
ndcg_cut_1000 all 0.1518
summary (QR)
map all 0.1335
bpref all 0.1647
recip_rank all 0.4888
P_10 all 0.2760
P_100 all 0.1076
P_1000 all 0.0108
recall_10 all 0.1117
recall_100 all 0.3921
recall_1000 all 0.3921
ndcg all 0.2864
ndcg_cut_10 all 0.2200
ndcg_cut_100 all 0.2864
ndcg_cut_1000 all 0.2864
summary (QS)
map all 0.0942
bpref all 0.1347
recip_rank all 0.3522
P_10 all 0.2050
P_100 all 0.0885
P_1000 all 0.0089
recall_10 all 0.0880
recall_100 all 0.3822
recall_1000 all 0.3822
ndcg all 0.2443
ndcg_cut_10 all 0.1448
ndcg_cut_100 all 0.2443
ndcg_cut_1000 all 0.2443
UCL_audio_1¶
Metadata | Participants| Input| Summary (QD) | Summary (QE) | Summary (QR) | Summary (QS)| Appendix
summary (QD)
map all 0.0906
bpref all 0.1071
recip_rank all 0.3489
P_10 all 0.1475
P_100 all 0.0645
P_1000 all 0.0065
recall_10 all 0.0958
recall_100 all 0.3838
recall_1000 all 0.3838
ndcg all 0.2424
ndcg_cut_10 all 0.1444
ndcg_cut_100 all 0.2424
ndcg_cut_1000 all 0.2424
summary (QE)
map all 0.0445
bpref all 0.0497
recip_rank all 0.2019
P_10 all 0.0675
P_100 all 0.0383
P_1000 all 0.0038
recall_10 all 0.0463
recall_100 all 0.2801
recall_1000 all 0.2801
ndcg all 0.1468
ndcg_cut_10 all 0.0684
ndcg_cut_100 all 0.1468
ndcg_cut_1000 all 0.1468
summary (QR)
map all 0.1335
bpref all 0.1647
recip_rank all 0.4888
P_10 all 0.2760
P_100 all 0.1076
P_1000 all 0.0108
recall_10 all 0.1117
recall_100 all 0.3921
recall_1000 all 0.3921
ndcg all 0.2864
ndcg_cut_10 all 0.2200
ndcg_cut_100 all 0.2864
ndcg_cut_1000 all 0.2864
summary (QS)
map all 0.0958
bpref all 0.1256
recip_rank all 0.4024
P_10 all 0.2000
P_100 all 0.0885
P_1000 all 0.0089
recall_10 all 0.0846
recall_100 all 0.3822
recall_1000 all 0.3822
ndcg all 0.2511
ndcg_cut_10 all 0.1618
ndcg_cut_100 all 0.2511
ndcg_cut_1000 all 0.2511
ms_mt5¶
Metadata | Participants| Input| Summary (QD) | Summary (QE) | Summary (QR) | Summary (QS)| Appendix
summary (QD)
map all 0.1184
bpref all 0.1485
recip_rank all 0.3532
P_10 all 0.1675
P_100 all 0.0727
P_1000 all 0.0140
recall_10 all 0.0995
recall_100 all 0.4076
recall_1000 all 0.7777
ndcg all 0.3647
ndcg_cut_10 all 0.1480
ndcg_cut_100 all 0.2570
ndcg_cut_1000 all 0.3647
summary (QE)
map all 0.0864
bpref all 0.0989
recip_rank all 0.2253
P_10 all 0.1150
P_100 all 0.0488
P_1000 all 0.0085
recall_10 all 0.0637
recall_100 all 0.3279
recall_1000 all 0.6279
ndcg all 0.2663
ndcg_cut_10 all 0.1083
ndcg_cut_100 all 0.1948
ndcg_cut_1000 all 0.2663
summary (QR)
map all 0.1866
bpref all 0.2388
recip_rank all 0.5374
P_10 all 0.2820
P_100 all 0.1202
P_1000 all 0.0229
recall_10 all 0.1314
recall_100 all 0.4235
recall_1000 all 0.7538
ndcg all 0.4428
ndcg_cut_10 all 0.2400
ndcg_cut_100 all 0.3282
ndcg_cut_1000 all 0.4428
summary (QS)
map all 0.1460
bpref all 0.1894
recip_rank all 0.4520
P_10 all 0.2300
P_100 all 0.1042
P_1000 all 0.0199
recall_10 all 0.0951
recall_100 all 0.4033
recall_1000 all 0.7879
ndcg all 0.4093
ndcg_cut_10 all 0.1862
ndcg_cut_100 all 0.2880
ndcg_cut_1000 all 0.4093
tp_mt5¶
Metadata | Participants| Input| Summary (QD) | Summary (QE) | Summary (QR) | Summary (QS)| Appendix
summary (QD)
map all 0.1478
bpref all 0.2071
recip_rank all 0.3664
P_10 all 0.2150
P_100 all 0.0822
P_1000 all 0.0140
recall_10 all 0.1304
recall_100 all 0.4633
recall_1000 all 0.7777
ndcg all 0.3881
ndcg_cut_10 all 0.1821
ndcg_cut_100 all 0.3006
ndcg_cut_1000 all 0.3881
summary (QE)
map all 0.1221
bpref all 0.1424
recip_rank all 0.3365
P_10 all 0.1775
P_100 all 0.0563
P_1000 all 0.0085
recall_10 all 0.1225
recall_100 all 0.3754
recall_1000 all 0.6279
ndcg all 0.3021
ndcg_cut_10 all 0.1715
ndcg_cut_100 all 0.2443
ndcg_cut_1000 all 0.3021
summary (QR)
map all 0.2703
bpref all 0.3238
recip_rank all 0.6449
P_10 all 0.4040
P_100 all 0.1454
P_1000 all 0.0229
recall_10 all 0.1694
recall_100 all 0.5130
recall_1000 all 0.7538
ndcg all 0.5110
ndcg_cut_10 all 0.3537
ndcg_cut_100 all 0.4284
ndcg_cut_1000 all 0.5110
summary (QS)
map all 0.2176
bpref all 0.2608
recip_rank all 0.5476
P_10 all 0.3475
P_100 all 0.1263
P_1000 all 0.0199
recall_10 all 0.1405
recall_100 all 0.4863
recall_1000 all 0.7879
ndcg all 0.4691
ndcg_cut_10 all 0.2876
ndcg_cut_100 all 0.3787
ndcg_cut_1000 all 0.4691
tp_mt5_f1¶
Metadata | Participants| Input| Summary (QD) | Summary (QE) | Summary (QR) | Summary (QS)| Appendix
summary (QD)
map all 0.1445
bpref all 0.1988
recip_rank all 0.3298
P_10 all 0.2100
P_100 all 0.0822
P_1000 all 0.0140
recall_10 all 0.1296
recall_100 all 0.4633
recall_1000 all 0.7777
ndcg all 0.3823
ndcg_cut_10 all 0.1681
ndcg_cut_100 all 0.2947
ndcg_cut_1000 all 0.3823
summary (QE)
map all 0.0972
bpref all 0.1286
recip_rank all 0.1856
P_10 all 0.1375
P_100 all 0.0563
P_1000 all 0.0085
recall_10 all 0.1064
recall_100 all 0.3754
recall_1000 all 0.6279
ndcg all 0.2703
ndcg_cut_10 all 0.1134
ndcg_cut_100 all 0.2125
ndcg_cut_1000 all 0.2703
summary (QR)
map all 0.2703
bpref all 0.3238
recip_rank all 0.6449
P_10 all 0.4040
P_100 all 0.1454
P_1000 all 0.0229
recall_10 all 0.1694
recall_100 all 0.5130
recall_1000 all 0.7538
ndcg all 0.5110
ndcg_cut_10 all 0.3537
ndcg_cut_100 all 0.4284
ndcg_cut_1000 all 0.5110
summary (QS)
map all 0.2091
bpref all 0.2605
recip_rank all 0.5181
P_10 all 0.3325
P_100 all 0.1263
P_1000 all 0.0199
recall_10 all 0.1307
recall_100 all 0.4863
recall_1000 all 0.7879
ndcg all 0.4619
ndcg_cut_10 all 0.2689
ndcg_cut_100 all 0.3716
ndcg_cut_1000 all 0.4619
tp_mt5_f2¶
Metadata | Participants| Input| Summary (QD) | Summary (QE) | Summary (QR) | Summary (QS)| Appendix
summary (QD)
map all 0.1451
bpref all 0.1978
recip_rank all 0.3232
P_10 all 0.2075
P_100 all 0.0822
P_1000 all 0.0140
recall_10 all 0.1360
recall_100 all 0.4633
recall_1000 all 0.7777
ndcg all 0.3814
ndcg_cut_10 all 0.1644
ndcg_cut_100 all 0.2938
ndcg_cut_1000 all 0.3814
summary (QE)
map all 0.0918
bpref all 0.1251
recip_rank all 0.1662
P_10 all 0.1200
P_100 all 0.0563
P_1000 all 0.0085
recall_10 all 0.1013
recall_100 all 0.3754
recall_1000 all 0.6279
ndcg all 0.2647
ndcg_cut_10 all 0.0987
ndcg_cut_100 all 0.2069
ndcg_cut_1000 all 0.2647
summary (QR)
map all 0.2703
bpref all 0.3238
recip_rank all 0.6449
P_10 all 0.4040
P_100 all 0.1454
P_1000 all 0.0229
recall_10 all 0.1694
recall_100 all 0.5130
recall_1000 all 0.7538
ndcg all 0.5110
ndcg_cut_10 all 0.3537
ndcg_cut_100 all 0.4284
ndcg_cut_1000 all 0.5110
summary (QS)
map all 0.2055
bpref all 0.2588
recip_rank all 0.5139
P_10 all 0.3075
P_100 all 0.1263
P_1000 all 0.0199
recall_10 all 0.1228
recall_100 all 0.4863
recall_1000 all 0.7879
ndcg all 0.4593
ndcg_cut_10 all 0.2497
ndcg_cut_100 all 0.3689
ndcg_cut_1000 all 0.4593
f_b25_coil¶
Metadata | Participants| Input| Summary (QD) | Summary (QE) | Summary (QR) | Summary (QS)| Appendix
summary (QD)
map all 0.1378
bpref all 0.1424
recip_rank all 0.3663
P_10 all 0.2050
P_100 all 0.0795
P_1000 all 0.0149
recall_10 all 0.1349
recall_100 all 0.4505
recall_1000 all 0.8282
ndcg all 0.3982
ndcg_cut_10 all 0.1777
ndcg_cut_100 all 0.2896
ndcg_cut_1000 all 0.3982
summary (QE)
map all 0.0747
bpref all 0.0762
recip_rank all 0.2343
P_10 all 0.1150
P_100 all 0.0455
P_1000 all 0.0092
recall_10 all 0.0827
recall_100 all 0.3300
recall_1000 all 0.6561
ndcg all 0.2693
ndcg_cut_10 all 0.1041
ndcg_cut_100 all 0.1825
ndcg_cut_1000 all 0.2693
summary (QR)
map all 0.2002
bpref all 0.2241
recip_rank all 0.5030
P_10 all 0.3260
P_100 all 0.1338
P_1000 all 0.0244
recall_10 all 0.1377
recall_100 all 0.4697
recall_1000 all 0.7968
ndcg all 0.4634
ndcg_cut_10 all 0.2574
ndcg_cut_100 all 0.3452
ndcg_cut_1000 all 0.4634
summary (QS)
map all 0.1675
bpref all 0.1909
recip_rank all 0.4452
P_10 all 0.2825
P_100 all 0.1140
P_1000 all 0.0217
recall_10 all 0.1415
recall_100 all 0.4630
recall_1000 all 0.8356
ndcg all 0.4451
ndcg_cut_10 all 0.2206
ndcg_cut_100 all 0.3194
ndcg_cut_1000 all 0.4451
f_b25_tct¶
Metadata | Participants| Input| Summary (QD) | Summary (QE) | Summary (QR) | Summary (QS)| Appendix
summary (QD)
map all 0.1624
bpref all 0.1794
recip_rank all 0.4445
P_10 all 0.2225
P_100 all 0.0887
P_1000 all 0.0153
recall_10 all 0.1502
recall_100 all 0.5019
recall_1000 all 0.8436
ndcg all 0.4275
ndcg_cut_10 all 0.2021
ndcg_cut_100 all 0.3276
ndcg_cut_1000 all 0.4275
summary (QE)
map all 0.0990
bpref all 0.1059
recip_rank all 0.3045
P_10 all 0.1350
P_100 all 0.0535
P_1000 all 0.0096
recall_10 all 0.0983
recall_100 all 0.3596
recall_1000 all 0.6716
ndcg all 0.2951
ndcg_cut_10 all 0.1281
ndcg_cut_100 all 0.2160
ndcg_cut_1000 all 0.2951
summary (QR)
map all 0.2362
bpref all 0.2571
recip_rank all 0.6020
P_10 all 0.3660
P_100 all 0.1490
P_1000 all 0.0254
recall_10 all 0.1544
recall_100 all 0.5130
recall_1000 all 0.8182
ndcg all 0.4987
ndcg_cut_10 all 0.3020
ndcg_cut_100 all 0.3885
ndcg_cut_1000 all 0.4987
summary (QS)
map all 0.2130
bpref all 0.2248
recip_rank all 0.5389
P_10 all 0.3250
P_100 all 0.1335
P_1000 all 0.0225
recall_10 all 0.1702
recall_100 all 0.5322
recall_1000 all 0.8609
ndcg all 0.4904
ndcg_cut_10 all 0.2742
ndcg_cut_100 all 0.3788
ndcg_cut_1000 all 0.4904
f_coil_tct¶
Metadata | Participants| Input| Summary (QD) | Summary (QE) | Summary (QR) | Summary (QS)| Appendix
summary (QD)
map all 0.1595
bpref all 0.1839
recip_rank all 0.4708
P_10 all 0.2200
P_100 all 0.0780
P_1000 all 0.0143
recall_10 all 0.1341
recall_100 all 0.4563
recall_1000 all 0.7835
ndcg all 0.4087
ndcg_cut_10 all 0.2036
ndcg_cut_100 all 0.3101
ndcg_cut_1000 all 0.4087
summary (QE)
map all 0.0859
bpref all 0.0974
recip_rank all 0.2959
P_10 all 0.1350
P_100 all 0.0455
P_1000 all 0.0087
recall_10 all 0.0836
recall_100 all 0.3305
recall_1000 all 0.5953
ndcg all 0.2681
ndcg_cut_10 all 0.1247
ndcg_cut_100 all 0.1966
ndcg_cut_1000 all 0.2681
summary (QR)
map all 0.2257
bpref all 0.2609
recip_rank all 0.6554
P_10 all 0.3540
P_100 all 0.1362
P_1000 all 0.0236
recall_10 all 0.1471
recall_100 all 0.4824
recall_1000 all 0.7722
ndcg all 0.4800
ndcg_cut_10 all 0.2978
ndcg_cut_100 all 0.3760
ndcg_cut_1000 all 0.4800
summary (QS)
map all 0.1982
bpref all 0.2282
recip_rank all 0.5505
P_10 all 0.3075
P_100 all 0.1195
P_1000 all 0.0209
recall_10 all 0.1571
recall_100 all 0.4849
recall_1000 all 0.8078
ndcg all 0.4647
ndcg_cut_10 all 0.2595
ndcg_cut_100 all 0.3550
ndcg_cut_1000 all 0.4647
s_tasb¶
Metadata | Participants| Input| Summary (QD) | Summary (QE) | Summary (QR) | Summary (QS)| Appendix
summary (QD)
map all 0.1237
bpref all 0.1641
recip_rank all 0.4159
P_10 all 0.1875
P_100 all 0.0690
P_1000 all 0.0117
recall_10 all 0.1131
recall_100 all 0.3834
recall_1000 all 0.6519
ndcg all 0.3362
ndcg_cut_10 all 0.1666
ndcg_cut_100 all 0.2600
ndcg_cut_1000 all 0.3362
summary (QE)
map all 0.0707
bpref all 0.0910
recip_rank all 0.2845
P_10 all 0.1025
P_100 all 0.0378
P_1000 all 0.0066
recall_10 all 0.0719
recall_100 all 0.2691
recall_1000 all 0.4869
ndcg all 0.2185
ndcg_cut_10 all 0.1023
ndcg_cut_100 all 0.1655
ndcg_cut_1000 all 0.2185
summary (QR)
map all 0.1758
bpref all 0.2327
recip_rank all 0.5692
P_10 all 0.3160
P_100 all 0.1140
P_1000 all 0.0198
recall_10 all 0.1287
recall_100 all 0.3940
recall_1000 all 0.6328
ndcg all 0.3927
ndcg_cut_10 all 0.2516
ndcg_cut_100 all 0.3066
ndcg_cut_1000 all 0.3927
summary (QS)
map all 0.1474
bpref all 0.1970
recip_rank all 0.5159
P_10 all 0.2650
P_100 all 0.0977
P_1000 all 0.0169
recall_10 all 0.1127
recall_100 all 0.3804
recall_1000 all 0.6431
ndcg all 0.3672
ndcg_cut_10 all 0.2111
ndcg_cut_100 all 0.2797
ndcg_cut_1000 all 0.3672
s_tct¶
Metadata | Participants| Input| Summary (QD) | Summary (QE) | Summary (QR) | Summary (QS)| Appendix
summary (QD)
map all 0.1312
bpref all 0.1712
recip_rank all 0.4273
P_10 all 0.1925
P_100 all 0.0643
P_1000 all 0.0115
recall_10 all 0.1265
recall_100 all 0.3646
recall_1000 all 0.6467
ndcg all 0.3522
ndcg_cut_10 all 0.1841
ndcg_cut_100 all 0.2732
ndcg_cut_1000 all 0.3522
summary (QE)
map all 0.0772
bpref all 0.1039
recip_rank all 0.2585
P_10 all 0.1175
P_100 all 0.0365
P_1000 all 0.0070
recall_10 all 0.0784
recall_100 all 0.2366
recall_1000 all 0.4882
ndcg all 0.2296
ndcg_cut_10 all 0.1175
ndcg_cut_100 all 0.1701
ndcg_cut_1000 all 0.2296
summary (QR)
map all 0.1826
bpref all 0.2470
recip_rank all 0.6140
P_10 all 0.3460
P_100 all 0.1114
P_1000 all 0.0187
recall_10 all 0.1459
recall_100 all 0.3886
recall_1000 all 0.6286
ndcg all 0.4122
ndcg_cut_10 all 0.2848
ndcg_cut_100 all 0.3305
ndcg_cut_1000 all 0.4122
summary (QS)
map all 0.1661
bpref all 0.2219
recip_rank all 0.5385
P_10 all 0.2950
P_100 all 0.1005
P_1000 all 0.0172
recall_10 all 0.1485
recall_100 all 0.4145
recall_1000 all 0.6657
ndcg all 0.4054
ndcg_cut_10 all 0.2506
ndcg_cut_100 all 0.3222
ndcg_cut_1000 all 0.4054
TUW_hybrid_ws¶
Metadata | Participants | Proceedings| Input| Summary (QD) | Summary (QE) | Summary (QR) | Summary (QS)| Appendix
summary (QD)
map all 0.1552
bpref all 0.1765
recip_rank all 0.4362
P_10 all 0.2050
P_100 all 0.0840
P_1000 all 0.0159
recall_10 all 0.1222
recall_100 all 0.4983
recall_1000 all 0.8897
ndcg all 0.4351
ndcg_cut_10 all 0.1933
ndcg_cut_100 all 0.3205
ndcg_cut_1000 all 0.4351
summary (QE)
map all 0.0908
bpref all 0.1020
recip_rank all 0.2814
P_10 all 0.1325
P_100 all 0.0445
P_1000 all 0.0100
recall_10 all 0.0893
recall_100 all 0.3542
recall_1000 all 0.7011
ndcg all 0.2869
ndcg_cut_10 all 0.1207
ndcg_cut_100 all 0.1957
ndcg_cut_1000 all 0.2869
summary (QR)
map all 0.2477
bpref all 0.2622
recip_rank all 0.6527
P_10 all 0.3840
P_100 all 0.1486
P_1000 all 0.0266
recall_10 all 0.1447
recall_100 all 0.5085
recall_1000 all 0.8613
ndcg all 0.5255
ndcg_cut_10 all 0.3205
ndcg_cut_100 all 0.3986
ndcg_cut_1000 all 0.5255
summary (QS)
map all 0.1845
bpref all 0.2317
recip_rank all 0.6192
P_10 all 0.2925
P_100 all 0.1117
P_1000 all 0.0232
recall_10 all 0.1236
recall_100 all 0.4549
recall_1000 all 0.8906
ndcg all 0.4847
ndcg_cut_10 all 0.2556
ndcg_cut_100 all 0.3399
ndcg_cut_1000 all 0.4847
TUW_hybrid_cat¶
Metadata | Participants | Proceedings| Input| Summary (QD) | Summary (QE) | Summary (QR) | Summary (QS)| Appendix
summary (QD)
map all 0.1369
bpref all 0.1654
recip_rank all 0.3521
P_10 all 0.1850
P_100 all 0.0785
P_1000 all 0.0161
recall_10 all 0.1307
recall_100 all 0.4674
recall_1000 all 0.9036
ndcg all 0.4179
ndcg_cut_10 all 0.1703
ndcg_cut_100 all 0.2940
ndcg_cut_1000 all 0.4179
summary (QE)
map all 0.1118
bpref all 0.1278
recip_rank all 0.2988
P_10 all 0.1500
P_100 all 0.0518
P_1000 all 0.0102
recall_10 all 0.1223
recall_100 all 0.3623
recall_1000 all 0.7219
ndcg all 0.3065
ndcg_cut_10 all 0.1437
ndcg_cut_100 all 0.2136
ndcg_cut_1000 all 0.3065
summary (QR)
map all 0.2446
bpref all 0.2879
recip_rank all 0.6088
P_10 all 0.3860
P_100 all 0.1432
P_1000 all 0.0269
recall_10 all 0.1632
recall_100 all 0.5168
recall_1000 all 0.8780
ndcg all 0.5315
ndcg_cut_10 all 0.3234
ndcg_cut_100 all 0.4024
ndcg_cut_1000 all 0.5315
summary (QS)
map all 0.1831
bpref all 0.2295
recip_rank all 0.5832
P_10 all 0.2675
P_100 all 0.1135
P_1000 all 0.0234
recall_10 all 0.1222
recall_100 all 0.4551
recall_1000 all 0.9040
ndcg all 0.4884
ndcg_cut_10 all 0.2433
ndcg_cut_100 all 0.3420
ndcg_cut_1000 all 0.4884
TUW_tasb_cat¶
Metadata | Participants | Proceedings| Input| Summary (QD) | Summary (QE) | Summary (QR) | Summary (QS)| Appendix
summary (QD)
map all 0.1326
bpref all 0.1727
recip_rank all 0.3699
P_10 all 0.1900
P_100 all 0.0755
P_1000 all 0.0140
recall_10 all 0.1246
recall_100 all 0.4612
recall_1000 all 0.7794
ndcg all 0.3894
ndcg_cut_10 all 0.1730
ndcg_cut_100 all 0.2915
ndcg_cut_1000 all 0.3894
summary (QE)
map all 0.1003
bpref all 0.1241
recip_rank all 0.2948
P_10 all 0.1450
P_100 all 0.0463
P_1000 all 0.0084
recall_10 all 0.1123
recall_100 all 0.3237
recall_1000 all 0.5906
ndcg all 0.2691
ndcg_cut_10 all 0.1353
ndcg_cut_100 all 0.1965
ndcg_cut_1000 all 0.2691
summary (QR)
map all 0.2287
bpref all 0.2811
recip_rank all 0.6499
P_10 all 0.3860
P_100 all 0.1390
P_1000 all 0.0238
recall_10 all 0.1545
recall_100 all 0.4929
recall_1000 all 0.7649
ndcg all 0.4952
ndcg_cut_10 all 0.3255
ndcg_cut_100 all 0.3929
ndcg_cut_1000 all 0.4952
summary (QS)
map all 0.1777
bpref all 0.2332
recip_rank all 0.6088
P_10 all 0.2775
P_100 all 0.1133
P_1000 all 0.0205
recall_10 all 0.1199
recall_100 all 0.4424
recall_1000 all 0.7867
ndcg all 0.4565
ndcg_cut_10 all 0.2533
ndcg_cut_100 all 0.3425
ndcg_cut_1000 all 0.4565
TUW_tasb192_ann¶
Metadata | Participants | Proceedings| Input| Summary (QD) | Summary (QE) | Summary (QR) | Summary (QS)| Appendix
summary (QD)
map all 0.1209
bpref all 0.1846
recip_rank all 0.4179
P_10 all 0.2100
P_100 all 0.0605
P_1000 all 0.0118
recall_10 all 0.1211
recall_100 all 0.3380
recall_1000 all 0.6583
ndcg all 0.3424
ndcg_cut_10 all 0.1884
ndcg_cut_100 all 0.2538
ndcg_cut_1000 all 0.3424
summary (QE)
map all 0.0903
bpref all 0.1322
recip_rank all 0.2933
P_10 all 0.1175
P_100 all 0.0310
P_1000 all 0.0069
recall_10 all 0.1054
recall_100 all 0.2358
recall_1000 all 0.4803
ndcg all 0.2273
ndcg_cut_10 all 0.1366
ndcg_cut_100 all 0.1674
ndcg_cut_1000 all 0.2273
summary (QR)
map all 0.1917
bpref all 0.2674
recip_rank all 0.5575
P_10 all 0.3720
P_100 all 0.1184
P_1000 all 0.0207
recall_10 all 0.1367
recall_100 all 0.4055
recall_1000 all 0.6707
ndcg all 0.4286
ndcg_cut_10 all 0.3082
ndcg_cut_100 all 0.3398
ndcg_cut_1000 all 0.4286
summary (QS)
map all 0.1598
bpref all 0.2431
recip_rank all 0.5056
P_10 all 0.3200
P_100 all 0.0943
P_1000 all 0.0175
recall_10 all 0.1284
recall_100 all 0.3683
recall_1000 all 0.6679
ndcg all 0.3940
ndcg_cut_10 all 0.2605
ndcg_cut_100 all 0.2979
ndcg_cut_1000 all 0.3940
osc_token¶
Metadata | Participants| Input| Summary (QD) | Summary (QE) | Summary (QR) | Summary (QS)| Appendix
summary (QD)
map all 0.0965
bpref all 0.1187
recip_rank all 0.4030
P_10 all 0.1625
P_100 all 0.0658
P_1000 all 0.0066
recall_10 all 0.1104
recall_100 all 0.3921
recall_1000 all 0.3921
ndcg all 0.2529
ndcg_cut_10 all 0.1584
ndcg_cut_100 all 0.2529
ndcg_cut_1000 all 0.2529
summary (QE)
map all 0.0735
bpref all 0.0869
recip_rank all 0.2922
P_10 all 0.1025
P_100 all 0.0425
P_1000 all 0.0043
recall_10 all 0.0704
recall_100 all 0.3229
recall_1000 all 0.3229
ndcg all 0.1914
ndcg_cut_10 all 0.1141
ndcg_cut_100 all 0.1914
ndcg_cut_1000 all 0.1914
summary (QR)
map all 0.1779
bpref all 0.2137
recip_rank all 0.5307
P_10 all 0.3180
P_100 all 0.1204
P_1000 all 0.0120
recall_10 all 0.1363
recall_100 all 0.4559
recall_1000 all 0.4559
ndcg all 0.3431
ndcg_cut_10 all 0.2617
ndcg_cut_100 all 0.3431
ndcg_cut_1000 all 0.3431
summary (QS)
map all 0.1127
bpref all 0.1441
recip_rank all 0.4490
P_10 all 0.2300
P_100 all 0.0925
P_1000 all 0.0093
recall_10 all 0.1024
recall_100 all 0.3872
recall_1000 all 0.3872
ndcg all 0.2747
ndcg_cut_10 all 0.1901
ndcg_cut_100 all 0.2747
ndcg_cut_1000 all 0.2747
osc_vector¶
Metadata | Participants| Input| Summary (QD) | Summary (QE) | Summary (QR) | Summary (QS)| Appendix
summary (QD)
map all 0.1011
bpref all 0.1466
recip_rank all 0.3723
P_10 all 0.1925
P_100 all 0.0568
P_1000 all 0.0057
recall_10 all 0.1166
recall_100 all 0.3549
recall_1000 all 0.3549
ndcg all 0.2403
ndcg_cut_10 all 0.1684
ndcg_cut_100 all 0.2403
ndcg_cut_1000 all 0.2403
summary (QE)
map all 0.0628
bpref all 0.0917
recip_rank all 0.1994
P_10 all 0.1200
P_100 all 0.0330
P_1000 all 0.0033
recall_10 all 0.0955
recall_100 all 0.2642
recall_1000 all 0.2642
ndcg all 0.1510
ndcg_cut_10 all 0.1045
ndcg_cut_100 all 0.1510
ndcg_cut_1000 all 0.1510
summary (QR)
map all 0.1601
bpref all 0.2150
recip_rank all 0.5618
P_10 all 0.3360
P_100 all 0.1070
P_1000 all 0.0107
recall_10 all 0.1362
recall_100 all 0.3709
recall_1000 all 0.3709
ndcg all 0.3137
ndcg_cut_10 all 0.2742
ndcg_cut_100 all 0.3137
ndcg_cut_1000 all 0.3137
summary (QS)
map all 0.1279
bpref all 0.1818
recip_rank all 0.4791
P_10 all 0.2725
P_100 all 0.0892
P_1000 all 0.0089
recall_10 all 0.1188
recall_100 all 0.3693
recall_1000 all 0.3693
ndcg all 0.2778
ndcg_cut_10 all 0.2154
ndcg_cut_100 all 0.2778
ndcg_cut_1000 all 0.2778
osc_vec_tok¶
Metadata | Participants| Input| Summary (QD) | Summary (QE) | Summary (QR) | Summary (QS)| Appendix
summary (QD)
map all 0.1234
bpref all 0.1709
recip_rank all 0.4218
P_10 all 0.2000
P_100 all 0.0688
P_1000 all 0.0069
recall_10 all 0.1347
recall_100 all 0.3921
recall_1000 all 0.3921
ndcg all 0.2800
ndcg_cut_10 all 0.1849
ndcg_cut_100 all 0.2800
ndcg_cut_1000 all 0.2800
summary (QE)
map all 0.0810
bpref all 0.1058
recip_rank all 0.2813
P_10 all 0.1500
P_100 all 0.0420
P_1000 all 0.0042
recall_10 all 0.1020
recall_100 all 0.3100
recall_1000 all 0.3100
ndcg all 0.1884
ndcg_cut_10 all 0.1331
ndcg_cut_100 all 0.1884
ndcg_cut_1000 all 0.1884
summary (QR)
map all 0.1911
bpref all 0.2320
recip_rank all 0.6907
P_10 all 0.3800
P_100 all 0.1200
P_1000 all 0.0120
recall_10 all 0.1556
recall_100 all 0.4124
recall_1000 all 0.4124
ndcg all 0.3541
ndcg_cut_10 all 0.3136
ndcg_cut_100 all 0.3541
ndcg_cut_1000 all 0.3541
summary (QS)
map all 0.1617
bpref all 0.2086
recip_rank all 0.6036
P_10 all 0.3250
P_100 all 0.0992
P_1000 all 0.0099
recall_10 all 0.1445
recall_100 all 0.3884
recall_1000 all 0.3884
ndcg all 0.3225
ndcg_cut_10 all 0.2747
ndcg_cut_100 all 0.3225
ndcg_cut_1000 all 0.3225
osc_tok_vec¶
Metadata | Participants| Input| Summary (QD) | Summary (QE) | Summary (QR) | Summary (QS)| Appendix
summary (QD)
map all 0.1202
bpref all 0.1619
recip_rank all 0.4298
P_10 all 0.2175
P_100 all 0.0658
P_1000 all 0.0066
recall_10 all 0.1440
recall_100 all 0.3921
recall_1000 all 0.3921
ndcg all 0.2712
ndcg_cut_10 all 0.1929
ndcg_cut_100 all 0.2712
ndcg_cut_1000 all 0.2712
summary (QE)
map all 0.0832
bpref all 0.0951
recip_rank all 0.2925
P_10 all 0.1375
P_100 all 0.0425
P_1000 all 0.0043
recall_10 all 0.0999
recall_100 all 0.3229
recall_1000 all 0.3229
ndcg all 0.1947
ndcg_cut_10 all 0.1296
ndcg_cut_100 all 0.1947
ndcg_cut_1000 all 0.1947
summary (QR)
map all 0.2297
bpref all 0.2668
recip_rank all 0.6732
P_10 all 0.4140
P_100 all 0.1204
P_1000 all 0.0120
recall_10 all 0.1892
recall_100 all 0.4559
recall_1000 all 0.4559
ndcg all 0.3888
ndcg_cut_10 all 0.3501
ndcg_cut_100 all 0.3888
ndcg_cut_1000 all 0.3888
summary (QS)
map all 0.1543
bpref all 0.1915
recip_rank all 0.5895
P_10 all 0.3250
P_100 all 0.0925
P_1000 all 0.0093
recall_10 all 0.1502
recall_100 all 0.3872
recall_1000 all 0.3872
ndcg all 0.3126
ndcg_cut_10 all 0.2748
ndcg_cut_100 all 0.3126
ndcg_cut_1000 all 0.3126