Results - Deep Learning 2020¶

roberta-large¶

summary (trec_eval)

map             all 0.4199
bpref           all 0.4412
recip_rank      all 0.9185
P_10            all 0.6378
P_100           all 0.1809
P_1000          all 0.0181
recall_10       all 0.2834
recall_100      all 0.5985
recall_1000     all 0.5985
ndcg            all 0.6059
ndcg_cut_10     all 0.6295
ndcg_cut_100    all 0.6060
ndcg_cut_1000   all 0.6059

fr_doc_roberta¶

summary (trec_eval)

map             all 0.4423
bpref           all 0.4648
recip_rank      all 0.9365
P_10            all 0.6578
P_100           all 0.1976
P_1000          all 0.0198
recall_10       all 0.2909
recall_100      all 0.6410
recall_1000     all 0.6410
ndcg            all 0.6389
ndcg_cut_10     all 0.6404
ndcg_cut_100    all 0.6390
ndcg_cut_1000   all 0.6389

rr-pass-roberta¶

summary (passages-eval)

ndcg_cut_10     all 0.7169
ndcg_cut_100    all 0.6416
ndcg_cut_1000   all 0.6735

summary (trec_eval)

map             all 0.4823
bpref           all 0.4876
recip_rank      all 0.8635
P_10            all 0.5463
P_100           all 0.1528
P_1000          all 0.0178
recall_10       all 0.3909
recall_100      all 0.6986
recall_1000     all 0.7526
ndcg            all 0.6783
ndcg_cut_10     all 0.6558
ndcg_cut_100    all 0.6587
ndcg_cut_1000   all 0.6783

fr_pass_roberta¶

summary (passages-eval)

ndcg_cut_10     all 0.7192
ndcg_cut_100    all 0.6662
ndcg_cut_1000   all 0.7231

summary (trec_eval)

map             all 0.4990
bpref           all 0.5011
recip_rank      all 0.8769
P_10            all 0.5519
P_100           all 0.1639
P_1000          all 0.0209
recall_10       all 0.3930
recall_100      all 0.7438
recall_1000     all 0.8396
ndcg            all 0.7186
ndcg_cut_10     all 0.6614
ndcg_cut_100    all 0.6816
ndcg_cut_1000   all 0.7186

pash_r1¶

summary (passages-eval)

ndcg_cut_10     all 0.7463
ndcg_cut_100    all 0.6478
ndcg_cut_1000   all 0.6797

summary (trec_eval)

map             all 0.4969
bpref           all 0.5024
recip_rank      all 0.8675
P_10            all 0.5870
P_100           all 0.1520
P_1000          all 0.0178
recall_10       all 0.4203
recall_100      all 0.6973
recall_1000     all 0.7526
ndcg            all 0.6835
ndcg_cut_10     all 0.6896
ndcg_cut_100    all 0.6634
ndcg_cut_1000   all 0.6835

CoRT-standalone¶

summary (passages-eval)

ndcg_cut_10     all 0.5926
ndcg_cut_100    all 0.4781
ndcg_cut_1000   all 0.5413

summary (trec_eval)

map             all 0.3308
bpref           all 0.3950
recip_rank      all 0.8112
P_10            all 0.4241
P_100           all 0.1085
P_1000          all 0.0163
recall_10       all 0.2767
recall_100      all 0.5301
recall_1000     all 0.6973
ndcg            all 0.5669
ndcg_cut_10     all 0.5223
ndcg_cut_100    all 0.5088
ndcg_cut_1000   all 0.5669

CoRT-bm25¶

summary (passages-eval)

ndcg_cut_10     all 0.5992
ndcg_cut_100    all 0.5671
ndcg_cut_1000   all 0.6630

summary (trec_eval)

map             all 0.3611
bpref           all 0.3670
recip_rank      all 0.8372
P_10            all 0.4370
P_100           all 0.1341
P_1000          all 0.0213
recall_10       all 0.3056
recall_100      all 0.6454
recall_1000     all 0.8496
ndcg            all 0.6384
ndcg_cut_10     all 0.5273
ndcg_cut_100    all 0.5643
ndcg_cut_1000   all 0.6384

CoRT-electra¶

summary (passages-eval)

ndcg_cut_10     all 0.7566
ndcg_cut_100    all 0.6853
ndcg_cut_1000   all 0.7438

summary (trec_eval)

map             all 0.5399
bpref           all 0.5442
recip_rank      all 0.8703
P_10            all 0.5889
P_100           all 0.1680
P_1000          all 0.0213
recall_10       all 0.4127
recall_100      all 0.7483
recall_1000     all 0.8496
ndcg            all 0.7460
ndcg_cut_10     all 0.6996
ndcg_cut_100    all 0.7093
ndcg_cut_1000   all 0.7460

pash_r2¶

summary (passages-eval)

ndcg_cut_10     all 0.8011
ndcg_cut_100    all 0.6703
ndcg_cut_1000   all 0.7020

summary (trec_eval)

map             all 0.5420
bpref           all 0.5458
recip_rank      all 0.9023
P_10            all 0.6426
P_100           all 0.1520
P_1000          all 0.0178
recall_10       all 0.4570
recall_100      all 0.6973
recall_1000     all 0.7526
ndcg            all 0.7153
ndcg_cut_10     all 0.7589
ndcg_cut_100    all 0.6952
ndcg_cut_1000   all 0.7153

relemb_mlm_0_2¶

summary (passages-eval)

ndcg_cut_10     all 0.6662
ndcg_cut_100    all 0.6063
ndcg_cut_1000   all 0.6520

summary (trec_eval)

map             all 0.4350
bpref           all 0.4361
recip_rank      all 0.7677
P_10            all 0.5222
P_100           all 0.1394
P_1000          all 0.0178
recall_10       all 0.3840
recall_100      all 0.6790
recall_1000     all 0.7526
ndcg            all 0.6395
ndcg_cut_10     all 0.5990
ndcg_cut_100    all 0.6090
ndcg_cut_1000   all 0.6395

ndrm3-orc-full¶

summary (trec_eval)

map             all 0.4280
bpref           all 0.4493
recip_rank      all 0.9444
P_10            all 0.6422
P_100           all 0.1944
P_1000          all 0.0194
recall_10       all 0.2919
recall_100      all 0.6382
recall_1000     all 0.6382
ndcg            all 0.6226
ndcg_cut_10     all 0.6249
ndcg_cut_100    all 0.6228
ndcg_cut_1000   all 0.6226

ndrm3-orc-re¶

summary (trec_eval)

map             all 0.4194
bpref           all 0.4346
recip_rank      all 0.9241
P_10            all 0.6422
P_100           all 0.1809
P_1000          all 0.0181
recall_10       all 0.2911
recall_100      all 0.5985
recall_1000     all 0.5985
ndcg            all 0.5960
ndcg_cut_10     all 0.6217
ndcg_cut_100    all 0.5962
ndcg_cut_1000   all 0.5960

ndrm3-full¶

summary (trec_eval)

map             all 0.4069
bpref           all 0.4315
recip_rank      all 0.9333
P_10            all 0.6311
P_100           all 0.1904
P_1000          all 0.0190
recall_10       all 0.2821
recall_100      all 0.6211
recall_1000     all 0.6211
ndcg            all 0.6104
ndcg_cut_10     all 0.6162
ndcg_cut_100    all 0.6105
ndcg_cut_1000   all 0.6104

ndrm3-re¶

summary (trec_eval)

map             all 0.4122
bpref           all 0.4216
recip_rank      all 0.9333
P_10            all 0.6333
P_100           all 0.1809
P_1000          all 0.0181
recall_10       all 0.2797
recall_100      all 0.5985
recall_1000     all 0.5985
ndcg            all 0.5938
ndcg_cut_10     all 0.6162
ndcg_cut_100    all 0.5940
ndcg_cut_1000   all 0.5938

ndrm1-full¶

summary (trec_eval)

map             all 0.3858
bpref           all 0.4218
recip_rank      all 0.9333
P_10            all 0.5889
P_100           all 0.1804
P_1000          all 0.0180
recall_10       all 0.2699
recall_100      all 0.5828
recall_1000     all 0.5828
ndcg            all 0.5955
ndcg_cut_10     all 0.5991
ndcg_cut_100    all 0.5956
ndcg_cut_1000   all 0.5955

ndrm1-re¶

summary (trec_eval)

map             all 0.4150
bpref           all 0.4285
recip_rank      all 0.9333
P_10            all 0.6267
P_100           all 0.1809
P_1000          all 0.0181
recall_10       all 0.2810
recall_100      all 0.5985
recall_1000     all 0.5985
ndcg            all 0.6018
ndcg_cut_10     all 0.6161
ndcg_cut_100    all 0.6020
ndcg_cut_1000   all 0.6018

ICIP_run1¶

summary (trec_eval)

map             all 0.4333
bpref           all 0.4488
recip_rank      all 0.9630
P_10            all 0.6556
P_100           all 0.1809
P_1000          all 0.0181
recall_10       all 0.3024
recall_100      all 0.5985
recall_1000     all 0.5985
ndcg            all 0.6173
ndcg_cut_10     all 0.6623
ndcg_cut_100    all 0.6174
ndcg_cut_1000   all 0.6173

ICIP_run3¶

summary (trec_eval)

map             all 0.4360
bpref           all 0.4451
recip_rank      all 0.9667
P_10            all 0.6533
P_100           all 0.1809
P_1000          all 0.0181
recall_10       all 0.2980
recall_100      all 0.5985
recall_1000     all 0.5985
ndcg            all 0.6166
ndcg_cut_10     all 0.6528
ndcg_cut_100    all 0.6167
ndcg_cut_1000   all 0.6166

ICIP_run2¶

summary (trec_eval)

map             all 0.4206
bpref           all 0.4350
recip_rank      all 0.9407
P_10            all 0.6311
P_100           all 0.1809
P_1000          all 0.0181
recall_10       all 0.2819
recall_100      all 0.5985
recall_1000     all 0.5985
ndcg            all 0.6119
ndcg_cut_10     all 0.6322
ndcg_cut_100    all 0.6120
ndcg_cut_1000   all 0.6119

TUW-TK-2Layer¶

summary (passages-eval)

ndcg_cut_10     all 0.6539
ndcg_cut_100    all 0.5927
ndcg_cut_1000   all 0.6423

summary (trec_eval)

map             all 0.4179
bpref           all 0.4283
recip_rank      all 0.7654
P_10            all 0.5019
P_100           all 0.1380
P_1000          all 0.0178
recall_10       all 0.3609
recall_100      all 0.6649
recall_1000     all 0.7526
ndcg            all 0.6328
ndcg_cut_10     all 0.5871
ndcg_cut_100    all 0.5994
ndcg_cut_1000   all 0.6328

TUW-TK-Sparse¶

summary (passages-eval)

ndcg_cut_10     all 0.6610
ndcg_cut_100    all 0.5944
ndcg_cut_1000   all 0.6407

summary (trec_eval)

map             all 0.4164
bpref           all 0.4300
recip_rank      all 0.7970
P_10            all 0.5167
P_100           all 0.1404
P_1000          all 0.0178
recall_10       all 0.3670
recall_100      all 0.6683
recall_1000     all 0.7526
ndcg            all 0.6281
ndcg_cut_10     all 0.5927
ndcg_cut_100    all 0.5968
ndcg_cut_1000   all 0.6281

TUW-TKL-2k¶

summary (trec_eval)

map             all 0.3810
bpref           all 0.3991
recip_rank      all 0.9296
P_10            all 0.6089
P_100           all 0.1809
P_1000          all 0.0181
recall_10       all 0.2655
recall_100      all 0.5985
recall_1000     all 0.5985
ndcg            all 0.5792
ndcg_cut_10     all 0.5852
ndcg_cut_100    all 0.5794
ndcg_cut_1000   all 0.5792

TUW-TKL-4k¶

summary (trec_eval)

map             all 0.3749
bpref           all 0.3953
recip_rank      all 0.9185
P_10            all 0.5778
P_100           all 0.1809
P_1000          all 0.0181
recall_10       all 0.2490
recall_100      all 0.5985
recall_1000     all 0.5985
ndcg            all 0.5785
ndcg_cut_10     all 0.5749
ndcg_cut_100    all 0.5786
ndcg_cut_1000   all 0.5785

pinganNLP1¶

summary (passages-eval)

ndcg_cut_10     all 0.7343
ndcg_cut_100    all 0.6474
ndcg_cut_1000   all 0.6783

summary (trec_eval)

map             all 0.4896
bpref           all 0.4945
recip_rank      all 0.8593
P_10            all 0.5778
P_100           all 0.1520
P_1000          all 0.0178
recall_10       all 0.4044
recall_100      all 0.6993
recall_1000     all 0.7526
ndcg            all 0.6773
ndcg_cut_10     all 0.6740
ndcg_cut_100    all 0.6579
ndcg_cut_1000   all 0.6773

pinganNLP2¶

summary (passages-eval)

ndcg_cut_10     all 0.7368
ndcg_cut_100    all 0.6474
ndcg_cut_1000   all 0.6783

summary (trec_eval)

map             all 0.4881
bpref           all 0.4930
recip_rank      all 0.8602
P_10            all 0.5796
P_100           all 0.1520
P_1000          all 0.0178
recall_10       all 0.4091
recall_100      all 0.6993
recall_1000     all 0.7526
ndcg            all 0.6770
ndcg_cut_10     all 0.6768
ndcg_cut_100    all 0.6576
ndcg_cut_1000   all 0.6770

uob_runid1¶

summary (trec_eval)

map             all 0.3786
bpref           all 0.4056
recip_rank      all 0.8852
P_10            all 0.5844
P_100           all 0.1809
P_1000          all 0.0181
recall_10       all 0.2614
recall_100      all 0.5985
recall_1000     all 0.5985
ndcg            all 0.5775
ndcg_cut_10     all 0.5781
ndcg_cut_100    all 0.5776
ndcg_cut_1000   all 0.5775

uob_runid2¶

summary (trec_eval)

map             all 0.3976
bpref           all 0.4173
recip_rank      all 0.9100
P_10            all 0.5956
P_100           all 0.1809
P_1000          all 0.0181
recall_10       all 0.2712
recall_100      all 0.5985
recall_1000     all 0.5985
ndcg            all 0.5859
ndcg_cut_10     all 0.5830
ndcg_cut_100    all 0.5860
ndcg_cut_1000   all 0.5859

uob_runid3¶

summary (trec_eval)

map             all 0.3948
bpref           all 0.4147
recip_rank      all 0.9259
P_10            all 0.6133
P_100           all 0.1809
P_1000          all 0.0181
recall_10       all 0.2794
recall_100      all 0.5985
recall_1000     all 0.5985
ndcg            all 0.5851
ndcg_cut_10     all 0.5949
ndcg_cut_100    all 0.5852
ndcg_cut_1000   all 0.5851

d_d2q_bm25rm3¶

Metadata | Participants| Input| Summary| Appendix

summary (trec_eval)

map             all 0.4228
bpref           all 0.4423
recip_rank      all 0.8147
P_10            all 0.5933
P_100           all 0.2027
P_1000          all 0.0203
recall_10       all 0.2594
recall_100      all 0.6555
recall_1000     all 0.6555
ndcg            all 0.5902
ndcg_cut_10     all 0.5407
ndcg_cut_100    all 0.5903
ndcg_cut_1000   all 0.5902

d_d2q_bm25¶

Metadata | Participants| Input| Summary| Appendix

summary (trec_eval)

map             all 0.4230
bpref           all 0.4480
recip_rank      all 0.9369
P_10            all 0.6089
P_100           all 0.1944
P_1000          all 0.0194
recall_10       all 0.2833
recall_100      all 0.6412
recall_1000     all 0.6412
ndcg            all 0.6115
ndcg_cut_10     all 0.5885
ndcg_cut_100    all 0.6117
ndcg_cut_1000   all 0.6115

d_bm25rm3¶

Metadata | Participants| Input| Summary| Appendix

summary (trec_eval)

map             all 0.4006
bpref           all 0.4127
recip_rank      all 0.8541
P_10            all 0.5600
P_100           all 0.1971
P_1000          all 0.0197
recall_10       all 0.2474
recall_100      all 0.6392
recall_1000     all 0.6392
ndcg            all 0.5726
ndcg_cut_10     all 0.5248
ndcg_cut_100    all 0.5727
ndcg_cut_1000   all 0.5726

d_bm25¶

Metadata | Participants| Input| Summary| Appendix

summary (trec_eval)

map             all 0.3791
bpref           all 0.4030
recip_rank      all 0.8521
P_10            all 0.5644
P_100           all 0.1836
P_1000          all 0.0184
recall_10       all 0.2671
recall_100      all 0.6110
recall_1000     all 0.6110
ndcg            all 0.5647
ndcg_cut_10     all 0.5271
ndcg_cut_100    all 0.5648
ndcg_cut_1000   all 0.5647

p_d2q_bm25rm3¶

summary (passages-eval)

ndcg_cut_10     all 0.6172
ndcg_cut_100    all 0.6206
ndcg_cut_1000   all 0.7041

summary (trec_eval)

map             all 0.4295
bpref           all 0.4359
recip_rank      all 0.7424
P_10            all 0.4759
P_100           all 0.1589
P_1000          all 0.0236
recall_10       all 0.3326
recall_100      all 0.7153
recall_1000     all 0.8699
ndcg            all 0.6652
ndcg_cut_10     all 0.5328
ndcg_cut_100    all 0.6029
ndcg_cut_1000   all 0.6652

p_d2q_bm25¶

summary (passages-eval)

ndcg_cut_10     all 0.6187
ndcg_cut_100    all 0.6081
ndcg_cut_1000   all 0.6840

summary (trec_eval)

map             all 0.4074
bpref           all 0.4182
recip_rank      all 0.7326
P_10            all 0.4796
P_100           all 0.1481
P_1000          all 0.0218
recall_10       all 0.3469
recall_100      all 0.7044
recall_1000     all 0.8452
ndcg            all 0.6500
ndcg_cut_10     all 0.5384
ndcg_cut_100    all 0.5929
ndcg_cut_1000   all 0.6500

p_bm25rm3¶

summary (passages-eval)

ndcg_cut_10     all 0.4821
ndcg_cut_100    all 0.5077
ndcg_cut_1000   all 0.6046

summary (trec_eval)

map             all 0.3019
bpref           all 0.2922
recip_rank      all 0.6360
P_10            all 0.3685
P_100           all 0.1239
P_1000          all 0.0207
recall_10       all 0.2393
recall_100      all 0.6046
recall_1000     all 0.8217
ndcg            all 0.5514
ndcg_cut_10     all 0.4028
ndcg_cut_100    all 0.4723
ndcg_cut_1000   all 0.5514

p_bm25¶

summary (passages-eval)

ndcg_cut_10     all 0.4796
ndcg_cut_100    all 0.4902
ndcg_cut_1000   all 0.5830

summary (trec_eval)

map             all 0.2856
bpref           all 0.2866
recip_rank      all 0.6585
P_10            all 0.3500
P_100           all 0.1135
P_1000          all 0.0190
recall_10       all 0.2467
recall_100      all 0.5599
recall_1000     all 0.7863
ndcg            all 0.5377
ndcg_cut_10     all 0.4040
ndcg_cut_100    all 0.4614
ndcg_cut_1000   all 0.5377

d_d2q_duo¶

summary (trec_eval)

map             all 0.5422
bpref           all 0.5617
recip_rank      all 0.9476
P_10            all 0.7333
P_100           all 0.2342
P_1000          all 0.0234
recall_10       all 0.3233
recall_100      all 0.7427
recall_1000     all 0.7427
ndcg            all 0.7089
ndcg_cut_10     all 0.6934
ndcg_cut_100    all 0.7090
ndcg_cut_1000   all 0.7089

d_d2q_rm3_duo¶

summary (trec_eval)

map             all 0.5427
bpref           all 0.5647
recip_rank      all 0.9476
P_10            all 0.7267
P_100           all 0.2387
P_1000          all 0.0239
recall_10       all 0.3212
recall_100      all 0.7469
recall_1000     all 0.7469
ndcg            all 0.7122
ndcg_cut_10     all 0.6900
ndcg_cut_100    all 0.7124
ndcg_cut_1000   all 0.7122

d_rm3_duo¶

summary (trec_eval)

map             all 0.5270
bpref           all 0.5464
recip_rank      all 0.9476
P_10            all 0.7222
P_100           all 0.2278
P_1000          all 0.0228
recall_10       all 0.3185
recall_100      all 0.7195
recall_1000     all 0.7195
ndcg            all 0.6929
ndcg_cut_10     all 0.6794
ndcg_cut_100    all 0.6930
ndcg_cut_1000   all 0.6929

p_d2q_bm25_duo¶

summary (passages-eval)

ndcg_cut_10     all 0.7837
ndcg_cut_100    all 0.7047
ndcg_cut_1000   all 0.7539

summary (trec_eval)

map             all 0.5609
bpref           all 0.5505
recip_rank      all 0.8798
P_10            all 0.6259
P_100           all 0.1733
P_1000          all 0.0218
recall_10       all 0.4413
recall_100      all 0.7636
recall_1000     all 0.8452
ndcg            all 0.7533
ndcg_cut_10     all 0.7269
ndcg_cut_100    all 0.7202
ndcg_cut_1000   all 0.7533

p_bm25rm3_duo¶

summary (passages-eval)

ndcg_cut_10     all 0.7583
ndcg_cut_100    all 0.6913
ndcg_cut_1000   all 0.7387

summary (trec_eval)

map             all 0.5355
bpref           all 0.5205
recip_rank      all 0.8759
P_10            all 0.5963
P_100           all 0.1672
P_1000          all 0.0207
recall_10       all 0.4244
recall_100      all 0.7483
recall_1000     all 0.8217
ndcg            all 0.7300
ndcg_cut_10     all 0.6994
ndcg_cut_100    all 0.7004
ndcg_cut_1000   all 0.7300

p_d2q_rm3_duo¶

summary (passages-eval)

ndcg_cut_10     all 0.7821
ndcg_cut_100    all 0.7071
ndcg_cut_1000   all 0.7732

summary (trec_eval)

map             all 0.5643
bpref           all 0.5607
recip_rank      all 0.8798
P_10            all 0.6241
P_100           all 0.1752
P_1000          all 0.0239
recall_10       all 0.4440
recall_100      all 0.7712
recall_1000     all 0.8749
ndcg            all 0.7670
ndcg_cut_10     all 0.7268
ndcg_cut_100    all 0.7232
ndcg_cut_1000   all 0.7670

BIT-run1¶

summary (trec_eval)

map             all 0.3466
bpref           all 0.3785
recip_rank      all 0.8389
P_10            all 0.5556
P_100           all 0.1800
P_1000          all 0.0180
recall_10       all 0.2390
recall_100      all 0.5763
recall_1000     all 0.5763
ndcg            all 0.5429
ndcg_cut_10     all 0.5239
ndcg_cut_100    all 0.5430
ndcg_cut_1000   all 0.5429

pash_r3¶

summary (passages-eval)

ndcg_cut_10     all 0.8031
ndcg_cut_100    all 0.6719
ndcg_cut_1000   all 0.7034

summary (trec_eval)

map             all 0.5445
bpref           all 0.5461
recip_rank      all 0.9147
P_10            all 0.6426
P_100           all 0.1520
P_1000          all 0.0178
recall_10       all 0.4569
recall_100      all 0.6973
recall_1000     all 0.7526
ndcg            all 0.7189
ndcg_cut_10     all 0.7628
ndcg_cut_100    all 0.6988
ndcg_cut_1000   all 0.7189

DoRA_Small¶

summary (passages-eval)

ndcg_cut_10     all 0.0484
ndcg_cut_100    all 0.0259
ndcg_cut_1000   all 0.0253

summary (trec_eval)

map             all 0.0088
bpref           all 0.0155
recip_rank      all 0.1287
P_10            all 0.0241
P_100           all 0.0039
P_1000          all 0.0004
recall_10       all 0.0124
recall_100      all 0.0156
recall_1000     all 0.0170
ndcg            all 0.0258
ndcg_cut_10     all 0.0339
ndcg_cut_100    all 0.0257
ndcg_cut_1000   all 0.0258

DoRA_Large¶

summary (passages-eval)

ndcg_cut_10     all 0.0414
ndcg_cut_100    all 0.0235
ndcg_cut_1000   all 0.0231

summary (trec_eval)

map             all 0.0079
bpref           all 0.0147
recip_rank      all 0.1111
P_10            all 0.0241
P_100           all 0.0039
P_1000          all 0.0004
recall_10       all 0.0125
recall_100      all 0.0156
recall_1000     all 0.0170
ndcg            all 0.0247
ndcg_cut_10     all 0.0310
ndcg_cut_100    all 0.0246
ndcg_cut_1000   all 0.0247

DoRA_Med¶

summary (passages-eval)

ndcg_cut_10     all 0.0431
ndcg_cut_100    all 0.0241
ndcg_cut_1000   all 0.0237

summary (trec_eval)

map             all 0.0087
bpref           all 0.0151
recip_rank      all 0.1075
P_10            all 0.0259
P_100           all 0.0039
P_1000          all 0.0004
recall_10       all 0.0122
recall_100      all 0.0156
recall_1000     all 0.0170
ndcg            all 0.0255
ndcg_cut_10     all 0.0326
ndcg_cut_100    all 0.0254
ndcg_cut_1000   all 0.0255

BIT-run2¶

summary (trec_eval)

map             all 0.3466
bpref           all 0.3757
recip_rank      all 0.8611
P_10            all 0.5578
P_100           all 0.1800
P_1000          all 0.0180
recall_10       all 0.2407
recall_100      all 0.5763
recall_1000     all 0.5763
ndcg            all 0.5446
ndcg_cut_10     all 0.5283
ndcg_cut_100    all 0.5447
ndcg_cut_1000   all 0.5446

DoRA_Large_1k¶

summary (passages-eval)

ndcg_cut_10     all 0.2661
ndcg_cut_100    all 0.3864
ndcg_cut_1000   all 0.4811

summary (trec_eval)

map             all 0.2072
bpref           all 0.2918
recip_rank      all 0.2740
P_10            all 0.2111
P_100           all 0.1100
P_1000          all 0.0178
recall_10       all 0.2046
recall_100      all 0.5662
recall_1000     all 0.7526
ndcg            all 0.4377
ndcg_cut_10     all 0.2156
ndcg_cut_100    all 0.3657
ndcg_cut_1000   all 0.4377

med_1k¶

summary (passages-eval)

ndcg_cut_10     all 0.2708
ndcg_cut_100    all 0.3980
ndcg_cut_1000   all 0.4827

summary (trec_eval)

map             all 0.2081
bpref           all 0.2998
recip_rank      all 0.2720
P_10            all 0.2185
P_100           all 0.1159
P_1000          all 0.0178
recall_10       all 0.2090
recall_100      all 0.5958
recall_1000     all 0.7526
ndcg            all 0.4390
ndcg_cut_10     all 0.2181
ndcg_cut_100    all 0.3770
ndcg_cut_1000   all 0.4390

mpii_run1¶

summary (trec_eval)

map             all 0.4030
bpref           all 0.4204
recip_rank      all 0.9000
P_10            all 0.6178
P_100           all 0.1809
P_1000          all 0.0181
recall_10       all 0.2685
recall_100      all 0.5985
recall_1000     all 0.5985
ndcg            all 0.5883
ndcg_cut_10     all 0.6017
ndcg_cut_100    all 0.5884
ndcg_cut_1000   all 0.5883

mpii_run2¶

summary (trec_eval)

map             all 0.4205
bpref           all 0.4415
recip_rank      all 0.8833
P_10            all 0.6489
P_100           all 0.1809
P_1000          all 0.0181
recall_10       all 0.2981
recall_100      all 0.5985
recall_1000     all 0.5985
ndcg            all 0.5891
ndcg_cut_10     all 0.6135
ndcg_cut_100    all 0.5892
ndcg_cut_1000   all 0.5891

mpii_run3¶

summary (trec_eval)

map             all 0.2587
bpref           all 0.2938
recip_rank      all 0.6388
P_10            all 0.3933
P_100           all 0.1809
P_1000          all 0.0181
recall_10       all 0.1658
recall_100      all 0.5985
recall_1000     all 0.5985
ndcg            all 0.4474
ndcg_cut_10     all 0.3286
ndcg_cut_100    all 0.4474
ndcg_cut_1000   all 0.4474

small_1k¶

summary (passages-eval)

ndcg_cut_10     all 0.2767
ndcg_cut_100    all 0.3923
ndcg_cut_1000   all 0.4841

summary (trec_eval)

map             all 0.2112
bpref           all 0.2919
recip_rank      all 0.2785
P_10            all 0.2130
P_100           all 0.1117
P_1000          all 0.0178
recall_10       all 0.2160
recall_100      all 0.5619
recall_1000     all 0.7526
ndcg            all 0.4415
ndcg_cut_10     all 0.2219
ndcg_cut_100    all 0.3694
ndcg_cut_1000   all 0.4415

pash_f1¶

summary (passages-eval)

ndcg_cut_10     all 0.7956
ndcg_cut_100    all 0.6756
ndcg_cut_1000   all 0.7074

summary (trec_eval)

map             all 0.5455
bpref           all 0.5473
recip_rank      all 0.8699
P_10            all 0.6463
P_100           all 0.1563
P_1000          all 0.0182
recall_10       all 0.4545
recall_100      all 0.7108
recall_1000     all 0.7667
ndcg            all 0.7186
ndcg_cut_10     all 0.7504
ndcg_cut_100    all 0.6981
ndcg_cut_1000   all 0.7186

pash_f2¶

summary (passages-eval)

ndcg_cut_10     all 0.7941
ndcg_cut_100    all 0.6704
ndcg_cut_1000   all 0.7019

summary (trec_eval)

map             all 0.5389
bpref           all 0.5415
recip_rank      all 0.8931
P_10            all 0.6389
P_100           all 0.1541
P_1000          all 0.0180
recall_10       all 0.4503
recall_100      all 0.7042
recall_1000     all 0.7595
ndcg            all 0.7146
ndcg_cut_10     all 0.7513
ndcg_cut_100    all 0.6945
ndcg_cut_1000   all 0.7146

bert_6¶

summary (passages-eval)

ndcg_cut_10     all 0.6149
ndcg_cut_100    all 0.5669
ndcg_cut_1000   all 0.6215

summary (trec_eval)

map             all 0.3760
bpref           all 0.3713
recip_rank      all 0.7386
P_10            all 0.4648
P_100           all 0.1337
P_1000          all 0.0178
recall_10       all 0.3263
recall_100      all 0.6510
recall_1000     all 0.7526
ndcg            all 0.6015
ndcg_cut_10     all 0.5339
ndcg_cut_100    all 0.5627
ndcg_cut_1000   all 0.6015

pinganNLP3¶

summary (passages-eval)

ndcg_cut_10     all 0.7352
ndcg_cut_100    all 0.6486
ndcg_cut_1000   all 0.6780

summary (trec_eval)

map             all 0.4918
bpref           all 0.5001
recip_rank      all 0.8586
P_10            all 0.5870
P_100           all 0.1530
P_1000          all 0.0178
recall_10       all 0.4072
recall_100      all 0.7009
recall_1000     all 0.7526
ndcg            all 0.6784
ndcg_cut_10     all 0.6790
ndcg_cut_100    all 0.6599
ndcg_cut_1000   all 0.6784

pash_f3¶

summary (passages-eval)

ndcg_cut_10     all 0.8005
ndcg_cut_100    all 0.6787
ndcg_cut_1000   all 0.7120

summary (trec_eval)

map             all 0.5504
bpref           all 0.5503
recip_rank      all 0.8885
P_10            all 0.6481
P_100           all 0.1563
P_1000          all 0.0183
recall_10       all 0.4553
recall_100      all 0.7108
recall_1000     all 0.7708
ndcg            all 0.7252
ndcg_cut_10     all 0.7561
ndcg_cut_100    all 0.7034
ndcg_cut_1000   all 0.7252

NLE_pr1¶

summary (passages-eval)

ndcg_cut_10     all 0.7325
ndcg_cut_100    all 0.6516
ndcg_cut_1000   all 0.6785

summary (trec_eval)

map             all 0.5050
bpref           all 0.5085
recip_rank      all 0.8551
P_10            all 0.5667
P_100           all 0.1644
P_1000          all 0.0196
recall_10       all 0.3981
recall_100      all 0.7252
recall_1000     all 0.7869
ndcg            all 0.7010
ndcg_cut_10     all 0.6671
ndcg_cut_100    all 0.6750
ndcg_cut_1000   all 0.7010

NLE_pr2¶

summary (passages-eval)

ndcg_cut_10     all 0.7341
ndcg_cut_100    all 0.6552
ndcg_cut_1000   all 0.6812

summary (trec_eval)

map             all 0.5117
bpref           all 0.5164
recip_rank      all 0.8454
P_10            all 0.5648
P_100           all 0.1643
P_1000          all 0.0196
recall_10       all 0.4048
recall_100      all 0.7259
recall_1000     all 0.7869
ndcg            all 0.7057
ndcg_cut_10     all 0.6715
ndcg_cut_100    all 0.6800
ndcg_cut_1000   all 0.7057

NLE_pr3¶

summary (passages-eval)

ndcg_cut_10     all 0.7458
ndcg_cut_100    all 0.6854
ndcg_cut_1000   all 0.7462

summary (trec_eval)

map             all 0.5245
bpref           all 0.5175
recip_rank      all 0.8440
P_10            all 0.5741
P_100           all 0.1709
P_1000          all 0.0221
recall_10       all 0.4033
recall_100      all 0.7634
recall_1000     all 0.8639
ndcg            all 0.7377
ndcg_cut_10     all 0.6796
ndcg_cut_100    all 0.6974
ndcg_cut_1000   all 0.7377

uogTrBaseDPH¶

Metadata | Participants| Input| Summary| Appendix

summary (trec_eval)

map             all 0.3070
bpref           all 0.3389
recip_rank      all 0.8415
P_10            all 0.5089
P_100           all 0.1533
P_1000          all 0.0153
recall_10       all 0.2212
recall_100      all 0.5178
recall_1000     all 0.5178
ndcg            all 0.4971
ndcg_cut_10     all 0.4871
ndcg_cut_100    all 0.4972
ndcg_cut_1000   all 0.4971

uogTrBaseL16¶

Metadata | Participants| Input| Summary| Appendix

summary (trec_eval)

map             all 0.3248
bpref           all 0.3495
recip_rank      all 0.8219
P_10            all 0.5289
P_100           all 0.1538
P_1000          all 0.0154
recall_10       all 0.2348
recall_100      all 0.5182
recall_1000     all 0.5182
ndcg            all 0.5051
ndcg_cut_10     all 0.4964
ndcg_cut_100    all 0.5052
ndcg_cut_1000   all 0.5051

uogTrBaseL17o¶

Metadata | Participants| Input| Summary| Appendix

summary (trec_eval)

map             all 0.3248
bpref           all 0.3510
recip_rank      all 0.7980
P_10            all 0.5333
P_100           all 0.1540
P_1000          all 0.0154
recall_10       all 0.2314
recall_100      all 0.5188
recall_1000     all 0.5188
ndcg            all 0.5089
ndcg_cut_10     all 0.5120
ndcg_cut_100    all 0.5090
ndcg_cut_1000   all 0.5089

uogTrBaseDPHQ¶

Metadata | Participants| Input| Summary| Appendix

summary (trec_eval)

map             all 0.3461
bpref           all 0.3685
recip_rank      all 0.8052
P_10            all 0.5444
P_100           all 0.1800
P_1000          all 0.0180
recall_10       all 0.2271
recall_100      all 0.5784
recall_1000     all 0.5784
ndcg            all 0.5344
ndcg_cut_10     all 0.5052
ndcg_cut_100    all 0.5345
ndcg_cut_1000   all 0.5344

uogTrBaseQL16¶

Metadata | Participants| Input| Summary| Appendix

summary (trec_eval)

map             all 0.3436
bpref           all 0.3717
recip_rank      all 0.7930
P_10            all 0.5311
P_100           all 0.1796
P_1000          all 0.0180
recall_10       all 0.2291
recall_100      all 0.5773
recall_1000     all 0.5773
ndcg            all 0.5356
ndcg_cut_10     all 0.4998
ndcg_cut_100    all 0.5357
ndcg_cut_1000   all 0.5356

uogTrBaseQL17o¶

Metadata | Participants| Input| Summary| Appendix

summary (trec_eval)

map             all 0.3529
bpref           all 0.3825
recip_rank      all 0.8276
P_10            all 0.5511
P_100           all 0.1793
P_1000          all 0.0179
recall_10       all 0.2437
recall_100      all 0.5770
recall_1000     all 0.5770
ndcg            all 0.5398
ndcg_cut_10     all 0.5203
ndcg_cut_100    all 0.5399
ndcg_cut_1000   all 0.5398

nlm-ens-bst-3¶

summary (passages-eval)

ndcg_cut_10     all 0.6803
ndcg_cut_100    all 0.6194
ndcg_cut_1000   all 0.6851

summary (trec_eval)

map             all 0.4526
bpref           all 0.4475
recip_rank      all 0.8491
P_10            all 0.5056
P_100           all 0.1469
P_1000          all 0.0198
recall_10       all 0.3580
recall_100      all 0.6877
recall_1000     all 0.7980
ndcg            all 0.6752
ndcg_cut_10     all 0.6103
ndcg_cut_100    all 0.6330
ndcg_cut_1000   all 0.6752

nlm-ens-bst-2¶

summary (passages-eval)

ndcg_cut_10     all 0.6934
ndcg_cut_100    all 0.6188
ndcg_cut_1000   all 0.6676

summary (trec_eval)

map             all 0.4598
bpref           all 0.4648
recip_rank      all 0.8203
P_10            all 0.5296
P_100           all 0.1448
P_1000          all 0.0184
recall_10       all 0.3849
recall_100      all 0.6829
recall_1000     all 0.7627
ndcg            all 0.6649
ndcg_cut_10     all 0.6270
ndcg_cut_100    all 0.6340
ndcg_cut_1000   all 0.6649

nlm-prfun-bert¶

summary (passages-eval)

ndcg_cut_10     all 0.6648
ndcg_cut_100    all 0.5946
ndcg_cut_1000   all 0.6435

summary (trec_eval)

map             all 0.4265
bpref           all 0.4278
recip_rank      all 0.8603
P_10            all 0.4963
P_100           all 0.1376
P_1000          all 0.0178
recall_10       all 0.3605
recall_100      all 0.6591
recall_1000     all 0.7415
ndcg            all 0.6440
ndcg_cut_10     all 0.6036
ndcg_cut_100    all 0.6118
ndcg_cut_1000   all 0.6440

nlm-bert-rr¶

summary (passages-eval)

ndcg_cut_10     all 0.6721
ndcg_cut_100    all 0.6025
ndcg_cut_1000   all 0.6471

summary (trec_eval)

map             all 0.4341
bpref           all 0.4446
recip_rank      all 0.7785
P_10            all 0.5259
P_100           all 0.1430
P_1000          all 0.0178
recall_10       all 0.3894
recall_100      all 0.6762
recall_1000     all 0.7526
ndcg            all 0.6409
ndcg_cut_10     all 0.6050
ndcg_cut_100    all 0.6118
ndcg_cut_1000   all 0.6409

BIT-run3¶

summary (trec_eval)

map             all 0.3267
bpref           all 0.3636
recip_rank      all 0.8296
P_10            all 0.5311
P_100           all 0.1771
P_1000          all 0.0177
recall_10       all 0.2313
recall_100      all 0.5689
recall_1000     all 0.5689
ndcg            all 0.5364
ndcg_cut_10     all 0.5063
ndcg_cut_100    all 0.5365
ndcg_cut_1000   all 0.5364

uogTrQCBMP¶

summary (trec_eval)

map             all 0.3752
bpref           all 0.3986
recip_rank      all 0.8722
P_10            all 0.6000
P_100           all 0.1798
P_1000          all 0.0180
recall_10       all 0.2574
recall_100      all 0.5777
recall_1000     all 0.5777
ndcg            all 0.5672
ndcg_cut_10     all 0.5791
ndcg_cut_100    all 0.5673
ndcg_cut_1000   all 0.5672

1¶

summary (passages-eval)

ndcg_cut_10     all 0.7271
ndcg_cut_100    all 0.6356
ndcg_cut_1000   all 0.6759

summary (trec_eval)

map             all 0.4899
bpref           all 0.5034
recip_rank      all 0.8691
P_10            all 0.5556
P_100           all 0.1507
P_1000          all 0.0178
recall_10       all 0.4093
recall_100      all 0.6920
recall_1000     all 0.7526
ndcg            all 0.6813
ndcg_cut_10     all 0.6670
ndcg_cut_100    all 0.6597
ndcg_cut_1000   all 0.6813

2¶

summary (passages-eval)

ndcg_cut_10     all 0.7113
ndcg_cut_100    all 0.6315
ndcg_cut_1000   all 0.6848

summary (trec_eval)

map             all 0.4866
bpref           all 0.5096
recip_rank      all 0.8507
P_10            all 0.5481
P_100           all 0.1606
P_1000          all 0.0202
recall_10       all 0.3944
recall_100      all 0.7183
recall_1000     all 0.8125
ndcg            all 0.7010
ndcg_cut_10     all 0.6531
ndcg_cut_100    all 0.6656
ndcg_cut_1000   all 0.7010

uogTrT20¶

summary (trec_eval)

map             all 0.3692
bpref           all 0.3826
recip_rank      all 0.8711
P_10            all 0.5667
P_100           all 0.1613
P_1000          all 0.0161
recall_10       all 0.2399
recall_100      all 0.5087
recall_1000     all 0.5087
ndcg            all 0.5295
ndcg_cut_10     all 0.5453
ndcg_cut_100    all 0.5296
ndcg_cut_1000   all 0.5295

bl_bcai_prox¶

Metadata | Participants| Input| Summary| Appendix

summary (trec_eval)

map             all 0.3766
bpref           all 0.4185
recip_rank      all 0.8164
P_10            all 0.5911
P_100           all 0.1889
P_1000          all 0.0189
recall_10       all 0.2670
recall_100      all 0.6065
recall_1000     all 0.6065
ndcg            all 0.5610
ndcg_cut_10     all 0.5364
ndcg_cut_100    all 0.5611
ndcg_cut_1000   all 0.5610

bm25_bert_token¶

summary (passages-eval)

ndcg_cut_10     all 0.4686
ndcg_cut_100    all 0.4555
ndcg_cut_1000   all 0.5525

summary (trec_eval)

map             all 0.2606
bpref           all 0.2651
recip_rank      all 0.6409
P_10            all 0.3352
P_100           all 0.1057
P_1000          all 0.0182
recall_10       all 0.2302
recall_100      all 0.5211
recall_1000     all 0.7625
ndcg            all 0.5125
ndcg_cut_10     all 0.3878
ndcg_cut_100    all 0.4330
ndcg_cut_1000   all 0.5125

bl_bcai_model1¶

Metadata | Participants| Input| Summary| Appendix

summary (trec_eval)

map             all 0.3774
bpref           all 0.4024
recip_rank      all 0.8358
P_10            all 0.5800
P_100           all 0.1867
P_1000          all 0.0187
recall_10       all 0.2480
recall_100      all 0.6056
recall_1000     all 0.6056
ndcg            all 0.5667
ndcg_cut_10     all 0.5378
ndcg_cut_100    all 0.5669
ndcg_cut_1000   all 0.5667

bl_bcai_multfld¶

Metadata | Participants| Input| Summary| Appendix

summary (trec_eval)

map             all 0.3829
bpref           all 0.4065
recip_rank      all 0.9195
P_10            all 0.5800
P_100           all 0.1816
P_1000          all 0.0182
recall_10       all 0.2689
recall_100      all 0.5921
recall_1000     all 0.5921
ndcg            all 0.5768
ndcg_cut_10     all 0.5629
ndcg_cut_100    all 0.5769
ndcg_cut_1000   all 0.5768

bcai_classic¶

Metadata | Participants| Input| Summary| Appendix

summary (trec_eval)

map             all 0.3906
bpref           all 0.4052
recip_rank      all 0.8648
P_10            all 0.5800
P_100           all 0.1900
P_1000          all 0.0190
recall_10       all 0.2548
recall_100      all 0.6098
recall_1000     all 0.6098
ndcg            all 0.5775
ndcg_cut_10     all 0.5557
ndcg_cut_100    all 0.5776
ndcg_cut_1000   all 0.5775

RMIT-Bart¶

summary (passages-eval)

ndcg_cut_10     all 0.7536
ndcg_cut_100    all 0.6800
ndcg_cut_1000   all 0.7190

summary (trec_eval)

map             all 0.5121
bpref           all 0.5143
recip_rank      all 0.8447
P_10            all 0.5944
P_100           all 0.1665
P_1000          all 0.0199
recall_10       all 0.4129
recall_100      all 0.7459
recall_1000     all 0.8093
ndcg            all 0.7137
ndcg_cut_10     all 0.6856
ndcg_cut_100    all 0.6875
ndcg_cut_1000   all 0.7137

indri-fdm¶

summary (passages-eval)

ndcg_cut_10     all 0.5003
ndcg_cut_100    all 0.5100
ndcg_cut_1000   all 0.6042

summary (trec_eval)

map             all 0.2989
bpref           all 0.2895
recip_rank      all 0.6498
P_10            all 0.3833
P_100           all 0.1231
P_1000          all 0.0203
recall_10       all 0.2454
recall_100      all 0.6213
recall_1000     all 0.8105
ndcg            all 0.5549
ndcg_cut_10     all 0.4222
ndcg_cut_100    all 0.4833
ndcg_cut_1000   all 0.5549

indri-sdm¶

summary (passages-eval)

ndcg_cut_10     all 0.4822
ndcg_cut_100    all 0.4943
ndcg_cut_1000   all 0.5900

summary (trec_eval)

map             all 0.2870
bpref           all 0.2851
recip_rank      all 0.6239
P_10            all 0.3685
P_100           all 0.1220
P_1000          all 0.0202
recall_10       all 0.2378
recall_100      all 0.6064
recall_1000     all 0.8074
ndcg            all 0.5410
ndcg_cut_10     all 0.4022
ndcg_cut_100    all 0.4667
ndcg_cut_1000   all 0.5410

terrier-InL2¶

summary (passages-eval)

ndcg_cut_10     all 0.4985
ndcg_cut_100    all 0.5300
ndcg_cut_1000   all 0.6146

summary (trec_eval)

map             all 0.3135
bpref           all 0.3086
recip_rank      all 0.6436
P_10            all 0.3537
P_100           all 0.1250
P_1000          all 0.0201
recall_10       all 0.2528
recall_100      all 0.6311
recall_1000     all 0.8083
ndcg            all 0.5642
ndcg_cut_10     all 0.4114
ndcg_cut_100    all 0.4977
ndcg_cut_1000   all 0.5642

terrier-BM25¶

summary (passages-eval)

ndcg_cut_10     all 0.4980
ndcg_cut_100    all 0.5124
ndcg_cut_1000   all 0.5998

summary (trec_eval)

map             all 0.3021
bpref           all 0.2995
recip_rank      all 0.6186
P_10            all 0.3667
P_100           all 0.1230
P_1000          all 0.0200
recall_10       all 0.2579
recall_100      all 0.5925
recall_1000     all 0.8035
ndcg            all 0.5525
ndcg_cut_10     all 0.4111
ndcg_cut_100    all 0.4795
ndcg_cut_1000   all 0.5525

indri-lmds¶

summary (passages-eval)

ndcg_cut_10     all 0.4912
ndcg_cut_100    all 0.5127
ndcg_cut_1000   all 0.6022

summary (trec_eval)

map             all 0.2961
bpref           all 0.2947
recip_rank      all 0.5866
P_10            all 0.3722
P_100           all 0.1239
P_1000          all 0.0203
recall_10       all 0.2531
recall_100      all 0.6258
recall_1000     all 0.8116
ndcg            all 0.5497
ndcg_cut_10     all 0.4056
ndcg_cut_100    all 0.4796
ndcg_cut_1000   all 0.5497

terrier-DPH¶

summary (passages-eval)

ndcg_cut_10     all 0.4671
ndcg_cut_100    all 0.4823
ndcg_cut_1000   all 0.5699

summary (trec_eval)

map             all 0.2758
bpref           all 0.2886
recip_rank      all 0.5667
P_10            all 0.3537
P_100           all 0.1189
P_1000          all 0.0193
recall_10       all 0.2322
recall_100      all 0.5778
recall_1000     all 0.7841
ndcg            all 0.5230
ndcg_cut_10     all 0.3819
ndcg_cut_100    all 0.4513
ndcg_cut_1000   all 0.5230

TF_IDF_d_2_t_50¶

summary (passages-eval)

ndcg_cut_10     all 0.4580
ndcg_cut_100    all 0.4920
ndcg_cut_1000   all 0.5908

summary (trec_eval)

map             all 0.2923
bpref           all 0.2827
recip_rank      all 0.5317
P_10            all 0.3278
P_100           all 0.1187
P_1000          all 0.0206
recall_10       all 0.2597
recall_100      all 0.6033
recall_1000     all 0.8103
ndcg            all 0.5414
ndcg_cut_10     all 0.3710
ndcg_cut_100    all 0.4608
ndcg_cut_1000   all 0.5414

DLH_d_5_t_25¶

summary (passages-eval)

ndcg_cut_10     all 0.4935
ndcg_cut_100    all 0.5219
ndcg_cut_1000   all 0.6225

summary (trec_eval)

map             all 0.3199
bpref           all 0.3166
recip_rank      all 0.5094
P_10            all 0.3667
P_100           all 0.1309
P_1000          all 0.0225
recall_10       all 0.2501
recall_100      all 0.6307
recall_1000     all 0.8413
ndcg            all 0.5593
ndcg_cut_10     all 0.3990
ndcg_cut_100    all 0.4768
ndcg_cut_1000   all 0.5593

indri-sdmf¶

summary (trec_eval)

map             all 0.3974
bpref           all 0.4186
recip_rank      all 0.8796
P_10            all 0.5889
P_100           all 0.2002
P_1000          all 0.0200
recall_10       all 0.2666
recall_100      all 0.6611
recall_1000     all 0.6611
ndcg            all 0.5967
ndcg_cut_10     all 0.5597
ndcg_cut_100    all 0.5968
ndcg_cut_1000   all 0.5967

RMIT_DPH¶

summary (trec_eval)

map             all 0.3879
bpref           all 0.4074
recip_rank      all 0.8278
P_10            all 0.5756
P_100           all 0.1951
P_1000          all 0.0195
recall_10       all 0.2471
recall_100      all 0.6218
recall_1000     all 0.6218
ndcg            all 0.5717
ndcg_cut_10     all 0.5280
ndcg_cut_100    all 0.5718
ndcg_cut_1000   all 0.5717

RMIT_DFRee¶

summary (trec_eval)

map             all 0.4087
bpref           all 0.4240
recip_rank      all 0.8756
P_10            all 0.5867
P_100           all 0.2033
P_1000          all 0.0203
recall_10       all 0.2658
recall_100      all 0.6656
recall_1000     all 0.6656
ndcg            all 0.5955
ndcg_cut_10     all 0.5431
ndcg_cut_100    all 0.5957
ndcg_cut_1000   all 0.5955

rmit_indri-sdm¶

Metadata | Participants| Input| Summary| Appendix

summary (trec_eval)

map             all 0.3780
bpref           all 0.4073
recip_rank      all 0.8470
P_10            all 0.5667
P_100           all 0.1944
P_1000          all 0.0194
recall_10       all 0.2557
recall_100      all 0.6424
recall_1000     all 0.6424
ndcg            all 0.5759
ndcg_cut_10     all 0.5328
ndcg_cut_100    all 0.5761
ndcg_cut_1000   all 0.5759

rmit_indri-fdm¶

Metadata | Participants| Input| Summary| Appendix

summary (trec_eval)

map             all 0.3859
bpref           all 0.4127
recip_rank      all 0.8481
P_10            all 0.5756
P_100           all 0.1964
P_1000          all 0.0196
recall_10       all 0.2538
recall_100      all 0.6523
recall_1000     all 0.6523
ndcg            all 0.5846
ndcg_cut_10     all 0.5416
ndcg_cut_100    all 0.5847
ndcg_cut_1000   all 0.5846

rterrier-dph¶

Metadata | Participants| Input| Summary| Appendix

summary (trec_eval)

map             all 0.3884
bpref           all 0.4156
recip_rank      all 0.8267
P_10            all 0.5644
P_100           all 0.1913
P_1000          all 0.0191
recall_10       all 0.2575
recall_100      all 0.6295
recall_1000     all 0.6295
ndcg            all 0.5741
ndcg_cut_10     all 0.5226
ndcg_cut_100    all 0.5742
ndcg_cut_1000   all 0.5741

rterrier-dph_sd¶

Metadata | Participants| Input| Summary| Appendix

summary (trec_eval)

map             all 0.3784
bpref           all 0.4040
recip_rank      all 0.8296
P_10            all 0.5467
P_100           all 0.1918
P_1000          all 0.0192
recall_10       all 0.2536
recall_100      all 0.6305
recall_1000     all 0.6305
ndcg            all 0.5676
ndcg_cut_10     all 0.5110
ndcg_cut_100    all 0.5677
ndcg_cut_1000   all 0.5676

rterrier-expC2¶

Metadata | Participants| Input| Summary| Appendix

summary (trec_eval)

map             all 0.3805
bpref           all 0.4076
recip_rank      all 0.8259
P_10            all 0.5933
P_100           all 0.1809
P_1000          all 0.0181
recall_10       all 0.2715
recall_100      all 0.6072
recall_1000     all 0.6072
ndcg            all 0.5689
ndcg_cut_10     all 0.5475
ndcg_cut_100    all 0.5690
ndcg_cut_1000   all 0.5689

rterrier-tfidf¶

Metadata | Participants| Input| Summary| Appendix

summary (trec_eval)

map             all 0.3734
bpref           all 0.4005
recip_rank      all 0.8241
P_10            all 0.5711
P_100           all 0.1796
P_1000          all 0.0180
recall_10       all 0.2675
recall_100      all 0.6075
recall_1000     all 0.6075
ndcg            all 0.5629
ndcg_cut_10     all 0.5317
ndcg_cut_100    all 0.5630
ndcg_cut_1000   all 0.5629

rterrier-tfidf2¶

Metadata | Participants| Input| Summary| Appendix

summary (trec_eval)

map             all 0.3607
bpref           all 0.3871
recip_rank      all 0.8407
P_10            all 0.5422
P_100           all 0.1749
P_1000          all 0.0175
recall_10       all 0.2489
recall_100      all 0.5923
recall_1000     all 0.5923
ndcg            all 0.5608
ndcg_cut_10     all 0.5219
ndcg_cut_100    all 0.5609
ndcg_cut_1000   all 0.5608

terrier-jskls¶

Metadata | Participants| Input| Summary| Appendix

summary (trec_eval)

map             all 0.4008
bpref           all 0.4237
recip_rank      all 0.8204
P_10            all 0.5800
P_100           all 0.1987
P_1000          all 0.0199
recall_10       all 0.2529
recall_100      all 0.6439
recall_1000     all 0.6439
ndcg            all 0.5851
ndcg_cut_10     all 0.5342
ndcg_cut_100    all 0.5852
ndcg_cut_1000   all 0.5851

bcai_class_pass¶

summary (passages-eval)

ndcg_cut_10     all 0.5600
ndcg_cut_100    all 0.5475
ndcg_cut_1000   all 0.6164

summary (trec_eval)

map             all 0.3374
bpref           all 0.3296
recip_rank      all 0.7115
P_10            all 0.4167
P_100           all 0.1304
P_1000          all 0.0185
recall_10       all 0.3122
recall_100      all 0.6382
recall_1000     all 0.7816
ndcg            all 0.5784
ndcg_cut_10     all 0.4825
ndcg_cut_100    all 0.5250
ndcg_cut_1000   all 0.5784

nlm-bm25-prf-1¶

summary (trec_eval)

map             all 0.2720
bpref           all 0.3010
recip_rank      all 0.8086
P_10            all 0.4756
P_100           all 0.1367
P_1000          all 0.0137
recall_10       all 0.2026
recall_100      all 0.4613
recall_1000     all 0.4613
ndcg            all 0.4678
ndcg_cut_10     all 0.4675
ndcg_cut_100    all 0.4679
ndcg_cut_1000   all 0.4678

bl_bcai_mdl1_vt¶

summary (passages-eval)

ndcg_cut_10     all 0.5667
ndcg_cut_100    all 0.5457
ndcg_cut_1000   all 0.6160

summary (trec_eval)

map             all 0.3380
bpref           all 0.3336
recip_rank      all 0.7037
P_10            all 0.4185
P_100           all 0.1281
P_1000          all 0.0185
recall_10       all 0.3128
recall_100      all 0.6372
recall_1000     all 0.7816
ndcg            all 0.5791
ndcg_cut_10     all 0.4885
ndcg_cut_100    all 0.5248
ndcg_cut_1000   all 0.5791

bl_bcai_mdl1_vs¶

summary (passages-eval)

ndcg_cut_10     all 0.5092
ndcg_cut_100    all 0.5293
ndcg_cut_1000   all 0.5984

summary (trec_eval)

map             all 0.3094
bpref           all 0.3058
recip_rank      all 0.6277
P_10            all 0.3741
P_100           all 0.1283
P_1000          all 0.0185
recall_10       all 0.2808
recall_100      all 0.6455
recall_1000     all 0.7816
ndcg            all 0.5508
ndcg_cut_10     all 0.4238
ndcg_cut_100    all 0.4990
ndcg_cut_1000   all 0.5508

nlm-bm25-prf-2¶

summary (trec_eval)

map             all 0.2912
bpref           all 0.3180
recip_rank      all 0.8099
P_10            all 0.4822
P_100           all 0.1458
P_1000          all 0.0146
recall_10       all 0.2099
recall_100      all 0.4911
recall_1000     all 0.4911
ndcg            all 0.4827
ndcg_cut_10     all 0.4705
ndcg_cut_100    all 0.4828
ndcg_cut_1000   all 0.4827

rindri-bm25¶

Metadata | Participants| Input| Summary| Appendix

summary (trec_eval)

map             all 0.3773
bpref           all 0.3990
recip_rank      all 0.8572
P_10            all 0.5600
P_100           all 0.1827
P_1000          all 0.0183
recall_10       all 0.2588
recall_100      all 0.6163
recall_1000     all 0.6163
ndcg            all 0.5749
ndcg_cut_10     all 0.5394
ndcg_cut_100    all 0.5750
ndcg_cut_1000   all 0.5749

bigIR-DCT-T5-F¶

summary (passages-eval)

ndcg_cut_10     all 0.7173
ndcg_cut_100    all 0.6704
ndcg_cut_1000   all 0.7304

summary (trec_eval)

map             all 0.5004
bpref           all 0.4999
recip_rank      all 0.8638
P_10            all 0.5593
P_100           all 0.1672
P_1000          all 0.0223
recall_10       all 0.3855
recall_100      all 0.7433
recall_1000     all 0.8362
ndcg            all 0.7215
ndcg_cut_10     all 0.6563
ndcg_cut_100    all 0.6832
ndcg_cut_1000   all 0.7215

bigIR-T5xp-T5-F¶

summary (passages-eval)

ndcg_cut_10     all 0.7034
ndcg_cut_100    all 0.6668
ndcg_cut_1000   all 0.7400

summary (trec_eval)

map             all 0.5001
bpref           all 0.5020
recip_rank      all 0.8579
P_10            all 0.5444
P_100           all 0.1674
P_1000          all 0.0235
recall_10       all 0.3772
recall_100      all 0.7524
recall_1000     all 0.8692
ndcg            all 0.7305
ndcg_cut_10     all 0.6466
ndcg_cut_100    all 0.6833
ndcg_cut_1000   all 0.7305

bigIR-T5-BERT-F¶

summary (passages-eval)

ndcg_cut_10     all 0.7073
ndcg_cut_100    all 0.6652
ndcg_cut_1000   all 0.7420

summary (trec_eval)

map             all 0.5101
bpref           all 0.5057
recip_rank      all 0.8478
P_10            all 0.5444
P_100           all 0.1685
P_1000          all 0.0235
recall_10       all 0.3882
recall_100      all 0.7485
recall_1000     all 0.8692
ndcg            all 0.7318
ndcg_cut_10     all 0.6504
ndcg_cut_100    all 0.6824
ndcg_cut_1000   all 0.7318

bigIR-BERT-R¶

summary (passages-eval)

ndcg_cut_10     all 0.7201
ndcg_cut_100    all 0.6371
ndcg_cut_1000   all 0.6732

summary (trec_eval)

map             all 0.4845
bpref           all 0.4832
recip_rank      all 0.8562
P_10            all 0.5556
P_100           all 0.1487
P_1000          all 0.0178
recall_10       all 0.3991
recall_100      all 0.6931
recall_1000     all 0.7526
ndcg            all 0.6777
ndcg_cut_10     all 0.6644
ndcg_cut_100    all 0.6543
ndcg_cut_1000   all 0.6777

bigIR-T5-R¶

summary (passages-eval)

ndcg_cut_10     all 0.7138
ndcg_cut_100    all 0.6414
ndcg_cut_1000   all 0.6741

summary (trec_eval)

map             all 0.4784
bpref           all 0.4743
recip_rank      all 0.8668
P_10            all 0.5537
P_100           all 0.1500
P_1000          all 0.0178
recall_10       all 0.3868
recall_100      all 0.6982
recall_1000     all 0.7526
ndcg            all 0.6767
ndcg_cut_10     all 0.6557
ndcg_cut_100    all 0.6563
ndcg_cut_1000   all 0.6767

uogTr31oR¶

summary (trec_eval)

map             all 0.3468
bpref           all 0.3699
recip_rank      all 0.8926
P_10            all 0.5622
P_100           all 0.1536
P_1000          all 0.0154
recall_10       all 0.2535
recall_100      all 0.5182
recall_1000     all 0.5182
ndcg            all 0.5283
ndcg_cut_10     all 0.5476
ndcg_cut_100    all 0.5284
ndcg_cut_1000   all 0.5283

bigIR-DH-T5-F¶

summary (trec_eval)

map             all 0.4177
bpref           all 0.4323
recip_rank      all 0.8902
P_10            all 0.6000
P_100           all 0.1984
P_1000          all 0.0198
recall_10       all 0.2630
recall_100      all 0.6422
recall_1000     all 0.6422
ndcg            all 0.6009
ndcg_cut_10     all 0.5734
ndcg_cut_100    all 0.6011
ndcg_cut_1000   all 0.6009

longformer_1¶

summary (trec_eval)

map             all 0.3503
bpref           all 0.3789
recip_rank      all 0.8889
P_10            all 0.5511
P_100           all 0.1809
P_1000          all 0.0181
recall_10       all 0.2449
recall_100      all 0.5985
recall_1000     all 0.5985
ndcg            all 0.5674
ndcg_cut_10     all 0.5520
ndcg_cut_100    all 0.5675
ndcg_cut_1000   all 0.5674

bigIR-DT-T5-F¶

summary (trec_eval)

map             all 0.3619
bpref           all 0.3788
recip_rank      all 0.9060
P_10            all 0.5400
P_100           all 0.1984
P_1000          all 0.0198
recall_10       all 0.2327
recall_100      all 0.6422
recall_1000     all 0.6422
ndcg            all 0.5848
ndcg_cut_10     all 0.5390
ndcg_cut_100    all 0.5849
ndcg_cut_1000   all 0.5848

bigIR-DH-T5-R¶

summary (trec_eval)

map             all 0.3842
bpref           all 0.4062
recip_rank      all 0.8889
P_10            all 0.5911
P_100           all 0.1809
P_1000          all 0.0181
recall_10       all 0.2563
recall_100      all 0.5985
recall_1000     all 0.5985
ndcg            all 0.5818
ndcg_cut_10     all 0.5846
ndcg_cut_100    all 0.5819
ndcg_cut_1000   all 0.5818

bigIR-DT-T5-R¶

summary (trec_eval)

map             all 0.3373
bpref           all 0.3564
recip_rank      all 0.9407
P_10            all 0.5311
P_100           all 0.1809
P_1000          all 0.0181
recall_10       all 0.2242
recall_100      all 0.5985
recall_1000     all 0.5985
ndcg            all 0.5681
ndcg_cut_10     all 0.5455
ndcg_cut_100    all 0.5683
ndcg_cut_1000   all 0.5681

bigIR-DTH-T5-R¶

summary (trec_eval)

map             all 0.3936
bpref           all 0.4169
recip_rank      all 0.9119
P_10            all 0.6022
P_100           all 0.1809
P_1000          all 0.0181
recall_10       all 0.2673
recall_100      all 0.5985
recall_1000     all 0.5985
ndcg            all 0.5905
ndcg_cut_10     all 0.6031
ndcg_cut_100    all 0.5906
ndcg_cut_1000   all 0.5905

bigIR-DTH-T5-F¶

summary (trec_eval)

map             all 0.4259
bpref           all 0.4429
recip_rank      all 0.8916
P_10            all 0.6067
P_100           all 0.1984
P_1000          all 0.0198
recall_10       all 0.2764
recall_100      all 0.6422
recall_1000     all 0.6422
ndcg            all 0.6102
ndcg_cut_10     all 0.5907
ndcg_cut_100    all 0.6104
ndcg_cut_1000   all 0.6102

bcai_bertl_pass¶

summary (passages-eval)

ndcg_cut_10     all 0.7151
ndcg_cut_100    all 0.6117
ndcg_cut_1000   all 0.7037

summary (trec_eval)

map             all 0.4641
bpref           all 0.4592
recip_rank      all 0.8453
P_10            all 0.5444
P_100           all 0.1313
P_1000          all 0.0204
recall_10       all 0.3755
recall_100      all 0.6376
recall_1000     all 0.8382
ndcg            all 0.6897
ndcg_cut_10     all 0.6453
ndcg_cut_100    all 0.6180
ndcg_cut_1000   all 0.6897

bcai_bertb_docv¶

Metadata | Participants| Input| Summary| Appendix

summary (trec_eval)

map             all 0.4308
bpref           all 0.4622
recip_rank      all 0.9259
P_10            all 0.6356
P_100           all 0.1969
P_1000          all 0.0197
recall_10       all 0.2967
recall_100      all 0.6255
recall_1000     all 0.6255
ndcg            all 0.6255
ndcg_cut_10     all 0.6278
ndcg_cut_100    all 0.6256
ndcg_cut_1000   all 0.6255