Runs - Common Core 2018¶

anserini_ax¶

Run ID: anserini_ax
Participant: Anserini
Track: Common Core
Year: 2018
Submission: 8/13/2018
Type: automatic
Task: main
MD5: e28756067300901c34c24702fd7d7fb4
Run description: This run uses BM25 as the base ranking model and then applies Axiomatic Reranking as the feedback method.

anserini_ax17¶

Run ID: anserini_ax17
Participant: Anserini
Track: Common Core
Year: 2018
Submission: 8/13/2018
Type: automatic
Task: main
MD5: c0540aaedc3ff565f2e67944b0dff600
Run description: This run uses BM25 as the base ranking model and then applies Axiomatic Reranking as the feedback method. When doing the reranking we use Core 17 New York Times collection as the expansion terms pool.

anserini_bm25¶

Run ID: anserini_bm25
Participant: Anserini
Track: Common Core
Year: 2018
Submission: 8/13/2018
Type: automatic
Task: main
MD5: aafbedfc2cf1113f6deab82a99ab595c
Run description: This run uses BM25 as the ranking model.

anserini_ql¶

Run ID: anserini_ql
Participant: Anserini
Track: Common Core
Year: 2018
Submission: 8/13/2018
Type: automatic
Task: main
MD5: 21d13e2529a1f181908befe7f6ffa73e
Run description: This run uses Dirichlet Language Model as the ranking model.

anserini_qlax¶

Run ID: anserini_qlax
Participant: Anserini
Track: Common Core
Year: 2018
Submission: 8/13/2018
Type: automatic
Task: main
MD5: ddf7cee69bb3d2282935f863450706ad
Run description: This run uses Dirichlet Language Model as the base ranking model and then applies Axiomatic Reranking as the feedback method.

anserini_qlax17¶

Run ID: anserini_qlax17
Participant: Anserini
Track: Common Core
Year: 2018
Submission: 8/13/2018
Type: automatic
Task: main
MD5: dcce134e902decbd6c6d0d028454544b
Run description: This run uses Dirichlet Language Model as the base ranking model and then applies Axiomatic Reranking as the feedback method. When doing the reranking we use Core 17 New York Times collection as the expansion terms pool.

anserini_qlrm3¶

Run ID: anserini_qlrm3
Participant: Anserini
Track: Common Core
Year: 2018
Submission: 8/13/2018
Type: automatic
Task: main
MD5: 1834a61218311eab214f61bef055c31e
Run description: This run uses Dirichlet Language Model as the base ranking model and then applies RM3 Reranking model.

anserini_qlsdm¶

Run ID: anserini_qlsdm
Participant: Anserini
Track: Common Core
Year: 2018
Submission: 8/13/2018
Type: automatic
Task: main
MD5: 43d3f80e993bd7b67a1aee392e1b173b
Run description: This run uses Dirichlet Language Model and we use the Sequential Dependency Model to model query topics. If the run or any of the information is incor

anserini_rm3¶

Run ID: anserini_rm3
Participant: Anserini
Track: Common Core
Year: 2018
Submission: 8/13/2018
Type: automatic
Task: main
MD5: e331704ee2f7a14b44639ab914757d28
Run description: This run uses BM25 as the base ranking model and then applies RM3 Reranking model.

anserini_sdm¶

Run ID: anserini_sdm
Participant: Anserini
Track: Common Core
Year: 2018
Submission: 8/13/2018
Type: automatic
Task: main
MD5: 07dd761131bcb60f58dc1aa4ce906c75
Run description: This run uses BM25 and we use the Sequential Dependency Model to model query topics.

b-BoE¶

Run ID: b-BoE
Participant: NOVASearch
Track: Common Core
Year: 2018
Submission: 8/21/2018
Type: relauto
Task: main
MD5: a34c6bed557505820e1bb6cc1651eada
Run description: This run uses the body field of the documents, in a bag of entities document representation. BM25, Language Model with Dirichlet Smoothing, Two-Staged Smoothing, TFIDF, TF, IDF and Coordinate Match retrieval models were used to generate features, that were used to rerank the documents using Ranklib's Coordinate Ascent

b-BoW¶

Run ID: b-BoW
Participant: NOVASearch
Track: Common Core
Year: 2018
Submission: 8/21/2018
Type: relauto
Task: main
MD5: b245b61c99479b82a9d4ec1d28284ad9
Run description: This run uses the document body field in a bag of words document representation. BM25, Language Model with Dirichlet Smoothing, Two-Staged Smoothing, TFIDF, TF, IDF and Coordinate Match retrieval models were used to generate features, that were used to rerank the documents using Ranklib's Coordinate Ascent

b-BoW-t-BoWBoE¶

Run ID: b-BoW-t-BoWBoE
Participant: NOVASearch
Track: Common Core
Year: 2018
Submission: 8/21/2018
Type: relauto
Task: main
MD5: 3cb6b610efdedab00d16f27f621cbb53
Run description: This run uses the document title field in a bag of words and bag of entities representation, and the body field in a bag of words document representation. BM25, Language Model with Dirichlet Smoothing, Two-Staged Smoothing, TFIDF, TF, IDF and Coordinate Match retrieval models were used to generate features, that were used to rerank the documents using Ranklib's Coordinate Ascent

b-BoWBoE-t-BoE¶

Run ID: b-BoWBoE-t-BoE
Participant: NOVASearch
Track: Common Core
Year: 2018
Submission: 8/21/2018
Type: relauto
Task: main
MD5: 977c66a5ee7497380e2dd70eaaa613fe
Run description: This run uses the document body field in a bag of words and bag of entities representation, and the title field in a bag of entities document representation. BM25, Language Model with Dirichlet Smoothing, Two-Staged Smoothing, TFIDF, TF, IDF and Coordinate Match retrieval models were used to generate features, that were used to rerank the documents using Ranklib's Coordinate Ascent. Unlike Run 6, here the entities weren't boosted.

BM25-b-BoW¶

Run ID: BM25-b-BoW
Participant: NOVASearch
Track: Common Core
Year: 2018
Submission: 8/21/2018
Type: relauto
Task: main
MD5: 71e271be970385c4045b829f095cbf6a
Run description: BM25 baseline over the document body field.

bt-BoE¶

Run ID: bt-BoE
Participant: NOVASearch
Track: Common Core
Year: 2018
Submission: 8/21/2018
Type: relauto
Task: main
MD5: 7a3ed3e12512f231897e36d0c0d23026
Run description: This run uses both title and body fields of the documents, bag of entities document representation. BM25, Language Model with Dirichlet Smoothing, Two-Staged Smoothing, TFIDF, TF, IDF and Coordinate Match retrieval models were used to generate features, that were used to rerank the documents using Ranklib's Coordinate Ascent

bt-BoW¶

Run ID: bt-BoW
Participant: NOVASearch
Track: Common Core
Year: 2018
Submission: 8/21/2018
Type: relauto
Task: main
MD5: 4a8b15be096360bffdcf7703776ddd24
Run description: This run uses both title and body fields of the documents, in a bag of words document representation. BM25, Language Model with Dirichlet Smoothing, Two-Staged Smoothing, TFIDF, TF, IDF and Coordinate Match retrieval models were used to generate features, that were used to rerank the documents using Ranklib's Coordinate Ascent

bt-BoWBoE¶

Run ID: bt-BoWBoE
Participant: NOVASearch
Track: Common Core
Year: 2018
Submission: 8/21/2018
Type: relauto
Task: main
MD5: 3125fb3f2eeb2e0c1ff89cd43607d76b
Run description: This run uses both title and body fields of the documents, in a bag of words and bag of entities document representation. BM25, Language Model with Dirichlet Smoothing, Two-Staged Smoothing, TFIDF, TF, IDF and Coordinate Match retrieval models were used to generate features, that were used to rerank the documents using Ranklib's Coordinate Ascent

eb-boost¶

Run ID: eb-boost
Participant: NOVASearch
Track: Common Core
Year: 2018
Submission: 8/21/2018
Type: relauto
Task: main
MD5: 7e0db6a049debf9c27033cd75b0e8fa6
Run description: This run uses the document body field in a bag of words and bag of entities representation, and the title field in a bag of entities document representation. BM25, Language Model with Dirichlet Smoothing, Two-Staged Smoothing, TFIDF, TF, IDF and Coordinate Match retrieval models were used to generate features, that were used to rerank the documents using Ranklib's Coordinate Ascent

feup-run1¶

Run ID: feup-run1
Participant: FEUP
Track: Common Core
Year: 2018
Submission: 7/31/2018
Type: automatic
Task: main
MD5: 60f800cb6e2640871d8ea1f639ef7800
Run description: Hypergraph-of-entity with only term nodes and document hyperedges.

feup-run2¶

Run ID: feup-run2
Participant: FEUP
Track: Common Core
Year: 2018
Submission: 7/31/2018
Type: automatic
Task: main
MD5: 94d54394dbf142ed9487332d6c021b94
Run description: Hypergraph-of-entity with term and entity nodes, and document, contained_in and related_to hyperedges, using external knowledge from DBpedia over named entities extracted from the first three paragraphs.

feup-run3¶

Run ID: feup-run3
Participant: FEUP
Track: Common Core
Year: 2018
Submission: 8/13/2018
Type: automatic
Task: main
MD5: 3c409b0afb066e883a393459f8653f10
Run description: Hypergraph-of-entity with only term nodes and document hyperedges, reranked for diversity using a document profile based on the following features: Keywords, Named Entities, Sentiment Analysis, Reading Complexity, Emotion Categories.

feup-run4¶

Run ID: feup-run4
Participant: FEUP
Track: Common Core
Year: 2018
Submission: 8/16/2018
Type: automatic
Task: main
MD5: 483be35ad65b61e2218704c4bd8a963f
Run description: Hypergraph-of-entity with term and entity nodes, and document, contained_in and related_to hyperedges, using external knowledge from DBpedia over named entities extracted from the first three paragraphs. Results are reranked for diversity using a document profile based on the following features: Sentiment Analysis, Reading Complexity. Limiting to 100 queries per topic.

feup-run5¶

Run ID: feup-run5
Participant: FEUP
Track: Common Core
Year: 2018
Submission: 8/13/2018
Type: automatic
Task: main
MD5: 1dad513190cb215414eb067af1e3a3a6
Run description: Hypergraph-of-entity with only term nodes and document hyperedges, reranked for strong diversity using a document profile based on the following features: Keywords, Named Entities, Sentiment Analysis, Reading Complexity, Emotion Categories.

feup-run6¶

Run ID: feup-run6
Participant: FEUP
Track: Common Core
Year: 2018
Submission: 8/16/2018
Type: automatic
Task: main
MD5: 96cc9297e0cf26f0332aeaaebf91b4eb
Run description: Hypergraph-of-entity with term and entity nodes, and document, contained_in and related_to hyperedges, using external knowledge from DBpedia over named entities extracted from the first three paragraphs. Results are reranked for diversity using a document profile based on the following features: Sentiment Analysis, Reading Complexity, Emotion Categories. Limiting to 100 queries per topic.

feup-run7¶

Run ID: feup-run7
Participant: FEUP
Track: Common Core
Year: 2018
Submission: 8/16/2018
Type: automatic
Task: main
MD5: dd38d543cdf038069ace1ede3bb68978
Run description: Hypergraph-of-entity with only term nodes and document hyperedges, reranked for diversity using a document profile based on the following features: Sentiment Analysis, Reading Complexity.

feup-run8¶

Run ID: feup-run8
Participant: FEUP
Track: Common Core
Year: 2018
Submission: 8/16/2018
Type: automatic
Task: main
MD5: 3c3b86f8d53f4809f3f7ada39f8b3ef1
Run description: Hypergraph-of-entity with only term nodes and document hyperedges, reranked for diversity using a document profile based on the following features: Named Entities, Sentiment Analysis, Reading Complexity, Emotion Categories. Including all possible values and weights of the features.

h2oloo_e7ax0.6¶

Run ID: h2oloo_e7ax0.6
Participant: h2oloo
Track: Common Core
Year: 2018
Submission: 8/20/2018
Type: relauto
Task: main
MD5: b887372d196d65ee9a7e4c09e6fda75b
Run description: Use 7 classifiers to rerank first 25 topics, BM25+axiom to run the rest 25 topics.

h2oloo_e7ax0.7¶

Run ID: h2oloo_e7ax0.7
Participant: h2oloo
Track: Common Core
Year: 2018
Submission: 8/20/2018
Type: relauto
Task: main
MD5: 3ebf4cf659162b421ce7fcc49e787fef
Run description: Use 7 classifiers to rerank first 25 topics, BM25+axiom to run the rest 25 topics.

h2oloo_e7rm30.6¶

Run ID: h2oloo_e7rm30.6
Participant: h2oloo
Track: Common Core
Year: 2018
Submission: 8/20/2018
Type: relauto
Task: main
MD5: 3dea04bb4f81f2921a1976d07de08bbf
Run description: Use 7 classifiers to run on the first 25 topics, BM25+RM3 on the rest 25 topics

h2oloo_e7rm30.7¶

Run ID: h2oloo_e7rm30.7
Participant: h2oloo
Track: Common Core
Year: 2018
Submission: 8/20/2018
Type: relauto
Task: main
MD5: 5838b2fc780b922cef97fb7aceb15003
Run description: Use 7 classifiers to run on the first 25 topics, BM25+RM3 on the rest 25 topics

h2oloo_enax0.6¶

Run ID: h2oloo_enax0.6
Participant: h2oloo
Track: Common Core
Year: 2018
Submission: 8/16/2018
Type: relauto
Task: main
MD5: 218f71e845a440fda659cda7f6eb5836
Run description: We used documents for same 25 topic in Robust04, Robust05, and Core17 to train classifiers.

h2oloo_enax0.7¶

Run ID: h2oloo_enax0.7
Participant: h2oloo
Track: Common Core
Year: 2018
Submission: 8/16/2018
Type: relauto
Task: main
MD5: 732acacab7452f9578886efec1db10ea
Run description: We used documents for same 25 topics in Robust04, Robust05, and Core17 to train classifier and take the ensemble

h2oloo_enrm30.6¶

Run ID: h2oloo_enrm30.6
Participant: h2oloo
Track: Common Core
Year: 2018
Submission: 8/20/2018
Type: relauto
Task: main
MD5: f48cd8f2ca3f412d047d359236a32378
Run description: Train 3 classifiers and ensemble on first 25 topics. BM25+RM3 for the rest 25 topics

h2oloo_LR2_rm3¶

Run ID: h2oloo_LR2_rm3
Participant: h2oloo
Track: Common Core
Year: 2018
Submission: 8/16/2018
Type: relauto
Task: main
MD5: 0f321ade7a3af86f3c5e4e1f64ee4aa9
Run description: This single model was trained on 25 topics in Robust04, Robust05, Core17 with LR and rerank on BM25+RM3.

h2oloo_LR2AX0.6¶

Run ID: h2oloo_LR2AX0.6
Participant: h2oloo
Track: Common Core
Year: 2018
Submission: 8/16/2018
Type: relauto
Task: main
MD5: 169adb32e5aadd006b0c1a2612dd0063
Run description: This submission contains the single model that was trained from Robust04, Robust05, Core17 over 25 topics, for the left 25 new topics, the results are just BM25+AX.

h2oloo_LRax0.6¶

Run ID: h2oloo_LRax0.6
Participant: h2oloo
Track: Common Core
Year: 2018
Submission: 8/20/2018
Type: relauto
Task: main
MD5: 9bd652f7b7f1a485fb49cd560bf7ae7a
Run description: Use Logistic Regression to train on the same topics in Robust04, Robust05, and Core17. Use BM25+axiom in the other topics.

jarir_cb_ind¶

Run ID: jarir_cb_ind
Participant: JARIR
Track: Common Core
Year: 2018
Submission: 8/20/2018
Type: automatic
Task: main
MD5: f73b65e0c8224d16a42651c658024d36
Run description: A query expansion method based on natural language processing and word embedding. Word2Vec's Cbow model was trained on the Washington Post Corpus with a vector size equal to 300. Then we selected expansion terms that are similar to the individual query terms. In this run, we used only the titles of queries. The Okapi BM25 ranking model is applied with default parameters.

jarir_cb_re¶

Run ID: jarir_cb_re
Participant: JARIR
Track: Common Core
Year: 2018
Submission: 8/19/2018
Type: automatic
Task: main
MD5: 0e856c66133cd0f0a49e4024b1ef4144
Run description: A query expansion method based on natural language processing and word embedding. Word2Vec's Cbow model was trained on the Washington Post Corpus with a vector size equal to 300. Then we selected expansion terms that are similar to the entire query. The new query terms are reweighed. In this run, we used only the titles of queries. The Okapi BM25 ranking model is applied with default parameters.

jarir_cbow¶

Run ID: jarir_cbow
Participant: JARIR
Track: Common Core
Year: 2018
Submission: 8/19/2018
Type: automatic
Task: main
MD5: 772e683585e6c0cc71df63d1aefaeb53
Run description: A query expansion method based on natural language processing and word embedding. Word2Vec's Cbow model was trained on the Washington Post Corpus with a vector size equal to 300. Then we selected expansion terms that are similar to the entire query. In this run, we used only the titles of queries. The Okapi BM25 ranking model is applied with default parameters.

jarir_sg_ind¶

Run ID: jarir_sg_ind
Participant: JARIR
Track: Common Core
Year: 2018
Submission: 8/20/2018
Type: automatic
Task: main
MD5: 233176c23dc6a448f9d1726f515696c6
Run description: A query expansion method based on natural language processing and word embedding. Word2Vec's Skip-Gram model was trained on the Washington Post Corpus with a vector size equal to 300. Then we selected expansion terms that are similar to the individual query terms. In this run, we used only the titles of queries. The Okapi BM25 ranking model is applied with default parameters.

jarir_sg_re¶

Run ID: jarir_sg_re
Participant: JARIR
Track: Common Core
Year: 2018
Submission: 8/19/2018
Type: automatic
Task: main
MD5: 0225a6bb265876deb5fd89fb54b69810
Run description: A query expansion method based on natural language processing and word embedding. Word2Vec's Skip-Gram model was trained on the Washington Post Corpus with a vector size equal to 300. Then we selected expansion terms that are similar to the entire query. The new query terms are reweighed. In this run, we used only the titles of queries. The Okapi BM25 ranking model is applied with default parameters.

jarir_skipgram¶

Run ID: jarir_skipgram
Participant: JARIR
Track: Common Core
Year: 2018
Submission: 8/20/2018
Type: automatic
Task: main
MD5: 9a77a09ec574aae7a59fe462ef0faade
Run description: A query expansion method based on natural language processing and word embedding. Word2Vec's Skip-Gram model was trained on the Washington Post Corpus with a vector size equal to 300. Then we selected expansion terms that are similar to the entire query. In this run, we used only the titles of queries. The Okapi BM25 ranking model is applied with default parameters.

RMITEXTGIGADA5¶

Run ID: RMITEXTGIGADA5
Participant: RMIT
Track: Common Core
Year: 2018
Submission: 8/20/2018
Type: relauto
Task: main
MD5: cf019907e4e7b1ccd747f8d8f5bc2b7a
Run description: External query expansion using the Gigaword + Tipster corpus.

RMITFDA4¶

Run ID: RMITFDA4
Participant: RMIT
Track: Common Core
Year: 2018
Submission: 8/20/2018
Type: relauto
Task: main
MD5: 13d0e12d6815dd90169a0af37ac0779b
Run description: Title query runs on Indri and Terrier with query expansion, and external expansion runs from Gigaword and Tipster fused into a single run using RRF. Query expansion parameters taken from NYT judgments (collection-wide, not per-topic). A baseline for how query variations compare to titles.

RMITUQVBestDM2¶

Run ID: RMITUQVBestDM2
Participant: RMIT
Track: Common Core
Year: 2018
Submission: 8/20/2018
Type: manual
Task: main
MD5: 311e877001c11893f700fd8bace81a21
Run description: Authors formed a judgment pool to the top-5 of RMITUQVDBFDM3 and a title query language model run. These judgments were used to select the best title-only query without fusion using the same systems as in RMITUQVDBFDM3.

RMITUQVDBFDM3¶

Run ID: RMITUQVDBFDM3
Participant: RMIT
Track: Common Core
Year: 2018
Submission: 8/20/2018
Type: manual
Task: main
MD5: dfc3526dae0803858c37e708279470c9
Run description: Query variations for the original TREC topics were generated by the authors. All query variations were run on systems with parameters shown to be effective on NYT using Indri and Terrier with query expansion, as well as external corpus query expansion using Gigaword and Gigaword+Tipster. This was fused to make a single run using RRF k=60. Documents found with duplicates were included in-place in the ranked list.

RMITUQVDBFNZDM1¶

Run ID: RMITUQVDBFNZDM1
Participant: RMIT
Track: Common Core
Year: 2018
Submission: 8/20/2018
Type: manual
Task: main
MD5: d7c2e97f39eca796c0e5ddcd392b11e9
Run description: Authors formed a judgment pool to the top-5 of RMITUQVDBFDM3 and a title query language model run. These judgments were used to remove any query variations with a zero score prior to rank fusion. The reduced set of queries using RMITUQVDBFDM3's approach, where documents found to have duplicates were included in-place in the ranked list.

sab18aqv45nytE1¶

Results | Participants | Input | Summary | Appendix

Run ID: sab18aqv45nytE1
Participant: Sabir
Track: Common Core
Year: 2018
Submission: 8/21/2018
Type: relauto
Task: main
MD5: c87e39450ec777b17877dd3169fb35a9
Run description: expansion by all terms occurring in known relevant documents of aquaint,v45,nyt.

sab18aqv45nytO1¶

Results | Participants | Input | Summary | Appendix

Run ID: sab18aqv45nytO1
Participant: Sabir
Track: Common Core
Year: 2018
Submission: 8/22/2018
Type: relauto
Task: main
MD5: a7bcdd494ea3f2f8e24f1129a98ba534
Run description: optimization with 250 terms occurring in known relevant documents of aquaint,v45,nyt.

sab18coreA¶

Results | Participants | Input | Summary | Appendix

Run ID: sab18coreA
Participant: Sabir
Track: Common Core
Year: 2018
Submission: 8/21/2018
Type: automatic
Task: main
MD5: 8539c640d3b7c97787421c36823c1453
Run description: Standard SMART Lnu.ltu weighted vector run

sab18coreE1¶

Results | Participants | Input | Summary | Appendix

Run ID: sab18coreE1
Participant: Sabir
Track: Common Core
Year: 2018
Submission: 8/21/2018
Type: relauto
Task: main
MD5: 28164589ff674db7da181e490ae56acf
Run description: Massive Rocchio feedback expansion by all terms occurring in known relevant documents of v45. Exact duplicate of run done in Robust and Core 2017.

sab18coreO1¶

Results | Participants | Input | Summary | Appendix

Run ID: sab18coreO1
Participant: Sabir
Track: Common Core
Year: 2018
Submission: 8/21/2018
Type: relauto
Task: main
MD5: 805ef351d44ef2b5e39da4c154aa0575
Run description: optimization of weights for 250 terms occurring in known relevant documents of v45. Exact duplicate of run done in Robust and Core 2017.

sab18nyt_E1¶

Results | Participants | Input | Summary | Appendix

Run ID: sab18nyt_E1
Participant: Sabir
Track: Common Core
Year: 2018
Submission: 8/21/2018
Type: relauto
Task: main
MD5: 50f72fda273f182167a7bd7a9bd58f99
Run description: massive expansion run with all terms occurring in known relevant documents of nyt (Core 2017).

sab18nyt_O1¶

Results | Participants | Input | Summary | Appendix

Run ID: sab18nyt_O1
Participant: Sabir
Track: Common Core
Year: 2018
Submission: 8/21/2018
Type: relauto
Task: main
MD5: d753057baaec3b2f053eb13ac34314c2
Run description: optimization of 250 terms occurring in known relevant documents of nyt (Core 2017).

sab18nyt_O1.50¶

Results | Participants | Input | Summary | Appendix

Run ID: sab18nyt_O1.50
Participant: Sabir
Track: Common Core
Year: 2018
Submission: 8/21/2018
Type: relauto
Task: main
MD5: d4ac31ef4d75ee09ece87d96cd8093a6
Run description: optimization of 50 terms occurring in known relevant documents of nyt (Core 2017).

sabaqv45nytO150¶

Results | Participants | Input | Summary | Appendix

Run ID: sabaqv45nytO150
Participant: Sabir
Track: Common Core
Year: 2018
Submission: 8/21/2018
Type: relauto
Task: main
MD5: 562abc59c6922686335698e3163da1b8
Run description: optimization of 50 terms occurring in known relevant documents of aquaint,v45,nyt.

sabopt50v45¶

Results | Participants | Input | Summary | Appendix

Run ID: sabopt50v45
Participant: Sabir
Track: Common Core
Year: 2018
Submission: 8/21/2018
Type: relauto
Task: main
MD5: 689680f1ae9029774f787fed3c8c71d1
Run description: optimization of weights for 250 terms occurring in known relevant documents of v45. Exact duplicate of run done in Robust and Core 2017.

t-BoE¶

Run ID: t-BoE
Participant: NOVASearch
Track: Common Core
Year: 2018
Submission: 8/21/2018
Type: relauto
Task: main
MD5: d7e9386dcecac9715a34282fec36df20
Run description: This run uses the title field of the document, in a bag of entities document representation. BM25, Language Model with Dirichlet Smoothing, Two-Staged Smoothing, TFIDF, TF, IDF and Coordinate Match retrieval models were used to generate features, that were used to rerank the documents using Ranklib's Coordinate Ascent

umass_bsdm¶

Run ID: umass_bsdm
Participant: UMass
Track: Common Core
Year: 2018
Submission: 8/21/2018
Type: automatic
Task: main
MD5: 4358af5872163b294dc5bbc538e30e16
Run description: BSDM model is a Sequential Dependence Model built on BM25 scoring.

umass_cbsdm¶

Run ID: umass_cbsdm
Participant: UMass
Track: Common Core
Year: 2018
Submission: 8/21/2018
Type: automatic
Task: main
MD5: 575442362009dd51e06d5a1608cdfbac
Run description: CBSDM model is a Sequential Dependence Model built on BM25 scoring with cheaper (more-efficient) feature functions than the traditional model and smart stopword removal (in unigrams and unordered features) using the inquery stopword list.

umass_ql¶

Run ID: umass_ql
Participant: UMass
Track: Common Core
Year: 2018
Submission: 8/21/2018
Type: automatic
Task: main
MD5: caaaa34b0f1eab84ca11c2daf1c79155
Run description: umass_ql is a straight-up implementation of the Query Likelihood Model.

umass_sdm¶

Run ID: umass_sdm
Participant: UMass
Track: Common Core
Year: 2018
Submission: 8/21/2018
Type: automatic
Task: main
MD5: e116906fb5f3b3066f2d822957a73784
Run description: umass_sdm is a straight-up implementation of the Sequential Dependence Model.

UWaterMDS_DS_A¶

Run ID: UWaterMDS_DS_A
Participant: UWaterlooMDS
Track: Common Core
Year: 2018
Submission: 8/20/2018
Type: manual
Task: main
MD5: 01033c97a160fc732c02261fc28e51c7
Run description: Manual search and judgments are initially used to find dozens of relevant documents. Then dynamic sampling is applied to select 300 sampled documents for human assessors to judge. The final machine learner ranks the judged relevant documents according to their relevance scores from high to low.

UWaterMDS_DS_B¶

Run ID: UWaterMDS_DS_B
Participant: UWaterlooMDS
Track: Common Core
Year: 2018
Submission: 8/20/2018
Type: manual
Task: main
MD5: 5a5b682b510ce46a755ef7a85a38e296
Run description: Manual search and judgments are initially used to find dozens of relevant documents. Then dynamic sampling is applied to select 300 sampled documents for human assessors to judge. The final machine learner ranks the judged relevant documents according to their relevance scores from low to high.

UWaterMDS_Rank¶

Run ID: UWaterMDS_Rank
Participant: UWaterlooMDS
Track: Common Core
Year: 2018
Submission: 8/21/2018
Type: manual
Task: main
MD5: 909920b948fe0dc1c1c182d0207b190e
Run description: Build a machine learner model from the judgments of ad-hoc search and dynamic sampling. Rank all the documents using the model according to their relevance scores.

UWaterMDS_SEQ¶

Run ID: UWaterMDS_SEQ
Participant: UWaterlooMDS
Track: Common Core
Year: 2018
Submission: 8/21/2018
Type: manual
Task: main
MD5: 4e7203f54535c82da20b65138e5043f4
Run description: The run is generated according to the order of our judgments. The earlier judged documents are ranked prior than the other documents.

uwmrg¶

Run ID: uwmrg
Participant: MRG_UWaterloo
Track: Common Core
Year: 2018
Submission: 8/12/2018
Type: automatic
Task: main
MD5: a4ddddaae38b61f73bdd4caf0d70bb94
Run description: Logistic Regression scoring, based on (up to) 11 pseudo-relevant documents: Google SERP page and top-10 documents linked-to by SERP page. For each topic, the pseudo-relevant documents for the other 49 topics were used as pseudo-nonrelevant documents.

uwmrgx¶

Run ID: uwmrgx
Participant: MRG_UWaterloo
Track: Common Core
Year: 2018
Submission: 8/12/2018
Type: automatic
Task: main
MD5: d1962acdcc2fb51a121493115703048e
Run description: Logistic Regression scoring, based on 1 pseudo-relevant document: Google SERP containing ten blue links and summaries. For each topic, the pseudo-relevant documents for the other 49 topics were used as pseudo-nonrelevant documents. This run did not use the linked-to documents, just the Google SERP.

webis-argument¶

Run ID: webis-argument
Participant: Webis
Track: Common Core
Year: 2018
Submission: 8/21/2018
Type: automatic
Task: main
MD5: a5e647d9be8cf65f339df7b8886ca090
Run description: - argumentation re-ranking with a higher priority for the queries demanding argumentation in arguments - argumentation extractor - textrank summarization

webis-baseline¶

Run ID: webis-baseline
Participant: Webis
Track: Common Core
Year: 2018
Submission: 8/21/2018
Type: automatic
Task: main
MD5: f1cc46b8a406b7ded4fb35d8fc5e2330
Run description: - ranking with no argumentation priority - multi-field ranking

webis-bm25f¶

Run ID: webis-bm25f
Participant: Webis
Track: Common Core
Year: 2018
Submission: 8/21/2018
Type: automatic
Task: main
MD5: 028ae72a093b56af56052ae444635dcf
Run description: - argumentative re-ranking with a low argumentation priority - multi-field ranking function - textrank