Runs - News 2021¶

300K_ENT_PH¶

Run ID: 300K_ENT_PH
Participant: L3i_Rochelle
Track: News
Year: 2021
Submission: 6/4/2021
Type: auto
Task: background
MD5: 3b836de63651e339a3cde706499b709c
Run description: 300Keywords+Cosine(Entity,Phrases,Keywords)

300K_ENT_PH_DN¶

Run ID: 300K_ENT_PH_DN
Participant: L3i_Rochelle
Track: News
Year: 2021
Submission: 6/5/2021
Type: auto
Task: background
MD5: bc23602c4006c79b0576da80745f9664
Run description: 300Keywords+Filter+Cosine(2xEntity+Event,Phrases,Keywords,Description,Narrative)

base¶

Run ID: base
Participant: IR-Cologne
Track: News
Year: 2021
Submission: 6/4/2021
Type: auto
Task: background
MD5: bc2bc8f6e743920aebb4c9b872c631c8
Run description: Baseline run with BM25 ranking.

bm25_sub_0.25¶

Run ID: bm25_sub_0.25
Participant: IR-Cologne
Track: News
Year: 2021
Submission: 6/4/2021
Type: auto
Task: bgsubs
MD5: f59d3a45463f6a286531aa1678fda4f5
Run description: Ranking depending on entities and relations and then reranking with weighted BM25 score (0.25) on subtopic text.

bm25_sub_0.5¶

Run ID: bm25_sub_0.5
Participant: IR-Cologne
Track: News
Year: 2021
Submission: 6/4/2021
Type: auto
Task: bgsubs
MD5: 0db845f43e14d56abd7435fae2564bf1
Run description: Ranking depending on entities and relations and then reranking with weighted BM25 score (0.5) on subtopic text.

bm25_sub_1.0¶

Run ID: bm25_sub_1.0
Participant: IR-Cologne
Track: News
Year: 2021
Submission: 6/4/2021
Type: auto
Task: bgsubs
MD5: df3c94c11e1a349dd94d50855a49d940
Run description: Ranking depending on entities and relations and then reranking with weighted BM25 score (1.0) on subtopic text.

bm25_sub_2¶

Run ID: bm25_sub_2
Participant: IR-Cologne
Track: News
Year: 2021
Submission: 6/4/2021
Type: auto
Task: bgsubs
MD5: 289727aaf0b9272cd9cdedb91742b260
Run description: Ranking depending on entities and relations and then reranking with weighted BM25 score (2) on subtopic text.

bm25_sub_4¶

Run ID: bm25_sub_4
Participant: IR-Cologne
Track: News
Year: 2021
Submission: 6/4/2021
Type: auto
Task: bgsubs
MD5: 8d8bc1848c155a156ca276c9f38d483c
Run description: Ranking depending on entities and relations and then reranking with weighted BM25 score (4) on subtopic text.

FUH_News_BG¶

Run ID: FUH_News_BG
Participant: FUH
Track: News
Year: 2021
Submission: 6/1/2021
Type: auto
Task: background
MD5: 017c47094b1fba449a7d7bfb2d2c3a56
Run description: Our method is based on a algorithm chain. Firstly, topic modeling is done using word stem analysis and TFIDF algorithms. Secondly, we calculate Graph Codes (a 2D projection of feature graphs with a model for metrics). Thirdly, we apply a metric calculation based on similarity and recommendations and finally, we calculate the background linking results as a ranked list.

FUH_News_ST¶

Run ID: FUH_News_ST
Participant: FUH
Track: News
Year: 2021
Submission: 6/2/2021
Type: auto
Task: bgsubs
MD5: fea0a2846d9719ce84c23c27190cfe65
Run description: Our method is based on a algorithm chain. Firstly, topic modeling is done using word stem analysis and TFIDF algorithms. Secondly, we calculate Graph Codes (a 2D projection of feature graphs with a model for metrics). Thirdly, we apply a metric calculation based on similarity and recommendations and finally, we calculate the background linking results as a ranked list.

KWVec¶

Run ID: KWVec
Participant: L3i_Rochelle
Track: News
Year: 2021
Submission: 6/5/2021
Type: auto
Task: background
MD5: 8cbb5534689bd558c4c15309c2a6f6a1
Run description: Use of Yake for keywords extraction. Use of SBERT dense vectors for creating document embeddings. Use of KNN and Cosine similarity for finding the most relevant documents. Editorial articles are kept but scored lower. Documents with titles similar to the topic are scored lower.

KWVec_sub¶

Run ID: KWVec_sub
Participant: L3i_Rochelle
Track: News
Year: 2021
Submission: 6/5/2021
Type: auto
Task: bgsubs
MD5: e896af07f5f48ef42466be50d3f4b904
Run description: Use of Yake for keywords extraction. Use of SBERT dense vectors for creating document embeddings. Use of KNN and Cosine similarity for finding the most relevant documents. Editorial articles are kept but scored lower. Documents with titles similar to the topic are scored lower. Documents are reranked based on the cosine similarity between the subtopic and the document body.

Lambda¶

Run ID: Lambda
Participant: L3i_Rochelle
Track: News
Year: 2021
Submission: 6/5/2021
Type: auto
Task: background
MD5: a7c1d33f51c16ac49c21afacc31ec04f
Run description: Multiple single queries are done to Elasticsearch based on keywords (YAKE) and embeddings (SBERT). E.g. title, body, lead. The scores from the queries are merged using a polynomial that was optimized using a Bayesian Optimization. The objective function was a weighted harmonic mean between the median NDCG@10 and quantiles. Optimized on 2018, 2019 and 2020; some extra documents were annotated regarding similar titles.

Lambda_narr¶

Run ID: Lambda_narr
Participant: L3i_Rochelle
Track: News
Year: 2021
Submission: 6/5/2021
Type: auto
Task: background
MD5: 0c747896592fb599086d74aa3e3061df
Run description: Multiple single queries are done to Elasticsearch based on keywords (YAKE) and embeddings (SBERT). E.g. title, body, lead. The scores from the queries are merged using a polynomial that was optimized using a Bayesian Optimization. The objective function was a weighted harmonic mean between the median NDCG@10 and quantiles. Optimized on 2018, 2019 and 2020; some extra documents were annotated regarding similar titles. Documents are reranked on the cosine similarity between the field narrative and the document's body.

Lambda_sub¶

Run ID: Lambda_sub
Participant: L3i_Rochelle
Track: News
Year: 2021
Submission: 6/5/2021
Type: auto
Task: bgsubs
MD5: 62515885fab269e535cb1dd5ab2105e6
Run description: Multiple single queries are done to Elasticsearch based on keywords (YAKE) and embeddings (SBERT). E.g. title, body, lead. The scores from the queries are merged using a polynomial that was optimized using a Bayesian Optimization. The objective function was a weighted harmonic mean between the median NDCG@10 and quantiles. Optimized on 2018, 2019 and 2020; some extra documents were annotated regarding similar titles. Documents are reranked on the cosine similarity between the subtopic and the document's body.

midd-direct¶

Run ID: midd-direct
Participant: middlebury
Track: News
Year: 2021
Submission: 6/4/2021
Type: auto
Task: background
MD5: 6c5099bc05ce1729d44175409125cce5
Run description: Coordinate Ascent Ranking; Local TF-IDF Similarities; Temporal features; Kicker Similarities; Clickbait features.

midd-rf¶

Run ID: midd-rf
Participant: middlebury
Track: News
Year: 2021
Submission: 6/4/2021
Type: auto
Task: background
MD5: df778e478b4d7b688a7741232a523fed
Run description: Random Forest Reranking; Local TF-IDF Similarities; Temporal features; Kicker Similarities; Clickbait features.

midd-transfer¶

Run ID: midd-transfer
Participant: middlebury
Track: News
Year: 2021
Submission: 6/4/2021
Type: auto
Task: background
MD5: 7a16ca5c2b9a8293846d73d85dac9b0f
Run description: Random Forest Ranking as supervision for linear model; Local TF-IDF Similarities; Temporal features; Kicker Similarities; Clickbait features.

midd-twostage¶

Run ID: midd-twostage
Participant: middlebury
Track: News
Year: 2021
Submission: 6/4/2021
Type: auto
Task: background
MD5: d56903036161af04cb61d4b5b47fc8e7
Run description: Random Forest Reranking of Shallow Pool; Local TF-IDF Similarities; Temporal features; Kicker Similarities; Clickbait features.

QU_LeadPar¶

Run ID: QU_LeadPar
Participant: QU
Track: News
Year: 2021
Submission: 6/6/2021
Type: auto
Task: background
MD5: 0c0006938a11eb69671a1b2bfc2f5315
Run description: In this run, we obtained a set of background links using the 16 leading paragraph of the query article along with its title as a search query againts an inverted index of V4 of the collection created using Lucene.

QU_SP_MBERT¶

Run ID: QU_SP_MBERT
Participant: QU
Track: News
Year: 2021
Submission: 6/5/2021
Type: auto
Task: bgsubs
MD5: 11e94ddb265e40035253cffe87bead94
Run description: In this run, we obtained initially a set of background links using the 16 leading paragraph of the query article as a search query, then we split this set into passages. We considered later each subtopic in the query article as an independent search query and we raranked the passages of the candidate set for each subtopic using monoBERT reranker model. Finally, we used the maximum passage score as a document score for the final run generation.

QU_SP_MSM¶

Run ID: QU_SP_MSM
Participant: QU
Track: News
Year: 2021
Submission: 6/5/2021
Type: auto
Task: bgsubs
MD5: b1a6f7386b814766b37cab678efe192b
Run description: In this run, we obtained initially a set of background links using the 16 leading paragraph of the query article as a search query, then we split this set into passages. We considered later each subtopic in the query article as an independent search query and we raranked the passages of the candidate set for each subtopic using a sentence transformer model that was trained on MSMARCO dataset. Finally, we used the maximum passage score as a document score for the final run generation.

QU_SS_MSM¶

Run ID: QU_SS_MSM
Participant: QU
Track: News
Year: 2021
Submission: 6/5/2021
Type: auto
Task: bgsubs
MD5: 10bb745d75ea0641541ff6c4b33a3a96
Run description: In this run, we obtained initially a set of background links using a concatenation of the subtopics description of the query article as a search query, then we split this set into passages. We considered later each subtopic in the query article as an independent search query and we raranked the passages of the candidate set for each subtopic using a sentence transformer model that was trained on MSMARCO dataset. Finally, we used the maximum passage score as a document score for the final run generation.

QU_YakeTruss¶

Run ID: QU_YakeTruss
Participant: QU
Track: News
Year: 2021
Submission: 6/6/2021
Type: auto
Task: background
MD5: e329ecbb19939af10fdd9d6137738493
Run description: In this run, we extracted a list of weighted keywords using Yake and KTruss keyword extraction methods then we applied linear interpolation with alpha =0.5 and selected the top 100 keywords as a search query. We then issued this search query against an inverted index of V4 of the collection created using Lucene.

rel00_ent07¶

Run ID: rel00_ent07
Participant: IR-Cologne
Track: News
Year: 2021
Submission: 6/4/2021
Type: auto
Task: background
MD5: e4f819683fecb1d2b5341a7db0ab2fe7
Run description: BM25 ranking but with entity depending reranking.

rel02_ent05¶

Run ID: rel02_ent05
Participant: IR-Cologne
Track: News
Year: 2021
Submission: 6/4/2021
Type: auto
Task: background
MD5: ec0a78821133ca37fa4c9a5a35ccc27c
Run description: BM25 ranking but with entity and relation depending reranking. Focus on entities.

rel05_ent02¶

Run ID: rel05_ent02
Participant: IR-Cologne
Track: News
Year: 2021
Submission: 6/4/2021
Type: auto
Task: background
MD5: 0b49aeccbed5a8a0444656c3f69f2157
Run description: BM25 ranking but with entity and relation depending reranking. Focus on relations.

rel07_ent00¶

Run ID: rel07_ent00
Participant: IR-Cologne
Track: News
Year: 2021
Submission: 6/4/2021
Type: auto
Task: background
MD5: b0fdcddb3a922d502dfa8d3d1962d4ed
Run description: BM25 ranking but with relation depending reranking.

S300K_ENT_P_DN2¶

Run ID: S300K_ENT_P_DN2
Participant: L3i_Rochelle
Track: News
Year: 2021
Submission: 6/7/2021
Type: auto
Task: bgsubs
MD5: 1263e75336809835f00022971d61a13a
Run description: 300Keywords+Cosine(Entities,Phrases,Description,Narrative)

S300K_ENT_PH_DN¶

Run ID: S300K_ENT_PH_DN
Participant: L3i_Rochelle
Track: News
Year: 2021
Submission: 6/6/2021
Type: auto
Task: bgsubs
MD5: 1fcbf2e7a775ad1e517ab74e94efb9e2
Run description: 300Keywords+Filter+Cosine(Entities,Phrases,Description,Narrative)

S300K_PH_DN¶

Run ID: S300K_PH_DN
Participant: L3i_Rochelle
Track: News
Year: 2021
Submission: 6/6/2021
Type: auto
Task: bgsubs
MD5: d98c6387b9294de95c79bbcf7573a096
Run description: 300Keywords+Cosine(Phrases,Description,Narrative)

SU_BiEnc¶

Run ID: SU_BiEnc
Participant: SU-NLP
Track: News
Year: 2021
Submission: 6/4/2021
Type: auto
Task: wikification
MD5: 06bb986857efc1155abd2dc76c3fb6eb
Run description: FLAIR + BiEncoder Pretrained models

SU_BiEnc_CrsEnc¶

Run ID: SU_BiEnc_CrsEnc
Participant: SU-NLP
Track: News
Year: 2021
Submission: 6/4/2021
Type: auto
Task: wikification
MD5: 0554d0ad7aa23ff14f53b9c205865003
Run description: FLAIR + BiEncoder + CrossEncoder Pretrained models

SU_ES¶

Run ID: SU_ES
Participant: SU-NLP
Track: News
Year: 2021
Submission: 6/5/2021
Type: auto
Task: wikification
MD5: 1273ad9261304ece9c3ca83e11040ffd
Run description: FLAIR + ES + TextOrder

SU_ES_CrsEnc¶

Run ID: SU_ES_CrsEnc
Participant: SU-NLP
Track: News
Year: 2021
Submission: 6/5/2021
Type: auto
Task: wikification
MD5: 186b482846b1330a30ec679331188f96
Run description: ElasticSearch + Crossencoder

SU_ES_CrsEnc_NF¶

Run ID: SU_ES_CrsEnc_NF
Participant: SU-NLP
Track: News
Year: 2021
Submission: 6/5/2021
Type: auto
Task: wikification
MD5: d6aa3fb9b16d69c35d132bbd9125cd6b
Run description: Flair + ElasticSearch + CrossEncoder + no textorder filtering

TKB48_Run1_DTQ¶

Run ID: TKB48_Run1_DTQ
Participant: TKB48
Track: News
Year: 2021
Submission: 6/3/2021
Type: auto
Task: background
MD5: 2a371f81c2472f1199d1e63a1593a036
Run description: desc + predicted queries rank by similarity: desc&content, desc&key words

TKB48_Run2_Tep¶

Run ID: TKB48_Run2_Tep
Participant: TKB48
Track: News
Year: 2021
Submission: 6/3/2021
Type: auto
Task: background
MD5: 8eba7e91671f1a588586b798194c1d01
Run description: desc + predicted queries; rank by the similarity of desc&content, desc&key words, recency weight

TKB48_Run3_skw¶

Run ID: TKB48_Run3_skw
Participant: TKB48
Track: News
Year: 2021
Submission: 6/3/2021
Type: auto
Task: background
MD5: dd898442a87e4f4fa07cf9ab78ba9f9a
Run description: desc + predicted queries rank by similarity: desc&content, desc&key words,keywords&keywords, recency weight

TKB48_Run4_tlkw¶

Run ID: TKB48_Run4_tlkw
Participant: TKB48
Track: News
Year: 2021
Submission: 6/3/2021
Type: auto
Task: background
MD5: 009baeeba03d89a8a4d56ea522dadc6c
Run description: title + desc + predicted queries, rank by the similarity of desc&content, desc&key words,keywords&keywords, recency weight

TKB48_SRun1_DS¶

Run ID: TKB48_SRun1_DS
Participant: TKB48
Track: News
Year: 2021
Submission: 6/3/2021
Type: auto
Task: bgsubs
MD5: 2a57798db4702d1e5255d490f67503c6
Run description: desc + subtopic; rank by the similarity of desc&content, desc&key words,keywords&keywords

TKB48_SRun2_Tep¶

Run ID: TKB48_SRun2_Tep
Participant: TKB48
Track: News
Year: 2021
Submission: 6/3/2021
Type: auto
Task: bgsubs
MD5: 517fd77bbe4cfc1da2cb9fd579931ba5
Run description: desc + subtopic; rank by the similarity of desc&content, desc&key words,keywords&keywords and recency weight

TKB48_SRun3_DTQ¶

Run ID: TKB48_SRun3_DTQ
Participant: TKB48
Track: News
Year: 2021
Submission: 6/3/2021
Type: auto
Task: bgsubs
MD5: 20cfc3362c85d7026ab6184c54bd56c2
Run description: predicted queries + desc +subtopic; rank by the similarity of desc&content, desc&key words,keywords&keywords and recency weight

TKB48_SRun4_ST¶

Run ID: TKB48_SRun4_ST
Participant: TKB48
Track: News
Year: 2021
Submission: 6/3/2021
Type: auto
Task: bgsubs
MD5: 373634d90f4193d4dd5f73168e3aff23
Run description: predicted queries + desc +subtopic; rank by the similarity of desc&content, desc&key words,keywords&keywords, subtopic&content, and recency weight

UW_UDHAV_S100¶

Run ID: UW_UDHAV_S100
Participant: Waterloo_Cormack
Track: News
Year: 2021
Submission: 6/5/2021
Type: auto
Task: bgsubs
MD5: 6da5d508686fc1a04b69084adf5a2402
Run description: This approach leverages the capability of BERT to learn contextual representations of the query in order to perform semantic search over the corpus. It uses a combination of BM25 and BERT for retrieval. This run includes 100 documents per topic, with a mixture of subtopics.

UW_UDHAVSETHI¶

Run ID: UW_UDHAVSETHI
Participant: Waterloo_Cormack
Track: News
Year: 2021
Submission: 6/5/2021
Type: auto
Task: background
MD5: 853235110cd3c74cd6dc7560bc25007c
Run description: This approach leverages the capability of BERT to learn contextual representations of the query in order to perform semantic search over the corpus. It uses a combination of BM25 and BERT for retrieval.

UW_UDHAVSETHI_S¶

Run ID: UW_UDHAVSETHI_S
Participant: Waterloo_Cormack
Track: News
Year: 2021
Submission: 6/5/2021
Type: auto
Task: bgsubs
MD5: 297b83dff29a44d37ceece91644f3651
Run description: This approach leverages the capability of BERT to learn contextual representations of the query in order to perform semantic search over the corpus. It uses a combination of BM25 and BERT for retrieval. This run includes subtopics.