Runs - News 2020¶
ans_bm25¶
- Run ID: ans_bm25
- Participant: anserini
- Track: News
- Year: 2020
- Submission: 7/10/2020
- Type: auto
- Task: background
- MD5:
3cbec1ac71203f438a6dec87cf87cfcc - Run description: BM25 with a query consisting of top 100 (tf-idf) terms extracted from topic document.
ans_bm25_rm3¶
- Run ID: ans_bm25_rm3
- Participant: anserini
- Track: News
- Year: 2020
- Submission: 7/10/2020
- Type: auto
- Task: background
- MD5:
ee9749d9e62a2d02b378c8360d28793e - Run description: BM25 + RM3 with a query consisting of top 100 (tf-idf) terms extracted from topic document.
ans_bm25_rm3_df¶
- Run ID: ans_bm25_rm3_df
- Participant: anserini
- Track: News
- Year: 2020
- Submission: 7/10/2020
- Type: auto
- Task: background
- MD5:
03d9b1fd0d4c3c22d07f91eba5ccf91b - Run description: BM25 + RM3 + Date Filter using a query consisting of top 100 (tf-idf) terms extracted from topic document. All retrieved documents for topic 896 were filtered out - a dummy was inserted for submission to be accepted.
BJTAG1¶
- Run ID: BJTAG1
- Participant: BJTAG
- Track: News
- Year: 2020
- Submission: 7/29/2020
- Type: auto
- Task: background
- MD5:
e9110d92602ede8cd6fa6465fbb001cc - Run description: This model used SAS Text analytics techniques such as CAS Search and CAS Text parsing actions.
BJTAG3¶
- Run ID: BJTAG3
- Participant: BJTAG
- Track: News
- Year: 2020
- Submission: 7/30/2020
- Type: auto
- Task: background
- MD5:
83b85128a10e850c3384b46225b19a63 - Run description: This model used SAS Text Analytics products such as CAS Search, TextParse actions. Also, used tf-idf and sentence transformer for calculating document similarity.
BJTAG_wiki1¶
- Run ID: BJTAG_wiki1
- Participant: BJTAG
- Track: News
- Year: 2020
- Submission: 7/30/2020
- Type: auto
- Task: wikification
- MD5:
6eec0fea1512685533e4cdcadc73dd87 - Run description: Note on the Wikification result (precedence 2) from the BJTAG: 0. The run tag "BJTAG_wiki1" is artificial because this task 2 doesn't need to output a run tag. 1. The start/len for the anchor text is based on the original data (i.e. such data as in is still there). 2. Only the first anchor text is output even if it appears several times in the topic text. 3. The score is for the ranking of the anchor text. 4. Each anchor text only has one link for it.
BJTAG_wiki2¶
- Run ID: BJTAG_wiki2
- Participant: BJTAG
- Track: News
- Year: 2020
- Submission: 7/31/2020
- Type: auto
- Task: wikification
- MD5:
d0ff8114767b82a58b6167211dbc4680 - Run description: Note on the Wikification result (precedence 1) from the BJTAG: 0. The run tag "BJTAG_wiki2" is artificial because this task 2 doesn't need to output a run tag. 1. The start/len for the anchor text is based on the original data (i.e. such data as in is still there). 2. Only the first anchor text is output even if it appears several times in the topic text. 3. The score is for the ranking of the anchor text. 4. Each anchor text only has one link for it.
clac-combined¶
Participants | Proceedings | Appendix
- Run ID: clac-combined
- Participant: CLAC_NEWS_2020
- Track: News
- Year: 2020
- Submission: 7/31/2020
- Type: auto
- Task: background
- MD5:
0d2dade00f42e964df00fd78dce6cdb7 - Run description: bm25 and gpt2 embedding combined score
clac-d2v2019¶
Participants | Proceedings | Appendix
- Run ID: clac-d2v2019
- Participant: CLAC_NEWS_2020
- Track: News
- Year: 2020
- Submission: 7/31/2020
- Type: auto
- Task: background
- MD5:
2f176c2c137de1c48fe94ca6309134fa - Run description: run from 2019 model. doc2vec with Jaccard similarity
clac-es-bm25¶
Participants | Proceedings | Appendix
- Run ID: clac-es-bm25
- Participant: CLAC_NEWS_2020
- Track: News
- Year: 2020
- Submission: 7/31/2020
- Type: auto
- Task: background
- MD5:
ba0aadf0091b047412b1c666e63590b6 - Run description: baseline with elasticsearch bm25
clac-gpt2-norm¶
Participants | Proceedings | Appendix
- Run ID: clac-gpt2-norm
- Participant: CLAC_NEWS_2020
- Track: News
- Year: 2020
- Submission: 7/31/2020
- Type: auto
- Task: background
- MD5:
63fdba4821fba83d37d40f00c42592ee - Run description: normalized gpt2 embedding vector space with custom proximity function
IRISINews1¶
Participants | Proceedings | Appendix
- Run ID: IRISINews1
- Participant: IRLABISI
- Track: News
- Year: 2020
- Submission: 7/10/2020
- Type: auto
- Task: background
- MD5:
5619a10f14d32dcdfd6bd68a143ee477 - Run description: Boosted terms extracted from content to perform query.
IRISINews2¶
Participants | Proceedings | Appendix
- Run ID: IRISINews2
- Participant: IRLABISI
- Track: News
- Year: 2020
- Submission: 7/10/2020
- Type: auto
- Task: background
- MD5:
5f9b684073bbbbbeb65d8b5c69d9c563 - Run description: This run is generated by merging two ranked lists with tuned weight. One is a list generated by boosted query. other is a list generated by vector similarity of two documents.
IRISINews3¶
Participants | Proceedings | Appendix
- Run ID: IRISINews3
- Participant: IRLABISI
- Track: News
- Year: 2020
- Submission: 7/10/2020
- Type: auto
- Task: background
- MD5:
74e1e7ee920085016da7098bb2a147eb - Run description: This method uses Named Entites extracted from the title of the articles to generate the query
IRISINews4¶
Participants | Proceedings | Appendix
- Run ID: IRISINews4
- Participant: IRLABISI
- Track: News
- Year: 2020
- Submission: 7/10/2020
- Type: auto
- Task: background
- MD5:
39fee2c692edc25b1991aeb28375898d - Run description: We construct 2 weighted queries, one generated from title and other from content. We merge the two resultsets with appropriate weights.
mlt_base¶
Participants | Proceedings | Appendix
- Run ID: mlt_base
- Participant: OSC
- Track: News
- Year: 2020
- Submission: 7/10/2020
- Type: auto
- Task: background
- MD5:
d25d739e7898df10fbb78a535ced4fb7 - Run description: Elasticsearch's More Like This
mlt_tune¶
Participants | Proceedings | Appendix
- Run ID: mlt_tune
- Participant: OSC
- Track: News
- Year: 2020
- Submission: 7/10/2020
- Type: auto
- Task: background
- MD5:
24d60aa6e219c754770429477a3d0682 - Run description: Elasticsearch's More Like This tuned by Quaerite a search tuning library
mlt_tune_ners¶
Participants | Proceedings | Appendix
- Run ID: mlt_tune_ners
- Participant: OSC
- Track: News
- Year: 2020
- Submission: 7/10/2020
- Type: auto
- Task: background
- MD5:
d58c4f1b0cb444ef1977077cd3045d48 - Run description: Elasticsearch's More Like This tuned by Quaerite, a search tuning library, augmented with curated NERs via Spacy
QU_KCR¶
- Run ID: QU_KCR
- Participant: QU
- Track: News
- Year: 2020
- Submission: 7/30/2020
- Type: auto
- Task: background
- MD5:
8604818671eb97ebe094840cab9678a3 - Run description: We basically retrieve an initial set of articles using BM25, then we construct a graph for the query as well as the retrieved articles and we decompose these graph using the K-Core method. Finally we rerank the initial set of articles based on graph similarity with the query.
QU_KTR¶
- Run ID: QU_KTR
- Participant: QU
- Track: News
- Year: 2020
- Submission: 7/30/2020
- Type: auto
- Task: background
- MD5:
6e0b0acb6fe122f426739f20ef90372e - Run description: We basically retrieve an initial set of articles based on BM25 scoring, then we construct a graph for the query as well as the retrieved articles and we decompose these graph using the K-Truss method. Finally we rerank the initial set of articles based on graph similarity with the query.
QUKC¶
- Run ID: QUKC
- Participant: QU
- Track: News
- Year: 2020
- Submission: 7/9/2020
- Type: auto
- Task: background
- MD5:
23189cb0e456107f16cd2d041db85361 - Run description: Basically, we analyze the topic article's text using K-CORE graph analysis method to extract the most influential keywords from the article. Then, we use these keywords as a query in an ad-hoc retrieval setting.
QUKT¶
- Run ID: QUKT
- Participant: QU
- Track: News
- Year: 2020
- Submission: 7/9/2020
- Type: auto
- Task: background
- MD5:
ecdfa0114974c53eb4bde4e3b390119b - Run description: Basically, we analyze the topic article's text using K-TRUSS graph analysis method to extract the most influential keywords from the article. Then, we use these keywords as a query in an ad-hoc retrieval setting.
ru_g_diversity¶
Participants | Proceedings | Appendix
- Run ID: ru_g_diversity
- Participant: RUIR
- Track: News
- Year: 2020
- Submission: 7/10/2020
- Type: auto
- Task: background
- MD5:
352bbc21eec27ee1ddc5f06ffa079c25 - Run description: Graph representation - similar as ru_graph. Filters top 5 results based on most present entity types, i.e. each doc in top 5 has most named entities of one type. External resources: - REL: Radboud Entity Linker - used for the retrieval of named entities - Wikipedia2vec: word2vec embeddings trained on Wikipedia 2019 - used for edge weights in graph.
ru_g_ne¶
Participants | Proceedings | Appendix
- Run ID: ru_g_ne
- Participant: RUIR
- Track: News
- Year: 2020
- Submission: 7/10/2020
- Type: auto
- Task: background
- MD5:
2703ef0fc72494405cf924a9fc1c0c84 - Run description: Graph representation - similar to ru_graph. Extract named entities from article and add as nodes to graph. External resources: - REL: Radboud Entity Linker - used for the retrieval of named entities - Wikipedia2vec: word2vec embeddings trained on Wikipedia 2019 - used for edge weights in graph.
ru_g_novelty¶
Participants | Proceedings | Appendix
- Run ID: ru_g_novelty
- Participant: RUIR
- Track: News
- Year: 2020
- Submission: 7/10/2020
- Type: auto
- Task: background
- MD5:
4eb44d12e59bb45d31120c7e5e4a640f - Run description: Graph representation - similar as ru_graph. Determines background relevance on nodes that are strongly related to common subgraph but are not in topic article. Also uses (external resource) for edge creation: - Wikipedia2vec: word2vec embeddings trained on Wikipedia 2019
ru_g_textrank¶
Participants | Proceedings | Appendix
- Run ID: ru_g_textrank
- Participant: RUIR
- Track: News
- Year: 2020
- Submission: 7/10/2020
- Type: auto
- Task: background
- MD5:
8f857692967fe4657a80bbfc0e5b11ac - Run description: Graph representation - similar as ru_graph. Recalculates node weights based on TextRank algorithm. External resource: - Wikipedia2vec: word2vec embeddings trained on Wikipedia 2019 - used for edge weights in graph.
ru_graph¶
Participants | Proceedings | Appendix
- Run ID: ru_graph
- Participant: RUIR
- Track: News
- Year: 2020
- Submission: 7/10/2020
- Type: auto
- Task: background
- MD5:
1d7d660d3126af6aecd010f1d879dbed - Run description: Graph representation of articles, relevance determined by overlap in graphs. Nodes consist of 100 terms with highest tf-idf score. Edges between nodes are based on cosine similarity using (external resource): - Wikipedia2vec: word2vec embeddings trained on Wikipedia 2019
SUNLP_BERT_RR¶
Participants | Proceedings | Appendix
- Run ID: SUNLP_BERT_RR
- Participant: SUNLP
- Track: News
- Year: 2020
- Submission: 7/10/2020
- Type: auto
- Task: background
- MD5:
8bf64338e69f4098300d45477ef19ef0 - Run description: This one is reranking docs with BERT.
SUNLP_BERT_Summ¶
Participants | Proceedings | Appendix
- Run ID: SUNLP_BERT_Summ
- Participant: SUNLP
- Track: News
- Year: 2020
- Submission: 7/10/2020
- Type: auto
- Task: background
- MD5:
c0f4c633a2800879df8ce3458201e2cf - Run description: This one uses the BERT summarizer model to get a summary to be used as the query.
SUNLP_doc2vec¶
Participants | Proceedings | Appendix
- Run ID: SUNLP_doc2vec
- Participant: SUNLP
- Track: News
- Year: 2020
- Submission: 7/31/2020
- Type: auto
- Task: wikification
- MD5:
9aa49416f97c432c242d4e9a74b4d1c8 - Run description: NER + Title match phrase + doc2vec
SUNLP_FullText¶
Participants | Proceedings | Appendix
- Run ID: SUNLP_FullText
- Participant: SUNLP
- Track: News
- Year: 2020
- Submission: 7/10/2020
- Type: auto
- Task: background
- MD5:
cb66d0df457f01f024162a2994818edc - Run description: This one uses the full content as query.
SUNLP_textorder¶
Participants | Proceedings | Appendix
- Run ID: SUNLP_textorder
- Participant: SUNLP
- Track: News
- Year: 2020
- Submission: 7/31/2020
- Type: auto
- Task: wikification
- MD5:
c87cbe7e34fef52e2ea6921da1794d2f - Run description: NER + title match phrase
SUNLP_USE¶
Participants | Proceedings | Appendix
- Run ID: SUNLP_USE
- Participant: SUNLP
- Track: News
- Year: 2020
- Submission: 7/10/2020
- Type: auto
- Task: background
- MD5:
f57e822d239164c8392d2804fc3e92ce - Run description: This one is using Universal Sentence Encoder during indexing and search.
tune_ners_embed¶
Participants | Proceedings | Appendix
- Run ID: tune_ners_embed
- Participant: OSC
- Track: News
- Year: 2020
- Submission: 7/10/2020
- Type: auto
- Task: background
- MD5:
a8d07a971c4ff8bc1a85d8242ab62891 - Run description: Elasticserch's More Like This query tuned with Quarite and augmented with NERs from Spacy and content embedding from Sentence Bert
tuw-ifs-1¶
Participants | Proceedings | Appendix
- Run ID: tuw-ifs-1
- Participant: TUW-IFS
- Track: News
- Year: 2020
- Submission: 7/10/2020
- Type: auto
- Task: wikification
- MD5:
f63b27d61e276ade81c39263d9cbeae6 - Run description: The features include commonness, topic modeling, and doc2vec similarity. The run used doc2vec pre-trained features that are not from Wikipedia dump provided by TREC.
tuw-ifs-2¶
Participants | Proceedings | Appendix
- Run ID: tuw-ifs-2
- Participant: TUW-IFS
- Track: News
- Year: 2020
- Submission: 7/10/2020
- Type: auto
- Task: wikification
- MD5:
1a7dfa0d342b62b34feefdf5a394212c - Run description: The features include commonness, topic modeling, and doc2vec similarity. The run used doc2vec pre-trained features that are not from Wikipedia dump provided by TREC. We give a float ranking on second column
tuw-ifs-3¶
Participants | Proceedings | Appendix
- Run ID: tuw-ifs-3
- Participant: TUW-IFS
- Track: News
- Year: 2020
- Submission: 7/30/2020
- Type: auto
- Task: wikification
- MD5:
3d43b7144e08d98c766516acec698d36 - Run description: The features include commonness, link probability, topic modeling, and doc2vec similarity. The run used doc2vec pre-trained features that are not from Wikipedia dump provided by TREC.
tuw-ifs-4¶
Participants | Proceedings | Appendix
- Run ID: tuw-ifs-4
- Participant: TUW-IFS
- Track: News
- Year: 2020
- Submission: 7/30/2020
- Type: auto
- Task: wikification
- MD5:
283b32de653f9e38dd9c8e4ff2ec9221 - Run description: The features include commonness, link probability, topic modeling, and doc2vec similarity, with different weighting. The run used doc2vec pre-trained features that are not from Wikipedia dump provided by TREC.
udel_fang_AW¶
Participants | Proceedings | Appendix
- Run ID: udel_fang_AW
- Participant: udel_fang
- Track: News
- Year: 2020
- Submission: 7/9/2020
- Type: auto
- Task: background
- MD5:
2f6f9db9285ad5231519e2fdf231e56e - Run description: DBpedia spotlight was used to annotate entities in the documents. Annotated documents are used to build an index. All words in the query documents are used as the query to search against the index. Only documents published before the published_date of the query document are returned.
udel_fang_CE¶
Participants | Proceedings | Appendix
- Run ID: udel_fang_CE
- Participant: udel_fang
- Track: News
- Year: 2020
- Submission: 7/9/2020
- Type: auto
- Task: background
- MD5:
e1c18a62a252886c0c0ae33aedf39192 - Run description: DBpedia spotlight was used to annotate entities in the documents. Annotated documents are used to build an index. All words in the query documents are used as the query to search against the index. Only documents published before the published_date of the query document are returned. Subtopics in the query document are mined by building a graph using entities in the documents and performing community analysis on the graph. Top 100 results from the initial results are re-ranked based on these subtopics.
udel_fang_CW¶
Participants | Proceedings | Appendix
- Run ID: udel_fang_CW
- Participant: udel_fang
- Track: News
- Year: 2020
- Submission: 7/9/2020
- Type: auto
- Task: background
- MD5:
31db4e9604ff12b33f809a962224c9ba - Run description: DBpedia spotlight was used to annotate entities in the documents. Annotated documents are used to build an index. All words in the query documents are used as the query to search against the index. Only documents published before the published_date of the query document are returned. Subtopics in the query document are mined by building a graph using entities in the documents and performing community analysis on the graph. The subtopics are expanded using non-entity words co-occur with them. Top 100 results from the initial results are re-ranked based on these expanded subtopics.