Skip to content

Runs - Complex Answer Retrieval 2017

CUISER

Participants

  • Run ID: CUISER
  • Participant: CUIS
  • Track: Complex Answer Retrieval
  • Year: 2017
  • Submission: 8/14/2017
  • Type: automatic
  • Task: entities
  • MD5: ff7981eda73b20a8c9921653c5d7653e
  • Run description: simply replace para_id in passage ranking results by article_id

CUISPR

Participants

  • Run ID: CUISPR
  • Participant: CUIS
  • Track: Complex Answer Retrieval
  • Year: 2017
  • Submission: 8/13/2017
  • Type: automatic
  • Task: passages
  • MD5: 3e53a5f6fc7f0972acf92353b2034a21
  • Run description: scoring pass 1: BM25 by Lucene 6 scoring pass 2: a slightly modified sequential dependence model

ECNU-runONE

Participants

  • Run ID: ECNU-runONE
  • Participant: ECNU
  • Track: Complex Answer Retrieval
  • Year: 2017
  • Submission: 8/13/2017
  • Type: automatic
  • Task: passages
  • MD5: 85b0a46ba4a7880e9ea19bf1eee006d6
  • Run description: First, we use lucene to select candidate paragraphs, and then we use bm25 score and word matching as features to training a ranking model with Ranklib.

mpii-nn4_pos_hperc

Participants | Proceedings

  • Run ID: mpii-nn4_pos_hperc
  • Participant: MPIID5
  • Track: Complex Answer Retrieval
  • Year: 2017
  • Submission: 8/14/2017
  • Type: automatic
  • Task: passages
  • MD5: 0f8337a4d4ab4c3f2ed8eb78f88a68e5
  • Run description: PACRR variant on BM25 candidate list

mpii-nn6_pos

Participants | Proceedings

  • Run ID: mpii-nn6_pos
  • Participant: MPIID5
  • Track: Complex Answer Retrieval
  • Year: 2017
  • Submission: 8/14/2017
  • Type: automatic
  • Task: passages
  • MD5: 87bfbbe8ebad228cda7095b8853f1c1d
  • Run description: PACRR variant on BM25 candidate list

mpii-nn6_pos_tprob

Participants | Proceedings

  • Run ID: mpii-nn6_pos_tprob
  • Participant: MPIID5
  • Track: Complex Answer Retrieval
  • Year: 2017
  • Submission: 8/14/2017
  • Type: automatic
  • Task: passages
  • MD5: 50985ffac8ca6635e42a74f1d8413c77
  • Run description: PACRR variant on BM25 candidate list

nyudl-ds

Participants

  • Run ID: nyudl-ds
  • Participant: NYUDL
  • Track: Complex Answer Retrieval
  • Year: 2017
  • Submission: 8/10/2017
  • Type: automatic
  • Task: passages
  • MD5: fb74dd2f7698bd305d7c9f7d55fb3c30
  • Run description: Simple Document Classifier using avg word embeddings in the documents as document vector and last hidden state of an LSTM as query vector. 2-layer feed forward neural net to select which documents are relevant given a query.
  • Code: https://github.com/nyu-dl/QueryReformulator

nyudl-qr

Participants

  • Run ID: nyudl-qr
  • Participant: NYUDL
  • Track: Complex Answer Retrieval
  • Year: 2017
  • Submission: 8/6/2017
  • Type: automatic
  • Task: passages
  • MD5: 2740426c216d4b9db637e5547a3a3744
  • Run description: Query reformulation with deep reinforcement learning.

nyudl-qrds

Participants

  • Run ID: nyudl-qrds
  • Participant: NYUDL
  • Track: Complex Answer Retrieval
  • Year: 2017
  • Submission: 8/12/2017
  • Type: automatic
  • Task: passages
  • MD5: 5da56a0cf26842f9b15bdd12412fa11c
  • Run description: Query reformulation using reinforcement learning + Lucene + Neural Net Classifier to select documents
  • Code: https://github.com/nyu-dl/QueryReformulator

top100-c-pr-bm25

Participants

  • Run ID: top100-c-pr-bm25
  • Participant: TREMA-UNH
  • Track: Complex Answer Retrieval
  • Year: 2017
  • Submission: 8/14/2017
  • Type: automatic
  • Task: entities
  • MD5: 2e9cbae8bba0c97862b5bb64d35e4d2a
  • Run description: ranks entities by degree centrality on a sub-graph that is extracted as follows: 1) edges in KG are associated with paragraph-long text 2) A BM25 model is used to retrieve edges in response to the query 3) edges are weighted according to their frequency 4) PageRank on this weighted graph is used to rank entities Support paragraphs are taken from the the paragraph associated with the entity's highest ranking edge

top100-rr-marg-bm25

Participants

  • Run ID: top100-rr-marg-bm25
  • Participant: TREMA-UNH
  • Track: Complex Answer Retrieval
  • Year: 2017
  • Submission: 8/14/2017
  • Type: automatic
  • Task: entities
  • MD5: 312b492c1a84d16925889bfff39ca2de
  • Run description: ranks entities by degree centrality on a sub-graph that is extracted as follows: 1) edges in KG are associated with paragraph-long text 2) A BM25 model is used to retrieve edges in response to the query 3) edges are weighted according to their reciprocal rank 4) DegreeCentrality on this weighted graph is used to rank entities Support paragraphs are taken from the the paragraph associated with the entity's highest ranking edge
  • Code: https://github.com/bgamari/mediawiki-annotate/tree/master/graph-expansion

top100-sc-ppr-bm25

Participants

  • Run ID: top100-sc-ppr-bm25
  • Participant: TREMA-UNH
  • Track: Complex Answer Retrieval
  • Year: 2017
  • Submission: 8/14/2017
  • Type: automatic
  • Task: entities
  • MD5: 8f327593fbd704c7c01e8cbe1e5b943c
  • Run description: ranks entities by degree centrality on a sub-graph that is extracted as follows: 1) edges in KG are associated with paragraph-long text 2) A BM25 model is used to retrieve edges in response to the query 3) edges are weighted according to their frequency 4) seed nodes are retrieved from an entity index of unprocessedtraing using Bm25 5) PersonalizedPageRank on this weighted graph with seed nodes is used to rank entities Support paragraphs are taken from the the paragraph associated with the entity's highest ranking edge
  • Code: https://github.com/bgamari/mediawiki-annotate/tree/master/graph-expansion

treccarict

Participants | Proceedings

  • Run ID: treccarict
  • Participant: ICTNET
  • Track: Complex Answer Retrieval
  • Year: 2017
  • Submission: 8/14/2017
  • Type: automatic
  • Task: passages
  • MD5: 27c5c0ec2697c4987ac374a5ae5bd529
  • Run description: Use BM25 to retrieval.

UNH-benchmarkY1test.bm25

Participants

  • Run ID: UNH-benchmarkY1test.bm25
  • Participant: TREMA-UNH
  • Track: Complex Answer Retrieval
  • Year: 2017
  • Submission: 8/13/2017
  • Type: automatic
  • Task: passages
  • MD5: 03ad8d7651c7d5db1a13b1f8fa559d6c
  • Run description: BM25 run using the concatenation of heading, parent headings and page title as keyword query

UNH-benchmarkY1test.expan

Participants

  • Run ID: UNH-benchmarkY1test.expan
  • Participant: TREMA-UNH
  • Track: Complex Answer Retrieval
  • Year: 2017
  • Submission: 8/13/2017
  • Type: automatic
  • Task: passages
  • MD5: fc669bb3324c57f9c773686c1eec785c
  • Run description: BM25 run using the concatenation of heading, parent headings and page title as keyword query. In addition query expansion with two sources: 1) the query is entity linked with TagMe. Terms from the first paragraph of the entity are used for expansion (like RM3) 2) if the same heading is contained in another article, then expansion terms from these sections will be used for expansion (like RM3). (This method is in spirit of the WikiKreator system) Balancing parameters are manually adjusted on test200

UTDHLTRIAR

Participants | Proceedings

  • Run ID: UTDHLTRIAR
  • Participant: UTDHLTRI
  • Track: Complex Answer Retrieval
  • Year: 2017
  • Submission: 8/13/2017
  • Type: automatic
  • Task: passages
  • MD5: 6c3e5a792698e1da4c8e6c43e2bfd406
  • Run description: 1. Initial Search 1.1 Parse the article title and secition names to use in part of query for Lucuene search using BM25 1.2 Use entities extracted from Dbpedia long abstracts data set from Dbpedia spotlight tagging to augment query 2. Re-Ranking 2.1 use Ranklib's Ada Rank implementation to re-rank paragraphs

UTDHLTRINN20

Participants | Proceedings

  • Run ID: UTDHLTRINN20
  • Participant: UTDHLTRI
  • Track: Complex Answer Retrieval
  • Year: 2017
  • Submission: 8/13/2017
  • Type: automatic
  • Task: passages
  • MD5: e240ef9dd1d68b65ce9940286d576af3
  • Run description: 1. Initial Search 1.1 Parse the article title and secition names to use in part of query for Lucuene search using BM25 1.2 Use entities extracted from Dbpedia long abstracts data set from Dbpedia spotlight tagging to augment query 2. Re-Ranking 2.1 use neural learning to rank model to re-rank paragraphs from Initial Search

UTDHLTRINN50

Participants | Proceedings

  • Run ID: UTDHLTRINN50
  • Participant: UTDHLTRI
  • Track: Complex Answer Retrieval
  • Year: 2017
  • Submission: 8/13/2017
  • Type: automatic
  • Task: passages
  • MD5: 626c47472f60934e13253b12f90b51d5
  • Run description: 1. Initial Search 1.1 Parse the article title and secition names to use in part of query for Lucuene search using BM25 1.2 Use entities extracted from Dbpedia long abstracts data set from Dbpedia spotlight tagging to augment query 2. Re-Ranking 2.1 use neural learning to rank model to re-rank paragraphs from Initial Search