Runs - Web 2013¶

clustmrfaf¶

Run ID: clustmrfaf
Participant: Technion
Track: Web
Year: 2013
Submission: 8/5/2013
Type: automatic
Task: adhoc
MD5: 623a7a010d89b490ea684f7136a9e2f8
Run description: We created an initial list of documents using the MRF retrieval method (documents assigned with a score below 50 by Waterloo's spam classifier were filtered out). SVMrank with 130 features was used to re-rank the top 1000 documents in the initial list. Then, the top 50 documents were clustered using the nearest neighbor clustering approach, and the ClustMRF cluster ranking method was used to rank the clusters. Finally, the ranking of clusters was transformed to a ranking of documents. The optimization metric was MAP.

clustmrfbf¶

Run ID: clustmrfbf
Participant: Technion
Track: Web
Year: 2013
Submission: 8/5/2013
Type: automatic
Task: adhoc
MD5: 4cdcd45c39e78b89386431bcaed32368
Run description: We created an initial list of documents using the MRF retrieval method (documents assigned with a score below 50 by Waterloo's spam classifier were filtered out). SVMrank with 130 features was used to re-rank the top 1000 documents in the initial list. Then, the top 50 documents were clustered using the nearest neighbor clustering approach, and the ClustMRF cluster ranking method was used to rank the clusters. Finally, the ranking of clusters was transformed to a ranking of documents. The optimization metric was MAP.

cwiwt13cpe¶

Run ID: cwiwt13cpe
Participant: CWI
Track: Web
Year: 2013
Submission: 8/4/2013
Type: automatic
Task: adhoc
MD5: 6f03ab7d9c6345d5c17468bb797fd575
Run description: Proximity Model

cwiwt13cps¶

Run ID: cwiwt13cps
Participant: CWI
Track: Web
Year: 2013
Submission: 8/4/2013
Type: automatic
Task: adhoc
MD5: 1f866791889406bca1e951ebc780e978
Run description: Proximity Model that uses stop words

cwiwt13kld¶

Run ID: cwiwt13kld
Participant: CWI
Track: Web
Year: 2013
Submission: 8/4/2013
Type: automatic
Task: adhoc
MD5: fed0b3a3088c57687ecddfda4b8ed645
Run description: KLD baseline

dlde¶

Run ID: dlde
Participant: DLDE
Track: Web
Year: 2013
Submission: 8/7/2013
Type: manual
Task: adhoc
MD5: 6ead4eb26521df101d326e9d64e39d44
Run description: we use google web search as additional resource to expend the query.

ICTNET13ADR1¶

Run ID: ICTNET13ADR1
Participant: ICTNET
Track: Web
Year: 2013
Submission: 8/4/2013
Type: automatic
Task: adhoc
MD5: 200a5dba16a693c18e1b9706362da958
Run description: consider "anchor perfect match" and "TPBF" at the same time, finally filled with ICTNET13ADR3. "TPBF" means "title perfect match" and "body and with the window size of 10" at the same time.

ICTNET13ADR2¶

Run ID: ICTNET13ADR2
Participant: ICTNET
Track: Web
Year: 2013
Submission: 8/4/2013
Type: automatic
Task: adhoc
MD5: 704ec11f751ca912bcbf153f64ce6ab6
Run description: Considerate "anchor perfect match" and "title and" at the same time, finally filled with the ICTNET13ADR3.

ICTNET13ADR3¶

Run ID: ICTNET13ADR3
Participant: ICTNET
Track: Web
Year: 2013
Submission: 8/2/2013
Type: automatic
Task: adhoc
MD5: 9e4158490b5276226033d1e2e8df516c
Run description: Only BM25 was applied on body field with And operator, and filled with results using Or operator when results is less than 10000. Note that only the docs whose Spam score is greater than 70 are retrieved.

ICTNET13RSR1¶

Run ID: ICTNET13RSR1
Participant: ICTNET
Track: Web
Year: 2013
Submission: 8/4/2013
Type: automatic
Task: risk
MD5: 89b79a155b7f84a3156738441834b44a
Run description: consider "anchor perfect match" and "TOBFNE" at the same time, finally filled with ICTNET13ADR3. "TOBFNE": remove stop words. apply Bm25(or) in title field; apply Bm25(And) in body field with window size of 10 when there are two words in topic; using named entity recognization to optimize search query in body field.

ICTNET13RSR2¶

Run ID: ICTNET13RSR2
Participant: ICTNET
Track: Web
Year: 2013
Submission: 8/4/2013
Type: automatic
Task: risk
MD5: ecdc2f3f4fee09a4b852d9d5b9e72707
Run description: Bm25(And) in anchor and than Bm25(Or) without stop words in title, finally filled with ICTNET13ADR3.

ICTNET13RSR3¶

Run ID: ICTNET13RSR3
Participant: ICTNET
Track: Web
Year: 2013
Submission: 8/4/2013
Type: automatic
Task: risk
MD5: bd3ec343ad47ba6b2be2d93ef2f3ac6a
Run description: perfect match in title, than perfect match in anchor(stemmed), finally filled with ICTNET13ADR3.

mmrbf¶

Run ID: mmrbf
Participant: Technion
Track: Web
Year: 2013
Submission: 8/5/2013
Type: automatic
Task: adhoc
MD5: 1ccadc8cf4d43d2e180467cffc658abd
Run description: We created an initial list of documents using the MRF retrieval method (documents assigned with a score below 50 by Waterloo's spam classifier were filtered out). SVMrank with 130 features was used to re-rank the top 1000 documents in the initial list. Then, the top 50 documents were re-ranked using the MMR method for diversifying search results. To estimate the similarity between the query and a document in MMR, the top 50 documents were clustered using the nearest neighbor clustering approach, and the ClustMRF cluster ranking method was used to rank the clusters.

msr_alpha0¶

Run ID: msr_alpha0
Participant: MSR_Redmond
Track: Web
Year: 2013
Submission: 8/8/2013
Type: manual
Task: adhoc
MD5: 627b8308471c7682aecc2b46ba50c93b
Run description: We used a English Wikipedia dump as a corpus to generate query expansion terms. The expansion terms were further filtered through a crowdsourced process.

msr_alpha0_95_4¶

Run ID: msr_alpha0_95_4
Participant: MSR_Redmond
Track: Web
Year: 2013
Submission: 8/8/2013
Type: manual
Task: adhoc
MD5: f5e5d335c31928f0ce8a8b4591b1eb5e
Run description: We used a English Wikipedia dump as a corpus to generate query expansion terms. The expansion terms were further filtered through a crowdsourced process.

msr_alpha1¶

Run ID: msr_alpha1
Participant: MSR_Redmond
Track: Web
Year: 2013
Submission: 8/8/2013
Type: manual
Task: risk
MD5: a55cd365c494c284f0b8140141f73f27
Run description: We used a English Wikipedia dump as a corpus to generate query expansion terms. The expansion terms were further filtered through a crowdsourced process.

msr_alpha10¶

Run ID: msr_alpha10
Participant: MSR_Redmond
Track: Web
Year: 2013
Submission: 8/8/2013
Type: manual
Task: risk
MD5: 16ae2bb77a65161649fd1170a4c41d3c
Run description: We used a English Wikipedia dump as a corpus to generate query expansion terms. The expansion terms were further filtered through a crowdsourced process.

msr_alpha5¶

Run ID: msr_alpha5
Participant: MSR_Redmond
Track: Web
Year: 2013
Submission: 8/8/2013
Type: manual
Task: risk
MD5: b91fe3074ef63a1e10628464a977befa
Run description: We used a English Wikipedia dump as a corpus to generate query expansion terms. The expansion terms were further filtered through a crowdsourced process.

RMITSC¶

Run ID: RMITSC
Participant: RMIT
Track: Web
Year: 2013
Submission: 8/8/2013
Type: automatic
Task: risk
MD5: 75217762b3f4add4838afb0fadab00c4
Run description: This run generated using score distribution of initial retrieval.

RMITSC75¶

Run ID: RMITSC75
Participant: RMIT
Track: Web
Year: 2013
Submission: 8/8/2013
Type: automatic
Task: risk
MD5: 2b80f3ee6356f76e58713d0367e51a64
Run description: This run generated using score distribution of initial retrieval.

RMITSCTh¶

Run ID: RMITSCTh
Participant: RMIT
Track: Web
Year: 2013
Submission: 8/8/2013
Type: automatic
Task: risk
MD5: 7a10e004e09ab7bff3656722e3c1ded0
Run description: This run generated using score distribution of initial retrieval.

udelCombUD¶

Run ID: udelCombUD
Participant: udel
Track: Web
Year: 2013
Submission: 8/7/2013
Type: automatic
Task: adhoc
MD5: d5630ad7df3d77a3daa7c45c970b25ea
Run description: Ranking Aggregation method that uses Bing related queries

udelManExp¶

Run ID: udelManExp
Participant: udel
Track: Web
Year: 2013
Submission: 8/8/2013
Type: manual
Task: risk
MD5: 696d342b97a296c7a51ef8d9b97a59c3
Run description: Manual Query Expansion

udelPseudo1¶

Run ID: udelPseudo1
Participant: udel
Track: Web
Year: 2013
Submission: 8/8/2013
Type: automatic
Task: adhoc
MD5: 7c685f84f9938c92a913d5192f772a98
Run description: pseudo-relevance feedback run where documents that have the highest cosine similarity with the top 5 docs returned by an aggregated run are promoted

udelPseudo1LM¶

Run ID: udelPseudo1LM
Participant: udel
Track: Web
Year: 2013
Submission: 8/8/2013
Type: automatic
Task: risk
MD5: c1c600f15587ef3ea8c2f8fbcb3be4f7
Run description: Rank aggregation method with LM as the baseline

udelPseudo2¶

Run ID: udelPseudo2
Participant: udel
Track: Web
Year: 2013
Submission: 8/8/2013
Type: automatic
Task: adhoc
MD5: bdb6e970077ce131e71f8e3641cc3de9
Run description: pseudo-relevance feedback run where documents that have the highest cosine similarity with the top 5 docs returned by an external search engine are promoted

udemFbWikiR¶

Run ID: udemFbWikiR
Participant: diro_web_13
Track: Web
Year: 2013
Submission: 8/7/2013
Type: automatic
Task: risk
MD5: cd64a9ca64ea11c7350783f0d0659cd9
Run description: Run of Quantum Language Models with window size of 1 and smoothing parameter to the default Indri value. Pseudo-Relevance Feedback (Indri Relevance Model) made on Wikipedia, with 5 feedback documents, 10 expansion terms and a combination weight 0.6. Spam filtering level is set to 30.

udemQlm1l¶

Run ID: udemQlm1l
Participant: diro_web_13
Track: Web
Year: 2013
Submission: 8/5/2013
Type: automatic
Task: adhoc
MD5: 1cbf65c87d68e05bf5b6ba1a60f9ec29
Run description: Vanilla run of Quantum Language Models with window size of 1 and smoothing parameter to the default Indri value. Spam filtering level is set to 30.

udemQlm1lFb¶

Run ID: udemQlm1lFb
Participant: diro_web_13
Track: Web
Year: 2013
Submission: 8/5/2013
Type: automatic
Task: adhoc
MD5: 858f4c65c7a3120789a590e2f5a8e1e6
Run description: Run of Quantum Language Models with window size of 1 and smoothing parameter to the default Indri value. Pseudo-Relevance Feedback (Indri Relevance Model) with 10 feedback documents, 10 expansion terms and a combination weight 0.8. Spam filtering level is set to 30.

udemQlm1lFbWiki¶

Run ID: udemQlm1lFbWiki
Participant: diro_web_13
Track: Web
Year: 2013
Submission: 8/5/2013
Type: automatic
Task: adhoc
MD5: 12e4928a897b93d03d1abd6d56dd18b2
Run description: Run of Quantum Language Models with window size of 1 and smoothing parameter to the default Indri value. Pseudo-Relevance Feedback (Indri Relevance Model) made on Wikipedia, with 5 feedback documents, 10 expansion terms and a combination weight 0.6. Spam filtering level is set to 30.

udemQlml1FbR¶

Run ID: udemQlml1FbR
Participant: diro_web_13
Track: Web
Year: 2013
Submission: 8/7/2013
Type: automatic
Task: risk
MD5: 5a6bcf6018a5ec21cc3a1c7c5621b401
Run description: Run of Quantum Language Models with window size of 1 and smoothing parameter to the default Indri value. Pseudo-Relevance Feedback (Indri Relevance Model) with 10 feedback documents, 10 expansion terms and a combination weight 0.8. Spam filtering level is set to 70.

udemQlml1R¶

Run ID: udemQlml1R
Participant: diro_web_13
Track: Web
Year: 2013
Submission: 8/7/2013
Type: automatic
Task: risk
MD5: b525356c4b2c2f8ce7496f02b43013f1
Run description: Run of Quantum Language Models with window size of 1 and smoothing parameter to the default Indri value. pam filtering level is set to 30.

UDInfolabWEB1¶

Run ID: UDInfolabWEB1
Participant: udel_fang
Track: Web
Year: 2013
Submission: 8/5/2013
Type: automatic
Task: adhoc
MD5: cf7e7e645ada78247df3475f361835bd
Run description: An axiomatic retrieval model in addition with semantic term matching based query expansion is used to do the retrieval. No external resources used.

UDInfolabWEB1R¶

Run ID: UDInfolabWEB1R
Participant: udel_fang
Track: Web
Year: 2013
Submission: 8/5/2013
Type: automatic
Task: risk
MD5: cf7e7e645ada78247df3475f361835bd
Run description: An axiomatic retrieval model in addition with semantic term matching based query expansion is used to do the retrieval. No external resources used.

UDInfolabWEB2¶

Run ID: UDInfolabWEB2
Participant: udel_fang
Track: Web
Year: 2013
Submission: 8/5/2013
Type: automatic
Task: adhoc
MD5: bfcfce6d55f70db61fbbe75f0559c1c9
Run description: An axiomatic retrieval model in addition with semantic term matching based query expansion is used to do the retrieval. Semantic terms are selected from snippets of commercial search engines.

UDInfolabWEB2R¶

Run ID: UDInfolabWEB2R
Participant: udel_fang
Track: Web
Year: 2013
Submission: 8/5/2013
Type: automatic
Task: risk
MD5: bfcfce6d55f70db61fbbe75f0559c1c9
Run description: An axiomatic retrieval model in addition with semantic term matching based query expansion is used to do the retrieval. Semantic terms are selected from snippets of commercial search engines.

UJS13LCRAd1¶

Run ID: UJS13LCRAd1
Participant: UJS
Track: Web
Year: 2013
Submission: 7/30/2013
Type: automatic
Task: adhoc
MD5: 7a170745984f75ea8c70dd5ea8771bf2
Run description: We use data fusion to get the result. LCR(Linear Combination with Regression) method is used to combine the raw retrieval results of three component systems with different weights from training. The three information retrieval systems are indri, terrier and mg4j. All ranklists of them are automaticlly achieved, and then they are filtered with top 15 percentage spam by the result of Waterloo to be a component parts.

UJS13LCRAd2¶

Run ID: UJS13LCRAd2
Participant: UJS
Track: Web
Year: 2013
Submission: 7/30/2013
Type: automatic
Task: adhoc
MD5: a806d6c0960d650a33916cadc496ee00
Run description: We use data fusion to get the result. LCR(Linear Combination with Regression) method is used to combine the raw retrieval results of three component systems with different weights from training. The three information retrieval systems are indri, terrier and mg4j, and all ranklists of them are automaticlly achieved without any manual activity.

UJS13Risk1¶

Run ID: UJS13Risk1
Participant: UJS
Track: Web
Year: 2013
Submission: 7/30/2013
Type: automatic
Task: risk
MD5: ac90a1e2a448d12b4675c3a320cb6247
Run description: To risk-sensitive task, four retrieval systems are used. We also use LCR(Linear Combination with Regression) data fusion method to combine results from indri, terrier, mg4j and a baseline run, which are filtered with top 10 percentage spam according to the result of Waterloo. We do not use the risk parameter Alpha to optimize this run .

UJS13Risk2¶

Run ID: UJS13Risk2
Participant: UJS
Track: Web
Year: 2013
Submission: 7/30/2013
Type: automatic
Task: risk
MD5: e33bbe8c09f38d5ffeb7d525695026ec
Run description: To risk-sensitive task, four retrieval systems are used. We also use LCR(Linear Combination with Regression) data fusion method to combine unfiltered results from indri, terrier, mg4j and a baseline run. We do not use the risk parameter Alpha to optimize this run .

uogTrADnLrb¶

Run ID: uogTrADnLrb
Participant: uogTr
Track: Web
Year: 2013
Submission: 8/8/2013
Type: automatic
Task: risk
MD5: 2f9dae25bda3732066891b2479c03f7f
Run description: Divergence from Randomness, Fat & QI generated features (63 total), state-of-the-art learning-to-rank encompassing risk.

uogTrAIwLmb¶

Run ID: uogTrAIwLmb
Participant: uogTr
Track: Web
Year: 2013
Submission: 8/7/2013
Type: automatic
Task: adhoc
MD5: e56705f5d97785553e93ea42276bf74a
Run description: Divergence from Independence, Fat & QI generated features (63 total), state-of-the-art learning-to-rank

uogTrAS1Lb¶

Run ID: uogTrAS1Lb
Participant: uogTr
Track: Web
Year: 2013
Submission: 8/8/2013
Type: automatic
Task: risk
MD5: 43a85176de06675c956a31601b6d2c13
Run description: Divergence from Randomness, Fat & QI generated features (63 total), state-of-the-art learning-to-rank. Selective approach to reduce risk.

uogTrAS2Lb¶

Run ID: uogTrAS2Lb
Participant: uogTr
Track: Web
Year: 2013
Submission: 8/8/2013
Type: automatic
Task: adhoc
MD5: 0ba5e9af712cb2b697aa77496d596132
Run description: Divergence from Randomness, Fat & QI generated features (63 total), state-of-the-art learning-to-rank techniques. A selective approach reduces risk.

uogTrBDnLaxw¶

Run ID: uogTrBDnLaxw
Participant: uogTr
Track: Web
Year: 2013
Submission: 8/8/2013
Type: automatic
Task: risk
MD5: 7a0069c7a700868d6ae4051c3c802618
Run description: Divergence from Randomness, Fat & QI generated features (63 total), state-of-the-art learning-to-rank and diversification.

uogTrBDnLmxw¶

Run ID: uogTrBDnLmxw
Participant: uogTr
Track: Web
Year: 2013
Submission: 8/8/2013
Type: automatic
Task: adhoc
MD5: 570bea3e5d9f2418e041bff23b8d93a0
Run description: Divergence from Randomness, Fat & QI generated features (63 total), state-of-the-art learning-to-rank and diversification

ut22base¶

Run ID: ut22base
Participant: ut
Track: Web
Year: 2013
Submission: 8/7/2013
Type: automatic
Task: adhoc
MD5: 0171de6d393c116036b5e6d49ea7c4b4
Run description: MIREX 0.3 baseline run language models with lambda = 0.9 on anchors

ut22spam¶

Run ID: ut22spam
Participant: ut
Track: Web
Year: 2013
Submission: 8/7/2013
Type: automatic
Task: adhoc
MD5: 6f3836b75dd48d39b29813d42c04f781
Run description: MIREX 0.3 baseline run language models with lambda = 0.9 on anchors with 50% Waterloo spam removed

ut22xact¶

Run ID: ut22xact
Participant: ut
Track: Web
Year: 2013
Submission: 8/7/2013
Type: automatic
Task: adhoc
MD5: 566ed6cc758f2250ce8650ef61c18099
Run description: MIREX 0.3 rank by number of exact query string matches in anchors

UWCWEB13RISK01¶

Run ID: UWCWEB13RISK01
Participant: UWaterlooCLAC
Track: Web
Year: 2013
Submission: 8/5/2013
Type: automatic
Task: risk
MD5: cd5e0cf75f9f6fa961b4f06bce963d79
Run description: We use the relevance feedback (rm3) method based on document weighting strategy to re-rank the baseline run. We use the top 20 documents appeared in the baseline run as feedback documents and use the rm3 method to select the top 20 terms to expand original query.

UWCWEB13RISK02¶

Run ID: UWCWEB13RISK02
Participant: UWaterlooCLAC
Track: Web
Year: 2013
Submission: 8/7/2013
Type: automatic
Task: risk
MD5: a433664a974335b4aafc582c90d1573f
Run description: We use the relevance feedback (rm3) method based on document weighting strategy to re-rank the baseline run. We use the top 20 documents appeared in the baseline run as feedback documents and use the rm3 method to select the top 20 terms to expand original query. For both runs, we use an index of the subset of ClueWeb2012 that only contains documents in the provided baseline runs (rm and ql).

webishybrid¶

Run ID: webishybrid
Participant: webis
Track: Web
Year: 2013
Submission: 8/2/2013
Type: automatic
Task: risk
MD5: 8512fc1c2d650eba7caef4df7b023a60
Run description: Segmentations may contain sources from Wikipedia.

webismixed¶

Run ID: webismixed
Participant: webis
Track: Web
Year: 2013
Submission: 8/2/2013
Type: automatic
Task: adhoc
MD5: 0dcf24dad0ff2d86d652df82652a25f3
Run description: Segmentations may contain sources from Wikipedia.

webisnaive¶

Run ID: webisnaive
Participant: webis
Track: Web
Year: 2013
Submission: 8/2/2013
Type: automatic
Task: risk
MD5: d4f8e0e1b719e242957786823474968f
Run description: Segmentations may contain sources from Wikipedia.

webisrandom¶

Run ID: webisrandom
Participant: webis
Track: Web
Year: 2013
Submission: 8/2/2013
Type: automatic
Task: adhoc
MD5: eb6cfca62a4eb6fd0b2de61f9f83c3ea
Run description: Segmentations may contain sources from Wikipedia.

webiswikibased¶

Run ID: webiswikibased
Participant: webis
Track: Web
Year: 2013
Submission: 8/2/2013
Type: automatic
Task: risk
MD5: 439bc8d635973a4c587ab86dca890201
Run description: Segmentations may contain sources from Wikipedia.

webiswtbaseline¶

Run ID: webiswtbaseline
Participant: webis
Track: Web
Year: 2013
Submission: 8/2/2013
Type: automatic
Task: adhoc
MD5: 322074e15a89e7d588f07b98b510b5e4
Run description: Segmentations may contain sources from Wikipedia.

wistud.runA¶

Run ID: wistud.runA
Participant: wistud
Track: Web
Year: 2013
Submission: 8/7/2013
Type: automatic
Task: adhoc
MD5: e5cf141dfebbd93aebc88e16cb0f1aef
Run description: Standard query likelihood run with spam filtering and de-duplication. The spam scores of UWaterloo have been used.

wistud.runB¶

Run ID: wistud.runB
Participant: wistud
Track: Web
Year: 2013
Submission: 8/7/2013
Type: automatic
Task: adhoc
MD5: d9dfc95a931f46c3b7aef93c167a2659
Run description: Standard query likelihood run with spam filtering and de-duplication. The spam scores of UWaterloo have been used. In contrast to wistud.runA, a different preprocessing pipeline was used.

wistud.runC¶

Run ID: wistud.runC
Participant: wistud
Track: Web
Year: 2013
Submission: 8/7/2013
Type: automatic
Task: adhoc
MD5: b326316c58c567bcc62d28bedf045e80
Run description: query likelihood run with strong spam filtering (UWaterloo spam scores)

wistud.runD¶

Run ID: wistud.runD
Participant: wistud
Track: Web
Year: 2013
Submission: 8/8/2013
Type: manual
Task: risk
MD5: 505078af485e909a52b88a023bfeaae7
Run description: queries expanded where possible with Wikipedia concepts.