Skip to content

Proceedings - Session 2011

Overview of the TREC 2011 Session Track

Evangelos Kanoulas, Mark M. Hall, Paul D. Clough, Ben Carterette, Mark Sanderson

Abstract

The TREC Session track ran for the second time in 2011. The track has the primary goal of providing test collections and evaluation measures for studying information retrieval over user sessions rather than one-time queries. These test collections are meant to be portable, reusable, statistically powerful, and open to anyone that wishes to work on the problem of retrieval over sessions. The second year has seen a near-complete overhaul of the track in terms of topic design, session data, and experimental evaluation. The changes are: 1. topics were formed from real user sessions with a search engine, and include queries, retrieved results, clicks, and dwell times; 2. retrieval tasks designed to study the effect of using increasing amounts of user data on retrieval effectiveness for the mth query in a session; 3. subtopic relevance judgments similar to the Web track diversity task. We believe the resulting test collection better models the interaction between system and user, though there is certainly still room for improvement. This overview is organized as follows: in Section 2 we describe the tasks participants were to perform. In Section 3 we describe the corpus, topics, and sessions that comprise the test collection. Section 4 gives some information about submitted runs. In Section 5 we describe relevance judging and evaluation measures, and Sections 6 and 7 present evaluation results and analysis. We conclude in Section 8 with some directions for the 2012 Session track.

Bibtex 
@inproceedings{DBLP:conf/trec/KanoulasHCCS11,
    author = {Evangelos Kanoulas and Mark M. Hall and Paul D. Clough and Ben Carterette and Mark Sanderson},
    editor = {Ellen M. Voorhees and Lori P. Buckland},
    title = {Overview of the {TREC} 2011 Session Track},
    booktitle = {Proceedings of The Twentieth Text REtrieval Conference, {TREC} 2011, Gaithersburg, Maryland, USA, November 15-18, 2011},
    series = {{NIST} Special Publication},
    volume = {500-296},
    publisher = {National Institute of Standards and Technology {(NIST)}},
    year = {2011},
    url = {http://trec.nist.gov/pubs/trec20/papers/SESSION.OVERVIEW.2011.pdf},
    timestamp = {Wed, 07 Dec 2022 00:00:00 +0100},
    biburl = {https://dblp.org/rec/conf/trec/KanoulasHCCS11.bib},
    bibsource = {dblp computer science bibliography, https://dblp.org}
}

RGU-ISTI-Essex at TREC 2011 Session Track

Ibrahim Adeyanju, Dawei Song, Franco Maria Nardini, M-Dyaa Albakour, Udo Kruschwitz

Abstract

Mining query recommendation from query logs has attracted a lot of attention in recent years. We propose to use query recommendations extracted from the logs of a web search engine to solve the session track tasks. The runs are obtained by using the Search Shortcuts recommender system. The Search Shortcuts technique uses an inverted index and the concept of “successful sessions” present in a web search engine's query log to produce effective recommendations for both frequent and rare/unseen queries. We adapt the above technique as a query expansion tool and use it to expand the given queries for Session Track at TREC 2011. The expansion is generated by using a method which aims to consider all past queries in the session. The expansion terms obtained are then used to build a global, uniformly weighted, representation of the user session (RL2). Furthermore, the expansion terms are then combined with a ranked list of results in order to boost terms appearing more frequently in the final results lists (RL3). Finally, we also integrate dwell times and the weighting method obtained taking both result lists and clicks into account for assigning weights to the terms to expand the final query of the session. In addition to that, we submitted a baseline run. It is based on the observation that using the term “wikipedia” to expand the query resulted in a better retrieval performance for the tasks at last year's session track at TREC 2010.

Bibtex 
@inproceedings{DBLP:conf/trec/AdeyanjuSNAK11,
    author = {Ibrahim Adeyanju and Dawei Song and Franco Maria Nardini and M{-}Dyaa Albakour and Udo Kruschwitz},
    editor = {Ellen M. Voorhees and Lori P. Buckland},
    title = {RGU-ISTI-Essex at {TREC} 2011 Session Track},
    booktitle = {Proceedings of The Twentieth Text REtrieval Conference, {TREC} 2011, Gaithersburg, Maryland, USA, November 15-18, 2011},
    series = {{NIST} Special Publication},
    volume = {500-296},
    publisher = {National Institute of Standards and Technology {(NIST)}},
    year = {2011},
    url = {http://trec.nist.gov/pubs/trec20/papers/RGU.session.update.pdf},
    timestamp = {Wed, 03 Feb 2021 08:31:23 +0100},
    biburl = {https://dblp.org/rec/conf/trec/AdeyanjuSNAK11.bib},
    bibsource = {dblp computer science bibliography, https://dblp.org}
}

University of Essex at the TREC 2011 Session Track

M-Dyaa Albakour, Udo Kruschwitz, Brendan Neville, Deirdre Lungley, Maria Fasli, Nikolaos Nanas

Abstract

This paper provides an overview of the experiments we carried out at the TREC 2011 Session Track. We propose two different approaches to tackle the task introduced this year. The first one relies on a biologically inspired adaptive model for information filtering to build a user profile of multiple topics of interests throughout the session. The learnt profile is then exploited in the retrieval process. The second approach is an extension of our anchor log technique we proposed in the previous year. We use the anchor logs to simulate queries in order to derive query expansions that are relevant to user information needs throughout the session.

Bibtex 
@inproceedings{DBLP:conf/trec/AlbakourKNLFN11,
    author = {M{-}Dyaa Albakour and Udo Kruschwitz and Brendan Neville and Deirdre Lungley and Maria Fasli and Nikolaos Nanas},
    editor = {Ellen M. Voorhees and Lori P. Buckland},
    title = {University of Essex at the {TREC} 2011 Session Track},
    booktitle = {Proceedings of The Twentieth Text REtrieval Conference, {TREC} 2011, Gaithersburg, Maryland, USA, November 15-18, 2011},
    series = {{NIST} Special Publication},
    volume = {500-296},
    publisher = {National Institute of Standards and Technology {(NIST)}},
    year = {2011},
    url = {http://trec.nist.gov/pubs/trec20/papers/essexuni.session.update.pdf},
    timestamp = {Thu, 12 Mar 2020 00:00:00 +0100},
    biburl = {https://dblp.org/rec/conf/trec/AlbakourKNLFN11.bib},
    bibsource = {dblp computer science bibliography, https://dblp.org}
}

Implicit Feedback and Document Filtering for Retrieval Over Query Sessions

Ben Carterette, Praveen Chandar

Abstract

The IR Lab at the University of Delaware participated in the 2011 Sessions track. The Sessions track features sequences of queries q 1 , ..., qm , with only q m being the subject for automatic retrieval. There are four separate experimental conditions for q m , each with a greater amount of data about user/system interaction for prior queries: 1. RL1: no interaction information; q m only. 2. RL2: previous queries q 1 , ..., qm−1 known to the system. 3. RL3: previous queries and retrieved results known to the system. 4. RL4: previous queries, retrieved results, and clicks on retrieved results known to the system. We used the different experimental conditions in the track to explore three research questions: 1. the effect of simple implicit feedback on retrieval results; 2. the effect of corpus filters on retrieval results; 3. the effect of duplicate detection and removal on retrieval results.

Bibtex 
@inproceedings{DBLP:conf/trec/CarteretteC11,
    author = {Ben Carterette and Praveen Chandar},
    editor = {Ellen M. Voorhees and Lori P. Buckland},
    title = {Implicit Feedback and Document Filtering for Retrieval Over Query Sessions},
    booktitle = {Proceedings of The Twentieth Text REtrieval Conference, {TREC} 2011, Gaithersburg, Maryland, USA, November 15-18, 2011},
    series = {{NIST} Special Publication},
    volume = {500-296},
    publisher = {National Institute of Standards and Technology {(NIST)}},
    year = {2011},
    url = {http://trec.nist.gov/pubs/trec20/papers/udel.session.pdf},
    timestamp = {Thu, 12 Mar 2020 00:00:00 +0100},
    biburl = {https://dblp.org/rec/conf/trec/CarteretteC11.bib},
    bibsource = {dblp computer science bibliography, https://dblp.org}
}

Webis at the TREC 2011 Session Track

Matthias Hagen, Jan Graßegger, Maximilian Michel, Benno Stein

Abstract

In this paper we give a brief overview of the Webis group's participation in the TREC 2011 Sessions track with an extended version of our last year's approach [HSV10]. The basic idea can be described as a conservative query expansion based on terms used in previous queries or terms contained in clicked snippets. Furthermore, a query's result set is reduced by removing documents shown for previous queries or documents containing important terms from non- clicked snippets.

Bibtex 
@inproceedings{DBLP:conf/trec/HagenGMS11,
    author = {Matthias Hagen and Jan Gra{\ss}egger and Maximilian Michel and Benno Stein},
    editor = {Ellen M. Voorhees and Lori P. Buckland},
    title = {Webis at the {TREC} 2011 Session Track},
    booktitle = {Proceedings of The Twentieth Text REtrieval Conference, {TREC} 2011, Gaithersburg, Maryland, USA, November 15-18, 2011},
    series = {{NIST} Special Publication},
    volume = {500-296},
    publisher = {National Institute of Standards and Technology {(NIST)}},
    year = {2011},
    url = {http://trec.nist.gov/pubs/trec20/papers/Webis.session.update.pdf},
    timestamp = {Thu, 12 Mar 2020 00:00:00 +0100},
    biburl = {https://dblp.org/rec/conf/trec/HagenGMS11.bib},
    bibsource = {dblp computer science bibliography, https://dblp.org}
}

The University of Amsterdam at the TREC 2011 Session Track

Bouke Huurnink, Richard Berendsen, Katja Hofmann, Edgar Meij, Maarten de Rijke

Abstract

We describe the participation of the University of Amsterdam's ILPS group in the Session track at TREC 2011.

Bibtex 
@inproceedings{DBLP:conf/trec/HuurninkBHMR11,
    author = {Bouke Huurnink and Richard Berendsen and Katja Hofmann and Edgar Meij and Maarten de Rijke},
    editor = {Ellen M. Voorhees and Lori P. Buckland},
    title = {The University of Amsterdam at the {TREC} 2011 Session Track},
    booktitle = {Proceedings of The Twentieth Text REtrieval Conference, {TREC} 2011, Gaithersburg, Maryland, USA, November 15-18, 2011},
    series = {{NIST} Special Publication},
    volume = {500-296},
    publisher = {National Institute of Standards and Technology {(NIST)}},
    year = {2011},
    url = {http://trec.nist.gov/pubs/trec20/papers/UvA.session.update.pdf},
    timestamp = {Thu, 12 Mar 2020 00:00:00 +0100},
    biburl = {https://dblp.org/rec/conf/trec/HuurninkBHMR11.bib},
    bibsource = {dblp computer science bibliography, https://dblp.org}
}

PITT at TREC 2011 Session Track

Jiepu Jiang, Shuguang Han, Jia Wu, Daqing He

Abstract

In this paper, we introduce our approaches for TREC 2011 session track. Our approaches focus on combining different query language models to model information needs in a search session. In RL1 stage, we build ad hoc retrieval system using sequential dependence model (SDM) on current query. In RL2 stage, we build query language models by combining SDM features (e.g. single term, ordered phrase, and unordered phrase) in both current query and previous queries in the session, which can significantly improve search performance. In RL3 and RL4, we combine query model in RL2 with two different pseudo-relevance feedback query models: in RL3, we use top ranked Wikipedia documents from RL2's results as pseudo-relevant documents; in RL4, snippets of the documents clicked by users in a search session are used. Our evaluation results indicate: texts of previous queries in a session are effective resources for estimating query models and improving search performance; mixing query model in RL2 with the query model estimated using click-through data (in RL4) can improve performance in evaluation setting that considers all subtopics, but no improvement is observed in evaluation setting that considers the only subtopic of current query; our methods of mixing query model in RL2 with query model in RL3 did not improve search performance over RL2 in any of the two evaluation settings.

Bibtex 
@inproceedings{DBLP:conf/trec/JiangHWH11,
    author = {Jiepu Jiang and Shuguang Han and Jia Wu and Daqing He},
    editor = {Ellen M. Voorhees and Lori P. Buckland},
    title = {{PITT} at {TREC} 2011 Session Track},
    booktitle = {Proceedings of The Twentieth Text REtrieval Conference, {TREC} 2011, Gaithersburg, Maryland, USA, November 15-18, 2011},
    series = {{NIST} Special Publication},
    volume = {500-296},
    publisher = {National Institute of Standards and Technology {(NIST)}},
    year = {2011},
    url = {http://trec.nist.gov/pubs/trec20/papers/PITTSIS.session.update.pdf},
    timestamp = {Thu, 12 Mar 2020 00:00:00 +0100},
    biburl = {https://dblp.org/rec/conf/trec/JiangHWH11.bib},
    bibsource = {dblp computer science bibliography, https://dblp.org}
}

RMIT at TREC 2011 Session Track

Lorena Leal Bando, Sadegh Kharazmi, Jasbir Dhaliwal, Mark Sanderson, Falk Scholer, Sargol Sadeghi, Fahad Alahmari

Abstract

The 2011 Session track aims to study retrieval system performance by pro- viding different components in a search session. We report on experiments and results based on query expansion techniques when lists of results are provided with or without clicked information. In contrast, a bag-of-words approach is employed as a baseline.

Bibtex 
@inproceedings{DBLP:conf/trec/LealKDSSSA11,
    author = {Lorena Leal Bando and Sadegh Kharazmi and Jasbir Dhaliwal and Mark Sanderson and Falk Scholer and Sargol Sadeghi and Fahad Alahmari},
    editor = {Ellen M. Voorhees and Lori P. Buckland},
    title = {{RMIT} at {TREC} 2011 Session Track},
    booktitle = {Proceedings of The Twentieth Text REtrieval Conference, {TREC} 2011, Gaithersburg, Maryland, USA, November 15-18, 2011},
    series = {{NIST} Special Publication},
    volume = {500-296},
    publisher = {National Institute of Standards and Technology {(NIST)}},
    year = {2011},
    url = {http://trec.nist.gov/pubs/trec20/papers/RMIT.session.update.pdf},
    timestamp = {Thu, 12 Mar 2020 00:00:00 +0100},
    biburl = {https://dblp.org/rec/conf/trec/LealKDSSSA11.bib},
    bibsource = {dblp computer science bibliography, https://dblp.org}
}

DUTIR at the Session Track in TREC 2011

Wenfei Liu, Hongfei Lin, Yunlong Ma, Tianshu Chang

Abstract

This paper presents the DUTIR submission to TREC 2011 Session Track, the task is to test whether systems can improve their performance for a given query by using previous queries and user interactions with the retrieval system. We use language model as the basic retrieval model. Query Expansion is applied to reformulate the original query. Some other technologies like Pseudo Relevance Feedback (PRF) are also applied. The results show that our methods are effective, but the results of Query Expansion method using previous queries and user interactions are unanticipated.

Bibtex 
@inproceedings{DBLP:conf/trec/LiuLMC11,
    author = {Wenfei Liu and Hongfei Lin and Yunlong Ma and Tianshu Chang},
    editor = {Ellen M. Voorhees and Lori P. Buckland},
    title = {{DUTIR} at the Session Track in {TREC} 2011},
    booktitle = {Proceedings of The Twentieth Text REtrieval Conference, {TREC} 2011, Gaithersburg, Maryland, USA, November 15-18, 2011},
    series = {{NIST} Special Publication},
    volume = {500-296},
    publisher = {National Institute of Standards and Technology {(NIST)}},
    year = {2011},
    url = {http://trec.nist.gov/pubs/trec20/papers/DUTIR.session.pdf},
    timestamp = {Thu, 12 Mar 2020 00:00:00 +0100},
    biburl = {https://dblp.org/rec/conf/trec/LiuLMC11.bib},
    bibsource = {dblp computer science bibliography, https://dblp.org}
}

Rutgers at the TREC 2011 Session Track

Chang Liu, Si Sun, Michael J. Cole, Nicholas J. Belkin

Abstract

At Rutgers, we approached the Session Track task as an issue of personalization, based on both the behaviors exhibited by the searcher during the course of an information-seeking episode, and a classification of the task that led the person to engage in information-seeking behavior. Our general approach is described in detail at the Web site of our project, and in the papers available there (http://comminfo.rutgers.edu/imls/poodle); in this section, we give an overview of our approach and how we applied results from our previous studies to the TREC 2011 Session Track. Subsequent sections give details of how we actually did things, our results, and our conclusions about the results. [...]

Bibtex 
@inproceedings{DBLP:conf/trec/LiuSCB11,
    author = {Chang Liu and Si Sun and Michael J. Cole and Nicholas J. Belkin},
    editor = {Ellen M. Voorhees and Lori P. Buckland},
    title = {Rutgers at the {TREC} 2011 Session Track},
    booktitle = {Proceedings of The Twentieth Text REtrieval Conference, {TREC} 2011, Gaithersburg, Maryland, USA, November 15-18, 2011},
    series = {{NIST} Special Publication},
    volume = {500-296},
    publisher = {National Institute of Standards and Technology {(NIST)}},
    year = {2011},
    url = {http://trec.nist.gov/pubs/trec20/papers/SCI\_TREC\_2011.session.pdf},
    timestamp = {Thu, 12 Mar 2020 00:00:00 +0100},
    biburl = {https://dblp.org/rec/conf/trec/LiuSCB11.bib},
    bibsource = {dblp computer science bibliography, https://dblp.org}
}

BUPT_WILDCAT at TREC 2011 Session Track

Tang Liu, Chuang Zhang, Yasi Gao, Wenjun Xiao, Hao Huang

Abstract

his paper is an overview of the runs carried out at TREC 2011 Session track, which proposes several approaches to improve the retrieval performance over one session including the search model based on user behavior, VSM_meta similarity model, optimization based on history ranked lists, optimization based on user‟s attention time and anchor log. The evaluation results show that our implementations are effective.

Bibtex 
@inproceedings{DBLP:conf/trec/LiuZGXH11,
    author = {Tang Liu and Chuang Zhang and Yasi Gao and Wenjun Xiao and Hao Huang},
    editor = {Ellen M. Voorhees and Lori P. Buckland},
    title = {BUPT{\_}WILDCAT at {TREC} 2011 Session Track},
    booktitle = {Proceedings of The Twentieth Text REtrieval Conference, {TREC} 2011, Gaithersburg, Maryland, USA, November 15-18, 2011},
    series = {{NIST} Special Publication},
    volume = {500-296},
    publisher = {National Institute of Standards and Technology {(NIST)}},
    year = {2011},
    url = {http://trec.nist.gov/pubs/trec20/papers/BUPT\_WILDCAT.session.update.pdf},
    timestamp = {Wed, 14 Apr 2021 01:00:00 +0200},
    biburl = {https://dblp.org/rec/conf/trec/LiuZGXH11.bib},
    bibsource = {dblp computer science bibliography, https://dblp.org}
}

ICTNET at Session Track TREC 2011

Mingxhuan Wei, Yuanhai Xue, Chen Xu, Xiaoming Yu, Yue Liu, Xueqi Cheng

Abstract

Following several methods of some past publications, among these we refined an important idea that related queries could be derived from queries submitted within the same session. As the browsers not just provide the previous queries but more informations,so we could use these informations to build a expert database for a special user, base on the database ,we could analysis some fixed behaviors of the user, so then forecast what information he really wants, rerank the results from search engine and give them to the user finally. The whole process is the session trec works on:Providing the really information to different user, so we can make the search engine more efficient and smarter. The rest of paper is structured as follows. In section 2 we discuss the ideal of session type. In section 3 we descripte the classification we used . In section 4 we explain the experiment and result we submitted ,Finally we make a brief conclusion and a plan for the future work.

Bibtex 
@inproceedings{DBLP:conf/trec/WeiXXYLC11,
    author = {Mingxhuan Wei and Yuanhai Xue and Chen Xu and Xiaoming Yu and Yue Liu and Xueqi Cheng},
    editor = {Ellen M. Voorhees and Lori P. Buckland},
    title = {{ICTNET} at Session Track {TREC} 2011},
    booktitle = {Proceedings of The Twentieth Text REtrieval Conference, {TREC} 2011, Gaithersburg, Maryland, USA, November 15-18, 2011},
    series = {{NIST} Special Publication},
    volume = {500-296},
    publisher = {National Institute of Standards and Technology {(NIST)}},
    year = {2011},
    url = {http://trec.nist.gov/pubs/trec20/papers/ICTNET.session.pdf},
    timestamp = {Thu, 12 Mar 2020 00:00:00 +0100},
    biburl = {https://dblp.org/rec/conf/trec/WeiXXYLC11.bib},
    bibsource = {dblp computer science bibliography, https://dblp.org}
}

CWI at TREC 2011: Session, Web, and Medical

Jiyin He, Vera Hollink, Corrado Boscarino, Arjen P. de Vries, Roberto Cornacchia

Abstract

We report on the participation of the Interactive Information Access group of the CWI Amsterdam in the web, session, and medical track at TREC 2011. In the web track we focus on the diversity task. We find that cluster-based subtopic modeling approaches improve diversification performance compared to a non-cluster-based subtopic modeling approach. While gain was observed on previous years' topic sets, diversification with the proposed approaches hurt the performance when compared to a non-diversified baseline run on this year's topic set. In the session track, we examine the effects of differentiating between 'good' and 'bad' users. We find that differentiation is useful as the use of search history appears to be mainly effective when the search is not going well. However, our current strategy is not effective for 'good' users. In addition, we studied the use of random walks on query graphs for formulating session history as search queries, but results are inconclusive. In the medical track, we found that the use of medical background resources for query expansion leads to small improvements in retrieval performance. Such resources appear to be especially useful to promote early precision.

Bibtex 
@inproceedings{DBLP:conf/trec/HeHBVC11,
    author = {Jiyin He and Vera Hollink and Corrado Boscarino and Arjen P. de Vries and Roberto Cornacchia},
    editor = {Ellen M. Voorhees and Lori P. Buckland},
    title = {{CWI} at {TREC} 2011: Session, Web, and Medical},
    booktitle = {Proceedings of The Twentieth Text REtrieval Conference, {TREC} 2011, Gaithersburg, Maryland, USA, November 15-18, 2011},
    series = {{NIST} Special Publication},
    volume = {500-296},
    publisher = {National Institute of Standards and Technology {(NIST)}},
    year = {2011},
    url = {http://trec.nist.gov/pubs/trec20/papers/CWI.session.web.medical.pdf},
    timestamp = {Thu, 12 Mar 2020 00:00:00 +0100},
    biburl = {https://dblp.org/rec/conf/trec/HeHBVC11.bib},
    bibsource = {dblp computer science bibliography, https://dblp.org}
}