Skip to content

Proceedings - Contextual Suggestion 2015

Overview of the TREC 2015 Contextual Suggestion Track

Adriel Dean-Hall, Charles L. A. Clarke, Jaap Kamps, Julia Kiseleva, Ellen M. Voorhees

Abstract

The TREC Contextual Suggestion Track evaluates point-of-interest (POI) recommendation systems, with the goal of creating open and reusable test collections for this purpose. The track imagines a traveler in a unknown city seeking sites to see and things to do that reflect his or her own personal interests, as inferred from their interests in their home city. Given a user's profile, consisting of a POI list and rating from a home city, participants make recommendations for attractions in a target city (i.e., a new context). For example, imagine a group of information retrieval researchers with a November evening to spend in beautiful Gaithersburg, Maryland. A contextual suggestion system might recommend a beer at the Dogfish Head Alehouse, dinner at the Flaming Pit, or even a trip into Washington on the metro to see the National Mall. This is the fourth year that the track has operated (since TREC 2012). If you are familiar with the track from previous years, here are the big changes this year: The track moved from the open web to a fixed set of documents. The track was split into two tasks: 1. A live task, in which participants set up a server and were sent requests over a period of about three weeks. 2. A batch task, which was similar to the task run in previous years. The live task reflects the track's long term goal of creating a “living lab” service for POI recommendation.

Bibtex 
@inproceedings{DBLP:conf/trec/Dean-HallCKKV15,
    author = {Adriel Dean{-}Hall and Charles L. A. Clarke and Jaap Kamps and Julia Kiseleva and Ellen M. Voorhees},
    editor = {Ellen M. Voorhees and Angela Ellis},
    title = {Overview of the {TREC} 2015 Contextual Suggestion Track},
    booktitle = {Proceedings of The Twenty-Fourth Text REtrieval Conference, {TREC} 2015, Gaithersburg, Maryland, USA, November 17-20, 2015},
    series = {{NIST} Special Publication},
    volume = {500-319},
    publisher = {National Institute of Standards and Technology {(NIST)}},
    year = {2015},
    url = {http://trec.nist.gov/pubs/trec24/papers/Overview-CX.pdf},
    timestamp = {Thu, 12 Mar 2020 00:00:00 +0100},
    biburl = {https://dblp.org/rec/conf/trec/Dean-HallCKKV15.bib},
    bibsource = {dblp computer science bibliography, https://dblp.org}
}

Contextual Suggestion using tag-description similarity

Manajit Chakraborty, Hitesh Agrawal, Himanshu Shekhar, C. Ravindranath Chowdary

Abstract

In this paper, we present our approach for the Contextual Suggestion track of 2015 Text REtrieval Conference (TREC). The task aims at providing recommendations on points of attraction for different kind of users and a varying context. Our group DPLAB IITBHU tries to address the problem from the perspective of how relevant the attractions are based on user profiles and rank them based on two similarity measures- wup similarity and another similarity measure proposed by us.

Bibtex 
@inproceedings{DBLP:conf/trec/ChakrabortyASC15,
    author = {Manajit Chakraborty and Hitesh Agrawal and Himanshu Shekhar and C. Ravindranath Chowdary},
    editor = {Ellen M. Voorhees and Angela Ellis},
    title = {Contextual Suggestion using tag-description similarity},
    booktitle = {Proceedings of The Twenty-Fourth Text REtrieval Conference, {TREC} 2015, Gaithersburg, Maryland, USA, November 17-20, 2015},
    series = {{NIST} Special Publication},
    volume = {500-319},
    publisher = {National Institute of Standards and Technology {(NIST)}},
    year = {2015},
    url = {http://trec.nist.gov/pubs/trec24/papers/DPLAB\_IITBHU-CX.pdf},
    timestamp = {Thu, 12 Mar 2020 00:00:00 +0100},
    biburl = {https://dblp.org/rec/conf/trec/ChakrabortyASC15.bib},
    bibsource = {dblp computer science bibliography, https://dblp.org}
}

BJUT at TREC 2015 Contextual Suggestion Track

Weitong Chen, Hanchen Li, Zhen Yang

Abstract

In this paper we described our efforts for TREC contextual suggestion task. Our goal of this year is to evaluate the effectiveness of: (1) predict user preferences of each scenic spot based on non-negtive matrix factorization, (2) automatic summarization method that leverages the information from multiple resources to generate the description for each candidate scenic spots; and (3) hybrid recommendation method that combing a variety of factors to construct a system of hybrid recommendation system. Finally, we conduct extensive experiments to evaluate the proposed framework on TREC 2015 Contextual Suggestion data set, and, as would be expected, the results demonstrate its generality and superior performance.

Bibtex 
@inproceedings{DBLP:conf/trec/ChenLY15,
    author = {Weitong Chen and Hanchen Li and Zhen Yang},
    editor = {Ellen M. Voorhees and Angela Ellis},
    title = {{BJUT} at {TREC} 2015 Contextual Suggestion Track},
    booktitle = {Proceedings of The Twenty-Fourth Text REtrieval Conference, {TREC} 2015, Gaithersburg, Maryland, USA, November 17-20, 2015},
    series = {{NIST} Special Publication},
    volume = {500-319},
    publisher = {National Institute of Standards and Technology {(NIST)}},
    year = {2015},
    url = {http://trec.nist.gov/pubs/trec24/papers/BJUT-CX.pdf},
    timestamp = {Thu, 12 Mar 2020 00:00:00 +0100},
    biburl = {https://dblp.org/rec/conf/trec/ChenLY15.bib},
    bibsource = {dblp computer science bibliography, https://dblp.org}
}

Parsimonious User and Group Profiling in Venue Recommendation

Seyyed Hadi Hashemi, Mostafa Dehghani, Jaap Kamps

Abstract

This paper presents the University of Amsterdam's participation in the TREC 2015 Contextual Suggestion Track. Creating e↵ective profiles for both users and contexts is the main key to build an e↵ective contextual suggestion system. To address these issues, we investigate building users' and groups' profiles for e↵ective contextual personalization and customization. Our main aim is to answer the questions: How to build a user-specific profile that penalizes terms having high probability in negative language models? Can parsimonious language models improve user and context profile's e↵ectiveness? How to combine both models and benefit from both a contextual customization using contextual group profiles and a contextual personalization using users profiles? Our main findings are the following: First, although using parsimonious language model leads to a more compact language model as users' profiles, the personalization performance is as good as using standard language models for building users' profiles. Second, we extensively analyze e↵ectiveness of three di↵erent approaches in taking the negative profiles into account, which improves performance of contextual suggestion models that just uses positive profiles. Third, we learn an e↵ective model for contextual customization and analyze the importance of different contexts in contextual suggestion task. Finally, we propose a linear combination of contextual customization and personalization, which improves the performance of contextual suggestion using either contextual customization or personalization based on all the common used IR metrics.

Bibtex 
@inproceedings{DBLP:conf/trec/HashemiDK15,
    author = {Seyyed Hadi Hashemi and Mostafa Dehghani and Jaap Kamps},
    editor = {Ellen M. Voorhees and Angela Ellis},
    title = {Parsimonious User and Group Profiling in Venue Recommendation},
    booktitle = {Proceedings of The Twenty-Fourth Text REtrieval Conference, {TREC} 2015, Gaithersburg, Maryland, USA, November 17-20, 2015},
    series = {{NIST} Special Publication},
    volume = {500-319},
    publisher = {National Institute of Standards and Technology {(NIST)}},
    year = {2015},
    url = {http://trec.nist.gov/pubs/trec24/papers/UAmsterdam-CX.pdf},
    timestamp = {Thu, 12 Mar 2020 00:00:00 +0100},
    biburl = {https://dblp.org/rec/conf/trec/HashemiDK15.bib},
    bibsource = {dblp computer science bibliography, https://dblp.org}
}

WaterlooClarke: TREC 2015 Contextual Suggestion Track

Hella Hoffmann, Pragnya Addala, Charles L. A. Clarke

Abstract

In this work we present a first attempt at developing a live system to solve the problem presented in the TREC 2015 contextual suggestion task1. The goal of this task is to tailor point-of-interest suggestions to users according to their preferences [3]. We present how we gathered data for the candidate points-of-interest, filtered some of the candidates and built a live system to return suggestions that would most likely interest the specific user. As part of TREC 2015, the contextual suggestion track is running for the fourth time [3, 4, 2] and this is the first time that the participants were required to build a live suggestion system. The general idea is to be able to make real-time suggestions for a particular person (based upon their profile) with a particular context. Unlike the previous years where the participants were asked to build their own candidate list of suggestions, this year the organizers themselves released a fixed set of candidate suggestions for 272 contexts, each context representing a city in the United States. Participants were required to set up a server that could listen to requests from users and respond with relevant suggestions. For each request, which is a profile-context pairing, a ranked list of up to 50 ranked suggestions was to be returned. [...]

Bibtex 
@inproceedings{DBLP:conf/trec/HoffmannAC15,
    author = {Hella Hoffmann and Pragnya Addala and Charles L. A. Clarke},
    editor = {Ellen M. Voorhees and Angela Ellis},
    title = {WaterlooClarke: {TREC} 2015 Contextual Suggestion Track},
    booktitle = {Proceedings of The Twenty-Fourth Text REtrieval Conference, {TREC} 2015, Gaithersburg, Maryland, USA, November 17-20, 2015},
    series = {{NIST} Special Publication},
    volume = {500-319},
    publisher = {National Institute of Standards and Technology {(NIST)}},
    year = {2015},
    url = {http://trec.nist.gov/pubs/trec24/papers/WaterlooClarke-CX.pdf},
    timestamp = {Thu, 12 Mar 2020 00:00:00 +0100},
    biburl = {https://dblp.org/rec/conf/trec/HoffmannAC15.bib},
    bibsource = {dblp computer science bibliography, https://dblp.org}
}

Laval University and Lakehead University Experiments at TREC 2015 Contextual Suggestion Track

Jian Mo, Luc Lamontagne, Richard Khoury

Abstract

In this paper we describe our effort on TREC Contextual Suggestion Track. We present a recommendation system that built upon ElasticSearch along with a machine learning re-ranking model. We utilize real world users' opinion as well as other information to build user profiles. With profile information, we then construct customized ElasticSearch queries to search on various fields. After that, a learning to rank regressor is implemented to give better ranking results. Track results of our submitted runs show the effectiveness of the system.

Bibtex 
@inproceedings{DBLP:conf/trec/MoLK15,
    author = {Jian Mo and Luc Lamontagne and Richard Khoury},
    editor = {Ellen M. Voorhees and Angela Ellis},
    title = {Laval University and Lakehead University Experiments at {TREC} 2015 Contextual Suggestion Track},
    booktitle = {Proceedings of The Twenty-Fourth Text REtrieval Conference, {TREC} 2015, Gaithersburg, Maryland, USA, November 17-20, 2015},
    series = {{NIST} Special Publication},
    volume = {500-319},
    publisher = {National Institute of Standards and Technology {(NIST)}},
    year = {2015},
    url = {http://trec.nist.gov/pubs/trec24/papers/LavallVA-CX.pdf},
    timestamp = {Thu, 12 Mar 2020 00:00:00 +0100},
    biburl = {https://dblp.org/rec/conf/trec/MoLK15.bib},
    bibsource = {dblp computer science bibliography, https://dblp.org}
}

Siena College's Institute of Artificial Intelligence TREC 2015 Contextual Suggestion Track

Aidan Trees, Kevin Danaher, Zach Siatkowski, Darren Lim, Tom Heritage

Abstract

An overview of Siena College's participation in the Contextual Suggestion track of the Twenty-Fourth Text Retrieval Conference (TREC) is provided in this report. Our goal was to first design a search technique for complex information on specified POI's (points of interest) from a collection set given by TREC. The second part of our task was to return a list of ranked suggestions dependent on a given context and a user's interests. Multiple API's were utilized for information retrieval on each particular POI including Google Places, Foursquare, and Yellow Pages. This process was repeated for not only the POI's being suggested to the user, but for the POI's rated by each user as well. From this information, profile preferences were created for individual users by examining the categories of the POI's that they had rated. To build these preferences, we designed a scoring algorithm to associate a value with each individual category returned by the API's. We finally created a ranking system that includes a unique penalty function to sort our suggestions of attractions specific to each of the users' interests.

Bibtex 
@inproceedings{DBLP:conf/trec/TreesDSLH15,
    author = {Aidan Trees and Kevin Danaher and Zach Siatkowski and Darren Lim and Tom Heritage},
    editor = {Ellen M. Voorhees and Angela Ellis},
    title = {Siena College's Institute of Artificial Intelligence {TREC} 2015 Contextual Suggestion Track},
    booktitle = {Proceedings of The Twenty-Fourth Text REtrieval Conference, {TREC} 2015, Gaithersburg, Maryland, USA, November 17-20, 2015},
    series = {{NIST} Special Publication},
    volume = {500-319},
    publisher = {National Institute of Standards and Technology {(NIST)}},
    year = {2015},
    url = {http://trec.nist.gov/pubs/trec24/papers/Siena\_SUCCESS-CX.pdf},
    timestamp = {Thu, 12 Mar 2020 00:00:00 +0100},
    biburl = {https://dblp.org/rec/conf/trec/TreesDSLH15.bib},
    bibsource = {dblp computer science bibliography, https://dblp.org}
}

Exploration of Semantic-aware Approach for Contextual Suggestion Using Knowledge from The Open Web

Yuan Wang, Jie Liu, Yalou Huang, Yongfeng Zhang, Yi Zhang, Xintong Zhang

Abstract

This paper describes our group's first attempt on the Contextual Suggestion Track of the Twenty-fourth Text REtrieval Conference (TREC 2015). The task aims to provide recommendations on attractions for various kinds of users under different and complex contexts. TREC provides two ways to participate in the track: one is to create a web server that can respond to contextual related queries called “Live Experiment”, the other is to submit run files that have all the responses to the released requests called “Batch Experiment”. For Live Experiment, due to lack of training data, our approach sticks closely to the defined relevance judgement criteria and context knowledge. We take linear interpolation to combine a variety of factors and contextual related knowledge. For Batch Experiment, we further consider domain preference under user attributes, and take existing Machine Learning based methods in principle. We show that feature engineering is a vital part for attraction suggestions. We find that the performance of suggestions to the provided user profiles and contexts has been improved using domain preference analysis.

Bibtex 
@inproceedings{DBLP:conf/trec/WangLHZZZ15,
    author = {Yuan Wang and Jie Liu and Yalou Huang and Yongfeng Zhang and Yi Zhang and Xintong Zhang},
    editor = {Ellen M. Voorhees and Angela Ellis},
    title = {Exploration of Semantic-aware Approach for Contextual Suggestion Using Knowledge from The Open Web},
    booktitle = {Proceedings of The Twenty-Fourth Text REtrieval Conference, {TREC} 2015, Gaithersburg, Maryland, USA, November 17-20, 2015},
    series = {{NIST} Special Publication},
    volume = {500-319},
    publisher = {National Institute of Standards and Technology {(NIST)}},
    year = {2015},
    url = {http://trec.nist.gov/pubs/trec24/papers/ucsc-CX.pdf},
    timestamp = {Mon, 22 Nov 2021 00:00:00 +0100},
    biburl = {https://dblp.org/rec/conf/trec/WangLHZZZ15.bib},
    bibsource = {dblp computer science bibliography, https://dblp.org}
}

University of Delaware at TREC 2015: Combining Opinion Profile Modeling with Complex Context Filtering for Contextual Suggestion

Peilin Yang, Hui Fang

Abstract

In this paper we describe our effort on TREC 2015 Contextual Suggestion Track. Using opinions from online resources to model both user profile and candidate profile has been proven to be effective on previous TREC. This year we also leverage the power of building profile based on opinions. Opinions from well known commercial online resources are collected in order to build the profiles. Two kinds of opinion representations are used for the two submitted runs. Linear interpolation is leveraged to rank the candidate suggestions. Additionally, an advanced context filter which considers all possible factors such as trip type and trip duration is applied to the ranking results so that unwanted venues are removed from the final ranking list. Official results of our submitted runs show the effectiveness of the proposed method.

Bibtex 
@inproceedings{DBLP:conf/trec/Yang015,
    author = {Peilin Yang and Hui Fang},
    editor = {Ellen M. Voorhees and Angela Ellis},
    title = {University of Delaware at {TREC} 2015: Combining Opinion Profile Modeling with Complex Context Filtering for Contextual Suggestion},
    booktitle = {Proceedings of The Twenty-Fourth Text REtrieval Conference, {TREC} 2015, Gaithersburg, Maryland, USA, November 17-20, 2015},
    series = {{NIST} Special Publication},
    volume = {500-319},
    publisher = {National Institute of Standards and Technology {(NIST)}},
    year = {2015},
    url = {http://trec.nist.gov/pubs/trec24/papers/udel\_fang-CX.pdf},
    timestamp = {Thu, 12 Mar 2020 00:00:00 +0100},
    biburl = {https://dblp.org/rec/conf/trec/Yang015.bib},
    bibsource = {dblp computer science bibliography, https://dblp.org}
}

University of Lugano at TREC 2015: Contextual Suggestion and Temporal Summarization Tracks

Mohammad Aliannejadi, Seyed Ali Bahrainian, Anastasia Giachanou, Fabio Crestani

Abstract

This technical report presents the work of the University of Lugano at TREC 2015 Contextual Suggestion and Temporal Summarization tracks. The first track that we report on, is the Contextual Suggestion. The goal of the Contextual Suggestion track is to develop systems that could generate user-specific suggestions that a user might potentially like. Our proposed method attempts to model the users' behavior and interest using a classifier, and enrich the basic model using additional data sources. Our results illustrate that our proposed method performed very well in terms of all used evaluation metrics. The second track that we report on, is the Temporal Summarization that aims to develop systems that can detect useful, new, and timely updates about a certain event. Our proposed method selects sentences that are relevant and novel to a specific event with the aim to create a summary for this event. The results showed that the proposed method is very e↵ective in terms of Latency Comprehensiveness (LC). However, the approach did not manage to obtain a good performance in terms of Expected Latency Gain (ELG).

Bibtex 
@inproceedings{DBLP:conf/trec/AliannejadiBGC15,
    author = {Mohammad Aliannejadi and Seyed Ali Bahrainian and Anastasia Giachanou and Fabio Crestani},
    editor = {Ellen M. Voorhees and Angela Ellis},
    title = {University of Lugano at {TREC} 2015: Contextual Suggestion and Temporal Summarization Tracks},
    booktitle = {Proceedings of The Twenty-Fourth Text REtrieval Conference, {TREC} 2015, Gaithersburg, Maryland, USA, November 17-20, 2015},
    series = {{NIST} Special Publication},
    volume = {500-319},
    publisher = {National Institute of Standards and Technology {(NIST)}},
    year = {2015},
    url = {http://trec.nist.gov/pubs/trec24/papers/USI-CXTS.pdf},
    timestamp = {Thu, 12 Mar 2020 00:00:00 +0100},
    biburl = {https://dblp.org/rec/conf/trec/AliannejadiBGC15.bib},
    bibsource = {dblp computer science bibliography, https://dblp.org}
}

University of Glasgow at TREC 2015: Experiments with Terrier in Contextual Suggestion, Temporal Summarisation and Dynamic Domain Tracks

Richard McCreadie, Saul Vargas, Craig MacDonald, Iadh Ounis, Stuart Mackie, Jarana Manotumruksa, Graham McDonald

Abstract

n TREC 2015, we focus on tackling the challenges posed by the Contextual Suggestion, Temporal Summarisation and Dynamic Domain tracks. For Contextual Suggestion, we investigate the use of user-generated data in location-based social networks (LBSN) to suggest venues. For Temporal Summarisation, we examine features for event summarisation that explicitly model the entities involved in the events. Meanwhile, for the Dynamic Domain track, we explore resource selection techniques for identifying the domain of interest and diversifying sub-topic intents.

Bibtex 
@inproceedings{DBLP:conf/trec/McCreadieVMOMMM15,
    author = {Richard McCreadie and Saul Vargas and Craig MacDonald and Iadh Ounis and Stuart Mackie and Jarana Manotumruksa and Graham McDonald},
    editor = {Ellen M. Voorhees and Angela Ellis},
    title = {University of Glasgow at {TREC} 2015: Experiments with Terrier in Contextual Suggestion, Temporal Summarisation and Dynamic Domain Tracks},
    booktitle = {Proceedings of The Twenty-Fourth Text REtrieval Conference, {TREC} 2015, Gaithersburg, Maryland, USA, November 17-20, 2015},
    series = {{NIST} Special Publication},
    volume = {500-319},
    publisher = {National Institute of Standards and Technology {(NIST)}},
    year = {2015},
    url = {http://trec.nist.gov/pubs/trec24/papers/uogTr-CXTSDD.pdf},
    timestamp = {Thu, 12 Mar 2020 00:00:00 +0100},
    biburl = {https://dblp.org/rec/conf/trec/McCreadieVMOMMM15.bib},
    bibsource = {dblp computer science bibliography, https://dblp.org}
}