Proceedings - Interactive Knowledge Assistance Track (IKAT) 2025¶

Conversational information seeking has evolved rapidly in the last few years with the development of large language models (LLMs) providing the basis for interpreting and responding in a naturalistic manner to user requests. iKAT emphasizes the creation and research of conversational search agents that adapt responses based on the user’s prior interactions and present context, maintaining a long-term memory of user-system interactions. This means that the same question might yield varied answers, contingent on the user’s profile and preferences. The challenge lies in enabling conversational search agents (CSA) to incorporate personalized context to guide users through the relevant information effectively. iKAT’s third year introduced an interactive conversation task, attracting seven teams and a total of 47 runs. Most of the runs leveraged LLMs in their pipelines, for single or multiple query rewriting, some also adopted agentic pipelines.

This year’s TREC iKAT track contains several tasks such as passage ranking, response generation and Personal Text Knowledge Base (PTKB) statement classification. We focus on response generation (only) task due to time and budget limitations. This task is to generate a response based on retrieved passages, given the additional context. We submitted two runs for this task mainly to explore the impact of user profiles on the generation quality of responses. In this short report, we describe the method that was used for generation and present the results.

This paper describes the GRILL Lab’s participation in the TREC 2025 Interactive Knowledge Assistance Track (IKAT) and the Retrieval-Augmented Generation (RAG) track, covering four sub-tasks: IKAT Passage Ranking/Response Generation, IKAT Simulation, RAG Retrieval Only, and RAG Full. Our approach centres on a modular, agentic pipeline that pursues high recall through iterative feedback. The system proceeds in three stages: (1) initial candidate generation via BM25; (2) document expansion using Query-by-Document techniques; and (3) an LLM-driven gap analysis phase in which the model identifies informational gaps and formulates supplementary queries. A key architectural feature is a fine-tuned GPT-4.1 nano binary relevance filter, trained on TREC CAsT 2022 and IKAT 2023 relevance judgments, which prunes irrelevant documents between each stage to contain topic drift.

The report describes the work conducted by several teams involved in the ANR GUIDANCE project for the iKAT evaluation campaign. The ANR GUIDANCE aims to advance research in General Purpose Dialogue-assisted Digital Information Access. This involves tackling challenges such as the design and adaptation of large language models (LLMs) for better information access, enhancing LLMs' generalization capabilities to new domains and languages, ensuring the truthfulness of outputs, and addressing the lack of open-access state-of-the-art models. The GUIDANCE project also seeks to unite the French Information Retrieval community and produce open-access resources for model evaluation and development.

We present GenAIus’s participation in the TREC 2025 Interactive Knowledge Assistance Track (iKAT), focusing on personalized and context-aware response generation in both offline and interactive settings. We develop a multi-stage pipeline that integrates conversation summarization, Personal Textual Knowledge Base (PTKB) classification, query rewriting, passage retrieval, and grounded response generation. To study the impact of conversational context modeling, we compare two configurations: conditioning on the full dialogue history versus using an evolving conversation summary updated at each turn. Experimental results show that full-history conditioning yields slightly stronger performance in offline generation and dialogue-level interactive metrics, while summary-based conditioning achieves comparable overall results with improvements in engagement and contextual efficiency. Both approaches rank within the top tier of participating systems, demonstrating the robustness of our pipeline and the viability of structured conversational summarization as a scalable alternative to full-history conditioning.

The 2025 TREC Interactive Knowledge Assistance Track (iKAT) featured both interactive and offline submission tasks. The former requires systems to operate under real-time constraints, making robustness and efficiency as important as accuracy, while the latter enables controlled evaluation of passage ranking and response generation with pre-defined datasets. To address this, we explored query rewriting and retrieval fusion as core strategies. We built our pipelines around Best-of-N selection and Reciprocal Rank Fusion (RRF) strategies to handle different submission tasks. Results show that reranking and fusion improve robustness while revealing trade-offs between effectiveness and efficiency across both tasks.

The TREC interactive Knowledge Assistant Track (iKAT) 2025 is the third edition of the iKAT shared task. It focuses on developing conversational assistants that can adapt their responses using personal user knowledge from a Personal Textual Knowledge Base (PTKB). This year’s edition also introduces a new interactive task that evaluates systems using a user simulator. Since query rewriting is an effective way to handle conversational context, we study the use of Large Language Models (LLMs) as query rewriters. In our runs, we generate multiple query aspects using the MQ4CS framework and frontier LLMs (GPT-4.1), as well as open-source LLMs finetuned for the task (Llama-8B). We also strengthen the approach with SPLADE-based sparse retrieval and cross-encoder reranking. Finally, we also explore a rewrite-free technique, based on learned sparse retrieval (LSR) using the DiSCo model. Our results show that multi-aspect query generation improves performance when paired with strong retrieval and reranking models. They also suggest that LLM-based query rewriting can support better personalization in conversational search.