Overview - Fair Ranking 2020

Proceedings | Data | Runs | Participants

For 2020, we again adopted an academic search task, where we have a corpus of academic article abstracts and queries submitted to a production academic search engine. The central goal of the Fair Ranking track is to provide fair exposure to different groups of authors (a group fairness framing). We recognize that there may be multiple group definitions (e.g. based on demographics, stature, topic) and hoped for the systems to be robust to these. We expected participants to develop systems that optimize for fairness and relevance for arbitrary group definitions, and did not reveal the exact group definitions until after the evaluation runs were submitted. The track contains two tasks, reranking and retrieval, with a shared evaluation.

Track coordinator(s):

• Asia J. Biega, Microsoft Research Montreal
• Fernando Diaz, Montreal Institute for Learning Algorithms
• Michael D. Ekstrand, Boise State University
• Sergey Feldman, Allen Institute for Artificial Intelligence
• Sebastian Kohlmeier, Allen Institute for Artificial Intelligence

Tasks:

• rerank: Rerank
• retrieval: Retrieval

Track Web Page: https://fair-trec.github.io/

---