Eli Lucherini will present his Pre-FPO "Analysis and Critique of Simulation Research on Recommender Systems" on Thursday, June 9, 2022 at 10:00 AM via Zoom.
Committee Members: Arvind Narayanan (Examiner, advisor), Olga Russakovsky (Examiner), Brandon Stewart (examiner), Jonathan Mayer (reader), Andrés Monroy-Hernández (reader).
All are welcome to attend.
Title: Analysis and Critique of Simulation Research on Recommender Systems
Abstract:
Simulation has emerged as a popular method to study the long-term societal consequences of recommender systems. This approach allows researchers to specify their theoretical model explicitly and observe the evolution of system-level outcomes over time. However, performing simulation-based studies often requires researchers to build their own simulation environments from the ground up, which creates a high barrier to entry, introduces room for implementation error, and makes it difficult to disentangle whether observed outcomes are due to the model or the implementation.
I will present T-RECS, an open-sourced Python package designed for researchers to simulate recommendation systems and other types of sociotechnical systems in which an algorithm mediates the interactions between multiple stakeholders, such as users and content creators. T-RECS promotes reproducibility in this area of research, provides a unified language for simulating sociotechnical systems, and removes the friction of implementing simulations from scratch.
More broadly, I will present a comparative analysis and offer a critique of current simulation research, showing how common shortcomings such as inconsistent results across studies and reproducibility issues can be addressed by the use of tools such as T-RECS.
References:
1. E. Lucherini, M. Sun, A. Winecoff, A. Narayanan. T-RECS: A Simulation Tool to Study the Societal Impact of Recommender Systems. Under submission.
2. A. Winecoff, M. Sun, E. Lucherini, A. Narayanan. Simulation as Experiment: An Empirical Critique of Simulation Research on Recommender Systems. At the Workshop on Synthetic Data and Simulation Methods for Recommender Systems Research, RecSys ’21.