Readers: Alex Pouget (University of Geneva), Barbara Engelhardt

Non-readers: Jonathan Pillow (advisor), Ryan Adams, Tom Griffiths

Abstract:

A central goal in neuroscience is to identify the strategies used by animals and humans as they make decisions, and to characterize the learning rules that shape these policies in the first place. In this talk, I will first discuss recent modeling work that revealed that — contrary to common wisdom — mice and humans use multiple strategies over the course of a single session to perform perceptual decision-making tasks.  Our modeling framework, based on hidden Markov models with Bernoulli Generalized Linear Model observations, allows us to characterize the nature of these policies and to label the trials associated with each strategy. Next, I will discuss work that sought to uncover the learning rules used by mice and rats as they learned to perform this type of task. Our model tracked trial-to-trial changes in the animals’ choice policies, and separated these changes into components explainable by a reinforcement learning rule, and components that remained unexplained.  Whereas the average contribution of the conventional REINFORCE learning rule to the policy update for mice learning a common task was just 30%, we found that adding baseline parameters allowed the learning rule to explain 92% of the animals' policy updates under our model. Finally, I will discuss work in progress that applies inverse reinforcement learning to the trajectories of mice exploring a maze environment.  Unlike artificial agents, mice are very good at exploring novel environments, and in this work, we will seek to uncover the extrinsic and intrinsic reward functions that power such exploration.