Anushri Arora will present her General Exam "Efficient Training of Maximal and Minimal Low-Rank RNNs" on Monday, May 12, 2025 at 11:00 AM in FC 008 and via zoom.

Zoom link: https://princeton.zoom.us/j/94357119848

Committee Members: Jonathan Pillow (advisor), Ben Eysenbach, Adji Bousso Dieng

Abstract:

Low-rank recurrent neural networks (RNNs) have recently gained prominence as a framework for understanding how neural systems solve complex cognitive tasks. However, fitting and interpreting these networks remains an important open problem. In this talk, I will introduce an alternative, yet mathematically equivalent perspective on low-rank RNNs that addresses these challenges. Specifically, we view individual neurons as nonlinear basis functions for representing an ordinary differential equation (ODE) of interest. Leveraging this perspective, we demonstrate how to embed arbitrary ODEs into the network using least-squares and recursive least-squares regression. First, this view provides geometric and algebraic insight highlighting how the choice of neuronal nonlinearity (e.g. tanh, ReLU) affects the network’s universal approximation capacity. Second, we show that this construction is mathematically equivalent to performing kernel regression in a high-dimensional feature space, thereby connecting our approach to a Gaussian Process in the infinite width limit. Third, by applying a variant of orthogonal matching pursuit, we derive the minimal network size required to implement any target ODE. Fourth, using active learning, we systematically select the smallest set of dynamical trajectories needed for training. Finally, we benchmark our approach against FORCE and Backpropagation through time (BPTT) demonstrating that our approach learns significantly faster and achieves higher accuracy with substantially smaller networks.

Reading List:

https://docs.google.com/document/d/19cgCy1iOIE3ckuIzXgEVZ3dEFrWJ20-ieyklysyHDTE/edit?usp=sharing

Everyone is invited to attend the talk, and those faculty wishing to remain for the oral exam following