Correction to the committee members, added below.
From: lriehl@cs.princeton.edu On Behalf Of
gradinfo--- via talks
Sent: Monday, May 12, 2025 10:27 AM
To: 'talks'
Subject: [talks] Anushri Arora will present her General Exam "Efficient
Training of Maximal and Minimal Low-Rank RNNs" on Monday, May 12, 2025 at
11:00 AM in FC 008 and via zoom.
Anushri Arora will present her General Exam "Efficient Training of Maximal
and Minimal Low-Rank RNNs" on Monday, May 12, 2025 at 11:00 AM in FC 008 and
via zoom.
Zoom link: https://princeton.zoom.us/j/94357119848
Committee Members: Jonathan Pillow (advisor), Ben Eysenbach, Adji Bousso
Dieng, Ryan Adams
Abstract:
Low-rank recurrent neural networks (RNNs) have recently gained prominence as
a framework for understanding how neural systems solve complex cognitive
tasks. However, fitting and interpreting these networks remains an important
open problem. In this talk, I will introduce an alternative, yet
mathematically equivalent perspective on low-rank RNNs that addresses these
challenges. Specifically, we view individual neurons as nonlinear basis
functions for representing an ordinary differential equation (ODE) of
interest. Leveraging this perspective, we demonstrate how to embed arbitrary
ODEs into the network using least-squares and recursive least-squares
regression. First, this view provides geometric and algebraic insight
highlighting how the choice of neuronal nonlinearity (e.g. tanh, ReLU)
affects the network's universal approximation capacity. Second, we show that
this construction is mathematically equivalent to performing kernel
regression in a high-dimensional feature space, thereby connecting our
approach to a Gaussian Process in the infinite width limit. Third, by
applying a variant of orthogonal matching pursuit, we derive the minimal
network size required to implement any target ODE. Fourth, using active
learning, we systematically select the smallest set of dynamical
trajectories needed for training. Finally, we benchmark our approach against
FORCE and Backpropagation through time (BPTT) demonstrating that our
approach learns significantly faster and achieves higher accuracy with
substantially smaller networks.
Reading List:
https://docs.google.com/document/d/19cgCy1iOIE3ckuIzXgEVZ3dEFrWJ20-ieyklysyH
DTE/edit?usp=sharing
Everyone is invited to attend the talk, and those faculty wishing to remain
for the oral exam following
