- talks - lists.cs.princeton.edu

Jimmy Wu will present his FPO "Spatial Representations for Learning Robotic Mobile Manipulation" on Tuesday, May 20, 2025 at 1:30 PM in CS 401 and Zoom.
by CS Grad Department 13 May '25

13 May '25

Jimmy Wu will present his FPO "Spatial Representations for Learning Robotic Mobile Manipulation" on Tuesday, May 20, 2025 at 1:30 PM in CS 401 and Zoom. Location: Zoom link: https://princeton.zoom.us/j/96975727674 The members of Jimmy’s committee are as follows: Examiners: Szymon Rusinkiewicz (Adviser), Thomas Funkhouser (Adviser), Anirudha Majumdar Readers: Jia Deng, Jeannette Bohg (Stanford University) A copy of his thesis is available upon request. Please email gradinfo(a)cs.princeton.edu if you would like a copy of the thesis. Everyone is invited to attend his talk. Abstract follows below: The development of intelligent mobile manipulators has long been a goal of robotics research, promising transformative applications. However, enabling robots to operate outside of controlled laboratory settings remains a formidable challenge. This dissertation explores the role of spatial representations in learning-based systems for mobile manipulation. We develop a sequence of spatially grounded frameworks that progressively increase robot capabilities. First, we introduce spatial action maps, an action representation that is spatially aligned with the robot’s state representation, allowing a policy to predict actions on a 2D map of possible target locations. This approach simplifies the perception-to-control mapping, yielding more sample-efficient reinforcement learning and improved task performance. We then extend this idea to multi-robot systems with spatial intention maps, in which the intentions of other robots are spatially encoded in the state representation. This enables implicit coordination between robots, leading to improved multi-agent task success and emergent cooperative strategies such as collision avoidance and task specialization. Next, we tackle dynamic manipulation tasks with multi-frequency spatial action maps, which operate on multiple temporal scales, allowing a single policy to exhibit both high-level deliberative planning and low-level reactive behaviors. Finally, we present TidyBot, a mobile manipulation system for open-world household cleanup that leverages foundation models in vision and language. We show that spatial grounding allows the robot to tap into the broad semantic understanding embedded in foundation models, enabling generalization to novel objects and scenarios. Together, these contributions demonstrate that spatially grounded learning is a powerful paradigm for robotic mobile manipulation, bringing us closer to practical, intelligent robotic systems capable of operating in the real world.

1 0

Andrew Sheinberg will present his General Exam "Laying Down the Sequence Train(ing) Tracks: Minimizing Communication and Reducing HBM Footprint via Local Computations and Orchestrated Dataflow" on 5/14/25 at 10:00am in CS 302.
by Gradinfo 12 May '25

12 May '25

Andrew Sheinberg will present his General Exam "Laying Down the Sequence Train(ing) Tracks: Minimizing Communication and Reducing HBM Footprint via Local Computations and Orchestrated Dataflow" on 5/14/25 at 10:00am in CS 302. Committee: Kai Li (adviser), Tri Dao, Ravi Netravali Abstract and Reading List: [ https://docs.google.com/document/d/1ARxwBp59w7Azys4_dZHU2noUUUGWUbCtlLeIUM4… | https://docs.google.com/document/d/1ARxwBp59w7Azys4_dZHU2noUUUGWUbCtlLeIUM4… ]

1 0

Anushri Arora will present her General Exam "Efficient Training of Maximal and Minimal Low-Rank RNNs" on Monday, May 12, 2025 at 11:00 AM in FC 008 and via zoom.
by gradinfo＠cs.princeton.edu 12 May '25

12 May '25

Anushri Arora will present her General Exam "Efficient Training of Maximal and Minimal Low-Rank RNNs" on Monday, May 12, 2025 at 11:00 AM in FC 008 and via zoom. Zoom link: https://princeton.zoom.us/j/94357119848 Committee Members: Jonathan Pillow (advisor), Ben Eysenbach, Adji Bousso Dieng Abstract: Low-rank recurrent neural networks (RNNs) have recently gained prominence as a framework for understanding how neural systems solve complex cognitive tasks. However, fitting and interpreting these networks remains an important open problem. In this talk, I will introduce an alternative, yet mathematically equivalent perspective on low-rank RNNs that addresses these challenges. Specifically, we view individual neurons as nonlinear basis functions for representing an ordinary differential equation (ODE) of interest. Leveraging this perspective, we demonstrate how to embed arbitrary ODEs into the network using least-squares and recursive least-squares regression. First, this view provides geometric and algebraic insight highlighting how the choice of neuronal nonlinearity (e.g. tanh, ReLU) affects the network's universal approximation capacity. Second, we show that this construction is mathematically equivalent to performing kernel regression in a high-dimensional feature space, thereby connecting our approach to a Gaussian Process in the infinite width limit. Third, by applying a variant of orthogonal matching pursuit, we derive the minimal network size required to implement any target ODE. Fourth, using active learning, we systematically select the smallest set of dynamical trajectories needed for training. Finally, we benchmark our approach against FORCE and Backpropagation through time (BPTT) demonstrating that our approach learns significantly faster and achieves higher accuracy with substantially smaller networks. Reading List: https://docs.google.com/document/d/19cgCy1iOIE3ckuIzXgEVZ3dEFrWJ20-ieyklysyH DTE/edit?usp=sharing Everyone is invited to attend the talk, and those faculty wishing to remain for the oral exam following

1 1

CS Colloquium speakers: week of February 26
by Emily C. Lawrence 12 May '25

12 May '25

CS Colloquium speakers Speaker: Jialin Ding, Amazon Web Services Date: Monday, February 26 Time: 12:30pm EST Location: CS 105 Host: Wyatt Lloyd Event page: https://www.cs.princeton.edu/events/26578 Register for live-stream online here: https://princeton.zoom.us/webinar/register/WN_10z_l8MYRr21ZOuPqA8Odw Title: Instance-Optimization: Rethinking Database Design for the Next 1000X Abstract: Modern database systems aim to support a large class of different use cases while simultaneously achieving high performance. However, as a result of their generality, databases often achieve adequate performance for the average use case but do not achieve the best performance for any individual use case. In this talk, I will describe my work on designing databases that use machine learning and optimization techniques to automatically achieve performance much closer to the optimal for each individual use case. In particular, I will present my work on instance-optimized database storage layouts, in which the co-design of data structures and optimization policies improves query performance in analytic databases by orders of magnitude. I will highlight how these instance-optimized data layouts address various challenges posed by real-world database workloads and how I implemented and deployed them in production within Amazon Redshift, a widely-used commercial database system. Bio: Jialin Ding is an Applied Scientist at AWS. Before that, he received his PhD in computer science from MIT, advised by Tim Kraska. He works broadly on applying machine learning and optimization techniques to improve data management systems, with a focus on building databases that automatically self-optimize to achieve high performance for any specific application. His work has appeared in top conferences such as SIGMOD, VLDB, and CIDR, and has been recognized by a Meta Research PhD Fellowship. To learn more about Jialin’s work, please visit [ https://jialinding.github.io/ | https://jialinding.github.io/ ] . CSML/CS/PSY Colloquium Speaker: Brenden Lake, New York University Date: Tuesday, February 27 Time: 12:30pm EST Location: CS 105 Host: Tom Griffiths Event page: https://csml.princeton.edu/events/csmlpsycs-seminar Register for live-stream online here: https://princeton.zoom.us/webinar/register/WN_aUfLfVQFSIqV2RDSYu4big Title: Towards more human-like learning in machines: Bridging the data and generalization gaps Abstract: There is an enormous data gap between how AI systems and children learn: The best LLMs now learn language from text with a word count in the trillions, whereas it would take a child roughly 100K years to reach those numbers through speech (Frank, 2023, "Bridging the data gap"). There is also a clear generalization gap: whereas machines struggle with systematic generalization, children can excel. For instance, once a child learns how to "skip," they immediately know how to "skip twice" or "skip around the room with their hands up" due to their compositional skills. In this talk, I'll describe two case studies in addressing these gaps. 1) The data gap: We train deep neural networks from scratch (using DINO, CLIP, etc.), not on large-scale data from the web, but through the eyes and ears of a single child. Using head-mounted video recordings from a child as training data (<200 hours of video slices over 26 months), we show how deep neural networks can perform challenging visual tasks, acquire many word-referent mappings, generalize to novel visual referents, and achieve multi-modal alignment. Our results demonstrate how today's AI models are capable of learning key aspects of children's early knowledge from realistic input. 2) The generalization gap: Can neural networks capture human-like systematic generalization? We address a 35-year-old debate catalyzed by Fodor and Pylyshyn's classic article, which argued that standard neural networks are not viable models of the mind because they lack systematic compositionality -- the algebraic ability to understand and produce novel combinations from known components. We'll show how neural networks can achieve human-like systematic generalization when trained through meta-learning for compositionality (MLC), a new method for optimizing the compositional skills of neural networks through practice. With MLC, neural networks can match human performance and solve several machine learning benchmarks. Given these findings, we'll discuss the paths forward for building machines that learn, generalize, and interact in more human-like ways based on more natural input. Related articles: Vong, W. K., Wang, W., Orhan, A. E., and Lake, B. M (2024). Grounded language acquisition through the eyes and ears of a single child. Science, 383, 504-511. Orhan, A. E., and Lake, B. M. (in press). Learning high-level visual representations from a child’s perspective without strong inductive biases. Nature Machine Intelligence. Lake, B. M. and Baroni, M. (2023). Human-like systematic generalization through a meta-learning neural network. Nature, 623, 115-121. Bio: Brenden M. Lake is an Assistant Professor of Data Science and Psychology at New York University. He received his M.S. and B.S. in Symbolic Systems from Stanford University in 2009, and his Ph.D. in Cognitive Science from MIT in 2014. He was a postdoctoral Data Science Fellow at NYU from 2014-2017. Brenden is a recipient of the Robert J. Glushko Prize for Outstanding Doctoral Dissertation in Cognitive Science and the MIT Technology Review 35 Innovators Under 35. His research was also selected by Scientific American as one of the 10 most important advances of 2016. Brenden's research focuses on computational problems that are easier for people than they are for machines, such as learning new concepts from just a few examples, learning by asking questions, learning by generating new goals, and learning by producing novel combinations of known components. CS Colloquium Speaker: Eric Mitchell, Stanford University Date: Thursday, February 29 Time: 12:30pm EST Location: CS 105 Host: Karthik Narasimhan Event page: https://www.cs.princeton.edu/events/26587 Register for live-stream online here: TBA Talk info TBA

1 24

[alg-ml-reading-group] Alg-ML Talk by Akshay Krishnamurthy on May 12, Monday, from 3pm - 4pm ET
by Emily C. Lawrence 09 May '25

09 May '25

Dr. Akshay Krishnamurthy from Microsoft Research will present his recent work on Understanding Inference Time Compute: Self-Improvement and Scaling . The talk is scheduled on on May 12, Monday. Bio: Akshay Krishnamurthy is a senior principal research manager at Microsoft Research, New York City. He previously spent two years as an assistant professor in the College of Information and Computer Sciences at the University of Massachusetts, Amherst and a year as a postdoctoral researcher at Microsoft Research, NYC. He completed my PhD in the Computer Science Department at Carnegie Mellon University, advised by Aarti Singh and received his undergraduate degree in EECS at UC Berkeley. His research interests are broadly in the areas of machine learning and statistics. He is most excited about interactive learning and decision making, including reinforcement learning, and recently has been thinking about how reinforcement learning can be used to improve modern generative AI systems. Feel free to grab a slot [ https://docs.google.com/spreadsheets/d/10LQsvqhbxw36MDLcxpnDPcG12nsrDl33yG0… | here ] to meet with our speaker! The talk will start at 3.00 pm ET. ____________________________________________________________________________ Time: 3 .00 PM ET on May 12 , Monday Physical Location: COS 402 ____________________________________________________________________________ Title: Understanding Inference Time Compute: Self-Improvement and Scaling Abstract: Inference-time compute has emerged as a new axis for scaling large language models, leading to breakthroughs in AI reasoning. Broadly speaking, inference-time compute methods involve allowing the language model to interact with a verifier to search for desirable, high-quality, or correct responses. While recent breakthroughs involve using a ground-truth verifier of correctness, it is also possible to invoke the language model itself or an otherwise learned model as verifiers. These latter protocols raise the possibility of self-improvement, whereby the AI system evaluates and refines its own generations to achieve higher performance. This talk presents new understanding of and new algorithms for language model self-improvement. The first part of the talk focuses on a new perspective on self-improvement that we refer to as sharpening, whereby we "sharpen" the model toward one placing large probability mass on high-quality sequence, as measured by the language model itself. We show how the sharpening process can be done purely at inference time or amortized into the model via post-training, thereby avoiding expensive inference-time computation. In the second part of the talk, we consider the more general setting of a learned reward model, show that the performance of naive-but-widely-used inference-time compute strategies does not improve monotonically with compute, and develop a new compute-monotone algorithm with optimal statistical performance. Based on joint works with Audrey Huang, Dhruv Rohatgi, Adam Block, Qinghua Liu, Jordan T. Ash, Cyril Zhang, Max Simchowitz, Dylan J. Foster and Nan Jiang. We hope to see you there! ____________________________________________________________________________ (Useful links: [ https://princeton-alg-ml.github.io/ | alg-ml-website ] , [ https://calendar.google.com/calendar/u/1?cid=Y185ZWQxMzVmOGMxN2JjZmNhYjAyOT… | alg-ml-calendar ] , [ https://lists.cs.princeton.edu/mailman/listinfo/alg-ml-reading-group | alg-ml-mailing-list ] ) Best, Zixuan , Gon, and Catherine _______________________________________________ alg-ml-reading-group mailing list [ mailto:alg-ml-reading-group@lists.cs.princeton.edu | alg-ml-reading-group(a)lists.cs.princeton.edu ] [ https://lists.cs.princeton.edu/mailman/listinfo/alg-ml-reading-group | https://lists.cs.princeton.edu/mailman/listinfo/alg-ml-reading-group ]

1 0

Elizabeth Mieczkowski will present her General Exam "Quantifying Collaborative Trade-offs with Insights from Distributed Systems" on Tuesday, May 13, 2025 at 12:00 PM in FC 008.
by CS Grad Department 08 May '25

08 May '25

Elizabeth Mieczkowski will present her General Exam "Quantifying Collaborative Trade-offs with Insights from Distributed Systems" on Tuesday, May 13, 2025 at 12:00 PM in FC 008. Zoom Link: https://princeton.zoom.us/j/6756891992 Committee Members: Tom Griffiths (advisor), Benjamin Eysenbach, Wyatt Lloyd, Natalia Velez Abstract: Collaboration enables multi-agent systems to achieve goals beyond the capabilities of any single agent. However, coordination is inherently complex and dynamic. How can we identify the underlying principles that should guide the division of labor and resources in multi-agent teams, and formalize the trade-offs that emerge during collaboration? I argue that research from distributed computing systems provides a powerful interpretability framework for multi-agent collaboration. This literature focuses on building efficient, scalable, and fault-tolerant systems, while exposing fundamental constraints—such as communication overhead and consistency—when computation is shared across machines. I first extend Amdahl’s Law, which predicts how serial bottlenecks constrain parallel speedup, to naturalistic multi-agent tasks and confirm its validity in a set of human experiments. I then apply this model to large-scale MARL simulations in Overcooked, SMAC, and MPE, demonstrating that specialized policies emerge as a function of task parallelizability. Finally, I demonstrate how the model can benchmark existing algorithms and human collaborative strategies. Reading List: https://docs.google.com/document/d/12GKMv_n99TcUJvj8iua44eHGkUbQAz2hwX_zUYj… Everyone is invited to attend the talk, and those faculty wishing to remain for the oral exam following are welcome to do so.

1 0

[Machine Learning Seminar] May 9: Virtual Talk by Prof. Yitao Liang on Minecraft and Vision-Language-Action Models
by Emily C. Lawrence 07 May '25

07 May '25

This Friday, Prof. Yitao Liang from Peking University will give a virtual talk about his research on Vision-Language-Action Models and their applications to open-world games (e.g., Minecraft). Time : May 9, from 10:00 to 11:00 a.m. ET at [ https://princeton.zoom.us/j/97152277086 | https://princeton.zoom.us/j/97152277086 ] We will also stream the talk together at Friend Center 008 . Title: Instruction-Following Control in an Open World: Vision-Language-Action Models and Beyond Abstract: With the emergence of large language models, the debate about whether an instruction-following long-horizon control agent will appear has resurfaced. However, GPT's emerging abilities seem difficult to reproduce in domains that require sequential control. In this talk, we will introduce various efforts by our group and other well-known research labs to develop such a model in open-world game domains (e.g., Minecraft). Due to its extremely high degree of freedom, traditional multi-task data-driven methods are unsustainable (we cannot afford to train on thousands of tasks simultaneously, it is too expensive). One possible direction is to leverage LLMs to achieve flexible task planning. In this talk, we present latest research in this direction, and how to obtain a universal prompt-based generic policy learning. Due to models’s increasingly impressive data fitting capability, how to properly evaluate in open-ended domains has also become a new frontier. We will also discuss how generative evaluation could facilitate a more principled evaluation. Bio: Yitao Liang ( [ mailto:yitaol@pku.edu.cn | yitaol(a)pku.edu.cn ] ) is an assistant professor and Boya Fellow at Peking University. He obtained his Ph.D. degree in Computer Science from UCLA, advised by Prof. Guy Van den Broeck. His research interests span reasoning, machine learning and AI agents. His work has received recognition from top AI conferences; for example, the best-paper honorable mention from AAMAS 2016, the best paper from RL for Real Life workshop in ICML 2019, a best paper runner-up from the LLD workshop in NeurIPS 2017, a best paper from the TEACH workshop in ICML2023. He regularly serves as area chairs and senior area chairs in top venues e.g., NeurIPS, ICML. Best, Wenzhe Li & Chi Jin

1 0

Ruyu Yan will present her General Exam "Nonlinear Perspective Photography via Geometry-aware Scene Completion" on Friday, May 9, 2025 at 1:00 PM in CS 402.
by CS Grad Department 06 May '25

06 May '25

Ruyu Yan will present her General Exam "Nonlinear Perspective Photography via Geometry-aware Scene Completion" on Friday, May 9, 2025 at 1:00 PM in CS 402. Committee Members: Adam Finkelstein (advisor), Felix Heide, Szymon Rusinkiewicz Abstract: Designers of modern mobile cameras aim to capture high-quality images using linear perspective projection. However, this projection often fails to align with users' natural perception, which is influenced by nonlinear phenomena; it also does not support creative composition needs, such as depth compression along a long street or tilt-shift effects. We introduce a computational photography system that enables nonlinear perspective projection as a composition process during capture on mobile devices. Our system reconstructs a low-resolution 3D scene representation using signed distance field fusion, allowing users to explore perspective transformations with interactive preview in the camera viewfinder. During capture, the system guides users to capture additional data to overcome challenges such as occlusions and the limited field of view of the physical lens. We then employ a 2D Gaussian splatting method to reconstruct a geometry proxy of the scene, which we use for warping the captured photos and exporting high-fidelity photos with nonlinear perspective. Reading List: https://docs.google.com/document/d/1yaZAtQX7YuhOO9343ITD-ieHEApp8teCrU9aQCq… Everyone is invited to attend the talk, and those faculty wishing to remain for the oral exam following are welcome to do so.

1 0

David Braun will present his General Exam "EASY-AS-PIE: Evolutionary Algorithm SYnthesizer Architecture Search via Pre-trained Instrument Embeddings" on Thursday, May 8, 2025 at 10:00 AM in CS 402.
by gradinfo＠cs.princeton.edu 06 May '25

06 May '25

David Braun will present his General Exam "EASY-AS-PIE: Evolutionary Algorithm SYnthesizer Architecture Search via Pre-trained Instrument Embeddings" on Thursday, May 8, 2025 at 10:00 AM in CS 402. Committee Members: Adam Finkelstein (advisor), Tri Dao, Mae Milano Abstract: Software synthesizers offer immense creative possibilities for sound design, but navigation of their high-dimensional parameter spaces poses a challenge for musicians. This has motivated efforts toward automatic synthesizer programming-systems that generate parameter settings given a target sound or description. However, such systems require an automated means of comparing the similarity of sounds, which is complicated due to psychoacoustic factors. Existing methods often address synthesizer programming using gradient descent or genetic algorithms. Gradient-based optimization faces challenges because synthesizer signal processing is highly recurrent and can involve gradient discontinuities. Conversely, evolutionary methods handle such discontinuities but typically operate with fixed synthesizer architectures, limiting creative potential. Allowing dynamic architectural changes exponentially increases search complexity due to the combinatorial explosion in connecting synthesizer modules. Neither approach is well-suited for automatic programming of a large corpus of target sounds. Nonetheless, automatic synthesizer programming remains a valuable research direction because synthesizers offer greater interpretability compared to black-box neural audio synthesis models, enabling musicians and sound designers to intuitively understand and manipulate generated sounds. Motivated by these considerations, we propose a novel framework that uniquely combines two established concepts: evolutionary algorithms for synthesizer architecture search and pre-trained instrument embeddings (specifically CLAP) for perceptual comparison of sound similarity. Crucially, our method efficiently optimizes synthesizer presets for thousands of distinct target sounds simultaneously. Our evolutionary framework involves a hierarchical genetic representation comprising distinct gene classes for oscillator settings, envelopes, various audio effects, and more. Each parameter is modeled with dedicated numeric genes, either continuous (e.g., an oscillator's sustain level) or discrete (e.g., filter type), and mutations are performed by sampling from precomputed parameter distributions extracted from real synthesizer presets. Since the genome includes information about modulation routings (e.g., "Envelope 1 modulating Reverb Size with a Depth of 40%"), the genetic mutation and crossover operations explore the synthesizer architecture space in addition to a purely numeric parameter space. Our evolutionary framework also incorporates two complementary archive structures to balance exploration and exploitation. A grid-based archive inspired by MAP-Elites uses dimensionality reduction on CLAP embeddings, enabling structured diversity-driven exploration of the preset space, untethered to target sounds. Concurrently, a second archive maintains champion presets for each target sound, ranked using cosine distance metrics in the embedding space, facilitating exploitation. Through ablation studies, we demonstrate that this dual-archive approach effectively improves alignment with a held-out audio dataset measured by Kernel Audio Distance (KAD). Moreover, when optimizing for a single target sound, our approach can flexibly use a different loss signal such as a multi-scale spectrogram loss while still benefiting from pre-existing archives. We show that the addition of architecture search in such a scenario results in superior reconstruction of query sounds compared to only modifying numeric parameters. This research demonstrates that embedding-driven evolutionary strategies combined with flexible synthesizer representations enable automated and diverse exploration of sounds.

1 1

Haoran Wan will present his General Exam "Facilitating the Co-Design of Applications and Cellular Network both Internally and Externally" on May 12, 2025 at 12pm in CS 302.
by Gradinfo 05 May '25

05 May '25

Haoran Wan will present his General Exam "Facilitating the Co-Design of Applications and Cellular Network both Internally and Externally" on May 12, 2025 at 12pm in CS 302. Committee: Kyle Jamieson (adviser), Maria Apostolaki, Wyatt Lloyd Title: Facilitating the Co-Design of Applications and Cellular Network both Internally and Externally Abstract: 5G has emerged as the dominant cellular Radio Access Network (RAN) technology, serving as a critical last-mile solution for modern connectivity. Building on significant innovations in the physical (PHY) and medium access control (MAC) layers over LTE, 5G promises substantially higher throughput and lower latency. However, wireless networks suffer from highly variable end-to-end throughput due to user movement, channel fading and interference. Compounding this issue, the base station keeps deep packet buffers for retransmissions to mitigate packet loss caused by the varying channels, which paradoxically causes the high end-to-end latency. The variable throughput and hidden buffer pose performance challenges for many applications, ranging from congestion control for bulk downloading to interactive videos. And the closed nature of the RAN inhibits co-design of applications and the cellular network. To bridge the gap and facilitate the co-design of applications and the cellular network, we worked from two perspectives: 1) Externally, we developed a non-intrusive telemetry tool called NR-Scope to decode the control information passively from the raw radio signals, providing the MAC/PHY throughput of the base station in sub-millisecond level granularity; 2) Internally, we designed a base station module called L4Span that predicts the egress rate of the hidden queue and exposes this information through explicit congestion notification (ECN) bits in the IP header. Our work paves the way for co-designing future cellular systems and applications to fully leverage 5G’s capabilities.

1 0