August 2024 - talks - lists.cs.princeton.edu

CS Colloquium speakers: week of February 26
by Emily C. Lawrence 01 Apr '25

01 Apr '25

CS Colloquium speakers Speaker: Jialin Ding, Amazon Web Services Date: Monday, February 26 Time: 12:30pm EST Location: CS 105 Host: Wyatt Lloyd Event page: https://www.cs.princeton.edu/events/26578 Register for live-stream online here: https://princeton.zoom.us/webinar/register/WN_10z_l8MYRr21ZOuPqA8Odw Title: Instance-Optimization: Rethinking Database Design for the Next 1000X Abstract: Modern database systems aim to support a large class of different use cases while simultaneously achieving high performance. However, as a result of their generality, databases often achieve adequate performance for the average use case but do not achieve the best performance for any individual use case. In this talk, I will describe my work on designing databases that use machine learning and optimization techniques to automatically achieve performance much closer to the optimal for each individual use case. In particular, I will present my work on instance-optimized database storage layouts, in which the co-design of data structures and optimization policies improves query performance in analytic databases by orders of magnitude. I will highlight how these instance-optimized data layouts address various challenges posed by real-world database workloads and how I implemented and deployed them in production within Amazon Redshift, a widely-used commercial database system. Bio: Jialin Ding is an Applied Scientist at AWS. Before that, he received his PhD in computer science from MIT, advised by Tim Kraska. He works broadly on applying machine learning and optimization techniques to improve data management systems, with a focus on building databases that automatically self-optimize to achieve high performance for any specific application. His work has appeared in top conferences such as SIGMOD, VLDB, and CIDR, and has been recognized by a Meta Research PhD Fellowship. To learn more about Jialin’s work, please visit [ https://jialinding.github.io/ | https://jialinding.github.io/ ] . CSML/CS/PSY Colloquium Speaker: Brenden Lake, New York University Date: Tuesday, February 27 Time: 12:30pm EST Location: CS 105 Host: Tom Griffiths Event page: https://csml.princeton.edu/events/csmlpsycs-seminar Register for live-stream online here: https://princeton.zoom.us/webinar/register/WN_aUfLfVQFSIqV2RDSYu4big Title: Towards more human-like learning in machines: Bridging the data and generalization gaps Abstract: There is an enormous data gap between how AI systems and children learn: The best LLMs now learn language from text with a word count in the trillions, whereas it would take a child roughly 100K years to reach those numbers through speech (Frank, 2023, "Bridging the data gap"). There is also a clear generalization gap: whereas machines struggle with systematic generalization, children can excel. For instance, once a child learns how to "skip," they immediately know how to "skip twice" or "skip around the room with their hands up" due to their compositional skills. In this talk, I'll describe two case studies in addressing these gaps. 1) The data gap: We train deep neural networks from scratch (using DINO, CLIP, etc.), not on large-scale data from the web, but through the eyes and ears of a single child. Using head-mounted video recordings from a child as training data (<200 hours of video slices over 26 months), we show how deep neural networks can perform challenging visual tasks, acquire many word-referent mappings, generalize to novel visual referents, and achieve multi-modal alignment. Our results demonstrate how today's AI models are capable of learning key aspects of children's early knowledge from realistic input. 2) The generalization gap: Can neural networks capture human-like systematic generalization? We address a 35-year-old debate catalyzed by Fodor and Pylyshyn's classic article, which argued that standard neural networks are not viable models of the mind because they lack systematic compositionality -- the algebraic ability to understand and produce novel combinations from known components. We'll show how neural networks can achieve human-like systematic generalization when trained through meta-learning for compositionality (MLC), a new method for optimizing the compositional skills of neural networks through practice. With MLC, neural networks can match human performance and solve several machine learning benchmarks. Given these findings, we'll discuss the paths forward for building machines that learn, generalize, and interact in more human-like ways based on more natural input. Related articles: Vong, W. K., Wang, W., Orhan, A. E., and Lake, B. M (2024). Grounded language acquisition through the eyes and ears of a single child. Science, 383, 504-511. Orhan, A. E., and Lake, B. M. (in press). Learning high-level visual representations from a child’s perspective without strong inductive biases. Nature Machine Intelligence. Lake, B. M. and Baroni, M. (2023). Human-like systematic generalization through a meta-learning neural network. Nature, 623, 115-121. Bio: Brenden M. Lake is an Assistant Professor of Data Science and Psychology at New York University. He received his M.S. and B.S. in Symbolic Systems from Stanford University in 2009, and his Ph.D. in Cognitive Science from MIT in 2014. He was a postdoctoral Data Science Fellow at NYU from 2014-2017. Brenden is a recipient of the Robert J. Glushko Prize for Outstanding Doctoral Dissertation in Cognitive Science and the MIT Technology Review 35 Innovators Under 35. His research was also selected by Scientific American as one of the 10 most important advances of 2016. Brenden's research focuses on computational problems that are easier for people than they are for machines, such as learning new concepts from just a few examples, learning by asking questions, learning by generating new goals, and learning by producing novel combinations of known components. CS Colloquium Speaker: Eric Mitchell, Stanford University Date: Thursday, February 29 Time: 12:30pm EST Location: CS 105 Host: Karthik Narasimhan Event page: https://www.cs.princeton.edu/events/26587 Register for live-stream online here: TBA Talk info TBA

1 19

CS Colloquium Speakers: week of March 4
by Emily C. Lawrence 03 Dec '24

03 Dec '24

CS Colloquium Speakers Speaker: Sewon Min, University of Washington Date: Monday, March 4 Time: 12:30pm EST Location: CS 105 Host: Danqi Chen Event page: https://www.cs.princeton.edu/events/26579 Register for live-stream online here: https://princeton.zoom.us/webinar/register/WN_5uVGEpXNTVe8eBNSLpxKtg Title: Rethinking Data Use in Large Language Models Abstract: Large language models (LMs) such as ChatGPT have revolutionized natural language processing and artificial intelligence more broadly. In this talk, I will discuss my research on understanding and advancing these models, centered around how they use the very large text corpora they are trained on. First, I will describe our efforts to understand how these models learn to perform new tasks after training, demonstrating that their so-called in context learning capabilities are almost entirely determined by what they learn from the training data. Next, I will introduce a new class of LMs—nonparametric LMs—that repurpose this training data as a data store from which they retrieve information for improved accuracy and updatability. I will describe my work on establishing the foundations of such models, including one of the first broadly used neural retrieval models and an approach that simplifies a traditional, two-stage pipeline into one. I will also discuss how nonparametric models open up new avenues for responsible data use, e.g., by segregating permissive and copyrighted text and using them differently. Finally, I will envision the next generation of LMs we should build, focusing on efficient scaling, improved factuality, and decentralization. Bio: Sewon Min is a Ph.D. candidate in the Paul G. Allen School of Computer Science & Engineering at the University of Washington. Her research focuses on language models (LMs): studying the science of LMs, and designing new model classes and learning methods that make LMs more performant and flexible. She also studies LMs in information-seeking, legal, and privacy contexts. She is a co-organizer of multiple tutorials and workshops, including most recently at ACL 2023 on Retrieval-based Language Models and Applications and upcoming at ICLR 2024 on Mathematical and Empirical Understanding of Foundation Models. She won a paper award at ACL 2023, received a J.P. Morgan Fellowship, and was named an EECS rising star in 2022. Speaker: Lianmin Zheng, University of California, Berkeley Date: Tuesday, March 5 Time: 12:30pm EST Location: CS 105 Host: Ravi Netravali Event page: https://www.cs.princeton.edu/events/26593 Register for live-stream online here: https://princeton.zoom.us/webinar/register/WN_ghiT2HbhSPqKBouJ-VwzFg Title: Scalable and Efficient Systems for Large Language Models Abstract: Large Language Models (LLMs) have been driving recent breakthroughs in AI. These advancements would not have been possible without the support of scalable and efficient infrastructure systems. In this talk, I will introduce several underlying systems I have designed and built to support the entire model lifecycle, from training to deployment to evaluation. First, I will present Alpa, a system for large-scale model-parallel training that automatically generates execution plans unifying data, operator, and pipeline parallelism. Next, I will discuss efficient deployment systems, covering the frontend programming interface and backend runtime optimizations for high-performance inference. Finally, I will complete the model lifecycle by presenting our model evaluation efforts, including the crowdsourced live benchmark platform, Chatbot Arena, and the automatic evaluation pipeline, LLM-as-a-Judge. These projects have collectively laid a solid foundation for large language model systems, being widely adopted by leading LLM developers and companies. I will conclude by outlining some future directions of machine learning systems, such as co-optimizing across the full stack for building AI-centric applications. Bio: Lianmin Zheng is a Ph.D. student in the EECS department at UC Berkeley, advised by Ion Stoica and Joseph E. Gonzalez. His research interests include machine learning systems, large language models, compilers, and distributed systems. He builds full-stack, scalable, and efficient systems to advance the development of AI. He co-founded LMSYS.org, where he leads impactful open-source large language model projects such as Vicuna and Chatbot Arena, which have received millions of downloads and served millions of users. He also co-organized the Big Model Tutorial at ICML 2022. He has received a Meta Ph.D. Fellowship, an IEEE Micro Best Paper Award, and an a16z open-source AI grant. Speaker: Emma Dauterman, University of California, Berkeley Date: Thursday, March 7 Time: 12:30pm EST Location: CS 105 Host: Amit Levy Event page: https://www.cs.princeton.edu/events/26585 Register for live-stream online here: https://princeton.zoom.us/webinar/register/WN_soNesOKXSFml9MPaTx_1iw Title: Secure systems from insecure components Abstract: In many computer systems today, an attacker that breaks one system component can steal data from millions of users. In this talk, I will present two systems that can withstand component compromise. I will describe (1) a single sign-on system that protects user security and privacy from a compromised single sign-on server, and (2) a secure-hardware-based backup service that protects user backups from compromised secure hardware devices. These systems provide strong security and privacy properties while taking into account practical constraints such as compatibility requirements, hardware limitations, and user expectations. Each splits user secrets across different system components, using new cryptographic tools to provide necessary functionality while protecting user data. Bio: Emma Dauterman is a Ph.D. candidate at UC Berkeley where she is advised by Raluca Ada Popa and Ion Stoica. Her research interests include computer security, systems, and applied cryptography. She has received the Microsoft Research Ada Lovelace fellowship, the NSF graduate research fellowship, and a UC Berkeley EECS excellence award.

1 22

Kun Woo Cho will present his FPO "Programmable Smart Radio Environments: From Theory to Hardware Implementation" on October 1, 2024 at 3pm in CS 302.
by Gradinfo 26 Sep '24

26 Sep '24

Kun Woo Cho will present his FPO "Programmable Smart Radio Environments: From Theory to Hardware Implementation" on October 1 , 2024 at 3pm in CS 302. Examiner: Prof. Kyle Jamieson (Advisor, CS), Prof. Yasaman Ghasempour (ECE), Prof. Andrew Appel (CS) Reader: Prof. Wyatt Lloyd (CS), Prof. Omid Abari (UCLA), Dr. Ranveer Chandra (MSR) All are welcome to attend. Title: Programmable Smart Radio Environments: From Theory to Hardware Implementation Abstract: Today ’s wireless networks are undergoing a rapid transformation, scaling in traffic vol- ume, spectral efficiency, and radio count as never seen before. This thesis addresses the critical challenges emerging from this evolution in next-generation (NextG) wire- less networks, focusing on realizing three key services: enhanced mobile broadband (Gbps or above), ultra-reliable low latency communication (order of milliseconds), and massive machine type communication (up to one million per squared kilometer). To meet these diverse and demanding requirements, this thesis poses a central question: Can we build a smarter radio environment controlled and learned by soft- ware, configuring itself in real-time to meet different application needs? Current approaches to handle uncontrolled wireless signals are end-to-end. Unfortunately, sending and receiving endpoints are limited in their ability to shape this inherent propagation behavior. For instance, although multiple antennas can shape the beam pattern departing the sender, it cannot control how the resulting signals arrive at the receiver after traversing environmental obstacles. By focusing on changing the envi- ronment itself rather than the communication endpoints, this thesis offers a significant shift in design paradigms for modern wireless networks. First, millimeter-wave (mmWave) technology offers multi-Gbps data rates due to its wide spectral bands. However, the high frequency of mmWave signals makes them vulnerable to blockage by walls, people, and obstacles, significantly limiting their practical applications. This thesis introduces mmWall, a programmable smart surface deployed on buildings to fully control the mmWave radio environment. Com- prising over 4,000 sub-millimeter meta-materials, mmWall can steer signals through the surface or reflect them, bringing outdoor mmWave signals indoors and bypassing obstacles. Also, this thesis proposes Wall-Street, a smart surface installed on vehicles to provide seamless and reliable mmWave connectivity in high-mobility scenarios. Second, satellite networks uses constellations of thousands of satellites to provide low latency communication and global coverage. However, these networks face two significant challenges: outages due to transient blockage, and complications in beam alignment between users and satellites due to the use of different frequency sub-bands in the uplink and downlink directions. Extending the concept of a programmable smart radio to satellite communications, this thesis introduces Wall-E, a dual-band smart surface that mitigates signal blockage and enhances reliability of satellite-to- ground links, and Monolith, a smart surface that boosts inter-satellite link capacity. Third, while these smart surfaces address challenges at the physical layer, we also tackle issues at the link/MAC layer, particularly in massive Internet of Things (IoT) networks. This thesis introduces Cross-Link Channel Prediction (CLCP), a ma- chine learning technique that learns the radio environment and accordingly allocates networking resources to a large number of IoT devices. This AI-driven approach com- plements my programmable surfaces, creating a comprehensive smart radio solution for NextG networks.

1 1

Yue Tan will present her FPO "FAASTEN: An Architecture and implementation for securing cloud applications" on September 5, 2024 at 11am in CS 302.
by Gradinfo 15 Aug '24

15 Aug '24

Yue Tan will present her FPO "FAASTEN: An Architecture and implementation for securing cloud applications" on September 5, 2024 at 11am in CS 302. The members of her committee are as follows: Amit Levy (advisor), Maria Apostolaki, Wyatt Lloyd Readers: Mae Milano and Marcus Peinado (Microsoft Research) All are welcome to attend. The FPO abstract follows below: Modern web applications have evolved into intricate networks of micro-applications, posing challenges in ensuring the security of user data confidentiality and integrity. In the current discretionary security paradigm, developers must enforce policies for every microapplication and all data paths that can arise at run-time. A promising alternative paradigm, decentralized information flow control (DIFC), suggests that the underlying system should enforce policies by tracking information flows. As the cloud has become a standard operating system for developing web applications, this paper proposes the first DIFC design for the cloud. We present Faasten, an architecture and implementation for securing cloud applications. Faasten includes a DIFC model and a FaaS-inspired system interface that implements the DIFC model. We provide a detailed interface security analysis and showcase how three representative cloud applications benefit from Faasten. We also show that Faasten introduces negligible latencies in controlling information flows and policy storage costs.

1 0

Samuel Ginzburg will present his FPO "VectorVisor: A Binary Translation Scheme for Throughput-Oriented GPU Acceleration" on Thursday, August 29, 2024 in CS 302 at 1pm.
by Gradinfo 15 Aug '24

15 Aug '24

Samuel Ginzburg will present his FPO "VectorVisor: A Binary Translation Scheme for Throughput-Oriented GPU Acceleration" on Thursday, August 29, 2024 in CS 302 at 1pm. The members of his FPO committee follows below: Examiners: Michael Freedman (Advisor), Wyatt Lloyd, and Amit Levy Readers: Mae Milano and Mohammad Shahrad (UBC) All are welcome to attend. Please see abstract below. Beyond conventional graphics applications, general-purpose GPU acceleration has had significant impact on machine learning and scientific computing workloads. Yet, it has failed to see widespread use for server-side applications, which we argue is because GPU programming models offer a level of abstraction that is either too low-level (e.g., OpenCL, CUDA) or too high-level (e.g., TensorFlow, Halide), depending on the language. Not all applications fit into either category, resulting in lost opportunities for GPU acceleration. We introduce VectorVisor, a vectorized binary translator that enables new opportunities for GPU acceleration by introducing a novel programming model for GPUs. With VectorVisor, many copies of the same server-side application are run concurrently on the GPU, where VectorVisor mimics the abstractions provided by CPU threads. To achieve this goal, we demonstrate how to (i) provide cross-platform support for system calls and recursion using continuations and (ii) make full use of the excess register file capacity and high memory bandwidth of GPUs. We then demonstrate that our binary translator is able to transparently accelerate certain classes of computebound workloads, gaining significant improvements in throughput-per-dollar of up to 2.9× compared to Intel x86-64 VMs in the cloud, and in some cases match the throughput-per-dollar of native CUDA baselines.

1 0

Kaifeng Lyu will present his FPO "Implicit Bias in Deep Learning Optimization: A Mathematical Examination" on Friday, August 9, 2024 at 10:30 AM in CS 402 and Zoom.
by CS Grad Department 02 Aug '24

02 Aug '24

Kaifeng Lyu will present his FPO " Implicit Bias in Deep Learning Optimization: A Mathematical Examination " on Friday, August 9, 2024 at 10:30 AM in CS 402 and Zoom. Location: Zoom link: https://princeton.zoom.us/j/99448789401 The members of Kaifeng ’s committee are as follows: Examiners: Sanjeev Arora (Adviser), Elad Hazan , Chi Jin Readers: Danqi Chen , Jason Lee A copy of his thesis is available upon request. Please email gradinfo(a)cs.princeton.edu if you would like a copy of the thesis. Everyone is invited to attend his talk. Abstract follows below: Deep learning has achieved remarkable success in recent years, yet training neural networks often involves a delicate combination of guesswork and hyperparameter tuning. A critical aspect of this process is the “implicit bias” of optimization methods, where minor changes in the optimization setup—without affecting the small training loss at convergence—can drastically shift the solution to which the model converges, thereby affecting test performance. This dissertation presents a collection of results that mathematically characterize this implicit bias in various training regimes. The first part of this dissertation explores how gradient descent, even without explicit regularization, can converge to solutions that maximize the margin. Previous results have established the first-order optimality of margin for homogeneous neural networks in general, but the global optimality of margin is not guaranteed due to their non-convex nature. This dissertation provides in-depth theoretical analyses when data has simple structures: for linearly separable data, we present both positive and negative results on whether the global optimality of margin can be attained. Furthermore, we show how this margin-based view can be used to explain interesting generalization phenomena in training neural networks with or without explicit regularization, including the simplicity bias and grokking phenomena. The second part of the dissertation presents two results that capture the implicit biases induced by finite learning rate. Many existing analyses, including the margin-based ones in the first part, describe implicit biases that hold even when the learning rate is infinitesimal. However, practical implementations use finite learning rates, which have been empirically observed to benefit generalization. We analyze how full-batch GD with finite learning rates, combined with key training components like normalization layers and weight decay, create a bias towards flatter minima, which are positively correlated with better generalization. Additionally, we study the implicit bias in stochastic optimization and derive rigorous approximations for the dynamics of adaptive gradient methods like Adam and RMSprop via Stochastic Differential Equations (SDEs) to capture the effect of finite learning rates. Based on this, we also derive the square root scaling rule as a practical guideline for adjusting the optimization hyperparameters of adaptive gradient methods when changing batch size.

1 0