Daniel Suo will present his FPO "Scaling Machine Learning in Practice" on Wednesday, May 10, 2023 at 3pm in CS 402.
Daniel Suo will present his FPO "Scaling Machine Learning in Practice" on Wednesday, May 10, 2023 at 3pm in CS 402. The members of his committee are as follows: Examiners: Kai Li (Adviser), Olga Troyanskaya, and Ryan Adams; Readers: Elad Hazan (Co-Adviser) and Naman Agarwal (Google LLC) All are welcome to attend. In recent years, machine learning has become pervasive, powering algorithmic clinicians, translators, and world-beating go masters. As practitioners build on this success, they repeatedly observe that scale–data, model size, compute–is critical. However, scale is now a challenge in and of itself; simple tasks such as gathering data become formidable, even prohibitive. In this talk, we discuss techniques for addressing scale in three areas: 1. Differential reinforcement learning for physical devices: reinforcement learning has emerged as a potential strategy for machines to make decisions in complex, dynamic environments. However, successful demonstrations have required vast experience to learn an optimal policy, making real-world physical applications particularly challenging. We present a method that uses limited experience to learn a differentiable simulator of a physical system (medical ventilator) and then uses gradient methods on the simulator to learn a state-of-the-art policy for controlling that system. 2. Practical optimization for deep learning: optimization is an essential aspect of deep learning. However, while a constellation of optimization algorithms dot the literature, the low burden of proof and empirical nature of deep learning has led practitioners to rely on defaults (i.e., Adagrad, Adam) rather than view optimization as a lever for progress. To rigorously test ideas in optimization, we introduce a comprehensive benchmark that currently includes 8 deep learning workloads and rules for training procedures, computational budget, and evaluation. 3. Scaling computer systems via thread scheduling: large global-scale applications are expensive and complex to operate let alone optimize. As a result, many simple parameters that govern important behaviors of these systems are simply set once and never touched again. However, we show that these parameters present low-hanging fruit for significant eciency improvements
Daniel Suo will present his FPO "Scaling Machine Learning in Practice" on Wednesday, May 10, 2023 at 3pm in CS 402. The members of his committee are as follows: Examiners: Kai Li (Adviser), Olga Troyanskaya, and Ryan Adams; Readers: Elad Hazan (Co-Adviser) and Naman Agarwal (Google LLC) All are welcome to attend. In recent years, machine learning has become pervasive, powering algorithmic clinicians, translators, and world-beating go masters. As practitioners build on this success, they repeatedly observe that scale–data, model size, compute–is critical. However, scale is now a challenge in and of itself; simple tasks such as gathering data become formidable, even prohibitive. In this talk, we discuss techniques for addressing scale in three areas: 1. Differential reinforcement learning for physical devices: reinforcement learning has emerged as a potential strategy for machines to make decisions in complex, dynamic environments. However, successful demonstrations have required vast experience to learn an optimal policy, making real-world physical applications particularly challenging. We present a method that uses limited experience to learn a differentiable simulator of a physical system (medical ventilator) and then uses gradient methods on the simulator to learn a state-of-the-art policy for controlling that system. 2. Practical optimization for deep learning: optimization is an essential aspect of deep learning. However, while a constellation of optimization algorithms dot the literature, the low burden of proof and empirical nature of deep learning has led practitioners to rely on defaults (i.e., Adagrad, Adam) rather than view optimization as a lever for progress. To rigorously test ideas in optimization, we introduce a comprehensive benchmark that currently includes 8 deep learning workloads and rules for training procedures, computational budget, and evaluation. 3. Scaling computer systems via thread scheduling: large global-scale applications are expensive and complex to operate let alone optimize. As a result, many simple parameters that govern important behaviors of these systems are simply set once and never touched again. However, we show that these parameters present low-hanging fruit for significant eciency improvements _______________________________________________ talks mailing list talks@lists.cs.princeton.edu To edit subscription settings or remove yourself, use this link: https://lists.cs.princeton.edu/mailman/listinfo/talks
participants (1)
-
Nicki Mahler