March 2021 - talks - lists.cs.princeton.edu

CS Colloquium Speaker: Huacheng Yu today at 12:30pm EST
by Emily C. Lawrence 04 Mar '21

04 Mar '21

CS Colloquium Speaker Huacheng Yu, Princeton University Host: Zeev Dvir Date: Thursday, March 4 Time: 12:30pm EST Event page: [ https://www.cs.princeton.edu/events/26051 | https://www.cs.princeton.edu/events/26051 ] Register for Zoom webinar here: [ https://princeton.zoom.us/webinar/register/WN_pNvnCKFHS_qoXLEEOSMOwA | https://princeton.zoom.us/webinar/register/WN_pNvnCKFHS_qoXLEEOSMOwA ] Title: Data Structures and Algorithms in Sublinear Computation Abstract: Sublinear algorithms are sustainable under the exponential increase of data volume and processing speed. Such algorithms use a sublinear amount of resources, e.g., spending time, space, or communication that is asymptotically smaller than the input data size. Typical examples include data structures, which compute query functions of the data in sublinear time, and streaming algorithms, which make one pass over massive data streams while maintaining a sublinear-sized memory. In this talk, I will give an overview of my work in sublinear computation, focusing on succinct data structures and distributed graph sketching algorithms. I will first discuss my work on a nearly optimal data structure for the dictionary problem, for which the textbook solution uses hash tables. Then, I will talk about detecting the connectivity of graphs using distributed sketching, and my recent work showing the optimality of a well-known sketching algorithm (the AGM sketch). I will conclude the talk with discussion on future directions and my other work in theoretical computer science. Bio: Huacheng Yu is an associate research scholar in the Department of Computer Science at Princeton University. His research interests include data structures and streaming algorithms, and other directions in theoretical computer science such as communication complexity and graph algorithms. Prior to Princeton, Huacheng was a postdoctoral researcher at Harvard University hosted by Jelani Nelson and Madhu Sudan. He received his Ph.D. from Stanford University (advised by Ryan Williams and Omer Reingold) and B.Eng from Tsinghua University, both in Computer Science.

1 0

CS Colloquium Speaker: Deepak Narayanan, Wednesday, March 10 at 4:30
by Emily C. Lawrence 03 Mar '21

03 Mar '21

CS Colloquium Speaker Deepak Narayanan, Stanford University Host: Kyle Jamieson Date: Wednesday, March 10 Time: 4:30pm EST Event page: [ https://www.cs.princeton.edu/events/26048 | https://www.cs.princeton.edu/events/26048 ] Register for Zoom webinar here: [ https://princeton.zoom.us/webinar/register/WN_Gry8g80-S06bARgqKzko_Q | https://princeton.zoom.us/webinar/register/WN_Gry8g80-S06bARgqKzko_Q ] Title: Resource-Efficient Execution for Deep Learning Abstract: Deep Learning models have enabled state-of-the-art results across a broad range of applications; however, training these models is extremely time- and resource-intensive, taking weeks on clusters with thousands of expensive accelerators in the extreme case. In this talk, I will describe two systems that improve the resource efficiency of model training. The first system, PipeDream, proposes the use of pipelining to accelerate distributed training. Pipeline parallelism facilitates model training with lower communication overhead than previous methods while still ensuring high compute resource utilization. Pipeline parallelism also enables the efficient training of large models that do not fit on a single worker. Pipeline parallelism is being used at Facebook, Microsoft, OpenAI, and Nvidia for efficient large-scale model training. The second system, Gavel, determines how resources in a shared cluster with heterogeneous compute resources (e.g., different types of hardware accelerators) should be partitioned among different users to optimize objectives specified over multiple training jobs. Gavel can improve various scheduling objectives, such as average completion time, makespan, or cloud computing resource cost, by up to 3.5x. I will conclude the talk with discussion on future directions for optimizing Machine Learning systems. Bio: Deepak Narayanan is a final-year PhD student at Stanford University advised by Prof. Matei Zaharia. He is interested in designing and building software to improve the runtime performance and efficiency of emerging machine learning and data analytics workloads on modern hardware. His work is supported by a NSF graduate fellowship.

1 0

Reminder: Huacheng Yu tomorrow at 12:30pm EST
by Emily C. Lawrence 03 Mar '21

03 Mar '21

CS Colloquium Speaker Huacheng Yu, Princeton University Host: Zeev Dvir Date: Thursday, March 4 Time: 12:30pm EST Event page: [ https://www.cs.princeton.edu/events/26051 | https://www.cs.princeton.edu/events/26051 ] Register for Zoom webinar here: [ https://princeton.zoom.us/webinar/register/WN_pNvnCKFHS_qoXLEEOSMOwA | https://princeton.zoom.us/webinar/register/WN_pNvnCKFHS_qoXLEEOSMOwA ] Title: Data Structures and Algorithms in Sublinear Computation Abstract: Sublinear algorithms are sustainable under the exponential increase of data volume and processing speed. Such algorithms use a sublinear amount of resources, e.g., spending time, space, or communication that is asymptotically smaller than the input data size. Typical examples include data structures, which compute query functions of the data in sublinear time, and streaming algorithms, which make one pass over massive data streams while maintaining a sublinear-sized memory. In this talk, I will give an overview of my work in sublinear computation, focusing on succinct data structures and distributed graph sketching algorithms. I will first discuss my work on a nearly optimal data structure for the dictionary problem, for which the textbook solution uses hash tables. Then, I will talk about detecting the connectivity of graphs using distributed sketching, and my recent work showing the optimality of a well-known sketching algorithm (the AGM sketch). I will conclude the talk with discussion on future directions and my other work in theoretical computer science. Bio: Huacheng Yu is an associate research scholar in the Department of Computer Science at Princeton University. His research interests include data structures and streaming algorithms, and other directions in theoretical computer science such as communication complexity and graph algorithms. Prior to Princeton, Huacheng was a postdoctoral researcher at Harvard University hosted by Jelani Nelson and Madhu Sudan. He received his Ph.D. from Stanford University (advised by Ryan Williams and Omer Reingold) and B.Eng from Tsinghua University, both in Computer Science.

1 0

Next Week's Virtual Wilks Statistics Seminar: Monday, March 8th at 12:30pm EST: Daniel Soudry, Technion
by Emily C. Lawrence 02 Mar '21

02 Mar '21

***Next Week’s Virtual Wilks Statistics Seminar*** DATE: Monday, March 8th, 2021 TIME: 12:30pm EST LOCATION: Virtual LINK: [ https://princeton.zoom.us/j/91215738939?pwd=RzNISGMxMTMydG5ZdjFEcS9aRnF4dz09 | https://princeton.zoom.us/j/91215738939?pwd=RzNISGMxMTMydG5ZdjFEcS9aRnF4dz09 ] SPEAKER: Daniel Soudry, Technion TITLE : Surprises in Deep Learning Training ABSTRACT: We make a few interesting observations regarding Deep Neural Networks (DNNS) training: 1. We find empirically that the neural gradients (i.e., backpropagated errors) have a near-lognormal distribution during training. We show how this can be used to analytically derive pruning thresholds and quantization formats - which enable efficient training. 2. DNNs are typically initialized with (a) random weights in order to (b) break symmetry and (c) promote feature diversity. We demonstrate a, b, and c are not necessary at initialization, in order to obtain high accuracy at the end of the training. 3. Quantization of full precision trained DNNs while retaining high accuracy typically requires "fine-tuning" the model on a large dataset. We generate synthetic data from the DNN Batch-Norm statistics; we then use it "fine-tune" the DNN to good performance, thus eliminating the need for real data . 4. Asynchronous training causes a degradation in generalization, even after training has converged to a steady-state. We close this gap by adjusting hyper-parameters according to a theoretical framework aiming to maintain minima stability. Bio: Daniel is an assistant professor and in the Electrical Engineering Department at the Technion, working in the areas of machine learning and neural networks. His recent works focus on resource efficiency and implicit bias in neural networks. He did his post-doc working with Prof. Liam Paninski in the Department of Statistics and the Center for Theoretical Neuroscience at Columbia University, and his Ph.D. in the Electrical Engineering Department at the Technion. He is the recipient of the Gruss Lipper Fellowship, the Taub Fellowship, the Goldberg Award, and Intel's Rising Star Faculty Award.

1 0

3/4/2021 ECE Seminar - Xiaojing Liao, Indiana University Bloomington
by Emily C. Lawrence 02 Mar '21

02 Mar '21

ECE SEMINAR Speaker: Xiaojing Liao, Indiana University Bloomington Title: Toward Automatically Evaluating Security Risks and Providing Cyber Threat Intelligence Day: Thursday, March 4, 2021 Time: 4:30 pm Room: Please register at [ https://princeton.zoom.us/meeting/register/tJEuduyrpjkqGtbe3yU5w-iYC0HxIIcM… | https://princeton.zoom.us/meeting/register/tJEuduyrpjkqGtbe3yU5w-iYC0HxIIcM… ] Host: Prateek Mittal Abstract: Program security analysis has been studied for decades. Various techniques, such as fuzzing, taint analysis, symbolic execution, have demonstrated their successes in vulnerability assessment. Today, the availability of a large amount of program semantic data (e.g., manuals, developer documentation, related web content), and the advance of artificial intelligence technologies make it increasingly feasible to simulate human intelligence in understanding program semantics to discover software vulnerability automatically. In this talk, I will discuss my research toward in-depth and systematic semantic supports for automatic vulnerability assessment. Particularly, I will focus on two systems — Advance and Dilution — which automatically analyzes the developer’s guide to infer potential security flaws and API misuse, respectively. Bio: Xiaojing Liao is an Assistant Professor in the Department of Computer Science at Indiana University Bloomington. Her research interests include data-driven security and privacy, with specific focuses on system security, cybercrime, as well as cyber-physical systems security and privacy. She has published papers on leading system security venues such as S&P (Oakland), Usenix Security, CCS, and NDSS. She is the recipient of the ACM SIGSAC Dissertations Award and NDSS Distinguish Paper Award.

1 0

CS Colloquium Speaker: Amy Zhang, Monday, March 8 at 4:30
by Emily C. Lawrence 02 Mar '21

02 Mar '21

CS Colloquium Speaker Amy Zhang, McGill University and the Mila Institute Date: Monday, March 8 Time: 4:30pm EST Host: Tom Griffiths Event page: [ https://www.cs.princeton.edu/events/26041 | https://www.cs.princeton.edu/events/26041 ] Register for Zoom webinar here: [ https://princeton.zoom.us/webinar/register/WN_yb-TRAQJSwyBTo7RSDfhUA | https://princeton.zoom.us/webinar/register/WN_yb-TRAQJSwyBTo7RSDfhUA ] Title: Exploiting Latent Structure and Bisimulation Metrics for Better Generalization in Reinforcement Learning Abstract: The advent of deep learning has shepherded unprecedented progress in various fields of machine learning. Despite recent advances in deep reinforcement learning (RL) algorithms, however, there is no method today that exhibits anywhere near the generalization that we have seen in computer vision and NLP. Indeed, one might ask whether deep RL algorithms are even capable of the kind of generalization that is needed for open-world environments. This challenge is fundamental and will not be solved with incremental algorithmic advances. In this talk, we propose to incorporate different assumptions that better reflect the real world and allow the design of novel algorithms with theoretical guarantees to address this fundamental problem. We first present how state abstractions can accelerate reinforcement learning from rich observations, such as images, without relying either on domain knowledge or pixel-reconstruction. Our goal is to learn state abstractions that both provide for effective downstream control and invariance to task-irrelevant details. We use bisimulation metrics to quantify behavioral similarity between states, and learn robust latent representations which encode only the task-relevant information from observations. We provide theoretical guarantees for the learned approximate abstraction and extend this notion to families of tasks with varying dynamics. Bio: I am a final year PhD candidate at McGill University and the Mila Institute, co-supervised by Profs. Joelle Pineau and Doina Precup. I am also a researcher at Facebook AI Research. My work focuses on bridging theory and practice through learning approximate state abstractions and learning representations for generalization in reinforcement learning. I previously obtained an M.Eng. in EECS and dual B.Sci. degrees in Mathematics and EECS from MIT.

1 0

CS Colloquium Speaker: Sanmi Koyejo, today at 12:30
by Emily C. Lawrence 01 Mar '21

01 Mar '21

CS Colloquium Speaker Sanmi Koyejo, University of Illinois at Urbana-Champaign Host: Barbara Engelhardt Date: Monday, March 1 Time: 12:30pm EST Event page: https://www.cs.princeton.edu/events/26040 Register for Zoom webinar here: https://princeton.zoom.us/webinar/register/WN_XMi3YQUVRn20UGAhEncHpQ Title: The Measurement and Mismeasurement of Trustworthy ML Abstract: Across healthcare, science, and engineering, we increasingly employ machine learning (ML) to automate decision-making that, in turn, affects our lives in profound ways. However, ML can fail, with significant and long-lasting consequences. Reliably measuring such failures is the first step towards building robust and trustworthy learning machines. Consider algorithmic fairness, where widely-deployed fairness metrics can exacerbate group disparities and result in discriminatory outcomes. Moreover, existing metrics are often incompatible. Hence, selecting fairness metrics is an open problem. Measurement is also crucial for robustness, particularly in federated learning with error-prone devices. Here, once again, models constructed using well-accepted robustness metrics can fail. Across ML applications, the dire consequences of mismeasurement are a recurring theme. This talk will outline emerging strategies for addressing the measurement gap in ML and how this impacts trustworthiness. Bio: Sanmi (Oluwasanmi) Koyejo is an Assistant Professor in the Department of Computer Science at the University of Illinois at Urbana-Champaign. Koyejo's research interests are in developing the principles and practice of trustworthy machine learning. Additionally, Koyejo focuses on applications to neuroscience and healthcare. Koyejo has been the recipient of several awards, including a best paper award from the conference on uncertainty in artificial intelligence (UAI), a Kavli Fellowship, an IJCAI early career spotlight, and a trainee award from the Organization for Human Brain Mapping (OHBM). Koyejo serves on the board of the Black in AI organization.

1 0