- talks - lists.cs.princeton.edu

Colloquium Speaker: Francis Bach, Friday 11/15 at 12:30pm
by Emily C. Lawrence 08 Nov '19

08 Nov '19

Distinguished Colloquium Speaker Francis Bach from INRIA Friday, November 15 - 12:30pm Computer Science - Room 105 Hosts: Barbara Engelhardt, Sanjeev Arora, Peter Ramadge [ https://www.cs.princeton.edu/events/25882 | https://www.cs.princeton.edu/events/25882 ] Can learning theory resist deep learning? Machine learning algorithms are ubiquitous in most scientific, industrial and personal domains, with many successful applications. As a scientific field, machine learning has always been characterized by the constant exchanges between theory and practice, with a stream of algorithms that exhibit both good empirical performance on real-world problems and some form of theoretical guarantees. Many of the recent and well publicized applications come from deep learning, where these exchanges are harder to make, in part because the objective functions used to train neural networks are not convex. In this talk, I will present recent results on the global convergence of gradient descent for some specific non-convex optimization problems, illustrating these difficulties and the associated pitfalls (joint work with Lénaïc Chizat and Edouard Oyallon ). Bio: Francis Bach is a researcher at INRIA, leading since 2011 the SIERRA project-team, which is part of the Computer Science Department at Ecole Normale Supérieure , and a joint team between CNRS, ENS and INRIA. Since 2016, he is an adjunct Professor at Ecole Normale Supérieure . He completed his Ph.D. in Computer Science at U.C. Berkeley, working with Professor Michael Jordan, and spent two years in the Mathematical Morphology group at Ecole des Mines de Paris, he then joined the WILLOW project-team at INRIA/ Ecole Normale Superieure /CNRS from 2007 to 2010. He obtained in 2009 a Starting Grant and in 2016 a Consolidator Grant from the European Research Council, and received the Inria young researcher prize in 2012, the ICML test-of-time award in 2014, as well as the Lagrange prize in continuous optimization in 2018. In 2015, he was program co-chair of the International Conference in Machine learning (ICML), and general chair in 2018; he is now co-editor- inchief of the Journal of Machine Learning Research. Francis Bach is primarily interested in machine learning, and especially in graphical models, sparse methods, kernel-based learning, large-scale convex optimization, computer vision and signal processing.

1 0

TODAY! Joint Princeton Optimization Seminar/CSML Seminar: 4:30pm in Sherrerd Hall 101: Elisa Celis, Yale University
by Emily Lawrence 07 Nov '19

07 Nov '19

Speaker: Elisa Celis Assistant Professor of Statistics and Data Science Yale University Title: Optimizing for Fairness in ML Day: Thursday, November 7, 2019 Time: 4:30-5:30 Room: Sherrerd 101 Host: CSML and ORFE, S. S. Wilks Memorial Seminar in Statistics Bio: Elisa Celis is an Assistant Professor of Statistics and Data Science at Yale University. Elisa’s research focuses on problems that arise at the interface of computation and machine learning and its societal ramifications. Specifically, she studies the manifestation of social and economic biases in our online lives via the algorithms that encode and perpetuate them. Her work spans multiple areas including social computing and crowdsourcing, data science, and algorithm design with a current emphasis on fairness and diversity in artificial intelligence and machine learning. Elisa received her B.Sci. from Harvey Mudd College in 2006 in computer science & mathematics, and a Ph.D. in computer science & engineering from the University of Washington in 2012. Prior to joining Yale, she was a research scientist at Xerox Research India where she managed the crowdsourcing research thrust worldwide, and a senior research scientist at École Polytechnique Fédérale de Lausanne (EPFL) in Switzerland in computer and information sciences, where she was a co-founder of the Computation Nature and Society Think Tank. Abstract: Recent events have made evident the fact that algorithms can be discriminatory, reinforce human prejudices, accelerate the spread of misinformation, and are generally not as objective as they are widely thought to be. In this talk, I will present some vignettes from my recent work addressing the problem of social bias in ML models. In particular, correcting for housing and employment discrimination in online advertising, auditing social and intersectional biases in contextual NLP systems, and diversifying image search results via simple and fast approaches that do not require labeled training data. This work leads to new ML approaches that have the ability to alleviate bias and increase diversity while simultaneously maintaining theoretical or empirical performance with respect to their original metrics.

1 0

Colloquium Speaker: Tushar Krishna, TODAY at 12:30
by Emily C. Lawrence 07 Nov '19

07 Nov '19

CS Colloquium Speaker Tushar Krishna from Georgia Tech Thursday, November 7 - 12:30pm E-Quad, B205 Host: Margaret Martonosi [ https://www.cs.princeton.edu/events/25888 | https://www.cs.princeton.edu/events/25888 ] Enabling Continuous Learning through Synaptic Plasticity in Hardware Ever since modern computers were invented, the dream of creating artificial intelligence (AI) has captivated humanity. We are fortunate to live in an era when, thanks to deep learning (DL), computer programs have paralleled, and in many cases even surpassed human level accuracy in tasks like visual perception and speech synthesis. However, we are still far away from realizing general-purpose AI. The problem lies in the fact that the development of supervised learning based DL solutions today is mostly open loop. A typical DL model is created by hand-tuning the deep neural network (DNN) topology by a team of experts over multiple iterations, followed by training over petabytes of labeled data. Once trained, the DNN provides high accuracy for the task at hand; if the task changes, however, the DNN model needs to be re-designed and re-trained before it can be deployed. A general-purpose AI system, in contrast, needs to have the ability to constantly interact with the environment and learn by adding and removing connections within the DNN autonomously, just like our brain does. This is known as synaptic plasticity. In this talk we will present our research efforts towards enabling general-purpose AI leveraging plasticity in both the algorithm and hardware. First, we will present GeneSys (MICRO 2018), a HW-SW prototype of a closed loop learning system for continuously evolving the structure and weights of a DNN for the task at hand using genetic algorithms, providing 100-10000x higher performance and energy-efficiency over state-of-the-art embedded and desktop CPU and GPU systems. Next, we will present a DNN accelerator substrate called MAERI (ASPLOS 2018), built using light-weight, non-blocking, reconfigurable interconnects, that supports efficient mapping of regular and irregular DNNs with arbitrary dataflows , providing ~100% utilization of all compute units, resulting in 3X speedup and energy-efficiency over our prior work Eyeriss (ISSCC 2016). Finally, time permitting, we will describe our research in enabling rapid design-space exploration and prototyping of hardware accelerators using our dataflow DSL + cost-model called MAESTRO (MICRO 2019). Bio: Tushar Krishna is an Assistant Professor in the School of Electrical and Computer Engineering at Georgia Tech. He also holds the ON Semiconductor Junior Professorship. He has a Ph.D. in Electrical Engineering and Computer Science from MIT (2014), a M.S.E in Electrical Engineering from Princeton University (2009), and a B.Tech in Electrical Engineering from the Indian Institute of Technology (IIT) Delhi (2007). Before joining Georgia Tech in 2015, Dr. Krishna spent a year as a post-doctoral researcher at Intel, Massachusetts. Dr. Krishna’s research spans computer architecture, interconnection networks, networks-on-chip ( NoC ) and deep learning accelerators - with a focus on optimizing data movement in modern computing systems. Three of his papers have been selected for IEEE Micro’s Top Picks from Computer Architecture, one more received an honorable mention, and two have won best paper awards. He received the National Science Foundation (NSF) CRII award in 2018, and both a Google Faculty Award and a Facebook Faculty Award in 2019. He also received the “Class of 1940 Course Survey Teaching Effectiveness” Award from Georgia Tech in 2018.

1 0

Speaker Ravi Chugh, Friday Nov 8 at 11am
by Emily Lawrence 06 Nov '19

06 Nov '19

Title: Bidirectional Programming with Direct Manipulation Speaker: Ravi Chugh, University of Chicago Date/Time: Friday, November 8th, 11 AM — 12 PM Location: CS 402 Abstract: Traditional text-based programming environments present users with mental hurdles and tedious, inefficient development workflows. In this talk, I'll describe techniques that aim to improve the user interfaces for programming. What new capabilities can programming languages offer to users, and what new interaction mechanisms can be used to expose the new capabilities? We are developing a prototype programming environment, called Sketch-n-Sketch, to explore these questions. Regarding capabilities, we present program synthesis and repair techniques for "bidirectionally" developing and evaluating programs. Regarding interaction mechanisms, we present "direct manipulation" interfaces for interacting with programs and their execution. These systems augment traditional text-based programming with new program development workflows, which we plan to further improve and evaluate in the future. Bio: Ravi Chugh has been Assistant Professor of Computer Science at the University of Chicago since 2014. His research and teaching interests revolve around programming language technology --- type systems, synthesis algorithms, and other program analysis techniques --- particularly as they relate to challenges in software engineering and human-computer interaction. Ravi received an NSF CAREER Award in 2017, and a Neubauer Fellowship from the University of Chicago in 2015 for innovative and effective undergraduate teaching. Ravi obtained his Ph.D. from the University of California, San Diego in 2013. Picture: http://people.cs.uchicago.edu/~rchugh/static/images/ravi-2018-10.jpg

1 0

Colloquium Speaker: Jeff Mogul, Wednesday 11/13 at 12:30pm
by Emily C. Lawrence 06 Nov '19

06 Nov '19

Distinguished CS Colloquium Speaker Jeff Mogul, Google Networking Infrastructure Wednesday, November 13 - 12:30pm Computer Science - Room 105 Host: Kyle Jamieson [ https://www.cs.princeton.edu/events/25892 | https://www.cs.princeton.edu/events/25892 ] Self-Driving Networks: Is it a good idea to mix AI and computer networks? Just as you wouldn't want to be in a car without seat belts, even if it's a self-driving car, you probably don't want your Self-Driving Network ( SelfDN ) to rely completely on control-theory or ML, no matter how sophisticated or formally verified. While autonomous systems promise to solve many problems that afflict networks, just as they promise to solve many problems that afflict our roadways, they are not immune from a multitude of practical problems and unexpected consequences. At Google, we have been moving towards highly-automated networking, and along the way we have learned a lot about the risks involved, and how to build robust networks in spite of the wonders of automation. I will talk about some of the problems that are likely to afflict SelfDNs as they enter the real world. Bio: Jeff Mogul works on fast, cheap, reliable, and flexible networking infrastructure for Google. Until 2013, he was a Fellow at HP Labs, doing research primarily on computer networks and operating systems issues for enterprise and cloud computer systems; previously, he worked at the DEC/Compaq Western Research Lab. He received his PhD from Stanford in 1986, an MS from Stanford in 1980, and an SB from MIT in 1979. He is an ACM Fellow. Jeff is the author or co-author of several Internet Standards; he contributed extensively to the HTTP/1.1 specification. He was an associate editor of Internetworking: Research and Experience, and has been the chair or co-chair of a variety of conferences and workshops, including SIGCOMM, OSDI, NSDI, USENIX, HotOS , and ANCS.

1 0

Reminder: Tushar Krishna, tomorrow at 12:30
by Emily C. Lawrence 06 Nov '19

06 Nov '19

CS Colloquium Speaker Tushar Krishna from Georgia Tech Thursday, November 7 - 12:30pm E-Quad, B205 Host: Margaret Martonosi [ https://www.cs.princeton.edu/events/25888 | https://www.cs.princeton.edu/events/25888 ] Enabling Continuous Learning through Synaptic Plasticity in Hardware Ever since modern computers were invented, the dream of creating artificial intelligence (AI) has captivated humanity. We are fortunate to live in an era when, thanks to deep learning (DL), computer programs have paralleled, and in many cases even surpassed human level accuracy in tasks like visual perception and speech synthesis. However, we are still far away from realizing general-purpose AI. The problem lies in the fact that the development of supervised learning based DL solutions today is mostly open loop. A typical DL model is created by hand-tuning the deep neural network (DNN) topology by a team of experts over multiple iterations, followed by training over petabytes of labeled data. Once trained, the DNN provides high accuracy for the task at hand; if the task changes, however, the DNN model needs to be re-designed and re-trained before it can be deployed. A general-purpose AI system, in contrast, needs to have the ability to constantly interact with the environment and learn by adding and removing connections within the DNN autonomously, just like our brain does. This is known as synaptic plasticity. In this talk we will present our research efforts towards enabling general-purpose AI leveraging plasticity in both the algorithm and hardware. First, we will present GeneSys (MICRO 2018), a HW-SW prototype of a closed loop learning system for continuously evolving the structure and weights of a DNN for the task at hand using genetic algorithms, providing 100-10000x higher performance and energy-efficiency over state-of-the-art embedded and desktop CPU and GPU systems. Next, we will present a DNN accelerator substrate called MAERI (ASPLOS 2018), built using light-weight, non-blocking, reconfigurable interconnects, that supports efficient mapping of regular and irregular DNNs with arbitrary dataflows , providing ~100% utilization of all compute units, resulting in 3X speedup and energy-efficiency over our prior work Eyeriss (ISSCC 2016). Finally, time permitting, we will describe our research in enabling rapid design-space exploration and prototyping of hardware accelerators using our dataflow DSL + cost-model called MAESTRO (MICRO 2019). Bio: Tushar Krishna is an Assistant Professor in the School of Electrical and Computer Engineering at Georgia Tech. He also holds the ON Semiconductor Junior Professorship. He has a Ph.D. in Electrical Engineering and Computer Science from MIT (2014), a M.S.E in Electrical Engineering from Princeton University (2009), and a B.Tech in Electrical Engineering from the Indian Institute of Technology (IIT) Delhi (2007). Before joining Georgia Tech in 2015, Dr. Krishna spent a year as a post-doctoral researcher at Intel, Massachusetts. Dr. Krishna’s research spans computer architecture, interconnection networks, networks-on-chip ( NoC ) and deep learning accelerators - with a focus on optimizing data movement in modern computing systems. Three of his papers have been selected for IEEE Micro’s Top Picks from Computer Architecture, one more received an honorable mention, and two have won best paper awards. He received the National Science Foundation (NSF) CRII award in 2018, and both a Google Faculty Award and a Facebook Faculty Award in 2019. He also received the “Class of 1940 Course Survey Teaching Effectiveness” Award from Georgia Tech in 2018.

1 0

Colloquium Speaker: Tushar Krishna, Thursday 11/7 at 12:30pm
by Emily C. Lawrence 31 Oct '19

31 Oct '19

CS Colloquium Speaker Tushar Krishna from Georgia Tech Thursday, November 7 - 12:30pm E-Quad, B205 Host: Margaret Martonosi [ https://www.cs.princeton.edu/events/25888 | https://www.cs.princeton.edu/events/25888 ] Enabling Continuous Learning through Synaptic Plasticity in Hardware Ever since modern computers were invented, the dream of creating artificial intelligence (AI) has captivated humanity. We are fortunate to live in an era when, thanks to deep learning (DL), computer programs have paralleled, and in many cases even surpassed human level accuracy in tasks like visual perception and speech synthesis. However, we are still far away from realizing general-purpose AI. The problem lies in the fact that the development of supervised learning based DL solutions today is mostly open loop. A typical DL model is created by hand-tuning the deep neural network (DNN) topology by a team of experts over multiple iterations, followed by training over petabytes of labeled data. Once trained, the DNN provides high accuracy for the task at hand; if the task changes, however, the DNN model needs to be re-designed and re-trained before it can be deployed. A general-purpose AI system, in contrast, needs to have the ability to constantly interact with the environment and learn by adding and removing connections within the DNN autonomously, just like our brain does. This is known as synaptic plasticity. In this talk we will present our research efforts towards enabling general-purpose AI leveraging plasticity in both the algorithm and hardware. First, we will present GeneSys (MICRO 2018), a HW-SW prototype of a closed loop learning system for continuously evolving the structure and weights of a DNN for the task at hand using genetic algorithms, providing 100-10000x higher performance and energy-efficiency over state-of-the-art embedded and desktop CPU and GPU systems. Next, we will present a DNN accelerator substrate called MAERI (ASPLOS 2018), built using light-weight, non-blocking, reconfigurable interconnects, that supports efficient mapping of regular and irregular DNNs with arbitrary dataflows , providing ~100% utilization of all compute units, resulting in 3X speedup and energy-efficiency over our prior work Eyeriss (ISSCC 2016). Finally, time permitting, we will describe our research in enabling rapid design-space exploration and prototyping of hardware accelerators using our dataflow DSL + cost-model called MAESTRO (MICRO 2019). Bio: Tushar Krishna is an Assistant Professor in the School of Electrical and Computer Engineering at Georgia Tech. He also holds the ON Semiconductor Junior Professorship. He has a Ph.D. in Electrical Engineering and Computer Science from MIT (2014), a M.S.E in Electrical Engineering from Princeton University (2009), and a B.Tech in Electrical Engineering from the Indian Institute of Technology (IIT) Delhi (2007). Before joining Georgia Tech in 2015, Dr. Krishna spent a year as a post-doctoral researcher at Intel, Massachusetts. Dr. Krishna’s research spans computer architecture, interconnection networks, networks-on-chip ( NoC ) and deep learning accelerators - with a focus on optimizing data movement in modern computing systems. Three of his papers have been selected for IEEE Micro’s Top Picks from Computer Architecture, one more received an honorable mention, and two have won best paper awards. He received the National Science Foundation (NSF) CRII award in 2018, and both a Google Faculty Award and a Facebook Faculty Award in 2019. He also received the “Class of 1940 Course Survey Teaching Effectiveness” Award from Georgia Tech in 2018.

1 0

Jordan Fix will present his FPO "Hardware MultiThreaded Transactions: Enabling Speculative MuliThreaded Pipeline Parallelization For Complex Programs" on Tuesday, 11/12/2019 at 10am in CS 402
by Nicki Mahler 28 Oct '19

28 Oct '19

Jordan Fix will present his FPO "Hardware MultiThreaded Transactions: Enabling Speculative MuliThreaded Pipeline Parallelization For Complex Programs" on Tuesday, 11/12/2019 at 10am in CS 402. The members of his committee are: David August (adviser); Readers: David August and David Callahan (Reservoir Labs); Examiners: Margaret Martonosi, David Wentzlaff (ELE), and Andrew Appel. A copy of his thesis is available upon request. Please email ngotsis(a)cs.princeton if you would like a copy of the thesis. Everyone is invited to attend his talk. The talk abstract follows below: Speculation with transactional memory systems helps programmers and compilers produce profitable thread-level parallel programs. Prior work shows that supporting transactions that can span multiple threads, rather than requiring transactions be contained within a single thread, enables new types of speculative parallelization techniques for both programmers and parallelizing compilers. Unfortunately, software support for multi-threaded transactions (MTXs) comes with significant additional inter-thread communication overhead for speculation validation. This overhead can make otherwise good parallelization unprofitable for programs with sizeable read and write sets. Some programs using these prior software MTXs overcame this problem through significant efforts by expert programmers to minimize these sets and optimize communication, capabilities which compiler technology has been unable to achieve to date. Instead, this dissertation makes speculative parallelization less laborious and more feasible through low-overhead speculation validation, presenting the first complete design, implementation, and evaluation of hardware MTXs. Even with maximal speculation validation of every load and store inside transactions of tens to hundreds of millions of instructions, profitable parallelization of complex programs can be achieved. Across 8 benchmarks, this system achieves a geomean speedup of 104% over sequential execution on a multicore machine with 4 cores, with modest increases in power and energy consumption. This work could be used as a building block to enable more realistic automatic parallelization of complex programs, by providing low-overhead, long-running, resilient transactions that support a diverse set of parallel paradigm options.

1 0

Nayana Prasad Nagendra will present her Pre FPO "A Frontend For All Workloads" on Monday, November 4, 2019 at 11:30am in CS 302
by Nicki Mahler 28 Oct '19

28 Oct '19

Nayana Prasad Nagendra will present her Pre FPO "A Frontend For All Workloads" on Monday, November 4, 2019 at 11:30am in CS 302. The members of her committee are as follows: David August (adviser), Tipp Moseley (Google), David Wentzlaff (ELE), and Sharad Malik (ELE). Everyone is invited to attend her talk. The abstract follows below. Warehouse-Scale Computing applications experience an order of magnitude more front-end stalls than desktop applications because their code footprints are well beyond the capacity of today’s processor structures, including instruction caches and branch target buffers. Current approaches to reducing front-end stalls rely, at least in part, on some subset of processor structures having sufficient capacity. Since working set sizes for important applications will continue to grow faster than processor structures (27% Year-over-Year growth in web-search instruction working set size for example vs. 7% for L1 instruction cache), these approaches are not viable. Instead, we propose Predicted Path Prefetching with Confidence Hedging, a front-end architecture that does not rely on the capacity to prevent front-end stalls by also prefetching the path not predicted when confidence in the path prediction is low. Initial results show many fewer instruction cache misses per kilo instructions with similar prefetch costs compared to existing techniques.

1 0

Colloquium Speaker Sham Kakade TODAY at 12:30pm
by Emily C. Lawrence 24 Oct '19

24 Oct '19

CS Colloquium Speaker Sham Kakade from University of Washington Thursday, October 24 - 12:30pm Friend Center Convocation Room Host: Sanjeev Arora [ https://www.cs.princeton.edu/events/25891 | https://www.cs.princeton.edu/events/25891 ] Representation, Modeling, and Optimization in Reinforcement Learning Reinforcement learning is now the dominant paradigm for how an agent learns to interact with the world. The approach has led to successes ranging across numerous domains, including game playing and robotics, and it holds much promise in new domains, from self-driving cars to interactive medical applications. Some of the central challenges are: -Representational learning: does having a good representation of the environment permit efficient reinforcement learning? -Modeling: should we explicitly build a model of our environment or, alternatively, should we directly learn how to act? -Optimization: in practice, deployed algorithms often use local search heuristics. Can we provably understand when these approaches are effective and provide faster and more robust alternatives? This talk will survey a number of results on these basic questions. Throughout, we will highlight the interplay of theory, algorithm design, and practice. Bio: Sham Kakade is a Washington Research Foundation Data Science Chair, with a joint appointment in the Department of Computer Science and the Department of Statistics at the University of Washington, and is a co-director for the Algorithmic Foundations of Data Science Institute. He works on the mathematical foundations of machine learning and AI. Sham's thesis helped in laying the foundations of the PAC-MDP framework for reinforcement learning. With his collaborators, his additional contributions include: one of the first provably efficient policy search methods, Conservative Policy Iteration, for reinforcement learning; developing the mathematical foundations for the widely used linear bandit models and the Gaussian process bandit models; the tensor and spectral methodologies for provable estimation of latent variable models (applicable to mixture of Gaussians, HMMs, and LDA); the first sharp analysis of the perturbed gradient descent algorithm, along with the design and analysis of numerous other convex and non-convex algorithms. He is the recipient of the IBM Goldberg best paper award (in 2007) for contributions to fast nearest neighbor search and the best paper, INFORMS Revenue Management and Pricing Section Prize (2014). He has been program chair for COLT 2011. Sham was an undergraduate at Caltech, where he studied physics and worked under the guidance of John Preskill in quantum computing. He then completed his Ph.D. in computational neuroscience at the Gatsby Unit at University College London, under the supervision of Peter Dayan. He was a postdoc at the Dept. of Computer Science, University of Pennsylvania , where he broadened his studies to include computational game theory and economics from the guidance of Michael Kearns. Sham has been a Principal Research Scientist at Microsoft Research, New England, an associate professor at the Department of Statistics, Wharton, UPenn , and an assistant professor at the Toyota Technological Institute at Chicago.

1 0