April 2021 - talks - lists.cs.princeton.edu

Kyle Genova will present his FPO "3D Representations for Learning to Reconstruct and Segment Shapes" on Friday, April 30, 2021 at 1PM via Zoom.
by jfarquer＠cs.princeton.edu 30 Apr '21

30 Apr '21

Kyle Genova will present his FPO "3D Representations for Learning to Reconstruct and Segment Shapes" on Friday, April 30, 2021 at 1PM via Zoom. Zoom link: <https://princeton.zoom.us/j/95639056934> https://princeton.zoom.us/j/95639056934 The members of Kyle's committee are as follows: Tom Funkhouser (adviser), Readers: Olga Russakovsky and Forrester Cole (Google), Examiners: Adam Finkelstein and Szymon Rusinkiewicz. A copy of his thesis is available upon request. Please email <mailto:gradinfo@cs.princeton.edu> gradinfo(a)cs.princeton.edu if you would like a copy of the thesis. Everyone is invited to attend his talk. Abstract follows below: The focus of this dissertation is the novel use of shape representations to empower 3D reasoning for reconstruction and segmentation. It is organized into three sections based on application: domain specific shape reconstruction (Chapter 2), general shape reconstruction (Chapter 3), and semantic segmentation (Chapter 4). In each chapter, we outline the setting and related work, and then introduce one or two approaches with a novel use of shape representation. Our key contribution is to use shape representation to enable new types of supervision and improve generalization when learning 3D priors. Because current reconstruction and segmentation methods share the use of learned 3D encoder and decoder architectures, these contributions apply to both tasks. In Chapters 2-4, we demonstrate experimentally that reconstruction and segmentation algorithms benefit from our choices of shape representation. A primary benefit of our approaches is enabling new types of supervision that require some property of the representation to be effective. Domain specific representation enables supervising 3D face reconstruction with a face recognition network for the first time, resulting in provably more recognizable reconstructions (Chapter 2). Our SIF representation learns shape correspondence from only reconstruction supervision (Chapter 3). Large, diverse image collections are already semantically labeled, making it possible to train 3D semantic segmentation models for datasets without point cloud annotations (Chapter 4). A secondary benefit is improved generalization by deriving better priors from existing supervision. We propose a new shape representation, LDIF, which is trained on existing 3D reconstruction data. LDIF learns robust local priors, improving generalization to unseen classes and shapes (Chapter 3). The addition of image-based supervision in segmentation algorithms improves generalization to cities with no 3D supervision (Chapter 4). We conclude that our choices of representation enable new supervision, better generalization, and learning useful 3D priors from readily available labels (e.g., labeled and unlabeled images, or unlabeled shape collections). We hypothesize that effective future representations will build on this trend by deriving higher level semantic priors from unannotated datasets and other inexpensive sources of supervision (Chapter 5).

2 1

Yi Zhang will present his Pre FPO "Advancing Deep Learning by Integrating Theory and Empirics" on Friday, April 30, 2021 at 2PM via Zoom.
by jfarquer＠cs.princeton.edu 30 Apr '21

30 Apr '21

Yi Zhang will present his Pre FPO "Advancing Deep Learning by Integrating Theory and Empirics" on Friday, April 30, 2021 at 2PM via Zoom. Zoom Link: https://us02web.zoom.us/j/6737994417?pwd=VUE2WWZXL1dTNDg1SWxFYjdyMGEvQT09 Committee: Examiners: Sanjeev Arora (advisor), Karthik Narasimhan, Jason D. Lee (ECE) Readers: Elad Hazan, Chi Jin (ECE) All are welcome to attend. Abstract: In sharp contrast to its remarkable empirical success stands a mathematical understanding of deep learning that is still in its infancy. Various puzzling behaviors of deep neural nets remain unexplained, and many widely deployed deep learning systems lack theoretical guarantees. This talk offers a perspective on the resemblance between deep learning research and natural sciences, especially modern physics at its formative stage where reciprocal interactions between theoretical and experimental studies fueled the growth. As the central object of study, deep neural networks are to deep learning as unknown particles are to physics. We can make significant progress by building theories inspired by empirical observations and verifying hypotheses with designed experiments. In this talk, I will present several of my representative works that follow this research philosophy. Firstly, I will introduce a finite-sample analysis of Generative Adversarial Networks that predicts the existence of degenerate solutions (i.e. mode collapse), which we confirm empirically using a principled test. Then I will present how the empirically identified 'noise stability' of deep neural networks trained on real-life data leads to a substantially stronger generalization measure for deep learning. Finally, I will describe our recent work on designing a simple test for measuring how much deep learning has overfitted to standard datasets.

1 0

CS Distinguished Colloquium Speaker: Jon Kleinberg, TODAY at 4:30pm
by Emily C. Lawrence 29 Apr '21

29 Apr '21

CS Distinguished Colloquium Speaker Jon Kleinberg, Cornell University Host: Sanjeev Arora Date: Thursday, April 29 Time: 4:30pm ET Event page: [ https://www.cs.princeton.edu/events/26075 | https://www.cs.princeton.edu/events/26075 ] Register for Zoom webinar here: [ https://princeton.zoom.us/webinar/register/WN_lh8warWtSVexsnJqz3tsHA | https://princeton.zoom.us/webinar/register/WN_lh8warWtSVexsnJqz3tsHA ] Title: Monoculture and Simplicity in an Ecosystem of Algorithmic Decision-Making Abstract: Algorithms are increasingly used to aid decision-making in high-stakes settings including employment, lending, healthcare, and the legal system. This development has led to an ecosystem of growing complexity in which algorithms and people interact around consequential decisions, often mediated by organizations and firms that may be in competition with one another. We consider two related sets of issues that arise in this setting. First, concerns have been raised about the effects of algorithmic monoculture, in which multiple decision-makers all rely on the same algorithm. In a set of models drawing on minimal assumptions, we show that when competing decision-makers converge on the use of the same algorithm as part of a decision pipeline, the result can potentially be harmful for social welfare even when the algorithm is more accurate than any decision-maker acting on their own. Second, we consider some of the canonical ways in which data is simplified over the course of these decision-making pipelines, showing how this process of simplification can introduce sources of bias in ways that connect to principles from the psychology of stereotype formation. The talk will be based on joint work with Sendhil Mullainathan and Manish Raghavan. Bio: Jon Kleinberg is the Tisch University Professor in the Departments of Computer Science and Information Science at Cornell University. His research focuses on the interaction of algorithms and networks, the roles they play in large-scale social and information systems, and their broader societal implications. He is a member of the National Academy of Sciences and the National Academy of Engineering, and the recipient of MacArthur, Packard, Simons, Sloan, and Vannevar Bush research fellowships, as well awards including the Harvey Prize, the Nevanlinna Prize, and the ACM Prize in Computing.

1 0

Zexuan Zhong will present his General Exam "Extracting Knowledge from Text and Pre-trained Language Models" on Tuesday, May 4, 2021 at 1:30PM via Zoom.
by jfarquer＠cs.princeton.edu 28 Apr '21

28 Apr '21

Zexuan Zhong will present his General Exam "Extracting Knowledge from Text and Pre-trained Language Models" on Tuesday, May 4, 2021 at 1:30PM via Zoom. Zoom Link: https://princeton.zoom.us/j/93687632836 The members of Zexuan's committee are as follows: Danqi Chen (adviser), Karthik Narasimhan, Sanjeev Arora Link to Abstract: https://docs.google.com/document/d/1HhfSSWiceWWs4hhP2ZfX0lD2fpfVC6qj7_zZPE4y 3XY/edit?usp=sharing Link to Reading List: https://docs.google.com/document/d/16Xn7Z9s8nniZASZlgcZbl6QQrYeP_DPucNywVw7l 8QY/edit?usp=sharing Everyone is invited to attend the talk, and those faculty wishing to remain for the oral exam following are welcome to do so.

1 0

Reminder: CS Distinguished Colloquium Speaker: Jon Kleinberg, tomorrow at 4:30pm
by Emily C. Lawrence 28 Apr '21

28 Apr '21

CS Distinguished Colloquium Speaker Jon Kleinberg, Cornell University Host: Sanjeev Arora Date: Thursday, April 29 Time: 4:30pm ET Event page: [ https://www.cs.princeton.edu/events/26075 | https://www.cs.princeton.edu/events/26075 ] Register for Zoom webinar here: [ https://princeton.zoom.us/webinar/register/WN_lh8warWtSVexsnJqz3tsHA | https://princeton.zoom.us/webinar/register/WN_lh8warWtSVexsnJqz3tsHA ] Title: Monoculture and Simplicity in an Ecosystem of Algorithmic Decision-Making Abstract: Algorithms are increasingly used to aid decision-making in high-stakes settings including employment, lending, healthcare, and the legal system. This development has led to an ecosystem of growing complexity in which algorithms and people interact around consequential decisions, often mediated by organizations and firms that may be in competition with one another. We consider two related sets of issues that arise in this setting. First, concerns have been raised about the effects of algorithmic monoculture, in which multiple decision-makers all rely on the same algorithm. In a set of models drawing on minimal assumptions, we show that when competing decision-makers converge on the use of the same algorithm as part of a decision pipeline, the result can potentially be harmful for social welfare even when the algorithm is more accurate than any decision-maker acting on their own. Second, we consider some of the canonical ways in which data is simplified over the course of these decision-making pipelines, showing how this process of simplification can introduce sources of bias in ways that connect to principles from the psychology of stereotype formation. The talk will be based on joint work with Sendhil Mullainathan and Manish Raghavan. Bio: Jon Kleinberg is the Tisch University Professor in the Departments of Computer Science and Information Science at Cornell University. His research focuses on the interaction of algorithms and networks, the roles they play in large-scale social and information systems, and their broader societal implications. He is a member of the National Academy of Sciences and the National Academy of Engineering, and the recipient of MacArthur, Packard, Simons, Sloan, and Vannevar Bush research fellowships, as well awards including the Harvey Prize, the Nevanlinna Prize, and the ACM Prize in Computing.

1 0

Lucas Salvador will present his MSE Talk ":Demystifying the GPU Stall Cycle: Evaluating the Diagnosis of GPU Bottlenecks" on Wednesday 4/28/2021 at 2PM via Zoom.
by GRADINFO 26 Apr '21

26 Apr '21

Lucas Salvador will present his MSE Talk " Demystifying the GPU Stall Cycle: Evaluating the Diagnosis of GPU Bottlenecks" on Wednesday 4/28/2021 at 2PM via Zoom. Zoom Link: [ https://princeton.zoom.us/j/96305827497 | https://princeton.zoom.us/j/96305827497 ] Committee: David August (Adviser), Mohammad Shahrad (Reader) All are welcome to attend. Abstract: Graphics Processing Units (GPUs) have become ubiquitous in all modern intensive computing environments, having transitioned from their early use in Graphics to a plethora of other general compute applications. These tasks are all equally demanding, yet their distinct access patterns generate specific strains in the GPU hardware that evidence different bottlenecks. In order to extract the maximum performance possible from a GPU, application designers and architecture engineers typically have to work back from the symptoms, relying in a series of "rules of thumb" and abstract tools to distill insights that may or may not be related to the actual application bottlenecks. This has led to a series of techniques that have attempted, through the use of detailed simulation, to isolate the cause of bottlenecks in GPU devices. These approaches rely on a hierarchy of stall causes that attributes different importance to specific bottlenecks in the GPU pipeline. In this work, we evaluate the validity of these hierarchical approaches by comparing their insights with the results we could expect from a "perfect" GPU, that is, a simulation (run on GPGPU-Sim) where each specific bottleneck is fully removed, one at a time. By comparing the results across a series of benchmarks, we expose the limitations of these approaches, indicating how the flawed insights may result in misguided optimization. We then further discuss the conceptual pitfalls that seem to be responsible for these discrepancies, and use these to outline a series of desired properties we would expect from a successful diagnosis technique.

1 0

Andrew Or will present his FPO “Abstracting Systems Challenges from Distributed Deep Learning” on Monday, 5/10/2021 at 10am via Zoom.
by jfarquer＠cs.princeton.edu 22 Apr '21

22 Apr '21

Andrew Or will present his FPO on Monday, 5/10/2021 at 10am via Zoom. Title: Abstracting Systems Challenges from Distributed Deep Learning Zoom link: https://princeton.zoom.us/j/96432320689?pwd=ZTE2Z1pGcDV4bFNmUEo5Z1VvaUw4Zz09 The members of his committee are as follows: Michael Freedman (Adviser); Readers: Amit Levy, Ravi Netravali; Examiners: Wyatt Lloyd, Kai Li, Michael Freedman. A copy of his thesis is available upon request. Please email gradinfo(a)cs.princeton.edu <mailto:gradinfo@cs.princeton.edu> if you would like a copy of the thesis. Everyone is invited to attend his talk. Abstract: State-of-the-art distributed deep learning systems, such as TensorFlow and PyTorch, are built on rigid assumptions that tightly couple model training and inference with the underlying hardware. First, they assume resource allocations must be fixed throughout the lifetime of a job, often leading to inefficient resource usage. Second, they require model hyperparameters to be retuned across different hardware configurations in order to achieve the same training result, posing a significant burden on the user. Due to these requirements, users are forced to juggle both systems challenges and application logic instead of being able to focus on just the latter. In this dissertation, we demonstrate that the above assumptions are not fundamental to distributed deep learning. We resolve these limitations by proposing two systems built on top of TensorFlow. The first is an autoscaling engine that, through trial-and-error, automatically determines the most resource-efficient hardware configuration for a given job. We propose pluggable heuristics tailored for deep learning workloads that incrementally guide the system towards such a configuration. Instead of repeatedly stopping the job and restarting it from checkpoints, which can lead to expensive hardware accelerators (e.g. GPUs, TPUs) going idle for minutes every time, our system adjusts the job’s all-reduce membership dynamically in between training steps without interrupting the job. The second system is VirtualFlow, which leverages a novel abstraction between the model and the underlying hardware called virtual node processing. From the perspective of the model, virtual nodes, instead of physical hardware accelerators, perform the computation. When multiple virtual nodes are mapped to a physical device, they are processed sequentially on that device. This representation offers users the flexibility to trade off computation time with resource requirement, allowing them to train their models using the same sets of hyperparameters across different hardware. Using this technique, VirtualFlow preserves application-level semantics while hiding hardware-level details from the user, enabling a variety of important new use cases such as experimentation, hyperparameter exploration, resource elasticity, and heterogeneous training. _____

1 0

CS Distinguished Colloquium Speaker: Jon Kleinberg, Thursday, 4/29 at 4:30pm
by Emily C. Lawrence 22 Apr '21

22 Apr '21

CS Distinguished Colloquium Speaker Jon Kleinberg, Cornell University Host: Sanjeev Arora Date: Thursday, April 29 Time: 4:30pm ET Event page: [ https://www.cs.princeton.edu/events/26075 | https://www.cs.princeton.edu/events/26075 ] Register for Zoom webinar here: [ https://princeton.zoom.us/webinar/register/WN_lh8warWtSVexsnJqz3tsHA | https://princeton.zoom.us/webinar/register/WN_lh8warWtSVexsnJqz3tsHA ] Title: Monoculture and Simplicity in an Ecosystem of Algorithmic Decision-Making Abstract: Algorithms are increasingly used to aid decision-making in high-stakes settings including employment, lending, healthcare, and the legal system. This development has led to an ecosystem of growing complexity in which algorithms and people interact around consequential decisions, often mediated by organizations and firms that may be in competition with one another. We consider two related sets of issues that arise in this setting. First, concerns have been raised about the effects of algorithmic monoculture, in which multiple decision-makers all rely on the same algorithm. In a set of models drawing on minimal assumptions, we show that when competing decision-makers converge on the use of the same algorithm as part of a decision pipeline, the result can potentially be harmful for social welfare even when the algorithm is more accurate than any decision-maker acting on their own. Second, we consider some of the canonical ways in which data is simplified over the course of these decision-making pipelines, showing how this process of simplification can introduce sources of bias in ways that connect to principles from the psychology of stereotype formation. The talk will be based on joint work with Sendhil Mullainathan and Manish Raghavan. Bio: Jon Kleinberg is the Tisch University Professor in the Departments of Computer Science and Information Science at Cornell University. His research focuses on the interaction of algorithms and networks, the roles they play in large-scale social and information systems, and their broader societal implications. He is a member of the National Academy of Sciences and the National Academy of Engineering, and the recipient of MacArthur, Packard, Simons, Sloan, and Vannevar Bush research fellowships, as well awards including the Harvey Prize, the Nevanlinna Prize, and the ACM Prize in Computing.

1 0

Sadhika Malladi will present her General Exam "A Mathematical Exploration of Why Language Models Help Solve Downstream Tasks" on Wednesday, May 5, 2021 at 1:30PM via Zoom.
by jfarquer＠cs.princeton.edu 21 Apr '21

21 Apr '21

Sadhika Malladi will present her General Exam "A Mathematical Exploration of Why Language Models Help Solve Downstream Tasks" on Wednesday, May 5, 2021 at 1:30 PM via Zoom. Zoom link: https://princeton.zoom.us/j/2722265961 Sadhika's committee are as follows: Sanjeev Arora (adviser), Elad Hazan, Karthik Narasimhan Link to Abstract: https://docs.google.com/document/d/1mpmrnTFHIRvHr2YyLvi1S3xEGt36WvQxWqECZsfY 00E/edit?usp=sharing Link to Reading List: https://docs.google.com/document/d/1fPdCU-OHv6DHAAMrOdz9oUQUrzqyTUivLjNFKTNV X9w/edit?usp=sharing Everyone is invited to attend the talk, and those faculty wishing to remain for the oral exam following are welcome to do so.

1 0

Xinyi Chen will present her General Exam "Black-box Control for Linear Dynamical Systems " on Wednesday, May 12, 2021 at 4PM via Zoom
by jfarquer＠cs.princeton.edu 21 Apr '21

21 Apr '21

Xinyi Chen will present her General Exam on Wednesday, May 12, 2021 at 4PM via Zoom. Title: Black-box Control for Linear Dynamical Systems Zoom link: https://princeton.zoom.us/j/8809833006 The members of Xinyi's committee are as follows: Elad Hazan (adviser), Sanjeev Arora, Jason D. Lee Link to Abstract: https://docs.google.com/document/d/1WSHycShlcY7viJNM2wkYYc4reCYH85zOAGOwOS4f 2sk/edit?usp=sharing Link to Reading List: https://docs.google.com/document/d/1VhTEHXUX0_gNVM1vwQSXahnTNBznND1f9JZbJ2D0 138/edit?usp=sharing Everyone is invited to attend the talk, and those faculty wishing to remain for the oral exam following are welcome to do so.

1 0