- talks - lists.cs.princeton.edu

Christina Shatford will present her MSE talk "Simultaneous Pattern and Detector Optimization for Visual Tags" on Thursday, April 21, 2022 at 3:00pm via Zoom
by Nicki Mahler 18 Apr '22

18 Apr '22

Christina Shatford will present her MSE talk "Simultaneous Pattern and Detector Optimization for Visual Tags" on Thursday, April 21, 2022 at 3:00pm via Zoom. Zoom link: [ https://princeton.zoom.us/j/91816794623 | https://princeton.zoom.us/j/91816794623 ] Committee members: Rusinkiewicz (Adviser) and Ryan Adams (Reader) Abstract: Accurate and robust object localization has been a long studied problem in computer vision. In this talk, we will propose a new approach for localizing visual tags in images. Instead of optimizing a detector to locate a tag with a static pattern, this approach involves the simultaneous optimization of both the detector and the pattern of the visual tag. We will show how this joint learning leads to increased localization accuracy when compared to only training the detector and using a selection of standard non-optimized tags, such as QR codes. All are welcome to attend.

1 0

Huihan Li will present her MSE Talk "Re-evaluating Conversational Question Answering" on Thursday, April 21st, 2022 at 3:30 PM via Zoom.
by Louis W. Riehl 14 Apr '22

14 Apr '22

Huihan Li will present her MSE Talk "Re-evaluating Conversational Question Answering" on Thursday, April 21st, 2022 at 3:30 PM via Zoom. Zoom Link: [ https://princeton.zoom.us/j/92763452837 | https://princeton.zoom.us/j/92763452837 ] Committee Members: Danqi Chen (advisor), Karthik Narasimhan (reader) All are welcome to attend. Re-evaluating Conversational Question Answering Conversational question answering aims to provide natural-language answers to users in information-seeking conversations. Existing conversational QA benchmarks compare models with pre-collected human-human conversations, using ground-truth answers provided in conversational history. It remains unclear whether we can rely on this static evaluation for model development and whether current systems can well generalize to real-world human-machine conversations. In this work, we conduct the first large-scale human evaluation of state-of-the-art conversational QA systems, where human evaluators converse with models and judge the correctness of their answers. We find that the distribution of human-machine conversations differs drastically from that of human-human conversations, and there is a disagreement between human and gold-history evaluation in terms of model ranking. We further investigate how to improve automatic evaluations, and propose a question rewriting mechanism based on predicted history, which better correlates with human judgments. Finally, we demonstrate that training conversational question answering systems using their own predictions benefits the models under the new automatic evaluation protocol. Louis Riehl Graduate Administrator Computer Science Department, CS213 Princeton University (609) 258-8014

1 0

Juan Duque will present his MSE talk, "Transformer Representations for Efficient Reinforcement Learning" on Thursday, April 21st at 3:00 PM (EDT) via Zoom.
by Louis W. Riehl 14 Apr '22

14 Apr '22

Juan Duque will present his MSE talk, "Transformer Representations for Efficient Reinforcement Learning" on Thursday, April 21st at 3:00 PM (EDT) via Zoom. Zoom Link: [ https://princeton.zoom.us/j/93980023607 | https://princeton.zoom.us/j/93980023607 ] Committee Members: Karthik Narasimhan (adviser) and Elad Hazan (reader) All are welcome to attend. Transformer Representations for Efficient Reinforcement Learning Reinforcement Learning algorithms are often trained from scratch at high computational costs. In this talk I will present a self-supervised learning framework that uses the Transformer architecture to generate useful representations for Reinforcement Learning. Our formulation uses objectives borrowed from Natural Language Processing and Imitation Learning to pre-train a Tranformer model from offline trajectories that is then fine-tuned on Atari games. We demonstrate how this setup allows for quick online convergence in under 100 thousand episodes. Louis Riehl Graduate Administrator Computer Science Department, CS213 Princeton University (609) 258-8014

1 0

Ishita Chaturvedi will present her General Exam "GPU Out-Of-Order Processing" on Wednesday, April 20, 2022 at 1:00 PM in Friend 202 and via zoom.
by Louis W. Riehl 14 Apr '22

14 Apr '22

Ishita Chaturvedi will present her General Exam "GPU Out-Of-Order Processing" on Wednesday, April 20, 2022 at 1:00 PM in Friend 202 and via zoom. Zoom link: [ https://princeton.zoom.us/my/ishita.c | https://princeton.zoom.us/my/ishita.c ] Committee Members: David August (advisor), David Wentzlaff, Tyler Sorensen Abstract: GPUs have become ubiquitous today for high-performance computing. While GPUs leverage massive multithreaded parallelism to obtain speedup, they still suffer from having multiple stall cycles. While some stall reasons can be fixed others must be tolerated. We introduce the Def-Ex GPU infrastructure which improves GPU performance using Out-of-Order instruction execution. We focus on the Deferred Execution Buffer which defers the execution of stalling instructions to issue them later. This gives an opportunity for the next instructions in the pipeline to execute, thus reducing the stall cycles. We see a maximum speedup of 26% and a geomean speedup of 14% on the Rodinia benchmark suite with the Deferred Execution Buffer. Reading List: [ https://docs.google.com/document/d/1LxXAI99yrbRsGa6kc56JCsnmBwQ8nzs3B2E9pTF… | https://docs.google.com/document/d/1LxXAI99yrbRsGa6kc56JCsnmBwQ8nzs3B2E9pTF… ] Everyone is invited to attend the talk, and those faculty wishing to remain for the oral exam following are welcome to do so. Louis Riehl Graduate Administrator Computer Science Department, CS213 Princeton University (609) 258-8014

1 0

Huihan Li will present her MSE talk "Re-evaluating Conversational Question Answering" on April 21, 3:30pm via Zoom
by Nicki Mahler 14 Apr '22

14 Apr '22

Huihan Li will present her MSE talk "Re-evaluating Conversational Question Answering" on April 21, 3:30pm via Zoom. Zoom link: [ https://princeton.zoom.us/j/92763452837 | https://princeton.zoom.us/j/92763452837 ] Huihan's committee is as follows: Danqi Chen (adviser) and Karthik Narasimhan (reader) All are welcome to attend. Abstract: Conversational question answering aims to provide natural-language answers to users ininformation-seeking conversations. Existing conversational QA benchmarks compare models with pre-collected human-human conversations, using ground-truth answers provided in conversational history. It remains unclear whether we can rely on this static evaluation for model development and whether current systems can well generalize to real-world human- machine conversations. In this work, we conduct the first large-scale human evaluation of state-of-the-art conversational QA systems, where human evaluators converse with models and judge the correctness of their answers. We find that the distribution of human-machine conversations differs drastically from that of human-human conversations, and there is a disagreement between human and gold-history evaluation in terms of model ranking. We further investigate how to improve automatic evaluations, and propose a question rewriting mechanism based on predicted history, which better correlates with human judgments. Finally, we demonstrate that training conversational question answering systems using their own predictions benefits the models under the new automatic evaluation protocol.

1 0

Mengzhou Xia will present her General Exam "Structured Pruning Learns Compact and Accurate Models" on Tuesday, April 19, 2022 at 1:00 PM in CS 302 and via Zoom.
by Louis W. Riehl 12 Apr '22

12 Apr '22

Mengzhou Xia will present her General Exam "Structured Pruning Learns Compact and Accurate Models" on Tuesday, April 19, 2022 at 1:00 PM in CS 302 and via Zoom. Zoom link: [ https://princeton.zoom.us/j/6256370935 | https://princeton.zoom.us/j/6256370935 ] Committee Members: Danqi Chen (advisor), Karthik Narasimhan, Kai Li Abstract: The growing size of neural language models has led to increased attention in model compression. The two predominant approaches are pruning, which gradually removes weights from a pre-trained model, and distillation, which trains a smaller compact model to match a larger one. Pruning methods can significantly reduce the model size but hardly achieve large speedups as distillation. However, distillation methods require large amounts of unlabeled data and are expensive to train. In this work, we propose a task-specific structured pruning method CoFi (Coarse- and Fine-grained Pruning), which delivers highly parallelizable subnetworks and matches the distillation methods in both accuracy and latency, without resorting to any unlabeled data. Our key insight is to jointly prune coarse-grained (e.g., layers) and fine-grained (e.g., heads and hidden units) modules, which controls the pruning decision of each parameter with masks of different granularity. We also devise a layerwise distillation strategy to transfer knowledge from unpruned to pruned models during optimization. Our experiments on GLUE and SQuAD datasets show that CoFi yields models with over 10x speedups with a small accuracy drop, showing its effectiveness and efficiency compared to previous pruning and distillation approaches. Reading List: [ https://docs.google.com/document/d/1cJzfOnk7OEn1ssGHZlpheGDRCZ9I9bO4Lcdk3EF… | https://docs.google.com/document/d/1cJzfOnk7OEn1ssGHZlpheGDRCZ9I9bO4Lcdk3EF… ] Everyone is invited to attend the talk, and those faculty wishing to remain for the oral exam following are welcome to do so. Louis Riehl Graduate Administrator Computer Science Department, CS213 Princeton University (609) 258-8014

1 0

Samyak Gupta will present his General Exam on April 15th, 2022 at 1pm via Zoom.
by Nicki Mahler 12 Apr '22

12 Apr '22

Samyak Gupta will present his General Exam on April 15th, 2022 at 1pm via Zoom. Zoom link: [ https://princeton.zoom.us/my/samyakg | https://princeton.zoom.us/my/samyakg ] The members of his committee are as follows: Kai Li (adviser), Danqi Chen, and Jon Cohen (Neuroscience) Link to Abstract and Reading list: [ https://docs.google.com/document/d/1t6BU6TflwycaOe7XECeUk7Tln19Aui9JqVOLhxk… | https://docs.google.com/document/d/1t6BU6TflwycaOe7XECeUk7Tln19Aui9JqVOLhxk… ]

1 1

Vishvak Murahari will present his General Exam "DataMUX: Data Multiplexing for Neural Networks" on Monday, April 18, 2022 at 3:30 pm in CS 302 and via Zoom.
by Louis W. Riehl 11 Apr '22

11 Apr '22

Vishvak Murahari will present his General Exam "DataMUX: Data Multiplexing for Neural Networks" on Monday, April 18, 2022 at 3:30 pm in CS 302 and via Zoom. Zoom Link: [ https://princeton.zoom.us/j/3829472199 | https://princeton.zoom.us/j/3829472199 ] Committee Members: Karthik Narasimhan (adviser), Danqi Chen, Sanjeev Arora Abstract: In this talk, we introduce data multiplexing (DataMUX), a technique that enables deep neural networks to process multiple inputs simultaneously using a single compact representation. DataMUX demonstrates that neural networks are capable of generating accurate predictions over mixtures of inputs, resulting in increased throughput with minimal extra memory requirements. Our approach uses two key components -- 1) a multiplexing layer that performs a fixed linear transformation to each input before combining them to create a mixed representation of the same size as a single input, which is then processed by the base network, and 2) a demultiplexing layer that converts the base network's output back into independent representations before producing predictions for each input. We show the viability of DataMUX for different architectures (Transformers, and to a lesser extent MLPs and CNNs) across six different tasks spanning sentence classification, named entity recognition and image classification. For instance, DataMUX for Transformers can multiplex up to 20x/40x inputs, achieving 11x/18x increase in throughput with minimal absolute performance drops of <2% and 4% respectively on MNLI, a natural language inference task. We also provide a theoretical construction for multiplexing in self-attention networks and analyze the effect of various design elements in DataMUX. Reading List: [ https://docs.google.com/document/d/1Pi-Pd7NwcScb1UfbHaGaE_g_-6-cwqXM_18yQ_u… | https://docs.google.com/document/d/1Pi-Pd7NwcScb1UfbHaGaE_g_-6-cwqXM_18yQ_u… ] Everyone is invited to attend the talk, and those faculty wishing to remain for the oral exam following are welcome to do so.

1 0

Samyak Gupta will present his General Exam "Attacking Federated Natural Language Modeling and Building Tools for Optimizing Cognitive Models" on Friday, April 15, 2022 at 1:00 PM in CS 302 and via zoom.
by Louis W. Riehl 11 Apr '22

11 Apr '22

Samyak Gupta will present his General Exam "Attacking Federated Natural Language Modeling and Building Tools for Optimizing Cognitive Models" on Friday, April 15, 2022 at 1:00 PM in CS 302 and via zoom. Zoom link: [ https://princeton.zoom.us/my/samyakg | https://princeton.zoom.us/my/samyakg ] Committee Members: Kai Li (advisor), Danqi Chen, Jon Cohen Abstract: In this talk, I will present two distinct areas of my work. In the first half of my talk, I describe an application related to privacy in federated learning. Federated learning allows distributed users to collaboratively train a model while keeping each user’s data private. As a result, it is actively being considered for privacy sensitive applications such as virtual mobile keyboards and analysis of electronic health records in hospitals. A growing body of recent work suggests that a clever eavesdropper may effectively reconstruct image data from gradients transmitted during federated learning. However, little progress has been made regarding the recovery of text data. Moreover, existing literature does not establish any metrics which may be used to quantify the severity of text data leakage. In this presentation, I present the first attack which can reconstruct sentences of private text data during training of a neural language model in federated learning. Our proposed method is built on beam search for text generation, by recovering a set of words from gradients and leveraging the latent memorization of text by large language models. We experiment with a GPT-2 model on several large datasets and demonstrate that our proposed attack can successfully recover private text data during training. Reading List: [ https://docs.google.com/document/d/1t6BU6TflwycaOe7XECeUk7Tln19Aui9JqVOLhxk… | https://docs.google.com/document/d/1t6BU6TflwycaOe7XECeUk7Tln19Aui9JqVOLhxk… ] Everyone is invited to attend the talk, and those faculty wishing to remain for the oral exam following are welcome to do so. Louis Riehl Graduate Administrator Computer Science Department, CS213 Princeton University (609) 258-8014

1 0

UPDATE: Tianyu Gao will present his General Exam "SimCSE: Simple Contrastive Learning of Sentence Embeddings" on Wednesday, April 13, 2022 at 3:00 PM in CS 402 and via Zoom.
by Louis Riehl 11 Apr '22

11 Apr '22

Tianyu Gao will present his General Exam "SimCSE: Simple Contrastive Learning of Sentence Embeddings" on Wednesday, April 13, 2022 at 3:00 PM in CS 402 and via Zoom. Zoom link: https://princeton.zoom.us/j/95387462065 Committee Members: Danqi Chen (advisor), Karthik Narasimhan, Sanjeev Arora Abstract: We present SimCSE, a simple contrastive learning framework that greatly advances the state-of-the-art sentence embeddings. We first describe an unsupervised approach, which takes an input sentence and predicts itself in a contrastive objective, with only standard dropout used as noise. This simple method works surprisingly well, performing on par with previous supervised counterparts. We find that dropout acts as minimal data augmentation and removing it leads to a representation collapse. Then, we propose a supervised approach, which incorporates annotated pairs from natural language inference datasets into our contrastive learning framework, by using "entailment" pairs as positives and "contradiction" pairs as hard negatives. We evaluate SimCSE on standard semantic textual similarity (STS) tasks, and our unsupervised and supervised models using BERT base achieve an average of 76.3% and 81.6% Spearman's correlation respectively, a 4.2% and 2.2% improvement compared to previous best results. We also show -- both theoretically and empirically -- that contrastive learning objective regularizes pre-trained embeddings' anisotropic space to be more uniform, and it better aligns positive pairs when supervised signals are available. Reading List: https://docs.google.com/document/d/1fSGNfvOSkPggmMZUv_rAp4MK0fo05EeL9enjoyep 0Og/edit?usp=sharing Everyone is invited to attend the talk, and those faculty wishing to remain for the oral exam following are welcome to do so.

1 0