April 2021 - talks - lists.cs.princeton.edu

Kritkorn Karntikoon will present his General Exam "A Sharp Bound on the s-Energy of a Symmetric Averaging System" on May 14, 2021 at 11AM via Zoom.
by jfarquer＠cs.princeton.edu 12 Apr '21

12 Apr '21

Kritkorn Karntikoon will present his General Exam on Friday, May 14, 2021 at 11AM via Zoom. Zoom Link: https://princeton.zoom.us/my/kritkorn The members of his committee are as follows: Bernard Chazelle (adviser), Robert Tarjan, Matt Weinberg Everyone is invited to attend his talk, and those faculty wishing to remain for the oral exam following are welcome to do so. His abstract and reading list follow below. Title: "A Sharp Bound on the s-Energy of a Symmetric Averaging System Abstract: https://docs.google.com/document/d/1BLzvkAW-MFiIg0JIih-FsyS0FyZEhu7tlOTNUyQX euY/edit?usp=sharing Reading List: https://docs.google.com/document/d/1HDHAH9_HK0Pu0h8R4djwSy4T0ADQxOBVxX2qxJUy lBY/edit?usp=sharing

1 0

Wei Hu will present his Pre FPO 'Understanding Deep Learning via Analyzing Dynamics of Gradient Descent' on Tuesday, April 13, 2021 at 4pm via Zoom.
by jfarquer＠cs.princeton.edu 10 Apr '21

10 Apr '21

Wei Hu will present his Pre FPO on Tuesday, April 13, 2021 at 4pm via Zoom. Zoom link: https://princeton.zoom.us/j/2180408086 Committee: Examiners: Sanjeev Arora (advisor), Mark Braverman, Chi Jin (ECE) Readers: Elad Hazan, Jason D. Lee (ECE) All are welcome to attend. Title: Understanding Deep Learning via Analyzing Dynamics of Gradient Descent Abstract: The phenomenal successes of deep learning build upon the mysterious abilities of gradient-based optimization algorithms. Not only can these algorithms often successfully optimize complicated non-convex training objectives, but the solutions found can also generalize remarkably well to unseen test data despite significant over-parameterization of the models. Classical approaches in optimization and learning theory that treat empirical risk minimization as a black box are insufficient to explain these mysteries in modern deep learning. In this talk, I will illustrate how we can make progress towards understanding optimization and generalization in deep learning by a more refined approach that opens the black box and analyzes the dynamics taken by the optimizer. In particular, I will present several results focusing on analyzing the dynamics of the gradient descent algorithm, including: (i) solving low-rank matrix completion via deep linear neural networks, (ii) positive and negative convergence results which reveal the optimal depth-width tradeoff for efficiently training deep linear neural networks, and (iii) the connection between wide neural networks and neural tangent kernels, and its theoretical and practical implications.

1 0

Theodore Sumers will present his General Exam "Communication as Optimal Control" on Monday, April 12, 2021 at 1PM via Zoom.
by jfarquer＠cs.princeton.edu 06 Apr '21

06 Apr '21

Theodore Sumers will present his General Exam on Monday, April 12, 2021 at 1PM via Zoom. The members of his committee are: Tom Griffiths (advisor), Karthik Narasimhan, Adele Goldberg Zoom link: https://princeton.zoom.us/s/5514275793 Everyone is invited to attend the talk, and those faculty wishing to remain for the oral exam are welcome to do so. His abstract and reading list follow below. Title: Communication as Optimal Control Abstract: Language is a powerful tool for communication. Humans routinely transmit vast amounts of information with a handful of words-- often by leaving other words unsaid. Yet the unbounded expressivity of language makes it difficult to model. As a result, psychological studies of communication often focus on coordination games with constrained vocabularies, while natural language interfaces assume literal, imperative statements. These paradigms provide experimental and practical control, but limit fully-fledged accounts of naturalistic language use and information transfer from humans to artificial agents. This exam uses reinforcement learning (RL) to explore human communication in more complex settings. I will first introduce RL-based language games, in which a speaker communicates abstract information (here, the reward function) to guide listener actions. I will show that naturalistic linguistic communication can be modeled with aspect-based sentiment analysis to recover the speaker's reward function. This is a novel contribution which facilitates intuitive human-agent interactions. I will then propose a general view of communication as optimal control over latent listener belief states. This perspective unifies belief- and action-based models of human communication, and can be viewed as a speaker inducing "optimal rewards" in the listener. Finally, I will close with a set of proposed experiments to elucidate the components of this model. Reading List: Books: 1. Richard Sutton and Andrew Barto. Reinforcement Learning: An Introduction. 2. Michael Tomasello. The Origins of Human Communication. Papers: * Communication and Joint Action * <https://www.ucl.ac.uk/ls/studypacks/Grice-Logic.pdf> Logic and Conversation (Grice, 1975) * <https://www.eva.mpg.de/documents/Cambridge/Tomasello_Understanding_BehBrain Sci_2005_1555292.pdf> Understanding and sharing intentions: The origins of cultural cognition (Tomasello et al, 2005) * <http://langcog.stanford.edu/papers_new/frank-2012-science.pdf> Predicting Pragmatic Reasoning in Language Games (Frank & Goodman, 2012) * <http://langcog.stanford.edu/papers_new/goodman-2016-tics.pdf> Pragmatic Language Interpretation as Probabilistic Inference (Goodman & Frank, 2016) * <https://www.aclweb.org/anthology/2020.emnlp-main.703.pdf> Experience Grounds Language (Bisk et al, 2020) * Human Teaching * <https://web.stanford.edu/~ngoodman/papers/shaftogg14.pdf> A rational account of pedagogical reasoning: Teaching by, and learning from, examples (Shafto et al, 2014) * <https://cocosci.princeton.edu/tom/papers/FasterTeachPOMDPPlanning.pdf> Faster Teaching via POMDP Planning (Rafferty et al, 2015) * RL and Rewards * <https://all.cs.umass.edu/pubs/2009/singh_l_b_09.pdf> Where Do Rewards Come From? (Singh et al, 2009) * <https://compdevlab.yale.edu/docs/2019/ToM_as_IRL_2019.pdf> Theory of Mind as Inverse Reinforcement Learning (Jara-Ettinger, 2019) * <https://arxiv.org/abs/1707.06354> Pragmatic-Pedagogic Value Alignment (Fisac et al, 2017) * <https://arxiv.org/abs/1706.03741> Deep reinforcement learning from human preferences (Christiano et al, 2017) * <https://arxiv.org/abs/2002.04833> Reward-rational Implicit Choice (Jeon et al 2020) * RL and Language * <https://arxiv.org/abs/1906.03926> A Survey of Reinforcement Learning Informed by Natural Language (Luketina et al, 2019) * <https://www.researchgate.net/publication/338957467_Robots_That_Use_Language > Robots that Use Language (Tellex et al, 2020) * <https://arxiv.org/abs/1708.00133> Grounding Language for Transfer in Deep Reinforcement Learning (Narasimhan et al, 2018)

1 0

Virtual Joint ORFE Department Colloquium/Wilks Seminar: 4:30pm ET: Peter Bartlett, University of California Berkeley
by Emily C. Lawrence 06 Apr '21

06 Apr '21

== Today’s Joint ORFE Department Colloquium/Wilks Seminar Announcement== DATE: Tuesday, April 6th, 2021 TIME: 4:30pm ET LOCATION: Virtual LINK: [ https://princeton.zoom.us/j/98180416928?pwd=cS8vaWJ0R2dMalVKR0x5SnB3TENSQT09 | https://princeton.zoom.us/j/98180416928?pwd=cS8vaWJ0R2dMalVKR0x5SnB3TENSQT09 ] SPEAKER: Peter Bartlett, University of California, Berkeley TITLE: Benign overfitting ABSTRACT: Deep learning methodology has revealed some major surprises from the perspective of statistical complexity: even without any explicit effort to control model complexity, these methods find prediction rules that give a near-perfect fit to noisy training data and yet exhibit excellent prediction performance in practice. We investigate this phenomenon of ‘benign overfitting’ in the setting of linear prediction, and give a characterization of linear regression problems for which the minimum norm interpolating prediction rule has near-optimal prediction accuracy. The characterization shows that overparameterization is essential: the number of directions in parameter space that are unimportant for prediction must significantly exceed the sample size. We discuss implications for deep networks, for robustness to adversarial examples, and for the rich variety of possible behaviors of excess risk as a function of dimension, and we describe extensions to ridge regression and barriers to analyzing benign overfitting based on model-dependent generalization bounds. Joint work with Phil Long, Gábor Lugosi, and Alex Tsigler. BIO: Peter Bartlett is a professor in the Division of Computer Science and the Department of Statistics. He is the co-author of the book Learning in Neural Networks: Theoretical Foundations. He has served as associate editor of the journals Machine Learning, Mathematics of Control Signals and Systems, the Journal of Machine Learning Research, the Journal of Artificial Intelligence Research, and the IEEE Transactions on Information Theory. He was awarded the Malcolm McIntosh Prize for Physical Scientist of the Year in Australia for his work in statistical learning theory. He was a Miller Institute Visiting Research Professor in Statistics and Computer Science at U.C. Berkeley in Fall 2001, and a fellow, senior fellow and professor in the Research School of Information Sciences and Engineering at the Australian National University's Institute for Advanced Studies (1993-2003). He is also an honorary professor in the Department of Computer Science and Electrical Engineering at the University of Queensland.

1 0

Ameet S. Deshpande will present his General Exam "Guiding Attention for Self-Supervised Learning with Transformers" on Thursday, April 29, 2021 at 11AM via Zoom.
by jfarquer＠cs.princeton.edu 06 Apr '21

06 Apr '21

Ameet S. Deshpande will present his General Exam on Thursday, April 29, 2021 at 11AM via Zoom. The members of his committee are as follows: Karthik Narasimhan (advisor), Danqi Chen, Sanjeev Arora Zoom link: <http://princeton.zoom.us/my/ameetsd> http://princeton.zoom.us/my/ameetsd Everyone is invited to attend the talk, and those faculty wishing to remain for the oral exam following are welcome to do so. His abstract and reading list follow below. Title: Guiding Attention for Self-Supervised Learning with Transformers Abstract: Recent advances in self-supervised pre-training have resulted in models with impressive downstream performance on several natural language processing (NLP) tasks. However, this has led to the development of enormous models, which often require days of training on non-commodity hardware (e.g., TPUs, distributed GPUs). Furthermore, studies have shown that it is quite challenging to successfully train these large Transformer models, requiring complicated learning schemes and extensive hyperparameter tuning. In this talk, we present a simple and effective technique for efficient self-supervised learning with bi-directional Transformers. Our approach is motivated by recent studies demonstrating that trained models' self-attention patterns contain a majority of non-linguistic regularities. We propose a computationally efficient auxiliary loss function to guide attention heads to conform to such patterns. Our method is agnostic to the actual pre-training objective. It results in faster convergence of models and better performance on downstream tasks compared to the baselines, achieving state-of-the-art results in low-resource settings. Reading List: Textbooks: 1. <https://web.stanford.edu/~jurafsky/slp3/ed3book_dec302020.pdf> Speech and Language Processing, by Jurafsky and Martin 2. <https://github.com/jacobeisenstein/gt-nlp-class/blob/master/notes/eisenstei n-nlp-notes.pdf> Natural Language Processing, by Jacob Eisenstein Papers: Transformers and pre-training in NLP 1. Vaswani, Ashish, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. "Attention is all you need." 2. Devlin, Jacob, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. "Bert: Pre-training of deep bidirectional transformers for language understanding." 3. Liu, Yinhan, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. "Roberta: A robustly optimized bert pretraining approach." 4. Clark, Kevin, Minh-Thang Luong, Quoc V. Le, and Christopher D. Manning. "Electra: Pre-training text encoders as discriminators rather than generators." Modifying self-attention modules in Transformers 1. Kitaev, Nikita, Łukasz Kaiser, and Anselm Levskaya. "Reformer: The efficient transformer." 2. Child, Rewon, Scott Gray, Alec Radford, and Ilya Sutskever. "Generating long sequences with sparse transformers." 3. Beltagy, Iz, Matthew E. Peters, and Arman Cohan. "Longformer: The long-document transformer." 4. Tay, Yi, Dara Bahri, Donald Metzler, Da-Cheng Juan, Zhe Zhao, and Che Zheng. "Synthesizer: Rethinking self-attention in transformer models." 5. Wang, Sinong, Belinda Li, Madian Khabsa, Han Fang, and Hao Ma. "Linformer: Self-attention with linear complexity." 6. Zaheer, Manzil, Guru Guruganesh, Avinava Dubey, Joshua Ainslie, Chris Alberti, Santiago Ontanon, Philip Pham et al. "Big bird: Transformers for longer sequences." Fixing self-attention patterns (Machine Translation) 1. You, Weiqiu, Simeng Sun, and Mohit Iyyer. "Hard-coded gaussian attention for neural machine translation." 2. Raganato, Alessandro, Yves Scherrer, and Jörg Tiedemann. "Fixed encoder self-attention patterns in transformer-based machine translation." Analyzing self-attention 1. Clark, Kevin, Urvashi Khandelwal, Omer Levy, and Christopher D. Manning. "What does bert look at? an analysis of bert's attention." 2. Kovaleva, Olga, Alexey Romanov, Anna Rogers, and Anna Rumshisky. "Revealing the dark secrets of BERT." 3. Voita, Elena, David Talbot, Fedor Moiseev, Rico Sennrich, and Ivan Titov. "Analyzing multi-head self-attention: Specialized heads do the heavy lifting, the rest can be pruned." 4. Michel, Paul, Omer Levy, and Graham Neubig. "Are sixteen heads really better than one?."

1 0

Qipeng Liu will present his Pre FPO "Cryptography in the age of quantum computers" on Monday, April 12, 2021 at 12pm via Zoom.
by jfarquer＠cs.princeton.edu 06 Apr '21

06 Apr '21

Qipeng Liu will present his Pre FPO "Cryptography in the age of quantum computers" on Monday, April 12, 2021 at 12pm via Zoom. Zoom link: <https://princeton.zoom.us/j/6541841088> https://princeton.zoom.us/j/6541841088 Committee: Mark Zhandry (advisor, examiner) Mark Braverman (examiner) Matt Weinberg (examiner) Ran Raz (reader) Gillat Kol (reader) Talk abstract follows below. All are welcome to attend. Abstract: It is widely believed that full-scale quantum computers will eventually be viable. Quantum computers pose threats to many existing cryptosystems (most prominently, Shor's algorithm), while raising the possibility of using quantum mechanical phenomena to achieve never-before-possible capabilities. My talk consists of the following topics: understanding security of existing cryptosystems in a quantum world and developing new schemes with the power of quantum computers. First, I will present my work on quantum query complexity: I will show tight bounds for multi-collision finding problem, and tight time-space tradeoffs for function inversion problem. The latter shows that even with a piece of preprocessed quantum advice, Grover's search can not be sped up. This technique can be extended to prove post-quantum non-uniform security of many existing cryptographic schemes. Second, I will present my work on post-quantum zero-knowledge proof: I will start by showing post-quantum constant-round zero-knowledge protocols with black-box simulators for NP do not exist, unless NP is in BQP; then I will show how to achieve non-interactive zero-knowledge, by proving the famous Fiat-Shamir transformation is secure in the quantum random oracle model. Third, I will present my work on quantum copy-protection, which aims to use the unclonability of quantum states to achieve programs that cannot be copied. Towards this goal, I will show that any unlearnable function can be copy-protected relative to classical oracles. For specific cryptographic functionalities (such as signatures and RPFs), I will show that they can be copy-protected in the plain model.

1 0

Johan Ospina will present his MSE Talk and Thesis "Crowdflow - creating a crowdsourced optical flow dataset" on Tuesday, April 20, 2021 at 3:30PM via Zoom.
by jfarquer＠cs.princeton.edu 01 Apr '21

01 Apr '21

Johan Ospina will present his MSE Talk and Thesis "Crowdflow - creating a crowdsourced optical flow dataset" on Tuesday, April 20, 2021 at 3:30PM via Zoom. Zoom Link: <https://princeton.zoom.us/j/6951559457> https://princeton.zoom.us/j/6951559457 Committee Members: Jia Deng (adviser) and Olga Russakovsky (reader) All are welcome to attend. Title: Crowdflow - creating a crowdsourced optical flow dataset. Abstract: Datasets for training Deep Learning solutions for Optical Flow have been for the most part created in synthetic rendering environments or in highly structured real life environments without much variance of environment. While CGI methods are able to produce accurate training data for learning algorithms, they are limited to either the quality of the rendering algorithms used or the variation of the library of 3D models they have access to. On the other end of the spectrum, the real life datasets that exist are limited in the types of environments they can image. Crowdflow aims to bring a crowdsourced dataset of sparse Optical Flow annotations in a wide variety of real life environments in order to measure how well current state of the art methods perform on a variety of real life scenes.

1 0

Ryan Amos will present his Pre FPO "Consumer protection with longitudinal data collection and analysis" on Thursday, April 1, 2021 at 1pm via Zoom.
by Nicki Mahler 01 Apr '21

01 Apr '21

Ryan Amos will present his Pre FPO "Consumer protection with longitudinal data collection and analysis" on Thursday, April 1, 2021 at 1pm via Zoom. Zoom link: [ https://princeton.zoom.us/j/98956078851?pwd=UWxReS9DcWc3aFlTWEFydTJTVXlnQT09 | https://princeton.zoom.us/j/98956078851?pwd=UWxReS9DcWc3aFlTWEFydTJTVXlnQT09 ] Committee: Examiners: Prateek Mittal (adviser), Ed Felten (adviser), Arvind Narayanan Readers: Jonathan Mayer, Joe Calandrino ( Federal Trade Commission) All are welcome to attend. Title: Consumer protection with longitudinal data collection and analysis Abstract: We'll discuss how we can use longitudinal data to understand issues surrounding consumer protection, particularly issues driven by misaligned incentives in advertising. In the first half, I'll present our work on collecting and analyzing our dataset of one million privacy policies from a twenty year timespan. Our work adds to the growing body demonstrating the inadequacy of the notice-and-choice model of privacy policy regulation. We find that privacy policies have become more difficult to read, and, by comparing our findings to prior web measurements, that they show a troubling lack of transparency. Our work helps contextualize the impact of the GDPR by showing how it stands out against a decade of privacy policy changes. In the second half, I'll present our ongoing work on collecting and analyzing a dataset of Yelp's review recommendation system, which we take as a proxy for fake review filtering. To understand the evolving game between fake reviewers and platforms, we frame our work differently from prior work -- rather than focusing on the reviews themselves, we focus on the platform's reactions to reviews. We find that a substantial proportion of filtered-out reviews are later reclassified, suggesting that Yelp takes an aggressive approach to removing reviews. I'll additionally discuss our ongoing work in this field and questions we want to answer.

1 1