- talks - lists.cs.princeton.edu

Sayash Kapoor will present his General Exam "Leakage and the reproducibility crisis in ML-based science" on Friday, May 6, 2022 at 9:00 AM in CS 402 and via zoom.
by Louis W. Riehl 29 Apr '22

29 Apr '22

Sayash Kapoor will present his General Exam "Leakage and the reproducibility crisis in ML-based science" on Friday, May 6, 2022 at 9:00 AM in CS 402 and via zoom. Zoom link: [ https://princeton.zoom.us/j/99164890280 | https://princeton.zoom.us/j/99164890280 ] Committee Members: Arvind Narayanan (advisor), Brandon Stewart, Olga Russakovsky Abstract: The use of Machine Learning (ML) methods for prediction and forecasting has become widespread across the quantitative sciences. However, there are many known methodological pitfalls, including data leakage, in ML-based science. In this work, we systematically investigate reproducibility issues in ML-based science. We show that data leakage is indeed a widespread problem and has led to severe reproducibility failures. Specifically, through a survey of literature in research communities that adopted ML methods, we find 15 fields where errors have been found, collectively affecting 304 papers and in some cases leading to wildly overoptimistic conclusions. Based on our survey, we present a fine-grained taxonomy of 8 types of leakage that range from textbook errors to open research problems. We argue for fundamental methodological changes to ML-based science so that cases of leakage can be caught before publication. To that end, we propose model cards for reporting scientific claims based on ML models and find that this would address all types of leakage identified in the 304 papers with errors in our survey. To investigate the impact of reproducibility errors and the efficacy of our model cards, we undertake a reproducibility study in a field where complex ML models are believed to vastly outperform older statistical models such as Logistic Regression (LR): Civil War prediction. We find that all papers claiming the superior performance of complex ML models compared to LR models fail to reproduce due to data leakage, and complex ML models don’t perform substantively better than decades-old LR models. While none of these errors could have been caught by reading the papers, our model cards enable the detection of leakage in each case. Reading List: [ https://docs.google.com/document/d/10Zt4UQTATXKb-aEOmNFi99xu4MbQiHcAXWc1hS3… | https://docs.google.com/document/d/10Zt4UQTATXKb-aEOmNFi99xu4MbQiHcAXWc1hS3… ] Everyone is invited to attend the talk, and those faculty wishing to remain for the oral exam following are welcome to do so. Louis Riehl Graduate Administrator Computer Science Department, CS213 Princeton University (609) 258-8014

1 0

Jiaqi Su will present her FPO "Studio-quality Speech Enhancement" on Thursday, May 5, 2022 at 3:30 PM in Friend 125 and Zoom.
by Louis W. Riehl 28 Apr '22

28 Apr '22

Jiaqi Su will present her FPO "Studio-quality Speech Enhancement" on Thursday, May 5, 2022 at 3:30 PM in Friend 125 and Zoom. Location: Zoom link: https://princeton.zoom.us/j/6073737365 & Friend 125 The members of Jiaqi’s committee are as follows: Examiners: Adam Finkelstein (Adviser), Olga Russakovsky, Karthik Narasimhan Readers: Szymon Rusinkiewicz, Zeyu Jin (Adobe Research) A copy of her thesis is available upon request. Please email gradinfo(a)cs.princeton.edu if you would like a copy of the thesis. Everyone is invited to attend her talk. Abstract follows below: Modern speech content such as podcasts, video narrations, and audiobooks typically requires high-quality audio to support a strong sense of presence and a pleasant listening experience. However, real-world recordings captured with consumer-grade equipment often suffer from quality degradations including noise, reverberation, equalization distortion, and loss of bandwidth. Conventional speech enhancement methods typically focus on removing a subset of quality degradations with the goal of improving signal clarity and intelligibility, but they fall short of studio quality. This dissertation addresses speech enhancement with a focus on improving the perceptual quality and aesthetics of recorded speech. It describes how to improve single-channel real-world consumer-grade recordings to sound like professional studio recordings – studio-quality speech enhancement. In pursuit of this problem, we identify three challenges: objective functions misaligned with human perception, the shortcomings of commonly used audio representations (i.e., spectrogram and waveform), and the lack of available high-quality speech data for training. This dissertation presents a waveform-to-waveform deep neural network solution that consists of two steps: (1) enhancement by removing all quality degradations at limited bandwidth (i.e., 16kHz sample rate), and (2) bandwidth extension from 16kHz to 48kHz to produce a high-fidelity signal. The first enhancement stage relies on a perceptually-motivated GAN framework that combines both waveform and spectrogram representations, and learns from simulated data covering a broad range of realistic recording scenarios. Next, the bandwidth extension stage shares a similar design as the enhancement method, but focuses on filling in missing high frequency details at 48kHz. Finally, we extend the studio-quality speech enhancement problem to a more general problem called acoustic matching to convert recordings to an arbitrary acoustic environment. Louis Riehl Graduate Administrator Computer Science Department, CS213 Princeton University (609) 258-8014

1 0

Jordan Holland will present his FPO "A Generic Framework for Network Traffic Analysis" on May 3, 2022 at 10am via Zoom
by Nicki Mahler 28 Apr '22

28 Apr '22

Jordan Holland will present his FPO "A Generic Framework for Network Traffic Analysis" on May 3, 2022 at 10am via Zoom. Zoom link: [ https://princeton.zoom.us/j/92501308623 | https://princeton.zoom.us/j/92501308623 ] Jordan's committee is as follows: Advisers: Nick Feamster and Prateek Mittal; Readers: Jonathan Mayer and Vitaly Shmatikov (Cornell Tech); Examiners: Nick Feamster, Ravi Netravali, and Prateek Mittal All are welcome to attend. Abstract: Researchers and practitioners rely on network traffic analysis techniques for a variety of critical network security and network management tasks. Ever-increasing traffic volumes and encryption rates have rendered traditional, signature-based solutions less effective. As such, newly developed methods almost universally leverage machine techniques. The development of new machine-learning based traffic analysis techniques shares a common methodological pipeline: curate a network traffic dataset, create a system to separate and associate labels with the traffic (e.g. flows, applications), engineer features for the task, and finally train models using the engineered features. Although this methodological pipeline shared across tasks, each instantiated pipeline is custom-built for the task at hand, requiring new traffic processing systems, features, and models. This dissertation questions the assumption that each stage in the shared methodological pipeline should be custom-built to each task, exploring if several stages of the common pipeline can be better accomplished using generic techniques. First, we examine the process of feature engineering and model training--two of the most manual and painstaking steps for any traffic analysis task. We develop nPrint, a unified packet representation that is amenable to representation learning and model training for a variety of tasks. We then integrate nPrint with automated machine learning to produce nPrintML, a generic feature engineering and model training solution. Next, we study the data collection and data processing steps of the common traffic analysis pipeline. Unlike other disciplines, such as image recognition, no standard dataset format or ''canonical'' task exists, forcing researchers to develop custom dataset formats and processing systems for each task. We survey existing literature to show that this approach has led to a reproducibility crisis, finding that the lack of a standardized dataset format and the extensive usage of ambiguous terminology are primary causes. We use these findings to develop pcapML, a system that enables reproducible network traffic analysis by providing a standardized dataset format that removes ambiguity in the definitions of traffic analysis tasks. The contributions chart new directions in network traffic analysis, demonstrating that generic methods can outperform many custom-built approaches and significantly enhance the ability to develop, reproduce, and compare new methods.

1 0

Rohan Prabhakar will present his General Exam "Using an On-Hardware Predictor For Function Multi-Versioning" on Thursday, May 5, 2022 at 9:00 AM via Zoom.
by Louis W. Riehl 28 Apr '22

28 Apr '22

Rohan Prabhakar will present his General Exam "Using an On-Hardware Predictor For Function Multi-Versioning" on Thursday, May 5, 2022 at 9:00 AM via Zoom. Zoom link: [ https://princeton.zoom.us/my/rohan.bp | https://princeton.zoom.us/my/rohan.bp ] Committee Members: David Wentzlaff (advisor), David August, Ruby Lee (ECE) Abstract: This work explores a system architecture that integrates traditional CPU components with a general-purpose, on-hardware, neural network predictor that has broad access to a wide variety of system features and state. We study the utility of such a system in runtime function multi-versioning, a program optimization technique that involves storing several versions of the same function in the program binary, each compiled with different compiler optimization flags. The key premise behind such multi-versioning is that the best choice of optimization flags is not constant but a function of microarchitectural state, input, and application phase. We approximate the availability of such a neural network predictor by building a software-only system that uses the integrated GPU of a commercially available processor to perform inference on a Long Short Term Memory (LSTM) that is continually supplied with hardware performance counter data. Our system combines this neural network, a static Source-to-Source transformation built using the Clang frontend, and a shared memory model to allow efficient runtime function multi-versioning of C and C++ programs. The static Source-to-Source transformation approach we employ permits using both different optimization flags and different compilers for the same function, all within one program. To evaluate the system and its performance, we use programs that are part of the SPEC 2017 benchmark suite. Reading List: [ http://docs.google.com/document/d/1EBIfSkfxU69gfimtwL_XnRXb6mYvszzflMTi3bMh… | http://docs.google.com/document/d/1EBIfSkfxU69gfimtwL_XnRXb6mYvszzflMTi3bMh… ] Everyone is invited to attend the talk, and those faculty wishing to remain for the oral exam following are welcome to do so. Louis Riehl Graduate Administrator Computer Science Department, CS213 Princeton University (609) 258-8014

1 0

Jens Tuyls will present his General Exam "Multi-Stage Episodic Control for Strategic Exploration in Text Games" on Wednesday, May 4, 2022 at 3 pm in CS 401 and via Zoom.
by Louis W. Riehl 27 Apr '22

27 Apr '22

Jens Tuyls will present his General Exam "Multi-Stage Episodic Control for Strategic Exploration in Text Games" on Wednesday, May 4, 2022 at 3 pm in CS 401 and via Zoom. Zoom Link: https://princeton.zoom.us/j/94658001342 Committee Members: Karthik Narasimhan (advisor), Danqi Chen, Sham Kakade (Harvard University) Abstract: Text adventure games present unique challenges to reinforcement learning methods due to their combinatorially large action spaces and sparse rewards. The interplay of these two factors is particularly demanding because large action spaces require extensive exploration, while sparse rewards provide limited feedback. This work proposes to tackle the explore-vs-exploit dilemma using a multi-stage approach that explicitly disentangles these two strategies within each episode. Our algorithm, called eXploit-Then-eXplore (XTX), begins each episode using an exploitation policy that imitates a set of promising trajectories from the past, and then switches over to an exploration policy aimed at discovering novel actions that lead to unseen state spaces. This policy decomposition allows us to combine global decisions about which parts of the game space to return to with curiosity-based local exploration in that space, motivated by how a human may approach these games. Our method significantly outperforms prior approaches by 27% and 11% average normalized score over 12 games from the Jericho benchmark (Hausknecht et al., 2020) in both deterministic and stochastic settings, respectively. On the game of Zork1, in particular, XTX obtains a score of 103, more than a 2x improvement over prior methods, and pushes past several known bottlenecks in the game that have plagued previous state-of-the-art methods. Reading List: https://docs.google.com/document/d/1y7viEOlx5HiUZWqM1AnHogpBnsWPFUBgLhZo279… Everyone is invited to attend the talk, and those faculty wishing to remain for the oral exam following are welcome to do so.

1 1

Mengying (Molly) Pan will present her General Exam "Query-Driven Packet Control in the Data Plane" on Wednesday, May 4, 2022 at 9:00 AM in CS 301and via zoom.
by Louis W. Riehl 27 Apr '22

27 Apr '22

Mengying (Molly) Pan will present her General Exam "Query-Driven Packet Control in the Data Plane" on Wednesday, May 4, 2022 at 9:00 AM in CS 301and via zoom. Zoom link: https://princeton.zoom.us/my/mollypan Committee Members: Jennifer Rexford (advisor), David Walker, Ravi Netravali Abstract: As network traffic grows in both volume and diversity, operators are longing for a flexible, real-time control over the packets. For example, operators may want to rate-limit large flows, report heavy downloaders, or drop unsolicited incoming packets—none of which is feasible in traditional fixed-function switches. Fortunately, the last decade has seen the emergence of high-speed programmable switches, which is making all these applications possible by integrating measurement and control entirely in the data plane. This enables operators to define their own packet control programs that can run and adapt at line rate. However, programming control applications on the hardware is hard. The existing data plane languages like P4 are simply too low-level for anyone other than the domain experts to write. Moreover, it requires expertise in network measurement to choose and configure data structures, and careful simulation and testing to confidently deploy such low-level programs. In this talk, we present CatQL, a high-level programming language for writing query-driven packet control in the data plane. It strikes a balance between expressiveness and restrictiveness. Borrowing concepts from query and functional languages, CatQL is intuitive, expressive and concise, allowing operators to easily specify a wide range of practical packet control intentions within ten lines of code. Meanwhile, by modeling the key hardware constraints, our CatQL compiler takes an input program, searches for the best choices and configurations of data structures, and translates it into a P4 program that fits onto the Intel Tofino target. Finally, we develop our library of P4 data structures in a modular and layered manner to allow a repeatable and efficient formal verification of their correctness properties. Overall, with CatQL, we hope to provide a simple, performant and reliable language for operators to easily write and deploy packet control applications. Reading List: https://docs.google.com/document/d/1hXf5VkrVW1OkhQOeY9x8xzRS1WRyKZNYIRQLHro… Everyone is invited to attend the talk, and those faculty wishing to remain for the oral exam following are welcome to do so. Louis Riehl Graduate Administrator Computer Science Department, CS213 Princeton University (609) 258-8014

1 0

Nanqinqin Li will present his General Exam "Achieving Cheap, Highly Available Fault Tolerance with Disaggregated Storage" on Friday, April 29, 2022 at 9:00 AM via Zoom
by Louis W. Riehl 27 Apr '22

27 Apr '22

Nanqinqin Li will present his General Exam "Achieving Cheap, Highly Available Fault Tolerance with Disaggregated Storage" on Friday, April 29, 2022 at 9:00 AM via Zoom. Zoom link: [ https://princeton.zoom.us/j/7608328107 | https://princeton.zoom.us/j/7608328107 ] Committee Members: Michael J. Freedman (advisor), Amit Levy, Ravi Netravali Abstract: High availability has long been achieved by replication at the application level. In a classic scheme called Primary-Backup, the primary application instance handles all client requests and forwards the execution log to the backup instances. Application-level replication, however, provides high availability at the cost of maintaining fully redundant replicas: they require as many resources (CPU, RAM, disk, etc.) as the primary. The ubiquity of disaggregated storage in cloud computing offers an attractive alternative. Instead of maintaining live replicas, a newly-launched backup instance recovers the application state from the underlying disaggregated storage where data is already replicated for high durability. This alternative provides fault tolerance at a much lower cost but suffers from long failover periods because it must sequentially first detects the primary failure and only then starts recovery on a backup instance. We propose speculative recovery to accelerate failover and improve the availability of this alternative. Instead of proceeding with failover sequentially, speculative recovery safely and efficiently parallelizes detecting primary failure and running recovery on a backup by using a clone of the application storage. Our implementation and evaluation show that speculative recovery reduces failover time considerably with similarly lower costs than application-level replication Reading List: [ https://docs.google.com/document/d/1gY1RS2JFBWmgG-X1TWq8JOxLnfZJ1eeiOX2Vtgm… | https://docs.google.com/document/d/1gY1RS2JFBWmgG-X1TWq8JOxLnfZJ1eeiOX2Vtgm… ] Everyone is invited to attend the talk, and those faculty wishing to remain for the oral exam following are welcome to do so. Louis Riehl Graduate Administrator Computer Science Department, CS213 Princeton University (609) 258-8014

1 0

Riley Simmons-Edler will present his FPO "Overcoming Sampling and Exploration Challenges in Deep Reinforcement Learning" on Tuesday, May 3, 2022 at 5:00 PM in CS 402 and Zoom.
by Louis W. Riehl 26 Apr '22

26 Apr '22

Riley Simmons-Edler will present his FPO "Overcoming Sampling and Exploration Challenges in Deep Reinforcement Learning" on Tuesday, May 3, 2022 at 5:00 PM in CS 402 and Zoom. Location: Zoom link: https://princeton.zoom.us/j/91838854729?pwd=MXhZOGtiem8wQkoyMjA4Z3hJNWxVQT09 and CS 402 The members of Riley’s committee are as follows: Examiners: H. Sebastian Seung (Adviser), Olga Russakovsky, Szymon Rusinkiewicz Readers: Karthik Narasimhan, Daniel Lee (Cornell Tech) A copy of his thesis is available upon request. Please email gradinfo(a)cs.princeton.edu if you would like a copy of the thesis. Everyone is invited to attend his talk. Abstract follows below: The combination of deep neural networks with reinforcement learning (RL) shows great promise for solving otherwise intractable learning tasks. However, practical demonstrations of deep reinforcement learning remain scarce. The challenges in using deep RL for a given task can be grouped into two categories, broadly “What to learn from experience?” and “What experience to learn from?” In this thesis, I describe work to address the second category. Specifically, problems of sampling actions, states, and trajectories which contain information relevant to learning tasks. I examine this challenge at three levels of algorithm design and task complexity, from algorithmic components to hybrid combination algorithms that break common RL conventions. In the first chapter, I describe work on stable and efficient sampling of actions that optimize a Q-function of continuous-valued actions. By combining a sample-based optimizer with neural network approximation, it is possible to obtain stability in training, computational efficiency, and precise inference. In the second chapter, I describe work on reward-aware exploration, the discovery of desirable behaviors where common sampling methods are insufficient. A teacher “exploration” agent discovers states and trajectories which maximize the amount a student “exploitation” agent learns on those experiences, and can enable the student agent to solve hard tasks which are otherwise impossible. In the third chapter, I describe work combining reinforcement learning with heuristic search, for use in task domains where the transition model is known, but where the combinatorics of the state space are intractable for traditional search. By combining deep Q-learning with a best-first tree search algorithm, it is possible to find solutions to program synthesis problems with dramatically fewer samples than with common search algorithms or RL alone. Lastly, I conclude with a summary of the major takeaways of this work, and discuss extensions and future directions for efficient sampling in RL. Louis Riehl Graduate Administrator Computer Science Department, CS213 Princeton University (609) 258-8014

1 0

Austin Hounsel will present his FPO "Measuring the Feasibility of DNS Privacy and Security" on Tuesday, 5/3/2022 at 1:00pm via Zoom.
by Nicki Mahler 26 Apr '22

26 Apr '22

Austin Hounsel will present his FPO "Measuring the Feasibility of DNS Privacy and Security" on Tuesday, 5/3/2022 at 1:00pm via Zoom. Zoom link: [ https://princeton.zoom.us/j/ | https://princeton.zoom.us/j/99974051371 ] His committee is as follows: Nick Feamster (adviser); Readers: Prateek Mittal and Kyle Jamieson; Examiners: Nick Feamster, Jennifer Rexford, Ravi Netravali. All are welcome to attend. Nearly every service on the Internet relies on the Domain Name System (DNS), which translates humanreadable names to IP addresses. Originally, the DNS was designed without security and privacy concerns in mind. This has left users’ DNS trac subject to eavesdropping, tampering, and censorship. In response, encrypted DNS protocols and DNS Security Extensions (DNSSEC) have seen increased deployment. Although these developments stand to improve users’ security and privacy, two questions remain unanswered. First, it is unclear whether encrypted DNS protocols yield acceptable performance for popular applications, such as web browsers. Second, it is unclear whether users can reliably retrieve and validate DNS records that are supposedly protected by DNSSEC. Without answers to these questions, popular applications may choose to not utilize these technologies, compromising the security and privacy of their users. This thesis studies the feasibility of deploying DNS security and privacy technologies for everyday use through various Internet measurements. We first measure the performance of encrypted DNS protocols from data centers and home networks through query response times and page load times. We find that although encrypted DNS protocols generally result in longer query response times than traditional, unencrypted DNS, these protocols can perform comparably with well-chosen connection timeouts, connection reuse, and the usage of popular recursive resolvers. We also find that despite generally higher query response times, web pages can load faster with encrypted DNS protocols. We then collaborate with a major browser vendor to measure how often users can successfully retrieve and validate records with DNSSEC. To do so, we deploy a measurement add-on to a globally-distributed random sample of the browser’s release users. The add-on issues requests for various records types for a domain name that we control using the browser’s locally configured recursive resolver. We find that many users were unable to retrieve the correct DNSSEC records they needed to perform validation. Such failure prevents users from being able to reliably determine whether their DNS trac is under attack or not.

1 0

Daniel Greenidge will present his General Exam "Efficient decoding of large-scale neural population responses with Gaussian-process multiclass regression" on Friday, April 29, 2022 at 3:00 PM via Zoom.
by Louis W. Riehl 26 Apr '22

26 Apr '22

Daniel Greenidge will present his General Exam "Efficient decoding of large-scale neural population responses with Gaussian-process multiclass regression" on Friday, April 29, 2022 at 3:00 PM via Zoom. Zoom link: [ https://princeton.zoom.us/j/99012342978 | https://princeton.zoom.us/j/99012342978 ] Committee Members: Jonathan Pillow (advisor), Adji Bousso Dieng, Ben Raphael Abstract: Linear classifiers are a tool used by neuroscientists to quantify the information content of neural population codes and the limits imposed by correlations in neural activity. However, the most commonly used classification methods are prone to overfitting on the high-dimensional, noisy datasets produced by modern neuroimaging technologies. Here, we introduce a novel classifier (also called “decoder” in the neuroscience literature) to overcome these limitations. Our approach, the Gaussian process multi-class decoder (GPMD), is well-suited to decoding a continuous low-dimensional variable from high-dimensional population activity, and provides a platform for assessing the importance of correlations in neural population codes. The GPMD is a multinomial logistic regression model with a Gaussian Process prior over the decoding weights. We provide a spectral variational inference method for fitting the GPMD to data, which scales to datasets with hundreds or thousands of neurons and performs especially well in datasets with more neurons than trials. We apply the GPMD to recordings from primary visual cortex in three different species: monkey, ferret, and mouse. Our classifier achieves state-of-the-art accuracy on all three datasets, and substantially outperforms independent Bayesian decoding, showing that knowledge of the correlation structure is essential for optimal decoding in all three species. We then use our classifier to examine the problems of model mismatch, cortical microcolumn structure, and how neural coding strategies are modulated in the presence of noise. Reading List: [ https://docs.google.com/document/d/1euprtAlmBXa7wBaI4u0kznMw5GfGDeyzB3Lb0jj… | https://docs.google.com/document/d/1euprtAlmBXa7wBaI4u0kznMw5GfGDeyzB3Lb0jj… ] Everyone is invited to attend the talk, and those faculty wishing to remain for the oral exam following are welcome to do so. Louis Riehl Graduate Administrator Computer Science Department, CS213 Princeton University (609) 258-8014

1 0