- talks - lists.cs.princeton.edu

Kathleen Chen will present her General Exam "A sequence-based global map of regulatory activity for deciphering human genetics" on Monday, September 27, 2021 at 11:30AM via Zoom
by jfarquer＠cs.princeton.edu 21 Sep '21

21 Sep '21

Kathleen Chen will present her General Exam "A sequence-based global map of regulatory activity for deciphering human genetics" on Monday, September 27, 2021 at 11:30AM via Zoom. Zoom Link: https://princeton.zoom.us/j/96378734253 The members of Kathleen's committee are as follows: Olga Troyanskaya (adviser), Mona Singh, Yuri Pritykin Title: A sequence-based global map of regulatory activity for deciphering human genetics Everyone is invited to attend the talk, and those faculty wishing to remain for the oral exam following are welcome to do so. Abstract: Sequence is at the basis of how the genome shapes chromatin organization, regulates gene expression, and impacts traits and diseases. Epigenomic profiling efforts have enabled large-scale identification of regulatory elements, yet we still lack a sequence-based map to systematically identify regulatory activities from any sequence, which is necessary for predicting the effects of any variant on these activities. We address this challenge with Sei, a new framework for integrating human genetics data with sequence information to discover the regulatory basis of traits and diseases. Our framework systematically learns a vocabulary for the regulatory activities of sequences, which we call sequence classes, using a new deep learning model that predicts a compendium of 21,907 chromatin profiles across >1,300 cell lines and tissues, the most comprehensive to-date. Sequence classes allow for a global view of sequence and variant effects by quantifying diverse regulatory activities, such as loss or gain of cell-type-specific enhancer function. We show that sequence class predictions are supported by experimental data, including tissue-specific gene expression, expression QTLs, and evolutionary constraints based on population allele frequencies. Finally, we applied our framework to human genetics data. Sequence classes uniquely provide a non-overlapping partitioning of GWAS heritability by tissue-specific regulatory activity categories, which we use to characterize the regulatory architecture of 47 traits and diseases from UK Biobank. Furthermore, the predicted loss or gain of sequence class activities suggest specific mechanistic hypotheses for individual regulatory pathogenic mutations. We provide this framework as a resource to further elucidate the sequence basis of human health and disease. Reading List: Textbook: Jones, N.C. and Pevzner, P.A. An Introduction to Bioinformatics Algorithms. Cambridge, MA: MIT Press (2004). Papers: 1. Zhou, J. et al. "Deep learning sequence-based ab initio prediction of variant effects on expression and disease risk." Nat Genetics vol. 50,8 (2018): 1171-1179. doi:10.1038/s41588-018-0160-6 2. Zhou, J., Park, C.Y, et al. Theesfeld, C.L. et al. "Whole-genome deep-learning analysis identifies contribution of noncoding mutations to autism risk." Nat Genetics 51, 973-980 (2019). 3. Kelley, D.R., et al. "Sequential regulatory activity prediction across chromosomes with convolutional neural networks." Genome Res 28(5), 739-750 (2018). 4. Avsec et al. "Base-resolution models of transcription-factor binding reveal soft motif syntax." Nat Genetics 53, 354-366 (2021). 5. Edwards, S.L. et al. "Beyond GWASs: illuminating the dark road from association to function." American Journal of Human Genetics vol. 93,5 (2013): 779-97. 6. Boix, C.A. et al. "Regulatory genomic circuitry of human disease loci by integrative epigenomics." Nature 590, 300-307 (2021). 7. Chen et al. "Determinants of transcription factor regulatory range." Nat Communications 11, 2472 (2020). 8. Rheinbay E. et al. "Analyses of non-coding somatic drivers in 2,658 cancer whole genomes." Nature 578, 102-111 (2020). 9. Weissbrod O. et al. "Functionally informed fine-mapping and polygenic localization of complex trait heritability." Nat Genetics 52, 1355-1363 (2020).

1 0

Comparative Literature talk: Dennis Tenen, 9/23 at 4:30PM
by Emily C. Lawrence 20 Sep '21

20 Sep '21

Dearest colleagues, The Department of Comparative Literature is proud to welcome Professor Dennis Tenen of Columbia University to give the first lecture in this year’s departmental lecture series. This year’s lecture series is entitled “Virtually Here, Virtually So ” and it aims at thinking through the many meanings of the word “virtual” i ncluding, but by no means limited to, virtual reality, things that are “virtually" the same yet profoundly different, and the similarities and discrepancies between our “true” selves and our “virtual” selves. Professor Tenen is the author of " Plain Text: The Poetics of Computation" (2017) and his work lies at the intersection of literary theory, new media studies, and coding studies. He will be presenting a lecture entitled “Media and Metaphor (Or How Literary Theory Figures in the Study of Human Computer Interaction)” on Thursday September 23rd at 4:30 PM in East Pyne 010 . The talk will be immediately followed by a small reception just outside of the auditorium. There are also limited spots available for dinner with Professor Tenen at 6:30 PM after the lecture at 6:30PM. If you are interested in joining us for dinner or have any questions or concerns, please address them to Daniel Dominguez ( [ mailto:dd10@princeton.edu | dd10(a)princeton.edu ] ). We hope everyone has a wonderful week and we sincerely hope you’ll join us this week to listen to Professor Tenen speak. Best wishes, Daniel Dominguez Series Organizer

1 0

Khiem Ngo will present his FPO "TOLERATING SLOWDOWNS IN REPLICATED STATE MACHINES" on Wednesday, August 25, 2021 at 3PM via Zoom.
by jfarquer＠cs.princeton.edu 19 Aug '21

19 Aug '21

Khiem Ngo will present his FPO "TOLERATING SLOWDOWNS IN REPLICATED STATE MACHINES" on Wednesday, August 25, 2021 at 3PM via Zoom. Zoom link: https://princeton.zoom.us/j/97848004548 The members of Khiem's committee are as follows: Wyatt Lloyd (Adviser); Readers: Amit Levy and Siddhartha Sen (Microsoft Research); Examiners: Michael Freedman, Ravi Netravali, Wyatt Lloyd. A copy of his thesis is available upon request. Please email gradinfo(a)cs.princeton.edu <mailto:gradinfo@cs.princeton.edu> if you would like a copy of the thesis. All are welcome to attend his talk. Abstract follows below: Replicated state machines (RSMs) are linearizable, fault-tolerant groups of replicas that are coordinated using a consensus algorithm. RSMs are used throughout large-scale sys[1]tems, such as distributed databases, cloud storage, and service managers. At such scale, it is common for some machines to be slow. In this dissertation, we address the prob[1]lem of developing slowdown-tolerant RSMs that continue to operate as fast RSMs despite the presence of slow replicas. We define s-slowdown-tolerance and identify why existing consensus protocols are not slowdown-tolerant. As the first and pragmatic step toward slowdown-tolerant RSMs, we design, implement, and evaluate two 1-slowdown-tolerant consensus protocols, Copilot and Latent Copilot. Copilot replication is the first 1-slowdown-tolerant consensus protocol: it delivers normal latency despite the slowdown of any 1 replica. Copilot uses two distinguished replicas-the pilot and copilot-to proactively add redundancy to all stages of processing a client's command. Copilot uses dependencies and deduplication to resolve potentially differing orderings proposed by the pilots. To avoid dependencies leading to either pilot being able to slow down the group, Copilot uses fast takeovers that allow a fast pilot to complete the ongoing work of a slow pilot. Copilot includes two optimizations-ping[1]pong batching and null dependency elimination-that improve its performance when there are 0 and 1 slow pilots respectively. Our evaluation of Copilot shows its performance is lower but competitive with Multi-Paxos and EPaxos when no replicas are slow. When a replica is slow, Copilot is the only protocol that avoids high latencies. Latent Copilot, a variant of Copilot, is another design and implementation of a 1- slowdown-tolerant consensus protocol. Latent Copilot operates with one active pilot, which actively proposes commands, and one latent pilot, which proposes commands only when necessary. In this way, Latent Copilot achieves an intermediate tradeoff between Multi[1]Paxos and Copilot in terms of throughput and slowdown tolerance. Our evaluation of La[1]iii tent Copilot shows that it achieves 1-slowdown-tolerance and that its performance with no slowdowns is comparable to Multi-Paxos. Finally, we generalize the design of Copilot replication to handle more than one slow[1]down. We provide the design of an s-slowdown-tolerant protocol that achieves slowdown tolerance despite s slow replicas.

1 0

Fermi Ma will present his FPO "Quantum Security and Fiat-Shamir for Cryptographic Protocols" on Tuesday, August 24, 2021 at 4pm via Zoom.
by jfarquer＠cs.princeton.edu 16 Aug '21

16 Aug '21

Fermi Ma will present his FPO "Quantum Security and Fiat-Shamir for Cryptographic Protocols" on Tuesday, August 24, 2021 at 4pm via Zoom. Zoom link: https://princeton.zoom.us/j/97642906902 The members of his committee are as follows: Mark Zhandry (Adviser); Readers: Zeev Dvir, Ran Raz; Examiners: Sanjeev Arora, Gillat Kol, and Mark Zhandry. A copy of his thesis is available upon request. Please email gradinfo(a)cs.princeton.edu <mailto:gradinfo@cs.princeton.edu> if you would like a copy of the thesis. Everyone is invited to attend his talk. Abstract: This talk will cover new results on the security of cryptographic protocols in the quantum setting and the Fiat-Shamir heuristic for removing interaction. (1) Quantum Rewinding. Many cryptosystems are proven secure by rewinding an interactive adversary to extract its responses across multiple invocations. Unfortunately, this technique only suffices for classical security, since recording the outputs of a quantum adversary may irreversibly damage its state. Obtaining a suitable quantum analogue of rewinding has been a long-standing open problem in post-quantum cryptography. In this talk, I will describe a general-purpose quantum rewinding procedure capable of extracting an unlimited number of responses from any quantum adversary; prior techniques were limited to a constant number. Our primary application is to prove that Kilian's succinct argument system for NP is post-quantum secure. (2) Quantum Secure Computation from One-Way Functions. Next, I will cover results on multi-party computation, a central primitive in cryptography that enables mutually distrusting parties to compute a shared function over their inputs while revealing no other information. We show that when all parties are quantum, secure multi-party computation can be based solely on the existence of the minimal cryptographic primitive: one-way functions. This is in stark contrast to the classical setting where such an implication is not known, and is considered unlikely. (3) The Security of Fiat-Shamir. The final part of this talk will cover the soundness of the Fiat-Shamir heuristic, a powerful technique that uses a cryptographic hash function to remove interaction from certain cryptographic protocols. We consider two popular applications of Fiat-Shamir: building non-interactive succinct arguments from Kilian's protocol and obtaining digital signatures from a wide range of identification protocols. We demonstrate significant barriers to soundly instantiating Fiat-Shamir for Kilian's succinct arguments using any concrete Fiat-Shamir hash function. We also raise the interesting possibility that natural identification protocols can be compiled with simple, non-cryptographic Fiat-Shamir hash functions.

1 0

Friday, August 13, 2021 - Princeton Wireless Distinguished Seminar Series - Sumit Roy, University of Washington
by Emily C. Lawrence 06 Aug '21

06 Aug '21

ECE Princeton Wireless Distinguished Seminar Series Speaker: Sumit Roy, University of Washington Title: Future Vehicular Radars: Two Aspects Day: Friday, August 13, 2021 Time: 1:00 pm Room: Please register here: [ https://princeton.zoom.us/meeting/register/tJAvd-2rqjkuHdQcCp7MUlGrW7CSb3Th… | https://princeton.zoom.us/meeting/register/tJAvd-2rqjkuHdQcCp7MUlGrW7CSb3Th… ] Host: Yasaman Ghasempour Abstract: This talk will present an overview of recent research in FUNLAB around the use of vehicular radar for advanced driver assistance systems (en route to a future vision of autonomous driving). Wideband (typically FMCW or chirp) radars are being increasingly deployed onboard vehicles as key high-resolution sensor for environmental mapping/imaging and various safety features. The talk will focus on the evolving role of radar `cognition’ in complex operating environments to address two important future challenges: 1. Mitigating multi-access interference among Radars (e.g. dense traffic scenario) The talk will first illustrate the impact of mutual interference on detection performance in Chirp/FMCW radars and then highlight some multi-access protocol design approaches for effective resource sharing among multiple radars. 2. Contributions to radar vision via new radar hardware (MIMO radar) + associated advanced signal processing (Synthetic Aperture principles) as well as Convolutional Neural Network (`Radar Net’) based machine learning approach for enhanced object detection/classification in challenging circumstances. Bio: Sumit Roy received the B. Tech. degree from the Indian Institute of Technology (Kanpur) in 1983, and the M. S. and Ph. D. degrees from the University of California (Santa Barbara), all in Electrical Engineering in 1985 and 1988 respectively, as well as an M. A. in Statistics and Applied Probability in 1988. He is currently Professor, Electrical & Comp. Engineering, appointed to a term Distinguished Professorship for Integrated Systems (2014-19); since Sep. 2020, he serves as a rotator for US Dept. of Defense’s Program Lead for Innovate Beyond 5G https://5g-to-xg.org. His research interests include fundamental design and evaluation of wireless communication and sensor network systems spanning a diversity of technologies and application areas: next-gen (5G & beyond) wireless LANs and cellular networks, heterogeneous network coexistence, spectrum sharing, software defined radio platforms, vehicular and sensor networking. He spent 2001-03 on academic leave at Intel Wireless Technology Lab as a Senior Researcher engaged in systems architecture and standards development for ultra-wideband systems (Wireless PANs) and next generation high-speed wireless LANs. During Jan-July 2008, he was Science Foundation of Ireland’s E.T.S. Walton Awardee for a sabbatical at University College, Dublin and also recipient of a Royal Acad. Engineering (UK) Distinguished Visiting Fellowship during summer 2011. He has served as IEEE Communications Society (ComSoc) Distinguished Lecturer and as Associate Editor for all the major ComSoc journals. He served 2 terms as (elected) member of Executive Committee, National Spectrum Consortium dedicated to efficient spectrum sharing between Federal and commercial networks and is the co-author of IEEE TAES 2016 Best paper award for work on Radar-Comm coexistence. He was elevated to IEEE Fellow by Communications Society in 2007 for ``contributions to multi-user communications theory and cross-layer design of wireless networking standards”.

1 0

Kevin Qipeng Liu will present his FPO "Cryptography in the Age of Quantum Computers 2.0" on Tuesday, July 27, 2021 at 4PM via Zoom.
by jfarquer＠cs.princeton.edu 22 Jul '21

22 Jul '21

Kevin Qipeng Liu will present his FPO "Cryptography in the Age of Quantum Computers 2.0" on Tuesday, July 27, 2021 at 4PM via Zoom. Zoom link: https://princeton.zoom.us/j/6541841088 The members of Kevin's committee are as follows: Mark Zhandry (Adviser); Readers: Ran Raz and Gillat Kol; Examiners: Mark Braverman, Matthew Weinberg and Mark Zhandry. A copy of his thesis is available upon request. Please email gradinfo(a)cs.princeton.edu <mailto:gradinfo@cs.princeton.edu> if you would like a copy of the thesis. Everyone is invited to attend his talk. Abstract follows below: People widely believe that full-scale quantum computers will eventually be viable in the near future. Quantum computers pose threats to many existing cryptosystems (most prominently, Shor's algorithm) while raising the possibility of using quantum[1]mechanical phenomena to achieve never-before-possible capabilities. First, I will present my work on quantum query complexity: I will show tight bounds for multi-collision finding problems and tight time-space tradeoffs for function inversion problems. The latter indicates that Grover's search cannot be sped up even with a piece of preprocessed quantum advice. This technique can be extended to prove the post-quantum non-uniform security of many existing cryptographic schemes. Second, I will present my work on post-quantum zero-knowledge proof. I will start by showing that post-quantum constant-round zero-knowledge protocols for NP with black-box simulators do not exist in the plain model unless NP is in BQP. Then, I will show how to achieve non-interactive zero-knowledge in the quantum random oracle model by proving that the famous Fiat-Shamir transformation is secure. Third, I will present my work on quantum copy-protection, which aims to use the unclonability of quantum states to achieve programs that cannot be copied. Towards this goal, I will show that any unlearnable function can be copy-protected relative to classical oracles. For specific cryptographic functionalities (such as signatures and RPFs), I will show that they can be copy-protected in the plain model.

1 0

Themistoklis Melissaris will present his FPO "Testing and Analyzing Correctness in Concurrent Systems: From Microprocessors to IoT and Distributed Systems" on Friday, July 30, 2021 at 12noon via Zoom.
by jfarquer＠cs.princeton.edu 20 Jul '21

20 Jul '21

Themistoklis Melissaris will present his FPO "Testing and Analyzing Correctness in Concurrent Systems: From Microprocessors to IoT and Distributed Systems" on Friday, July 30, 2021 at 12PM via Zoom. Zoom Link: https://princeton.zoom.us/j/98466211870 The members of his committee are as follows: Margaret Martonosi (Adviser); Kelly Shaw (Co-Adviser); Readers: Kyle Jamieson, Kelly Shaw (Williams College); Examiners: Wyatt Lloyd, David August, Margaret Martonosi. A copy of his thesis is available upon request. Please email gradinfo(a)cs.princeton.edu <mailto:gradinfo@cs.princeton.edu> if you would like a copy of the thesis. Everyone is invited to attend the talk. Abstract follows below: Applications such as the Internet of Things (IoT), multimedia streaming and self driving cars are drivers for explosive data growth. This data growth along with the slowdown of Moore's law presents new challenges for computing and spurs innovation for the next generation of hardware and software. To extract performance at manageable power, systems ranging from microprocessors to IoT devices and distributed systems increasingly leverage parallelism and concurrency. However, the performance and power benefits of parallelism come at the cost of increased complexity in designing, specifying and testing correctness of these systems. This dissertation draws insights across the domains of computer architecture, ΙοΤ and distributed systems, resulting in a set of practical tools and methodologies that allow for rigorous and efficient testing of correctness on multiprocessors, IoT systems and distributed systems. More specifically, this thesis introduces PerpLE, a tool that enables consistency testing for multiprocessors without per-iteration synchronization and relies on property based testing using litmus tests. This dissertation demonstrates how PerpLE speeds up testing performance by orders of magnitude against the state of the art, enabling fast error discovery in commercial microprocessors. Additionally, this dissertation presents Musli Tool, a practical tool for empirical consistency testing of distributed systems that leverages techniques and methodologies based on the computer architecture concept of litmus tests. Musli Tool automates testing by generating property-based tests and increases testing performance and fast portability in open source distributed systems. Finally, this thesis presents a study of rigorous concurrency testing on commercial and open source IoT systems. The study demonstrates correctness deficiencies of IoT systems and identifies the implications of race conditions, lack of atomicity and weak consistency on applications. To mitigate erroneous behavior for IoT applications, the OKAPI tool is introduced. This dissertation presents how to vastly improve the performance and efficiency of consistency testing by leveraging a synchronization free variant of litmus tests and by developing tools to enable analysis and generation of tests based on this new methodology. This thesis also demonstrates how to take lessons learned from testing correctness in microprocessors and adapting to other domains of concurrent systems, particularly distributed systems and IoT.

1 0

Orestis Plevrakis will present his FPO "New Exploration and Identification schemes for Linear Dynamics: the cases of Convex costs and Censored Data" on Thursday, July 22, 2021 at 1PM via Zoom.
by jfarquer＠cs.princeton.edu 15 Jul '21

15 Jul '21

Orestis Plevrakis will present his FPO "New Exploration and Identification schemes for Linear Dynamics: the cases of Convex costs and Censored Data" on Thursday, July 22, 2021 at 1PM via Zoom. Zoom link: <https://princeton.zoom.us/j/92035947506> https://princeton.zoom.us/j/92035947506 The members of Orestis' committee are as follows: Sanjeev Arora (Adviser); Readers: Chi Jin and Jason Lee; Examiners: Mark Braverman, Elad Hazan and Sanjeev Arora. A copy of his thesis is available upon request. Please email <mailto:gradinfo@cs.princeton.edu> gradinfo(a)cs.princeton.edu if you would like a copy of the thesis. Everyone is invited to attend his talk. Abstract is as follows: Adaptive control, i.e., learning to control an unknown linear dynamical system, has a long history in statistics and control theory. However, sharp non-asymptotic bounds were only recently obtained, and are outputs of an ongoing research program aiming to apply tools from learning theory and optimization, to control theory. A common feature of all the major results of the program has been the learning-part of the proposed algorithms, i.e., the exploration strategy. All these algorithms apply the so-called "certainty-equivalence", where the learner spends the first steps applying random controls (e.g. Gaussian noise), then estimates the dynamics via least-squares, and thereafter optimizes the control-policy using these estimates. Even though this is the simplest exploration strategy, it is optimal for the classical linear quadratic regulator (LQR). In this thesis, I study 1) the problem of adaptive control when the cost can be any convex function (thus generalizing LQR), and 2) the problem of learning a linear dynamical system (LDS) from "censored" observations, i.e., the state is observed only when it falls inside some set (e.g., the state can be the position of an object and the set can be the camera-frame). For the first problem, certainty-equivalence is suboptimal. For the second, the least-squares solution is not even consistent. The contribution of the thesis is the development of new computationally and statistically efficient algorithms for both problems. Specifically, 1. For online control with convex costs, we consider the objective of regret vs. the class of disturbance-feedback-controllers. Leveraging ideas from convex geometry, we design an exploration strategy that achieves optimal regret, improving upon the previous best known bounds, which correspond to algorithms applying certainty-equivalence. 2. The problem of learning an LDS from censored observations was first considered by Lee and Maddala (1985), and Zeger and Brookmeyer (1986). We develop a new second order method: Stochastic Online Newton with Switching Gradients, building on the Online Newton Step of Hazan et al. (2007). This method is the first computationally and statistically efficient algorithm for this problem.

1 0

Alejandro Newell will present his Pre-FPO "Learning to solve structured vision problems" on Tuesday, July 20, 2021 at 2pm via Zoom.
by jfarquer＠cs.princeton.edu 13 Jul '21

13 Jul '21

Alejandro Newell will present his Pre-FPO "Learning to solve structured vision problems" on Tuesday, July 20, 2021 at 2pm via Zoom. Zoom Link: https://princeton.zoom.us/j/7228079201 Committee: Examiners: Jia Deng (Adviser), Adam Finkelstein, Felix Heide Readers: Olga Russakovsky, Szmon Rusinkiewicz All are welcome to attend. Title: Learning to solve structured vision problems Abstract: We want computer vision models to understand the rich world captured in images and video. This means not just recognizing objects, but seeing how they relate and interact. In the first half of my talk, I will present my work developing a general system for capturing visual relationships. Combining our contributions in both neural architecture and loss design, we enable deep models to express arbitrary semantic graphs. This allows models to connect parts of the image into structured outputs such as body parts into poses. These models outperform prior work in various challenging vision tasks. In the second half of the talk, I will cover ongoing work that moves beyond single images into video. Connections across time are often captured at the pixel level with optical flow. We show how to improve existing flow systems with feedback mechanisms inspired by self-supervised losses. I will also discuss future work extending such systems to downstream video tasks. Together, the presented body of work lays the foundations for systems that understand the world and its interactions across space and time.

1 0

Wei Hu will present his FPO "Understanding Deep Learning via Analyzing Dynamics of Gradient Descent" on Friday, July 23, 2021 at 9:30AM via zoom.
by jfarquer＠cs.princeton.edu 13 Jul '21

13 Jul '21

Wei Hu will present his FPO "Understanding Deep Learning via Analyzing Dynamics of Gradient Descent" on Friday, July 23, 2021 at 9:30AM via Zoom. Zoom link: https://princeton.zoom.us/j/2180408086 The members of Wei's committee are as follows: Sanjeev Arora (Adviser); Readers: Elad Hazan and Jason Lee; Examiners: Mark Braverman, Chi Jin and Sanjeev Arora. A copy of his thesis is available upon request. Please email gradinfo(a)cs.princeton.edu <mailto:gradinfo@cs.princeton.edu> if you would like a copy of the thesis. Everyone is invited to attend his talk. Abstract follows below: The phenomenal successes of deep learning build upon the mysterious abilities of gradient-based optimization algorithms. Not only can these algorithms often successfully optimize complicated non-convex training objectives, but the solutions found can also generalize remarkably well to unseen test data despite significant over-parameterization of the models. Classical approaches in optimization and generalization theories that treat empirical risk minimization as a black box are insufficient to explain these mysteries in modern deep learning. This dissertation illustrates how we can make progress toward understanding optimization and generalization in deep learning by a more refined approach that opens the black box and analyzes the dynamics taken by the optimizer. In particular, we present several theoretical results that take into account the learning dynamics of the gradient descent algorithm. In the first part, we provide global convergence guarantees of gradient descent for training deep linear networks under various initialization schemes. Our results characterize the effect of width, depth and initialization on the speed of optimization. In addition, we identify an auto-balancing effect of gradient flow, which we prove to hold generally in homogeneous neural networks (including those with ReLU activation). In the second part, we study the implicit regularization induced by gradient descent, which is believed to be the key to mathematically understanding generalization in deep learning. We present results in both linear and non-linear neural networks, which characterize how gradient descent implicitly favors simple solutions. In the third part, we focus on the setting where neural networks are over-parameterized to have sufficiently large width. Through the connection to neural tangent kernels, we perform a fine-grained analysis of optimization and generalization, which explains several empirically observed phenomena. Built on these theoretical principles, we further design a new simple and effective method for training neural networks on noisily labeled data.

1 0