Yi Zhang will present his Pre FPO “Advancing Deep Learning by Integrating Theory and Empirics” on Friday, April 30, 2021 at 2PM via Zoom.

Zoom Link: https://us02web.zoom.us/j/6737994417?pwd=VUE2WWZXL1dTNDg1SWxFYjdyMGEvQT09

Committee:

Examiners: Sanjeev Arora (advisor), Karthik Narasimhan, Jason D. Lee (ECE)

Readers: Elad Hazan, Chi Jin (ECE)

All are welcome to attend.

Abstract:

In sharp contrast to its remarkable empirical success stands a mathematical understanding of deep learning that is still in its infancy. Various puzzling behaviors of deep neural nets remain unexplained, and many widely deployed deep learning systems lack theoretical guarantees. This talk offers a perspective on the resemblance between deep learning research and natural sciences, especially modern physics at its formative stage where reciprocal interactions between theoretical and experimental studies fueled the growth. As the central object of study, deep neural networks are to deep learning as unknown particles are to physics. We can make significant progress by building theories inspired by empirical observations and verifying hypotheses with designed experiments. In this talk, I will present several of my representative works that follow this research philosophy. Firstly, I will introduce a finite-sample analysis of Generative Adversarial Networks that predicts the existence of degenerate solutions (i.e. mode collapse), which we confirm empirically using a principled test. Then I will present how the empirically identified 'noise stability' of deep neural networks trained on real-life data leads to a substantially stronger generalization measure for deep learning. Finally, I will describe our recent work on designing a simple test for measuring how much deep learning has overfitted to standard datasets.