<html><body><div style="font-family: garamond,new york,times,serif; font-size: 12pt; color: #000000"><div data-marker="__QUOTED_TEXT__"><p style="margin-left: 50px;" data-mce-style="margin-left: 50px;">Tengyu Ma will present his FPO, "Non-convex Optimization for Machine Learning: Design, Analysis, and Understanding" on Thursday, 10/12/2017 at 3pm, in CS 402.</p><p style="margin-left: 50px;" data-mce-style="margin-left: 50px;">The members of his committee are: Sanjeev Arora (adviser); Readers: Rong GE (Duke), Sanjeev Arora, and David Steurer (ETH Zurich); Nonreaders: Sanjeev Arora, Elad Hazan, and Mark Braverman.</p><p style="margin-left: 50px;" data-mce-style="margin-left: 50px;">A copy of his thesis, is available in Room 310. Everyone is invited to attend his talk. The talk abstract follows below:</p><p style="margin-left: 50px;" data-mce-style="margin-left: 50px;">Non-convex optimization is ubiquitous in modern machine learning: recent breakthroughs<br>in deep learning require optimizing non-convex training objective functions;<br>problems that admit accurate convex relaxation can often be solved more efficiently<br>with non-convex formulations. However, the theoretical understanding of optimizing<br>non-convex functions remained rather limited to the intractability result of optimizing<br>worst case non-convex functions. Can we extend the algorithmic frontier by effi-<br>ciently optimizing a family of interesting non-convex functions? Can we successfully<br>apply non-convex optimization to machine learning problems with provable guarantees?<br>How do we interpret the complicated models in machine learning that demand<br>non-convex optimizers?<br>Towards addressing these questions, in this thesis, we theoretically studied various<br>machine learning models including sparse coding, topic models, and matrix completion,<br>linear dynamical systems, and word embeddings.<br>We first consider how to find a coarse solution to serve as a good starting point<br>for local improvement algorithms such as stochastic gradient descent. We propose ef-<br>ficient methods for sparse coding and topic inference with better provable guarantees.<br>Second, we propose a framework for analyzing local improvement algorithms with<br>a good starting point. We apply it successfully to the sparse coding problem and<br>demonstrate its advantage over other related frameworks.<br>Then, we consider minimizing a family of non-convex functions that local improvement<br>algorithms can succeed efficiently from random or arbitrary initialization<br>— the family of functions of which all local minima are also global. The challenge, in<br>turn, becomes proving the objective function belongs to this class. We establish such<br>type of results for the natural learning objectives for matrix completion and learning<br>linear dynamical systems.<br>iii<br>Finally, we make steps towards interpreting the non-linear models that require<br>non-convex training algorithms. We reflect on the principles of word embeddings in<br>natural language processing. We give a generative model for the texts, using which<br>we explain why different non-convex formulations such as word2vec and GloVe can<br>learn similar word embeddings with the surprising performance — analogous words<br>have embeddings with similar differences. </p><div><p style="margin: 0px;" data-mce-style="margin: 0px;"><br></p><div><br></div></div></div></div></body></html>