<html><head><style type='text/css'>p { margin: 0; }</style></head><body><div style='font-family: arial,helvetica,sans-serif; font-size: 10pt; color: #000000'>Tengyu Ma will present his research seminar/general exam on Wednesday January 22 at <br>2PM in Room 301 (note room!). The members of his committee are: Sanjeev Arora (advisor), <br>Moses Charikar, and David Blei. Everyone is invited to attend his talk, and those faculty <br>wishing to remain for the oral exam following are welcome to do so. His abstract and <br>reading list follow below.<br><br><hr id="zwchr"><div style="color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt;"><b></b><div dir="ltr"><br><div style="font-family:arial,sans-serif;font-size:13px">abstract </div><div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">
I will mainly talk about our recent paper on deep learning <i>provable bound for learning some deep representations.</i> <span style="font-size:12.727272033691406px">Deep learning, a modern variant of artificial neural net, has been a very powerful tool in recent years for many machine learning tasks such as image recognition, speech recognition and natural language processing. Variations of However, theoretically, it still remains mysterious why these modern heuristics and tricks together with millions of data result in big improvements in so many important questions. With the motivation to resolve these mysteries, this paper proposes a generative model for deep network, and provides an provable algorithm that can learn all the parameters and hidden variables unsupervisedly using polynomial time and polynomial samples generated from this network under some assumptions</span></div>
<div style="font-family:arial,sans-serif;font-size:13px"><span style="font-size:12.727272033691406px"><br></span></div><div style="font-family:arial,sans-serif;font-size:13px"><span style="font-size:12.727272033691406px">In more detail, the network we study has constant number of layers, each consecutive two layers are connected via a random sparse bipartite graph/matrix. The observed data at bottom are generated as follows: First a sparse hidden vector at the top layer is chosen, then the top layer activates some of the second layer nodes by multiplying the connection matrix and applying another point-wise threshold function, and then adding some small noise. Then the second layer activates the third using the same rules with the new connection matrix, so on and so forth. Our algorithms exploits the old Hebbian rule: "Things fire together wire together", that is, if two nodes in the same layer are correlated, then they are likely to connect to the same ancestor in the layer above them. </span><span style="font-size:12.727272033691406px">This is the joint work with Sanjeev Arora, Aditya Bhaskara and Rong Ge. </span></div>
<p class="">Reading list: </p><p class="">0. Algorithm Design, Jon Kleinberg, Éva Tardos</p>
<p class="">1. Ankur Moitra's lecture notes for the course Algorithmic Aspects of Machine Learning</p>
<p class=""><span class=""><a href="http://people.csail.mit.edu/moitra/996.html" target="_blank">http://people.csail.mit.edu/moitra/996.html</a></span></p>
<p class="">2. Daniel Hsu's lecture notes for COMS 4772 Advanced Machine Learning (Fall 2013)</p>
<p class=""><span class=""><a href="http://www.cs.columbia.edu/~djhsu/4772/" target="_blank">http://www.cs.columbia.edu/~djhsu/4772/</a></span></p>
<p class="">3. Machine Learning: A Probabilistic Perspective By Kevin P. Murphy (provided I could find it in library) </p>
<p class=""><span class=""><a href="http://mitpress.mit.edu/sites/default/files/titles/content/9780262018029_toc_0001.pdf" target="_blank">http://mitpress.mit.edu/sites/default/files/titles/content/9780262018029_toc_0001.pdf</a></span></p>
<p class="">4. Representation Learning: A Review and New Perspectives, by Yoshua Bengio, Aaron Courville, Pascal Vincent</p>
<p class="">5. Visualizing and Understanding Convolutional Networks<span class=""><a href="http://arxiv.org/abs/1311.2901" target="_blank">http://arxiv.org/abs/1311.2901</a></span></p>
<p class="">6. ImageNet Classification with Deep Convolutional Neural Networks by Krizhevsky, A., Sutskever, I. and Hinton, G. E.</p>
<p class="">7. Generalized Denoising Auto-Encoders as Generative Models Bengio et al. </p>
<p class=""><span class=""><a href="http://arxiv.org/abs/1305.6663" target="_blank">http://arxiv.org/abs/1305.6663</a></span></p>
<p class="">8. On the Provable Convergence of Alternating Minimization for Matrix Completion</p>
<p class="">9. Tensor Decompositions for Learning Latent Variable Models by Anandkumar, Ge, Hsu, Kakade, Telgarsky</p>
<p class="">10. A. Kumar, R. Kannan, Clustering with Spectral Norm and the k-Means Algorithm</p>
<p class="">11. V. Chandrasekaran, P. Parrilo, A. Willsky, Latent Variable Graphical Model Selection via Convex Optimization</p></div>
</div><br></div></body></html>