[Ml-stat-talks] Ben Recht talk
ehazan at CS.Princeton.EDU
Mon Sep 14 15:17:37 EDT 2015
Ben Recht of Berkeley will give a joint PACM / CS colloquium, Sep 28
4:30pm at 214 Fine Hall, details below. Ben is one of the most innovative
machine learning researchers today, highly recommended!
Title: The Resilience of the Perceptron
Abstract: The most widely used optimization method in machine learning
practice is the Perceptron Algorithm, also known as the Stochastic Gradient
Method (SGM). This method has been used since the fifties to build
statistical estimators, iteratively improving models by correcting errors
observed on single data points. SGM is not only scalable, robust, and
simple to implement, but achieves the state-of-the-art performance in many
different domains. In contemporary systems, SGM powers enterprise
analytics systems and is the workhorse tool used to train complex
pattern-recognition systems in speech and vision.
In this talk, I will explore why SGM has had such staying power, focusing
on notions of stability and robustness. I will first discuss how SGM is
robust to perturbations of the model and the updates. From a computing
systems perspective, this robustness enables parallel implementations with
minimal communication, with no locking or synchronization, and with strong
spatial locality. I will then show how SGM is robust to perturbations of
the data itself, and prove that any model trained with stochastic gradient
method in a reasonable amount of time attains small generalization error.
I will subsequently provide a new interpretation of common practices in
neural networks, and provide a formal rationale for many popular techniques
in training large, deep models.
Bio: Benjamin Recht is an Associate Professor in the Department of
Electrical Engineering and Computer Sciences and the Department of
Statistics at the University of California, Berkeley. His work focuses on
applications of optimization, mathematical statistics, and randomized
algorithms in data analysis and machine learning. He is the recipient of a
Presidential Early Career Awards for Scientists and Engineers, an Alfred P.
Sloan Research Fellowship, the 2012 SIAM/MOS Lagrange Prize in Continuous
Optimization, the 2014 Jamon Lecture Prize, and the 2015 William O. Baker
Award for Initiatives in Research.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Ml-stat-talks