Changyan Wang will present his MSE Talk "Data Echoing with Adagrad' on Thursday, April 22, 2021 at 1:30PM via Zoom. Zoom Link: https://princeton.zoom.us/j/3023000473 Committee: Yoram Singer (adviser), Mark Braverman (reader) All are welcome to attend. Abstract: In modern neural networks, hardware accelerators such as GPU's have been highly successful in speeding up training. However, GPU's are not suitable for accelerating all parts of the training pipelines: operations such as data I/O and preprocessing are still run on the CPU. As accelerators improve, these non-accelerated operations are becoming bottlenecks in some workloads. Data echoing is an approach to reduce this load on the CPU, by having the GPU run multiple iterations on each batch provided on the CPU. Previous work with data echoing focused on standard SGD or proximal SGD: in this project, we experimented with and analyzed data echoing with a proximal version of Adagrad.