Ethan Tseng will present his FPO "Neural Proxy Functions for Computational Cameras and Displays" on Friday, May 2, 2025 at 2:30 PM in CS 105 and Zoom. Location: Zoom link: https://princeton.zoom.us/j/97976329281 The members of Ethan’s committee are as follows: Examiners: Felix Heide (Adviser), Adam Finkelstein, Jia Deng Readers: Szymon Rusinkiewicz, Tian-Ming Fu A copy of his thesis is available upon request. Please email gradinfo@cs.princeton.edu if you would like a copy of the thesis. Everyone is invited to attend his talk. Abstract follows below: Cameras and displays serve as portals between the physical world and the digital realm. We use cameras to take pictures of our environment and we use computer vision algorithms to analyze these digital images. These images and their associated analyses can then be shared with others through programmable displays ranging from smartphone screens to mixed reality headsets. However, despite their importance as gateways to the physical world, these hardware devices are still usually developed in isolation from the downstream software. We would like to be able to “learn” new cameras and displays in a manner similar to how modern machine learning models learn new functions by being trained on large volumes of data. Today’s machine learning models are typically trained using some form of gradient descent and the parameters of the model are progressively updated over many iterations. Extending the same design strategy to cameras and displays is challenging because gradients may not be available (e.g., proprietary black-box systems) or the gradients may be too computationally expensive to acquire (e.g., full wave physics models). This thesis introduces neural proxy functions to address and overcome these obstacles. Neural proxy functions act as differentiable approximations for systems that do not readily admit gradients. The gradients provided by neural proxy functions allow for the co-evolution of hardware and software, extending the reach of machine learning beyond the digital world. We demonstrate the utility of neural proxy functions through three research contributions: (1) Hyperparameter optimization of image signal processors that connect high-level visual machine learning (e.g., object detection) to low-level irradiance measurements recorded by a camera sensor. (2) Fast differentiable forward models for optical stacks consisting of both traditional optics and emerging nanophotonic elements. (3) Design of neural algorithms and optics for high-performance holographic displays.