Yiming Zuo will present his General Exam "Efficient View Synthesis with Point-based Representation" on Wednesday, April 26, 2023 at 3:00 PM in CS 302. Committee Members: Jia Deng (advisor), Szymon Rusinkiewicz, Felix Heide Abstract: We address the task of view synthesis, generating novel views of a scene given a set of images as input. In many recent works such as NeRF, the scene geometry is parameterized using neural implicit representations (i.e., MLPs). Implicit neural representations have achieved impressive visual quality but have drawbacks in computational efficiency. NeRF needs to evaluate the neural network at hundreds of 3D points along the ray, which is wasteful because most of the 3D spaces are unoccupied. NeRF’s implicit representation also makes it inflexible for scene editing operations such as deformation, which is important for downstream applications including augmented reality and video games. An intriguing question is whether we can achieve state-of-the-art visual quality by using explicit representations such as point clouds. The basic framework of point-based neural rendering is to represent the scene as a featurized point cloud. Although this framework has been studied in several recent works (Aliev et al., 2020; Wiles et al., 2020; Lassner & Zollhofer, 2021), the overall rendering quality still lags behind NeRF. Our approach adopts this basic framework but introduces a novel technique we call “Sculpted Neural Points (SNP)”, which significantly improves the robustness to the errors and holes in the reconstructed point cloud. We further propose a few novel designs in the point-based rendering pipeline that boost the performance. Compared to previous works that use point cloud-based representation, ours is the first model that achieves better rendering quality than NeRF, while being 100x faster in rendering, and reducing the training time by 66%. Finally, we show that our model allows fine-grained scene editing in a user-friendly way. Reading List: [ https://docs.google.com/document/d/15U9PNheu3dgfYe-_YZVA5vkLfVsYAwzGz2GDMDlf... | https://docs.google.com/document/d/15U9PNheu3dgfYe-_YZVA5vkLfVsYAwzGz2GDMDlf... ] Everyone is invited to attend the talk, and those faculty wishing to remain for the oral exam following are welcome to do so. Louis Riehl Graduate Administrator Computer Science Department, CS213 Princeton University (609) 258-8014