[talks] ​​​​​Riley Simmons-Edler will present his general exam on Friday, May 12, 2017 in CS 401 at 10am.

Nicki Gotsis ngotsis at CS.Princeton.EDU
Mon May 8 10:42:14 EDT 2017

Riley Simmons-Edler will present his general exam on Friday, May 12, 2017 in CS 401 at 10am.

The members of his committee are: Tom Funkhouser, Sebastian Seung, and Szymon Rusinkiewicz.

Everyone is invited to attend his talk, and those faculty wishing to remain for the oral exam following are welcome to do so. His abstract and reading list follow below.

Title: "Self-Supervised and Adversarial Image Intrinsic Decomposition"


Image decomposition into intrinsic components, such as albedo and 
lighting, is a highly underconstrained problem. For any given image, 
there are infinitely many possible predictions that satisfy the 
relationship albedo * lighting = color. However, as humans we can define 
a distribution of "plausible" predictions for a given image using a 
combination of our internal model of what a valid intrinsic looks like 
and the known content of the original image. It is therefore reasonable 
to think it is viable to train a deep neural network to do this task by 
constraining the output in a similar fashion. Here, I present my work on 
using self-supervision and adversarial neural networks to learn an 
albedo and lighting decomposition for real photographs, where no ground 
truth data exists nor can exist. By using an adversarial network trained 
on a distribution of synthetic albedo and lighting, I project a given 
color image into the space of possible intrinsics, while using 
self-supervision via a reconstruction loss from the intrinsics to the 
input image to constrain the predictions to reproduce the input image. 
This combination of loss terms makes it possible to learn an accurate 
transformation from color input to intrinsic factors without any paired 
training data, allowing for training directly on real images and thus 
improved performance compared to previous approaches which can only use 
synthetic training data.

Textbook: "Deep Learning," Goodfellow, Bengio, and Courville, 2016


Directly Related to my project(methods and previous application works):

"Direct Intrinsics: Learning Albedo-Shading Decomposition by 
Convolutional Regression," Narihira, Maire, and Yu, 2015

"DARN: a Deep Adversarial Residual Network for Intrinsic Image 
Decomposition," Lettry, Vanhoey, and  Van Gool, 2016

"Wasserstein GAN," Arjovsky, Chintala, and Bottou, 2017

"Multi-Scale Context Aggregation by Dilated Convolutions," Yu and 
Koltun, 2016

Other Similar Influential Works(in image to image translation):

"Learning from Simulated and Unsupervised Images through Adversarial 
Training," Shrivastava et al, 2016

"Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial 
Networks," Zhu, Park, Isola, and Efros, 2017

Classic Papers:

"Generative Adversarial Nets," Goodfellow et al, 2014

"Imagenet Classification with Deep Convolutional Neural Networks," 
Krizhevsky, Sutskever, and Hinton, 2012

"Backpropagation Applied to Handwritten Zip Code Recognition," LeCun et 
al, 1989 (the original convolutional neural networks paper)

More information about the talks mailing list