[talks] C Toler-Franklin preFPO

[Melissa Lawson]

Corey Toler-Franklin will present her preFPO on Wednesday May 26 at 2:30PM in Room 402.
The members of her committee are:  Szymon Rusinkiewicz, advisor; Tom Funkhouser and 
Holly Rushmeier (Yale), readers; Adam Finkelstein and David Blei, nonreaders.  Everyone is 
invited to attend her talk.  Her abstract follow below.
---------------------------------------------

Title: Matching, Visualizing and Archiving Cultural Heritage Artifacts Using Multi-Channel Images
Abstract

Recent advancements in low-cost acquisition technologies have made it more practical to acquire real-world datasets on a large scale. This has lead to a number of computer-based solutions for reassembling, archiving and visualizing cultural heritage artifacts.  In this thesis, we combine aspects of these technologies in novel ways and introduce algorithms to improve upon their overall efficiency and robustness. First, we introduce a 2D acquisition system to address the challenge of acquiring higher resolution color and normal maps than those available with 3D scanning devices. Next, we incorporate our normal maps into a novel multi-cue matching system for reassembling small fragments of artifacts.  We then present a non-photorealistic rendering pipeline for illustrating geometrically complex objects using images with multiple channels of information. Finally, we propose a data mining approach for visualizing historic artifacts from digital image collections. 

State-of-the-art 3D acquisition systems capture 3D geometry at archeological sites using affordable, off-the shelf scanners. Although multiple scans at varying viewpoints are required to assemble a complete model, robust registration and alignment algorithms, as well as new work-flow methodologies, significantly reduce the post-processing time.  However, the color and normal maps obtained from these systems lack the subtle sub-millimeter details necessary for careful analysis, and high fidelity documentation. We introduce an algorithm that generates higher resolution normal maps and diffuse reflectance (true color texture), while minimizing acquisition time. Using shape from shading, we compute our normal maps from high resolution color scans of the object taken at four orientations on a 2D flat-bed scanner.  A key contribution of our work is a novel calibration process to measure the observed brightness as a function of the surface normal.  This calibration is important because the scanner's light is linear (rather than a point), and we cannot solve for the surface normal using the traditional formulation of the Lambertian lighting law. High resolution digital SLR cameras provide alternative solutions when objects are too large or fragile to place on a scanner. However, they require more control over the ambient light in the environment and additional manual effort to continually re-position a hand-held flash. They lack the high resolutions we obtain from the scanner.

Several projects have been explored to leverage these newly acquired datasets for computer-assisted reassembly, and have proven successful in some domains. However, current matching algorithms do not perform well when artifacts have deteriorated over many years. One limitation is their reliance on previous acquisition methods that do not capture fine surface details that are important matching cues when features such as color, 2D contours or 3D geometry are no longer reliable.  We introduce a set of feature descriptors that are based not only on color and shape, but also normal maps with a high data quality.  Rather than rely exclusively on one form of data, we use machine-learning techniques to combine descriptors in a multi-cue matching framework. We have tested our system on three datasets of fresco fragments:  Theran Frescos from the site of Akrotiri, Greece; Roman frescoes from Kerkrade in the Netherlands; and a Synthetic fresco created by conservators in a style similar to Akrotiri frescos. We demonstrate that multi-cue matching using different subsets of features leads to different tradeoffs between efficiency and effectiveness. We observe that individual feature performance varies from dataset to dataset and discuss the implications of feature importance for matching in this domain. Our results show good retrieval performance, significantly improving upon the match prediction rate of state-of the-art 3D matching algorithms.

The Illustrative depictions found in biology or medical text-books are one possible method of archiving and distributing historic information.  Using a datatype that stores both color and normals, RGBN images, we develop 2D analogs to 3D NPR rendering equations. Our approach extends signal processing tools such as scale-space analysis and segmentation for this new data type. We investigate stylize depiction techniques such as toon shading, line drawing and exaggerated shading. By incorporating some 3D information, we reveal fine details while maintaining the simplicity of a 2D implementation. Our results achieve levels of detail that are impractical to create with more conventional methods like manual 3D modeling or 3D scanning.

Museums use digital cameras to archive their collections. When combined with the internet, digital collections create another powerful tool for disseminating information about history and culture. However, most systems permit only simplistic navigation through a fixed set of color or grayscale images.  We propose a visualization technique that examines many images in the frequency domain to reveal information not easily apparent in conventional images, such as embedded watermarks, faded pigments or patterns in materials. In addition to RGBN images, our multi-channel image stacks combine many types of images including images with varied lighting and multiple wave lengths. We present algorithms to efficiently navigate a large search space, extract salient information across multiple images, and generate effective visualizations without manual adjustment of a large set of parameters.  Unlike current techniques that are restricted to a single input image, we are able to reduce data loss and limit artifacts. We demonstrate these multi-channel image stacks using digital image collections provided by several institutions including the Smithsonian National Museum of the American Indian, the Library of Congress, and Cultural Heritage Imaging.