Ari Seff will present his FPO "Learning-Aided Design with Structured Generative Modeling" on June 4, 2021 at 2PM via Zoom.

Zoom link: https://princeton.zoom.us/j/96743727531

The members of Ari’s committee are as follows:

Readers: Ryan Adams(Adviser), Szymon Rusinkiewicz

Examiners: Tom Griffiths, Olga Russakovsky, Abigail Doyle (Chemistry)

A copy of his thesis is available upon request. Please email gradinfo@cs.princeton.edu if you would like a copy of the thesis.

Everyone is invited to attend his talk.

Title: Learning-Aided Design with Structured Generative Modeling

Abstract follows below:

Humans exhibit a remarkable ability to design new tools and systems for solving problems. At the heart of this capability lies a rich suite of evolved pattern recognition modules, enabling the reuse and modification of previously existing solutions. Machine learning offers the intriguing possibility of automatically capturing the patterns that arise in various design domains, in particular via generative modeling. By leveraging the general tool of distribution approximation over data, we may hope to computationally encode common design patterns, leading to applications that enable more efficient workflows in engineering, design, and even scientific discovery.

But this promising route faces challenges. Integral to the above fields are discrete domains, such as molecules and graphs. In contrast to the Euclidean spaces where generative modeling has recently thrived (e.g., images and audio), discrete domains often exhibit irregular structure, strict validity constraints, and non-canonical representations, presenting unique challenges to modeling. In addition, machine learning as applied to engineering still lacks maturity, and as a result there is a need for curated datasets, benchmarks, and shared pipelines.

First, we discuss a novel generative model for discrete domains, known as reversible inductive construction. Building off of generative interpretations of denoising autoencoders, the model employs a Markov chain where transitions are restricted to a set of local operations that preserve validity. This approach constrains the generative model to only produce valid objects, requires the learner to only discover local modifications to the objects, and avoids marginalization over an unknown and potentially large space of construction histories. Next, we present SketchGraphs, a dataset and processing pipeline that contains 15 million real-world CAD sketches paired with their ground truth geometric constraint graphs. Unlike previously available CAD data, explicit supervision is provided regarding the designer-imposed geometric relationships between primitives, making the typically latent construction operations directly accessible. We demonstrate and establish benchmarks for two use cases of the dataset: generative modeling of sketches and conditional generation of likely constraints given unconstrained geometry. Lastly, we propose a method for CAD sketch inference from images, leveraging a conditional generative model that consumes a raster image of a hand-drawn sketch infers the depicted primitives and constraints. Evaluation of this model on SketchGraphs demonstrates its potential to enable new design workflows.