April 2024 - talks - lists.cs.princeton.edu

Fangyin Wei will present her FPO "Learning to Edit 3D Objects and Scenes" on Monday, April 22, 2024 at 1:30 PM in CS 302
by gradinfo＠cs.princeton.edu 18 Apr '24

18 Apr '24

Fangyin Wei will present her FPO "Learning to Edit 3D Objects and Scenes" on Monday, April 22, 2024 at 1:30 PM in CS 302 Location: CS 302 The members of Fangyin's committee are as follows: Examiners: Szymon Rusinkiewicz (Adviser), Thomas Funkhouser (Adviser), Jia Deng Readers: Felix Heide, Olga Russakovsky A copy of her thesis is available upon request. Please email gradinfo(a)cs.princeton.edu if you would like a copy of the thesis. Everyone is invited to attend her talk. Abstract follows below: 3D editing plays a key role in many fields ranging from AR/VR, industrial and art design, to robotics. However, existing 3D editing tools either (i) demand labor-intensive manual efforts and struggle to scale to many examples, or (ii) use optimization and machine learning but produce unsatisfactory results (e.g., losing details, supporting only coarse editing, etc.). These shortcomings often arise from editing in geometric space rather than structure-aware semantic space, where the latter is the key to automatic 3D editing at scale. While learning a structure-aware space will result in significantly improved efficiency and accuracy, labeled datasets to train 3D editing models don't exist. In this dissertation, we present novel approaches for learning to edit 3D objects and scenes in structure-aware semantic space with noisy or no supervision. We first address how to extract the underlying structure to edit 3D objects, with a focus on editing two critical properties: semantic shape parts and articulations. Our semantic editing method enables specific edits to an object's semantic parameters (e.g., the pose of a person's arm or the length of an airplane's wing), leading to better preservation of input details and improved accuracy compared to previous work. Next, we introduce a 3D annotation-free method that learns to model geometry, articulation, and appearance of articulated objects from color images. The model works on an entire category (as opposed to typical NeRF extensions that only overfit on a single scene) and enables various applications such as few-shot reconstruction and static object animation. It also generalizes to real-world captures. Then, we tackle how to extract structure for scene editing. We present an automatic system that removes clutter (frequently moving objects such as clothes or chairs) from 3D scenes and inpaints the resulting holes with coherent geometry and texture. We address challenges including the lack of well-defined clutter annotations, entangled semantics and geometry, and multi-view inconsistency. In summary, this dissertation demonstrates techniques to exploit the underlying structure of 3D data for editing. Our work opens up new research directions such as leveraging structures from other modalities (e.g., text, images) to empower 3D editing models with stronger semantic understanding.

1 0

Jack Defay will present his MSE talk “Synthetic Data Generation for 6DOF Object Rearrangement” Monday, April 22 at 3:00 PM in CS 105.
by CS Grad Department 17 Apr '24

17 Apr '24

Jack Defay will present his MSE talk “Synthetic Data Generation for 6DOF Object Rearrangement” Monday, April 22 at 3:00 PM in CS 105. Advisor: Jia Deng; Reader: Olga Russakovsky Abstract: State of the art methods for object rearrangement tasks will need better labeled, and fuller distributions of, data to train on. Many difficult object classes, like shiny or transparent objects, are important for deployment scenarios like a kitchen. Furthermore, challenges to labeling, like occlusions, are essential for a model to be able to handle in more complex or cluttered environments. We approach these challenges in a few ways: improved data collection techniques, semi-automatic data labeling pipelines, and synthetic data generation. We believe that synthetic data will continue to close the sim to real gap, and due to its better coverage of the underlying distribution and perfect labels, will become a more cost effective solution to training complex vision problems like 6DOF object rearrangement. This paper focuses on techniques for synthetic data generation for the object rearrangement task as well as methods to improve the real world data collection pipeline for benchmarking the synthetic datasets. CS Grad Calendar: [ https://calendar.google.com/calendar/event?action=TEMPLATE&tmeid=MW1yc2FwZH… | https://calendar.google.com/calendar/event?action=TEMPLATE&tmeid=MW1yc2FwZH… ]

1 0

Gillian Chu will present her General Exam "Probabilistic Models for Evolutionary Tree Reconstruction" on Monday, April 22, 2024 at 3:00 PM in CS 201.
by CS Grad Department 17 Apr '24

17 Apr '24

Gillian Chu will present her General Exam "Probabilistic Models for Evolutionary Tree Reconstruction" on Monday, April 22, 2024 at 3:00 PM in Friend 202. Committee Members: Ben Raphael (advisor), Mona Singh, Yuri Pritykin Abstract: Lineage tracing is the tracking of cell divisions from a single cell to a resulting multicellular organism. Recent experimental work in developmental biology has resulted in a new generation of dynamic lineage tracing technologies which edit a cell to accumulate heritable CRISPR edits over generations. The subsequent computational problem of reconstructing the cell lineage tree from such dynamic lineage tracing data has also been the subject of much recent work. However, several unaddressed computational challenges remain. For instance, one key unaddressed challenge is how to handle the high rates (up to 40%) of heterogeneous (heritable silencing and stochastic dropout) missing data. In this work, we address these challenges by presenting a novel probabilistic approach: (a) we propose a new model of dynamic lineage traced sequence evolution with probabilistic mixed-type missing (PMM) data, and (b) present Lineage Analysis via Maximum Likelihood (LAML), which infers the (i) cell phylogeny with (ii) time-scaled branch lengths and (iii) missing data rates under this model. Our work demonstrates a systematic bias in previous mutation-based approaches to branch length estimation, with important implications for downstream analyses. Crucially, using the LAML time-scaled tree we can place evolutionary events (such as cancer cell migration to and from different anatomical locations) in experimental time. From a lineage-traced mouse model of metastasis, we observe three “epochs” of metastasis with distinct cell migration patterns. Reading List: https://docs.google.com/document/d/1Pjmf-OA0QhbapMoPxzRP62_ukFQ6GfCaKeIOjBF… Everyone is invited to attend the talk, and those faculty wishing to remain for the oral exam following are welcome to do so.

1 0

Andrea Wynn will present her MSE talk "Learning Human-Like Representations to Enable Learning Human Values" today (April 16) at 10:00 AM in CS 301.
by gradinfo＠cs.princeton.edu 17 Apr '24

17 Apr '24

Andrea Wynn will present her MSE talk "Learning Human-Like Representations to Enable Learning Human Values" today (April 16) at 10:00 AM in CS 301. Advisor: Prof. Danqi Chen; Reader: Karthik Narasimhan CS Grad Calendar: https://calendar.google.com/calendar/event?action=TEMPLATE <https://calendar.google.com/calendar/event?action=TEMPLATE&tmeid=MDQybnM4NW g0NXJ2bW50cjUyNWRsZ3E4OGQgYWNnMDc5YmxzbzRtczNza2tmZThwa2lyb2dAZw&tmsrc=acg07 9blso4ms3skkfe8pkirog%40group.calendar.google.com> &tmeid=MDQybnM4NWg0NXJ2bW50cjUyNWRsZ3E4OGQgYWNnMDc5YmxzbzRtczNza2tmZThwa2lyb 2dAZw&tmsrc=acg079blso4ms3skkfe8pkirog%40group.calendar.google.com

1 2

Hongyu Wen will present her General Exam "A Real-World Benchmark for Non-Lambertian Optical Flow" on Friday, April 19, 2024 at 3:00 PM in Friend 202 .
by CS Grad Department 16 Apr '24

16 Apr '24

Hongyu Wen will present her General Exam " A Real-World Benchmark for Non-Lambertian Optical Flow " on Friday, April 19, 2024 at 3:00 PM in Friend 202 . Committee Members: Jia Deng (advisor), Felix Heide , Ellen Zhong Abstract: Achieving 3D understanding of non-Lambertian objects is an important task with many useful applications, but most existing algorithms struggle to deal with such objects. One major obstacle towards progress in this field is the lack of holistic non-Lambertian benchmarks---most benchmarks have low scene and object diversity, and none provide multi-layer 3D annotations for objects occluded by transparent surfaces. We introduce a real world benchmark, Layered Flow, containing multi-layer ground truth annotation for optical flow of non-Lambertian objects. Compared to previous benchmarks, our benchmark exhibits greater scene and object diversity, with 15k high quality optical flow and stereo pairs taken over 185 indoor and outdoor scenes and 360 unique objects. Using LayeredFlow as evaluation data, we propose a new task called multi-layer optical flow. To provide training data for this task, we introduce a large-scale densely-annotated synthetic dataset containing 60k images within 30 scenes tailored for non-Lambertian objects. Training on our synthetic dataset enable model to predict multi-layer optical flow, while fine-tuning existing optical flow methods on the dataset notably boosts their performance on non-Lambertian objects without compromising the performance on diffuse objects. Reading List: https://docs.google.com/document/d/1BL5xNaZVqjog4MOldJeklfA6po4zJmxDZU2VLuD… Everyone is invited to attend the talk, and those faculty wishing to remain for the oral exam following are welcome to do so.

1 0

Chenghan Zhou will present her MSE talk "Profitable Manipulations of Cryptographic Self-Sortition are Statistically Detectable" on April 25, 2024 in CS 402.
by Gradinfo 16 Apr '24

16 Apr '24

Chenghan Zhou will present her MSE talk "Profitable Manipulations of Cryptographic Self-Sortition are Statistically Detectable" on April 25, 2024 in CS 402. Thesis advisor: Matt Weinberg Thesis reader: Mark Braverman Public seminar time: Apr 25, 2024, 12:30 - 1pm Location: CS402 Thesis Title: Profitable Manipulations of Cryptographic Self-Sortition are Statistically Detectable Abstract: Cryptographic Self-Selection is a common primitive underlying leader-selection for Proof-of-Stake blockchain protocols. The concept was first popularized in Algorand [CM19], who also observed that the protocol might be manipulable. [FHWY22] provide a concrete manipulation that is strictly profitable for a staker of any size (and also prove upper bounds on the gains from manipulation). Separately, [BW23] initiate the study of undetectable profitable manipulations of consensus protocols with a focus on the seminal Selfish Mining strategy [ES14] for Bitcoin’s Proof-of-Work longest-chain protocol. They design a Selfish Mining variant that, for sufficiently large miners, is strictly profitable yet also indistinguishable to an onlooker from routine latency (that is, a sufficiently large profit-maximizing miner could use their strategy to strictly profit over being honest in a way that still appears to the rest of the network as though everyone is honest but experiencing mildly higher latency. This avoids any risk of negatively impacting the value of the underlying cryptocurrency due to attack detection). We investigate the detectability of profitable manipulations of the canonical cryptographic self-selection leader selection protocol introduced in [CM19] and studied in [FHWY22], and establish that every strictly profitable manipulation is statistically detectable. Specifically, we consider an onlooker who sees only the random seed of each round (and does not need to see any other broadcasts by any other players). We show that the distribution of the sequence of random seeds when any player is profitably manipulating the protocol is inconsistent with any distribution that could arise by honest stakers being offline or timing out (for a natural stylized model of honest timeouts).

1 0

Shruthi Santhanam will present her MSE talk "Interpretable 3D Scene Perception, Tracking and Generation" Friday, April 19 at 12:30 PM in Friend 202.
by gradinfo＠cs.princeton.edu 16 Apr '24

16 Apr '24

Shruthi Santhanam will present her MSE talk "Interpretable 3D Scene Perception, Tracking and Generation" Friday, April 19 at 12:30 PM in Friend 202. Advisor: Felix Heide; Reader: Vikram Ramaswamy Abstract: 3D scene understanding is a many-pronged problem that is fundamental to countless real-world applications. These are often deployed in safety-critical situations, meaning that both accuracy and interpretability are crucial. Current methods use non-learned algorithmic approaches, which may be more explainable but often have lower accuracies, or neural networks, which are generally not interpretable. In this thesis, we present two works that tackle different aspects of interpretable 3D scene understanding-perception, generation, and tracking. First, we combine scene perception and generation by reframing 3D object detection as a conditional generative process. Generative methods have shown to be effective learners of empirical priors, and we exploit this property by using a probabilistic diffusion model, conditioned on an input observation, to model each detected object explicitly on a ground plane. Then, we use a diffusion-based neural rendering method to reconstruct the scene, allowing for simultaneous 3D object detection and multi-object scene generation. This closed-loop pipeline gives us interpretability "for free" via the generated ground plane information. Second, we present a new intrinsically interpretable online 3D multi-object tracking method, based on graph attention networks (GAT). Using the tracking-by-detection framework, we use a GAT to perform data association on a track-detection bipartite graph between sequential frames. Furthermore, attention-based scores are used to visualize the GAT's message passing behavior on a model level, as well as explain specific outputs. Along with excellent accuracy across datasets, this method provides much-needed online interpretability to deep learning-based tracking, setting the stage for safe and scalable applications. CS Grad Calendar: https://calendar.google.com/calendar/event?action=TEMPLATE <https://calendar.google.com/calendar/event?action=TEMPLATE&tmeid=MXZyaDA4dW 5rZXFuNDB2NDBpY3NuYjQ3b2sgYWNnMDc5YmxzbzRtczNza2tmZThwa2lyb2dAZw&tmsrc=acg07 9blso4ms3skkfe8pkirog%40group.calendar.google.com> &tmeid=MXZyaDA4dW5rZXFuNDB2NDBpY3NuYjQ3b2sgYWNnMDc5YmxzbzRtczNza2tmZThwa2lyb 2dAZw&tmsrc=acg079blso4ms3skkfe8pkirog%40group.calendar.google.com

1 0

Princeton Robotics Seminar: Jeannette Bohg (Stanford), Friday @ 11 AM
by Emily C. Lawrence 16 Apr '24

16 Apr '24

Join us this Friday (4/19) for the Robotics Seminar! Speaker: [ https://web.stanford.edu/~bohg/ | Jeannette Bohg ] , Stanford Date: Friday, April 19, 2024 Time: 11am ET Location: Computer Science 105 Event page: [ https://robo.princeton.edu/seminar | https://robo.princeton.edu/seminar ] PhD students and postdocs can [ https://docs.google.com/forms/d/e/1FAIpQLSczj2FTIvPhV_6f0jQpM5XZTJ_c-dp0DfH… | signup to join Jeannette for lunch ] . There will also be a Robotics Social at 4:00 PM in the F-Wing Cafe Area - all are welcome to attend! Title: Enabling Cross-Embodiment Learning Abstract: In this talk, I will investigate the problem of learning manipulation skills across a diverse set of robotic embodiments. Conventionally, manipulation skills are learned separately for every task, environment and robot. However, in domains like Computer Vision and Natural Language Processing we have seen that one of the main contributing factors to generalisable models is large amounts of diverse data. If we were able to have one robot learn a new task even from data recorded with a different robot, then we could already scale up training data to a much larger degree for each robot embodiment. In this talk, I will present a new, large-scale datasets that was put together across multiple industry and academic research labs to make it possible to explore the possibility of cross-embodiment learning in the context of robotic manipulation, alongside experimental results that provide an example of effective cross-robot policies. Given this dataset, I will also present multiple alternative ways to learn cross-embodiment policies. These example approaches will include (1) UniGrasp - a model that allows to synthesise grasps with new hands, (2) XIRL - an approach to automatically discover and learn vision-based reward functions from cross-embodiment demonstration videos and (3) Equivact - an approach that leverages equivariance to learn sensorimotor policies that generalise to scenarios that are traditionally out-of-distribution. Bio: Jeannette Bohg is an Assistant Professor of Computer Science at Stanford University. She was a group leader at the Autonomous Motion Department (AMD) of the MPI for Intelligent Systems until September 2017. Before joining AMD in January 2012, Jeannette Bohg was a PhD student at the Division of Robotics, Perception and Learning (RPL) at KTH in Stockholm. In her thesis, she proposed novel methods towards multi-modal scene understanding for robotic grasping. She also studied at Chalmers in Gothenburg and at the Technical University in Dresden where she received her Master in Art and Technology and her Diploma in Computer Science, respectively. Her research focuses on perception and learning for autonomous robotic manipulation and grasping. She is specifically interested in developing methods that are goal-directed, real-time and multi-modal such that they can provide meaningful feedback for execution and learning. Jeannette Bohg has received several Early Career and Best Paper awards, most notably the 2019 IEEE Robotics and Automation Society Early Career Award and the 2020 Robotics: Science and Systems Early Career Award.

1 0

ECE Seminar: Yanjing Li, Monday April 22 at 4:30pm
by Emily C. Lawrence 16 Apr '24

16 Apr '24

Speaker: Yanjing Li, University of Chicago Date: Monday, April 22 Time: 4:30pm Location: B205, EQuad Host: Prof. David Wentzlaff Event link: [ https://ece.princeton.edu/events/how-does-one-bit-flip-corrupt-entire-deep-… | https://ece.princeton.edu/events/how-does-one-bit-flip-corrupt-entire-deep-… ] Title: How does one bit-flip corrupt an entire deep neural network, and what to do about it Abstract: Deep neural networks are increasingly susceptible to hardware failures. The impact of hardware failures on these workloads is severe – even a single bit-flip can corrupt an entire network during both training and inference. The urgency of tackling this challenge, known as the Silent Data Corruption challenge in a broader context, has been widely raised by both the industry and academia. In this talk, I will first present the first in-depth resilience study targeting DNN workloads and hardware failures that occur in the logic portion of deep learning accelerator systems, including a comprehensive characterization of hardware failure effects, along with the fundamental understanding of how hardware failures propagate in hardware devices and interact with the workloads. Next, based on the insights obtained from our study, I will present ultra lightweight yet highly effective techniques to mitigate hardware failures in deep learning accelerator systems. Bio: Yanjing Li is an Assistant Professor in the Department of Computer Science at the University of Chicago. Prior to joining the university, she was a senior research scientist at Intel Labs. Professor Li received her Ph.D. in Electrical Engineering from Stanford University, an M.S. in Mathematical Sciences (with honors) and a B.S. in Electrical and Computer Engineering (with a double major in Computer Science) from Carnegie Mellon University. Professor Li has received various awards, including the NSF CAREER Award, DAC under-40 innovators award, Google research scholar award, Intel Labs Gordy academy award (highest honor in Intel Labs) and several other Intel recognition awards, outstanding dissertation award (European Design and Automation Association), and multiple best paper awards (for example, IEEE VLSI Test Symposium and IEEE International Test Conference). Professor Li’s seminal work on in-field self-test and diagnostics has been adopted by various companies including Intel, Nvdia, Synopsys, TI, and MentorGraphics, and more.

1 0

CSML Seminar: Machine Learning for Structural Biology - TOMORROW at 12:30pm
by Emily C. Lawrence 15 Apr '24

15 Apr '24

Speaker: Ellen Zhong, Assistant Professor, Princeton University, Computer Science Title: Machine Learning for Structural Biology Day: April 16, 2024 Place: CSML, Bendheim House, 26 Prospect Ave, Classroom Room 103 Time: 12:30-1:30 PM Lunch available from 12:00 Host: Tom Griffiths, Director CSML Abstract: Structural biology has been transformed by breakthroughs in deep learning methods for protein structure prediction. In parallel, advances in cryo-electron microscopy (cryo-EM) have produced new opportunities to study the structure and dynamics of proteins and other biomolecular complexes through imaging. In this seminar, I will overview cryoDRGN and related methods that leverage the representation power of deep neural networks for 3D reconstruction of protein structures from cryo-EM images. Extended to real datasets and released as open-source tools, these methods have been used to discover new protein structures and visualize continuous trajectories of protein motion. I will discuss extensions of the method for scalable and robust reconstruction, analysis of the learned generative model, and visualization of dynamic macromolecular machines in situ. Finally, I will discuss how recent advances in machine learning for protein structure prediction (e.g. AlphaFold) can complement methods for cryo-EM structure determination and what key algorithmic challenges remain to realize the next era of structural biology. Bio: Ellen Zhong is an Assistant Professor of Computer Science at Princeton University. She is interested in problems at the intersection of AI and biology. Her research develops machine learning methods for computational and structural biology problems with a focus on protein structure determination with cryo-electron microscopy (cryo-EM). She obtained her Ph.D. from MIT in 2022, advised by Bonnie Berger and Joey Davis, where she developed deep learning algorithms for 3D reconstruction of dynamic protein structures from cryo-EM images. She has interned at DeepMind with John Jumper and the AlphaFold team and previously worked on molecular dynamics algorithms and infrastructure for drug discovery at D. E. Shaw Research. She obtained her B.S. from the University of Virginia where she worked with Michael Shirts on computational methods for studying protein folding. For more information about her research and group, please visit her group website: https://ezlab.princeton.edu/

1 0