- talks - lists.cs.princeton.edu

Jack Defay will present his MSE talk “Synthetic Data Generation for 6DOF Object Rearrangement” Monday, April 22 at 3:00 PM in CS 105.
by CS Grad Department 17 Apr '24

17 Apr '24

Jack Defay will present his MSE talk “Synthetic Data Generation for 6DOF Object Rearrangement” Monday, April 22 at 3:00 PM in CS 105. Advisor: Jia Deng; Reader: Olga Russakovsky Abstract: State of the art methods for object rearrangement tasks will need better labeled, and fuller distributions of, data to train on. Many difficult object classes, like shiny or transparent objects, are important for deployment scenarios like a kitchen. Furthermore, challenges to labeling, like occlusions, are essential for a model to be able to handle in more complex or cluttered environments. We approach these challenges in a few ways: improved data collection techniques, semi-automatic data labeling pipelines, and synthetic data generation. We believe that synthetic data will continue to close the sim to real gap, and due to its better coverage of the underlying distribution and perfect labels, will become a more cost effective solution to training complex vision problems like 6DOF object rearrangement. This paper focuses on techniques for synthetic data generation for the object rearrangement task as well as methods to improve the real world data collection pipeline for benchmarking the synthetic datasets. CS Grad Calendar: [ https://calendar.google.com/calendar/event?action=TEMPLATE&tmeid=MW1yc2FwZH… | https://calendar.google.com/calendar/event?action=TEMPLATE&tmeid=MW1yc2FwZH… ]

1 0

Andrea Wynn will present her MSE talk "Learning Human-Like Representations to Enable Learning Human Values" today (April 16) at 10:00 AM in CS 301.
by gradinfo＠cs.princeton.edu 17 Apr '24

17 Apr '24

Andrea Wynn will present her MSE talk "Learning Human-Like Representations to Enable Learning Human Values" today (April 16) at 10:00 AM in CS 301. Advisor: Prof. Danqi Chen; Reader: Karthik Narasimhan CS Grad Calendar: https://calendar.google.com/calendar/event?action=TEMPLATE <https://calendar.google.com/calendar/event?action=TEMPLATE&tmeid=MDQybnM4NW g0NXJ2bW50cjUyNWRsZ3E4OGQgYWNnMDc5YmxzbzRtczNza2tmZThwa2lyb2dAZw&tmsrc=acg07 9blso4ms3skkfe8pkirog%40group.calendar.google.com> &tmeid=MDQybnM4NWg0NXJ2bW50cjUyNWRsZ3E4OGQgYWNnMDc5YmxzbzRtczNza2tmZThwa2lyb 2dAZw&tmsrc=acg079blso4ms3skkfe8pkirog%40group.calendar.google.com

1 2

Hongyu Wen will present her General Exam "A Real-World Benchmark for Non-Lambertian Optical Flow" on Friday, April 19, 2024 at 3:00 PM in Friend 202 .
by CS Grad Department 16 Apr '24

16 Apr '24

Hongyu Wen will present her General Exam " A Real-World Benchmark for Non-Lambertian Optical Flow " on Friday, April 19, 2024 at 3:00 PM in Friend 202 . Committee Members: Jia Deng (advisor), Felix Heide , Ellen Zhong Abstract: Achieving 3D understanding of non-Lambertian objects is an important task with many useful applications, but most existing algorithms struggle to deal with such objects. One major obstacle towards progress in this field is the lack of holistic non-Lambertian benchmarks---most benchmarks have low scene and object diversity, and none provide multi-layer 3D annotations for objects occluded by transparent surfaces. We introduce a real world benchmark, Layered Flow, containing multi-layer ground truth annotation for optical flow of non-Lambertian objects. Compared to previous benchmarks, our benchmark exhibits greater scene and object diversity, with 15k high quality optical flow and stereo pairs taken over 185 indoor and outdoor scenes and 360 unique objects. Using LayeredFlow as evaluation data, we propose a new task called multi-layer optical flow. To provide training data for this task, we introduce a large-scale densely-annotated synthetic dataset containing 60k images within 30 scenes tailored for non-Lambertian objects. Training on our synthetic dataset enable model to predict multi-layer optical flow, while fine-tuning existing optical flow methods on the dataset notably boosts their performance on non-Lambertian objects without compromising the performance on diffuse objects. Reading List: https://docs.google.com/document/d/1BL5xNaZVqjog4MOldJeklfA6po4zJmxDZU2VLuD… Everyone is invited to attend the talk, and those faculty wishing to remain for the oral exam following are welcome to do so.

1 0

Chenghan Zhou will present her MSE talk "Profitable Manipulations of Cryptographic Self-Sortition are Statistically Detectable" on April 25, 2024 in CS 402.
by Gradinfo 16 Apr '24

16 Apr '24

Chenghan Zhou will present her MSE talk "Profitable Manipulations of Cryptographic Self-Sortition are Statistically Detectable" on April 25, 2024 in CS 402. Thesis advisor: Matt Weinberg Thesis reader: Mark Braverman Public seminar time: Apr 25, 2024, 12:30 - 1pm Location: CS402 Thesis Title: Profitable Manipulations of Cryptographic Self-Sortition are Statistically Detectable Abstract: Cryptographic Self-Selection is a common primitive underlying leader-selection for Proof-of-Stake blockchain protocols. The concept was first popularized in Algorand [CM19], who also observed that the protocol might be manipulable. [FHWY22] provide a concrete manipulation that is strictly profitable for a staker of any size (and also prove upper bounds on the gains from manipulation). Separately, [BW23] initiate the study of undetectable profitable manipulations of consensus protocols with a focus on the seminal Selfish Mining strategy [ES14] for Bitcoin’s Proof-of-Work longest-chain protocol. They design a Selfish Mining variant that, for sufficiently large miners, is strictly profitable yet also indistinguishable to an onlooker from routine latency (that is, a sufficiently large profit-maximizing miner could use their strategy to strictly profit over being honest in a way that still appears to the rest of the network as though everyone is honest but experiencing mildly higher latency. This avoids any risk of negatively impacting the value of the underlying cryptocurrency due to attack detection). We investigate the detectability of profitable manipulations of the canonical cryptographic self-selection leader selection protocol introduced in [CM19] and studied in [FHWY22], and establish that every strictly profitable manipulation is statistically detectable. Specifically, we consider an onlooker who sees only the random seed of each round (and does not need to see any other broadcasts by any other players). We show that the distribution of the sequence of random seeds when any player is profitably manipulating the protocol is inconsistent with any distribution that could arise by honest stakers being offline or timing out (for a natural stylized model of honest timeouts).

1 0

Shruthi Santhanam will present her MSE talk "Interpretable 3D Scene Perception, Tracking and Generation" Friday, April 19 at 12:30 PM in Friend 202.
by gradinfo＠cs.princeton.edu 16 Apr '24

16 Apr '24

Shruthi Santhanam will present her MSE talk "Interpretable 3D Scene Perception, Tracking and Generation" Friday, April 19 at 12:30 PM in Friend 202. Advisor: Felix Heide; Reader: Vikram Ramaswamy Abstract: 3D scene understanding is a many-pronged problem that is fundamental to countless real-world applications. These are often deployed in safety-critical situations, meaning that both accuracy and interpretability are crucial. Current methods use non-learned algorithmic approaches, which may be more explainable but often have lower accuracies, or neural networks, which are generally not interpretable. In this thesis, we present two works that tackle different aspects of interpretable 3D scene understanding-perception, generation, and tracking. First, we combine scene perception and generation by reframing 3D object detection as a conditional generative process. Generative methods have shown to be effective learners of empirical priors, and we exploit this property by using a probabilistic diffusion model, conditioned on an input observation, to model each detected object explicitly on a ground plane. Then, we use a diffusion-based neural rendering method to reconstruct the scene, allowing for simultaneous 3D object detection and multi-object scene generation. This closed-loop pipeline gives us interpretability "for free" via the generated ground plane information. Second, we present a new intrinsically interpretable online 3D multi-object tracking method, based on graph attention networks (GAT). Using the tracking-by-detection framework, we use a GAT to perform data association on a track-detection bipartite graph between sequential frames. Furthermore, attention-based scores are used to visualize the GAT's message passing behavior on a model level, as well as explain specific outputs. Along with excellent accuracy across datasets, this method provides much-needed online interpretability to deep learning-based tracking, setting the stage for safe and scalable applications. CS Grad Calendar: https://calendar.google.com/calendar/event?action=TEMPLATE <https://calendar.google.com/calendar/event?action=TEMPLATE&tmeid=MXZyaDA4dW 5rZXFuNDB2NDBpY3NuYjQ3b2sgYWNnMDc5YmxzbzRtczNza2tmZThwa2lyb2dAZw&tmsrc=acg07 9blso4ms3skkfe8pkirog%40group.calendar.google.com> &tmeid=MXZyaDA4dW5rZXFuNDB2NDBpY3NuYjQ3b2sgYWNnMDc5YmxzbzRtczNza2tmZThwa2lyb 2dAZw&tmsrc=acg079blso4ms3skkfe8pkirog%40group.calendar.google.com

1 0

ECE Seminar: Yanjing Li, Monday April 22 at 4:30pm
by Emily C. Lawrence 16 Apr '24

16 Apr '24

Speaker: Yanjing Li, University of Chicago Date: Monday, April 22 Time: 4:30pm Location: B205, EQuad Host: Prof. David Wentzlaff Event link: [ https://ece.princeton.edu/events/how-does-one-bit-flip-corrupt-entire-deep-… | https://ece.princeton.edu/events/how-does-one-bit-flip-corrupt-entire-deep-… ] Title: How does one bit-flip corrupt an entire deep neural network, and what to do about it Abstract: Deep neural networks are increasingly susceptible to hardware failures. The impact of hardware failures on these workloads is severe – even a single bit-flip can corrupt an entire network during both training and inference. The urgency of tackling this challenge, known as the Silent Data Corruption challenge in a broader context, has been widely raised by both the industry and academia. In this talk, I will first present the first in-depth resilience study targeting DNN workloads and hardware failures that occur in the logic portion of deep learning accelerator systems, including a comprehensive characterization of hardware failure effects, along with the fundamental understanding of how hardware failures propagate in hardware devices and interact with the workloads. Next, based on the insights obtained from our study, I will present ultra lightweight yet highly effective techniques to mitigate hardware failures in deep learning accelerator systems. Bio: Yanjing Li is an Assistant Professor in the Department of Computer Science at the University of Chicago. Prior to joining the university, she was a senior research scientist at Intel Labs. Professor Li received her Ph.D. in Electrical Engineering from Stanford University, an M.S. in Mathematical Sciences (with honors) and a B.S. in Electrical and Computer Engineering (with a double major in Computer Science) from Carnegie Mellon University. Professor Li has received various awards, including the NSF CAREER Award, DAC under-40 innovators award, Google research scholar award, Intel Labs Gordy academy award (highest honor in Intel Labs) and several other Intel recognition awards, outstanding dissertation award (European Design and Automation Association), and multiple best paper awards (for example, IEEE VLSI Test Symposium and IEEE International Test Conference). Professor Li’s seminal work on in-field self-test and diagnostics has been adopted by various companies including Intel, Nvdia, Synopsys, TI, and MentorGraphics, and more.

1 0

Princeton Robotics Seminar: Jeannette Bohg (Stanford), Friday @ 11 AM
by Emily C. Lawrence 16 Apr '24

16 Apr '24

Join us this Friday (4/19) for the Robotics Seminar! Speaker: [ https://web.stanford.edu/~bohg/ | Jeannette Bohg ] , Stanford Date: Friday, April 19, 2024 Time: 11am ET Location: Computer Science 105 Event page: [ https://robo.princeton.edu/seminar | https://robo.princeton.edu/seminar ] PhD students and postdocs can [ https://docs.google.com/forms/d/e/1FAIpQLSczj2FTIvPhV_6f0jQpM5XZTJ_c-dp0DfH… | signup to join Jeannette for lunch ] . There will also be a Robotics Social at 4:00 PM in the F-Wing Cafe Area - all are welcome to attend! Title: Enabling Cross-Embodiment Learning Abstract: In this talk, I will investigate the problem of learning manipulation skills across a diverse set of robotic embodiments. Conventionally, manipulation skills are learned separately for every task, environment and robot. However, in domains like Computer Vision and Natural Language Processing we have seen that one of the main contributing factors to generalisable models is large amounts of diverse data. If we were able to have one robot learn a new task even from data recorded with a different robot, then we could already scale up training data to a much larger degree for each robot embodiment. In this talk, I will present a new, large-scale datasets that was put together across multiple industry and academic research labs to make it possible to explore the possibility of cross-embodiment learning in the context of robotic manipulation, alongside experimental results that provide an example of effective cross-robot policies. Given this dataset, I will also present multiple alternative ways to learn cross-embodiment policies. These example approaches will include (1) UniGrasp - a model that allows to synthesise grasps with new hands, (2) XIRL - an approach to automatically discover and learn vision-based reward functions from cross-embodiment demonstration videos and (3) Equivact - an approach that leverages equivariance to learn sensorimotor policies that generalise to scenarios that are traditionally out-of-distribution. Bio: Jeannette Bohg is an Assistant Professor of Computer Science at Stanford University. She was a group leader at the Autonomous Motion Department (AMD) of the MPI for Intelligent Systems until September 2017. Before joining AMD in January 2012, Jeannette Bohg was a PhD student at the Division of Robotics, Perception and Learning (RPL) at KTH in Stockholm. In her thesis, she proposed novel methods towards multi-modal scene understanding for robotic grasping. She also studied at Chalmers in Gothenburg and at the Technical University in Dresden where she received her Master in Art and Technology and her Diploma in Computer Science, respectively. Her research focuses on perception and learning for autonomous robotic manipulation and grasping. She is specifically interested in developing methods that are goal-directed, real-time and multi-modal such that they can provide meaningful feedback for execution and learning. Jeannette Bohg has received several Early Career and Best Paper awards, most notably the 2019 IEEE Robotics and Automation Society Early Career Award and the 2020 Robotics: Science and Systems Early Career Award.

1 0

CSML Seminar: Machine Learning for Structural Biology - TOMORROW at 12:30pm
by Emily C. Lawrence 15 Apr '24

15 Apr '24

Speaker: Ellen Zhong, Assistant Professor, Princeton University, Computer Science Title: Machine Learning for Structural Biology Day: April 16, 2024 Place: CSML, Bendheim House, 26 Prospect Ave, Classroom Room 103 Time: 12:30-1:30 PM Lunch available from 12:00 Host: Tom Griffiths, Director CSML Abstract: Structural biology has been transformed by breakthroughs in deep learning methods for protein structure prediction. In parallel, advances in cryo-electron microscopy (cryo-EM) have produced new opportunities to study the structure and dynamics of proteins and other biomolecular complexes through imaging. In this seminar, I will overview cryoDRGN and related methods that leverage the representation power of deep neural networks for 3D reconstruction of protein structures from cryo-EM images. Extended to real datasets and released as open-source tools, these methods have been used to discover new protein structures and visualize continuous trajectories of protein motion. I will discuss extensions of the method for scalable and robust reconstruction, analysis of the learned generative model, and visualization of dynamic macromolecular machines in situ. Finally, I will discuss how recent advances in machine learning for protein structure prediction (e.g. AlphaFold) can complement methods for cryo-EM structure determination and what key algorithmic challenges remain to realize the next era of structural biology. Bio: Ellen Zhong is an Assistant Professor of Computer Science at Princeton University. She is interested in problems at the intersection of AI and biology. Her research develops machine learning methods for computational and structural biology problems with a focus on protein structure determination with cryo-electron microscopy (cryo-EM). She obtained her Ph.D. from MIT in 2022, advised by Bonnie Berger and Joey Davis, where she developed deep learning algorithms for 3D reconstruction of dynamic protein structures from cryo-EM images. She has interned at DeepMind with John Jumper and the AlphaFold team and previously worked on molecular dynamics algorithms and infrastructure for drug discovery at D. E. Shaw Research. She obtained her B.S. from the University of Virginia where she worked with Michael Shirts on computational methods for studying protein folding. For more information about her research and group, please visit her group website: https://ezlab.princeton.edu/

1 0

Ted Sumers will present his FPO "Grounding Communication in Real-World Action" on Wednesday, April 24, 2024 at 11:00 AM in CS 301
by gradinfo＠cs.princeton.edu 11 Apr '24

11 Apr '24

Ted Sumers will present his FPO "Grounding Communication in Real-World Action" on Wednesday, April 24, 2024 at 11:00 AM in CS 301 Location: CS 301 The members of Ted's committee are as follows: Examiners: Tom Griffiths (Adviser), Ryan Adams, Adele Goldberg Readers: Karthik Narasimhan, Dylan Hadfield-Menell (MIT), Tom Griffiths (Adviser) A copy of his thesis is available upon request. Please email gradinfo(a)cs.princeton.edu <mailto:gradinfo@cs.princeton.edu> if you would like a copy of the thesis. Everyone is invited to attend his talk. Abstract follows below: This dissertation bridges psychology and artificial intelligence (AI) to develop agents capable of learning through communication with humans. The first half establishes a foundation by comparing the efficacy of language and demonstration for transmitting complex concepts. Experiments reveal language's superior ability to convey abstract rules, suggesting its importance for social learning. I then connect computational models of pragmatic language understanding to reinforcement learning settings, grounding a speaker's utility in their listener's decision problem. Behavioral evidence validates this as a model of human language use. Building on these insights, the second half develops AI agents capable of learning from such language. I first extend the computational model to incorporate both commands and teaching. Experiments show this allows an AI listener to robustly infer the human's latent reward function. I then introduce the problem of learning from fully natural language and contribute two novel approaches: utilizing aspect-based sentiment analysis and a inference network learned end-to-end. Behavioral evaluations demonstrate these models successfully learn from interactive human feedback. Together, this dissertation provides a formal computational theory of the cognitive mechanisms supporting human social learning and embeds them in artificial agents. I discuss implications both for large language models and the continued development of AI agents that acquire and use information through genuine dialogue. This work suggests that building machines to learn as humans do - socially and linguistically - is a promising path towards beneficial artificial intelligence.

1 0

Fangyin Wei will present her FPO "Learning to Edit 3D Objects and Scenes" on Monday, April 22, 2024 at 1:30 PM in CS 302
by gradinfo＠cs.princeton.edu 10 Apr '24

10 Apr '24

Fangyin Wei will present her FPO "Learning to Edit 3D Objects and Scenes" on Monday, April 22, 2024 at 1:30 PM in CS 302 Location: CS 302 The members of Fangyin's committee are as follows: Examiners: Szymon Rusinkiewicz (Adviser), Thomas Funkhouser (Adviser), Jia Deng Readers: Felix Heide, Olga Russakovsky A copy of her thesis is available upon request. Please email gradinfo(a)cs.princeton.edu <mailto:gradinfo@cs.princeton.edu> if you would like a copy of the thesis. Everyone is invited to attend her talk. Abstract follows below: 3D editing plays a key role in many fields ranging from AR/VR, industrial and art design, to robotics. However, existing 3D editing tools either (i) demand labor-intensive manual efforts and struggle to scale to many examples, or (ii) use optimization and machine learning but produce unsatisfactory results (e.g., losing details, supporting only coarse editing, etc.). These shortcomings often arise from editing in geometric space rather than structure-aware semantic space, where the latter is the key to automatic 3D editing at scale. While learning a structure-aware space will result in significantly improved efficiency and accuracy, labeled datasets to train 3D editing models don't exist. In this dissertation, we present novel approaches for learning to edit 3D objects and scenes in structure-aware semantic space with noisy or no supervision. We first address how to extract the underlying structure to edit 3D objects, with a focus on editing two critical properties: semantic shape parts and articulations. Our semantic editing method enables specific edits to an object's semantic parameters (e.g., the pose of a person's arm or the length of an airplane's wing), leading to better preservation of input details and improved accuracy compared to previous work. Next, we introduce a 3D annotation-free method that learns to model geometry, articulation, and appearance of articulated objects from color images. The model works on an entire category (as opposed to typical NeRF extensions that only overfit on a single scene) and enables various applications such as few-shot reconstruction and static object animation. It also generalizes to real-world captures. Then, we tackle how to extract structure for scene editing. We present an automatic system that removes clutter (frequently moving objects such as clothes or chairs) from 3D scenes and inpaints the resulting holes with coherent geometry and texture. We address challenges including the lack of well-defined clutter annotations, entangled semantics and geometry, and multi-view inconsistency. In summary, this dissertation demonstrates techniques to exploit the underlying structure of 3D data for editing. Our work opens up new research directions such as leveraging structures from other modalities (e.g., text, images) to empower 3D editing models with stronger semantic understanding.

1 0