Prachi Sinha will present her MSE talk "Augmenting Concept-Labeled Datasets for Interpretability Using Counterfactual Generation" TODAY Monday, April 22 at 12:30 PM in CS 402.
Prachi Sinha will present her MSE talk "Augmenting Concept-Labeled Datasets for Interpretability Using Counterfactual Generation" Monday, April 22 at 12:30 PM in CS 402. Advisor: Vikram Ramaswamy, Reader: Olga Russakovsky Abstract: Concept-based interpretability methods use a pre-defined set of human-understandable concepts to explain a model's output. These explanations are learned by "probing" the model using a concept-labeled dataset and learning how concepts can be linearly combined to predict outputs. However, there are several limitations to current methods. Explanations are highly dependent on the probe dataset, and they may reflect correlations between concepts and outputs, rather than causation. To address these issues, we augment COCO, a concept-labeled dataset, with counterfactual images in which particular semantic concepts have been removed. Using such counterfactual images, we attempt to improve the accuracy of explanations for a scene classification model, and more directly learn the causal relationships between concepts and outputs. We find that explanation accuracy improves in certain cases, but is generally still limited by the probe dataset's alignment with the model, and that counterfactual images elucidate issues with assigning weights to concepts when there are strong correlations between concepts. CS Grad Calendar: https://calendar.google.com/calendar/event?action=TEMPLATE <https://calendar.google.com/calendar/event?action=TEMPLATE&tmeid=NzljZTNoZm 1ibWc1NGU4ZmtodjJkdHRuMmggYWNnMDc5YmxzbzRtczNza2tmZThwa2lyb2dAZw&tmsrc=acg07 9blso4ms3skkfe8pkirog%40group.calendar.google.com> &tmeid=NzljZTNoZm1ibWc1NGU4ZmtodjJkdHRuMmggYWNnMDc5YmxzbzRtczNza2tmZThwa2lyb 2dAZw&tmsrc=acg079blso4ms3skkfe8pkirog%40group.calendar.google.com
Rishwanth Raghu will present his MSE talk "Tools for analysis of large scale cryo-EM and cryo-ET data" Thursday, April 25 at 1:30 PM in CS 301. Advisor: Ellen Zhong, Reader: Ben Raphael Abstract: Cryo-electron microscopy (cryo-EM) is a powerful technique for visualizing the structure and dynamics of macromolecules in near-physiological conditions. Owing to cryo-EM's ability to image biomolecules in varying native conformations, methods are increasingly being developed to reconstruct heterogeneous ensembles of 3D biomolecular structures from 2D cryo-EM image datasets. However, the field lacks standardized benchmarks for quantitative evaluation of these methods. We present CryoBench, a set of challenging synthetic cryo-EM datasets and novel metrics for the evaluation of heterogeneous reconstruction methods, along with benchmarking analysis of existing state-of-the-art methods. Next, we explore the semantic segmentation task for cryo-electron tomography (cryo-ET), a related biophysical technique for visualizing macromolecules within their cellular environments. We present work towards the generalizable segmentation of large-scale cryo-ET data given only sparse manual annotations. CS Grad Calendar: https://calendar.google.com/calendar/event?action=TEMPLATE <https://calendar.google.com/calendar/event?action=TEMPLATE&tmeid=MW1hanY5MD Q1bGRhaHF1NmVpcHRqMnBzZ2kgYWNnMDc5YmxzbzRtczNza2tmZThwa2lyb2dAZw&tmsrc=acg07 9blso4ms3skkfe8pkirog%40group.calendar.google.com> &tmeid=MW1hanY5MDQ1bGRhaHF1NmVpcHRqMnBzZ2kgYWNnMDc5YmxzbzRtczNza2tmZThwa2lyb 2dAZw&tmsrc=acg079blso4ms3skkfe8pkirog%40group.calendar.google.com
Rishwanth Raghu will present his MSE talk "Tools for analysis of large scale cryo-EM and cryo-ET data" Thursday, April 25 at 1:00 PM in CS 301. Advisor: Ellen Zhong, Reader: Ben Raphael Abstract: Cryo-electron microscopy (cryo-EM) is a powerful technique for visualizing the structure and dynamics of macromolecules in near-physiological conditions. Owing to cryo-EM's ability to image biomolecules in varying native conformations, methods are increasingly being developed to reconstruct heterogeneous ensembles of 3D biomolecular structures from 2D cryo-EM image datasets. However, the field lacks standardized benchmarks for quantitative evaluation of these methods. We present CryoBench, a set of challenging synthetic cryo-EM datasets and novel metrics for the evaluation of heterogeneous reconstruction methods, along with benchmarking analysis of existing state-of-the-art methods. Next, we explore the semantic segmentation task for cryo-electron tomography (cryo-ET), a related biophysical technique for visualizing macromolecules within their cellular environments. We present work towards the generalizable segmentation of large-scale cryo-ET data given only sparse manual annotations. CS Grad Calendar: https://calendar.google.com/calendar/event?action=TEMPLATE <https://calendar.google.com/calendar/event?action=TEMPLATE&tmeid=MW1hanY5MD Q1bGRhaHF1NmVpcHRqMnBzZ2kgYWNnMDc5YmxzbzRtczNza2tmZThwa2lyb2dAZw&tmsrc=acg07 9blso4ms3skkfe8pkirog%40group.calendar.google.com> &tmeid=MW1hanY5MDQ1bGRhaHF1NmVpcHRqMnBzZ2kgYWNnMDc5YmxzbzRtczNza2tmZThwa2lyb 2dAZw&tmsrc=acg079blso4ms3skkfe8pkirog%40group.calendar.google.com
Sabhya Chhabria will present his MSE talk "Exploring LLM Insight Discovery: InsightBench and MCMC Prompting" Wednesday April 24, at10:00 AM in CS 401 Advisor: Danqi Chen, Reader: Karthik Narasimhan Abstract: Large Language Models (LLMs) have become increasingly adept at existing benchmarks that seek to evaluate their abilities on traditional tasks such as language understanding, reasoning, programming and math questions. In this work, we introduce InsightBench, a new benchmark that seeks to evaluate LLMs on insight problems. Insight problems are those problems where the path to a solution is not immediately obvious. They require some form of creative thinking and the answer usually strikes in an "aha" moment. In this work, we evaluate several popular LLMs (both open and closed sourced) of varying sizes on InsightBench. Our results are able to establish that insight problems (creative problem solving) remain a critical failure point of language models. We further try to bolster LLM performance on certain InsightBench tasks by using Markov Chain Monte Carlo prompting schemes, allowing LLMs to explore large search spaces to solve insight problems. CS Grad Calendar: https://calendar.google.com/calendar/event?action=TEMPLATE&tmeid=MWtlMnFucWZ rNWs4dTZmYWg2c25qZTNxZ24gYWNnMDc5YmxzbzRtczNza2tmZThwa2lyb2dAZw&tmsrc=acg079 blso4ms3skkfe8pkirog%40group.calendar.google.com
Constance Ferragu will present her MSE talk "Vendi-Decoding: Diverse Sampling for Protein Sequence Design" Wednesday, April 24 at 9:30 AM in CS 402. Advisor: Adji Bousso Dieng Reader: Olga Troyanskaya Abstract: Protein sequence models have become increasingly valuable for the design of novel proteins. These models learn distributions over amino acids at sequence positions. However, decoding from these models poses a significant challenge due to the exponentially large sequence space. A significant limitation is the lack of diversity and exploration. These methods tend to prioritize decoding high-likelihood tokens, resulting in repetitive or similar sequences. Generating diverse sequences with high naturalness is crucial for thorough exploration of the sequence space, essential for the discovery of novel protein sequences. In this thesis, we propose Vendi Decoding, a sequence decoding algorithm designed to improve the efficiency of exploring sequence space and the diversity of decoded sequence sets. Our method leverages the Vendi Score, a statistical measure of diversity, to select edit positions that will most effectively improve our diversity objective and to guide the model's hidden representations towards diverse decoding steps. Our results demonstrate that Vendi Decoding can iteratively refine a seed sequence into a set of diverse sequences more rapidly, while ensuring that the quality of sequences does not deteriorate. CS Grad Calendar: https://calendar.google.com/calendar/event?action=TEMPLATE <https://calendar.google.com/calendar/event?action=TEMPLATE&tmeid=NWNtcmM2dX FiYjdjcXZmYTdqdmU0Y3VhdGggYWNnMDc5YmxzbzRtczNza2tmZThwa2lyb2dAZw&tmsrc=acg07 9blso4ms3skkfe8pkirog%40group.calendar.google.com> &tmeid=NWNtcmM2dXFiYjdjcXZmYTdqdmU0Y3VhdGggYWNnMDc5YmxzbzRtczNza2tmZThwa2lyb 2dAZw&tmsrc=acg079blso4ms3skkfe8pkirog%40group.calendar.google.com
participants (1)
-
gradinfo＠cs.princeton.edu