Austin Wang will present his General Exam "Semantic Supervision for Simple and Scalable Zero-shot Generalization" on Monday, May 1, 2023 at 1:30 PM in CS 401 . Committee Members: Karthik Narasimhan(advisor), Olga Russakovsky, Danqi Chen Abstract: Zero-shot learning is the problem of predicting instances over classes not seen during training. One approach to zero-shot learning is providing auxiliary class information to the model. Prior work along this vein have largely used expensive per-instance annotation or singular class-level descriptions, but per-instance descriptions are hard to scale and single class descriptions may not be rich enough. Furthermore, these works have used natural-language descriptions exclusively, simple bi-encoders models, and modality or task-specific methods. These approaches have several limitations: text supervision may not always be available or optimal and bi-encoders may only learn coarse relations between inputs and class descriptions. In this work, we present Semsup, a novel approach that uses (1) a scalable multiple description sampling method which improves performance over single descriptions, (2) alternative description formats such as JSON that are easy to generate and outperform text on certain settings, and (3) hybrid lexical-semantic similarity to leverage fine-grained information in class descriptions. We demonstrate the effectiveness of Semsup across four datasets, two modalities, and three generalization settings. For example, across text and image datasets, Semsup increases unseen class generalization accuracy by 15 points on average compared to the closest baseline. Reading List: [ https://docs.google.com/document/d/1S4U2c1IHB5yIHDjQ8yIFLxcKmIbdAm94R1Re3znx... | https://docs.google.com/document/d/1S4U2c1IHB5yIHDjQ8yIFLxcKmIbdAm94R1Re3znx... ] Everyone is invited to attend the talk, and those faculty wishing to remain for the oral exam following are welcome to do so. Louis Riehl Graduate Administrator Computer Science Department, CS213 Princeton University (609) 258-8014