Speaker: Ofir Press, Princeton University Date: Thursday, April 10 Time: 12:30pm EST Location: CS 105 Host: Peter Henderson Event page: https://www.cs.princeton.edu/events/towards-autonomous-language-model-system... Register for live-stream online here: https://princeton.zoom.us/webinar/register/WN_fVUS0ZFATVesABqXSe-JUw Title: Towards Autonomous Language Model Systems Abstract: Language models (LMs) are increasingly used to assist users in day to day tasks such as programming (Github Copilot) or search (Google's AI Overviews). But can we build language model systems that are able to autonomously complete entire tasks end-to-end? In this talk I'll discuss our efforts to build autonomous LM systems, focusing on the software engineering domain. I'll present SWE-bench, our novel method for measuring AI systems on their abilities to fix real issues in popular software libraries. I'll then discuss SWE-agent, our system for solving SWE-bench tasks. SWE-bench and SWE-agent are used by many leading AI orgs in academia and industry including OpenAI, Anthropic, Meta, and Google, and SWE-bench has been downloaded over 2 million times. These projects show that academics on tight budgets are able to have substantial impact in steering the research community towards building autonomous systems that can complete challenging tasks. Bio: Ofir Press is a postdoc at Princeton University where he mainly works with Karthik Narasimhan's lab. He previously completed his PhD at the University of Washington in Seattle, where he was advised by Noah Smith. During his PhD he spent two years at Facebook AI Research Labs on Luke Zettlemoyer's team.