Please see revised date/time for Jane's MSE talk:
Thursday, April 27, 2023 at 1pm in COS 402
See original announcement below.
From: "Nicki Gotsis"
To: "talks"
Sent: Monday, April 17, 2023 9:00:00 PM
Subject: [talks] Jane Pan will present her MSE talk "What In-Context Learning “Learns” In-Context: Disentangling Task Recognition and Task Learning" on Monday, April 24, 2023 at 2pm in CS 402.
Jane Pan will present her MSE talk "What In-Context Learning “Learns” In-Context: Disentangling Task Recognition and Task Learning" on Monday, April 24, 2023 at 2pm in CS 402.
The members of her committee are as follows:
Advisor: Danqi Chen; Reader: Karthik Narasimhan
Title: What In-Context Learning “Learns” In-Context: Disentangling Task Recognition and Task Learning
Abstract:
Large language models (LLMs) exploit in-context learning (ICL) to solve tasks with only
a few demonstrations, but its mechanisms are not yet well-understood. Some works suggest that LLMs only recall already learned concepts from pre-training, while others hint that
ICL performs implicit learning over demonstrations. We characterize two ways through
which ICL leverages demonstrations: task recognition (TR) captures the extent to which
LLMs can recognize a task through demonstrations – even without ground-truth labels –
and apply their pre-trained priors; task learning (TL) is the ability to capture new input-
label mappings unseen in pre-training. Using a wide range of classification datasets and two
LLM families (GPT-3 and OPT), we design controlled experiments to disentangle the roles
of TR and TL in ICL. We show that (1) models can achieve non-trivial performance with
only TR, and TR does not further improve with larger models or more demonstrations; (2)
LLMs acquire TL as the model size scales; and (3) the correlation between model sizes and TL
performance strengthens with the number of demonstrations. Our findings unravel two dif-
ferent forces behind ICL and we advocate for discriminating them in future ICL research due
to their distinct nature.
_______________________________________________
talks mailing list
talks@lists.cs.princeton.edu
To edit subscription settings or remove yourself, use this link:
https://lists.cs.princeton.edu/mailman/listinfo/talks
