Prediction and Locality

in Language and Language Models

 

Richard Futrell

Department of Language Science

University of California, Irvine

 

Thursday, December 19, 2024

4:30-6:00 p.m.

1-S-5 Green Hall

 

 

 

I argue that language models share information-processing constraints with humans, arising from the shared core task of language models and the brain: predicting upcoming input. I show that we can use constraints on sequential prediction to explain aspects of human language processing, as reflected in eyetracking and reading time data, as well as various universal properties of languages, as reflected in patterns of word order in massively cross-linguistic text corpora. Furthermore, the same constraints explain language model performance when learning human-like versus non-human-like languages. I argue that this approach points to an information-theoretic explanation for the nature of human language, and links the science of language with neuroscience and machine learning.

 

Richard Futrell is Associate Professor of Language Science at the University of California, Irvine. His research focuses on language in humans and machines.