Probabilistic Modeling in Psycholinguistics: A Survey and Apologia

Dan Jurafsky
University of Colorado, Boulder
Department of Linguistics, Department of Computer Science
Institute of Cognitive Science, & Center for Spoken Language Research

A wide variety of evidence suggests that humans are probabilistic reasoners, and that this key role for probabilities extends to language processing. In language comprehension, probabilities of linguistic structure at every level (phonology, morphology, the lexicon, syntax) play a role in access and disambiguation of linguistic structure. In language production, probability plays a role in choosing among structures to generate, and helps determine the surface form that words take. In learning, probability plays a role in segmentation and in generalization. The implications are far-reaching and exciting. Human language processing is "probabilistic all the way down".

These experimental results have been modeled with many different probabilistic formalisms. Language comprehension, in particular, has been addressed by Markov models, stochastic grammars, Bayesian belief networks, and other models from information theory and rational agent theory.

The survey part of the talk will cover the experimental results on the role of probability in psycholinguistics, and various probabilistic models of language comprehension. The apologia part of the talk will address a number of common challenges to probabilistic modeling, ranging from "Surely you don't really believe this" to "But this is obvious and trivially true".


AMLaP Conference, Saarbrücken, September 2001