To what extent can statistical information implicit in the linguistic environment provide a 'low-level' account of certain aspects of language processing behaviour? Using eyetracking data recorded during the reading of naturalistic text, we explored the relationships between ways of characterising linguistic experience and lexical processing effort, as revealed by eye movement measures.
It is well-established that prior context can influence the effort involved in processing a given word during reading. This effect of context is assumed to be predictive in nature; if the preceding material constrains the identity and/or meaning (e.g., Rayner & Well, 1996) of word[n], then it is less likely to be fixated and if fixated, its fixation time is shorter than if word[n] is not as predictable. What is not established is the source of this effect: are constraint effects due solely to 'high-level' processes that infer the identity and/or meaning of word[n] from the current sentence/discourse model? Or do 'low-level' factors, derived from knowledge of words' distributional behaviour in one's language experience, contribute to predictability effects?
A low-level account may take the form of simple Hebbian learning; when one encounters two co-occurring words, the connections between their neural representations are strengthened. The corpus-derived conditional bigram probability, p(word[n]|word[n-1]), is a good candidate for modelling this low-level information.
We obtained eye movement records for 10 subjects each reading approximately 1,000 words of light fiction. Using multiple regression techniques for repeated measures data (Lorch & Myers, 1990), we explored the viability of several experiential measures as predictors of two eye movement variables: whether a word is skipped or not (Skip) and first pass fixation duration (Gaze). Word length in letters (WL) was also included as a predictor. The two corpus-based measures (log-transformed word frequency: WF; log-transformed bigram probability: BiP) were derived from the 10M word spoken language part of the British National Corpus. Smoothed bigram probabilities were computed using the CMU-Cambridge Language Modeling Toolkit (Clarkson & Rosenfeld, 1997).
A logistic regression on Skip with WL, WF and BiP as predictors revealed reliable independent effects of WL and WF only. However, a linear regression on Gaze using the same predictor variables indicated significant independent effects of WL and BiP only. Because frequency and bigram probability are highly intercorrelated, it is possible that a single variable that combines the prior probability of a word occurring (estimated by its relative frequency) with its predictability may offer a more parsimonious view. We have been experimenting with Bayesian techniques for combining these two sources of probabilistic information; initial results indicate that a combined variable explains more Gaze variance than either WF or BiP.
The conclusion we draw from these analyses is that for the amount of time the eyes spend on a word before moving on, it is the experience-governed probability of the word occurring, not its frequency or simple predictability which offers the best account. Whether or not a word is skipped does not appear best explained as the consequences of low-level prediction, but as a function of at least its length and frequency.
References
Clarkson, P. & Rosenfeld, R. (1997) Statistical language modelling using the CMU-Cambridge Toolkit. In Proceedings of Eurospeech '97.
Lorch, R. F. & Myers, J. L. (1990). Regression analyses of repeated measures data in cognitive research. Journal of Experimental Psychology: Learning, Memory, and Cognition, 16, 149-157.
Rayner, K. & Well, A. D. (1996). Effects of contextual constraint on eye movements in reading: A further examination. Psychonomic Bulletin & Review, 3, 504-509.