It is well known that verbs play a key role in determining parsing preferences (e.g., Clifton, Jr. et al., 1984; MacDonald et al., 1994) and constitute the central elements over which we build conceptual-structure representations of utterances (Jackendoff, 1990). In a recent series of studies, Altmann and Kamide (1999) showed that when verbs highly constrain the nature of noun complements - such as in (1a) - saccade onsets to drawings of objects related to the verbs are significantly faster than in less-constraining constructions such as in (1b).
In this paper, we present a study on the role of visual representations in the access to argument structure and conceptual structure of verbs during sentence processing. We used an eye-tracking paradigm in which subjects heard sentences while looking at pictures of objects displayed on a computer screen. We controlled saccade onsets and fixations by presenting a red dot on the screen placed equidistant from a set of pictures of objects. Subjects had to fixate on the red dot until it turned green - time at which they had to look at the object(s) that the sentence referred to. In addition to the picture of the target (the noun related to the verb, in the experimental condition), we presented sets of pictures representing linguistic (phonological), conceptual (semantically related) and visual (shape and colour) competitors. We manipulated the time at which the "light" turned green - from the onset of the noun up to 200 ms after its offset.
We contrasted two types of sentences, a highly constraining causative construction (2a), in which there is a close conceptual relation between verb and direct object, and a neutral one with a transitive perception verb (2b).
We found significant differences between probe points in both types of constructions suggesting that early during the presentation of the noun subjects initiate a saccade to the object labeled by the noun. However, we did not find a difference between the verb conceptually related to its direct object (causative; (2a)) and the neutral transitive verb (perception; (2b)) during the presentation of the target noun, and only marginally significant differences 200 ms after the offset of the noun. Our results, thus, failed to support the view that verb- specific representations cause faster eye movements to objects related to typical complements of verbs.
Although it is clear that integration between visual and linguistic representations is crucial for language interpretation in real-world contexts, we argue that the language comprehension system relies primarily on structural analyses of linguistic representations which are independent from conceptual interpretation.
References
Altmann, G. T. M., & Kamide, Y. (1999). Incremental interpretation at verbs: restricting the domain of subsequent reference. Cognition, 73, 247-264.
Clifton, J., Charles, Frazier, L., & Conniene, C. (1984). Lexical expectations in sentence comprehension. Journal of Verbal Learning and Verbal Behavior, 23, 696-708.
Dowty, D. (1991). Thematic proto-roles and argument selection. Language, 67, 547-619.
Jackendoff, R. (1990). Semantic Structures. Cambridge, MA: MIT Press.
MacDonald, M. C., Pearlmutter, N. J., & Seidenberg, M. S. (1994). Lexical nature of syntactic ambiguity resolution. Psychological Review, 4, 676-703.