(Pea)nuts and Bolts of Visual Narrative: Structure and Meaning in Sequential Image Comprehension

Autor/inn/en	Cohn, Neil; Paczynski, Martin; Jackendoff, Ray; Holcomb, Phillip J.; Kuperberg, Gina R.
Titel	(Pea)nuts and Bolts of Visual Narrative: Structure and Meaning in Sequential Image Comprehension
Quelle	In: Cognitive Psychology, 65 (2012) 1, S.1-38 (38 Seiten) PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0010-0285
DOI	10.1016/j.cogpsych.2012.01.003
Schlagwörter	Sentences; Semantics; Syntax; Language Processing; Semiotics; Experiments; Visual Stimuli; Comparative Analysis; Interaction; Sentence Structure; Comprehension; Reading Comprehension + Suchen Sie Ihr Suchwort? Sentence analysis; Satzanalyse; Semantik; Sprachverarbeitung; Semiotik; Erprobung; Interaktion; Satzbau; Satzstruktur; Verstehen; Verständnis; Leseverstehen
Abstract	Just as syntax differentiates coherent sentences from scrambled word strings, the comprehension of sequential images must also use a cognitive system to distinguish coherent narrative sequences from random strings of images. We conducted experiments analogous to two classic studies of language processing to examine the contributions of narrative structure and semantic relatedness to processing sequential images. We compared four types of comic strips: (1) Normal sequences with both structure and meaning, (2) Semantic Only sequences (in which the panels were related to a common semantic theme, but had no narrative structure), (3) Structural Only sequences (narrative structure but no semantic relatedness), and (4) Scrambled sequences of randomly-ordered panels. In Experiment 1, participants monitored for target panels in sequences presented panel-by-panel. Reaction times were slowest to panels in Scrambled sequences, intermediate in both Structural Only and Semantic Only sequences, and fastest in Normal sequences. This suggests that both semantic relatedness and narrative structure offer advantages to processing. Experiment 2 measured ERPs to all panels across the whole sequence. The N300/N400 was largest to panels in both the Scrambled and Structural Only sequences, intermediate in Semantic Only sequences and smallest in the Normal sequences. This implies that a combination of narrative structure and semantic relatedness can facilitate semantic processing of upcoming panels (as reflected by the N300/N400). Also, panels in the Scrambled sequences evoked a larger left-lateralized anterior negativity than panels in the Structural Only sequences. This localized effect was distinct from the N300/N400, and appeared despite the fact that these two sequence types were matched on local semantic relatedness between individual panels. These findings suggest that sequential image comprehension uses a narrative structure that may be independent of semantic relatedness. Altogether, we argue that the comprehension of visual narrative is guided by an interaction between structure and meaning. (Contains 9 figures and 4 tables.) (As Provided).
Anmerkungen	Elsevier. 6277 Sea Harbor Drive, Orlando, FL 32887-4800. Tel: 877-839-7126; Tel: 407-345-4020; Fax: 407-363-1354; e-mail: usjcs@elsevier.com; Web site: http://www.elsevier.com
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2017/4/10