This page features downloadable academic papers on visual language research. Also, see my introductory writings for more accessible writing for laypeople, and the Visual Linguist blog for more casual and exploratory writing, along with updates about ongoing research. For more detailed information about visual language and its study, check out my book, The Visual Language of Comics.
- PDF – open pdfs or links in new window.
- NEW – recently added papers.
- *** – recommended introductory papers.
Papers are organized by their major topics as best as possible. Because aspects of visual language are interconnected, some papers may discuss multiple topics. For a chronological listing of papers, see my vitae.
The theory of visual language covers a wide range of topics, just like the study of spoken or signed languages. These papers discuss the broader relationship between visual language, comics, and lingusitics, and how to study them.
*** Cohn, Neil. 2016. Sequential images are not universal, or Caveats for using visual narratives in experimental tasks. In Papafragou, A., Grodner, D., Mirman, D., & Trueswell, J.C. (Eds.) (2016). Proceedings of the 38th Annual Conference of the Cognitive Science Society. Austin, TX: Cognitive Science Society. (PDF)
*** Cohn, Neil. 2015. Climbing trees and seeing stars: Combinatorial structure in comics and diverse domains. In Toivonen, Ida, Piroska Csúri & Emilie van der Zee. (Ed). Structures in the Mind: Essays on Language, Music, and Cognition in Honor of Ray Jackendoff (pp. 379-392). Cambridge: MIT Press. (PDF)
Cohn, Neil. 2014. Building a better “comic theory”: Shortcomings of theoretical research on comics how to overcome them. Studies in Comics. 5(1): 57-75. (PDF)
*** Cohn, Neil. 2012. Comics, linguistics, and visual language: The past and future of a field. In Bramlett, Frank (ed). Linguistics and the Study of Comics (pp. 92-118). New York: Palgrave Macmillan. (PDF)
Cohn, Neil. 2010. Japanese Visual Language: The structure of manga. In Manga: An Anthology of Global and Cultural Perspectives, edited by T. Johnson-Woods (pp. 187-203). New York: Continuum Books. (PDF)
Cohn, Neil. 2005. Un-defining "comics." International Journal of Comic Art. 7(2): 236-248. (PDF)
My main project in visual langauge research has always been exploring how sequences of images create meaning in the mind and brain. I am primarily interested in how such competence is structured, how it is processed in the brain, and how that system relates to other cognitive domains like verbal discourse and narrative, event structure, and even musical cognition. Because the theory of narrative grammar is fairly complex, students and researchers interested in the diagnostics and procedures of implementing it should read this tutorial on How to analyze visual narrative (PDF).
Cohn, Neil. 2015. How to analyze visual narrative. Visual Language Lab Resource (PDF)
Cohn, Neil. 2015. Narrative conjunction’s junction function: The interface of narrative grammar and semantics in sequential images. Journal of Pragmatics 88: 105-132 (PDF)
*** Cohn, Neil. 2014. The architecture of visual narrative comprehension: the interaction of narrative structure and page layout in understanding comics. Frontiers in Psychology. 5:680 (Online version & PDF, Japanese PDF)
Cohn, Neil. 2013. Visual narrative structure. Cognitive Science 37(3): 413-452. (PDF)
Cohn, Neil. 2010. The limits of time and transitions: Challenges to theories of sequential image comprehension. Studies in Comics 1(1): 127-147. (PDF)
Experimental and corpus papers
NEW Cohn, Neil and Marta Kutas. 2017. What’s your neural function, visual narrative conjunction? Grammar, meaning, and fluency in sequential image processing. Cognitive Research: Principles and Implications. 2(27): 1-13. (PDF, Online version, Poster summary)
NEW Cohn, Neil and Patrick Bender. 2017. Drawing the line between constituent structure and coherence relations in visual narratives. Journal of Experimental Psychology: Learning, Memory, and Cognition. 43(2): 289-301. (PDF)
Foulsham, Tom, Dean Wybrow, and Neil Cohn. 2016. Reading without words: Eye-movements in the comprehension of comic strips. Applied Cognitive Psychology. 30: 566-579 (PDF)
Hagmann, Carl Erick, and Neil Cohn. 2016. The pieces fit: Constituent structure and global coherence of visual narrative in RSVP. Acta Psychologica. 164:157-164. (PDF)
Cohn, Neil, and Marta Kutas. 2015. Getting a cue before getting a clue: Event-related potentials to inference in visual narrative comprehension. Neuropsychologia 77: 267-278. (PDF)
Cohn, Neil, and Eva Wittenberg. 2015. Action starring narratives and events: Structure and inference in visual narrative comprehension. Journal of Cognitive Psychology. 27(7): 812-828. (PDF)
Cohn, Neil, Ray Jackendoff, Phillip Holcomb, and Gina Kuperberg. 2014. The grammar of visual narratives: Neural evidence for constituent structure in visual narrative comprehension. Neuropsychologia. 64: 63-70. (PDF, Video summary)
Cohn, Neil. 2014. You’re a good structure, Charlie Brown: The distribution of narrative categories in comic strips. Cognitive Science, 38(7): 1317-1359. (PDF)
Cohn, Neil, and Martin Paczynski. 2013. Prediction, events, and the advantage of Agents: The processing of semantic roles in visual narrative. Cognitive Psychology. 67 (3):73-97. (PDF)
Cohn, Neil, Martin Paczynski, Ray Jackendoff, Phillip Holcomb, and Gina Kuperberg. 2012. (Pea)nuts and bolts of visual narrative: Structure and meaning in sequential image comprehension. Cognitive Psychology 65(1): 1-38. (PDF, Poster version)
Cohn, Neil, Amaro Taylor-Weiner, and Suzanne Grossman. 2012. Framing attention in Japanese and American comics: Cross-cultural differences in attentional structure. Frontiers in Cultural Psychology. 3: 1-12. (PDF)
Cohn, Neil. 2011. A different kind of cultural frame: An analysis of panels in American comics and Japanese manga. Image [&] Narrative 12 (1): 120-134. (PDF)
Visual languages rarely appear in isolation, and usually accompany written language. These papers explore these types of multimodal interactions.
NEW Cohn, Neil, Ryan Taylor, and Kaitlin Pederson. 2017. A picture is worth more words over time: Multimodality and narrative structure across eight decades of American superhero comics. Multimodal Communication. 6(1): 19-37. (PDF, Video Presentation)
NEW Manfredi, Mirella, Neil Cohn, and Marta Kutas. 2017. When a hit sounds like a kiss: an electrophysiological exploration of semantic processing in visual narrative. Brain and Language. 169: 28-38. (PDF)
Pratha, Nimish K., Natalie Avunjian, and Neil Cohn. 2016. Pow, punch, pika, and chu: The structure of sound effects in genres of American comics and Japanese manga. Multimodal Communication. 5(2): 93-109. (PDF, Video Presentation)
*** Cohn, Neil. 2016. A multimodal parallel architecture: A cognitive framework for multimodal interactions. Cognition 146: 304-323. (PDF)
These papers discuss aspects of comic page layouts—their "External Compositional Structure."
Pederson, Kaitlin and Neil Cohn. 2016. The changing pages of comics: Page layouts across eight decades of American superhero comics. Studies in Comics. 7(1): 7-28. (PDF, Video Presentation)
Cohn, Neil and Hannah Campbell. 2015. Navigating comics II: Constraints on the reading order of page layouts. Applied Cognitive Psychology. 29: 193-199. (PDF)
Cohn, Neil. 2013. Navigating comics: An empirical and theoretical approach to strategies of reading comic page layouts. Frontiers in Cognitive Science. 4: 1-15. (Online version & PDF)
These papers discuss the cognition of drawing: how we create meaning by making graphic marks. They explore how drawing is structured in the mind/brain, how this ability is developed, and the effect of culture on these abilities.
Cohn, Neil. 2014. Framing “I can’t draw”: The influence of cultural frames on the development of drawing. Culture & Psychology. 20(1): 102-117. (PDF)
*** Cohn, Neil. 2012. Explaining "I can't draw": Parallels between the structure and development of language and drawing. Human Development 55(4): 167-192. (PDF)
These papers cover aspects of visual language "vocabulary"—the meaningful elements of images.
NEW Cohn, Neil, Vivian Wong, Kaitlin Pederson & Ryan Taylor. (2017). Path salience in motion events from verbal and visual languages. In G. Gunzelmann, A. Howes, T. Tenbrink, & E. J. Davelaar (Eds.), Proceedings of the 39th Annual Conference of the Cognitive Science Society (pp. 1794-1799). Austin, TX: Cognitive Science Society. (PDF, Poster PDF)
Cohn, Neil, Beena Murthy, and Tom Foulsham. 2016. Meaning above the head: Combinatorial constraints on the visual vocabulary of comics. Journal of Cognitive Psychology. 28(5): 559-574. (PDF)
Cohn, Neil and Sean Ehly. 2016. The vocabulary of manga: Visual morphology in dialects of Japanese Visual Language. Journal of Pragmatics. 92: 17-29. (PDF, Listing of visual morphemes in JVL)
Cohn, Neil, and Stephen Maher. 2015. The notion of the motion: The neurocognition of motion lines in visual narratives. Brain Research. 1601: 73-84. (PDF, Poster version).
Cohn, Neil. 2013. Beyond speech balloons and thought bubbles: The integration of text and image. Semiotica. 2013(197): 35-63. (PDF)
Cohn, Neil. 2010. Extra! Extra! Semantics in comics!: The conceptual structure of Chicago Tribune advertisements. Journal of Pragmatics 42 (11): 3138–3146. (PDF)
Cohn, Neil. 2007. A visual lexicon. Public Journal of Semiotics. 1(1): 35-56. (PDF)