Research of "visual language" under that name is relatively new. However, there have been scattered studies on the structure and comprehension of sequential images across numerous disciplines for many years. It is my intent to unify this research into a cohesive understanding, as well as to sponsor future research under the umbrella of "visual language research."
This page provides further resources for people interested in pursing this line of research, beyond my own downloadable papers and books.
Have a resource you'd like to see here that might be useful for your research needs? If so, please email me.
I often receive emails asking what the best way is to go about researching the structure of "comics." My methods are drawn directly from how language is studied in linguistics and psychology research, and I believe that these provide the best ways in which we can understand about the structure and comprehension of visual language.
For specific advice on methodology for how to go about doing research, you might want to check out my blog posts on Advice for Scholarship.
I have been developing several classes for teaching visual language theory examining both structure and cognition. Below, I list sample syllabi. If you are further interested in teaching a class like this, please email me for suggestions about homework assignments and additional readings.
- Language of Comics: Introduction to Visual Language, Winter 2013 (PDF)
- Language of Comics: Introduction to Visual Language, Fall 2015 (PDF)
- Cognition of Comics: Visual Language Research, Spring 2013 (PDF)
My theory of Visual Narrative Grammar has provided a way to explain how people comprehend sequential images, and has been shown experimentally to reflect how people's brains comprehend narrative sequential images. However, the complexity of this theory makes it somewhat challenging to implement when students or researchers may wish to use it to analyze sequences. This tutorial provides a step-by-step guide for the procedures and diagnostics that should be used when analyzing sequential images.
- How to analyze visual narratives: A tutorial in Visual Narrative Grammar (PDF)
In my experiments, I have tried to assess how "fluent" my participants are in the visual language used in comics by generating a number that can be correlated with experimental findings. I now call this metric the "Visual Language Fluency Index" (VLFI). If you are running experiments using any aspect of this visual language and would like to use this measure, please download the following zip file which contains pdf documentation about the measure, a questionnaire for participants, and an Excel spreadsheet for generating the fluency score.
- Visual Language Fluency Index package (ZIP folder)
In formulating visual language theory, I have introduced many new terms. This glossary provides a list of many of these terms.
Previously, this website provided a reference bibliography listing papers that have explored the comprehension of sequential images and/or their parts. We are currently exploring ways to offer a more comprehensive and interactive database that can further provide better resources along these lines.
For now though, ongoing summaries and "reviews" of papers related to sequential image understanding and visual communication are available at the Visual Linguist Blog with the tag "reviews" (direct link).
There are not many books about visual language besides my own, but there are definitely books that I recommend for people to read that can better enrich their understanding of language, visual narratives, and related aspects of cognition.
This landmark book includes the most extensive treatment to date of the sand narratives used by Aboriginals in Central Australia. It focuses on the multimodal interactions between the spoken, signed, and drawn modalities in the overall scope of narratives. It is highly recommended!
This far-reaching collection expands conceptual metaphor theory into various domains of multimodal combinations. Of particular note are the several chapters about metaphor in comics and cartoons.
Considered one of the most significant books on language of the last thirty years, this book should be considered basic reading for anyone seriously considering the study of language. Since Ray Jackendoff was my mentor, this book provides the foundation for the theoretical architecture of language and the mind on which visual language theory is built.
A somewhat old study at this point, this fascinating work describes what happened when two researchers taught Navajo how to create their own films. Of particular interest to the structural aspects of visual language theory are the interesting ways that they structured their visual narratives. It raises many questions about cross-cultural cognition and visual narrative. The whole book is available online here.
Perhaps the best books I can recommend about how to draw, especially since "Commander" Mark was my childhood drawing teacher. His methods reinforce much of the arguments made in visual language theory, that drawing is about learning basic graphc scheams that build on each other as a visual vocabulary. Recommended for anyone learning how to draw.