Research of "visual language" under that name is relatively new. However, there have been scattered studies on the structure and comprehension of sequential images across numerous disciplines for many years. It is my intent to unify this research into a cohesive understanding, as well as to sponsor future research under the umbrella of "visual language theory."
This page provides further resources for people interested in pursing this line of research, beyond my own downloadable papers and books.
Have a resource you'd like to see here that might be useful for your research needs? If so, please email me.
I often receive emails asking what the best way is to go about researching the structure of comics. My methods are drawn directly from how language is studied in linguistics and psychology research, and I believe that these provide the best ways in which we can understand about the structure and comprehension of visual language.
For specific advice on methodology for how to go about doing research, you might want to check out my blog posts on Advice for Scholarship.
Previously, this website provided a reference bibliography listing papers that have explored the comprehension of sequential images and/or their parts. We are currently exploring ways to offer a more comprehensive and interactive database that can further provide better resources along these lines.
Nevertheless, ongoing summaries and "reviews" of papers related to sequential image understanding and visual communication are available at the Visual Linguist Blog with the tag "reviews" (direct link). See also the books listed under "recommended reading" below.
In formulating visual language theory, many new terms have been introduced. This glossary provides a list of many of these terms.
There are not many books about visual language besides my own, but there are definitely books that I recommend for people to read that can better enrich their understanding of language, visual narratives, and related aspects of cognition.
This landmark book includes the most extensive treatment to date of the sand narratives used by Aboriginals in Central Australia. It focuses on the multimodal interactions between the spoken, signed, and drawn modalities in the overall scope of narratives. It is highly recommended!
This far-reaching collection expands conceptual metaphor theory into various domains of multimodal combinations. Of particular note are the several chapters about metaphor in comics and cartoons.
Considered one of the most significant books on language of the last thirty years, this book should be considered basic reading for anyone seriously considering the study of language. Since Ray Jackendoff was Neil Cohn's mentor, this book provides the foundation for the theoretical architecture of language and the mind on which visual language theory is built.
A somewhat old study at this point, this fascinating work describes what happened when two researchers taught Navajo how to create their own films. Of particular interest to the structural aspects of visual language theory are the interesting ways that they structured their visual narratives. It raises many questions about cross-cultural cognition and visual narrative. The whole book is available online here.