VINCENT TAOHSUN CHANG

National. Chengchi University (TAIWAN)

Visual meaning in multimodal discourse: an experimental pragmatic approach

Pragmática / Pragmatics

How does our knowledge of language and of context endow us to understand what we are told, to resolve ambiguities, to grasp both explicit and implicit contents, and to appreciate non-literal expressions—metaphor, irony, pun, hyperbole, humour, poetic effects, and, non-verbal communication? These issues have often been approached within linguistic pragmatics and psycholinguistics, whilst with only limited interactions between the two. This paper thus aims to explore the audience's perception, comprehension and interpretation of visual image in multimodal communication , reexamining the explanatory adequacy of Relevance framework (Sperber & Wilson 1986/1995) and exploring the significant novelty of experimental pragmatics (Noveck & Sperber 2005).

Research questions:
This paper attempts to investigate implicit meaning conveyed in multimodal discourse, trying to explain and render plausible interpretations to the following research questions:

1. What explicit and implicit information and cognitive contextual effects would be perceived and inferred by the (different) receptors through the integration of the visual images and slogans employed within institutionalised discourse / specialised communication?
2. The implicit meaning, especially weak implicatures involving feelings, attitudes, emotions and impressions, will fall into an indeterminate range. Can we regard these as scalar implicatures with different functional loadings or weights?
3. Which inferred salient implicature of implicit meanings could possibly be served as default value?
4. Will the results of this study enhance the explanatory power of Neo-Gricean pragmatic theories in terms of language, cognition and communication, e.g. Relevance Theory (Sperber & Wilson 1986/1995; Noveck & Sperber 2005)?

Methodology
The current study is to conduct an experiment by randomly sampling one dozen students from different departments at NCCU to look into six captions designed for the Olympics 2008 released by Mainland China. The subjects are to watch (and hence process) the six captions (for around one to two minutes) first, as shown below; and then report/narrate the pictures they just perceived and processed respectively as much as they possibly can (for three to five minutes or so). These activated meanings are then tape-recorded for further analysis.

Expected results:
What the subjects have (actively, creatively and imaginatively) inferred are supposed to fall into a wide range of weak implicatures along with strong implicatures, depending on the different degrees of involvement and shared background knowledge in terms of cultural, social, political aspects etc. At the least case, the subjects could process and tell what is all about the slogan "???, ????! (Olympic Fever Heats the Whole Beijing!)" — the explicit meaning. Or if unfortunately not, then the communication is unsuccessful, which somehow still conforming the principle of relevance. After all, by the same token, we couldn't expect that all the students in one class would understand one joke simultaneously, or would process therein at the same speed. Considering "…there is no clear cut-off point between assumptions strongly backed by the communicator, and the assumptions derived from the utterance on the addressee's sole responsibility," (cf. fn. 2) we group/classify the implicatures according to the order that they previously narrated. We compare then those implicatures inferred and derived by those subjects to see the overlapping (or quasi-overlapping/similar) parts to re-/organise and find the functional loadings — scalar implicatures (cf. Noveck & Sperber 2005). Also expectedly to assign the salient meaning(s) of them as default value accordingly.

Keywords: experimental pragmatics, implicature, multimodal discourse, relevance, visual meaning
References
Forceville, Charles J. (2005). Multimodal metaphors in commercials. Paper for "The pragmatics of multimodal representations" panel at the 9th International Pragmatics Association (IPrA) Conference, July 10-15, Riva del Garda, Italy.
Noveck, Ira A. and Dan Sperber (Eds.). (2005). Experimental Pragmatics. (Palgrave Studies in Pragmatics, Languages and Cognition). Palgrave Macmillan.
Pilkington, Adrian. (1992). Poetic Effects. Lingua 87: 29-51.
Sperber, Dan, and Deirdre Wilson. (1986/1995). Relevance: Communication and Cognition. 2nd ed. Oxford: Blackwell.
Tanaka, Keiko. (1994). Advertising Language: A Pragmatic Approach to Advertisements in Britain and Japan. London: Routledge.

Asociación de Jóvenes Lingüistas
ajl2006@gmail.com