Kress on arbitrariness

Gunther Kress, in his book “Multimodality: A Social Semiotic Approach to Contemporary Communication”, asserts a lot of things about the nature of communication, but the one thing that struck me was that he challenged one of the fundamentals of linguistics that Introduction to Linguistics students learn on the first day.

That is, on pages 64-66, he argues against the arbitrariness of language. For those non-linguists reading this, or those who need a refresher, arbitrariness refers to the fact that there’s no inherent connection between the sounds and the words we use and the object in the world we’re trying to reference. That is, there is nothing about the sequence of sounds [t] [r] [i] (“tree”) that can somehow clue us in to the fact that we’re talking about the tall leafy branchy thing. Similarly, the construction of our sentences and our rules for making words are arbitrary – there’s no inherent reason that the subject comes before the verb in English, aside from “it’s the rule”.

Kress, in arguing against this, says:
“In Saussurian semiotics, if I want to be understood, I do so by learning the social rules of use of the semiotic resources which those around me know and use. If I don’t know them, I’m in trouble. In Social Semiotics, if I want to be understood, by preference I use the resources that those around me know and use to make the signs which I need to make. If I am not familiar with those resources, I make signs in which the form strongly suggests the meaning I want to communicate. Many of us have found ourselves in the latter situation and survived, using signs of gesture, of drawing, of pointing. Those signs however have to be as transparent, as iconic as I can possibly make them.”

He goes on to further argue that, “the signifier of ‘tree-ness’ is not a sequence of sounds, that is, not [a:br], but an existing lexical-item-as-signifier ‘tree’ used in its potential for becoming a new sign. The meaning-potential of the signifier ‘tree’ is the sum of all the instances in which I have encountered the sign ‘tree’ as signifier: that enables me to make a prediction about its aptness as a signifier for the new sign that I want to make now. The signified TREE and the signifier ‘tree’ are elements at the same level and of the same kind: not as in Saussure’s assumption where one is a semantic entity and the other is a phonetic one, one an entity of meaning and the other an entity of sound.”

It was somewhat difficult for me to parse these pages, but from what I can gather, Kress’s argument is this:
1. In terms of language, you can’t separate the signifier and the signified. That is, you can’t have some entity in the world TREE that has the name “tree”, and keep them separate. Once you name it, you connect the phonetic representation (the spoken language) with the meaning.
2. We know that “tree” means TREE because we’ve heard it a million (probably) times before in our lives. Therefore, if we know that the rule in this particular area of the world is that the spoken word “tree” means the tall thing with branches and leaves, then we know that because we’ve heard it many times before.
3. Perhaps the assignment of phonetic sounds to meaning is arbitrary, but after that it becomes a social rule to use these sounds with this meaning, and if there’s a social rule, it’s not entirely arbitrary. And since you can’t separate the referent and the name, you can’t separate the TREE from the social rule that says “hey, we call this thing a ‘tree’.”
4. Furthermore, the words we use aren’t entirely arbitrary either. They come from other languages, or past versions of our language. They bear similarities to language forms that came before them. Descendants aren’t arbitrary.

I’m sure many linguists have thought much about whether language is really arbitrary or not, and what “arbitrary” even entails. It was a useful exercise, for me, to read someone else’s thought process on it. I’m not sure I entirely buy it, especially the idea that you can’t separate the word from the thing in the world. There are plenty of things we don’t have words for, but that we know they exist. His point that hand signals are iconic, and therefore not arbitrary, also misses the mark with me – viewers have to know that you are trying to represent the tree with your hands and not just waving about and being silly. Although now this is getting me into thinking about the iconicity of gesture.

The moral of the story: question everything, even basic assumptions.

Also, that the science of linguistic meaning is deep, and once you step into the water, you can’t escape so easily…


Books – “Analyzing Multimodal Interaction” by Sigrid Norris

I have believed ever since I first studied discourse analysis that there’s a lot more to interaction than just the language and the talk. When linguists who study conversation make their transcripts, the large bulk of it is a documentation of the words said, with pauses recorded in milliseconds and maybe an indication of laughter.

Norris’s book shows us that there are many more communicative modes than just the verbal — and that these modes greatly affect communication. She challenges discourse analysts to take down notes about posture, head angling, gaze direction, use of objects in the environment, and to take into account all of these other modes of communication which may impact the direction of conversation.

For instance, she has a long analysis of a woman who is talking on the phone with a client at the same time as she is playing with her baby daughter. While some may see the playing with the baby as a distraction that impacts the flow of the conversation, Norris points out that her subject is actually managing the two behaviors quite regularly. When she addresses the client on the phone, she directs her gaze away from the baby towards an imaginary interlocutor, mediated by the telephone. When the client is talking to her, she takes that time to direct her gaze at the baby. The subject can then smile and make faces at the baby because she isn’t currently needing her mouth/lips/cheeks to talk to her client. This effective managing of gaze and facial gesture helps this particular subject manage two interactions simultaneously. The way that we humans do this almost subconsciously is incredible. We are really inherently social beings, and we have so many skills available to us to facilitate social behavior.

One of the most interesting things that I found in the book is the idea of modal density, which is (basically) how much we focus on one particular communicative mode in an interaction. For instance, when you are talking on the phone, you focus heavily on ther verbal component of interaction, because with the phone that’s all you have available. However, say you are out to lunch with a friend — you use your body position (facing the other person or looking away), objects in the environment (you might look at something or point it out and use it as a conversation topic), gestures and facial expressions, AND verbal conversation all at the same time, and all of these things have a component in interaction. As another example, say you are watching a movie with your significant other. There is likely little verbal interaction, but instead you are using a media device (the film) as a component in the interaction as being a shared object to orient your interaction around; furthermore, you have gestures, body position (you may have your arm around that person indicating a physical intimacy, or you may be sitting up next to each other, indicating some distance), and non-linguistic verbal cues like laughter or winces that are related to the shared object of interaction (the movie).

In thinking about this kind of modal density, I have thought about some of my own interactions with people. I am notorious for disliking conversations on the phone. I think now, having this knowledge, that it’s because of the heavy use of only one mode in interaction, that being the verbal mode; I am a person who likes very much to share experience and comment on observations in the environment. That’s why I like to meet with people in real life, go to shopping malls to spend time together, watch a sporting event, or hang out at a coffeeshop. I think that this even extends to computer-mediated communication, because online you can link another person to a news article or website that you want to talk about, or watch a YouTube video together, or play an online game.

This is exactly the kind of viewpoint that I was seeking when I first encountered discourse analysis and realized that there was something missing. Documenting everything that goes on is, of course, a lot harder than just documenting speech; however, it is important that we do so, because there’s more to communication than just language, and more to linguistics than language as well.