Start Submission Become a Reviewer

Reading: Towards understanding nonmanuality: A semiotic treatment of signers’ head movements

Download

A- A+
Alt. Display

Research

Towards understanding nonmanuality: A semiotic treatment of signers’ head movements

Author:

Anna Puupponen

Department of Language and Communication Studies (Sign Language Centre), University of Jyväskylä, FI
X close

Abstract

This article discusses a certain type of nonmanual action, signers’ head movements, from a semiotic perspective. It presents a typology of head movements and their iconic, indexical and symbolic features based on Peircean and post-Peircean semiotics. The paper argues for the view that (i) indexical strategies are very prominent in head movements, (ii) iconic features are most evident in enacting, while non-enacting description is less common, (iii) symbolic types for tokens are infrequent, although some movements—such as nodding and shaking the head—may become more conventional or schematized, and (iv) different types of head movements involve different proportions of iconicity, indexicality and symbolicity as well as different degrees of control in their production and interpretation. The treatment of head movements is extended to a discussion of semiotic versatility in the signification of actions of a signer’s body, as well as to the treatment of nonmanuals in the theoretical description of sign languages. Finally, the paper presents a perspective on nonmanuals in which different nonmanual cues are examples of how signification, and human cognition in general, are closely connected to the embodied experience of existing and navigating in the physical and social world around us.
How to Cite: Puupponen, A. (2019). Towards understanding nonmanuality: A semiotic treatment of signers’ head movements. Glossa: A Journal of General Linguistics, 4(1), 39. DOI: http://doi.org/10.5334/gjgl.709
139
Views
59
Downloads
11
Twitter
  Published on 19 Mar 2019
 Accepted on 05 Dec 2018            Submitted on 31 May 2018

1 Introduction

Signers’ head movements, body movements and facial expressions—i.e. nonmanuality—can be approached in several different ways. Since, in sign language linguistics, nonmanuality has been observed to organize texts and discourse (e.g. Wilbur 2000; Sandler 2012) along with the kinematic features of the movements of signers’ hands (e.g. Wilbur & Schick 1987; Brentari 1998; Wilbur 1999; Crasborn 2012), it has often been compared to, for example, spoken language prosody. On the other hand, different nonmanual elements have been analyzed as grammatical, and their functions have been associated with, for example, morphological and syntactic phenomena (e.g. Pfau & Quer 2010; Herrmann & Pendzich 2014) as well as with several aspects of interaction such as illocutionary acts, backchannelling, and emotive (Jakobson 1980) meanings (e.g. van der Kooij & Crasborn 2006; Puupponen et al. 2015; Schoonjans 2017). Finally, some nonmanual signals have been shown to be very similar to the bodily gesturing of speakers (see e.g. Schoonjans 2017; Johnston 2018). This may be taken as evidence for the view that signers’ and speakers’ nonmanual actions are based on the embodied features of human experience, action and interaction (e.g. Streeck 2015), and that this interaction consists of composite utterances in which nonmanual actions combine with manual and/or vocal actions (Enfield 2009; Kendon 2004).

In other words, (a) nonmanuality can be approached like the prosody of spoken languages, (b) nonmanual elements can be seen as parallel to grammatical units such as words and affixes, and (c) nonmanuality can be approached the same way as speakers’ bodily gestures in multimodal interaction. However, the above-mentioned approaches are in many ways overlapping and none of them alone is enough to exhaustively describe the role of nonmanuality in signed languages and communication. It has been widely acknowledged that different nonmanual and manual signals are simultaneously layered in a complex way in the articulation of signers (e.g. Wilbur 2000), that nonmanual signals are multifunctional (e.g. Pfau & Quer 2010; Herrmann & Steinbach 2011), and that distinguishing between prosodic, grammatical and gestural (e.g. affective) nonmanuals is in many ways difficult (Pfau & Quer 2010; Herrmann & Steinbach 2011; see also Puupponen et al. 2015). This raises the question whether categorical distinctions between “the prosodic”, “the grammatical” and “the gestural” actually exist in the nonmanual actions of signers.

I suggest here that the treatment of nonmanuals, along with many other phenomena in sign language linguistics, has been affected by preconceptual biases, brought about by structuralist semiotics and the criteria for natural language—criteria defined on the basis of written forms of spoken languages (see also e.g. Liddell 2003; Johnston 2013a; b; Dingemanse 2017; Ferrara & Hodge 2018). These biases result in an overemphasis on the symbolic status of nonmanual signals—an eagerness to demonstrate that certain nonmanuals have a categorical status as markers of grammatical phenomena similarly to words, morphemes or signs—and in the presupposition of links between prosodic aspects of speech and certain nonmanual signals.

I argue, further, that in the attempt to distinguish between “gestural” and “grammatical” nonmanuals, the argumentation has focused mostly on facial signals (so-called facial bias). As the expression of emotion is a common feature of human facial expression, this has resulted in associating “nonmanual gesturality” with the “signaling of affect” (see e.g. Herrmann & Steinbach 2011; Herrmann & Pendzich 2014). This oversimplifies both the concept of affect1 as something gestural and the concept of nonmanual gesturality as something affective. In relation to the former, affect may be expressed, for example, with phonemes, morphemes, morpho-syntactic constructions, whole sentences, prosodic features (e.g. pauses, stress or intonation), and actions of different parts of the body. With regard to the latter, nonmanual gesturality (like speakers’ hand gesturing, see e.g. Ekman & Friesen 1969) is not only affective: gradient and unconventionalized nonmanual signals that may be associated with the co-speech gesturing with the face, head and body, may also enact or otherwise signal other types of information than emotion.

To continue along these lines, I argue that the notion that there are conventionalized and categorical nonmanuals (e.g. grammatical nonmanuals) that can be clearly distinguished from gradient, uncategorical nonmanuals (e.g. affective signals) is (i) premature; (ii) mainly based on neurolinguistic perception studies of the actions of the signer’s face (e.g. Corina 1989; Corina et al. 1999; McCullough et al. 2005); (iii) that there is not enough empirical evidence of whether such categorical distinctions can be found in the actual language use of signers; (iv) that such presumptions may result in the hand-picking of specific nonmanual actions as relevant subject matter for linguistic analysis of SLs while ignoring others; and (v) that the results of some recent studies on, for example, signers’ mouth actions and head movements do not confirm this type of theoretical distinction (Johnston et al. 2015; Puupponen et al. 2015; Johnston 2018). Finally, I suggest that these (pre)conceptual biases already emerge as a so-called manual bias in the way we refer to the actions of signers’ face, head and body, that is, in the use of the term NON-manuals as a conceptual trash bin for ‘everything else besides what is done with the hands’.

As an attempt to step away from such conceptual inconsistencies, this article aims to give a theoretical description of one type of nonmanual activity—signers’ head movements—from a wide semiotic perspective.2 More specifically, the paper seeks to apply C.S. Peirce’s (1894; 1903a) semiotic theory of signs to the description of signers’ head movements. This means that head movements are not classified as prosodic and/or grammatical and/or gestural. Instead they are seen as semiotic signs that connect to their objects in different ways, and which therefore require different types of interpretations. The motivation behind this approach is, firstly, that it allows one to discuss the significance of head movements without differentiating between “language” and “paralinguistic”. Peirce’s semiotics offers a framework which does not have its origins in a particular system of signs, such as natural language, and therefore does not differentiate between linguistic and non-linguistic signs. In this way, the level of abstraction in Peirce’s semiotics is well suited to a relatively uninvestigated topic such as signers’ head movements and their functions in signed interaction. It does not force one to exclude from the analysis elements which signify but may not form, for example, conventionalized form-function pairings; that is, that may not be considered linguistically significant from the point of view of structuralist semiotics (e.g. Saussure 1916). A sign may be an element that connects to its object through analogical association of qualities or through a physical spatio-temporal connection in a given instance, such as proximity of timing, and it may even be a unique single event (see Section 2). This includes the notion that meaning is not a static and arbitrary connection between a signifier and a signified but instead a dynamic and motivated phenomenon (Hanks 1990; Enfield 2009).

Secondly, Peirce’s semiotics is very suitable for investigating a phenomenon that includes a lot of social and situational variation. It has been shown that the ways in which signers move their head while signing varies a great deal between different signers and different discourse strategies (Puupponen et al. 2016; Jantunen 2017; Puupponen 2018), and that the movements of individual signers do not form distinct formal categories but, instead, head movement types such as a nod or a thrust may be described as prototypes that the actual head movement events in the discourse resemble more or less (Puupponen et al. 2015). In actual language use, movements occur that share features with, for example, both the prototypical nod and the prototypical thrust, and which therefore are examples of the gradient periphery between different head movement types (ibid.). All in all, head movement types—sequences of movement such as nods or shakes of the head—are un-categorical and multifunctional and show a lot of variation both in form and in form-function patterning. In Peirce’s semiotic theory, a sign becomes a sign through the process of semiosis, in which it need not be interpreted according to the rules of some surrounding system (Nieminen 2010). Which interpretation emerges depends on the situation in which the sign is observed, and it can be unique. Therefore, variation in how signs stand for their objects becomes a default feature rather than an exception to a rule (see Section 2).

Although Peirce’s semiotics may not be the most common framework for the analysis of different elements in sign languages, the discussion on the iconic, indexical and symbolic features in head movements, presented in Section 3, is connected to several previous theoretical notions on the nature of signification put forward in sign language research. Discussion of, for example, the lexicons of sign languages (e.g. Johnston & Schembri 1999), strategies in signaling meaning in sign languages (Ferrara & Hodge 2018), and mouth actions in Australian sign language (Johnston et al. 2015), have been connected to Peircean semiotics. In addition to Peirce’s semiotics, the treatment of signers’ head movements in the current paper relies on the more recent discussions on semiotics in linguistic anthropology, psycholinguistics, cognitive linguistics and gesture studies (e.g. Parmentier 1994; 2006; Clark 1996; Kendon 2004; Kockelman 2005; Enfield 2009).

This paper presents a theoretical view—a semiotic typology—of signers’ head movements and discusses their iconic, indexical and symbolic features using examples from a sign language corpus. In addition, it discusses the semiotic versatility in the signification brought about by the actions of the signer’s body. The examples and discussion on head movements draw from work on a variety of data, including corpus narratives and dialogues, other semi-structured monologues, and dialogue motion capture recordings of signers of Finnish Sign Language3 (FinSL) (see e.g. Puupponen 2012; Puupponen et al. 2014; 2015; 2016; Jantunen et al. 2016b; Puupponen 2018). In these studies, the analysis was done within the usage-based cognitive-functional framework, and nonmanuals were approached according to their forms and functions in actual communication. The analysis of head movements was based on the view that language is a part of the overall physiological, cognitive and social activity of humans, and therefore that language is inherently connected to other types of human action. The same approach is taken in the current paper. Furthermore, language is seen as including both conventionalized and distinct elements as well as gradient, un-categorical, and unconventionalized features. Conventions in symbolic form-meaning pairing and the schematization of grammatical phenomena are considered to emerge in the language as the result of patterns that develop through frequency of use (see Hopper 1998). Finally, although all the examples in the paper come from FinSL, it is presumed that many of the conclusions reached here about the nature of head movements could also be applied to other sign—and in some respects spoken—languages, due to the common physiological and psychological ground in human interaction. This issue is discussed further in Section 6 of the current paper.

The paper is organized as follows: first, in Section 1.1 a short overview is given of the data and analysis from which the current paper draws, followed by Section 2, which presents Peirce’s Theory of Signs and some post-Peircean semiotics, insofar as it is relevant for the discussion of the head movement typology presented in Section 3. The following sections discuss the role of head movements in relation to the semiotic versatility of the signification of signers’ bodily actions (Section 4), the implementation of nonmanuals in a theory of language (Section 5), and the connections between head movements and the embodied human experience of being in the world (Section 6). Final conclusions are presented in Section 7.

1.1 An overview of the data

The data used as the basis of the current paper includes corpus narratives and dialogues (CFINSL), pre-structured monologue data (SLM), and motion capture dialogue data (MOCAP), all collected or processed at the Sign Language Centre in the University of Jyväskylä, Finland. The corpus narratives consist of altogether 12 retellings of two picture-books, The Snowman, and Frog, where are you?, by signers between the ages of 20 and 60, recorded in a dialogue setting with multiple cameras (e.g. Jantunen et al. 2016b; Puupponen et al. 2016; Puupponen 2018). The overall duration of the material is 45 minutes and 12 seconds and it has been annotated in ELAN for manual signs and sentences, syntactic structure, head movement types, and torso movement types (Jantunen et al. 2016b; Puupponen et al. 2016; Puupponen 2018). From the 12 narratives, the interplay between head and torso movements has been annotated and analyzed for a subset of 6 narratives, and the functions of these overlapping torso and head movements has been annotated for a subset of 4 narratives (for a detailed discussion, see Puupponen 2018). The corpus dialogue data consists of altogether 6 dialogues from the same signers as in the narratives. In these dialogues, the signers discuss their experiences and thoughts on events and other matters related to the deaf world. The dialogue data has been annotated for manual signs and sentences. The examples from the dialogue data used in the current paper have been identified by means of an overall visual examination of the data, and with the search functions available in ELAN.

From the other semi-structured monologues, the discussion in the current paper focuses on a 1 minute 8-second-long stretch of video material, recorded by the Finnish National Association for the Deaf, in which a native FinSL signer discusses the relationship between young deaf people and the activities of the traditional national deaf associations. The data (consisting also of computer-vision based motion analysis) has been annotated for manual signs and sentences, as well as for head movements and their functions (see Puupponen 2012). Finally, in relation to the dialogue motion capture recordings, the data consists of altogether 2 minutes 15 seconds of synchronized motion capture data and digital video material of two continuous dialogues in which two FinSL signers talk about their work, studies, and everyday language use. The data has been annotated for manual signs, head movement types and functions of different head movements, and the analysis included both a detailed description of the different forms of head movements on the basis of the quantitative motion capture data, as well as an analysis of the form-function pairing of different types of head movements (a detailed description of the data, annotation and analysis can be found in Puupponen et al. 2015).

2 The semiotic framework

2.1 Universal categories & semiosis

C.S. Peirce’s semiotic theory (e.g. 1894; 1903a) forms the basis of the semiotic framework for the discussion on signers’ head movements in the current paper. It deals with all kinds of signification, that is, the process of how things become signs for something else. According to Peirce, knowledge, and the world in general, are comprised of signs. Peirce’s conception of signification is based on so-called universal categories, a phenomenological triadic distinction that forms the basis of his philosophy (Peirce 1903b). According to Peirce, different aspects of reality can be described on the basis of how things exist experientially. The first of the categories, Firstness, is feeling itself: a raw experience of something without thought or conscious processing ([Quality of] Feeling). It is an experience that is not compared, proportioned or connected with any other issue; it is a sort of observation of a quality without a relation to another entity, for example an experience of color. (Peirce 1894; 1903b.) Secondness (which he also called Reaction or Struggle) is an element in which two things are in an influential connection with each other. It is a feeling that materializes in relation to another feeling, such as a reflex-like motion emerging because of a stimulus, or an eye-blink emerging in order to moisten one’s eye. According to Peirce, things in our experiences and consciousness come true as these types of relations, while single raw feelings exist only when we are not fully awake. (Peirce 1894; 1903b.)

The third category, or Thirdness, is thinking: a conscious process in which one learns that something is controlled or governed by a rule or a habit. Thirdness is a tripartite relation in which two things are connected through a third thing, which is a habit or a law. Signification is always an example of thirdness. Something stands for some other thing, and in so doing, conveys an idea of this other thing. For example, an eye-blink, the physical event caused by the necessity of keeping one’s eye moist and clean, becomes a sign when it co-occurs with the boundaries of signed discourse and therefore functions as one of the signals that help us perceive discourse structure. Only the elements of thirdness are intelligent, non-mechanical processes that include reasoning, signs and learning. (Peirce 1894.) Actually, according to Peirce, human cognition—thinking, learning, rationality—is in itself an act of signification (it comes true through signification) and therefore an example of the third universal category.

The core concept of Peirce’s signification is semiosis, the event involved in the interpretation of a sign. Semiosis is a tripartite relation which consists of the sign, the object, and the interpretant. In semiosis, things become signs because they are interpreted as standing for something else, their objects (Peirce 1903a).

“I define a sign as anything which is so determined by something else, called its Object, and so determines an effect upon a person, which effect I call its interpretant, that the latter is thereby mediately determined by the former.” (Peirce 1903a: 478.)

In Peirce’s semiosis anything may be a sign. A sign is a sign only through this process of interpretation, because of the fact that it ends up being interpreted as a sign of something. In semiosis, an entity or an event, such as a flash of lightning, becomes a sign when it is interpreted as standing for its object, a thunderstorm. As Kockelman (2005) points out, the social practice of joint attention is another example of this. In a situation in which a child turns their attention to what their parent is observing, the change in the child’s attention is the interpretant, the direction of the parent’s attention is the sign, and the thing that the parent is attending to is the object (Kockelman 2005). Nieminen (2010) argues that identifying different forms of a language is in itself already an interpretation of a sign in the Peircean sense, and noticing and distinguishing boundaries in the speech signal are examples of semiotic processes, even though these signs would or could not have a meaning in the Saussurean (1916) sense. In other words, signs may signify without being interpreted as meaningful. Some signs do not have a static, conventionalized way of being interpreted as standing for some precise event or entity. They do, however, connect to other signs to form intentional and meaningful social actions (Enfield 2009). The characteristics of a semiotic sign will be discussed in more detail in Section 2.2.

The second part of Peirce’s semiosis, the object, is what the sign stands for (Peirce 1894; 1903a; Kockelman 2005). As was pointed out above, an object may be something perceivable, such as something a person is directing their attention to (Peirce 1903a; Kockelman 2005). On the other hand, according to Kockelman (2005: 242), it may be something less precise and more abstract, such as something that directs us to a certain behavior (e.g. stroking a cat) once we have observed a sign (e.g. the cat’s purring sound). The purring stands for something, but that something is not as easy to determine as, for example, the object of a pointing action would be. The object may emerge as a synthesis of possible interpretations, such as seeing the purring of a cat as a sign for a purpose which the cat is trying to achieve (e.g. it wants the stroking to continue) (Kockelman 2005). Kockelman (2005) continues that objects may be types which emerge through different contexts of use (e.g. a proposition of an assertion) or they may be tokens in particular contexts of use (e.g. the state of affairs in a specific assertion). Peirce himself also distinguished between so-called immediate objects and dynamic objects. Peirce’s immediate object is an object that does not exist independently of the sign; it is the representational object of the sign in a certain sign event, that is, a token (Peirce 1906; Kockelman 2005). A dynamic object is independent of the sign and it determines whether the sign exists. It is not dependent on a particular sign event, and so it is a type rather than a token (Peirce 1906; Kockelman 2005). Type and token are important notions also in relation to the characteristics of a semiotic sign, which will be discussed further in Section 2.2.

The third part of the triadic Peircean semiosis, the interpretant, is what happens in our minds when we experience that a sign stands for its object (Peirce 1903a). It is what our mind does as we “read” or “translate” the sign, that is, when we realize that one thing stands for something else. (Peirce 1903a; Atkin 2013.) Kockelman (2005: 251) points out that the Peircean concept of interpretant does not have to be mental or verbal, instead it may be “embodied in actual behavior”. According to Kockelman, these embodied interpretants may be bodily processes without any physical effort or commitment. They are changes in bodily state, such as a feeling of pain or blushing. These interpretants are themselves often perceived as signs by the person experiencing them, or by someone else who is witnessing them (Kockelman 2005). For example, feeling pain while burning one’s finger on a hot stove is an embodied interpretant of the sign, the heat from the stove. This feeling of pain is at the same time an interpreter and a sign. The feeling is simultaneously an effect from a previous sign (the thing causing the pain) and a sign for the next event, a reflex-like hand motion away from the heat. The motion event is a new interpretant which includes action. And so on. According to Kockelman (2005: 239), “most objects and interpretants are themselves signs – and so the three-fold relationality continues indefinitely: every component of one third is simultaneously (and/or sequentially) a component of another third.” Other embodied interpretants may be energetic behavioral processes or representations. The former “involve physical or mental effort”; people cause them to happen, without necessarily involving “purpose, intention or planning”, like, for example, stretching one’s neck in order to see the source of a loud sound. The latter involve signs with propositional contents or thoughts, such as the concept underlying the word ‘cat’ (Kockelman 2005).

In Peirce’s semiosis, signs are first recognized as signs, after which the function or meaning of the sign is gradually defined in the process of semiosis, which moves towards its final interpretation. This process may include several stages, in which the object of the previous stage (the immediate object) emerges as a sign for the following stage of interpretation, and which, in the end, reaches the final stage (the dynamic object). However, this process may not be linear or straightforward and it need not reach a final conclusion. All signs do not have an inherent characteristic that ensures that they are interpreted in a certain way (Peirce 1903a; Kockelman 2005; Nieminen 2010).

2.2 Sign

Peirce’s classification of signs—in its many versions—may be the best known part of Peirce’s semiotic thinking. The discussion of head movements given in the current paper has its roots in Peirce’s first and interim typologies of signs, published in 1894 and 1903. The interim typology is often regarded as the most complete and coherent presentation of his classification of semiotic signs. (e.g. Atkin 2013). In this typology, signs are defined according to three dimensions, which are based on Peirce’s three universal categories of experience: (i) what signs are themselves, (ii) what signs are when viewing the connection between signs and their objects, and (iii) what signs are on the basis of their interpretant. (Peirce 1903a.) For the purposes of the current paper I will briefly present dimension (i), after which the discussion will focus on dimension (ii) which will be referred to as the ground of the sign and which is the basis of the discussion of signers’ head movements in Section 3 of this paper.

When describing dimension (i), Peirce (1903a) defined signs as qualisigns, sinsigns or legisigns. A qualisign is a quality, a characteristic that may perform as a sign. An example of a qualisign is the quality of a color, such as the property of ‘redness’. Actually, a qualisign is not really a sign, and it does not function as a sign in reality. Instead it is an abstract quality which is potentially a sign, such as an impression of a movement that might be a sign of a sign language. When a qualisign manifests and is interpreted as a sign, it always becomes a sinsign. A sinsign is a single actually existing entity or event which functions as a sign. Although a sinsign is not just a quality—like a qualisign—it is a sign because of its qualities, that is, because of its qualisigns. An example of a sinsign would be, for example, a color sample. Finally, a legisign is a law or a convention that functions as a sign. It is a general type that has been established as significant, and which manifests as replicas, tokens of that type. These replicas are sinsigns of the legisign. According to Peirce, single instances of a certain word in a text are all occurrences (i.e. sinsigns) of the same word (i.e. legisign). A replica is a sinsign which is significant because of the law that creates the legisign. It is not an exceptional incident which becomes significant for other reasons, as are other types of sinsigns. (Peirce 1903a; Parmentier 1994; Kockelman 2005; Nieminen 2010.)

Let us look at three different signs: a word that has been uttered, a national flag, and an empty bucket close to wild blueberry bushes in the forest. Of these, an uttered word and a national flag are based on a shared norm: the sign is connected to its object through a convention. A particular word such as ‘cat’ uttered at a given moment (i.e. a sinsign) is a token (i.e.replica) of a more general type (i.e. a legisign): the normative connection between any instance of the word and its shared meaning or function (Peirce 1903a; Kockelman 2005). However, all sinsigns are not replicas: they do not have a type. The bucket in the forest may become a sign when interpreted as indicating that someone is planning or has at some point planned to pick the berries nearby. It can be interpreted as a sign for someone not far away from the location of the bucket (‘someone is coming back soon to continue with this task’). On the other hand, it might result in the thought that the bucket has been left there by accident (‘someone has forgotten their bucket’) or in concern (‘I hope nothing unexpected and serious has happened to the owner of this bucket’). The interpretation of the sign depends on the situation and may vary according to the interpreter: there is no rule or norm for it to be interpreted in one particular way. It is a single token without a type. In addition, a bucket in itself, if put into another context, does not invoke the same interpretation; it does not necessarily result in any interpretation of signification. If it is interpreted as standing for something else, it is a singular event which functions as a sign (Peirce 1903a). Kockelman (2005) calls these types of signs singularities and goes on to say that most sign events are singularities, and that semiosis is actually an inductive, rather than a deductive process. That is, the meaning of a type is induced through a token, not the other way around; or the meaning of a token emerges from other co-occurring tokens, potentially abstracting to a type (Kockelman 2005).

2.3 Ground of the sign: Iconicity, indexicality and symbolicity

In relation to dimension (ii), Peirce (1903a) defines signs as icons, indices and symbols on the basis of the connection between the sign and its object, that is, the ground of the sign. An icon is a sign that represents its object through its qualities. An icon and its object share these qualities, that is, an icon resembles its object in some way. An example would be a good portrait of a person. (Peirce 1903a.) An index, on the other hand, connects to its object primarily through an actual connection, contiguity in time and space, such as an effect or an impact (Peirce 1903a; Kockelman 2005). According to Peirce, this connection does not induce an analogical association due to the similarities between the sign and its object, as with iconic signs. Instead, an index is physically connected to its object. This connection is only noticed; interpretation emerges when the physical connection is observed to be conventional (Peirce 1903a). Examples of indexical signs would be smoke (as an index of fire), the sound of rain, a footprint, a pheromone, or a weather-vane. The physical connection between a sign and its object is what makes an index a sign, although indexical signs may have iconic features as well (Peirce 1903a). For example, a sign such as the aforementioned weather-vane includes the association of the similarity of the direction of the object’s orientation and the direction of the wind (i.e. iconicity) (Parmentier 1994; Kockelman 2005). Finally, a symbol represents its object through a law-like relationship and is therefore connected to it by means of a convention. Examples of symbols are the lexical words or signs of a natural language. From this conventionality it follows that symbols denote that something is of a certain kind, rather than indicating a single particular thing (Peirce 1894). According to Peirce (1903a), symbols are always types, generalizations of single entities. This means that symbols are always legisigns which manifest as replicas, and the object of a symbol is more general than specific.

According to Peirce, icons, indices and symbols are essential for human cognition and reasoning (Peirce 1984). However, Peirce describes these different representative types as features rather than categories. Peirce (1984: 10) states that in reasoning we must always “use a mixture of likenesses, indices, and symbols”. Furthermore, iconic, indexical and symbolic features may, and usually do, coexist in single signs. Already in his early account of semiotic signs, Peirce (1894) points out that a single sign may simultaneously be an icon and an index, and that symbols often have an imitative origin. An icon does not necessarily have indexical or symbolic features although this is possible, whereas an index must have iconic features (i.e. information about or qualities corresponding to its object), and a symbol must have iconic and indexical features (i.e. directing attention and providing information concerning it or indexing a code) (Peirce 1903a; Silverstein 1955; Parmentier 1994; Kockelman 2005). This same inclusivity can be found in Peirce’s Universal Categories: Secondness includes Firstness, and Thirdness includes both Firstness and Secondness. As Kockelman (2005: 246) points out, because of these inclusive relations “it is best to talk about iconic, indexical, or symbolic grounds, rather than to talk about icons, indices, and symbols per se”.

Let us consider one frequent example of this connectedness, a footprint. One can easily see how a footprint in the sand has iconic, indexical as well as symbolic features. The indexicality of the footprint lies in its physical connection to its source: the weight of a person’s foot leaving a trace in the sand. The indexicality in the sign links an event, i.e. someone walking, to something in time and space, i.e. that the walking occurred in that particular location. In addition, the indexicality links the sign to the foot itself, to the person who left the trace behind. This type of spatial, temporal and causal contiguity forms the indexical ground of the sign (Peirce 1903a; Kockelman 2005; Enfield 2009). On the other hand, identifying the similarity between a trace on the ground and a characteristic of its source is based on iconicity: association between the qualities of the sign, i.e. the shape and size of the footprint, and the qualities of its object, i.e. the shape and size of a foot. This association is a key characteristic in what makes a footprint a footprint, in just the same way as is the physical phenomenon through which it exists. Finally, although we may see only parts of a footprint on muddy ground, on asphalt, or on a beach – left by a shoe, a bare foot, or high heels – we still might recognize what it is: we are guided by our knowledge (which is based on convention) of what footprints are, how they are formed and what they look like (and that they are often imperfect, so to speak). The different semiotic grounds of the sign come together when we use and interpret signs, and construct meaning.

In the following discussion on signers’ head movements, with iconicity I refer to a relation between the sign and its object which is based on perceptual analogies (see Peirce 1903a; Parmentier 1994; Kockelman 2005; Enfield 2009; Dingemanse et al. 2015). Indexicality is defined as a relation between the sign and its object that is based on spatial, temporal or causal contiguity (see Peirce 1903a; Parmentier 1994; Kockelman 2005; Enfield 2009). Symbolicity is referred to as a relation between the sign and its object that is based on a norm in a community according to which the sign is regarded as standing for the object (see Peirce 1903a; Kockelman 2005; Enfield 2009). All of these three types of signification are referred to as the grounds of a sign or semiotic strategies. The paper makes no distinction between these two terms. Conventions and automatization emerging in one person’s communicative actions are referred to as entrenchment, while socially shared conventions are referred to as conventionality (e.g. Wilcox & Xavier 2013).

2.4 Utterances as actions: Semiotic complexity and pragmatic unity

If we broaden our perspective from single semiotic signs to whole utterances, intentional social actions occurring in discourse, it is evident that they are semiotically complex, especially if one counts the signals from different sign mediums—such as speech, movements of the body etc.—as parts of one and the same utterance (e.g. Kendon 2004; Enfield 2009). According to Enfield (2009: 15), in a communicative action, signs from different mediums come together into a “communicative move that incorporates multiple signs of multiple types”, which he calls a composite utterance. Some of these signs are more conventional (e.g. words, emblems), some unconventional (e.g. pitch, depicting hand gestures), while others are hybrids of conventional and unconventional features (e.g. pointing actions). The interpretation of these holistic chunks of semiotic content happens through “recognition and bringing together of these multiple signs” while presuming “pragmatic unity despite semiotic complexity” (Enfield 2009: 11). That is, although utterances are semiotically complex and consist of signification done simultaneously and multimodally with different sign mediums, these different layers of signification are connected and come to exist together as parts of one event which is driven by intentional social action (Enfield 2009). While investigating signals such as head movements in signed interaction, it is important to acknowledge that their meanings and functions do not emerge, and are not interpreted in, isolation. They are a part of multimodal social actions of communication in which different semiotic resources are available.

In order to maintain clarity in the argumentation regarding the semiotic features of signers’ head movements and the semiotic versatility in signers’ communicative actions, I wish to make the following terminological and conceptual distinctions: physical medium, sensory modality, sign medium (i.e. modality of information), and semiotic dimension (applied from e.g. Kress and van Leeuwen 2001; Bernsen 2002; de Ruiter et al. 2003; Enfield 2009). With physical medium I refer to the medium of phenomena such as light, movement and space, which are relevant for our sensory systems and sensory, mental and social activities. With sensory modality I refer to sensory systems such as vision, hearing, touch and balance. The term sign medium will be used to refer to the actions of different body parts: the hands, face, head and torso.4 With sign medium Enfield (2009) refers to modalities of semiotic information such as speech, hand gesture, eye-gaze and so on. For the purposes of the current paper and its approach to language (see Section 1), gesture is not considered to form a coherent sign medium, and it is not differentiated from the “linguistic component” (cf. Enfield 2009: 13). Finally, with semiotic dimension I refer to those features in signals of a specific sign medium that can independently vary simultaneously with other dimensions (Kress and van Leeuwen 2001; Enfield 2009). These could be, for example, the loudness of sound (which cannot be loud and quiet simultaneously) or the amplitude of the displacement of a hand movement (which cannot be a long and a short distance simultaneously). Different parts of a signer’s body may produce a variety of movements and positions (i.e. potential signs in different sign mediums) using additional kinematic features available for signification, such as velocity and displacement amplitude (i.e. semiotic dimensions).

3 Iconicity, indexicality and symbolicity in head movements

In this section I will present a typology of signers’ head movements according to the semiotic framework presented in the previous section. The discussion will focus on the grounds of different types of signs (i.e. head movements), that is, in the iconic, indexical and symbolic connections between the signs and their objects. As is the case with the underlying theory, iconicity, indexicality and symbolicity are seen in the discussion as connected and overlapping semiotic features of head movements. Head movements of a certain type may therefore show iconic, indexical and symbolic features all at the same time. The main characteristics of the typology are presented in Figure 1 with relation to the different semiotic strategies (i.e. iconicity, indexicality and symbolicity).

Figure 1 

A typology of head movements according to semiotic strategies of iconicity, indexicality and symbolicity.

According to the typology illustrated in Figure 1, all head movements involve iconicity, indexicality and symbolicity, but in different types of movements some semiotic strategies are especially prominent and central for interpretation. This is demonstrated by the stacking of certain types of head movements to the proximity of strategic “poles” of iconicity, indexicality or symbolicity. The following sections discuss the iconic, indexical and symbolic features in head movements (3.1–3.3) and the different proportions of these strategies in different types of movements (3.4).

3.1 Iconic features in head movements

According to the typology in Figure 1, the iconic strategy is important in head movements which resemble their objects either through enactment or metaphorically (see Figure 1). When a head movement has an iconic relation to its object, its interpretation involves association of the similarities between some characteristics of the sign and some characteristics of its object. Enacting head movements, such as the one in Figure 2, mimically enact the head movements of a discourse referent. Figure 2 presents an example of an utterance in which the head movements represent the head movements of a character in a story (a boy). While in the beginning of the example it is debatable whether the head tilt enacts the actions of a referent, the subsequent sideways rotations of the head (i.e. head turns left and right) are clearly enacting movements depicting the actions of the referent (looking left and right). During the enacting head turns, the hands produce the indexical verbal sign LOOK, also directed to the left and right side of the signing space. The indexicality of manual signs is briefly discussed in Section 4.2 of the paper.

Figure 2 

An example of pantomimic head movements (head turns). CFINSL_005_05_00:00:35.5

Enacting head movements are signs that connect to their objects most prominently through iconicity. They are a part of an embodied depiction of the thoughts, sayings or actions of discourse referents, in which the content is enacted from the referents’ point of view—a discourse strategy often referred to as constructed action (e.g. Hodge & Ferrara 2014; Cormier et al. 2015). In the enactment, the actions of referents are projected in a size and manner that resembles real life (e.g. Ferrara & Halvorsen 2017), which in the case of human referents equals the embodied, visual and haptic engagement of human beings in the world. This is in contrast to observer-viewpoint description with fully or partly lexical signs (Johnston & Schembri 2010; Johnston 2012), in which actions are presented as distanced, small-scale description without enactment (e.g. Ferrara & Halvorsen 2017; Ferrara & Hodge 2018). These two strategies may also overlap. As depiction involves actions of different parts of the signer’s body, the hands may produce descriptive contents from the observer-viewpoint while other parts of the body are involved in enactment, as is the case in the example in Figure 2. In some recent studies discussing the different degrees of enactment (Cormier et al. 2015; Jantunen et al. 2018a; b), utterances consisting of both observer-viewpoint description and enactment are referred to as reduced or subtle enactment, whereas overt enactment refers to enactments in which hands are involved in the mimic depiction. Furthermore, formally similar or identical manual actions may emerge in either observer-viewpoint description or enactment, depending on the situation (e.g. Johnston & Ferrara 2012). Finally, in enacting movements, the direction of the head movement or the orientation of the face may indicate the relations and semantic roles between discourse referents.

The second type of iconicity in actions of the head, metaphorical iconicity, emerges, for example, in those head movements in which the direction of the movement is associated with an abstract image of a sagittal time line (Figure 3). In this case, forward and backward directed movements of the head or the whole body refer to the linear organization of things in time. Content associated with the future, or with subsequent events in the organization of things (‘later’, ‘after’), is produced with movements directed forward, whereas meanings related to the present, the past or anterior (‘earlier’, ‘before’) are produced with movements directed backward. I suggest that in these movements there is an iconic connection between specific features of the sign, i.e. the direction of the movement, and its object. The object in this case is a metaphorical spatial conceptualization of time (see Cooperrider & Nunez 2009): a visual and embodied mental image, common in the Western world, according to which time is spatially mapped, with the ego as a reference point, so that the future is something ‘in front of us’ and the past ‘behind us’ (Cooperrider & Nunez 2009; see also Lakoff & Johnson 1980; Wilcox 2000; Taub 2001; Cienki & Müller 2008). These metaphorically iconic sagittal head movements are optional. They may emerge in utterances with deictic signs, such as the FinSL signs FUTURE, and NOW, with manually produced indexical time-lines, or the temporal organization of events may be expressed without any particular head movements. At present, we do not know how frequent they actually are in utterances with time references, and whether they can alone anchor manually signed content to conceptual time-lines, without manual indexicality.

Figure 3 

An example of a metaphorically iconic head movement (head pull). CFINSL_005_06_00:03:56.

However, according to earlier research on head movements in FinSL (Puupponen et al. 2015), the connection between the direction of a head movement and the linear organization of things in time is not categorical. The surrounding discourse context is a relevant factor in how head movements are directed: the movements may be coincidental, caused by kinematic features of manual movements (Woll 2009; Puupponen et al. 2015), or the contrast between ‘future’ and ‘past’ or ‘later event’ and ‘earlier event’ may be emphasized with movements produced contrariwise (i.e. ‘future’ backwards, ‘past’ forwards). In this case it is indeed this contrastive function which is relevant in the movements: two or more things are placed in opposite, alternative, comparative, or corrective relations to each other (see Johnston 1992; Wilbur & Patschke 1999; van der Kooij & Crasborn 2006; Crasborn & van der Kooij 2013; Puupponen et al. 2015; Jantunen 2016; Lackner 2017). I suggest here that when the contrastive function of head movements is not connected to a metaphoric concept of a sagittal time-line, its source of iconicity changes (see section 3.4) and the indexical ground becomes increasingly relevant. Contrastive head movements include indexical features of organizing the discourse structure and creating a spatial relation between the contrasted manually signed contents, which is discussed further in Section 3.2.2. It should also be noted that the indexical placement for showing opposition, alternatives etc. may be considered to include iconicity, that is, analogical association (e.g. opposites visualized at opposite sides of the signing space). Finally, metaphoric time-lines in sign languages are not only sagittal: the organization of events in time may be visualized also horizontally or circularly, as has been noted also in the manual gesturing of speakers from several different cultural surroundings (for a summary, see Cooperrider & Nunez 2009). However, in the data from which the current paper draws, head movements have not been found to iconically represent these other time metaphors.

In sum, the iconic strategies in signs produced with the head are most prominent and crucial in enactment, although iconicity is more or less involved in the production and interpretation of other types of head movements as well (see section 3.4). Enacting movements vary in form according to the needs of the depiction: movements and positions of the head imitate their object, that is, the activity of the head of a discourse referent, rather freely. They are often one part of a holistic iconicity in which their meaning emerges only in relation to the actions of other parts of the body: the body, head, and face form a unified depiction that is interpreted as a whole, and the head movement alone cannot be seen as having an independent meaning (head movements and the holistic nature of signers’ actions are discussed in Section 4.1). However, enactment need not involve all the aforementioned sign mediums.

Non-enacting iconic head movements are, for example, movements visually representing a time metaphor. In this case the central imitating feature was found to be the direction of the movement. Together with the direction of the manual movements, the movements of the signers’ body embody and visualize the imagistic dimensions of human temporal reasoning. On the basis of preliminary examination of the data, it is suggested here that head movements which have non-enacting iconic features are less common than enacting movements. However, this observation is given as a hypothesis for further investigation as it is not based on frequency counts.

3.2 Indexical features in head movements

3.2.1 Indexicality of head movements in enactment and visualized time metaphors

Enacting and time-metaphor movements of the head, presented in the previous section, also have indexical relations to their objects. As was mentioned above, enacting head movements may have indexical functions that point out semantic relations between referents. Orientation of the head and face, direction of gaze, and direction of manually produced enactment/description together indicate spatial relations between referents (e.g. Hodge & Ferrara 2014; Schembri et al. 2018). Head movements indicating referents are discussed in more detail in the next section (3.2.2). In general, the communicative action of enacting a referent can be considered to involve indexical as well as iconic signification. When the enactment involves a so-called invisible surrogate (Liddell 2003)—that is, a referent, for example an inanimate object, is handled by an enacted referent without any explicit visualization of the object that undergoes the handling—characteristics in the form of the enactment are affected (or according to Enfield 2009determined) by the characteristics of the referent. In the gesturing of speakers, Enfield (2009) sees this type of enacting as an example of spatio-temporal continuity (i.e. indexicality) between the sign and its object.

Metaphorically iconic head movements that are connected to an abstract image of a sagittal time-line are also clearly indexical. The movement of the signer’s upper body or head is an index, functioning as a pointer referring to a “vague temporal region” (Cooperrider & Nunez 2009); it indicates temporal organization and the alignment of content in the discourse. Referring to a later or an earlier event always includes a relation: the event in time is presented in relation to a temporal reference point, an origo, which in this case is the ego. If one wants to emphasize with a head movement that the content that is being signed is placed in a later time or in the future, the forward directed movement holds the presumption that the origo is somewhere “behind it” in the present or (more indefinitely) in an earlier time. Temporally deictic words and signs in natural languages, such as the English adverbs ‘now’ and ‘yesterday’ or the aforementioned FinSL signs FUTURE and NOW, are traditionally recognized as indexical (although also symbolic) signs which include situating referents in temporal relations. Head movements connected to time metaphors are, however, different in their indexicality: they are not used alone to create temporal relations but, rather, visually emphasize relations that are created with lexical signs.

3.2.2 Indicating referents

According to the typology, indexical features are prominent in head movements that indicate discourse referents or indicate discourse structure. Movements that indicate discourse referents have been analyzed as signs that direct attention to their objects or bring objects to someone’s attention (see Clark 2003; Cooperrider & Nunez 2009). The first type of movement is pointing movements of the head referring to specific referents. These movements function as manual pointing: they project a vector from a specific body part to a direction, location or an entity (Kita 2003). They may identify—introduce, re-introduce and maintain—referents which are present at the moment of signing or they may be used to refer to imaginary referents, that is, metaphoric locations in the signing space in front of the signer to which referents have been placed earlier in the discourse (for an overview see e.g. Liddell 2003; Perniss & Özyürek 2015). This strategy has been referred to as abstract deixis in relation to co-speech gesturing (McNeill et al. 1993). When pointing with manual signs, eye-gaze and movements of the head are directed to these locations, they organize discourse with anaphoric references. With speakers, reference tracking with manual gestures has been found to occur more in the re-introduction of referents than while maintaining them (Gullberg 2006; Perniss & Özyürek 2015). It is unclear whether pointing with the head and eye gaze alone may be enough when an imaginary referent is established for the first time in the discourse.

Pointing head movements may emerge together with indexical manual signs or independently, without manual signs. Syntactically they can function as pronominal core arguments of a clause (see Puupponen et al. 2015). When pointing with the whole head one may, for example, refer to the addressee in a conversation (Puupponen et al. 2015) or objects present in the communicative situation. In those occurrences identified in the data in Puupponen et al. (2015), they were nod-like movements which include tilting the head sagittally or sideways while pointing in the same direction as the gaze. In the study, referring to specific referents by pointing with the head was less frequent than other types of head movements (Puupponen et al. 2015). However, in different types of communicative situations signers may presumably point with their head in multiple different ways. It is suggested here, as a hypothesis for future research, that pointing with the head emerges more often as giving emphasis to, or as a consequence of, manual movements in indexical signs rather than independently without manual signs. Another potential context for independently occurring head pointing may be situations in which the hands of the signer are in other ways occupied. Independent pointing movements of the head are presumed to be more common in discourse data than in narratives, but in order to throw more light on this, pointing with the head should be investigated further with data consisting of multiple different discourse genres, including discussions on both present and imaginary referents and reference to locations both near and far from the interactants.

Indicating a referent can also be done with sideways movements of the head while connecting the content of manually produced signs to imaginary referents established earlier in the discourse. The signer may, for example, change the orientation of the face along with the eye gaze, tilt the head sideways, or lean the whole upper body sideways, or produce a combination of these movement features. In some of the movements, the orientation of the face changes according to the direction of the gaze. These are referred to as gaze-aligned movements (see also Enfield 2009). In these movements the head does not independently point as if projecting a vector to the object to which the signer is guiding attention. The direction of the gaze points while the movement of the head aligns with the gaze and potentially emphasizes the pointing. To this may be added a turning of the shoulders, torso or the whole body in the same direction as the gaze. When manual signs are produced during this action, their interpretation is anchored to the reference point of the pointing (introduced or pre-established). In gaze-aligned movements it is not easy to define whether the gaze and head movement direct attention to something or draw something to someone’s attention.

In some movements of the head or the whole upper body, the head moves towards an introduced or previously established location (i.e. an imaginary referent) while the face and gaze may be oriented towards an addressee (as in Figure 4) or to the imaginary referent. As with gaze-aligned movements, these movements anchor the simultaneously occurring manually signed contents to the reference point of an imaginary referent (introduced, re-introduced or maintained). In the head movement typology, this type of indication of a referent with the head is referred to as placing. With placing movement I refer to a movement which, instead of projecting a vector in the direction of a referent, “actually positions a temporal [or other] entity in space” (Cooperrider & Nunez 2009: 190; brackets added to the original quote by the writer). This definition is connected to Clark (2003), who distinguished between directing someone’s attention to something (i.e. pointing) and offering something for someone’s attention (i.e. placing).6

Figure 4 

An example of an utterance in which the head tilts towards an imaginary referent (young deaf people), specified in the signing space earlier in the discourse, and connects the signed content to this theme (see also Puupponen 2012). The head turn during the tilt shows exclusion and it is analyzed here as enacting. SLM_2011_00:00:18.

Placing movements of the head, such as the one in Figure 4, are often longer in duration than pointing movements identifying referents. However, differences in the average durations of these movement types is an empirical question for future research. The anchoring of manually signed content may be done while the head moves or turns sideways, or there may be a sideways hold or a static position of the head during which the content is produced. They are not used for explicit pronominal reference and therefore they do not function as core arguments of a clause. They are, however, used together with manual pointing and pointing with the gaze to track reference in discourse (see also Lackner 2017). They may be optional, as indexical manual signs perform the same function, but they are presumed to occur quite often: the way the discourse is indexically organized is shown in the whole body of the signer, not only in the hands. On the other hand, they may also include enactment (as is the case in Figure 4). In general, it is suggested here that in many cases the difference between pointing and placing functions may not be clear cut. When it comes to movements of the head and eye gaze, in the same way as moving towards something may not be easy to distinguish from projecting a vector in the direction of something, directing attention to something or bringing something to someone’s attention may not always be distinguishable notions. The interplay between gaze and head movements as well as their indexical functions is a matter which should be dealt with in more detail in future research.

3.2.3 Indicating discourse structure

Head movements which do not indicate referents but only indicate discourse structure organize and parse texts and conversations by binding elements together into continuous stretches, by showing points of transition between these stretches, or by emphasizing elements during the stretches. In the head movement typology these are referred to as parsing movements, separative movements and emphatic movements, and they occur either together with movements of the torso or without them. Parsing movements bind elements together into stretches of discourse without placing or pointing (Figure 5). They are simply movement contours which bind together manual signs in relation to the syntactic structure and organization of the discourse (phrases, clauses, sentences, text episodes, turns in conversation). In sign language linguistics, these types of movements have been previously referred to as prosodic “domain markers” (e.g. Wilbur 2000; Pfau & Quer 2010; Puupponen et al. 2015; see also Sandler 2012). Formally they may not differ from placing movements: they are sagittal and sideways head movements which are relatively long in duration and may include a static hold or position of the head. In FinSL (Puupponen 2012; Jantunen et al. 2016a; Puupponen 2018) and Israeli sign language (Sandler 2011), for example, the parsing function has been associated especially with sideways directed movements of the head and upper body, although sagittal parsing is not uncommon. It should be emphasized, however, that head and body movements do not bind elements together on their own. Instead, parsing is done together with the kinematic features of manual movements as well as with the timing of different facial expressions (e.g. Ormel & Crasborn 2012; Sandler 2012).

Figure 5 

An example of a sequence of utterances in a narrative in which a parsing movement of the head and upper body binds together manual signs and shows the area of a metacomment on the content of the story.7 CFINSL_008_05_00:05:43.

As mentioned in Section 3.1, movements of the head may also create contrast between elements in signed discourse (Figure 6). In this case the head (or the whole upper body) moves, for example, sideways while the content is being signed, after which the head comes back and moves again while content is signed that is in an oppositional, alternative, corrective or comparative relation with the previously signed content (see also Johnston 1992; Wilbur & Patschke 1999; van der Kooij & Crasborn 2006; Puupponen et al. 2015; Jantunen 2016). They are indexical embodied visualizations of a relation between two or more entities, and they may introduce referents which are referred to with anaphoric pointing later in the discourse. As in the example in Figure 6, the contrastive relation always includes at least two elements which are contrasted, that is, it always involves a relation of two entities or events. These contrasted elements may be expressed manually with only single signs or with whole sentences, the choice naturally affecting the duration of the different contrastive head movements. These manual elements may be contrasted with only head movements or also with manual conjunctions (see e.g. Jantunen 2016). Contrastive movements usually include two or three sagittal and/or sideways movement sequences (including the retraction phases of the movements), depending on the number of elements that are put in a contrastive relation. The movement sequences may be produced in distinctly different or even opposite directions, or in similar directions but with tilts to opposite sides. Further, when contrasting only two things, the head movement may emerge with only one of them, as is the case in the example in Figure 6. This is enough to create a perceptible distinction between the two things.

Figure 6 

An example of an utterance in which movements of the head and upper body signal contrast between manually signed elements. The 3rd gloss from the right is FINLAND-SWEDISH and the abbreviation at the end of the 4th gloss, ‘h2’ refers to the non-dominant hand. CFINSL_008_06_00:14:50.

Contrastive movements are parsing movements with a particular function. In this case their indexicality may identify referents (placing) as well as discourse structure (parsing), although they do not necessarily have to include a placing function. The temporal placement of the movement is enough to create a contrast. Furthermore, as was pointed out earlier, it can be argued that signaling opposition, alternatives and comparison by means of placing movements to opposite sides of the signing space invokes association of analogy (i.e. iconicity) as well as indexical signification. The stance taken in this paper is that this is one type of iconicity, and that contrastive movements are a good example of how different semiotic grounds come together in actions of the head.

Movements separating stretches of signing emerge on the boundaries between clauses, sentences, text episodes or turns in conversation. They are movements that are relatively short in duration – such as single head nods – and indicate linear transitions between signed sequences (Figure 7). For example, in a study comparing corpus narratives of Finnish and Swedish sign languages, a tendency was found for single head nods to align with syntactic boundaries and to occur sentence-finally in both languages (Puupponen et al. 2015; 2016). In sign language linguistics, these types of elements have traditionally been referred to as prosodic “boundary markers” or “edge markers” (e.g. Wilbur 2000; Pfau & Quer 2010; Puupponen et al. 2015). Stretches of discourse are also separated simply by the transitions between parsing head and torso movements, that is, they are not always pointed out by distinct movement events such as nods. The holds and changes of direction between longer movement contours are in themselves elements that organize the signing string into perceptible chunks, just like phonetic features which indicate word and sentence boundaries in speech.

Figure 7 

An example of a separative head nod indicating a transition between two stretches of discourse. CFINSL_008_05_00:06:40.

Finally, in the head movement typology, many different types and forms of movements are classified as emphatic when they reinforce manual movements, and hence also the content conveyed by those manual signs. They may be produced along the movement path of manual signs, or in the opposite direction (hands and head moving towards or away from each other), which intensifies the manual movements. It is suggested here, as a hypothesis for further investigation, that emphatic head movements may be reflex-like consequences of manual motion or movements involving more control in their production and interpretation (see Enfield 2009; Kockelman 2005; Section 4.1) and underlining the content that is being signed manually (Figure 8).

Figure 8 

An example of head movements emphasizing manually signed content. CFINSL_008_06_00:05:54.

In sum, parsing, separative, and emphatic movements all indicate the structure of larger episodes of discourse or smaller syntactic sequences. They guide one’s attention to the organization of the signed content and signal relations between textual elements. It is suggested here, as a hypothesis for future research, that parsing and separative movements may include less control in their production and interpretation, while emphatic movements may be more controlled, depending on the situation in which they occur. This issue will be discussed further in Section 4.1 (see Kockelman 2005; Enfield 2009). Some parsing movements and contrastive movements, presented earlier, may also be considered emphatic. This exemplifies the fact that the typology presented in this paper describes head movements through prototypes, not mutually exclusive categories. Different functions in the signification made with head movements may overlap.

3.2.4 Enactment or reflex-based signaling of embodied experiences? Inclusive and exclusive head movements

Indexical features also emerge in head movements showing inclusion (e.g. ‘participation’, ‘going along’) or exclusion (e.g. ‘avoidance’, ‘rejection’) (see also Wilbur & Patschke 1998; van der Kooij et al. 2006; Puupponen et al. 2015). In these instances, the head movement has its roots in a pattern of human behavior typical in the Western world: when we want to be included in something, we approach it, when something is rejected, we move away from it. The origins of these types of movements can be seen in physical reactions, reflexes, and typicalities in the actions of human beings, that is, in the physical and social behavior of humans. They are indices which stand for something else through a natural connection, cues for something through reality, just as blushing, an angry expression, or the sound of rain are indices.8 We may have a shared experience of certain types of behavioral patterns, which means that they may become more or less schematic and recognizable, as is argued in section 3.3 that discusses headshakes and nodding of the head. An example of an exclusive head movement was given in Figure 4.

On the other hand, movements of the head or the whole body showing inclusive and exclusive meanings may in many cases be interpreted as enacting movements. They may be viewed as iconic depictions of the behavior (experience or feeling) of a referent. When they emerge in sequences of overt enactment (Cormier et al. 2015; see Section 3.1), they may be easily analyzed as a part of the representation of a referent’s inclusive/exclusive actions. However, when the signer’s upper body leans forward with signs such as SIGN-UP and JOIN, for example, without distinctive enactment from other parts of the body, it is difficult to say whether the movement should be considered enacting or not. Whether or not inclusive and exclusive movements always include enactment is an open question and, in the end, the answer may depend on how one defines enactment in signing and which formal features are associated with it. A recent phonetic study of FinSL (Jantunen et al. 2018a) brings out interesting issues concerning this question as it shows that in more subtle degrees of constructed action, eye gaze—which is traditionally seen as the most significant feature distinguishing constructed action from standard narration—need not be included in the enactment.

Enacting or not, inclusive and exclusive movements include indexical reference points. In forward-directed head movements with content such as ‘participation’, the direction of the movement is produced with relation to a spatial reference point. The reference point may be unspecified and non-anaphoric, in which case more content is not anchored to it during the discourse. It is suggested here that in this case the reference point is usually placed in front of the signer, and inclusive head movements are directed towards the point (forward) while exclusive movements are directed away from the point (backward). If the reference point is more specified and anaphoric—a present or imaginary referent introduced earlier in the discourse—the movement is produced towards or away from that reference point, not necessarily forward or backward in the sagittal plane. Considering the discussion above on the enacting vs. non-enacting nature of inclusive and exclusive head movements, another question is whether anaphoric and specified movements are easier to interpret as enacting than non-anaphoric movements.

3.3 Symbolic features in head movements

The symbolicity of head movements is most evident in negative headshakes and single nods or repeated nodding of the head with positive functions, which form more conventional form-function pairs than primarily iconic or indexical head movements. Headshakes and nods are globally widespread gestures frequent in the communication of both signers and speakers. Although there are observations of some variation in their functions across the world (e.g. LaBarre 1947; Samovar et al. 2007), they are somewhat conventionalized signs for negative (e.g. disapproval, disbelief, negation, grief) and affirmative meanings (approval, agreement, understanding, assuring). In the data from which the current paper draws, they may occur together with manual signs (Figure 9) or independently as responses to questions or as backchannelling cues. They have a more decontextualized meaning-like function than other types of head movements discussed in this paper, that is, their interpretation is not only context-specific and they can be considered as tokens of types.

Figure 9 

An example of a headshake that changes the meaning of a sentence from affirmative to negative together with the facial expression. CFINSL_008_06_00:19:50.

Although the form-function connections in negative headshakes and affirmative nodding show more conventionality than in iconic or indexical head movements, this does not mean that their functions are clear cut. Headshakes, for instance, may convey different types of meaning related to negation or surprise: disbelief, hesitation, doubt, downplaying, disagreement, negation, sorrow, sympathy towards a negative issue, and so on (Kendon 2002; Johnston 2018; Puhto 2018). When a signer signals something negative with a headshake, this may be associated with the negation of a proposition in the discourse or it may convey meanings related to the way the signer’s personal feelings, thoughts or attitudes are positioned against what is said in the discourse (Puupponen 2012; Johnston 2018). Therefore, when we view symbolic head movements as semiotic signs, it is evident that the object to which they connect are not clear cut, but rather a gradient field of different types of functions that are more or less closely connected.

In addition, symbolic head movements are not only symbolic: the connection between the sign and its object is indexical for both of them. Negative shakes and positive nods are indices which show different degrees of conventionalization. For example, the rotation of the head in a headshake may be argued to have its origin in a gesture of rejection, a reflex that is already typical in infancy (e.g. a child avoiding a spoon while being fed). In the 19th century Charles Darwin (1872) suggested that nodding and turning the head have their roots in the way babies lean towards milk (nodding) or reject it (turning). In addition to cyclical shaking movements, this type of function (i.e. a physical act of rejection) may form the basis for different types of exclusive and negative single head turns found in different sign languages (see Figure 4). On the other hand, it has been suggested that head turns stem from avoiding eye contact (Stern & Bender 1974). With relation to the indexical origin of head nods, they have been argued to have their roots in the act of bowing (Morris 1994) and in the development of reptiles already 280 million years ago (Givens 2013). Some scholars have also argued that head nods and head turns may be traced back to human mirror systems and their effect on babies’ abilities to imitate their parents’ movements and emotions from early on (Meltzoff 2002; Braten & Colwyn 2007; Thagard 2010). The current paper argues for the view that symbolic head movements in sign languages are schematized indexical signs which can be said to differ in their degree of conventionality and to vary in their meaning according to the communicative situation.

Finally, symbolicity occurs also in other head movement types, although not in the Peircean sense of legisigns. As with speakers’ manual gestures, so too with signers’ nonmanual actions the conventionality may lie in the “types of communicative action” (Enfield 2009; see also Kendon 1988; Okrent 2002), rather than requiring specific forms for specific functions. The ways in which iconic and indexical movements are produced have recognizable features through association and inductive processes even though they do not emerge as types for tokens in the same way as do shakes and nodding of the head. Their recognizability may involve understanding the habits of a single signer (i.e. entrenchment) or conventions shared among a group of signers. Nods and shakes, on the other hand, are easy to associate with convention because they convey meaning that can be understood roughly along the same line of interpretation, whatever the context.

3.4 Proportions of different grounds in head movements

As has been presented in the preceding sections, different types of head movements involve iconicity, indexicality and symbolicity in overlapping ways. For example, enacting and time-metaphor head movements include both iconic and indexical features, movements indicating referents by placing them in contrastive relations may have iconic features, and symbolicity emerges in different ways in different types of head movements. It may, however, be that these different strategies of signification emerge in different proportions in different head movement types. In order to demonstrate this, I will borrow a visualization suggested by Capirci (2018) for the iconic-indexical-symbolic relations of linguistic signs (Figure 10).

Figure 10 

The proportions of different grounds in different head movement types (visualization according to Capirci 2018).

As suggested by Figure 10, the proportions of iconic (yellow), indexical (green), and symbolic (red) grounds are different in different types of head movements. While negative headshakes include a lot of indexicality and symbolicity, their iconicity lies mainly in the associational recognition of tokens to types (see Peirce 1903a; Silverstein 1955; Parmentier 1994; Kockelman 2005). With movements indicating discourse structure (parsing, separating, emphatic) and indicating referents (pointing, placing), the indexical ground is strong whereas iconicity lies in the association of the types of communicative action (together with other sign mediums) to the token sign events, that is, in recognizing their symbolic ground (see Enfield 2009). In addition, as pointed out earlier, in contrastive movements, for example, the spatial relations between the contrasted element may show also other sources of iconic association. As for enacting and time-metaphor movements, the iconic ground is strong but they include a lot of indexicality as well. The symbolicity of these movements lies in the recognition of patterns in the communicative action, that is, in connecting the token enactments and token time-metaphor movements to the “general manner of execution” (Enfield 2009: 19) in which they are done among signers. Finally, one very visible feature in Figure 10 is how redness—symbolicity—is not the most evident feature in all head movements. Symbolicity may not be the prime motor of this type of signification, but still head movements are significant. This, as I argue in Section 5, speaks for the need for a theory of language that embraces less symbolic and unconventional elements instead of pushing them to the periphery.

4 Head movements and the semiotic versatility of signers’ actions

4.1 Semiotic complexity & pragmatic unity in signers’ actions

As presented in Section 2.5, in a semiotic analysis of signers’ head movements it should be noted that head movements in a given communicative situation are one part of a semiotically complex whole—a composite utterance (Enfield 2009)—consisting of several signs from different sign mediums and driven by a unified intentional social action (see ibid.). This applies for all head movements, also movements such as headshakes that are produced without any manual action, for example, in response to an interlocutor’s utterance (i.e. they may emerge together with eye contact, facial expressions, and so on). No signals produced with the head in natural discourse emerge in isolation and they cannot be interpreted as isolated entities.

There are, however, differences in what kind of components signals of the head are in clusters of semiotic features. Some of the head movements discussed in Section 3, such as a headshake can be interpreted as conveying meanings. Head movements indicating discourse structure, on the other hand, cannot in themselves be associated with a specific meaning. The function of a parsing or separating head movement comes about in its relation to the whole signed utterance, in its temporal alignment with the actions of the hands. This means that its functions emerge through a physical, actual relation—a co-occurrence or proximity of timing—which makes it an indexical sign (Peirce 1903a; Parmentier 1994; 2006; Kockelman 2005). A sentence-final head nod or a parsing movement of the head (or the whole upper body) organize the discourse by indicating changes or continuities in the structure along the lines of, ‘here is a change in the structure of discourse’ or ‘this is a continuous stretch of discourse’.

Semiotically, a separating head nod or a parsing sideways movement of the head are signs that do not have a clear and easily determined object (cf. Kockelman 2005). Instead, the object is an abstract notion that emerges in the co-occurrence of head movements with manual actions: ‘something is happening here that is distinguishable from what has come before and/or is subsequently going to come in the discourse’. This is in line with Kockelman’s (2005: 242) notion of the abstract object which is “less precise”, “less consistent” and does not necessitate “intentional states such as knowing” (see also Section 2.1). The interpretant of this type of a sign is the feeling or state that comes about when observing the sign, and which may be vaguer and more subconscious than consciously acknowledged: ‘the processing and interpretation of these contents is smoother and less obstructed because this signal organizes the structure of discourse’. As a hypothesis for further investigation, it is suggested here that a head movement indicating discourse structure involves less control in both its interpretation and its production, and that it therefore shows a lower degree of agency than, for example, a headshake (Kockelman 2005; Enfield 2009). It may be that the existence of such signs becomes evident only when there are perceivable errors or disfluencies in the signing as a result of the atypical use of these signals.

These types of signals are similar to what Enfield (2009) calls implicit indexical resolution. Although Enfield seems to use the term to refer to indexical signs with an implicit placing function (see also Clark 2003)—that is, to signs that link other signs in an utterance to specific points in space with mere placement (e.g. a no-smoking sign)—it is partly applicable to signs indicating discourse structure without a placing function. Both types of signs (i.e. a no-smoking sign and, e.g. a separative head nod) are implicit in their indexicality, that is, they come about through the physical situation in which they emerge or to which they are positioned without any explicit indexical content. Despite their implicitness, both sign types are, however, indexically connected to other signs in the semiotic cluster. A no-smoking sign links the interpretation of the content of the sign to a specific location without actually pointing to it (e.g. with an arrow). A head movement indicating discourse structure is linked to the linear structure of signed utterances, or larger stretches of discourse, by mere temporal placement, the proximity of timing. The interpretation of these head movements is interrelated with the holistic interpretation of the whole composite utterance and, further, with the textual entirety that they build together. This unity in interpretation is possible because of contextual association (Enfield 2009), among other factors. The timing and other indexical proximity of these signals are triggers that guide the interpreter to assume that they are “part of one signifying action” (Enfield 2009: 16) together with other signs in the utterance. To summarize, as a hypothesis it is suggested here that head movements include different degrees of control, as has been found to be the case with, for example, eye-blinks in signed discourse (e.g. Wilbur 2000). Movements indicating discourse structure involve less control than, for example, more conventionalized head movements such as headshakes or head movements indicating referents. It is also suggested that different head movement types differ in how dependent their interpretation is on other co-occurring signs in other mediums. While a headshake is connected to its composite utterance, its function does not rely on other signs to the same extent as the function of a separative head nod. Nonetheless, they are all part of intentional communicative actions which, according to Enfield (2005: 17), are driven by a pragmatic unity: the “unified, single, addressed utterance–meaning”.

4.2 Signification with the head and other parts of a signer’s body: Central semiotic features

It is not surprising that different sign mediums in sign language discourse form a whole, and that different body parts share features in terms of what types of (and how) signs are created with them. However, I argue that there are differences in the central semiotic features signaled with different body parts. This semiotic versatility (Wagner et al. 2014: 209) is a feature that is evident in the communication and interaction of speakers as well as signers. Hands convey information regarding shape and size better than speech, while the face is well suited for expressing feelings and attitudes (Wagner et al. 2014). I argue, further, that in addition to approaching the signification of signers as a holistic social action that embodies a cluster of semiotic signs, it is important to discuss the potential differences in the pivotal features of these sign mediums (hands, face, head, body). This requires both that we investigate signification while focusing on signers’ specific body part(s) and that we investigate the interrelations of these different parts of signing. In this way we might find out whether or not the central features of different sign mediums in signer’s communicative actions are similar and learn more about the roles of and interplay between different nonmanual signals in signed discourse.

The view taken in this paper on the semiotic versatility of signers’ actions is summarized in Table 1 with regard to each sign medium. As is shown in the table, manual signs include conventional signs, unconventional signs and hybrids of the two (see Enfield 2009). These signs show different levels of iconicity, indexicality and symbolicity (as defined in the current paper), and it is not uncommon at all for signs to present all these features simultaneously (Ferrara & Hodge 2018).9 Manual signs have iconic features which require the analogical association of the similarities between qualities of the sign and qualities of its object. A handshape consisting of a stretched index finger used for referring to a small object such as a pen is an example of an iconic feature in a manual sign, as is the signaling of size with tracing movements or qualities of actions with manual kinematics. On the other hand, pointing and placing movements (indicating referents) and some manual kinematics and manual signs (indicating discourse structure) are examples of the indexicality of manual signs. The symbolicity of manual actions can be seen in the shared norms among signers of a language that have emerged through frequency of use and which enable lexicalization processes. Both description with partly lexical signs and enactment are common strategies through which fully lexical signs may emerge in a sign language (e.g. signs that describe whole entities, shape, size or handling) (e.g. Johnston & Schembri 1999; 2010; Liddell 2003; Johnston & Ferrara 2012; Ferrara & Halvorsen 2018; Ferrara & Hodge 2018). Especially interesting is the fact that both of these iconic strategies have a two-way connection to the established lexicon: while signs may conventionalize and become fully lexical signs, this process may move in the opposite direction. That is, fully lexical signs may “turn back” to a more gradient type of signing, such as enactment, a process which is often referred to as delexicalization (Johnston & Schembri 1999; Cormier et al. 2012; Johnston & Ferrara 2012). Johnston & Ferrara (2012: 237) refer to this phenomenon as the “two faces of the sign”. To conclude, the manual elements of a sign language form gradient categories (see e.g. Jantunen 2017). This is the result of the different strategies available in discourse.

Sign Medium Central semiotic features

Hands Conventional signs, unconventional signs and hybrids (see Enfield 2009), including
  • - Observer-viewpoint description: fully & partly lexical signs
  • - Indicating referents: pointing; placing
  • - Signaling one’s own emotions and attitudes (manual kinematics)
  • - Indicating discourse structure: parsing, separating, emphasizing (manual kinematics, pauses, pointing; buoys etc.)
  • - Signaling grammatical information: e.g. qualities of actions or events (manual kinematics)
  • - Etc.
Enactment

Face Lower part: some hybrids + unconventional signs, including
  • - Observer-viewpoint description: signaling the qualities of entities and their actions, emotions, attitudes (mouth gestures & whole facial expressions)
  • - Signaling one’s own emotions and attitudes (whole facial expressions)
  • - Indicating discourse structure: parsing (the spreading of mouthings)
  • - Signaling language contact (mouthings)
  • - Potential for indicating grammatical information
  • - Potential for conventionalized symbolic signs (especially mouthing)
Upper part: unconventional signs, including
  • - Indicating referents: pointing and placing (gaze)
  • - Indicating discourse structure: parsing & emphasis (brows & eye aperture); separating (blinks)
Enactment

Head A few hybrids + unconventional signs, including
  • - Conventionalized reaction-based indices: negative-shakes, affirmative nod(ding)
  • - Indicating referents: pointing & placing
  • - Indicating discourse structure: parsing, separating, emphasizing
  • - Movements visualizing (time) metaphors
Enactment

Torso/Whole body No/few hybrids (shrugging?) + unconventional signs, including
  • - Indicating referents: placing
  • - Indicating discourse structure: parsing, emphasizing
  • - Movements visualizing (time) metaphors
Enactment

Table 1

The central semiotic features of the actions of different parts of signers’ body: hands (e.g. Padden 1990; Johnston & Schembri 1999; Liddell 2003; Johnston 2013a; b; Jantunen 2016; 2017; 2019), face (e.g. Metzger 1998; Wilbur 2000; Boyes Braem & Sutton-Spence 2001; Thompson et al. 2006; 2009; Sandler 2012; Johnston et al. 2015), head and torso.

A signer’s face conveys a lot of information as well. All in all, signers’ facial expressions are elements that characterize something. They convey information about the qualities of referents and their actions, emotions or attitudes. For this reason, parallels have been drawn between facial expressions and grammatical classes of spoken languages, such as adverbs and adjectives (mm. Wilbur 2000; Pfau & Quer 2010; Herrmann & Pendzich 2014). These labels have been given in particular to mouth actions—elements produced with the lower parts of the face—and to whole facial expressions in general. However, a recent study (Johnston et al. 2015) has challenged the view that elements produced with the face are “linguistic markers with a grammatical function” (Herrmann & Pendzich 2014: 2149). When the form-function pairing of signers’ mouth actions was studied from a large set of video material, they were found to show patterning, but not to function as conventionalized grammatical markers (Johnston et al. 2015). Instead, their use was found to be idiosyncratic, and different types of mouth actions were placed on a continuum from indexical to iconic, unconventional and finally potentially grammatical (Johnston et al. 2015). Upper parts of the face, on the other hand, have a range of indexical functions. Firstly, the direction of the eye gaze is used for pointing at present or imaginary referents or placing imaginary referents to the attention of interlocutors (e.g. Wilbur 2000; Schembri et al. 2018). Secondly, eye-brow movements and eye aperture may parse and emphasize elements, such as old and new information, or the lack of information (see e.g. Wilbur 2000; Sandler 2012). Finally, some eye-blinks, biologically driven actions for keeping the eyes moist, separate stretches of signing by indicating transitions between them (Wilbur 2000; Sandler 2012).

All in all, elements produced with a signer’s face form a semiotically interesting research topic. Although different parts of the face may be distinguished on the basis of the elements they produce, the facial expression of a signer is often a unified whole which is interpreted imagistically in its entirety. Apart from the eye gaze and blinking, the upper and lower parts of the face should in this case be investigated as a whole, not only locally. Eye gaze and blinking, on the other hand, may function more independently also during more unified facial expressions (pointing and separating functions). It may be that the signals produced with the face vary in how dependent their interpretation is on other co-occurring signs in other mediums, in the same way as was suggested for different head movements. Finally, it should be noted that when it comes to actions of the face, it is difficult to distinguish between enacting, the non-enacting description of qualities of a referent, and the expression of the signer’s own feelings and attitudes (see e.g. de Vos et al. 2009). The face forms a semiotically complex sign medium, which should in some cases be approached as a cluster of smaller and more specific semiotic mediums.

As can be seen from Table 1, the semiotic repertoire of head movements is in some ways more limited than that of the hands and face. The capacity to reflect or imitate visible features in the world is not as diverse for signs produced with the head as it is for signs produced with the hands and the face. The physiology and anatomy of the head (see Puupponen 2018) alone limit the potential for imitation in head movements, and actions of the head are often more approximate than those of the hands and face. Head movements may depict referents, their actions and their qualities, but this is mostly enactment. Non-enacting iconic features are not as common. Furthermore, symbolicity (as defined in the current paper) is not as prominent in the actions of the head as in that of the hands, as was presented in Section 3. Head movements seem to rarely form types for tokens, they involve more analogical association and spatio-temporal contiguity, and they include different degrees of control in their production and interpretation. The conventionality of head movements emerges most probably in the recognizable manner in which certain communicative actions are produced.

Compared with head movements, the actions of the (upper) body have an even more restricted variation in semiotic functions. As is the case with signs produced with the head, also body movements parse and emphasize elements, indicate referents by placing, and enact referents. Symbolic types for tokens are presumably rare and may emerge only in movements such as shrugging the shoulders. However, although the semiotic repertoire in signs’ representative nature is more restricted than that of the hands and face, this does not mean that the body does not convey a lot of information. The diversity and richness of actions of the body is shown, for example, in the ways in which signers and speakers convey information related to their identity, state of mind, emotions and social distance with body movements that may not be intended as communicative actions (see e.g. Streeck et al. 2011).

The semiotic versatility of actions of different parts of signers’ bodies may be summarized as follows: (i) Hands produce more fully lexical symbolic signs (i.e. types for tokens) than other parts of the body, indicate referents and discourse structure, signal one’s own emotions and attitudes, and show semiotic flexibility (signs may emerge both as lexical signs and as more gradient and unconventional enacting); (ii) The face can enact, describe without enacting, indicate referents, and indicate discourse structure, and some signals may become more conventional symbolic signs; (iii) The head can indicate referents, indicate discourse structure, enact referents, and connect to (time) metaphors but symbolic types for tokens are rare, as is non-enacting description, (iv) The (upper) body indicates referents and discourse structure with a slightly smaller repertoire than the head, enacts referents, and connects to time metaphors. As can be seen from the summary, indexicality and iconicity are everywhere while symbolic types for tokens are primarily produced with the hands. As a result, I suggest that the potential for communicative actions of different parts of the body to produce symbolic signs in the sense of Peircean legisigns is strongest for the hands, possible for the face, rare for the head and unusual for the upper body. Thus, the signification with different parts of signers’ bodies forms a continuum, from hands to body, between an abundance of symbolic content and very little symbolic content.

These differences in the central semiotic features of actions of different parts of the signer’s body come back to the physical characteristics of these different sign mediums. There are differences in what kind of information can easily be conveyed with different parts of the body. Borrowing from Enfield (2009: 18), hand movements “are well suited to iconic–indexical meaning thanks to their rich potential for sharing perceptible qualities in common with physical objects and events”. As Wagner et al. (2014) pointed out, the face is a convenient medium for signaling emotions and attitudes. However, tracing the movements or shape of entities is not a feature which is enabled by the physical characteristics of the face; the quantity or volume of entities, on the other hand, may be signaled with, for example, different mouth actions. The physical characteristics of the head and body are not well suited for signals that share physical qualities with physical objects and events, except for showing the actions and existence of referents through enactment. However, as with the hands and eye gaze, with the head and the body one may produce signals which point or place, for example. In this function one may also see differences between the different mediums. As the size of the body part that produces the pointing action becomes larger, so too the object of the pointing becomes less precise. Pointing with the head or the whole body is in many cases—especially when referring to referents that are not physically present—more approximate than pointing with the index finger or the whole hand/arm. Furthermore, while placing movements of the hands function well for indicating precise spatial relations between entities (micro-level), placing movements of the head or the whole body are better suited for large scale thematic, textual and syntactic sequencing (macro-level). Finally, semiotic versatility in the signs produced with different body parts applies to speakers as well as signers. When looking at, for example, the comprehensive literature review on co-speech head gesturing carried out by Wagner et al. (2014), there are evident similarities between speakers’ and signers’ head movements, and comparing their forms and functions raises many interesting questions for future research.

5 Including non-manuality in a theory of language

The actions of a signer’s face, head and body are idiosyncratic, multifunctional, and gradient elements in sign languages. The “units” produced nonmanually do not form discrete categories, and they are not directly parallel to words and affixes in spoken languages (except, e.g. deictic words, onomatopoeia and ideophones, see e.g. Dingemanse 2017). Signers’ head movements, for example, are often optional (see Section 3; also Puupponen et al. 2015; 2016). If one disregards the challenge of defining the linguistic status of nonmanuals and studies first their essence as semiotic signs, we find that signers nonmanually produce signs which vary in the extent to which they contain iconic, indexical and symbolic features. This type of wide approach could include in its analysis elements which might not interest a linguist or a gesture researcher, such as movements which are not produced or interpreted as means of communication but which are rather just a canvas onto which semiotically motivated features present themselves (e.g. a physical tremor). These movements can also be interpreted as signals for something, although they do not actually have communicative intent (stretching one’s back, for example). In other words, the investigation may be focused on elements that are communicational in a wider or a more restricted respect, depending on the theoretical approach. In the current paper, the focus has been on head movements that are part of intentional communicative actions (see Kendon 2004; Enfield 2009), but which may show different degrees of control in their production and interpretation while still organizing information into understandable structures. However, the head movement typology presented in the current paper could be extended to include treatment of non-intentionally communicative head movements.

The strength of a more or less wide approach is that it does not automatically exclude elements: when the focus is wide, elements are not left outside the scope of the study (although things do align as peripheral and prototypical in the long run). If the focus is more predetermined—e.g. a formalism for spoken natural language—phenomena which do not fit the premises of the underlying theory may be omitted from the investigation. In the case of nonmanuality, this might lead to nonmanual cues, which are considered “paralinguistic” from the viewpoint of a specific linguistic theory, not being included in the analysis. Given that gradience and unconventionality are characteristic of nonmanuality, this may lead to the exclusion of a significant amount of nonmanual activity from the investigation. On the other hand, it may be that nonmanuals are investigated more broadly, but the interpretation of their functions is affected by assumptions determined by the underlying theory. If the theory presupposes that the elements which it describes are categorical and can be explained according to a fixed set of rules, rules may be formulated without sufficient evidence of their generalizability, or elements that do not “materialize” according to these rules may be seen as exceptions, anomalies. In both cases there is a risk that the theoretical view of the subject—in this case nonmanuality—will become distorted.

Kendon (2014: 3) suggests that, instead of trying to define an interface between “language” and “non-language”, we should try to “distinguish these different [semiotic] systems, at the same time analysing their interrelations”. With relation to Peircean semiotics, this would mean extending the analysis from symbolic signs with possible iconic and indexical features to include also iconic and indexical signs which are not legisigns, that is, which do not show the level of conventionalization that symbolic types for tokens do. In the investigation of sign languages this means that, in addition to including in the theoretical description both the conventional/distinctive and gradient/unconventional features of manual elements, one would also have to include the (mostly) gradient/unconventional nonmanual cues. As Kendon (ibid.) points out, the pursuit of understanding how meaningful utterances are successfully conveyed “will require that we incorporate in a systematic way these other systems that do not admit of a formal-linguistic analysis” (see also Liddell 2003). The arguments made about nonmanuality in the current paper follow this line of thought, with the exception that nonmanuals traditionally defined as “linguistic” and “nonlinguistic” are not seen as semiotically two different systems. Instead, nonmanuality is understood as a part of signers’ embodied signification, in which semiotic signs are created and interpreted according to their iconic, indexical and potentially symbolic features. I argue that not only manual signs but also nonmanual signs have two faces (see Johnston & Ferrara 2012). Types for tokens may emerge from iconic and indexical nonmanual cues, and on the other hand, elements which may have a more conventional or schematized form-function connection may be used in gradient and unconventional depiction. To be more precise, the stance taken in this paper is that nonmanual signs have many faces. Nonmanuals are not two-faced as they rarely show a similar level of symbolicity as conventional fully lexical manual signs. Nonmanual cues come into existence in each signification process (semiosis) and their interpretation calls for several options: understanding a typicality or habit; the observation of an actual relationship between coexisting things; and an imagistic recognition of likeness.

On the basis of the issues discussed in this paper I argue that, in order to understand nonmanuals and their role in a sign language, we must also take into account non-symbolic, unconventional and gradient nonmanual signs. These cues cannot be defined simply as either language or non-language as their interpretation varies situationally, they are often optional, and their form-function patterning shows levels of entrenchment (see e.g. Wilcox & Xavier 2013) rather than actual conventionality. Therefore, in order to understand how different types of semiotic processes are connected when signers communicate with meaningful utterances, we must look at how signers act and what they do with language. This means including in the theoretical description of a sign language both elements which form discrete categories and elements which form gradient categories or only show tendencies/inclinations (e.g. gestural features of speech and co-speech gesturing). This means, for example, that variation is regarded as a premise—something which is difficult to achieve if one follows strictly any given formalism. Cognitive and functional approaches to language give room for including both strongly symbolic and less symbolic dimensions of signification in a theory of language.

6 Beyond signs? Nonmanuals and embodiment in interaction

Whether or not one includes the gradient and unconventional aspects of nonmanuals in a theoretical description of language, we are likely to agree that nonmanual cues are semiotic signs that are significant for signed interaction. Another question is how one sees the connection between the nonmanual and manual actions of signers and human bodily actions in general, and how these are related to our cognitive and social reality. A fact which is relevant for all signification in sign languages is that the physical ground for iconic, indexical and symbolic nonmanual signs includes not only visual but also sensorimotor processes. The discussion presented in this section supports the view that human interaction emerges through the moving, active “enculturated living body” (Streeck 2015: 432) engaging the world kinesthetically and haptically as well as visually and/or auditorily. Signification, and human cognition in general, are inextricably connected to the ways we navigate in our animate and inanimate surroundings. When it comes to the topic of the current paper, this issue is evident, for example, in the exclusive and inclusive movements of the head or the whole upper body, presented in Section 3.2.2. I argue for the view that these movements—as well as many other head movements—are acts of signification that are visibly connected to the embodied human experience, action and existence in the world.

Whether or not all nonmanual actions should be seen as semiotic signs is, however, another matter. According to Streeck (2015: 430), “transparently meaningful” bodily actions exist beyond semiotic signification:

“The bodily actions of others can be transparently meaningful for us without thereby becoming signs; signs originate when actions are performed specifically for communicative purposes. [–] Intelligible, embodied social action does not require that its significance be expressed—or expressible—in signs.”

In this approach, semiotic signification is seen as a part of a larger ecology of interaction in which the embodied agency of a human being comes together situationally with other actors, medias of communication (including languaging-gesturing), skills, conventions, animate and inanimate surroundings, and the storage of information in these surroundings (Ingold 2011; Streeck 2015). Together these different dimensions create a web in which semiotic action, communicating with signs, forms one part. According to Streeck (2015), embodied communication is only partly semiotic. An interesting question for future research is, however, what exactly are these embodied actions which are significant for interaction but which are not signs, and further, how are they interpreted?

In the end, at the core of this discussion is how one defines a sign and the act of signification. The Peircean definition of a sign is very broad. Signs need not involve conventional or communicative intent but they may still be interpreted as signs, and through habit or regularity of interpretation they may become conventionalized signs with new requirements regarding their interpretation. For example, many indexical signs may be reflex-like, physical, subconscious or semi-conscious activities, as was pointed out in Sections 2 and 3 of the current paper. Events such as blushing or directing one’s gaze may emerge involuntarily, with less control, but can still be interpreted as significant. Signs may be physical events, such as dark clouds or the sound of water drops hitting a surface, which are not consciously communicated but are still interpreted as indices for approaching or already falling rain. This ability to interpret signs, to view events, actions or entities as standing for something else, is the dimension of signification which is emphasized in Peircean semiotics. Considering this, one might argue that Streeck’s transparently meaningful actions are in fact signs in the Peircean sense, if they are interpreted as significant. This would, however, entail that although all actions and events are not signification, all communication and cognition are signification and interpretation of signs.

It is evident that, according to Peirce (1894; 1903a; b), not all action is signification. The universal categories in our experience of reality include different levels of consciousness, while only thirdness is actually linked to thinking. In Peircean trichonomies, the different levels are not disconnected but show different aspects of human experience, thought and action. Peirce’s qualisigns, for example, are qualities that may potentially become signs, but they do not actually perform as such. We are surrounded and involved with different types of embodied activity, which includes different levels of social interpretation. With regard to nonmanuals, it is suggested here, as a hypothesis for future research, that signs produced with the signer’s face, head or body may be produced with more or less control and intention (see Figure 1) and still be interpreted as signs for something else because they connect to co-occurring signs from other sign mediums (see Enfield 2009). It is also suggested that actions such as blinking or nodding one’s head sentence-finally in signed discourse may be produced with less control than other head movements, and that they may not involve conscious interpretation, either. Nonetheless, whether or not they can be seen as including intentionality or reference by themselves, they organize the discourse (production, interpretation and negotiation of information) moment by moment in interaction. It is an interesting question how this embodied activity is relevant for interaction. The matter of whether these actions elicit representation or not is another interesting question for future research, that is, how these types of signs actually guide our action, thought and communication.

7 Conclusion

This paper presented a semiotic typology of signers’ head movements and their iconic, indexical and symbolic features, as well as a discussion of semiotic diversity in the actions of a signer’s body. It presented the view that indexicality is a prominent feature in the actions of the head, and that it is most evident in movements that indicate referents (pointing, placing), discourse structure (parsing, separating and emphasizing), or information behind a reaction (nodding, shaking, exclusive/inclusive movements). A few of these signals may become more conventional or schematized, as has happened with headshakes and head nodding, but these symbolic types for tokens are rare. The conventionality of head movements is more likely to emerge in the recognizable manner in which certain communicative actions are produced than as shared norms that connect specific forms to specific functions. In addition, iconic signification is central in head movements that enact referents or visually connect to (time) metaphors. All in all, non-enacting description of referents is presumed to be a less prominent feature in head movements. Furthermore, the paper suggested that different types of head movements involve different proportions of iconic, indexical and symbolic strategies of signification, and that, in general, symbolicity is not the most common strategy.

The paper concluded that while all head movements are a part of semiotically complex utterances—consisting of several signs from different sign mediums and driven by unified intentional social actions—they differ in how dependent their interpretation is on other co-occurring signs from other sign mediums, and in the degree of control exercised in their production and interpretation. The paper suggested that in signification with different parts of the signer’s body, there are some differences in the central semiotic features, and that the actions of the head and the body are involved in a more restricted repertoire of signification in intentionally communicative actions than are the hands and the face. This means that, although head and body movements are in many ways significant and emerge frequently in discourse, they form a collection of signs with less semiotic diversity than is shown by signs produced with the hands and the face. Their interpretation mostly involves an observation of spatio-temporal contiguity (i.e. indexicality) and analogical association of perceivable qualities (i.e. iconicity). Finally, the functions of nonmanuals such as head movements are seen as inseparably connected to human embodied experience and action in the world, regardless of their degree of conventionality or schematicity. This is seen as support for a wide functional approach that considers non-symbolic and unconventional head and body movements and other nonmanual signs to have an essential place in the theoretical description of languages.

Notes

1Affect is a concept which is defined in different ways according to the specific field of interest. In many cases it is used as a synonym for “emotion”, while some fields (e.g. philosophy and social theory) distinguish between affect—a vague influence which is not yet clearly “meaningful”—and emotion—a subjective and personal quality of experience which has a function and a meaning, and which can be expressed linguistically (e.g. Massumi 1995).

2For a literature review on the forms and functions of head movements in sign languages see, e.g. Puupponen et al. (2015); Lackner (2017); Puupponen (2018).

3Finnish sign language is a language used by approximately 10000–14000 people in Finland, of whom an estimated 4000–5000 are deaf or hard-of-hearing signers (FAD 2018).

4In a more detailed discussion about the actions of signers’ hands and face one might find it necessary to divide them further into more specific mediums, such as dominant and non-dominant hand, upper face, lower face and gaze.

5Figure 2 and the following figures are each provided with a reference to the data (see Section 1.1), including a time-code.

6Clark’s (2003) placement includes the concrete placement of actual physical objects.

7Figure 5 and some of the following figures include a grid in order to demonstrate more clearly the movements of the signer’s head.

8They are indexical signs when someone interprets them as meaningful. They are also simultaneously interpretants (Kockelman 2005), the results of preceding sign events that have caused them to occur (see Section 2).

9Ferrara & Hodge (2018) apply Clark’s (1996) notions of description, indication and depiction to the analysis of these different semiotic strategies in signing and speaking.

Abbreviations

INDEX = a pointing action with the index finger (except pointing to the 1st person), ONE-LIST = a manual list construction referring to the first listed item or topic, h2 = a sign is produced with the non-dominant hand, [–] = gloss for a sign requires several words, e.g. FIRST-TIME, [+] = several signs are produced as a prosodically unified sequence, e.g. YOUNGER+INDEX

Acknowledgements

The author wishes to thank Eleanor Underwood for checking the English of the paper. Also the financial support of Emil Aaltonen Foundation, and the Academy of Finland under grant 269089 (ProGram), is gratefully acknowledged.

Competing Interests

The author has no competing interests to declare.

References

  1. Atkin, Albert. 2013. Peirce’s Theory of signs. In Edward N. Zalta (ed.), The Stanford encyclopedia of philosophy (Summer 2013 Edition). (http://plato.stanford.edu/archives/sum2013/entries/peirce-semiotics) (Accessed 2018-08-31). 

  2. Braten, Stein & Colwyn Trevarthen. 2007. Chapter 1: Prologue. In Stein Braten (ed.), On being moved: From mirror neurons to empathy, 21–34. Amsterdam: John Benjamins. DOI: https://doi.org/10.1075/aicr.68 

  3. Brentari, Diane. 1998. A prosodic model of sign language phonology. Cambridge, MA: MIT Press. 

  4. Capirci, Olga. 2018. Visible bodily action in the emergence and development of speakers’ and signers’ languaging. A paper presented at the ISGS 8 conference. Cape Town, South Africa, July 4–8, 2018. 

  5. Cienki, Alan & Cornelia Müller (eds.). 2008. Metaphor and gesture. Amsterdam: John Benjamins. DOI: https://doi.org/10.1075/gs.3 

  6. Clark, Herbert H. 1996. Using language. Cambridge: Cambridge University Press. DOI: https://doi.org/10.1017/CBO9780511620539 

  7. Clark, Herbert H. 2003. Pointing and placing. In Sotaro Kita (ed.), Pointing: Where language, culture, and cognition meet, 243–268. Hillsdale, NJ: Erlbaum. DOI: https://doi.org/10.1075/gest.4.2.08gul 

  8. Cooperrider, Kensy & Rafael Nunez. 2009. Across time, across the body: Transversal temporal gestures. Gesture 9(2). 181–206. DOI: https://doi.org/10.1075/gest.9.2.02coo 

  9. Corina, David P. 1989. Recognition of affective and noncanonical linguistic facial expressions in hearing and deaf subjects. Brain Cognition 9(2). 227–37. DOI: https://doi.org/10.1016/0278-2626(89)90032-8 

  10. Corina, David P., Ursula Bellugi & J. Reilly. 1999. Neuropsychological studies of linguistic and affective facial expressions in deaf signers. Language and Speech 42(2–3). 307–31. DOI: https://doi.org/10.1177/00238309990420020801 

  11. Cormier, Kearsy, David Quinto-Pozos, Zed Sevcikova & Adam Schembri. 2012. Lexicalisation and delexicalisation processes in sign languages: Comparing depicting constructions and viewpoint gestures. Language & Communication 32(4). 329–348. DOI: https://doi.org/10.1016/j.langcom.2012.09.004 

  12. Cormier, Kearsy, Sandra Smith & Zed Sevcikova. 2015. Rethinking constructed action. Sign Language & Linguistics 18(2). 167–204. DOI: https://doi.org/10.1075/sll.18.2.01cor 

  13. Crasborn, Onno. 2012. Phonetics. In Roland Pfau, Markus Steinbach & Bencie Woll (eds.), Sign language: An international handbook (Handbooks of Linguistics and Communication Science 37), 4–20. Berlin: De Gruyter Mouton. 

  14. Crasborn, Onno & Els van der Kooij. 2013. The phonology of focus in Sign Language of the Netherlands. Journal of Linguistics 49(3). 515–565. DOI: https://doi.org/10.1017/S0022226713000054 

  15. Darwin, Charles R. 1872. The expression of the emotions in man and animals. 1st edition; London: John Murray. DOI: https://doi.org/10.1037/10001-000 

  16. de Saussure, Ferdinand. 1916. Cours de linguistique générale [Course in general linguistics]. Charles Bally, Albert Sechehaye & Albert Riedlinger (eds). Translated by Roy Harris from de Saussure. 5th impression (1998). London: Duckworth. 

  17. de Vos, Connie, Els van der Kooij & Onno Crasborn. 2009. Mixed signals: Combining linguistic and affective functions of eyebrows in questions in Sign Language of the Netherlands. Language and Speech 52(2). 315–339. DOI: https://doi.org/10.1177/0023830909103177 

  18. Dingemanse, Mark. 2017. On the margins of language: Ideophones, interjections and dependencies in linguistic theory. In Nick J. Enfield (ed.), Dependencies in language, 195–203. Berlin: Language Science Press. DOI: https://doi.org/10.5281/zenodo.573773 

  19. Dingemanse, Mark, Damián E. Blasi, Gary Lupyan, Morten H. Christiansen & Padraic Monaghan. 2015. Arbitrariness, iconicity, and systematicity in language. Trends in Cognitive Sciences 19(10). 603–15. DOI: https://doi.org/10.1016/j.tics.2015.07.013 

  20. Ekman, Paul & Wallace V. Friesen. 1969. The repertoire or nonverbal behavior: Categories, origins, usage, and coding. Semiotica 1. 49–98. DOI: https://doi.org/10.1515/semi.1969.1.1.49 

  21. Enfield, Nick J. 2009. The anatomy of meaning. Cambridge, UK: Cambridge University Press. DOI: https://doi.org/10.1017/CBO9780511576737 

  22. FAD = Finnish association of the deaf. 2018. Viittomakielet ja viittomakieliset [Sign languages and sign language users]. (www.kuurojenliitto.fi/fi/viittomakielet/viittomakielet-ja-viittomakieliset) (Accessed: 2018-09-17). 

  23. Ferrara, Lindsey & Gabriella Hodge. 2018. Language as description, indication, and depiction. Frontiers of Psychology 9(716). DOI: https://doi.org/10.3389/fpsyg.2018.00716 

  24. Ferrara, Lindsey & Rolf Piene Halvorsen. 2017. Depicting and describing meanings with iconic signs in Norwegian Sign Language. Gesture 16(3). 371–395. DOI: https://doi.org/10.1075/gest.00001.fer 

  25. Givens, David B. 2013. The nonverbal dictionary of gestures, signs & body language cues. Spokane, Washington, DC: Centre for Nonverbal Studies Press. 

  26. Gullberg, Marianne. 2006. Handling discourse: Gestures, reference tracking, and communication strategies in early L2. Language Learning 56(1). 155–196. DOI: https://doi.org/10.1111/j.0023-8333.2006.00344.x 

  27. Hanks, William F. 1990. Referential practice: Language and lived space among the Maya. Chicago, IL: University of Chicago Press. 

  28. Herrmann, Annika & Nina-Kristin Pendzich. 2014. Nonmanual gestures in sign languages. In Cornelia Müller, Alan Cienki, Ellen Fricke, Silva H. Ladewig, David McNeill & Jana Bressem (eds.), Body – language – communication: An international handbook on multimodality in human interaction (Handbooks of Linguistics and Communication Science 38.2), 2149–2163. Berlin/Boston: De Gruyter Mouton. DOI: https://doi.org/10.1515/9783110302028.2149 

  29. Hodge, Gabrielle & Lindsey Ferrara. 2014. Showing the story: Enactment as performance in Auslan narratives. In Lauren Gawne & Jill Vaughan (eds.), Selected papers from the 44th conference of the Australian Linguistic Society, 372–397. Melbourne: University of Melbourne. 

  30. Hopper, Paul J. 1998. Emergent grammar. In Michael Tomasello (ed.), The new psychology of language: Cognitive and functional approaches to language structure, 155–175. Mahwah, NJ: Erlbaum. 

  31. Ingold, Tim. 2011. Being alive: Essays on movement, knowledge, and description. London: Routledge. DOI: https://doi.org/10.4324/9780203818336 

  32. Jakobson, Roman. 1980. The framework of language (Michigan Studies in the Humanities 1). Ann Arbor, MI: Michigan Slavic Publications. 

  33. Jantunen, Tommi. 2016. Clausal coordination in Finnish Sign Language. Studies in Language 40(1). 204–234. DOI: https://doi.org/10.1075/sl.40.1.07jan 

  34. Jantunen, Tommi. 2017. Constructed action, the clause and the nature of syntax in Finnish Sign Language. Open Linguistics 3. 65–85. DOI: https://doi.org/10.1515/opli-2017-0004 

  35. Jantunen, Tommi. 2019. Elliptical phenomena in Finnish Sign Language. In Jeroen van Craenenbroeck & Tanja Temmerman (eds.), The Oxford handbook of ellipsis, 765–784. Oxford: Oxford University Press. DOI: https://doi.org/10.1093/oxfordhb/9780198712398.001.0001 

  36. Jantunen, Tommi, Birgitta Burger & Anna Puupponen. 2018a. Constructed action types and eye behavior in Finnish Sign Language. A paper presented at the 8th Conference of the International Society for Gesture Studies (ISGS 8) organized in Cape Town. South Africa, on July 4–8 2018. 

  37. Jantunen, Tommi, Birgitta Burger & Anna Puupponen. 2018b. The kinematics of constructed action in sign language narration: A motion capture study. A paper presented at the 51st Annual meeting of the Societas Linguistica Europaea (SLE 2018) organized in Tallinn. Estonia, August 29–September 1 2018. 

  38. Jantunen, Tommi, Johanna Mesch, Anna Puupponen & Jorma Laaksonen. 2016a. On the rhythm of head movements in Finnish and Swedish Sign Language sentences. In Jon Barnes, Alejna Brugos, Stefanie Shattuck-Hufnagel & Nanette Veilleux (eds.), Proceedings of Speech Prosody 2016, 850–853. Baixas, France: International Speech Communication Association. DOI: https://doi.org/10.21437/SpeechProsody.2016-174 

  39. Jantunen, Tommi, Outi Pippuri, Tuija Wainio, Anna Puupponen & Jorma Laaksonen. 2016b. Annotated video corpus on FinSL with Kinect and computer-vision data. In Eleni Efthimiou, Fotinea Stavroula-Evita, Thomas Hanke, Julie Hochgesang, Jette Kristoffersen & Johanna Mesch (eds.), Proceedings of the 7th Workshop on the Representation and Processing of Sign Languages: Corpus mining, 93–100. Paris: ELRA. 

  40. Johnston, Trevor. 1992. The realization of the linguistic metafunctions in a Sign Language. Language Sciences 14(4). 317–53. DOI: https://doi.org/10.1016/0388-0001(92)90021-6 

  41. Johnston, Trevor. 2013a. Towards a comparative semiotics of pointing actions in signed and spoken languages. Gesture 13(2). 109–142. DOI: https://doi.org/10.1075/gest.13.2.01joh 

  42. Johnston, Trevor. 2013b. Formational and functional characteristics of pointing signs in a corpus of Auslan (Australian sign language): Are the data sufficient to posit a grammatical class of ‘pronouns’ in Auslan? Corpus Linguistics and Linguistic Theory 9(1). 109–159. DOI: https://doi.org/10.1515/cllt-2013-0012 

  43. Johnston, Trevor. 2018. The role of headshake in negation in Auslan (Australian Sign Language): implications for signed language typology. Linguistic Typology 22(2). 185–231. DOI: https://doi.org/10.1515/lingty-2018-0008 

  44. Johnston, Trevor & Adam Schembri. 2010. Variation, lexicalization and grammaticalization in signed languages. Langage et Société 131. 19–35. DOI: https://doi.org/10.3917/ls.131.0019 

  45. Johnston, Trevor, Jane van Roekel & Adam Schembri. 2015. On the conventionalization of mouth actions in Auslan (Australian Sign Language). Language and Speech 59(1). 3–42. DOI: https://doi.org/10.1177/0023830915569334 

  46. Johnston, Trevor & Lindsey Ferrara. 2012. Lexicalization in signed languages: When is an idiom not an idiom? Selected papers from UK-CLA meetings 1. 229–248. 

  47. Kendon, Adam. 1988. Sign languages of Aboriginal Australia: Cultural, semiotic, and communicative perspectives. Cambridge: Cambridge University Press. 

  48. Kendon, Adam. 2002. Some uses of the head shake. Gesture 2(2). 147–182. DOI: https://doi.org/10.1075/gest.2.2.03ken 

  49. Kendon, Adam. 2004. Gesture: Visible action as utterance. Cambridge: Cambridge University Press. DOI: https://doi.org/10.1017/CBO9780511807572 

  50. Kendon, Adam. 2014. Semiotic diversity in utterance production and the concept of ‘language’. In Philosophical Transactions of the Royal Society B 369. DOI: https://doi.org/10.1098/rstb.2013.0293 

  51. Kita, Sotaro. 2003. Pointing: A foundational building block of human communication. In Sotaro Kita (ed.), Pointing: Where language, culture, and cognition meet, 1–8. Mahwah, NJ: Lawrence Erlbaum Associates. DOI: https://doi.org/10.1075/gest.4.2.08gul 

  52. Kockelman, Paul. 2005. The semiotic stance. Semiotica 157(1–4). 233–304. DOI: https://doi.org/10.1515/semi.2005.2005.157.1-4.233 

  53. LaBarre, Weston. 1947 (reprint 2006). The cultural basis of emotions and gestures. Journal of Personality 16(1). 49–68. DOI: https://doi.org/10.1111/j.1467-6494.1947.tb01075.x 

  54. Lackner, Andrea. 2017. Functions of head and body movements in Austrian Sign Language. Berlin/Boston: De Gruyter Mouton. DOI: https://doi.org/10.1515/9781501507779 

  55. Lakoff, George & Mark Johnson. 1980. Metaphors we live by. Chicago, IL: University of Chicago Press. DOI: https://doi.org/10.7208/chicago/9780226470993.001.0001 

  56. Liddell, Scott K. 2003. Grammar, gesture and meaning in American Sign Language. Cambridge: Cambridge University Press. DOI: https://doi.org/10.1017/CBO9780511615054 

  57. McCullough, Stephen, Karen Emmorey & Martin I. Sereno. 2005. Neural organization for recognition of grammatical and emotional facial expressions in deaf ASL signers and hearing nonsigners. Brain Research: Cognitive Brain Research 22(2). 193–203. DOI: https://doi.org/10.1016/j.cogbrainres.2004.08.012 

  58. McNeill, David, Justine Cassell & Elena T. Levy 1993. Abstract deixis. Semiotica 95(1). 5–19. DOI: https://doi.org/10.1515/semi.1993.95.1-2.5 

  59. Metzger, Melanie. 1998. Eye gaze and pronominal reference in American Sign Language. In Ceil Lucas (ed.), Pinky extension and eye gaze: Language use in deaf communities, 170–181. Washington, DC: Gallaudet University Press. 

  60. Morris, Desmond. 1994. Bodytalk: The meaning of human gestures. New York: Crown Publishers. 

  61. Nieminen, Tommi. 2010. Lajien synty: tekstilaji kielitieteen semioottisessa metateoriassa [Origin of genres: Genre in the semiotic metatheory of linguistics]. Jyväskylä: University of Jyväskylä PhD dissertation. 

  62. Okrent, Arika. 2002. A modality-free notion of gesture and how it can help us with the morpheme vs. gesture question in sign language linguistics (or at least give us some criteria to work with). In Richard P. Meier, Kearsy Cormier & David Quinto-Pozos (eds.), Modality and structure in signed and spoken languages, 175–198. Cambridge: Cambridge University Press. DOI: https://doi.org/10.1017/CBO9780511486777 

  63. Ormel, Ellen & Onno Crasborn. 2012. Prosodic correlates of sentences in signed languages. Sign Language Studies 12(2). 279–315. DOI: https://doi.org/10.1353/sls.2011.0019 

  64. Parmentier, Richard J. 1994. Signs and society: studies in semiotic anthropology. Bloomington, IN: Indiana University Press. DOI: https://doi.org/10.1525/jlin.1995.5.1.95 

  65. Peirce, Charles S. 1894. What is a sign? In Nathan Houser, Andre De Tienne, Jonathan R. Eller, Albert C. Lewis, Cathy L. Clark & D. Bront Davis (eds.), The Essential Peirce, Volume 2: Selected Philosophical Writings (1893–1913), 4–10. Bloomington, IN: Indiana University Press. 

  66. Peirce, Charles S. 1903a. Nomenclature and divisions of triadic relations, as far as they are determined. In Nathan Houser, Andre De Tienne, Jonathan R. Eller, Albert C. Lewis, Cathy L. Clark & D. Bront Davis (eds.), The Essential Peirce, Volume 2: Selected Philosophical Writings (1893–1913), 289–299. Bloomington, IN: Indiana University Press. 

  67. Peirce, Charles S. 1903b. On phenomenology. In Nathan Houser, Andre De Tienne, Jonathan R. Eller, Albert C. Lewis, Cathy L. Clark & D. Bront Davis (eds.), The Essential Peirce, Volume 2: Selected Philosophical Writings (1893–1913), 145–159. Bloomington, IN: Indiana University Press. 

  68. Peirce, Charles S. 1906. Prolegomena to an apology for pragmaticism. The Monist 16(4). 492–546. Oxford University Press Publication (https://www.jstor.org/stable/27899680) (Accessed 2018-09-17). DOI: https://doi.org/10.5840/monist190616436 

  69. Perniss, Pamela & Asli Özyürek. 2015. Visible cohesion: A comparison of reference tracking in sign, speech, and co-speech gesture. Topics in Cognitive Science 7(1). 36–60. DOI: https://doi.org/10.1111/tops.12122 

  70. Pfau, Roland & Josep Quer. 2010. Nonmanuals: Their grammatical and prosodic roles. In Diane Brentari (ed.), Sign languages: A Cambridge language survey, 381–402. Cambridge: Cambridge University Press. DOI: https://doi.org/10.1017/CBO9780511712203 

  71. Puhto, Janne. 2018. Päänpudistuksen käyttötavat ja frekvenssit suomalaisessa viittomakielessä. [The uses and frequencies of the headshake in Finnish Sign Language.] Jyväskylä: University of Jyväskylä MA thesis. 

  72. Puupponen, Anna. 2012. Horisontaaliset ja vertikaaliset päänliikkeet suomalaisessa viittomakielessä [Horizontal and vertical head movements in Finnish Sign Language]. Jyväskylä: University of Jyväskylä MA thesis. 

  73. Puupponen, Anna. 2018. The relationship between the movements and positions of the head and the torso in Finnish Sign Language. Sign Language Studies 18(2). 175–214. DOI: https://doi.org/10.1353/sls.2018.0000 

  74. Puupponen, Anna, Tommi Jantunen & Johanna Mesch. 2016. The alignment of head nods with syntactic units in Finnish Sign Language and Swedish Sign Language. In Jon Barnes, Alejna Brugos, Stefanie Shattuck-Hufnagel & Nanette Veilleux (eds.), Proceedings of Speech Prosody 2016, 168–72. Baixas, France: International Speech Communication Association. DOI: https://doi.org/10.21437/SpeechProsody.2016-35 

  75. Puupponen, Anna, Tommi Jantunen, Ritva Takkinen, Tuija Wainio & Outi Pippuri. 2014. Taking non-manuality into account in collecting and analyzing Finnish Sign Language video data. In Eleni Efthimiou, Fotinea Stavroula-Evita, Thomas Hanke, Julie Hochgesang, Jette Kristoffersen & Johanna Mesch (eds.), Proceedings of the 6th Workshop on the Representation and Processing of Sign Languages: Beyond the manual channel, 143–48. Paris: ELRA. 

  76. Puupponen, Anna, Tuija Wainio, Birgitta Burger & Tommi Jantunen. 2015. Head Movements in Finnish Sign Language on the basis of motion capture data: A study of the form and function of nods, nodding, head thrusts, and head pulls. Sign Language & Linguistics 18(1). 41–89. DOI: https://doi.org/10.1075/sll.18.1.02puu 

  77. Samovar, Larry A., Richard E. Porter & Edwin R. McDaniel. 2007. Communication between cultures (Wadsworth Series in Communication Studies, 6th ed.). Belmont CA: Thompson/Wadsworth. 

  78. Sandler, Wendy. 2012. Visual prosody. In Roland Pfau, Markus Steinbach & Bencie Woll (eds.), Sign language: An international handbook (Handbooks of Linguistics and Communication Science 37), 55–76. Berlin: De Gruyter Mouton. DOI: https://doi.org/10.1515/9783110261325.55 

  79. Schembri, Adam, Kearsy Cormier & Jordan Fenlon. 2018. Indicating verbs as typologically unique constructions: Reconsidering verb ‘agreement’ in sign languages. Glossa: a Journal of General Linguistics 3(1). 89. DOI: https://doi.org/10.5334/gjgl.468 

  80. Schoonjans, Steven. 2017. Nonmanual downtoning in German co-speech gesture and in German Sign Language. Yearbook of the German Cognitive Linguistics Association 5(1). 85–100. DOI: https://doi.org/10.1515/gcla-2017-0006 

  81. Stern, Daniel & Estelle Bender. 1974. An ethological study of children approaching a strange adult. In Richard C. Friedman, Ralph M. Richart & Raymond L. Va de Wiele (eds.), Sex differences in behavior, 233–258. New York: John Wiley & Sons. 

  82. Streeck, Jürgen. 2015. Embodiment in human communication. Annual Review of Anthropology 44. 419–438. DOI: https://doi.org/10.1146/annurev-anthro-102214-014045 

  83. Streeck, Jürgen, Charles Goodwin & Curtis LeBaron (eds.). 2011. Embodied interaction: Language and body in the material world. Cambridge: Cambridge University Press. 

  84. Taub, Sarah F. 2001. Language from the body: Iconicity and metaphor in American Sign Language. Cambridge: Cambridge University Press. DOI: https://doi.org/10.1017/CBO9780511509629 

  85. Thagard, Paul. 2010. The brain and the meaning of life. Princeton, NJ: Princeton University Press. DOI: https://doi.org/10.1515/9781400834617 

  86. Thompson, Robin, Karen Emmorey & Robert Kluender. 2006. The relationship between eye gaze and verb agreement in American Sign Language: An eye-tracking study. Natural Language and Linguistic Theory 24(2). 571–604. DOI: https://doi.org/10.1007/s11049-005-1829-y 

  87. Thompson, Robin, Karen Emmorey & Robert Kluender. 2009. Learning to look: The acquisition of eye-gaze agreement during the production of ASL verbs. Bilingualism: Cognition and Language 12(4). 393–409. DOI: https://doi.org/10.1017/S1366728909990277 

  88. van der Kooij, Els, Onno Crasborn & Wim Emmerik. 2006. Explaining prosodic body leans in Sign Language of the Netherlands: Pragmatics required. Journal of Pragmatics 38. 1598–1614. DOI: https://doi.org/10.1016/j.pragma.2005.07.006 

  89. Wagner, Petra, Zofia Malisz & Stefan Kopp. 2014. Gesture and speech in interaction: An overview. Speech Communication 57. 209–232. DOI: https://doi.org/10.1016/j.specom.2013.09.008 

  90. Wilbur, Ronnie B. 1999. Syntactic correlates of brow raise in ASL. Sign Language & Linguistics 2(1). 3–41. DOI: https://doi.org/10.1075/sll.2.1.03wil 

  91. Wilbur, Ronnie B. 2000. Phonological and prosodic layering of nonmanuals in American Sign Language. In Karen Emmorey & Harlan Lane (eds.), The signs of language revisited, 215–244. Mahwah, NJ: Lawrence Erlbaum. 

  92. Wilbur, Ronnie B. & Brenda S. Schick. 1987. The effects of linguistic stress on ASL signs. Language & Speech 30(4). 301–323. DOI: https://doi.org/10.1177/002383098703000402 

  93. Wilbur, Ronnie B. & Cynthia Patschke. 1998. Body leans and the marking of contrast in American Sign Language. Journal of Pragmatics 30(3). 275–303. DOI: https://doi.org/10.1016/S0378-2166(98)00003-4 

  94. Wilcox, Phyllis P. 2000. Metaphor in American Sign Language. Washington, DC: Gallaudet University Press. 

  95. Wilcox, Sherman & André Nogueira Xavier. 2013. A framework for unifying spoken language, signed language, and gesture. Todas as Letras-Revista de Língua e Literatura 15(1). 88–110. 

  96. Woll, Bencie. 2009. Do mouths sign? Do hands speak? Echo phonology as a window on language genesis. In Rudolf Botha & Henriette de Swart (eds.), Language evolution: The view from restricted linguistic systems, 203–224. Utrecht: LOT Occasional Series. 

comments powered by Disqus