1 Introduction
Look at the comic by Nathan W. Pyle in (1).
- (1)
Why is it funny? In one sentence, the duck interprets (or pretends to interpret) the directionality of the food-tossing motion depicted in the prohibition sign as part of the at-issue content of the sign—contrary to the intent of the sign, which aims to prohibit duck-feeding in general.2
Now, the duck could’ve made its life easier and avoided the need to bring in a ladder by choosing to treat other aspects of the depiction of a food-tossing event in the sign as at-issue—say, by demanding that the human toss four rather than three food morsels or that the morsels be tossed in a different spatiotemporal arrangement (e.g., one by one rather than all at once). It could’ve also demanded that the human toss the food with something other than their hand, that they put it on the ground rather than toss it, etc. In all these cases, the effect would be similar, although not exactly the same, and would be due to a mismatch between the intent of the prohibition sign and the at-issue interpretation of its aspects that are not meant to be interpreted as such.
In this paper, I point out empirical parallels between the at-issue vs. not-at-issue distinction in pictures and a specific subcase of this distinction in natural language and discuss the ensuing implications for our understanding of how meaning works across domains. This paper, thus, continues the tradition of applying formal linguistics tools to pictorial representations (Greenberg 2011; 2018; 2019; 2021; Abusch 2012; 2019; Abusch & Rooth 2017; Maier & Bimpikou 2019; Rooth & Abusch 2019; Cohn 2020, a.o.), but it expands on it in two specific ways. First, it raises the question of how the at-issue vs. not-at-issue distinction operates in the pictorial domain, which, to my knowledge, has only been raised by Schlenker (2019) so far. And second, it asks how meaning–form mappings operate architecturally in pictorial representations, a question that has been raised for sequences of images in pictorial narratives (Cohn 2020) and for highly conventionalized cases of semantic composition within smaller structures (e.g., speech and thought balloons in Maier 2019 or tagging in Greenberg 2019), but not for single image iconic pictorial representations of events or individuals. The findings of this paper point to substantial parallels between pictures and natural language at the level of pragmatic reasoning about meaning as well as at the level of compositional structuring of meaning.
The paper is organized as follows.
In section 2, I make descriptive generalizations about at-issueness in pictures. In particular, I demonstrate that which aspects of a given pictorial representation are interpreted as its at-issue content is determined through multi-factorial pragmatic reasoning. I furthermore show that in this respect not-at-issue pictorial content resembles non-restricting modifiers in language, i.e., instances of subsective modifiers that are truth-conditionally vacuous in the context of a given utterance.
In section 3, I provide evidence that some pieces of pictorial content are, in fact, treated as modifiers within a compositional structure, rather than sublexical pieces of holistic meanings, based on their behavior during ellipsis/anaphora resolution and alternative generation under only. I, therefore, conclude that the at-issue vs. not-at-issue distinction for such pieces of content is indeed best captured via the notions of restricting vs. non-restricting modification rather than the more obscure notion of “cosuppositions”, introduced in Schlenker 2018a for co-speech gestures and facial expressions and extended to other types of meaning-bearing expressions in Schlenker 2018b and to pieces of pictorial content in Schlenker 2019. This is in line with the similar treatment of various “secondary channel” meaning-bearing expressions in spoken language utterances that behave like subsective modifiers in Esipova 2019a; c; 2020.
Section 4 summarizes the main empirical, theoretical, and methodological points of the paper and makes suggestions for future research.
2 Pragmatics of (not-)at-issueness in pictures
2.1 At-issue vs. not-at-issue content
The at-issue vs. not-at-issue distinction is used in many different ways in the literature. It is often “diagnosed” via a range of “tests” (projection from under semantic operators, targetability by direct responses, ability to address questions under discussion, etc.), which seem to target distinct—even if potentially correlating—properties (see, e.g., Koev 2018 for an overview). For the purposes of this paper, I will set aside the question of whether “not-at-issueness” is a linguistically meaningful notion, or whether this is a descriptive umbrella term for several distinct types of content (presupposed, backgrounded, non-truth-conditional, etc.), which can pattern together along this or that dimension for potentially independent reasons.
Instead, I will mostly focus on a specific property often attributed to not-at-issue content, namely projection, i.e., failure of a given piece of content to interact with the semantic operators in whose syntactic scope the expression contributing said piece of content appears to be. A related property is contextual truth-conditional vacuity, i.e., lack of effect on the truth conditions of the utterance in a given context. Both can be illustrated for lexical presuppositions, i.e., presuppositions triggered by specific lexical items (which, similarly, need not be a homogeneous class). For instance, in (2), again contributes an inference that Kim has applied for this grant before. This contribution can’t be interpreted locally under the negated modal can’t, i.e., the sentence doesn’t have the reading that Kim isn’t allowed to apply for this grant if she has applied for it before—a reading under which the contribution of again would be at-issue. In other words, the contribution of again projects from under the negated modal. Relatedly, the contribution of again doesn’t affect the truth conditions of the utterance, i.e., in its context, (2) is truth-conditionally equivalent to the version of itself without again.
(2) | Kim can’t apply for this grant again. |
→ Kim has applied for this grant before. | |
≠ It’s not allowed that (Kim has applied for this grant before and applies again now). | |
= Kim can’t apply for this grant. |
This paper will focus on the at-issue vs. not-at-issue distinction in prohibition signs. The goal of a prohibition sign is to prohibit, and at the compositional level, they ostensibly involve embedding under negation or a negative modal,3 often represented by the circle-backslash symbol, resulting in an interpretation similar to No X! So, while the prohibition message intended by the sign in (1) could be paraphrased as No duck feeding! (or something more general like No waterfowl/wildlife feeding!), the duck chooses to interpret it along the lines of No duck feeding from above!4 In other words, it interprets the directionality of the tossing motion in the sign as at-issue while it wasn’t intended as such.
Note that in the case of prohibition signs projection perfectly coincides with another common conception of (not-)at-issueness, namely, (ir)relevance for the question under discussion (QUD), as the QUD in the case of prohibition signs is fixed to what is prohibited. This is not to say that projection from under semantic operators and ability to address some antecedent question always go hand in hand (cf. Simons et al. 2010 claiming that they do), however, prohibition signs allow us to side-step this issue. The built-in negation and the typically very clear pragmatic intent of prohibition signs, thus, make them a very convenient test ground for which parts of pictorial representations are interpreted as at-issue.5
2.2 Pictorial content, iconicity, and stylization
In order for a given surface aspect of a pictorial representation in a prohibition sign to be understood as an essential part of what’s prohibited, it has to be interpreted iconically, i.e., as part of the picture’s semantic content to begin with. However, when a given surface aspect of a pictorial representation is not interpreted as an essential part of what’s prohibited, it can be either because it is interpreted as part of the picture’s not-at-issue content, or because it is not interpreted as part of the picture’s content in the first place.6
For instance, in (1), the directionality of the food-tossing motion is not meant to be at-issue (all duck-feeding events are prohibited, including non-prototypical ones), but it is arguably still interpreted as part of the picture’s content, since a prototypical duck-feeding event involves tossing a few morsels of food from a higher position in space to a lower one.7 However, the exact number of the dots representing food morsels in the picture is arguably not interpreted iconically: while a plurality of food morsels is usually involved in a prototypical duck-feeding event (i.e., the fact that the number of the dots is higher than one does factor into the picture’s content), there doesn’t have to be exactly three. In other words, under this construal, the semantic content of this part of the picture would simply be ‘plurality of food morsels’, and the fact that there are three dots—as opposed to, say, four—is just a purely surface form property of this specific pictorial representation. This is not unlike how some phonetic contrasts are phonemic and can, thus, distinguish meaning in some languages, but not others (e.g., palatalization is phonemic in Russian, but not in English), except which contrasts are phonemic and, thus, potentially meaningful is fixed within a given language, but is typically established on a case by case basis in pictures.
Let me note, however, that three seems to often be the default number to represent a plurality of individuated atoms across modalities (see, e.g., Schlenker & Lamberton 2019 on how three repetitions can indicate any cardinality higher than two in sign language), i.e., the meaning–form mapping here seems to be fairly conventionalized. Some other aspects of pictorial representations, especially in information signs, can be due to stylization conventions specific to pictures (discussed, e.g., in Greenberg 2021) and, thus, will also not be interpreted hyper-iconically. This is reminiscent of onomatopoeic words and ideophones in spoken language, which have a regularized surface form and a conventionalized meaning–form mapping, although this mapping does have an iconic source. For instance, when I say The ball went swish, I aim to represent an event that produces—or at least can produce—a sound that resembles [swɪʃ], i.e., the surface form of swish does factor into its meaning contribution, but I do not intend for it to be interpreted hyper-iconically, i.e., the sound produced by the event does not have to be exactly [swɪʃ].
Of course, by the same logic, the directionality in the sign in (1) is only meant to be part of the picture’s content as a categorical distinction, but the exact angle of the relative position of the event participants is not interpreted iconically. It is, after all, not an accident that the duck chose to completely revert the relative position of the event participants rather than change the angle only slightly. Thus, the issue of resolving the mapping from the picture itself to its intended content is sensitive to which distinctions count as categorical. Of course, in the absence of pre-established categories (like in language), what counts as a categorical distinction for the purposes of a given picture will be highly context-dependent. This paper is not meant to investigate this issue in any detail, but it cannot be side-stepped entirely, since being part of a picture’s content is a pre-requisite for participating in the at-issue vs. not-at-issue distinction. Thus, this issue will keep resurfacing throughout the paper, and I will make my assumptions about what is and what isn’t part of the picture’s content clear whenever relevant.
2.3 Lexical meaning vs. pragmatic reasoning
So, to sum up, I assume that manner (tossing) and directionality (downwards) in the sign in (1) do contribute content, but the content they contribute is not intended to interact with the prohibition semantic operator, i.e., this content projects and is, thus, not-at-issue. But how do we decide which parts of a picture’s content are at-issue and which aren’t? In the case of lexical presuppositions, such as in (2), the at-issue vs. not-at-issue distinction is the property of a specific lexical item. Even if one does not assume that this distinction is hardcoded into the lexical entry of a given item, but rather arises via pragmatic reasoning, said reasoning still relies heavily on the lexical meaning of a given item.8 For instance, the fact that again almost inevitably contributes not-at-issue content is inextricably linked to its lexical semantics. So is the fact that it contrasts in this respect with the prefix re-, which makes a similar meaning contribution that also has a tendency to be not-at-issue, but is distinct enough from that of again to be able to make an at-issue contribution fairly easily, at least in some contexts:
(3) | Context: A grant coordinator is talking to an applicant about their application for a specific grant and wants to find out if they are applying for the first time or not, as different policies apply to first-time and non-first-time applicants. | |
a. | Are you reapplying? | |
b. | ??Are you applying again? | |
Intended in (a,b): ≈ ‘Are you applying for the second time?’ |
However, as I will show in this subsection, the nature of the at-issue vs. not-at-issue distinction in prohibition signs is not as rigidly determined by the lexical semantics of a given aspect of a pictorial representation.9 Instead, the way the at-issue vs. not-at-issue distinction is established for pictorial content is determined via multi-factorial pragmatic reasoning, in a way that is reminiscent of restricting vs. non-restricting modifiers in language.
The restricting vs. non-restricting distinction arises for subsective modifiers and is illustrated in (4). Under the restricting reading, obnoxious is intended to affect the truth conditions of the sentence, i.e., (4) isn’t contextually truth-conditionally equivalent to the version of itself without obnoxious. Instead, obnoxious is meant to pick out a potentially smaller subpart of the denotation of the expression it composes with. Under the non-restricting reading, however, obnoxious is meant to be contextually truth-conditionally vacuous, i.e., the sentence is meant to be contextually truth-conditionally equivalent to the version of itself without obnoxious. Instead, obnoxious is meant to contribute additional information about the denotation of the expression it composes with.
(4) | I don’t want any obnoxious semanticists at my talk. | |
a. | Restricting reading: | |
≠ I don’t want any semanticists at my talk. | ||
↛ All semanticists are obnoxious. | ||
b. | Non-restricting reading: | |
= I don’t want any semanticists at my talk. | ||
→ All semanticists are obnoxious. |
In other words, restricting modifiers are at-issue and non-restricting modifiers are not-at-issue. As I argue in Esipova 2019a; b, the difference between the two readings isn’t due to lexical ambiguity or different compositional structures. Thus, under both readings of (4), obnoxious composes with its sister like a regular subsective modifier; it is just that in the second case the subset it returns happens to be contextually equivalent to the input set. When we interpret sentences like (4), we are trying to figure out which reading was intended by the speaker, and many factors will come into play. These factors include, but are not limited to: our prior knowledge about the speaker’s beliefs and other contextual information; lexical semantics of the modifier (e.g., evaluative/subjective modifiers are often more likely to be non-restricting, because they, on the one hand, don’t make good restricting modifiers, and, on the other, are typically licensed by relevance considerations);10 focus (which tends to signal relevance for the question under discussion and, thus, correlates with the at-issue interpretation); and even the surface configuration (e.g., gestural subsective modifiers co-occurring with the spoken expressions they modify tend to be non-restricting). For instance, in (4), we might decide that the non-restricting interpretation is more likely if we have some pre-existing reasons to believe that the speaker considers all semanticists obnoxious. Or we might have no such prior beliefs, but might still entertain the non-restricting interpretation because of the evaluative and subjective nature of the adjective obnoxious. In the latter case, we would further reason about what would make (4) contextually truth-conditionally equivalent to the version of itself without obnoxious, and one obvious way in which that could be true is if the speaker thinks that all semanticists are obnoxious. The reader is referred to Esipova 2019a; b; for further, more in-depth discussion of the restricting vs. non-restricting distinction in natural language modifiers.
I will now show that the at-issue vs. not-at-issue distinction in pictures resembles the restricting vs. non-restricting modifier distinction in language in that it is also highly pragmatic. In section 3, I will, furthermore, argue that there are also structural similarities between the two, at least for the cases at hand—in other words, that the not-at-issue pieces of pictorial content we have encountered so far are, in fact, non-restricting modifiers.
Thus, when directionality in signs is interpreted as part of the sign’s content, it can be either at-issue or not-at-issue, depending on further pragmatic considerations. For instance, in (5), the directionality of the arrows is at-issue.
In pictorially very similar ‘No U-turn!’ signs, however, whether the arrow bends leftward or rightward is not-at-issue and is determined by whether the sign is from an area with right-hand (e.g., the Norwegian version in (6a)) or left-hand (e.g., the New Zealand version in (6b)) traffic.14
But to know whether a given piece of pictorial content is at-issue or not, one needs to have very specific world knowledge, for instance, that feeding wildlife in general is likely to be prohibited, but not specifically feeding wildlife from above. Even more obviously, correctly interpreting road traffic signs requires knowing traffic rules and a great deal of conventions. For instance, one could easily imagine a hypothetical rule that would prohibit U-turns leftward, but not rightward—but this is not a practical rule to have given how traffic works.
As anticipated in subsection 2.2, the same is true when it comes to distinguishing between aspects of pictorial representations that are interpreted iconically, i.e., as part of the picture’s content, but are not-at-issue, and those that are not meant to be interpreted iconically in the first place. For instance, in the sign prohibiting crowding in (7), certain aspects of how the plurality is depicted are meant to be interpreted iconically: while the exact number of the atoms depicted still isn’t meant to be interpreted precisely, their higher-than-the-default number and closer-than-the-default arrangement are meaningful—and, furthermore, at-issue. In contrast, while the sign in (8) also contains five atomic representations that are quite densely arranged, the sign doesn’t prohibit overfeeding birds, nor does it seem to convey anything about a prototypical amount or arrangement of food morsels fed to birds beyond it being a plurality (in particular, the difference between the three dots in the sign in (1) and the five dots in (8) doesn’t seem to be meaningful). Once again, this difference in interpretation arises due to world knowledge, namely, that crowding is something that is routinely prohibited, but over- or underfeeding wildlife is not, as well as that feeding a bird usually involves a plurality of food morsels, but the cardinality of that plurality can vary quite a lot.
(7) | 17 |
(8) | 18 |
2.4 The role of contrastive alternatives
Apart from world knowledge, contrast with explicit permitted alternatives also helps disambiguating between at-issue and not-at-issue readings of pictorial content in prohibition signs. This is, once again, reminiscent of how contrasting two (or more) alternatives brings about the at-issue interpretation of the content that is the locus of contrast, as illustrated for modifiers in (9) (see, e.g., Esipova 2018 for further discussion).19
(9) | I don’t want any obnoxious semanticists at my talk, but you can invite some nice semanticists. |
≠ I don’t want any semanticists at my talk, but you can invite some semanticists. | |
↛ All semanticists are {obnoxious, nice}. |
A case in point for pictures are public bathroom signs depicting acceptable and unacceptable ways of relieving oneself, such as the set of signs in (10) from a Sochi bathroom, which made the news during the 2014 Winter Olympics.
(10) | 20 |
First, the top right sign and the one below it in (10) don’t prohibit passing waste in general, but doing so in a specific position, which is in clear contrast to the sign in (11) that bans all public urination, but also conveys a not-at-issue meaning that people urinating in public typically do so in a standing position.21
(11) | 22 |
Second, the arrangement of the dots depicting pee droplets in the top right sign in (10) is meant to be interpreted a part of the picture’s content (once again, in contrast to the arrangement of the dots representing the food morsels in the sign in (1) or in (8)), but it conveys a not-at-issue meaning that peeing while standing results in droplets of pee spraying all over the place. Of course, to interpret these signs correctly, one needs a great deal of world knowledge, but also, the presence of the explicit permitted alternative in the top left corner (assuming that (10) provides the exhaustive list of the permitted alternatives) helps rule out the—hypothetically possible—at-issue interpretation of the pee droplet arrangement in the top right sign, whereby peeing while standing is OK as long as one avoids pee spraying.
The German bathroom signs in (12) make similar points explicitly, by contrasting acceptable and unacceptable alternatives and by providing spoken language instructions; once again, the arrangement of the droplet-depicting dots in these signs is meaningful, but not-at-issue.
(12) | 23 |
2.5 Optionality and licensing of non-restricting modifiers
Note that the spraying arrangement of the droplet-depicting dots in the prohibition signs in (10) and (12) is optional. Thus, the sign in (13) makes the same at-issue point, i.e., it prohibits peeing while standing, but without depicting all the mereological consequences of doing so.
(13) | 24 |
However, while the more complex depictions of the pluralities in (10) and (12) aren’t necessary, they are justified, as they explain the rationale behind the rules. As briefly mentioned before in subsection 2.3, this is similar to how non-restricting modifiers in language often need to be licensed by relevance considerations.
That said, not all non-restricting modifiers obey this relevance constraint. For instance, in language, modifiers that are morphosyntactically obligatory, such as phi-features on pronouns, which I argue to be modifiers that are obligatorily non-restricting for configurational reasons in Esipova 2019a, don’t have to contribute contextually relevant information.25 It is also possible that some non-restricting modifiers are non-obligatory, but don’t result in extra production effort and can, thus, circumvent the relevance constraint.
Similar considerations seem to apply to pictures: in the sign in (1), some relative spatial arrangement between the depictions of the agent and the goal of the tossing event is inevitable, so one might as well go with the prototypical one, without requiring that the downward direction of the tossing motion be particularly relevant. However, in a bathroom sign like the ones above, the artist can choose whether to go with a schematized, unembellished depiction of a pee stream or with an iconically modified version, in which case the modification has to be relevant in the context of the sign.
2.6 The role of prototypicality and exemplification
It is useful to think about how prototypicality, which has been mentioned a few times above, affects pictorial representations in prohibition signs and if/how it plays into establishing which parts of a depiction are meant to be iconic and which of the iconic ones are meant to be at-issue. In this subsection I outline some thoughts on this issue.
When one creates a sign prohibiting X, they have to depict a specific instance of X. This depiction can be stylized to some extent, but it still has to be recognizable as an instance of X. This already encourages one to depict a maximally prototypical X (although, of course, there’s still place for humor, artistic expression, bad artistry, etc.).
However, for some aspects of a given pictorial representation, one might have to make choices that are more or less arbitrary. In particular, if no obvious prototypical value for a given obligatory aspect of a pictorial representation exists, the artist may choose the value at random. For instance, the number of food morsels in ‘Don’t feed wildlife!’ signs can often be argued to be iconic and arbitrary rather than non-iconic and default or iconic and prototypical. Alex Warstadt (p.c.) pointed out to me that this is similar to how some speakers of English can use non-default gender on bound pronouns in an arbitrary way in some cases, without any universal inferences:
(14) | If you make a friend, you should be kind to her. |
↛ If you make a friend, that friend will have “she” pronouns. |
Next, sometimes a sign will deviate from prototypicality for practical reasons. An example of such deviation is illustrated in the sign in (15), which will resurface in the next section. As pointed out to me by Patrick Grosz (p.c.), this sign arguably does not represent the most prototypical coyote feeding event in terms of manner, since if a person was to feed a coyote, they would probably toss the food to it from a distance. However, the artist might have had to sacrifice prototypicality for practical reasons, since the depictions of the agent and the recipient of the feeding event have to co-exist within a small space while still being large enough to be visible. Note that, given this arguably non-prototypical manner of feeding, the directionality is, in fact, prototypical: if a human was to feed a coyote by handing the food to it, they would most likely do it from above.
(15) | 26 |
The nature of the food item in (15) doesn’t seem to be an obvious characteristic of a prototypical coyote-feeding event either.27 However, the food item choice in this case might be random, ironic, or, as pointed out to me by Paloma Jeretič (p.c.), it could be a prototypical food item one might have on them when they encounter a coyote rather than a prototypical food item one might want to feed to a coyote to properly satisfy its nutritional needs.
All these considerations will affect whether a pictorial representation will intend to convey anything about a prototypical X or not, but, once again, there doesn’t seem to be a general recipe. One has to do their own reasoning in each specific case, based on world knowledge, sign-related conventions adopted within a given community, etc.
Now, figuring out what’s at-issue and what’s not in a given sign prohibiting X is not directly about figuring out which aspects of the depicted instance of X are prototypical—it’s about figuring out what X is, i.e., what the sign intends to prohibit in the first place, abstracting away from all the irrelevant aspects of the specific depiction, whether or not they are meant to depict properties of a prototypical X. That said, making sure the at-issue message of the sign gets identified correctly is probably a factor that adds to the pressure of depicting a prototypical X. If a given depicted instance of X deviates from a prototypical X too much and for non-obvious reasons, that might make someone interpreting the sign wonder why that is so and whether the non-prototypical aspects of the depiction aren’t, in fact, meant to be part of its at-issue content.
The flip side of this is that since people are usually aware of the convention to depict prototypical entities in prohibition signs, they might draw generic inferences in the absence of the relevant world knowledge. For instance, if an outsider were to see the very Magrittean sign in (16a), which is an actual sign from a hiking trail on the island of Lokrum in Croatia and which differs from the more common and highly conventionalized ‘No smoking!’ sign depicting a cigarette in (16b), they might infer that in that area people tend to smoke pipes rather than cigarettes. Alternatively, they could conclude that only pipe-smoking is prohibited, implausible as it may sound. According to Chris Barker and Helen Koulidobrova (p.c.), however, the sign is intended to mean something like ‘No smoking, not even pipes!’, in order to counteract the—apparently, common—claims from the locals that the standard cigarette-depicting ‘No smoking!’ sign does not apply to pipe-smoking (which is, of course, related to the fact that many locals do indeed smoke pipes). Naturally, such intricate pragmatics is lost on people without the relevant world knowledge.
I would also like to point out that this discussion connects to the idea in Ebert 2017 that co-speech gestures sometimes perform the exemplification function. Ebert’s claim was about co-nominal pointing, but I think that (i) this idea can apply to depictive gestures, too, and (ii) exemplification gestures do not have to be co-speech. Thus, the idea of an ‘instance of X → prototypical instance of X → generic inference about X’ chain of reasoning as well as the various cases when the first link in the chain is broken are worth exploring for gestures as well.
This chain of reasoning can also apply to pictorial content other than prohibition signs that also aim to depict a prototypical example of some X, such as various cases of illustrations (in menus, textbooks, ads, etc.), which can result in various inferences about prototypical X.30 However, for the at-issue vs. not-at-issue distinction to arise, such content needs to be further presented in a context in which such a distinction would be applicable. For instance, the context would need to raise a QUD meant to be addressed by the picture at hand, so that we then can reason about which aspects of this picture are relevant for said QUD, which is one of the ways of conceptualizing at-issueness (again, see Koev 2018 for an overview).
For instance, if the central diagram in (17) is presented in the context of the questions ‘How to properly perform sumo deadlift?’ and/or ‘Which muscles does sumo deadlift work?’, many parts of its content will not be relevant to these questions. For example, while the depicted double-overhand grip is arguably the default grip for a prototypical sumo deadlift, it is not the only possible grip. In fact, the text under the diagram explicitly recommends using the double-overhand grip by default, but a mixed over-under grip for heavier loads, and the lifter in the top right diagram is using a mixed grip. The depicted grip could then be said to be not-at-issue in such contexts. This would be in contrast to diagrams intended to depict the different grips one can use on a deadlift or diagrams depicting lifts in which changing the grip changes the anatomy of the movement (e.g., various rowing movements). Even more obviously, the lifter in the central diagram appears to have a female body, but, of course, the sex of the lifter, would be irrelevant for the questions above. However, in this case, a better analysis would probably be in terms of an arbitrary value, akin to (14), rather than a prototypical one.
(17) | 31 |
The question of the at-issue vs. not-at-issue distinction in such cases can and should be investigated further, but since prohibition signs have a much clearer pragmatics and a built-in “test” for a specific property associated with (not-)at-issueness, namely, projection, this paper will continue to focus on them.
The upshot of this entire section is that there is no sure way to tell a priori which aspects of a given pictorial representation are meant to be part of its at-issue content and which are not, based solely on the geometric properties of the representation itself. Furthermore, among those aspects that are not part of a picture’s at-issue content, there is no sure a priori way to tell which of them are not meant to be part of the picture’s content to begin with, which are meant to depict accidental properties of this specific instance of X, and which are meant to depict properties of a prototypical X. Establishing all this requires a great deal of world knowledge and sometimes access to explicit contrastive alternatives, and even then there is often place for ambiguity.
3 Structural properties of (not-)at-issueness in pictures
3.1 Compositionally independent vs. sublexical not-at-issue content: what’s at stake?
In the previous section, I have drawn some empirical parallels between modifiers in language and various pieces of pictorial content such as directionality of events and mereological properties of individuals. In particular, I have likened the at-issue vs. not-at-issue distinction observed for such pieces of pictorial content to that observed for restricting vs. non-restricting interpretations of modifiers in language, based on the highly pragmatic nature of this distinction in both cases. In this section, I will reinforce this parallel and will argue that such pieces of pictorial content are, in fact, treated as modifiers within a compositional structure, rather than sublexical components of holistic pictorial meanings.
This is in contrast to a less specific treatment of such pieces of pictorial content as contributing “cosuppositions”, i.e., assertion-dependent presuppositions posited in Schlenker 2018a as inferences contributed by co-speech gestures, such as in (18), and co-speech facial expressions with an evaluative component, both of which are often not-at-issue,32 and extended to other types of meaning-bearing expressions in Schlenker 2018b and to meaningful subcomponents of pictures in Schlenker 2019.
(18) | a. | Lea shouldn’t be bringing her dogLARGE.33 |
Likely interpretation: | ||
= Lea shouldn’t be bringing her dog. | ||
→ Lea’s dog is large. | ||
b. | Zoe shouldn’t be shooting at the targetLONGBOW. | |
Likely interpretation: | ||
= Zoe shouldn’t be shooting at the target. | ||
→ If Zoe shoots at the target, she’ll be shooting a longbow. |
In the cases discussed by Schlenker, the form of “cosuppositions” is more or less equivalent to the inferences one would obtain if they were to treat the expressions contributing them as non-restricting modifiers. However, the notion of “cosuppositions” does not commit one to a stance (i) on the compositional status of the pieces of content that contribute them (whether these pieces of content are sublexical or compositionally independent, and if the latter, what kind of composition strategy they use), nor (ii) on how these inferences arise. In Esipova (2019a; c; 2020), I argue that, at least in the case of gestures and some evaluative facial expressions, Schlenker’s “cosuppositions” are, in fact, non-restricting modifier inferences, i.e., that (i) they are contributed by compositionally independent pieces of meaning, namely, subsective modifiers, and (ii) arise just like other non-restricting modifier inferences, i.e., via multi-factorial pragmatic reasoning.34
While the compositional independence of gestures in (18) is self-evident, this is not necessarily so for various pieces of pictorial content. In what follows I will show that there are reasons to believe that such pieces of pictorial content as directionality of events and mereological properties of pluralities are, in fact, compositionally independent and are, in particular, interpreted as modifiers. Therefore, the at-issue vs. not-at-issue distinction for them should be analyzed in terms of restricting vs. non-restricting modification rather than the more arcane notion of “cosuppositions”. Whether the latter notion might be needed elsewhere, i.e., whether we can find sublexical assertion-dependent presuppositions (or, alternatively, assertion-dependent presuppositions that arise between two expressions that are independent, but do not compositionally integrate with one another), remains to be seen, but the data presented in this section suggest that there is no need for “cosuppositions” when it comes to explaining the behavior of certain not-at-issue pieces of pictorial content.
3.2 Compositionally independent vs. sublexical content in speech and gesture
As observed in Esipova 2019a and expanded upon in Sailor & Colasanti 2020, when recovering ellipsis antecedents and computing alternatives negated by only, it is in principle possible to ignore non-restricting—and, therefore, not-at-issue—modifiers.35 The same holds when recovering antecedents for expressions like same. The relevant facts are illustrated in (19), where the non-restricting interpretation of the target modifier is enforced by the extralinguistic context, and the continuation sentences make sure that the modifier is ignored in the target environments.
(19) | Context: Lea has only one dog. | ||
a. | A: | Lea won’t be bringing {her enormous dog, her dogLARGE} to the party, as one of the guests is allergic. | |
B: | Ah, OK, then I won’t [ ] either. Even though mine is a York, and those are sometimes claimed to be hypoallergenic, but better safe than sorry. | ||
↛ B’s dog is large. | |||
(variation of Sailor & Colasanti 2020: (15a)) | |||
b. | Well, if Lea’s gonna bring {her enormous dog, her dogLARGE} to the party, then I’ll do the same. Especially because mine is tiny. | ||
↛ The speaker’s dog is large. | |||
c. | Only Lea brought {her enormous dog, her dogLARGE} to the party. No one else did. Not even Mia, who has a tiny, innocuous chihuahua. | ||
↛ Every other relevant person’s dog is large. |
Sailor & Colasanti (2020) furthermore observe that restricting—and, therefore, at-issue—modifiers categorically cannot be ignored under ellipsis. Unsurprisingly, the same is true for the other two environments. This is shown in (20), where the restricting interpretation of the target modifiers is enforced by the extralinguistic context and by evoking the salient alternatives with the help of contrastive focus.36 In all these cases, the continuations that would require ignoring the modifier in the target environments are completely infelicitous. If anything, the utterances before the continuations come with an inference that the speaker/relevant alternatives to Lea have multiple dogs of various sizes as well.
(20) | Context: Lea has two dogs, a small one and a large one. | ||
a. | A: | Lea won’t be bringing {her enormous dog, her dogLARGE}. | |
B: | Ah, OK, then I won’t [ ] either. #Even though mine is a York. | ||
= I won’t be bringing my large dog. | |||
b. | Well, if Lea’s gonna bring {her enormous dog, her dogLARGE} to the party, then I’ll do the same. #Especially because mine is tiny. | ||
= I’ll bring my large dog. | |||
c. | Only Lea brought {her enormous dog, her dogLARGE} to the party. No one else did. #Not even Mia, who has a tiny, innocuous chihuahua. | ||
→ No one else brought their large dog. |
As observed in Esipova 2019a, the paradigm in (19) is in contrast to sublexical presuppositions, i.e., presuppositions arising as part of the lexical meaning of certain words, which cannot be ignored in such environments:
(21) | a. | Jackie stopped smoking, {but Daisy didn’t, and Daisy did the same}. |
→ Daisy used to smoke. | ||
b. | Out of these three women, only Jackie stopped smoking. | |
→ Each of these three women used to smoke. | ||
c. | O-Ren knows that she is in danger, {but Vernita doesn’t, and the same is true for Vernita}. | |
→ Vernita is in danger. (under the sloppy reading) | ||
d. | Out of these three women only O-Ren knows that she is in danger. | |
→ Each of these three women is in danger. (under the sloppy reading) | ||
e. | Kim regrets cheating on the exam, {but Abernathy doesn’t, and the same is true for Abernathy}. | |
→ Abernathy cheated on the exam. | ||
f. | Out of these three women only Kim regrets cheating on the exam. | |
→ Each of these three women cheated on the exam. | ||
(Esipova 2019a: (7.4), adjusted) |
Note that it might be possible to locally accommodate the presupposition in some of the cases in (21), in which case the relevant inferences won’t arise, but then the presupposition still won’t be ignored, but will rather be treated as part of the at-issue content (e.g., Daisy didn’t in (21) will be read as ‘It’s not the case that (Daisy used to smoke and no longer does)’).
Next, Schlenker & Chemla (2018) observed that directional properties of gestures can be ignored under ellipsis and only,37 in a way reminiscent of phi-features on pronouns. They furthermore point out a contrast with some sublexical presuppositions, but do not attribute said contrast to the distinction between compositionally independent and sublexical meaning. However, both directional properties of gestures and phi-features on pronouns are analyzed as modifiers from the point of view of semantic composition38 in Esipova 2019a and are, thus, expected to pattern with other compositionally independent pieces of meaning during ellipsis/anaphora resolution and under only. A relevant paradigm is given in (22).39
(22) | Context: Zoe and Skyler are practicing face punches with sparring partners. | ||
a. | Zoei punchedPUNCH-HIGH heri sparring partner, {but Skyler didn’t, and Skyler did the same}. | ||
b. | Only Zoei punchedPUNCH-HIGH heri sparring partner. | ||
(a,b): | |||
→ Zoe’s sparring partner is taller than Zoe.40 | |||
→ Zoe has “she” pronouns. | |||
↛ Skyler’s sparring partner is taller than Skyler. | |||
↛ Skyler has “she” pronouns. | (Esipova 2019a: (7.2), adjusted) |
3.3 Compositional independence of directionality and mereology in pictures
If compositional independence is indeed a prerequisite for being able to be ignored in ellipsis/anaphora resolution and under only, we can use the behavior of a given piece of content in these environments to test whether it is compositionally independent or sublexical. Note that if a given piece of content cannot be ignored in these environments even when it is not-at-issue, we cannot necessarily conclude that it is not compositionally independent, as compositional independence is a necessary, but not a sufficient condition for such ignorability.
It turns out that directionality in pictures patterns with modifiers (and can be, in particular, compared directly to directional properties of gestures discussed at the end of the previous subsection). (23) and (24) show that when directionality is not-at-issue, it can be ignored in all the target environments (I use gapping in Russian as my ellipsis environment in (23), because Russian allows more gapping possibilities than English).
- (23)
- Context: In a zoo, the coyote enclosure has the following sign:
- The speaker nods at this sign and says:
- a.
- Žirafov
- giraffe.PL.ACC
- tože
- too
- nel’zja
- not-allowed
- [ ].
- ‘One can’t [feed] giraffes either.’ (Russian)
- b.
- A
- and/but
- žirafov
- giraffe.PL.ACC
- možno
- allowed
- [ ].
- ‘But one can [feed] giraffes.’ (Russian)
- c.
- The same applies to giraffes.
- d.
- This only applies to coyotes.
- (a–d): ↛ {Giraffes, all other relevant animals} are typically fed from above.
- (24)
- Context: Zoe and Lea are driving from country A, with left-hand traffic, to country B, with right-hand traffic. They are 0.5 km before the border when they see the following sign:
- Zoe nods at this sign and says:
- a.
- Eŝë
- more
- kilometr
- kilometer
- budet
- will-be
- nel’zja
- not-allowed
- [ ].
- ‘One can’t [do a U-turn] for another kilometer.’ (Russian)
- b.
- Čerez
- in
- kilometr
- kilometer
- budet
- will-be
- možno
- allowed
- [ ].
- ‘One can [do a U-turn] in a kilometer.’ (Russian)
- c.
- We won’t be able to [ ] for another kilometer.
- d.
- We’ll be able to [ ] in a kilometer.
- e.
- The same applies after we cross the border.
- f.
- This only applies until we cross the border.
- (a–f): ↛ U-turns in country B are rightward.
Conversely, when directionality is at-issue, it cannot be ignored in the target environments:
- (25)
- Context: Zoe and Lea are driving down the road and see the following sign:
- Zoe nods at this sign and says:
- a.
- Na
- on
- sledujuŝem
- next
- perekrëstke
- intersection
- tože
- too
- nel’zja
- not-allowed
- [ ].
- ✓ ‘One can’t [turn left] at the next intersection either.’
- ✕ ‘One can’t [turn] at the next intersection either.’ (Russian)
- b.
- A
- and/but
- na
- on
- sledujuŝem
- next
- perekrëstke
- intersection
- možno
- allowed
- [ ].
- ✓ ‘But one can [turn left] at the next intersection.’
- ✕ ‘But one can [turn] at the next intersection.’ (Russian)
- c.
- At the next intersection you can’t [ ] either.
- = At the next intersection you can’t turn left.
- ≠ At the next intersection you can’t turn.
- d.
- But at the next intersection you can [ ].
- = At the next intersection you can turn left.
- ≠ At the next intersection you can turn.
- e.
- The same applies at the next intersection.
- = Turning left is prohibited at the next intersection.
- ≠ Turning is prohibited at the next intersection.
- f.
- This only applies at this intersection.
- → It’s not the case that turning left is prohibited at all other relevant intersections.
- ↛ It’s not the case that turning is prohibited at all other relevant intersections.
The same holds for mereological properties of individuals in pictures. For instance, while feeding birds does typically involve giving them multiple small morsels of food, such as bread crumbs, that’s not true for all animals. Thus, the mereological properties of food in (26) can be ignored in all the target environments, since they are not-at-issue.
- (26)
- Context: The speaker nods at the following sign:
- a.
- Drugix
- other
- životnyx
- animal.PL.ACC
- tože
- too
- nel’zja
- not-allowed
- [ ].
- ‘One can’t [feed] other animals either.’ (Russian)
- b.
- A
- and/but
- drugix
- other
- životnyx
- animal.PL.ACC
- možno
- allowed
- [ ].
- ‘But one can [feed] other animals.’ (Russian)
- c.
- The same applies to other animals.
- d.
- This only applies to birds.
- (a–d): ↛ All other animals are typically fed multiple small morsels of food.
In (27), however, the mereological properties of the depicted plurality are at-issue and, thus, cannot be ignored in any of the target environments.
- (27)
- Context: The speaker sees the following sign near the exit and nods at it:
- a.
- Vnutri
- inside
- tože
- too
- ne
- not
- stoit
- should
- [ ].
- ✓ ‘[Standing in a crowd] isn’t recommended inside either.’
- ✕ ‘[Standing] isn’t recommended inside either.’ (Russian)
- b.
- A
- and/but
- vnutri
- inside
- možno
- allowed
- [ ].
- ✓ ‘[Standing in a crowd] is allowed inside.’
- ✕ ‘[Standing] is allowed inside.’ (Russian)
- c.
- The same applies inside.
- = People can’t stand in a crowd inside.
- ≠ People can’t stand inside.
- d.
- This only applies to the area near the exit.
- → It’s not the case that people can’t stand in a crowd at all other relevant areas.
- ↛ It’s not the case that people can’t stand at all other relevant areas.
The fact that directionality of events and mereological properties of individuals in pictures pattern with modifiers rather than sublexical content for the purposes of ellipsis/anaphora resolution and alternative generation under only suggests that interpreting the content of single image iconic pictorial representations involves a syntactic level of representation that feeds semantic composition, and, at least in the cases discussed here, these pieces of pictorial content are independent at that level of representation. Thus, the at-issue vs. not-at-issue distinction for these pieces of pictorial content is best captured in terms of restricting vs. non-restricting modification, in line with the treatment of subsective modifier gestures and facial expressions in Esipova 2019a; c; 2020.
Of course, it is possible that in some of the cases above it is not that a given piece of pictorial content is ignored when recovering the antecedent for ellipsis or anaphora, but rather that it is outside of the targeted constituent. That, of course, would still mean that this piece of content is compositionally independent, so the broad conclusion of this section would still hold.
4 Conclusion
In this paper, I have made several points about how we interpret the meaning of pictures:
Some aspects of pictorial representations are not meant to be interpreted iconically, i.e., as part of the picture’s content. But whether a given aspect of a pictorial representation is part of its content is not fixed based on what that aspect is, but instead depends on various pragmatic factors. Thus, the exact number of the dots representing food morsels and their arrangement are typically not meant to be interpreted iconically in the various ‘Don’t feed wildlife!’ signs (or, if they are, they are more or less arbitrary), but similar aspects of pictorial representations factor into the picture’s content much more in the ‘No crowding!’ sign in (7) or in the bathroom signs in (10) and (12).
Some aspects of pictorial representations are meant to be interpreted as part of the picture’s content, but are intended to be not-at-issue. Once again, whether this is the case within a given representation depends on various pragmatic factors. For instance, directionality is part of the picture’s not-at-issue content in the prohibition sign in (1) and in the ‘No U-Turn!’ signs in (6), but it is at-issue in the ‘No left turn!’ and ‘No way out!’ signs in (5). Similarly, various mereological properties of pluralities are not-at-issue in the bathroom signs in (10) and (12), but they are at-issue in the ‘No crowding!’ sign in (7). Furthermore, in some cases, not-at-issue pieces of content, such as the droplet arrangement in (10) and (12), seem to be subject to relevance considerations, similarly to non-restricting modifiers in language.
Such pieces of pictorial content as directionality of events and mereological properties of individuals pattern with modifiers in natural language for the purposes of ellipsis/anaphora resolution and alternative generation under only. That suggests that processing pictorial content involves a syntactic level of representation whose output is interpreted compositionally, and directionality of events and mereological properties of pluralities are independent at this level of representation and are, furthermore, treated as subsective modifiers (at least in the examples discussed here). Thus, the at-issue vs. not-at-issue distinction for such pieces of pictorial content should be analyzed in terms of restricting vs. non-restricting modification, following similar ideas for subsective modifier gestures and facial expressions in Esipova 2019a; c; 2020.
Methodologically speaking, prohibition signs proved useful in getting at the at-issue vs. not-at-issue distinction in pictures, as they, by virtue, involve embedding under negation. Relatedly, the pragmatics of prohibition signs is usually very clear, especially in the presence of explicit permitted alternatives, which makes judgements about what’s at-issue and what’s not fairly straight-forward and, furthermore, highlights the role of pragmatics for this distinction in pictures.
Another methodological take-away is that ellipsis/anaphora resolution and alternative generation under only have once again yielded empirical distinctions that can be used to test for presence of semantic composition in and beyond language. That said, we need to understand the relationship between these phenomena and composition better before drawing any definitive conclusions.
I would like to finish with a cautionary note about what I mean when I say that pictorial representations are interpreted compositionally. Based on the data discussed in this paper, the potential cognitive claim we could make is fairly weak: as humans, we can abstract away from certain aspects of a given pictorial representation and construe of it as a depiction of, say, a feeding or a peeing event tout court—and we can furthermore use these construals when we integrate pictorial content with spoken content, as we did in the ellipsis/anaphora/only examples in the previous section. However, we don’t simply ignore those aspects that we abstracted away from, as we can furthermore reason about whether they are still part of the picture’s content, and, if so, whether they qualify the depicted instance of a certain event or individual as a prototypical member of its class, how they contribute to the pragmatic intent of the pictorial representation, etc. That, of course, does not mean that the actual mental representations we have when processing pictorial content compositionally are exactly the same as the representations we have when processing content of natural language utterances compositionally.
However, we can investigate the extent to which compositional structures in pictures actually resemble those in natural language experimentally in the future. One sample experimental setup would involve presenting a participant with a video stimulus showing an event and then a drawing meant to depict the same event, but containing deviations from the original stimulus, e.g., different type of action, different type of object involved in the action, different directionality of the action, different shape of the object, etc. The participant will then be asked if the drawing accurately represents the original event (with the response options being on a pseudo-continuous scale from ‘yes’ to ‘no’, or including discrete options such as ‘yes’, ‘no’, ‘yes, but…’, etc.). These data could then be compared to similar data for linguistic utterances describing the stimulus event, with deviations targeting various types of syntactic objects (verbs, their arguments, different types of modifiers). Similarly, we can collect quantitative comparative data for ellipsis/anaphora resolution and alternative computation under only for different aspects of event depictions/descriptions in pictures vs. language. In addition, we can also include gesture-only utterances depicting the stimulus event to further probe the universality of compositional representations of event depictions/descriptions across domains.
Notes
- https://www.facebook.com/nathanwpyle2/photos/a.1377156059035720/2745700495514596/. [^]
- It is also funny because the duck is purportedly vocalizing, but is, in fact, producing pictorial utterances; the joke wouldn’t work as is if we tried to represent the duck’s utterances with spoken language only. However, the spirit of the joke could be preserved in the following exchange, for example:
[^]
(i) Duck: Toss me some food! Human: *points to the prohibition sign* Duck: OK, then put the food on the ground. - Said modal would be akin, for instance, to Russian nel’zja ‘not-allowed’/’not-possible’. It is not clear to me whether the modal is part of the lexical meaning of the circle-backslash symbol or is external to it, similarly to how it is external to no in No X! This issue is orthogonal to the goals of this paper, however. [^]
- Bold indicates prosodic contrastive focus marking, which helps enforce the at-issue interpretation of the modifier. [^]
- See also, e.g., Nöth 1995; Oversteegen & Schilperoord 2014 for a more general discussion of truth and negation in pictures. [^]
- The framing of the distinctions discussed in this subsection greatly benefited from the suggestions of an anonymous reviewer. [^]
- Relatedly, multiple commentators on the original Facebook post with the comic noted that the last panel evokes regurgitation rather than feeding, even though that clearly wasn’t the intended interpretation. [^]
- This issue is known in the presupposition literature as the triggering problem (see, e.g., Abusch 2010). [^]
- I use the terms lexical semantics, lexical meaning, etc. to refer to the meaning of a given expression that enters the compositional derivation, whether it is fully conventionalized and, thus, retrieved from the mental lexicon, is outputted by some module of interpreting iconic forms, or is computed via some combination of the two. However, the fact that we do not observe the same rigidity with respect to the at-issue vs. not-at-issue distinction in pictures as with some lexical presupposition triggers in language is likely related to the differences in conventionalization levels between the two. [^]
- See, e.g., Schlenker 2005; Leffel 2014 for discussion of the relevance constraint on non-restricting modifiers. [^]
- https://commons.wikimedia.org/wiki/File:New_Zealand_road_sign_R3-1.svg. [^]
- https://www.shutterstock.com/image-vector/no-ban-stop-signs-way-out-265647986. [^]
- Alicia Parrish (p.c.) pointed out to me that these signs are likely to be making an existential rather than prohibitory statement, saying that there is no exit in a certain direction, not that there is one, but one is not allowed to use it (e.g., it’s an emergency exit only). This is compatible with the backslash circle being a negative root modal, with the modal flavor ambiguous between deontic and ability, similarly to the Russian modal nel’zja mentioned before in footnote 3. [^]
- Thanks to Patrick Grosz (p.c.) for pointing out this example to me. [^]
- https://commons.wikimedia.org/wiki/File:Norwegian-road-sign-332.0.svg. [^]
- https://commons.wikimedia.org/wiki/File:New_Zealand_road_sign_R3-3.svg. [^]
- https://www.shutterstock.com/image-vector/no-crowd-vector-icon-flat-symbol-1610348866. [^]
- https://www.shutterstock.com/image-vector/stop-dont-feed-animals-sign-vector-418712962. [^]
- As I have already briefly noted in subsection 2.3, in language, an important phenomenon associated with contrastive alternatives is (contrastive) focus marking. It remains to be seen if there are any ways of marking focus in pictures. [^]
- https://twitter.com/SebToots/status/429546115604938752. [^]
- I thank Patrick Grosz (p.c.) for bringing out signs banning public urination to my attention. [^]
- https://www.mysecuritysign.com/persons-prosecuted-for-urinating-sign/sku-k2-0650. [^]
- https://immi.de/wc-schild-bitte-im-sitzen-pinkeln-saubere-toilette/. [^]
- https://www.shutterstock.com/image-illustration/do-not-pee-closet-while-standing-778632148. [^]
- Although, as I point out in Esipova 2019a: fn. 90, some speakers of English use gender-neutral they for people of any gender when the gender of the referent is not relevant, even if it is known. [^]
- https://www.flickr.com/photos/adactio/2156791720/. [^]
- In fact, one of the comments under the picture makes a joke about the nature of the food item in the sign in a way that makes it at-issue, contra to the intent of the sign: ‘Sign was cut off – “Do not feed the coyotes chocolate chip biscuits, they like honey snaps.”’ [^]
- The original photo (cropped for the example) was taken by Chris Barker. Many thanks to Chris for letting me use it here. [^]
- https://commons.wikimedia.org/wiki/File:No_Smoking.svg. [^]
- The generic nature of such pictures is discussed, in particular, in Greenberg 2018. [^]
- Delavier, Frédéric. 2001. Strength Training Anatomy, 1st edition. Human Kinetics, p. 72. [^]
- But not always; see Esipova 2019a; b; c; 2020 for details. [^]
- I write gesture labels in ALL CAPS. Co-speech gestures are written as subscripts, with underlining indicating their approximate temporal alignment without making any syntactic claims. New gestures are accompanied with illustrations, placed at their approximate onset. [^]
- I, furthermore, maintain that not all co-speech gestures or facial expressions are subsective modifiers and, thus, not all of them have a potential to give rise to non-restricting modifier interpretations; Schlenker’s notion of “cosuppositions” can’t apply in such cases either. The question of whether any compositionally independent pieces of pictorial content can be something other than main predicates, their arguments, or subsective (or even just intersective) modifiers is beyond the scope of this paper. However, let me note that, just like in the case of gestures, it is unlikely that purely iconic depictions can encode more complex meanings, i.e., arguably some level of conventionalization is required for more complex meanings. For instance, the backslash circle is clearly not a subsective modifier, but it is highly conventionalized. Other examples of more complex and more conventionalized semantic composition in the pictorial domain include, for instance, speech and thought balloons, whose content is analyzed as quoted in Maier 2019, and tags, discussed in Greenberg 2019, which, in the first approximation, are good candidates for supplements in the pictorial domain. [^]
- This is not to say that it is always easy. A lot of factors seem to affect whether a given non-restricting modifier is ignorable in a given context, often with gradient results. However, for the purposes of this paper what matters is the categorical distinction between pieces of meaning that are in principle ignorable and those that are not. [^]
- Note that restricting interpretations of co-speech gestures are degraded to a varying extent; see Esipova 2019a; b for details. [^]
- When they are not-at-issue; Schlenker & Chemla (2018) seem to assume that they always are. While that’s certainly true for phi-features on referring pronouns, that’s not necessarily true for directional properties of gestures, as discussed in Esipova 2019a. [^]
- I.e., whether or not phi-features are adjuncts or heads in the syntactic sense is irrelevant; what matters is their semantic type. [^]
- Schlenker & Chemla focus on non-co-speech gestures, but this difference is irrelevant for the issue at hand. [^]
- Or more precisely, Zoe’s sparring partner’s face is higher that Zoe’s. [^]
Acknowledgements
I am very grateful to two anonymous reviewers at Glossa for their helpful and insightful feedback, as well as Johan Rooryck for the open-minded effort he dedicated to this submission as an editor. I would also like to thank the participants of ‘Mean & W(h)ine’ and ‘Semantics Group’ at NYU for discussion. Special thanks to Patrick Grosz for many insightful comments.
Funding information
This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 891493.
Competing interests
The author has no competing interests to declare.
References
Abusch, Dorit. 2010. Presupposition triggering from alternatives. Journal of Semantics 27(1). 37–80. DOI: http://doi.org/10.1093/jos/ffp009
Abusch, Dorit. 2012. Applying discourse semantics and pragmatics to coreference in picture sequences. In Emmanuel Chemla, Vincent Homer & Grégoire Winterstein (eds.), Proceedings of Sinn und Bedeutung 17. 9–25.
Abusch, Dorit. 2019. Possible worlds semantics for pictures. To appear in Matthewson, Lisa, Cécile Meier, Hotze Rullmann, and Thomas Ede Zimmermann (eds.), The Companion to Semantics. Oxford: Wiley. https://compling.cis.cornell.edu/mr249/papers/abusch-possibleapr-29-2019.pdf.
Abusch, Dorit & Mats Rooth. 2017. The formal semantics of free perception in pictorial narratives. In Alexandre Cremers, Thom van Gessel & Floris Roelofsen (eds.), Proceedings of the 21st Amsterdam Colloquium. 85–95.
Cohn, Neil. 2020. Visual narrative comprehension: Universal or not? Psychonomic Bulletin and Review 2(27). 266–285. DOI: http://doi.org/10.3758/s13423-019-01670-1
Ebert, Cornelia. 2017. Co-speech vs. post-speech gestures. Talk given at Language and cognition workshop in memory of Peter Bosch, Osnabrück, February.
Esipova, Maria. 2018. Focus on what’s not at issue: Gestures, presuppositions, supplements under contrastive focus. In Uli Sauerland & Stephanie Solt (eds.), Proceedings of Sinn und Bedeutung 22, 385–402. DOI: http://doi.org/10.21248/zaspil.60.2018.473
Esipova, Maria. 2019a. Composition and projection in speech and gesture. New York University dissertation. https://ling.auf.net/lingbuzz/004676. DOI: http://doi.org/10.3765/salt.v29i0.4600
Esipova, Maria. 2019b. Composition and projection of co-speech gestures. In Katherine Blake, Forrest Davis, Kaelyn Lamp & Joseph Rhyne (eds.), Proceedings of Semantics and Linguistic Theory (SALT) 29, 111–137. DOI: http://doi.org/10.3765/salt.v29i0.4600
Esipova, Maria. 2019c. Towards a uniform super-linguistic theory of projection. In Julian J. Schlöder, Dean McHugh & Floris Roelofsen (eds.), Proceedings of the 22nd Amsterdam Colloquium, 553–562.
Esipova, Maria. 2020. Composure and composition. Ms., University of Oslo, in revision. https://ling.auf.net/lingbuzz/005003.
Greenberg, Gabriel. 2011. The semiotic spectrum. Rutgers University dissertation. DOI: http://doi.org/10.7282/T3HT2ND1
Greenberg, Gabriel. 2018. Content and target in pictorial representation. Ergo 5(33). 865–898. DOI: http://doi.org/10.3998/ergo.12405314.0005.033
Greenberg, Gabriel. 2019. Tagging: Semantics at the iconic/symbolic interface. In Julian J. Schlöder, Dean McHugh & Floris Roelofsen (eds.), Proceedings of the 22nd Amsterdam Colloquium, 11–20.
Greenberg, Gabriel. 2021. Semantics of pictorial space. Review of Philosophy and Psychology. DOI: http://doi.org/10.1007/s13164-020-00513-6
Koev, Todor. 2018. Notions of at-issueness. Language and Linguistic Compass 12(12). e12306. DOI: http://doi.org/10.1111/lnc3.12306
Leffel, Timothy. 2014. The semantics of modification: Adjectives, nouns, and order. New York University dissertation. https://ling.auf.net/lingbuzz/002212.
Maier, Emar. 2019. Picturing words: the semantics of speech balloons. In Julian J. Schlöder, Dean McHugh & Floris Roelofsen (eds.), Proceedings of the 22nd Amsterdam Colloquium, 584–592.
Maier, Emar & Sofia Bimpikou. 2019. Shifting perspectives in pictorial narratives. In M. Teresa Espinal, Elena Castroviejo, Manuel Leonetti, Louise McNally & Cristina Real-Puigdollers (eds.), Proceedings of Sinn und Bedeutung 23, 91–105.
Nöth, Winfred. 1995. SRB Insights: Can pictures lie? Semiotic Review of Books 6(2). 10–12.
Oversteegen, Eleonore & Joost Schilperoord. 2014. Can pictures say no or not? Negation and denial in the visual mode. Journal of Pragmatics 67. 89–106. DOI: http://doi.org/10.1016/j.pragma.2014.03.009
Rooth, Mats & Dorit Abusch. 2019. Indexing across media. In Julian J. Schlöder, Dean McHugh & Floris Roelofsen (eds.), Proceedings of the 22nd Amsterdam Colloquium, 612–624.
Sailor, Craig & Valentina Colasanti. 2020. Co-speech gestures under ellipsis: a first look. Talk given at The 94th Annual Meeting of the LSA, New Orleans.
Schlenker, Philippe. 2005. Minimize restrictors! (Notes on definite descriptions, condition C and epithets). In Emar Maier, Corien Bary & Janneke Huitink (eds.), Proceedings of Sinn und Bedeutung 9, 385–416.
Schlenker, Philippe. 2018a. Gesture projection and cosuppositions. Linguistics and Philosophy 41(3). 295–365. DOI: http://doi.org/10.1007/s10988-017-9225-8
Schlenker, Philippe. 2018b. Iconic pragmatics. Natural Language & Linguistic Theory 36(3). 877–936. DOI: http://doi.org/10.1007/s11049-017-9392-x
Schlenker, Philippe. 2019. Extending the typology of iconic presuppositions. Invited talk at the workshop Linguistic Investigations Beyond Language: Gestures, Body Movement and Primate Linguistics, ZAS Berlin.
Schlenker, Philippe & Emmanuel Chemla. 2018. Gestural agreement. Natural Language & Linguistic Theory 36(2). 587–625. DOI: http://doi.org/10.1007/s11049-017-9378-8
Schlenker, Philippe & Jonathan Lamberton. 2019. Iconic plurality. Linguistics and Philosophy 42(1). 45–108. DOI: http://doi.org/10.1007/s10988-018-9236-0
Simons, Mandy, Judith Tonhauser, David Beaver & Craige Roberts. 2010. What projects and why. In Nan Li & David Lutz (eds.), Proceedings of SALT 20, 309–327. DOI: http://doi.org/10.3765/salt.v20i0.2584