1 Theoretical background

1.1 What constitutes an idiom?

“Idiom” is a pre-theoretic notion, the meaning of which native speakers normally have some intuitive sense of, yet it is hard to pin down a single theory-neutral definition of what constitutes an idiom. Prototypical idioms exhibit certain basic properties, but the question of whether these should be considered defining properties demarcating the set of idioms is subject to debate.

For example, the meaning of idioms is normally understood to be figurative, yet some works argue that (non-literal) idioms and (literal) collocations should be treated as one class (e.g., Bruening 2020), and idioms themselves often consist of literal parts (e.g., miss the boat ~ ‘miss an opportunity’).1 Idioms are generally thought of as multi-word expressions, yet some works argue that words can also constitute phrasal idioms (e.g., Marantz 1996). Idioms are generally thought of as lexically fixed units, yet some idioms allow at least partial lexical substitution (e.g., grasp/clutch/seize at straws). Idioms have also historically been thought of as non-compositional, yet the idiomatic meaning of some idioms can arguably be distributed over their parts, and in that sense is compositional.

Thus, any theoretical investigation of idioms requires postulating a working definition of idioms. In the current research, we will treat idioms as (i) multilexemic expressions of which (ii) at least some part is non-literal (following Bruening 2020), and whose (iii) meaning is conventional, i.e., compositionally unpredictable based on the literal meanings of its constituents (following Nediger 2017; Horvath & Siloni 2019).

1.2 Why are idioms interesting?

At the heart of most generative theories of syntax and semantics lies the notion of language as a compositional phenomenon. That is, the meaning of a phrase or sentence is thought to be incrementally composed of the meaning of its constituent parts (words or lexemes), inserted from the lexicon, according to the hierarchical structure of syntax. While such an architecture of grammar is, by and large, well equipped at handling standard uses of language, this does not seem to be the case for idioms.

On the one hand, the meaning of an idiom (e.g., pull strings ~ ‘exert influence’) cannot be derived via standard composition of the meanings of its constituents in the same way a literal phrase can (e.g., pull weeds, denoting a literal event of pulling, of which weeds are the object). Rather, it seems to be conventionalized as a unit, requiring some form of listing akin to lexical items. On the other hand, like other compositional phrases (and unlike lexical items), the parts of many idioms seem to be accessible to syntactic manipulation: pull strings, for example, arguably retains its idiomatic meaning when passivized (“strings were pulled”), relativized (“the strings that he pulled”) or modified (“pull the political strings”), among others. This apparent dual nature of idioms – namely, that they behave in some senses like lexical items and in other senses like syntactically complex phrases – has rendered them a subject of interest for generative linguists throughout the years, most notably as a diagnostic for movement and syntactic structure (e.g., Vergnaud 1974; Chomsky 1981; Larson 1988).

Further complicating the matter, idioms seem to differ from one another in their syntactic flexibility: as opposed to pull strings, for example, kick the bucket (~ ‘die’) arguably does not retain its idiomatic meaning when passivized (#“the bucket was kicked”), relativized (#“the bucket that he kicked”) or modified (#“kick the political bucket”). Indeed, idioms have been shown to be less syntactically flexible on average than literal phrases (e.g., Fraser 1970; Nunberg et al. 1994; Wierzba et al. 2023a).

Seeing as idioms are by no means a marginal phenomenon of language (Jackendoff 1997), their behavior raises several important questions: (i) What kind of linguistic entities do idioms constitute, and how can their variable syntactic flexibility patterns be accounted for? (ii) What does the behavior of idioms reveal about the nature of the lexical component(s)? (iii) What can idioms teach us about the architecture of grammar? And is the standard architecture equipped to describe and account for idiom interpretation?

The current research focuses on experimentally examining the question of idiom syntactic flexibility, but its results have a direct bearing on the other questions at hand.

1.3 The syntactic flexibility patterns of idioms

Preliminary accounts of idiom syntactic flexibility patterns were of a rather stipulative nature. Wèinreich (1969), for example, suggested specifying the “transformational properties” of each idiom as part of its lexical entry, rendering the variability entirely idiosyncratic. Fraser (1970) suggested a slightly more systematic account, according to which idioms are arranged along a “frozenness hierarchy”, with each level of the hierarchy representing a class of transformations, and each idiom assigned to a certain level and able to undergo any transformation lower on the hierarchy. The lack of reasoning behind the ordering of transformations, as well as behind the assignment of idioms to levels, however, left much to be desired.

In an attempt to provide a more principled account, Nunberg et al. (1994) suggested a distinction between semantically decomposable idioms (which they termed “idiomatically combining expressions”) and semantically non-decomposable idioms (which they termed “idiomatic phrases”). The former are defined as “idioms whose parts carry identifiable parts of their idiomatic meanings”(496), and the latter as idioms “whose idiomatic interpretations cannot be distributed over their parts”(497).

For (a prototypical) example, the idiom pull strings mentioned above is decomposable, as its idiomatic meaning ‘exert influence’ can be distributed over its parts (where pull ~ ‘exert’ and strings ~ ‘influence’), whereas the idiom kick the bucket is non-decomposable, as its idiomatic meaning ‘die’ cannot be distributed over its parts.

According to Nunberg et al. (1994), decomposable idioms like pull strings are syntactically flexible because their idiomatic meaning is, in fact, compositional, whereas non-decomposable idioms like kick the bucket are syntactically inflexible because their idiomatic meaning is not. Such a theory is appealing, as it enables incorporating idioms within a compositional framework, with decomposable idioms behaving like literal compositional phrases, and non-decomposable idioms behaving like fixed lexical items.

Nunberg et al.’s argumentation has since led several authors (e.g., Nediger 2017; Bargmann & Sailer 2018; Wierzba et al. 2023b) to suggest that the inflexibility of non-decomposable idioms could follow independently from the general semantic restrictions imposed on phrase constituents by various syntactic operations. Thus, for example, the same semantic restrictions preventing the topicalization of someone in (1b) could also be responsible for preventing the topicalization of the bucket in (1c) under idiomatic interpretation:

    1. (1)
    1. a.
    1.   John he doesn’t like.
    1.  
    1. b.
    1. #Someone he doesn’t like.
    1.  
    1. c.
    1. #The bucket he didn’t kick.

Such an account avoids the need to posit separate modes of storage for decomposable and non-decomposable idioms (as suggested by Nunberg et al.), allowing instead for a unified treatment of all phrases, as will be further discussed in section 4.2.

2 Previous research

Nunberg et al.’s influential theory has since been supported by various works (e.g., McClure 2011; Hladnik 2017; Corver et al. 2019) and experimentally examined in several languages. These experiments have yielded mixed results, depending on the language, decomposability classification method, and experimental setup. In English, for example, some experiments report a significant effect of decomposability on syntactic flexibility for a range of syntactic operations (e.g., Gibbs & Nayak 1989), while others find no such effect (e.g., Wierzba et al. 2023a,b). Attempts to replicate English results in Italian have been largely unsuccessful (e.g., Tabossi et al. 2008), whereas in German, decomposability was found to affect syntactic flexibility for a host of operations (e.g., Wierzba et al. 2023a,b). Meanwhile, several studies that averaged across syntactic operations were able to detect a correlation between decomposability and syntactic flexibility, though the effects were rather weak (e.g., Tabossi et al. 2011 in Italian; Nediger 2017 in English). Decomposability classification has also been debated, with some studies reporting high levels of consistency and intersubject agreement (e.g., Gibbs & Nayak 1989), and others unable to replicate these results (e.g., Tabossi et al. 2008). Overall, the current empirical picture seems to point to a possible (weak) effect of decomposability on syntactic flexibility, potentially subject to cross-task and cross-linguistic differences, but its existence still needs to be further examined.

3 The current research

The current study aimed to experimentally examine the effects of decomposability on the syntactic flexibility of idioms as compared to non-idioms in Hebrew, a language not yet studied in this regard.

In classifying idiom decomposability, we chose not to use an idiom-paraphrase task (as in Gibbs & Nayak 1989), to avoid potential effects of paraphrase choice on perceived decomposability (see Maher 2013 for a fuller discussion). Instead, we relied on advanced linguistics students for classification, to whom the non-trivial knowledge required for such a classification would be more readily accessible (as argued, e.g., by Titone & Connine 1994). This choice also advantageously separated the participant pool, with decomposability classification performed by different individuals than those who subsequently assessed syntactic flexibility (naive participants), thereby ensuring the independence of observations. Finally, following concerns raised by Wierzba et al. (2023b), we also conducted a familiarity pretest to ensure that selected decomposable and non-decomposable idioms were familiar to a similar and sufficient extent.

Three separate experiments were then run on the selected idioms, each examining a different syntactic operation: (i) pronominalization, (ii) fronting, and (iii) adjectival modification. These were chosen with a view to assessing the effect of decomposability on a broad range of operations, as well ensuring as little overlap as possible in terms of the mechanisms that could potentially impose semantic restrictions on idioms (i.e., interpretation of idiom constituents through co-reference, movement of idiom constituents, and lexical insertion within idiom structure, respectively).

As in the decomposability pretest, our experiments did not use idiom-paraphrase similarity judgments (unlike Gibbs & Nayak 1989), to avoid potential effects of paraphrase choice on participants’ assessments of syntactic flexibility. Instead, we ran an acceptability rating task (as employed in Nediger 2017; Wierzba et al. 2023b) using a 7-point Likert scale. Our design included both a baseline consisting of idioms in their unmodified (canonical) form (the importance of which is exemplified in Wierzba et al. 2023a,b), as well as a direct comparison to the same phrases interpreted literally (addressing a possible confound in Wierzba et al. 2023a,b). The latter was achieved through introduction of preceding discourse contexts that elicited each of the desired interpretations.2 The introduction of discourse contexts also provided pragmatic motivation for the use of given idioms and structures, thereby avoiding underestimation of their acceptability (see Tabossi et al. 2009; Wierzba et al. 2023a for further discussion on discourse context effects).3

Overall, for each VP-idiom in each experiment, a set of four sentences was constructed, with the phrase appearing once in its canonical form and once under the relevant syntactic operation, once in a literal context and once in an idiomatic context. Decomposable and non-decomposable idioms were paired, resulting in a Latin square design consisting of eight sets of experimental sentences per experiment, assessing the effects of three factors: Type of Idiom (decomposable vs. non-decomposable), Type of Meaning (literal vs. idiomatic), and Type of Structure (baseline vs. modified). The sentences in each set were constructed to be as structurally similar as possible, to minimize effects other than those of the manipulated factors.

If the syntactic flexibility patterns of idioms are indeed the result of differences in idiom decomposability, we would expect to find a three-way interaction in all three experiments. That is, we would expect non-decomposable idioms to be rated significantly less acceptable than decomposable idioms when syntactically modified (as compared to in canonical form) and in idiomatic contexts (as compared to the same phrases in literal contexts). Moreover, we expect to replicate the finding that idioms are generally less syntactically flexible than non-idioms, with the aforementioned three-way interaction (if detected) indicating that the effect is at least partly due to the relative inflexibility of non-decomposable idioms. Finally, if the syntactic inflexibility of non-decomposable idioms is the result of semantic restrictions imposed on their constituents (rather than of their being fundamentally inflexible entities), we may expect to see differences in the nature and size of effect between experiments, assuming different operations could potentially impose different semantic restrictions and to different extents.

In systematically assessing the correlation between decomposability and syntactic flexibility, the current study aims to contribute to our understanding of the nature of idioms as compared to non-idioms. The extent to which the variable syntactic behavior of idioms can be explained by differences in semantic decomposability carries implications for our ability to incorporate idioms within a compositional framework of grammar, and in turn, for our conception of idiom storage and interpretation, as well as our overall conception of the lexicon and its interaction with the syntax and semantics, as will be further discussed in Section 4.

3.1 Pretests

3.1.1 Decomposability classification pretest

22 advanced students of linguistics from Tel Aviv University took part in the pretest.

Participants were first explained the notion of decomposability in detail, based on the definitions provided by Nunberg et al. (1994), with two canonical examples of each category provided as illustration.

Participants were then provided with a link to the pretest, uploaded on Google Forms, where they were asked to classify 30 VP-idioms (half of which were presumed to belong to each category) as either decomposable (D) or non-decomposable (ND). Participants also had the option of selecting “I’m unsure” (X) in the event that they were unable to come to a clear-cut decision.

The idioms were presented in pseudo-randomized order, alongside an example of their canonical use, in order to ensure that participants understood the intended interpretation.

The results of the decomposability classification pretest are presented in Figure 1.

Figure 1: Decomposability classification of 30 VP-idioms.

3.1.2 Familiarity pretest

33 participants with no prior linguistic background took part in the subsequent familiarity pretest, which was conducted on the same 30 VP-idioms. As these idioms were all relatively familiar, 10 further VP-idioms presumed to be less familiar were also included in the pretest to allow for a wider range of ratings.

Participants were provided with a link to the pretest, uploaded on Google Forms, where they were asked to judge on a 5-point Likert scale how common they thought each idiom was – that is, how often they felt they encountered each idiom relative to other phrases (with 1 indicating ‘never’ and 5 indicating ‘very often’).

As in the decomposability pretest, the idioms were presented in pseudo-randomized order, alongside an example of their canonical use.

Results of the familiarity pretest are shown in Figure 2.4

Figure 2: Mean familiarity ratings of the 30 VP-idioms included in the decomposability classification pretest. Idiom numbers match those that appear in Figure 1.

3.1.3 Idiom selection

Of the 30 VP-idioms, 8 decomposable idioms and 8 non-decomposable idioms were selected for the experiments.5

All selected idioms were deemed to belong to their respective categories by at least two thirds of pretest participants, with mean agreement rates standing at 82.95% (SD = 7.97%) for decomposable idioms and 84.41% (SD = 12.19%) for non-decomposable idioms.

In addition, all selected idioms received a mean familiarity rating of above average,6 with mean familiarity ratings standing at 3.41 (SD = 0.61, Range = 1.33) for decomposable idioms and 3.31 (SD = 0.73, Range = 1.18) for non-decomposable idioms, as illustrated in Figure 3.

Figure 3: Scatter plot of mean familiarity ratings for selected decomposable and non-decomposable idioms.

For a complete list of selected idioms, see Appendix A.

For the degree of variation in familiarity ratings between experimental sets, see Appendix B.

3.2 Experiment 1: pronominalization

Experiment 1 tested the effects of decomposability on pronominalization, namely the ability of idiom chunks to serve as antecedents for pronouns. Nunberg et al. (1994), and others consequently (e.g., Nediger 2017), have argued that for a constituent to be able to serve as an antecedent for a pronoun, it must refer to something implicit or explicit in the discourse – that is, it must have individual interpretation or reference. Hence, non-decomposable idioms should not be able to undergo pronominalization.

For example, John in (2a) is referential and hence allows pronominalization, whereas someone in (2b) is non-referential and hence does not; similarly, idiomatic strings in (2c) has idiomatic reference (strings ~ ‘influence’) and hence allows pronominalization, whereas idiomatic bucket in (2d) has no independent idiomatic reference and hence does not.

    1. (2)
    1. a.
    1.   I thought he’d met John, but he didn’t meet him.
    1.  
    1. b.
    1. #I thought he’d met someone, but he didn’t meet him.
    1.  
    1. c.
    1.   I thought he’d pulled the strings, but he didn’t pull them.
    1.  
    1. d.
    1. #I thought he’d kicked the bucket, but he didn’t kick it.

3.2.1 Design and materials

As aforementioned, the selected 8 decomposable and 8 non-decomposable idioms were paired, resulting in 8 sets. For each set, eight sentences were constructed involving three factors: (i) Type of Idiom (Decomposable vs. Non-Decomposable), (ii) Type of Meaning (Literal vs. Idiomatic), and (iii) Type of Structure (Baseline vs. Modified). This resulted in eight conditions per set, divided into: a Decomposable Subset: (a) DLB, (b) DLM, (c) DIB, (d) DIM; and a Non-Decomposable Subset: (e) NDLB, (f) NDLM, (g) NDIB, (h) NDIM.

Pragmatically suitable literal and idiomatic contexts (specifically, polarity contexts, as advised by Wierzba et al. 2023a) were constructed for each idiom to support the desired reading.

An example set is illustrated below, consisting of the paired decomposable idiom kataf et ha-perot (lit. ‘pick the fruits’, fig. ‘reap the rewards’) and non-decomposable idiom axal et ha-kova (lit. ‘eat the hat’, fig. ‘prove to be mistaken’).7

    1. (3)
    1. Decomposable subset:
    2. literal context:
    3. kše-higati la-mata huftati legalot še-kol ha-perot adayin al ha-ecim.
    4. ‘When I arrived at the orchard I was surprised to find that all the fruits were still on the trees.’
    1.  
    1. a.
    1. canonical form:
    1. hayiti
    2. was.1sg
    1. batu’ax
    2. sure
    1. še-ha-oved
    2. that-the-worker
    1. yiktof
    2. pick.fut
    1. et
    2. acc
    1. ha-perot.
    2. the-fruits
    1. ‘I was sure the worker would pick the fruits.’
    1.  
    1. b.
    1. pronominalization:
    1. hayiti
    2. was.1sg
    1. batu’ax
    2. sure
    1. še-ha-oved
    2. that-the-worker
    1. yiktof
    2. pick.fut
    1. et
    2. acc
    1. ha-perot
    2. the-fruits
    1. aval
    2. but
    1. le-hafta’ati
    2. to-surprise.gen.1sg
    1. hu
    2. he
    1. lo
    2. neg
    1. kataf
    2. pick.pst
    1. otam.
    2. them
    1. ‘I was sure the worker would pick the fruits but to my surprise he didn’t pick them.’
    1.  
    1. idiomatic context:
    2. kše-pagašti et ha-yazam huftati legalot še-ha-proyekt še-hu hiški’a bo kol-kax harbe nixšal.
    3. ‘When I met the entrepreneur I was surprised to discover that the project he had invested so much in failed.’
    1.  
    1. c.
    1. canonical form:
    1. hayiti
    2. was.1sg
    1. batu’ax
    2. sure
    1. še-ha-yazam
    2. that-the-entrepreneur
    1. yiktof
    2. pick.fut
    1. et
    2. acc
    1. ha-perot.
    2. the-fruits
    1. ‘I was sure the entrepreneur would reap the rewards.’
    1.  
    1. d.
    1. pronominalization:
    1. hayiti
    2. was.1sg
    1. batu’ax
    2. sure
    1. še-ha-yazam
    2. that-the-entrepreneur
    1. yiktof
    2. pick.fut
    1. et
    2. acc
    1. ha-perot
    2. the-fruits
    1. aval
    2. but
    1. le-hafta’ati
    2. to-surprise.gen.1sg
    1. hu
    2. he
    1. lo
    2. neg
    1. kataf
    2. pick.pst
    1. otam.
    2. them
    1. ‘I was sure the entrepreneur would reap the rewards but to my surprise he didn’t reap them.’
    1.  
    1. Non-decomposable subset:
    2. literal context:
    3. piniti et ha-aron biglal be’ayat aš aval šaxaxti be-toxo et ha-kova.
    4. ‘I cleared the closet because of a moth problem but I forgot the hat inside it.’
    1.  
    1. e.
    1. canonical form:
    1. hayiti
    2. was.1sg
    1. batu’ax
    2. sure
    1. še-ha-aš
    2. that-the-moth
    1. yoxal
    2. eat.fut
    1. et
    2. acc
    1. ha-kova.
    2. the-hat
    1. ‘I was sure the moth would eat the hat.’
    1.  
    1. f.
    1. pronominalization:
    1. hayiti
    2. was.1sg
    1. batu’ax
    2. sure
    1. še-ha-aš
    2. that-the-moth
    1. yoxal
    2. eat.fut
    1. et
    2. acc
    1. ha-kova
    2. the-hat
    1. aval
    2. but
    1. le-hafta’ati
    2. to-surprise.gen.1sg
    1. hu
    2. he
    1. lo
    2. neg
    1. axal
    2. eat.pst
    1. oto.
    2. it
    1. ‘I was sure the moth would eat the hat but to my surprise he didn’t eat it.’
    1.  
    1. idiomatic context:
    2. etmol pagašti et exad ha-maški’im še baxar be-zmano lo lehaški’a ba-startap ha-muclax šelanu.
    3. ‘Yesterday I met one of the investors who previously chose not to invest in our successful start-up.’
    1.  
    1. g.
    1. canonical form:
    1. hayiti
    2. was.1sg
    1. batu’ax
    2. sure
    1. še-ha-maški’a
    2. that-the-investor
    1. yoxal
    2. eat.fut
    1. et
    2. acc
    1. ha-kova.
    2. the-hat
    1. ‘I was sure the investor would eat his hat.’
    1.  
    1. h.
    1. pronominalization:
    1. hayiti
    2. was.1sg
    1. batu’ax
    2. sure
    1. še-ha-maški’a
    2. that-the-investor
    1. yoxal
    2. eat.fut
    1. et
    2. acc
    1. ha-kova
    2. the-hat
    1. aval
    2. but
    1. le-hafta’ati
    2. to-surprise.gen.1sg
    1. hu
    2. he
    1. lo
    2. neg
    1. axal
    2. eat.pst
    1. oto.
    2. it
    1. ‘I was sure the investor would eat his hat but to my surprise he didn’t eat it.’

The experimental items were distributed into four lists and presented in pseudo-randomized order, with participants evenly assigned to lists. Each list included a total of 48 sentences: 16 experimental items and 32 filler items of parallel length and varying acceptability. All in all, each participant saw two items from each experimental condition (a-h).

3.2.2 Participants and procedure

The experiment was set up using the Ibex Farm web platform (Zehr & Schwarz 2018).

In the experiment, participants were presented with sentences and a preceding context, one at a time, and asked to rate how acceptable each sentence sounded to them on a scale of 1 (“unacceptable”) to 7 (“acceptable”).

Participants were informed that the preceding context was only provided to describe the circumstance under which the sentence was uttered, and that they were not asked to rate the plausibility of the scenario described, but only how natural or acceptable the sentence itself sounded in Hebrew. They were also encouraged to answer intuitively and use the full range of the scale. For the complete set of instructions, see Appendix C.

63 native speakers of Hebrew took part in the experiment. 37 were recruited via prolific.co and received £3 for participation. Pre-screening filters were set to only include participants aged 18–35 situated in Israel, whose first language was Hebrew. On average, the experiment took around 15 minutes to complete.

3.2.3 Results

We opted to run Cumulative Link Mixed Models (CLMMs) for statistical analysis rather than Linear Mixed Models (LMMs), as the latter have been shown to increase Type I and Type II errors and impact effect size estimates when applied to ordinal rating data (Liddell & Kruschke 2018; Veríssimo 2021). All three manipulated factors were sum-coded in order to treat the levels of each factor symmetrically.8 Participants and sets (eight per experiment, as detailed above) were treated as random effects.9

The four highest ranked fillers and four lowest ranked fillers across participants were used as an exclusion criteria. Participants for whom the mean ranking of either of these groups deviated by more than one standard deviation from the same items’ mean ranking (±1 SD) across participants, or for whom more than one of these items showed a similar deviation in its individual ranking, were excluded from the study. All in all, 3 participant were excluded, resulting in a total of 60 participants.

According to the CLMM fit, a main effect was found for all three factors: Type of Idiom, Type of Meaning, and Type of Structure. That is, decomposable idioms were judged to be significantly more acceptable overall than non-decomposable idioms (p = 0.0059); literal phrases were judged to be significantly more acceptable overall than idiomatic ones (p < 0.001); and phrases in canonical form were judged to be significantly more acceptable overall than pronominalized phrases (p < 0.001). However, no significant interaction was found between any of the factors, as can be seen in Table 1, and as illustrated in Figure 4.

Table 1: Summary of CLMM statistical results for Experiment 1. Final model formula: Ranking ~ TypeOfIdiom * TypeOfMeaning * TypeOfStructure + (1 + TypeOfIdiom + TypeOfMeaning * TypeOfStructure | Participant) + (1 + TypeOfIdiom * TypeOfMeaning * TypeOfStructure | Set).

Contrast β SE z p
TypeOfIdiom 0.50 0.18 2.75 0.0059**
TypeOfMeaning 0.58 0.16 3.71 <0.001***
TypeOfStructure 0.57 0.14 4.02 <0.001***
TypeOfIdiom:TypeOfMeaning –0.08 0.10 –0.76 0.4501
TypeOfIdiom:TypeOfStructure 0.09 0.12 0.75 0.4532
TypeOfMeaning:TypeOfStructure –0.07 0.13 –0.52 0.6013
TypeOfIdiom:TypeOfMeaning:TypeOfStructure 0.09 0.08 1.01 0.3144

Figure 4: Box plot representing the acceptability results of Experiment 1, split by Type of Meaning (Literal vs. Idiomatic), Type of Idiom (Decomposable vs. Non-Decomposable), and Type of Structure (Baseline vs. Modified).

To ensure that the ratings of the phrases in literal contexts did not confound our results, we also examined the idiomatic contexts in isolation (Type of Meaning = Idiomatic), but still found no significant interaction between decomposability and syntactic flexibility, as can be seen in Table 2.

Table 2: Summary of CLMM statistical results for Experiment 1 for idiomatic contexts only. Final model formula: Ranking ~ TypeOfIdiom * TypeOfStructure + (1 + TypeOfIdiom * TypeOfStructure | Participant) + (1 + TypeOfIdiom * TypeOfStructure | Set).

Contrast β SE z p
TypeOfIdiom 0.58 0.15 3.79 <0.001***
TypeOfStructure 0.64 0.17 3.81 <0.001***
TypeOfIdiom:TypeOfStructure –0.01 0.13 –0.11 0.9108

3.2.4 Discussion

As can be seen, neither the predicted three-way interaction between our experimental factors, nor the predicted two-way interaction between Type of Meaning (literal vs. idiomatic) and Type of Structure (baseline vs. modified), were attested in Experiment 1. Thus, this experiment provided no evidence that non-decomposable idioms are less likely to undergo pronominalization than decomposable idioms, nor did it provide evidence that idioms are generally less likely to do so than non-idioms. We return to the implications of these results in Section 3.5 and in the general discussion.

3.3 Experiment 2: fronting

Experiment 2 tested the effects of decomposability on fronting, namely the ability to move idiom chunks to the left periphery. Various constraints on the fronting, or topicalization, of DP constituents have been previously argued for (mostly in English), such as the requirement that fronted constituents be either definite or generic (Fellbaum 1980), anaphoric or generic (Kuno 1972), referential and specific (Kiss 2002), or referential or generic (Nediger 2017). Under these assumptions, which partly overlap,10 non-decomposable idioms, whose DP constituents are presumably none of the above (as they carry no independent meaning), are not expected to allow fronting.

For example, John in (4a) and books in (4b) allow fronting because they are referential and generic respectively, whereas somebody in (4c) is not and hence does not allow fronting; similarly, idiomatic strings in (4d) allows fronting because it has independent idiomatic reference (strings ~ ‘influence’), whereas the bucket in (4e) does not and hence does not allow fronting.

    1. (4)
    1. a.
    1.   John he doesn’t like.
    1.  
    1. b.
    1.   Books he doesn’t like.
    1.  
    1. c.
    1. #Somebody he doesn’t like.
    1.  
    1. d.
    1.   The strings he didn’t pull.
    1.  
    1. e.
    1. #The bucket he didn’t kick.

Meanwhile, contrastive contexts have been argued to facilitate the interpretation of topicalized constituents (e.g., Kiss 2002; Nediger 2017). The contrast can presumably be either between different instances of whatever the object refers to (5a), or between different situations altogether (5b):

    1. (5)
    1. a.
    1. natali
    2. nataly
    1. omnam
    2. indeed
    1. fisfesa
    2. missed.f
    1. et
    2. acc
    1. ha-rakevet
    2. the-train
    1. ha-mukdemet
    2. the-early
    1. aval
    2. but
    1. la-rakevet
    2. to.the-train
    1. ha-meuxeret
    2. the-late
    1. hi
    2. she
    1. hispika.
    2. managed.f
    1. ‘Nataly may have missed the early train but the late train she managed to make.’
    1.  
    1. b.
    1. natali
    2. nataly
    1. omnam
    2. indeed
    1. hit’orera
    2. woke.up.f
    1. be-ixur
    2. in-delay
    1. aval
    2. but
    1. et
    2. acc
    1. ha-rakevet
    2. the-train
    1. hi
    2. she
    1. lo
    2. neg
    1. fisfesa.
    2. missed.f
    1. ‘Nataly may have woken up late but the train she didn’t miss.’

Seeing as in the current study, effort was made to ensure that the syntactic operations examined did not overlap, we elected to construct sentences with contrastive contexts of the type in (5b), considering those of the type in (5a) require the modification of idiom constituents in addition to fronting, which could impose separate semantic restrictions (to be assessed in isolation in Experiment 3).

3.3.1 Design and materials

The same design used in Experiment 1, involving 8 sets of paired decomposable and non-decomposable idioms, was used in Experiment 2.

An example set is illustrated below, consisting of the same paired decomposable idiom kataf et ha-perot (lit. ‘pick the fruits’, fig. ‘reap the rewards’) and non-decomposable idiom axal et ha-kova (lit. ‘eat the hat’, fig. ‘prove to be mistaken’).

    1. (6)
    1. Decomposable subset:
    2. literal context:
    3. be-ikvot ha-macav higi’u mitnadvim lesaye’a la-xakla’im ba-avodot ba-mata.
    4. ‘In light of the situation, volunteers came to help the farmers with their work at the orchard.’
    1.  
    1. a.
    1. canonical form:
    1. ha-mitnadvim
    2. The-volunteers
    1. asu
    2. did
    1. et
    2. acc
    1. rov
    2. most
    1. ha-avodot
    2. the-works
    1. ba-mata
    2. in.the-orchard
    1. aval
    2. but
    1. hem
    2. they
    1. lo
    2. neg
    1. katfu
    2. picked
    1. et
    2. ACC
    1. ha-perot.
    2. the-fruits
    1. ‘The volunteers did most of the work in the orchard, but they didn’t pick the fruits.’
    1.  
    1. b.
    1. fronting:
    1. ha-mitnadvim
    2. The-volunteers
    1. asu
    2. did
    1. et
    2. acc
    1. rov
    2. most
    1. ha-avodot
    2. the-works
    1. ba-mata
    2. in.the-orchard
    1. aval
    2. but
    1. et
    2. acc
    1. ha-perot
    2. the-fruits
    1. hem
    2. they
    1. lo
    2. neg
    1. katfu.
    2. picked
    1. ‘The volunteers did most of the work in the orchard, but the fruits they didn’t pick.’
    1.  
    1. idiomatic context:
    2. ha-proyekt ha-xadaš šel xevrat ha-haytek hitgala ke-haclaxa ve-ha-menahalim zaxu le-švaxim.
    3. ‘The high tech company’s new project turned out to be a success and the managers garnered praise.’
    1.  
    1. c.
    1. canonical form:
    1. ha-metaxnetim
    2. The-programmers
    1. asu
    2. did
    1. et
    2. acc
    1. rov
    2. most
    1. ha-avoda
    2. the-work
    1. ba-proyekt
    2. in.the-project
    1. aval
    2. but
    1. hem
    2. they
    1. lo
    2. neg
    1. katfu
    2. picked
    1. et
    2. acc
    1. ha-perot.
    2. the-fruits
    1. ‘The programmers did most of the work on the project but they didn’t reap the rewards.’
    1.  
    1. d.
    1. fronting:
    1. ha-metaxnetim
    2. The-programmers
    1. asu
    2. did
    1. et
    2. acc
    1. rov
    2. most
    1. ha-avoda
    2. the-work
    1. ba-proyekt
    2. in.the-project
    1. aval
    2. but
    1. et
    2. acc
    1. ha-perot
    2. the-fruits
    1. hem
    2. they
    1. lo
    2. neg
    1. katfu.
    2. picked
    1. ‘The programmers did most of the work on the project but the rewards they didn’t reap.’
    1.  
    1. Non-decomposable subset:
    2. literal context:
    3. kše-giliti še-ašim hištaltu li al ha-aron paxadeti yoter mi-kol še-hem yaharsu li et ha-kova ha-xadaš.
    4. ‘When I found out that moths took over my closet, I feared more than anything that they would ruin my new hat.’
    1.  
    1. e.
    1. canonical form:
    1. ha-ašim
    2. The-moths
    1. harsu
    2. ruined
    1. et
    2. acc
    1. rov
    2. most
    1. ha-bgadim
    2. the-clothes
    1. ba-aron
    2. in.the-closet
    1. aval
    2. but
    1. hem
    2. they
    1. lo
    2. neg
    1. axlu
    2. ate
    1. et
    2. acc
    1. ha-kova.
    2. the-hat
    1. ‘The moths ruined most of the clothes in my closet but they didn’t eat the hat.’
    1.  
    1. f.
    1. fronting:
    1. ha-ašim
    2. The-moths
    1. harsu
    2. ruined
    1. et
    2. acc
    1. rov
    2. most
    1. ha-bgadim
    2. the-clothes
    1. ba-aron
    2. in.the-closet
    1. aval
    2. but
    1. et
    2. acc
    1. ha-kova
    2. the-hat
    1. hem
    2. they
    1. lo
    2. neg
    1. axlu.
    2. ate
    1. ‘The moths ruined most of the clothes in my closet but the hat they didn’t eat.’
    1.  
    1. idiomatic context:
    2. lifney ha-mitkafa va’ada šel mumxim pirsema du’ax še-ta’an še-le-israel lo niškefet šum sakana mi-cafon.
    3. ‘Before the attack, a committee of experts published a report that claimed no danger is threatening Israel from the north.’
    1.  
    1. g.
    1. canonical form:
    1. ha-mumxim
    2. The-experts
    1. ta’u
    2. wrong.pst
    1. legabey
    2. about
    1. rov
    2. most
    1. ha-taxaziyot
    2. the-forecasts
    1. ba-du’ax
    2. in.the-report
    1. aval
    2. but
    1. hem
    2. they
    1. lo
    2. neg
    1. axlu
    2. ate
    1. et
    2. acc
    1. ha-kova.
    2. the-hat
    1. ‘The experts were wrong about most of the forecasts in the report but they didn’t eat their hat.’
    1.  
    1. h.
    1. fronting:
    1. ha-mumxim
    2. The-experts
    1. ta’u
    2. wrong.pst
    1. legabey
    2. about
    1. rov
    2. most
    1. ha-taxaziyot
    2. the-forecasts
    1. ba-du’ax
    2. in.the-report
    1. aval
    2. but
    1. et
    2. acc
    1. ha-kova
    2. the-hat
    1. hem
    2. they
    1. lo
    2. neg
    1. axlu.
    2. ate
    1. ‘The experts were wrong about most of the forecasts in the report but their hat they didn’t eat.’

3.3.2 Participants and procedure

The experiment was set up in a manner similar to that described in Experiment 1.

64 native speakers of Hebrew took part in the experiment, 38 of which were recruited via prolific.co and received £3 for participation, and 23 of which were recruited via Facebook and received ₪15 for participation.

3.3.3 Results

As in Experiment 1, we ran CLMMs for statistical analysis, with all three manipulated factors sum-coded in the same direction, and using the same random effects structure. 4 participants were excluded from the study according to the same exclusion criteria as in Experiment 1, resulting in a total of 60 participants.

According to the CLMM fit, a main effect was found for all three factors: Type of Idiom, Type of Meaning, and Type of Structure. That is, decomposable idioms were judged to be significantly more acceptable overall than non-decomposable idioms (p = 0.0207); literal phrases were judged to be significantly more acceptable overall than idiomatic ones (p = 0.0051); and phrases in canonical form were judged to be significantly more acceptable overall than phrases with fronted objects (p < 0.001). However, no other significant interaction was found between the factors, as can be seen in Table 3, and as illustrated in Figure 5.

Table 3: Summary of CLMM statistical results for Experiment 2. Final model formula: Ranking ~ TypeOfIdiom * TypeOfMeaning * TypeOfStructure + (1 + TypeOfIdiom * TypeOfMeaning * TypeOfStructure | Participant) + (1 + TypeOfIdiom * TypeOfMeaning | Set).

Contrast β SE z p
TypeOfIdiom 0.33 0.14 2.31 0.0207*
TypeOfMeaning 0.46 0.17 2.80 0.0051**
TypeOfStructure 0.54 0.11 4.75 <0.001***
TypeOfIdiom:TypeOfMeaning –0.30 0.19 –1.58 0.1140
TypeOfIdiom:TypeOfStructure 0.01 0.08 –0.16 0.8756
TypeOfMeaning:TypeOfStructure –0.09 0.07 –1.25 0.2122
TypeOfIdiom:TypeOfMeaning:TypeOfStructure 0.09 0.07 1.20 0.2317

Figure 5: Box plot representing the acceptability results of Experiment 2, split by Type of Meaning (Literal vs. Idiomatic), Type of Idiom (Decomposable vs. Non-Decomposable), and Type of Structure (Baseline vs. Modified).

To ensure that the ratings of the phrases in literal contexts did not confound our results, we also examined the idiomatic contexts in isolation (Type of Meaning = Idiomatic), but still found no significant interaction between decomposability and syntactic flexibility, as can be seen in Table 4.

Table 4: Summary of CLMM statistical results for Experiment 2 for idiomatic contexts only. Final model formula: Ranking ~ TypeOfIdiom * TypeOfStructure + (1 + TypeOfIdiom * TypeOfStructure | Participant) + (1 + TypeOfIdiom * TypeOfStructure | Set).

Contrast β SE z p
TypeOfIdiom 0.62 0.24 2.61 0.0089*
TypeOfStructure 0.68 0.23 2.96 0.0031**
TypeOfIdiom:TypeOfStructure –0.12 0.14 –0.84 0.4026

3.3.4 Discussion

As can be seen, neither the predicted three-way interaction between our experimental factors, nor the predicted two-way interaction between Type of Meaning (literal vs. idiomatic) and Type of Structure (baseline vs. modified), were attested in Experiment 2. Thus, this experiment provided no evidence that non-decomposable idioms are less likely to undergo fronting than decomposable idioms, nor did it provide evidence that idioms are generally less likely to do so than non-idioms. We return to the implications of these results in Section 3.5 and in the general discussion.

3.4 Experiment 3: adjectival modification

Experiment 3 tested the effects of decomposability on adjectival modification, namely the ability to modify idiom chunks through use of an adjective. In an early and influential paper, Ernst (1981) distinguishes between two types of adjectival modification: internal and external.11

In internal modification, the adjective, which syntactically attaches to the idiom-internal noun phrase, also semantically modifies its idiomatic denotation. For example, for the idiom pull strings (fig. ~ ‘exert influence’), one can arguably “pull political strings”, with political semantically modifying the idiomatic denotation of strings, such that the meaning of the expression can be roughly understood as ‘exert political influence’.

In external modification, on the other hand, while the adjective syntactically attaches to the idiom-internal noun phrase, it does not modify its idiomatic denotation, but rather that of the entire phrase. For example, for the idiom kick the bucket (fig. ~ ‘die’), one can arguably “kick the political bucket” in certain contexts, but political does not semantically modify the idiomatic denotation of bucket (which does not exist), but rather that of the phrase as a whole, such that the meaning can be adverbially paraphrased as “politically, kick the bucket”.

Generally, non-decomposable idioms should not allow internal modification, as an idiom chunk must carry individual idiomatic meaning for it to be semantically modified at the idiomatic level (as argued by Nunberg et al. 1994; McClure 2011; Nediger 2017). Non-decomposable idioms could however potentially be able to undergo external modification, as such modification semantically targets the phrase as a whole, which has an idiomatic denotation.

Thus, in the current experiment, we opted to construct sentences that involve internal modification (i.e., adjectival modification that is not adverbially paraphrasable), which decomposable idioms are expected to allow but not non-decomposable idioms.12

3.4.1 Design and materials

The same design used in Experiments 1 and 2, involving 8 sets of paired decomposable and non-decomposable idioms, was used in Experiment 3.

An example set is illustrated below, consisting of the same paired decomposable idiom kataf et ha-perot (lit. ‘pick the fruits’, fig. ‘reap the rewards’) and non-decomposable idiom axal et ha-kova (lit. ‘eat the hat’, fig. ‘prove to be mistaken’).

    1. (7)
    1. Decomposable subset:
    2. literal context:
    3. biglal ha-šitfonot lo nitan haya lekayem ha-šana katif mexani.
    4. ‘Because of the floods, it was impossible to conduct a mechanical fruit harvest this year.’
    1.  
    1. a.
    1. canonical form:
    1. ha-po’el
    2. The-worker
    1. ne’elac
    2. forced.unacc
    1. liktof
    2. pick.inf
    1. et
    2. acc
    1. ha-perot.
    2. the-fruits
    1. ‘The worker was forced to pick the fruits.’
    1.  
    1. b.
    1. adjectival modification:
    1. ha-po’el
    2. The-worker
    1. ne’elac
    2. forced.unacc
    1. liktof
    2. pick.inf
    1. et
    2. acc
    1. ha-perot
    2. the-fruits
    1. ha-bšelim.
    2. the-ripe
    1. ‘The worker was forced to pick the ripe fruits.’
    1.  
    1. idiomatic context:
    2. nesi rusya haya ha-marvi’ax ha-ikari me-ha-hitarvut ha-cvait be-surya.
    3. ‘The Russian president was the main beneficiary of the military intervention in Syria.’
    1.  
    1. c.
    1. canonical form:
    1. putin
    2. putin
    1. hiskil
    2. wizened
    1. liktof
    2. pick.inf
    1. et
    2. acc
    1. ha-perot.
    2. the-fruits
    1. ‘Putin managed to reap the rewards.’
    1.  
    1. d.
    1. adjectival modification:
    1. putin
    2. putin
    1. hiskil
    2. wizened
    1. liktof
    2. pick.inf
    1. et
    2. acc
    1. ha-perot
    2. the-fruits
    1. ha-mediniyim.
    2. the-political
    1. ‘Putin managed to reap the political rewards.’
    1.  
    1. Non-decomposable subset:
    2. literal context:
    3. kiviti še-ha-tarsis neged ašim yacil et ha-kova ha-yarok šeli aval le-ca’ari hu lo asa et ha-avoda.
    4. ‘I hoped the anti-moth spray would save my green hat but sadly it didn’t do the job.’
    1.  
    1. e.
    1. canonical form:
    1. ha-aš
    2. The-moth
    1. hispik
    2. made.it
    1. le’exol
    2. eat.inf
    1. et
    2. acc
    1. ha-kova.
    2. the-hat
    1. ‘The moth managed to eat the hat.’
    1.  
    1. f.
    1. adjectival modification:
    1. ha-aš
    2. The-moth
    1. hispik
    2. made.it
    1. le’exol
    2. eat.inf
    1. et
    2. acc
    1. ha-kova
    2. the-hat
    1. ha-yarok.
    2. the-green
    1. ‘The moth managed to eat the green hat.’
    1.  
    1. idiomatic context:
    2. havtaxoteha šel ha-xevra lefate’ax rexev otonomi laxalutin ad sof 2022 hitbadu.
    3. ‘The company’s promises to develop a completely autonomous vehicle by the end of 2022 proved false.’
    1.  
    1. g.
    1. canonical form:
    1. ha-xevra
    2. The-company
    1. ne’elca
    2. forced.unacc
    1. le’exol
    2. eat.inf
    1. et
    2. acc
    1. ha-kova.
    2. the-hat
    1. ‘The company was forced to eat its hat.’
    1.  
    1. h.
    1. adjectival modification:
    1. ha-xevra
    2. The-company
    1. ne’elca
    2. forced.unacc
    1. le’exol
    2. eat.inf
    1. et
    2. acc
    1. ha-kova
    2. the-hat
    1. ha-texnologi.
    2. the-technological
    1. ‘The company was forced to eat its technological hat.’

3.4.2 Participants and procedure

The experiment was set up in a manner similar to that described in Experiments 1 and 2.

67 native speakers of Hebrew took part in the experiment, 65 of which were recruited via Facebook and received₪ 15 for participation.

3.4.3 Results

As in Experiments 1 and 2, we ran CLMMs for statistical analysis, with all three manipulated factors sum-coded in the same direction, and using the same random effects structure. 7 participants were excluded from the study according to the same exclusion criteria as in Experiments 1 and 2, resulting in a total of 60 participants.

According to the CLMM fit, a main effect was found for all three factors: Type of Idiom, Type of Meaning, and Type of Structure. That is, decomposable idioms were judged to be significantly more acceptable overall than non-decomposable idioms (p = 0.0142); literal phrases were judged to be significantly more acceptable overall than idiomatic ones (p < 0.001); and phrases in canonical form were judged to be significantly more acceptable overall than phrases with adjectivally modified objects (p = 0.0180).

In addition, a significant interaction was found between Type of Meaning and Type of Structure (p = 0.0066), such that adjectivally modified phrases were judged to be significantly less acceptable than phrases in their canonical form in idiomatic contexts, but not so in literal contexts (irrespective of decomposability). No other significant interaction was found between the factors, as can be seen in Table 5, and as illustrated in Figure 6. The two-way interaction between Type of Meaning and Type of Structure is illustrated in Figure 7.

Table 5: Summary of CLMM statistical results for Experiment 3. Final model formula: Ranking ~ TypeOfIdiom * TypeOfMeaning * TypeOfStructure + (1 + TypeOfIdiom * TypeOfMeaning * TypeOfStructure | Participant) + (1 + TypeOfIdiom * TypeOfMeaning * TypeOfStructure | Set).

Contrast β SE z p
TypeOfIdiom 0.65 0.26 2.45 0.0142*
TypeOfMeaning 0.71 0.16 4.31 <0.001***
TypeOfStructure 0.28 0.12 2.37 0.0180*
TypeOfIdiom:TypeOfMeaning –0.15 0.15 –1.01 0.3136
TypeOfIdiom:TypeOfStructure –0.07 0.14 –0.54 0.5928
TypeOfMeaning:TypeOfStructure –0.30 0.11 –2.72 0.0066**
TypeOfIdiom:TypeOfMeaning:TypeOfStructure 0.14 0.12 1.22 0.2237

Figure 6: Box plot representing the acceptability results of Experiment 3, split by the Type of Meaning (Literal vs. Idiomatic), Type of Idiom (Decomposable vs. Non-Decomposable), and Type of Structure (Baseline vs. Modified).

Figure 7: Box plot representing the interaction between Type of Meaning (Literal vs. Idiomatic) and Type of Structure (Baseline vs. Modified) in Experiment 3.

To ensure that the ratings of the phrases in literal contexts did not confound our results, we also examined the idiomatic contexts in isolation (Type of Meaning = Idiomatic), but still found no significant interaction between decomposability and syntactic flexibility, as can be seen in Table 6.

Table 6: Summary of CLMM statistical results for Experiment 3 for idiomatic contexts only. Final model formula: Ranking ~ TypeOfIdiom * TypeOfStructure + (1 + TypeOfIdiom * TypeOfStructure | Participant) + (1 + TypeOfIdiom * TypeOfStructure | Set).

Contrast β SE z p
TypeOfIdiom 0.81 0.21 3.84 <0.001***
TypeOfStructure 0.59 0.15 4.01 <0.001***
TypeOfIdiom:TypeOfStructure –0.23 0.20 –1.14 0.2545

3.4.4 Discussion

As can be seen, the predicted three-way interaction between our experimental factors was not attested in Experiment 2, but contrary to the two previous experiments, the predicted two-way interaction between Type of Meaning (literal vs. idiomatic) and Type of Structure (baseline vs. modified) was. Thus, while this experiment provided no evidence that non-decomposable idioms are less likely to undergo adjectival modification than decomposable idioms, it did provide evidence that idioms are generally less likely to do so than non-idioms. We return to the implications of these results in the following section and in the general discussion.

3.5 Combined post-hoc analysis

Experiments 1-3 did not yield the predicted three-way interaction between Type of Idiom, Type of Meaning, and Type of Structure. However, since the contrast in acceptability between idioms in baseline and modified form was consistently larger for non-decomposable idioms than for decomposable idioms across all three operations, just not in a statistically significant way, we conducted a post-hoc analysis combining the data of all three experiments. This was done to increase statistical power and avoid the possibility of a Type II error (false negative), especially given that the frequentist analysis we employed cannot provide evidence for the null hypothesis.13

As in the three isolated experiments, we ran CLMMs for statistical analysis. According to the CLMM fit, a main effect was found for all three factors: Type of Idiom, Type of Meaning, and Type of Structure. That is, decomposable idioms were judged to be significantly more acceptable overall than non-decomposable idioms (p < 0.001); literal phrases were judged to be significantly more acceptable overall than idiomatic ones (p < 0.001); and phrases in canonical form were judged to be significantly more acceptable overall than modified phrases (p < 0.001).

In addition, a significant two-way interaction was found between Type of Meaning and Type of Structure (p = 0.0058), such that modified phrases were judged to be significantly less acceptable than phrases in their canonical form in idiomatic contexts, but not so in literal contexts (irrespective of decomposability).

Finally, a significant three-way interaction was found between our experimental factors (p = 0.0495), as originally predicted. That is, non-decomposable idioms were judged to be significantly less acceptable than decomposable idioms when syntactically modified (as compared to in canonical form) and in idiomatic contexts (as compared to the same phrases in literal contexts). These results can be seen in Table 7, and are illustrated in Figure 8.

Table 7: Summary of CLMM statistical results for three combined experiments. Final model formula: Ranking ~ TypeOfIdiom * TypeOfMeaning * TypeOfStructure + (1 + TypeOfIdiom * TypeOfMeaning * TypeOfStructure | Participant) + (1 + TypeOfIdiom * TypeOfMeaning * TypeOfStructure | Set).

Contrast β SE z p
TypeOfIdiom 0.48 0.12 4.14 <0.001***
TypeOfMeaning 0.56 0.09 5.98 <0.001***
TypeOfStructure 0.45 0.07 6.65 <0.001***
TypeOfIdiom:TypeOfMeaning –0.16 0.08 –1.92 0.0551
TypeOfIdiom:TypeOfStructure 0.002 0.06 –0.03 0.9784
TypeOfMeaning:TypeOfStructure –0.15 0.05 –2.76 0.0058**
TypeOfIdiom:TypeOfMeaning:TypeOfStructure 0.10 0.05 1.96 0.0495*

Figure 8: Box plot representing the acceptability results of the three combined experiments, split by Type of Meaning (Literal vs. Idiomatic), Type of Idiom (Decomposable vs. Non-Decomposable), and Type of Structure (Baseline vs. Modified).

4 General discussion

4.1 Summary and discussion of experimental findings

In a series of three experiments, we sought to examine the effect of semantic decomposability on idiom flexibility in terms of three syntactic operations: pronominalization, fronting, and adjectival modification.

Through a revised decomposability classification task, we were able to classify a set of 16 VP-idioms as either decomposable or non-decomposable with high inter-subject agreement, while also controlling for idiom familiarity. By selecting only verb phrases that allow for both a literal and an idiomatic reading, we were able to assess the effects of both idiomaticity and decomposability on syntactic flexibility within a single experimental design.

Assuming that the variable syntactic behavior of idioms reflects differences in semantic decomposability, we predicted a three-way interaction between our experimental factors. Specifically, non-decomposable idioms were expected to be rated significantly less acceptable than decomposable idioms when syntactically modified (as opposed to in canonical form) and in idiomatic contexts (as opposed to the same phrases in literal contexts). We also expected a two-way interaction between idiomaticity (i.e., Type of Meaning) and syntactic flexibility (i.e., Type of Structure), replicating previous findings that idioms are generally less syntactically flexible than literal phrases.

Individually, none of the three experiments yielded the predicted three-way interaction, and thus provided no evidence that non-decomposable idioms are less syntactically flexible than decomposable idioms. However, the results of the combined post-hoc analysis suggest that this absence may have stemmed from limited statistical power: a significant (though weak) three-way interaction did emerge in the unified analysis of all three experiments.

The predicted two-way interaction between Type of Meaning and Type of Structure was also observed in the combined post-hoc analysis. Individually, only Experiment 3 (adjectival modification) yielded such an effect.14 The aforementioned three-way interaction supports the notion that this overall difference in syntactic flexibility between idioms and literal phrases may be attributed, at least in part, to the effect of non-decomposability.

Because the combined analysis was not planned, and the significance of the main effect of interest (i.e., the three-way interaction) was marginal (p = 0.0495), these results should be interpreted cautiously. Thus, while the current research aligns with previous work in other languages suggesting a possible (weak) effect of decomposability on syntactic flexibility, the empirical cross-linguistic picture remains inconclusive, and warrants further experimental examination.

4.2 Implications for idiom storage and interpretation

As discussed in Section 1.2, the apparent dual nature of idioms – namely, that they behave in some senses like lexical items and in other senses like syntactically complex phrases – seems to pose challenges to classical assumptions regarding the lexicon-syntax interface. If idioms are stored as lexical units, how are their components accessible to syntactic manipulation? If the components making up idioms are stored individually, as in literal phrases, how is their apparently non-compositional meaning achieved? This duality is reflected in the various accounts proposed in the literature throughout the years of how idiom entries are stored and interact with the syntax. Broadly speaking, these accounts can be categorized into two groups.

Some theories, which we will term “holistic idiom storage theories”, assume idioms to be stored as one lexical unit, thus introducing more complex entities into the lexical inventory. These differ in their conceptualization of the type of information encoded in idiom entries (e.g., strings in Katz & Postal 1964; purely semantic representations in Chafe 1968; a combination of syntactic structure and phonological information in Jackendoff 1997 and Nediger 2017), as well as the mechanisms by which idioms are interpreted (e.g., Semantic Mutation in Chafe 1968; Unification in Jackendoff 1997; Matching in Nediger 2017).

Other theories, which we will term “componential idiom storage theories”, assume that idiomatic meaning is associated not with the idiom as a whole, but rather with each of its components (e.g., Everaert 2010; Bargmann & Sailer 2018) or with its head (e.g., Kay & Sag 2014; Horvath & Siloni 2019). These theories generally take the interpretation mechanisms of idioms to be no different than those of literal phrases, with co-occurrence or selectional restrictions specifying the context under which idiomatic interpretation arises. Under this heading, we also include non-lexicalist theories such as Distributed Morphology (e.g., Marantz 1997; Harley 2014), notwithstanding the fundamental difference that contextual meaning is only listed and introduced post-syntactically (as well as the fact that meaning can be ascribed to sub-word units, which is irrelevant to the current debate).

As aforementioned, the results of the current study point to a possible (weak) effect of decomposability on syntactic flexibility, but future studies are needed to replicate them before any definitive conclusions can be drawn. Assuming that these results can be replicated, their tentative implications on idiom storage and interpretation are as follows.

As mentioned in Section 1.3, Nunberg et al. (1994) suggest that the effect of semantic decomposability on syntactic flexibility arises from decomposable and non-decomposable idioms being stored differently: while decomposable idioms are componentially stored, and hence flexible, non-decomposable idioms are holistically stored, and hence inflexible. Later work (e.g., Nediger 2017; Bargmann & Sailer 2018; Wierzba et al. 2023b) suggests instead that the inflexibility of non-decomposable idioms could follow independently from the general semantic restrictions imposed on idiom constituents by various syntactic operations. This view is typically associated with componential idiom storage theories, under which parts of non-decomposable idioms (and specifically, the object in VP-idioms) are assumed to carry either a null interpretation (e.g., Everaert 2010) or a redundant interpretation (e.g., Bargmann & Sailer 2018), thus violating semantic restrictions when syntactically manipulated (as detailed in Sections 3.2–3.4).

Prima facie, the weak nature of the detected effect of decomposability on syntactic flexibility, along with the relative flexibility exhibited by all idioms in our study (including non-decomposable ones), appears to be less compatible with non-decomposable idioms being holistically stored.15 A holistic storage account would seem to predict non-decomposable idioms to be categorically inflexible, explaining the “strong correlation between semantic analyzability and transformational productivity” (Nunberg et al. 1994: 508), but this correlation is not borne out by our results.16

A semantic restrictions account of non-decomposable idiom inflexibility, on the other hand, would appear less rigid, since the scope of such restrictions could differ across operations, and since possible ways of bypassing certain semantic constraints have been argued for in other languages.17 Our original aim to independently analyze syntactic operations involving different mechanisms (namely, co-reference, movement, and lexical insertion) was meant to examine this possibility. However, the need for a combined post-hoc analysis to achieve sufficient statistical power ultimately prevented us from pursuing this line of inquiry further.

The latter account of non-decomposable idiom inflexibility also appears theoretically simpler, and hence preferable, since semantic restrictions on syntactically manipulated constituents have been argued for independently of idiom behavior, and since it allows for a unified, compositional treatment of all phrases, avoiding the need to introduce more complex entities into the lexical inventory.

As for decomposability classification, while our pretest showed that it is generally possible to classify at least a subset of idioms as decomposable or not with high intersubject agreement, the indeterminate classification of a considerable portion of idioms in the pretest appears to cast some doubt on the categorical nature of the classification. One possible explanation is that the classification is indeed categorical, but speaker dependent. That is, the meaning attributed to a given idiom may not be uniform across speakers, leading them to categorize idioms differently as either decomposable or non-decomposable, differences that could, in turn, be reflected in their syntactic flexibility.

Alternatively, decomposability may be better conceived of as a graded property rather than a categorical one, with certain idioms more amenable to attributing parts of their (often complex and nuanced) meaning to their constituents than others. Thus, for example, an indeterminately classified idiom such as herim et ha-kfafa (lit. ‘pick up the glove’, fig. ‘accept the challenge presented’) may in principle allow parts of its meaning to be attributed to its constituents (with pick up ~ ‘accept’ and glove ~ ‘challenge presented’). However, doing so may not be as natural as for a more definitively decomposable idiom such as kataf et ha-perot (lit. ‘pick the fruits’, fig. ‘reap the rewards’), which is based on a direct analogy. While the results of the current research do not provide any insight as to which of these accounts is preferable, the former would seem to be more compatible with current notions of idiom representation under both holistic and componential idiom storage theories, none of which involve any graded mechanisms.18,19

5. Conclusion and outlook

The current study aimed to contribute to the cross-linguistic empirical landscape regarding the purported role of decomposability in shaping the syntactic flexibility patterns of idioms. In a series of three experiments, we systematically examined the effect of decomposability on the syntactic flexibility of Hebrew idioms, a language not yet studied in this regard.

While none of the experiments provided evidence that non-decomposable idioms are less syntactically flexible than decomposable idioms, a post-hoc analysis suggests that this may have been due to a lack of statistical power: a (weak) effect of decomposability on syntactic flexibility was detected when analyzing the data of all three experiments combined.

These results align with previous work in other languages reporting a (weak) effect of decomposability on syntactic flexibility, specifically when averaging across syntactic operations (e.g., Tabossi et al. 2011 for Italian; Nediger 2017 for English). However, the fact that such an effect was only found in a post-hoc analysis suggests that these results should be interpreted with caution until they are further examined and replicated.

The current study points to several directions for future research. First, and perhaps most evidently, the statistical power of the three individual experiments needs to be increased in order to examine the effect of decomposability on syntactic flexibility for each of the operations in isolation, as originally intended. This, we argued, could help assess the possibility of operation-dependent effects of decomposability, which, in turn, could have implications for competing theories of idiom storage (namely, by providing support for a semantic restrictions account of non-decomposable idiom inflexibility). In addition, the range of syntactic operations assessed should be expanded to determine whether our findings extend to other operations involving different mechanisms, such as passivization, nominalization, or relativization.

Second, it is worthwhile to explore the question of whether speaker-dependent effects can explain the indeterminate decomposability classification of some idioms, thereby maintaining a categorical view of decomposability. The syntactic behavior of indeterminately classified idioms should also be further assessed (as was done in Wierzba et al. 2023b), as should the question of whether a graded treatment of decomposability can better account for the syntactic flexibility patterns of idioms than a categorical one. Such an investigation could also have important implications for idiom storage and representation theories.

Finally, works such as Bargmann & Sailer (2018) have argued that the semantic restrictions imposed by various syntactic operations may be differently instantiated in different languages. Future research – both theoretical or experimental – could assess the effect of semantic restrictions on idiom flexibility more directly through comparison of the behavior of parallel idioms in different languages, as well as thorough examination of the instantiation of semantic restrictions by various syntactic operations in specific languages and their relation to the syntactic flexibility patterns of idioms.

Abbreviations

1 first person
acc accusative
f feminine
fut future
gen genitive
inf infinitive
neg negation
pl plural
pst past
sg singular
unacc unaccusative

Data availability

All materials, collected data, analysis scripts and additional files for this article (including Appendices A-D) are available in our Open Science Framework (OSF) repository under: https://osf.io/jhbgr/overview?view_only=5d05f7ebeeae411084344514e72a3313.

Ethics and consent

The research received the approval of the University Institutional Review Board Ethics Committee (proposal no. 0007991-2). All participants in our experiments gave their informed consent before participating.

Funding information

This research was supported by the Cukier-Goldstein-Goren Center for Mind, Cognition and Language’s collaborative research funding grant.

Acknowledgements

We would like to thank the editor Sol Lago for her attentive support and assistance throughout the review process, as well as the three anonymous reviewers, whose contribution to the improvement of the paper cannot be overstated. We would also like to thank Aya Meltzer-Asscher and Nitzan Trainin for their invaluable input with regard to the design and statistical analysis of the experiments, as well as Julia Horvath for her insightful commentary at early stages of this research. Last but not least, we would like to thank the graduate students and doctoral graduates from the Department of Linguistics at Tel Aviv University who participated in the various dry runs and pretests involved in this research for both their time and insights, and specifically, Amit Benalal, Aviv Azar, Daniel Lachs Rabinovich, Dori Sharabani, Noa Brandel, Nofar Rimon, Mandy Cartner, and Yuval Katz.

Competing interests

The authors have no competing interests to declare.

Notes

  1. Tildes are used instead of equal signs for idiom paraphrases, as we adopt previous assertions that “no exact paraphrases of these expressions exist” (Nunberg et al. 1994) and that “idioms cannot be paraphrased without loss” (Vega-Moreno 2002). [^]
  2. This was especially important considering the idiom superiority effect, according to which idiomatic interpretations are only suppressed in non-felicitous contexts (Noveck et al. 2023). [^]
  3. To ensure the pragmatic suitability of the contexts we constructed, we consulted with heTenTen 2014 (Baroni et al. 2009), a billion-token web-crawled Hebrew corpus available on Sketch Engine (Kilgarriff et al. 2004), to identify contexts in which the relevant idioms have been attested to appear. [^]
  4. The mean familiarity ratings of the 10 “filler” VP-idioms are not shown, but were indeed found to be significantly lower than those of the original 30 VP-idioms. [^]
  5. The original set also included four V+PP idioms, which were eventually excluded for uniformity’s sake. In addition, the idiom ikem et ha-af (lit. ‘bend the nose’, fig. ‘show dissatisfaction’) was also excluded due to difficulties in formulating literal contexts that were deemed natural enough. [^]
  6. Bar one decomposable and one non-decomposable idiom, which were selected as the best remaining options with high enough decomposability classification ratings. [^]
  7. The transcription style used throughout the paper is in line with conventions common in the generative literature (e.g., Borer 1995; Armon-Lotem et al. 2008). [^]
  8. The direction of sum-coding in all three experiments was selected as follows: decomposable idioms were coded as 1 and non-decomposable idioms as -1; literal expressions were coded as 1 and idiomatic expressions as -1; and baseline structure was coded as 1 and modified structure as -1. [^]
  9. Following the recommendations of Barr et al. (2013), we incorporated maximal converging random effects structures for all our models. [^]
  10. For example, definite DPs are generally thought to be specific and referential; referential DPs are generally thought to showcase anaphoric potential. [^]
  11. Ernst also describes a third type of adjectival modification, conjunction modification, where the adjective semantically modifies the literal denotation of the noun to which it syntactically attaches, but we gloss over it here, as the distinction between decomposable and non-decomposable idioms pertains to the difference in patterns of idiomatic association. [^]
  12. To ensure that our selected adjectives were not ruled out on pragmatic grounds (i.e., that they fit into the metaphor denoted by the idiom, as advocated by Ernst (1981) and Nediger (2017)), we again consulted with heTenTen 2014 to find adjectives with at least some attested uses in modifying the relevant idiom. [^]
  13. We thank an anonymous reviewer for this suggestion. [^]
  14. This could potentially suggest that lexical insertion-type operations are more restrictive of idioms than co-reference or movement-type operations – presumably because they intervene between idiom constituents upon entering the derivation (assuming adjectives are not introduced counter-cyclically via Adjoin-like operations). However, lack of statistical power prevented us from pursuing this hypothesis in any concrete way. [^]
  15. For a visual representation of average acceptability scores per idiom, see Figure D1 in the appendix. [^]
  16. Nunberg et al. (1994) do mention that certain German non-decomposable idioms have been attested to undergo operations such as scrambling, verb-second, and object fronting. They argue that this is not incompatible with their theory since in German these operations constitute “an alternative realization of the combination of a subject and an idiomatic phrase” rather than a “lexical process”. However, their arguments regarding object fronting, for example, that “unlike English topicalization.. the discourse function of the German construction is not dependent on assigning a meaning to the initial element,” seem much more straightforwardly explained under a semantic restriction account of the inflexibility of non-decomposable idioms. [^]
  17. Fanselow (2004), e.g., argues that German allows for a form of object fronting termed pars pro toto fronting, where though only a chunk of the phrase is syntactically fronted, it is in fact the entire phrase that is semantically topicalized. [^]
  18. We ran an additional post-hoc analysis treating decomposability (i.e., Type of Idiom) as a continuous predictor, but its results did not differ significantly from those treating decomposability as a categorical predictor. See Appendix D for the full analysis. [^]
  19. Under the hybrid theory suggested by Nediger (2017), according to which idiomatic interpretation is achieved by post-syntactic matching of holistic idiom entries with derivationally built structures, the extent to which an idiom’s meaning is distributable over its parts could potentially dictate the extent of violation of semantic restrictions at the interfaces, allowing for graded effects. [^]

References

Armon-Lotem, Sharon & Danon, Gabi & Rothstein, Susan (eds.). 2008. Current issues in generative Hebrew linguistics. Amsterdam/Philadelphia: John Benjamins Publishing Company. DOI:  http://doi.org/10.1075/la.134

Bargmann, Sascha & Sailer, Manfred. 2018. The syntactic flexibility of semantically non-decomposable idioms. In Sailer, Manfred & Markantonatou, Stella (eds.), Multiword expressions: Insights from a multi-lingual perspective, 1–29. Berlin: Language Science Press. DOI:  http://doi.org/10.5281/zenodo.1182587

Baroni, Marco & Bernardini, Silvia & Ferraresi, Adriano & Zanchetta, Eros. 2009. The WaCky wide web: A collection of very large linguistically processed web-crawled corpora. Language Resources and Evaluation 43(3). 209–226. DOI:  http://doi.org/10.1007/s10579-009-9081-4

Barr, Dale J. & Levy, Roger & Scheepers, Christoph & Tily, Harry J. 2013. Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of Memory and Language 68(3). 255–278. DOI:  http://doi.org/10.1016/j.jml.2012.11.001

Borer, Hagit. 1995. The ups and downs of Hebrew verb movement. Natural Language & Linguistic Theory 13. 527–606. DOI:  http://doi.org/10.1007/BF00992740

Bruening, Benjamin. 2020. Idioms, collocations, and structure. Natural Language & Linguistic Theory 38(2). 365–424. DOI:  http://doi.org/10.1007/s11049-019-09451-0

Chafe, Wallace. 1968. Idiomaticity as an anomaly in the Chomskyan paradigm. Foundations of Language 4(2). 109–127.

Chomsky, Noam. 1981. Lectures on government and binding: The Pisa lectures. Dordrecht: Foris.

Corver, Norbert & Van Craenenbroeck, Jeroen & Harwood, William & Hladnik, Marko & Leufkens, Sterre & Temmerman, Tanja. 2019. Introduction: The compositionality and syntactic flexibility of verbal idioms. Linguistics 57(4). 725–733. DOI:  http://doi.org/10.1515/ling-2019-0021

Ernst, Thomas. 1981. Grist for the linguistic mill: Idioms and “extra” adjectives. Journal of Linguistic Research 1(3). 51–68.

Everaert, Martin. 2010. The lexical encoding of idioms. In Rappaport-Hovav, Malka & Doron, Edit & Sichel, Ivy (eds.), Lexical semantics, syntax, and event structure, 76–98. Oxford: Oxford University Press. DOI:  http://doi.org/10.1093/acprof:oso/9780199544325.003.0005

Fanselow, Gisbert. 2004. Cyclic phonology-syntax interaction: Movement to first position in German. Universitätsverlag Potsdam Working Papers of the SFB 632: Interdisciplinary studies on information structure 1. 1–42.

Fellbaum, Christiane. 1980. Functional structure and surface structure. Ann Arbor, MI: University Microfilms International.

Fraser, Bruce. 1970. Idioms within a transformational grammar. Foundations of Language 6(1). 22–42.

Gibbs, Raymond W. & Nayak, Nandini P. 1989. Psycholinguistic studies on the syntactic behavior of idioms. Cognitive Psychology 21. 100–138. DOI:  http://doi.org/10.1016/0010-0285(89)90004-2

Harley, Heidi. 2014. On the identity of roots. Theoretical Linguistics 40(3–4). 225–276. DOI:  http://doi.org/10.1515/tl-2014-0010

Hladnik, Marko. 2017. Variability in the syntax of idioms. ELOPE: English Language Overseas Perspectives and Enquiries 14(2). 25–37. DOI:  http://doi.org/10.4312/elope.14.2.25-37

Horvath, Julia & Siloni, Tal. 2019. Idioms: The type-sensitive storage model. Linguistics 57(4). 853–891. DOI:  http://doi.org/10.1515/ling-2019-0017

Jackendoff, Ray. 1997. The architecture of the language faculty. Cambridge, MA: MIT Press.

Katz, Jerrold & Postal, Paul. 1964. An integrated theory of linguistic descriptions. Cambridge, MA: MIT Press.

Kay, Paul & Sag, Ivan. 2014. A lexical theory of phrasal idioms. Ms., University of California, Berkeley.

Kilgarriff, Adam & Rychlý, Pavel & Smrz, Pavel & Tugwell, David. 2004. The Sketch Engine, 105–115.

Kiss, Katalin É. 2002. The syntax of Hungarian. Cambridge: Cambridge University Press. DOI:  http://doi.org/10.1017/CBO9780511755088

Kuno, Susumu. 1972. Functional sentence perspective: A case study from Japanese and English. Linguistic Inquiry 3. 269–320.

Larson, Richard K. 1988. On the double object construction. Linguistic Inquiry 19(3). 335–391.

Liddell, Torrin M. & Kruschke, John K. 2018. Analyzing ordinal data with metric models: What could possibly go wrong? Journal of Experimental Social Psychology 79. 328–348. DOI:  http://doi.org/10.1016/j.jesp.2018.08.009

Maher, Zachary. 2013. Opening a can of worms: Idiom flexibility, decomposability, and the mental lexicon. BA/MA thesis, Yale University.

Marantz, Alec. 1996. ‘Cat’ as a phrasal idiom. Ms., MIT.

Marantz, Alec. 1997. No escape from syntax: Don’t try morphological analysis in the privacy of your own lexicon. University of Pennsylvania Working Papers in Linguistics 4(2). 14–25.

McClure, Scott. 2011. Modification in non-combining idioms. Semantics & Pragmatics 4. 1–7. DOI:  http://doi.org/10.3765/sp.4.7

Nediger, Will Alexander. 2017. Unifying structure-building in human language: The minimalist syntax of idioms. PhD thesis, University of Michigan.

Noveck, Ira A. & Griffen, Nicholas & Mazzarella, Diana. 2023. Taking stock of an idiom’s background assumptions: An alternative relevance theoretic account. Frontiers in Psychology 14. DOI:  http://doi.org/10.3389/fpsyg.2023.1117847

Nunberg, Geoffrey & Sag, Ivan A. & Wasow, Thomas. 1994. Idioms. Language 70(3). 491–538. DOI:  http://doi.org/10.2307/416483

Tabossi, Patrizia & Arduino, Lisa & Fanari, Rachele. 2011. Descriptive norms for 245 Italian idiomatic expressions. Behavior Research Methods 43(1). 110–123. DOI:  http://doi.org/10.3758/s13428-010-0018-z

Tabossi, Patrizia & Fanari, Rachele & Wolf, Kinou. 2008. Processing idiomatic expressions: effects of semantic compositionality. Journal of Experimental Psychology: Learning, Memory, and Cognition 34(2). 313–327. DOI:  http://doi.org/10.1037/0278-7393.34.2.313

Tabossi, Patrizia & Wolf, Kinou & Koterle, Sara. 2009. Idiom syntax: idiosyncratic or principled? Journal of Memory and Language 61. 77–96. DOI:  http://doi.org/10.1016/j.jml.2009.03.003

Titone, Debra A., & Connine, Cynthia M. 1994. Descriptive norms for 171 idiomatic expressions: Familiarity, compositionality, predictability, and literality. Metaphor and Symbolic Activity 9(4). 247–270. DOI:  http://doi.org/10.1207/s15327868ms0904_1

Vega-Moreno, Rosa Elena. 2002. Representing and processing idioms. UCL Working Papers in Linguistics 13. 73–107.

Vergnaud, Jean Roger. 1974. French relative clauses. PhD thesis, MIT.

Veríssimo, João. 2021. Analysis of rating scales: A pervasive problem in bilingualism research and a solution with Bayesian ordinal models. Bilingualism: Language and Cognition 24. 842–848. DOI:  http://doi.org/10.1017/S1366728921000316

Wèinreich, Uriel. 1969. Problems in the analysis of idioms. In Puhvel, Jaan (ed.), Substance and structure of language, 23–82. Oakland: University of California Press. DOI:  http://doi.org/10.1525/9780520316218-003

Wierzba, Marta & Brown, Jessica M. M. & Fanselow, Gisbert. 2023a. The syntactic flexibility of German and English idioms: Evidence from acceptability rating experiments. Journal of Linguistics. 431–466. DOI:  http://doi.org/10.1017/S0022226723000105

Wierzba, Marta & Brown, Jessica M. M. & Fanselow, Gisbert. 2023b. Sources of variability in the syntactic flexibility of idioms. Glossa: A Journal of General Linguistics 8(1). 1–41. DOI:  http://doi.org/10.16995/glossa.8502

Zehr, Jérémy & Schwarz, Florian. 2018. PennController for Internet Based Experiments (IBEX). DOI:  http://doi.org/10.17605/OSF.IO/MD832