2.1 Where and how is a VP omitted?

2.2 What makes English missing VP sentences acceptable?

In contrast to the debate on why a VP gets omitted, there is less discussion about why ungrammatical missing VP sentences can be perceived as more acceptable. Implicit in existing accounts is the idea that the parser comes to develop certain expectations about the VPs in these sentences. These expectations are then borne out, producing a sense of completeness.

We can distinguish between two variants of this idea. The first variant is found in accounts like Gibson & Thomas’s and computational models of the illusion (also see Christiansen & Chater 1999; Christiansen & MacDonald 2009). Memory limitations cause the parser to expect only two VPs. Since the parser observes only two VPs, these predictions are borne out.

In contrast, interference accounts implicitly assume that the parser expects to fill all three VP nodes, even though the parser ends up leaving the intermediate VP node empty and fails to notice it. Regardless of why exactly the empty node goes unnoticed, the result is that the parser mistakenly concludes that all three nodes have been filled and the sentence is complete.

Here, we offer an alternative non-structural perspective, based on the premise that a core parsing objective is to establish thematic relations between the arguments and predicates in a sentence: who did what to whom, so to speak. Setting aside what processes cause VPs to be omitted, we suggest that missing VP sentences are acceptable not because of the parser’s structural expectations, but because thematic relations can be successfully built between all remaining arguments and predicates. Because no argument or predicate is thematically “orphaned,” speakers perceive a sense of completeness, which translates into an illusion of grammaticality.

For illustration, consider the missing VP sentence in (3c): The novel that the horror author who the publishing company had recently fired was banned by the local library. In this sentence, all three NPs (more precisely, the arguments they represent) are related to some VP predicate: NP1 the novel is related to VP2 was banned by the local library, while NP2 the horror author and NP3 the publishing company are related to VP1 had recently fired. Conversely, the two predicates in VP1 and VP2 collectively require three arguments: two for had recently fired, and one for was banned by the local library. These requirements are satisfied by the three NPs. It is easy to see from Figure 1 that every argument and predicate is connected to something else. (For comparison purposes, we depict thematic relations for a grammatical DCE sentence in Figure 2).

Figure 1

Thematic relations (depicted with arrows) in missing VP sentence (3c) [_NP1 The novel] that [_NP2 the horror author] who [_NP3 the publishing company] [_VP1 had recently fired] [_VP2 was banned by the local library].

Figure 2

Thematic relations in grammatical DCE sentence (3a) [_NP1 The novel] that [_NP2 the horror author] who [_NP3 the publishing company] [_VP1 had recently fired] [_VP2 had typed quickly] [_VP3 was banned by the local library].

An important question here is what kind of representation is implicated in this thematic relations account. It is difficult to say definitively that the relevant representation must be syntactic and that thematic relations are read off from that structure. In a grammatical DCE sentence, NP1 and NP2 should each be thematically and syntactically related to two verbs: the verb of the relative clause modifying them and the verb that they are the subjects of, as reflected in Figure 2. It is unclear what happens to these syntactic relations in a missing VP sentence, where there is only one verb each for NP1 and NP2.

For this reason, we do not rule out the possibility that in addition to a syntactic representation, there is a shallower representation that lets the parser track what thematic relations exist in a sentence. We remain agnostic about the specifics of this shallower representation, for example, whether relations are encoded directly between constituents (NPs, verbs) or the arguments and predicates they represent.

Of course, our suggestion that shallow representations are involved seems inconsistent with a body of psycholinguistic research. There is substantial evidence that the parser incrementally builds a rich, connected syntactic representation, as revealed by the finding that the real-time building of various dependencies is sensitive to syntactic constraints (for filler-gap dependencies, see Stowe 1986; Traxler & Pickering 1996; Phillips 2006; Wagers & Phillips 2009; etc.; for anaphoric dependencies, see Sturt 2003; Kazanina et al. 2007; Aoshima et al. 2009, etc.).

This inconsistency can be resolved if we consider the exceptional syntactic complexity and memory load associated with DCE sentences (Miller & Isard 1964; Resnik 1992; Gibson & Thomas 1999, etc.). Under such circumstances, the parser might find it difficult to build and maintain a connected structure for interpretation. Consequently, it might rely on a shallower interpretive strategy, directly tracking how arguments and predicates are linked, so that it can produce a passable interpretation.

As presented, the thematic relations account, as well as the structural forgetting and interference accounts, predict missing VP sentences to be highly acceptable, contrary to experimental findings (Gibson & Thomas 1999 and Experiment 3 here; Frank & Ernst 2019). The low ratings might be attributable to the length and structural complexity of these sentences. Alternatively, the proposed parsing mechanisms might be less deterministic than depicted, so missing VP sentences do not consistently produce an illusion of grammaticality.

Overall, the thematic relations hypothesis makes very similar predictions as existing accounts about missing VP sentences. However, a key difference is that it does not require the parser to overlook the fact that the second NP in these sentences is missing a VP. Rather, the absence of a VP is essentially offset by the fact that this NP can still be linked to another predicate. One way to test this hypothesis, then, would be to consider a double center-embedding scenario where this link is unavailable, so the second NP (or its equivalent) runs the risk of being “orphaned.” Mandarin Chinese provides just such a test case.

3.1 Center-embedding in Mandarin

Mandarin has head-final relative clauses that precede a particle de, which in turn precedes the head noun. Suppose that a noun NP3 is modified by a subject relative clause with an object NP2. This complex noun phrase has [V-NP2-de]-NP3 word order. Since Mandarin is a Subject-Verb-Object language, a sentence where this complex NP is the object has Subject-V1-[V2-NP2-de]-NP3 word order. Inserting another subject relative clause to modify NP2 yields a DCE sentence with Subject-V1-[V2-[V3-NP1-de]-NP2-de]-NP3 word order (4).

A missing NP sentence can be constructed by omitting one of the des and NPs (5), by analogy with missing VP sentences.

As the English “translation” of (5) shows, a missing NP sentence is ungrammatical: the three transitive verbs each require an object, but there are only two NPs in object positions. However, impressionistically, such a sentence is as acceptable as its grammatical counterpart (4), if not more so – it is this relative acceptability that we refer to as the missing NP illusion.

3.2 Ruling out null object or headless relative clause analyses

Mandarin has null object and “headless” relative clause constructions in which NPs are omitted. One might wonder whether missing NP sentences might be instances of these constructions and therefore grammatical. This section argues against this possibility. First, null objects (6a) and headless relative clauses (6b) constructions elicit a specific percept of ill-formedness when presented out of context, when no antecedent is available (e.g. Huang 1984; Li 2014). In contrast, missing NP sentences are relatively acceptable even out of context.

Second, missing NP sentences are syntactically distinct from both constructions. In null object sentences, verbs usually appear at the end of a clause (embedded or otherwise), unless adverbial material happens to be present (6a). In missing NP sentences, none of the verbs appear in such a position. Both V1 and V2 precede another verb that appears to start an embedded clause, while V3 immediately precedes a direct object NP. As for headless relative clause sentences like (6b), only the head NP is omitted. This contrasts with missing NP sentences, where an NP and a de are missing.

3.3 Challenges for existing accounts

Abstractly, missing NP sentences resemble missing VP sentences: missing NP sentences feature a sequence of three verbs and two NPs, while missing VP sentences feature a sequence of three NPs and two VPs. However, it turns out to be challenging to adapt hypotheses proposed for the missing VP illusion to account for the missing NP illusion. Under the structural forgetting and interference accounts, there is a distinct possibility that the middle transitive verb would be “orphaned” thematically, without agent and theme arguments, in turn raising questions about why missing NP sentences should be relatively acceptable. The language experience account, on the other hand, predicts that Mandarin should not exhibit such an illusion.

We first elaborate on the challenge for the structural forgetting and interference accounts. In Mandarin, as in English, the parser forgets or overlooks the absence of a constituent in a DCE sentence. However, unlike in English, the absence of this constituent – an NP – should cause a more severe disruption in terms of thematic relations, which should reduce acceptability. The three verbs in missing NP sentences are transitive, each requiring two arguments. However, including the main clause’s subject, there are only three NPs. Intuitively, there are not enough NPs.

More specifically, let us consider missing NP sentences from an incremental parsing perspective, following these accounts. (We have simplified the following description, for exposition purposes. We refer interested readers to Jäger et al. 2015 for an overview of the local ambiguities that a Mandarin parser must navigate in parsing a relative clause.)

Suppose that a missing NP sentence like (5) is parsed in the following manner. First, the parser observes an NP, prime minister, and a transitive V1 meet. It interprets the NP as V1’s subject and agent argument. Observing V1 triggers a prediction for an NP object, its theme argument.

Next, when a transitive V2 rebuked is seen rather than V1’s object, V2 is recognized as the main verb of a subordinate clause modifying V1’s object. Being transitive, V2 triggers predictions for an NP object and a de. To the extent that the parser analyzes this subordinate clause as a subject relative clause, it should also expect the NP appearing after de to be interpreted as V2’s agent.

Lastly, when a transitive V3 hear is seen rather than V2’s object, V3 is recognized as the main verb of a subordinate clause modifying V2’s object. The parser again should predict an NP object (V3’s theme) and de. Again, to the extent that the parser analyzes this subordinate clause as a subject relative clause, it should expect the NP appearing after de to be interpreted as V3’s agent.

Both structural forgetting and interference hypotheses claim that the parser forgets or overlooks the predictions associated with V2. Consequently, the parser should pair the first post-verb NP and de with V3 and the second NP with V1. To the extent that the parser analyzes V3 as a subject relative clause, the second NP should also get interpreted as V3’s agent.

Figure 3 summarizes the thematic relations built in this scenario. The three NPs – the main clause subject, NP1, and NP2 – are all linked to verbs. As for the three verbs, only two have arguments: V1 and V3; V2 is “orphaned.” If missing VP sentences are acceptable because the parser can build thematic relations between predicates and arguments, one might expect missing NP sentences to be less acceptable than what they are reported to be. (For comparison, Figure 4 depicts thematic relations for a grammatical DCE sentence.)

Figure 3

Hypothetical thematic relations in Mandarin missing NP sentence (5).

Figure 4

Thematic relations in grammatical Mandarin DCE sentence (4).

Alternatively, one might wonder if the “orphaning” of V2 in Figure 3/(5) is only superficial. Since Mandarin allows null arguments, perhaps the parser can supply arguments for V2 from the discourse context or world knowledge, so V2 can satisfy its argument structure needs without being linked to other NPs in (5). If so, it should be easy to find a grammatical version of (5) that has overt arguments for V2 and is consistent with the thematic relations in Figure 3. But there is no acceptable way to add two more overt NPs for V2 to a missing NP sentence like (5).

We next turn to the challenge for the language experience hypothesis. It has been suggested that this hypothesis, when paired with an account like structural forgetting, can explain cross-linguistic variation in the missing VP illusion (Vasishth et al. 2010, Engelmann & Vasishth 2009, Frank et al. 2016; Futrell et al. 2020). We argue that this approach does not improve matters here. In fact, adopting it might yield the prediction that Mandarin should lack such an illusion.

The root of this problem lies in the syntax of Mandarin noun phrases: they are uniformly head-final; all complements and modifiers are marked overtly with de. Consequently, sequences of nouns, whether interleaved with de or not, are common in Mandarin: they appear in compound (7a) and possessive structures (7b). They also appear when a noun has a subordinate clause that contains an object, regardless of whether the clause is a complement (7c) or relative clause (7d). In fact, subject relative clauses, which correlate with noun sequences (7d), are reported to occur more frequently than object relative clauses (Hsiao & Gibson 2003; Vasishth et al. 2013).

Mandarin speakers should therefore be familiar with processing sequences of nouns, just as German and Dutch speakers are familiar with processing sequences of verbs. In addition, given Mandarin’s VO word order and head-final NPs, it is not unusual for a verb to be separated from the head of its object NP by a modifier, such as a possessor or a relative clause. The net effect is that Mandarin speakers should find grammatical DCE sentences, which feature long verb-object dependencies and sequences of nouns, easier to process than missing NP sentences.

Of course, the arguments above assume that the missing NP illusion exists and Mandarin speakers interpret missing NP sentences where V2 is thematically orphaned. The next section reports an experiment designed to validate these assumptions. As a preview, we find that missing NP sentences are relatively acceptable, but no verb is orphaned. Instead, speakers can and do interpret V2 as taking NP2 as an argument.

4.1 Materials

24 DCE sentences were constructed, each with six versions corresponding to different experimental conditions. Two of these conditions were grammatical DCE sentences and “plausible” missing NP sentences, designed to test for the missing NP illusion. The remaining conditions were “implausible” missing NP sentences, designed to determine which verbs are thematically connected with which NPs in missing NP illusions. We distributed these 24 sentences into six lists using a Latin Square design, so that each list contained four sentences per condition, and no two sentences in each list were variants of each other.

In addition, 48 fillers were constructed. Fillers were similar in length and syntactically complex, featuring subordinate clauses, conjoined clauses, and non-canonical word order. 36 fillers were acceptable sentences and 12 were unacceptable ones. This distribution was chosen so that participants would read an equal number of acceptable and unacceptable sentences, assuming that participants find all target sentences less acceptable than the acceptable fillers. All sentences were presented in simplified Chinese characters.

In all target sentences, a duration or frequency adverb appeared between the NP objects and the clause-final particle de. In (4), repeated below as (8), the duration adverb bùjiǔ “not long ago” appears between NP1 tānwū-àn “corruption case” and de, and the frequency adverb hǎojǐcì “several times” between NP2 fǎguān “judge” and de.

These adverbs served to disambiguate the reading of de. Without an adverb, de in this position – between two nouns like tānwū-àn “corruption case” and fǎguān “judge” – is in principle ambiguous. De can mark the end of a relative clause, in which case tānwū-àn is inside the relative clause and fǎguān is the head of the relative clause. Alternatively, de can be understood as a possessive marker connecting two NPs, so “corruption case” and “judge” form a single complex NP (9). The adverb blocks the possession reading.

DCE sentences like (8) were contrasted with “plausible” missing NP variants like (10), which end prematurely at NP2. These variants were plausible in the respect that all likely verb-argument relations were semantically plausible. For instance, (10) is plausible in that NP2, fǎguān “judge” is a plausible theme argument for either V1, jiējiàn “meet,” or V2, zébèi-guo “rebuked,” and a plausible agent of V3 shěnlǐ “hear.” Of course, they were not plausible in the sense that they have a fully coherent meaning, since the sentences were incomplete.

We chose verbs, aspect markers, and adverbs so as to maximize the possibility that participants parse a missing NP sentence as doubly center-embedded. First, we tried to pick V1 and V2 verbs with a clear NP complement bias, avoiding verbs that can also readily appear with only a clause-like complement, such as verbs describing speech, cognition, or desire, like xǐhuān “like.” The goal was to prevent participants from assigning an unintended, alternative grammatical parse with only a single level of embedding (11b); we expand on this point in Experiment 2. To identify these verbs, we relied on the intuitions of the first author, a Mandarin native speaker. We also verified these intuitions by checking what argument structure frames are associated with each verb in the “frame files” in the Chinese Proposition Bank (Xue & Palmer 2008).

Second, verbs within a sentence also had distinct lexical semantics, and where possible, distinct grammatical aspect morphology (such as experiential -guo or perfective -le, translated using the English past tense). As a generalization, two verbs can be conjoined in Mandarin without an overt marker when the verbs have similar semantics and are in the same grammatical aspect. By controlling for these factors, we discouraged participants from coercing a reading of a missing NP sentence where two of the three verbs, such as “criticize” and “rebuke” in (12), are conjoined. A conjunction analysis would produce a grammatical sentence, as illustrated by the English translation in (12b).

Third, we added past-oriented temporal adverbs like céng(jīng) “previously,” yǐ(jīng) “already,” gāng “just recently,” zhīqián “in the past” after the first verb and after the second verb, as suggested by an anonymous reviewer. These temporal adverbs seem to encourage these verbs to be parsed as taking a complex NP object, rather than a complement clause.

Four other conditions were created by modifying the plausible missing NP condition, so that the NPs following the verbs would be perceived as inappropriate arguments for plausibility and/or animacy reasons, depending on which of the three verbs the NP was interpreted with. These manipulations parallel Gibson & Thomas’s manipulations for their English missing VP experiment (3), which were also intended to detect which nouns and verbs are interpreted together.

Three of these conditions manipulated verbs to alter the plausibility of NP2 as an argument. In the first condition, V1 (and associated adverbs, aspect, and modals) was chosen so that NP2 would be an unlikely or inappropriate theme (13a). While we do not translate sentences like (13a) because doing so would presuppose a parse, the intuition for this condition is as follows: if speakers interpret NP2 fǎguān “judge” as the theme argument of V1 chóngzǔ “reorganize,” they should assign a lower acceptability rating to the sentence, since one reorganizes institutions, not judges.

Likewise, in the second condition, V2 was chosen so that NP2 would be an inappropriate theme argument (13b), if NP2 were so interpreted. In the third condition, V3, its object (NP1), and adverbs were chosen so that NP2 would be an unlikely or inappropriate agent (13c): judges do not lead national basketball teams. For ease of reference, we call these the “implausible V1 object,” “implausible V2 object,” and “implausible V3 subject” conditions.

The fourth and last condition altered NP1 so that it would be an implausible theme object of V3, the immediately preceding verb (14). We label this the “implausible V3 object” condition.

Obviously, there are other implausible conditions that we could have created. We chose the above four conditions, however, because there is prior reason to think that speakers might interpret the NPs and verbs as standing in these thematic relations. It is plausible that NP2 gets interpreted as V1’s object, by analogy to English missing VP sentences, where the first NP is interpreted as the second VP’s subject. One might also expect NP2 to get interpreted as V2’s object and as V3’s subject, as these are the interpretations that NP2 would receive in grammatical DCE sentences. Similarly, one would expect NP1 to be interpreted as V3’s object, by analogy to grammatical DCE sentences and because NP1 appears in a canonical object position.

4.2 Participants

Participants were 60 self-identified Mandarin Chinese native speakers born in mainland China. All participants were above the age of 18. They were recruited over Prolific and compensated US$3.17 for their time, based on the time estimated necessary for completing the experiment and Prolific’s recommended hourly rate of US$9.50.

We recruited participants by first asking them to complete a ten-question multiple choice screening test on Prolific, for which they were compensated US$0.81 for their time (again based on Prolific’s recommended rate). In addition to ensuring that participants could read simplified Chinese proficiently, the test also checked for familiarity with Mandarin syntax and vocabulary, including idiomatic expressions that non-native or heritage speakers might find difficult. We selected only participants who answered at least eight questions correctly.

4.3 Procedure

Sentences were presented using Ibex (created by Alex Drummond). Participants were instructed to rate the acceptability of each sentence based on their intuitions, with a 7-point Likert scale, where “1” was “very incoherent (bù tōngshùn), totally unacceptable” and “7” was “very coherent, totally acceptable.” For the first four practice sentences, participants saw brief comments about each sentence. For example, they were instructed that a sentence that was incoherent (with word order violations) or described an implausible scenario should receive a low rating. All ungrammatical practice sentences had word order or plausibility issues, except for the following ungrammatical sentence (the fourth practice sentence), a non-DCE sentence missing an NP (15). This sentence was modeled after one of Gibson & Thomas’s English practice sentences, a non-DCE sentence missing a VP. Following Gibson & Thomas, participants were told that this sentence should receive a low rating because it was not coherent, without further explanation. Participants were not exposed to missing NP or DCE sentences nor told how they should rate them.

Each sentence appeared with a rating scale. Participants rated sentences using the keyboard or by clicking on the rating scale. Even though there was no time limit, participants were also instructed to respond as quickly as possible, to discourage them from re-reading sentences and noticing any structural anomalies.

4.4 Data analysis

The target sentences in this experiment were fairly long. To ensure that participants had taken the time to read a target sentence before rating it, we excluded responses that were provided in less than 3 seconds after the presentation of the sentence. This eliminated 2.8% of responses.

Acceptability ratings for all conditions were analyzed with a single cumulative link mixed effects model in R version 3.3.2 (R Development Core Team 2019) with the ordinal package (Christensen 2019). The model had conditions as a fixed effect and random intercepts and slopes for both participant and items.

4.5 Results

Averaged ratings for the conditions are presented in Table 1; no condition was particularly acceptable in absolute terms. Of the six conditions, the ungrammatical plausible missing NP condition received the highest ratings. Put differently, native speakers did not find grammatical DCE sentences more acceptable.

Table 2 presents results of the statistical analysis, with the plausible missing NP condition as the baseline. While this condition received higher ratings than the grammatical DCE condition, the difference was not significant (p = 0.15). On the other hand, all four implausible missing NP conditions were less acceptable than the plausible missing NP condition (all p < 0.01).

4.6 Discussion

Experiment 1 confirmed informal impressions of a missing NP illusion. The plausible missing NP condition received numerically higher ratings than its grammatical counterparts. Even though the difference was not significant, calling it a grammatically illusion is still justified. Unlike other kinds of ungrammatical sentences, these missing NP sentences are clearly rated as no worse than their grammatical counterparts. The illusion is analogous to the missing VP illusion, despite differences in word order and thematic relations.

This result has a parallel with Gibson & Thomas 1999, which also reported no significant contrast between missing VP sentences and their grammatical counterparts in English. It is possible, as Gibson & Thomas suggest, that the offline nature of the task let participants reread sentences and more easily notice that missing NP sentences are ungrammatical. For transparency, we also note that whether the numerical difference is significant depends on the statistical model. When we fitted the data with a model with only random intercepts, the difference turned out to be significant, like the other four implausible conditions (all p < 0.01).

More interesting are the contrasts for the other conditions. They suggest that speakers can relate the second post-verb NP (NP2) to all three verbs: as the theme argument (object) of V1 and V2 and as the subject (agent or experiencer) of V3. This finding is unexpected, having no analog in the missing VP illusion. In a missing VP sentence, the second VP is the equivalent of the Mandarin second NP, but there is no evidence that English speakers interpret this VP as taking all three NPs as its arguments, in particular, with both NP1 and NP2 as agent or experiencer arguments (subjects).

These findings do not follow easily from the structural forgetting and interference accounts. Under both accounts, the parser should have forgotten or overlooked V2’s predictions for an NP object and a de. Since the object is interpreted as V2’s theme argument and the NP that appears immediately after de is interpreted as V2’s agent, the forgetting or overlooking of the object and de should cause V2 to become thematically orphaned, without a theme or agent (see Figure 3). This should in turn lower the acceptability of a missing NP sentence. The data, however, show that plausible missing NP sentences are relatively acceptable and V2 is not orphaned.

While we did not predict that missing NP sentences would feature such a set of thematic relations, these results turn out to be consistent with our thematic relations hypothesis, which attributes the center-embedding illusion to the parser establishing thematic relations between all arguments and predicates in the sentence. More specifically, in a missing NP sentence, all NPs (including the main clause subject) are thematically related to some verbal predicate. The subject of the sentence, appearing in a canonical subject position preceding V1, is interpreted as V1’s experiencer or agent. The two post-verb NPs are interpreted as arguments of all three verbs, as ratings for the implausible conditions show. Conversely, every verbal predicate also gets at least one argument: V1 has an agent/experiencer in its subject and a theme argument in NP2; V3 has an agent in NP2 and a theme in NP1. While the experiment did not test whether V2 has an agent argument, our results indicate that it has a theme argument in NP2.

Finally, the illusion is not predicted by an account in which center-embedding illusions (or the lack thereof) are derived from language experience. Given the noun-final nature of Mandarin nominal expressions and the presence of noun sequences in the language, such an account predicts that Mandarin should lack the illusion, a point we return to in Section 7 (Experiment 4).

5.1 Materials

We constructed six Subject–Main verb–Adverb–Embedded verb–NP sentences for each of the 42 unique V1 and V2 verbs used in Experiment 1. (In principle, there should have been 48 verbs, but six verbs occur in both V1 and V2 positions.) Within this set of six sentences, one (or two, for the six overlapping verbs) was constructed by reusing constituents from Experiment 1, so that the resulting sentence resembled the original grammatical DCE sentence. An example of this can be seen in (17), where the main verb is zébèi-guo “rebuked”: we used NP3 bùzhǎng “minister” as the subject, the adverb gāng “recently,” V3 shěnlǐ “hear” as the embedded verb, and NP1 tānwū-àn “corruption case” as the NP. We added modal auxiliaries, aspect markers, or adverbs to the verbs of interest where necessary for felicity.

For the other sentences in each set, we replaced all lexical items other than the main verb and adverb. We made sure the replacement verbs and NPs were as plausible as possible given the choice of main verb. For instance, another sentence was constructed with xuézhě “scholar” as the subject, tàntǎo “investigate” as the embedded verb, and shèhuì wèntí “social issue” as the object (19). These are arguably as compatible with zébèi-guo “rebuked” – a scholar might rebuke someone’s investigation of social issues to the same extent as a government minister might rebuke someone’s hearing of a corruption case. If these sentences are still unacceptable, we can then reasonably attribute that to structural factors.

We also created six sentences for each baseline verb, using the same frame.

We wanted to keep the experiment short to help participants focus on the task. To that end, we randomly sorted the 42 verbs into two sets of 21 verbs. We then added the three baseline verbs to each set, forming two sets of 24 verbs. Within each set of 24 verbs, the six sentences for each verb were distributed to produce six lists, such that each verb appears once in each list. Put differently, there were a total of 12 lists (2 sets of 24 verbs × 6 lists), each containing 24 target sentences (one sentence for each of the V1, V2, and baseline verbs).

Finally, we added 24 filler items to each list, so there were as many filler items as target and baseline sentences. Filler sentences were similar in length and structure, consisting of a subject, an adverb, an auxiliary or a second adverb, and a verb and an NP object. 21 of these sentences were grammatical, while the remaining three were not. On the assumption that the non-baseline target items were unacceptable, this distribution ensured that participants saw an equal number of acceptable and unacceptable sentences.

5.2 Participants

Participants were 36 native speakers of Mandarin on Prolific, recruited from the 60 participants who completed Experiment 1. The recruitment process was blind, in the sense that participants’ demographic information and their responses from Experiment 1 were not taken into consideration. Each participant received US$1.60, based on our estimate of the time needed to complete the experiment and Prolific’s recommended hourly rate.

5.3 Procedure

Recall that the V1 and V2 verbs were sorted into two sets and six lists of sentences were constructed for each set. We randomly assigned three participants to each list, so that a total of 18 participants rated sentences for each set of verbs. In other words, through this arrangement, we collected 18 responses for each V1 and V2 verb.

As in Experiment 1, participants rated the sentences on Ibex, with the same 7-point Likert scale. Before starting the experiment, they saw seven practice sentences. For the first four practice sentences, participants saw brief comments about each sentence and what kind of ratings to assign. The fourth practice sentence here was a sentence involving an apparent subcategorization violation (an intransitive verb occurring with a post-verb NP).

5.4 Data analysis

After data collection, we noticed errors in three of the target sentences (one featuring the verb dào “visit, go to” and two featuring jiārù “join”); these sentences have been excluded from analysis. To ensure that participants had taken the time to read a target sentence before rating it, we also excluded responses provided in less than one second after the sentence was presented. Altogether, 1.2% of responses were excluded.

We analyzed the results with a cumulative link mixed effects model for each set of verbs. There were two conditions (baseline verbs vs. V1 and V2 verbs), by-subject and by-verb random intercepts, and a by-subject random slope in each model.

5.5 Results

Table 3 presents a summary of acceptability ratings. The sentences for the three baseline verbs received uniformly very high ratings, while the sentences for the V1 and V2 verbs received much lower ratings. Statistical analysis confirmed that the difference is significant (p < 0.001, Table 4). There was also some variation within the class of V1 and V2 verbs, but no verb had ratings comparable to the baseline verbs.

5.6 Discussion

The results provide clear evidence that the V1 and V2 verbs in Experiment 1 do not allow VP complements containing a past-oriented time adverb. It is therefore unlikely that the missing NP sentences in Experiment 1 were relatively acceptable because some of those sentences had a grammatical single center-embedding parse in which V1 or V2 verbs take such a VP complement.

To sum up: the results of Experiments 1 and 2 jointly indicate that Mandarin Chinese exhibits a center-embedding illusion analogous to the missing VP illusion reported for English. The implausibility manipulations in Experiment 1 further reveal an interesting difference: in a missing NP sentence, the two NPs that appear after the verbs get interpreted as objects and subjects of the verbs in a way that lacks a parallel in English missing VP sentences. These interpretations are also not predicted by existing accounts of the center-embedding illusion.

However, these remarks presume that our description of how missing VP sentences are interpreted is accurate. So that we have a stronger basis for comparing Mandarin with English, we ran a replication of Gibson & Thomas 1999. Our goal here was not to test their claims about structural forgetting. Rather, we were interested in using their manipulations to run an experiment parallel to Experiment 1: verifying whether English exhibits the missing VP illusion, and if so, which NPs serve as the arguments of which verb in a missing VP sentence.

6.1 Materials

Experimental materials were based on the 12 items listed on pp. 247–248 of Gibson & Thomas 1999. These were used to generate sentences for the four conditions in (3), repeated below as (20). We follow Gibson & Thomas in labeling these conditions as grammatical DCE, missing VP1, missing VP2, and missing VP3 sentences.

As noted previously, the missing VP conditions feature plausibility manipulations to confirm claims that speakers treat the third NP as the first VP’s subject, while treating the first NP (but not the second NP) as the subject of the second VP. If so, one expects a plausibility violation in the missing VP1 condition (20b): the first VP had typed quickly would take the third NP the publishing company as an agent. A similar violation would occur in the missing VP3 condition (20d), where the second VP, had typed quickly, takes the first NP the novel as its agent.

The 12 test sentences were distributed across four lists in a Latin Square design, so that in each list were three sentences per condition, and no two sentences were variants of each other. Following Gibson & Thomas, test sentences were mixed with 48 filler sentences of comparable length and complexity, featuring various kinds of adjunct, relative, and complement clauses. 30 of these sentences were well-formed and acceptable and 18 were ill-formed and unacceptable. This ratio was chosen so that participants would see as many acceptable sentences as unacceptable sentences, on the assumption that all four experimental conditions are relatively unacceptable.

We also prepared 7 practice items. Three of these items were grammatical and comprehensible while the other four items were not. One of the four ungrammatical practice sentences was the following non-DCE sentence missing a VP, used by Gibson & Thomas as a practice sentence: The form was stamped by the bureaucrat who worked at the ministry where everyone who had walked strangely.

6.2 Participants

Participants were 32 US-based workers on Amazon Mechanical Turk. All 32 participants were self-identified native speakers of American English and had passed a native speaker proficiency test, which tested for knowledge of relatively subtle grammatical rules and constraints of American English. Participants received US$2.50.

6.3 Procedure

The procedure was largely the same as that of Experiment 1. We departed from Gibson & Thomas in our choice of scale. Gibson & Thomas used a 5-point scale, where “5” indicated “hard to understand” and “1” “easy to understand.” To be consistent with Experiment 1, we used a 7-point scale and reversed it, so “1” was used for unacceptable sentences and “7” for acceptable ones.

For familiarization purposes, participants first judged the practice sentences. For the first three practice sentences, participants saw brief comments about each sentence and what kind of ratings to assign, e.g. a sentence should have received a low rating because it described an implausible scenario. The fourth practice sentence was Gibson & Thomas’s non-DCE sentence that lacked a VP. Following Gibson & Thomas, the comment for that sentence indicated that they should give similar sentences a low rating, although it did not explain why.

6.4 Data analysis

To ensure that participants took the time to read a target sentence before judging it for acceptability, we excluded responses that were provided in 3 seconds or less upon the presentation of the sentence; doing so eliminated about 6% of responses.

As was the case for Experiment 1, ratings were analyzed with a cumulative link mixed effects model, with conditions as a fixed effect and random intercepts and slopes for both participant and items. Ratings for the missing VP2 condition were used as a baseline.

6.5 Results

Table 5 shows that in absolute terms, all four conditions received low to medium acceptability ratings. Of the three missing VP conditions, the missing VP2 condition was rated the most acceptable; this outcome was also observed by Gibson & Thomas and Frank & Ernst (2019).

Statistically, the missing VP2 condition was significantly more acceptable than the missing VP1 condition (p = 0.001) and the missing VP3 condition (p < 0.001) (Table 6). Although the missing VP2 condition also received numerically higher ratings than the grammatical DCE condition, the difference is not significant (p = 0.357).

6.6 Discussion

Our results replicated Gibson & Thomas’s key findings. The fact that ungrammatical missing VP2 sentences were not less acceptable than grammatical DCE sentences indicates a grammaticality illusion. We also confirmed their observations about missing VP sentences: the first VP has a subject in the third NP, while the second VP has a subject in the first NP but not the second NP. In other words, the second VP is thematically related to the first NP, but not the second. This result contrasts with the finding for Mandarin missing NP sentences in Experiment 1, where the second post-verb NP (the equivalent of the second VP in English) is thematically related to both the first and second verbs (the equivalent of the first and second NPs in English).

Experiment 3 also provides another baseline to verify if a grammatical parse is at all possible for Mandarin missing NP sentences. Specifically, if the plausible missing NP sentences in Experiment 1 are ungrammatical (but illusory), they should have the same acceptability profile as missing VP2 sentences, whose ungrammaticality is not in doubt.

Post-hoc analyses of these experiments, discussed in more detail in Appendix 1, bear out this prediction. For example, plausible missing NP sentences, like missing VP2 sentences, generally receive lower ratings compared to unambiguously grammatical filler items, which are structurally similar (Figures 5 and 6).

Figure 5

Distribution of ratings across trials in Mandarin plausible missing NP condition and grammatical fillers.

Figure 6

Distribution of ratings across trials in English missing VP2 condition and grammatical fillers.

Ratings for plausible missing NP sentences also varied substantially within and across participants as well as items (Figures 7 and 8), which is expected if these sentences are ungrammatical but elicit the illusion probabilistically. The same variability was found in missing VP2 sentences (Figures 9 and 10). These parallels provide strong evidence that missing NP sentences are ungrammatical.

Figure 7

Distribution of ratings for Mandarin plausible missing NP sentences, by participant. (bold line in the middle of boxplot = median rating).

Figure 8

Distribution of ratings for Mandarin plausible missing NP sentences, by item. (bold line in middle of boxplot = median rating).

Figure 9

Distribution of ratings for English missing VP2 sentences, by participant. (bold line in the middle of boxplot = median rating).

Figure 10

Distribution of ratings for English missing VP2 sentences, by item. (bold line in middle of boxplot = median rating).

To sum up, Experiments 1 and 3 show that ungrammatical DCE sentences with missing NPs and VPs in Mandarin and English exhibit a grammaticality illusion, even though these ungrammatical sentences are assigned very different thematic relations.

These findings have implications for cross-linguistic accounts of center-embedding illusions. Ideally, an account must be fine-grained enough to explain the differences in thematic relations between Mandarin and English. However, it should also capture a higher-level generalization, namely, that Mandarin and English speakers experience a grammaticality illusion, while German and Dutch speakers are less likely to do so. In the next section, we address the second issue on the cross-linguistic distribution of the illusion. In particular, we evaluate an approach that attributes the distribution to cross-linguistic differences in language experience.

7.1 The model

We chose Futrell et al.’s model over simple recurrent neural network (RNN) models of the illusion used by Christiansen, Vasishth, Frank, and colleagues for three reasons. First, as mentioned, Futrell et al.’s model explicitly derives the presence and absence of the illusion in English and German as a consequence of the fact that German relative clauses are always verb-final, while English relative clauses are not. Second, the code is publicly available, which lets us reproduce their analysis faithfully. Third, as they argue, RNN models can be thought of as a “special case” of a lossy-context surprisal model (Futrell et al. 2020: 17–18).

While Futrell et al.’s model was presented as a model of reading times for DCE sentences (Vasishth et al. 2010), it is reasonable to use it to model acceptability judgments. Frank & Ernst (2019) note that Dutch and German have similar verb-final syntax and DCE sentences in both languages have the same reading time profiles. They further point out that reading time differences between grammatical DCE and missing VP sentences in Dutch and English correlate with an acceptability difference.

Futrell et al.’s model uses a simple probabilistic grammar, simulating English and German as a set of intransitive sentences. The focus on intransitive sentences is presumably driven by the goal to model the parsing of a prototypical DCE sentence, in which the subject’s head noun is separated from the verb by a relative clause containing another relative clause.

In this grammar (Table 7), a noun is modified by a relative clause 25% of the time. For English, 20% of relative clauses are verb-final, reflecting the fact that verb-final object relative clauses are infrequent in naturalistic contexts. For German, all relative clauses are verb-final. With these parameters, the grammar generates sequences of syntactic categories (corresponding to sentences) and their probabilities. Noise is then applied to these sequences, so that each syntactic category is deleted with a probability of 20%, producing sequences of varying grammaticality. It is this noisier set of sequences that the model is exposed to.

We extend the model with a simple probabilistic grammar of Mandarin. Since the prototypical Mandarin DCE sentence features a transitive verb separated from its object’s head noun by a doubly embedded relative clause, we have this grammar generate transitive sentences instead. Following Futrell et al., we assume that 20% of relative clauses are object relative clauses.

One might wonder whether these parameter values are realistic. With these values, the grammars for all three languages assign low, but non-zero, probabilities to grammatical DCE sentences before noise is applied, even though DCE sentences are probably effectively non-existent in naturalistic contexts. However, this model is arguably not intended to show how actual differences in language statistics can derive differences in how DCE sentences are parsed. Rather, it is a proof of concept, demonstrating how the parsing of DCE sentences can be conditioned by general word order differences, such as whether relative clauses are uniformly verb-final. These parameter values serve as simplifying assumptions necessary for the demonstration of this point.

7.2 Model results

We first ran the model for English and German/Dutch to replicate Futrell et al.’s results. Doing so also allows us to confirm that their choice of parameter values would be appropriate for Mandarin, for the sake of fairness. Having done that, we ran the model for Mandarin.

The English and German/Dutch models were presented with the DCE sequence NCNCNVV (N = noun, V = verb, C = relative pronoun). The model then calculated the probability that this sequence is followed by a V or an end-of-sentence symbol. To the extent that the English model generates the illusion, the surprisal of observing the end-of-sentence symbol should be lower than the surprisal for a third verb, even though the grammar generates DCE sentences. We predict the opposite contrast for German/Dutch. These predictions are borne out (Table 8).

The Mandarin model was presented with the DCE sequence NVVVNCN (here, C = de), and it calculated the probability that this sequence would be followed by a C (de) or the end-of-sentence symbol. As Table 8 shows, the model predicts the absence of an illusion: the observation of C, the grammatical continuation, has a lower surprisal value.

We also ran a post-hoc analysis for Mandarin, varying the conditional probability of subject relative clauses. Doing so let us assess the alternative hypothesis that Mandarin exhibits the illusion because subject relative clauses are less common in the language; Hsiao & Gibson (2003) estimate the rate of subject relative clauses to be about 57.5% (although Vasishth et al. (2013) estimate it to be higher, around 73%). The model predicts that Mandarin only starts exhibiting the illusion when the probability of subject relative clauses falls below 20% (Table 9), which is unrealistically low. We therefore discount this alternative hypothesis.

An anonymous reviewer raised the possibility that the results might be an artifact of the deletion rate. To address this issue, we reran the model with different deletion rates. As Table 10 shows, when the deletion rate is low (e.g. below 0.3), the model predicts that Mandarin lacks the illusion, patterning like German/Dutch: grammatical continuations elicit a lower surprisal than ungrammatical continuations. When the deletion rate is higher, the model predicts that Mandarin starts exhibiting the illusion, like English.

One possible takeaway is that the deletion rate should have been higher. For instance, contrary to our current assumptions (and Futrell et al.’s), perhaps the deletion rate increases with processing load, so that the deletion rate is higher for DCE sentences than for simpler sentences. This is not an implausible scenario. However, it requires a richer theory of deletion and representations beyond what is now available. For instance, one would need to specify when in incremental processing and by how much the deletion rate starts to rise (cf. Gibson & Thomas 1999). More generally, there are also potential consequences regarding how missing VP/NP sentences are interpreted, since this operation ostensibly deletes words from a linguistic representation. As far as we can tell, though, these issues have not been spelled out, so it is unclear how we can explain existing findings. For these reasons, we will not adopt this alternative perspective here, but flag it for future research.