2.3.1 Methodological differences

We now delve into the differences between the previous studies with respect to the methods they have employed. Adger et al. (2017) collected yes-no responses to a question asking about coreference with the only matching referent in the sentence. They did so by highlighting the R-expression and the pronoun, and asking if participants could use the sentence when the two referred to the same individual. Sentences were presented without context. Only those participants who passed a test where they had to indicate the impossibility of coreference with surface-level principle C violations were included in the sample.

Bruening & Al Khalaf (2019) collected a forced choice response between the two matching referents in the sentence, measuring which referent was preferred among participants. Although we cannot tell whether participants truly disallow a referent, the authors argue that the preference-based task is fairly simple and natural. If a grammatical constraint rules out either referent, responses should be close to zero, while chance level performance would indicate no such constraint. It is acknowledged that other factors, such as word order and the alternation between cataphoric and anaphoric reference may distort the picture further. The authors mention that rates should be ‘significantly different from zero’ if there is no reconstruction, though it is unclear what exactly counts as significantly different. Sentences were presented without context in this experiment, too.

Stockwell & Meltzer-Asscher & Sportiche (2021; 2022) used a sliding Likert scale for each intended reading ranging from 0 to 7, arguably the most complex method. In the relevant condition, participants had to judge the naturalness of the reading violating principle C and of the reading where the pronoun referred to an unknown referent. There were thus two independent tasks per trial. In this experiment, too, sentences were presented without specific contexts, but participants were informed before the experiment that they should imagine being at a party and joining an ongoing conversation. Although the aim of this complex task was likely to allow for more nuanced responses, the authors report that the mean ratings of the two readings in each condition add up roughly to 8.0. This indicates that an increased rating for one reading led to a decreased rating for the other reading. Rather than improving the precision of the measurement, it appears that the complexity encouraged people to engage in simplification strategies of their own.

In the experiment by Salzmann & Wierzba & Georgi (2023), participants had to complete two forced choice tasks. The two questions were whether the sentence could be interpreted such that (i) the referent in the matrix clause could corefer with the pronoun or (ii) the referent in the PP could corefer with the pronoun, respectively. The intention was to collect explicit judgments on both interpretive options. In addition, the authors informed participants that the sentences may have multiple interpretations, even if one of them is more readily available than the other, and that they should consider each of the options carefully. In a training sequence before the experiment, participants were shown an ambiguous sentence to illustrate this point (Maria hat Anna besucht, weil sie nett ist ‘Mary visited Anna because she is nice’). Though other groups do not report about their instructions in as much detail, it appears that Salzmann & Wierzba & Georgi (2023) explicitly communicated the intention of the task to their participants, which on the one hand, ensures that they understood it, but on the other hand, may also encourage participants to spend more time thinking about the items and being less spontaneous with their judgments. In general, there is nothing inherently wrong about instructing participants more explicitly or inviting them to think about their responses. However, in the case of coreferent judgments, which are known to be highly susceptible to pragmatic factors, participants may end up accepting the coreferent reading regardless of a syntactic violation if they take enough time to contemplate.

While there is some level of uncertainty associated with giving people only one forced choice task, providing them with more complex tasks and instructing them to consider each option carefully may induce a bias to be more accepting and potentially consider non-syntactic cues that may facilitate the acceptability of a violation. Given the involvement of non-syntactic factors in coreference resolution, these factors may have additionally influenced the outcomes.

2.3.2 Item structure

Let us now turn to the differences between the experimental items themselves across studies. Authors vary with respect to whether they presented participants with interrogative or declarative sentences embedding the wh-dependency. This coincides with the presence of an alternative referent that the pronoun could saliently refer to.

Notice the three-way opposition in previous experiments regarding alternative referents: in the case of Adger et al. (2017), the possibility of an alternative referent was not mentioned at all, Stockwell & Meltzer-Asscher & Sportiche (2021; 2022) gave participants scales to assess coreference with the referent in the sentence and someone else, while Bruening & Al Khalaf (2019) and Salzmann & Wierzba & Georgi (2023) included a named alternative referent in their experimental items. If not given an alternative at all or if the alternative is not particularly salient, participants may be biased to resolve pronominal reference with whatever referent is available (Gordon & Hendrick 1998). It is therefore unclear if, in the case of Adger et al. (2017), fairly high coreference rates were obtained because there was no reconstruction, or if people simply chose the referent present in the sentence because it is less costly than postulating another referent in the discourse model without any prompt to do so. On the other hand, the properties of the alternative referent included by Bruening & Al Khalaf (2019) and Salzmann & Wierzba & Georgi (2023) may have the opposite effect. In both experiments, the referent was the subject of the matrix clause, making it a strong contender for coreference and a potential distraction from the embedded referent. Participants may have responded less favorably to the referent in the PP simply because the alternative referent was extremely prominent.

There are also some potential confounds resulting from the choice of verbs. In particular, Salzmann & Wierzba & Georgi (2023) used psych verbs to maintain minimal pairs across conditions with subject and object extraction, allowing for pairs such as which story about Hanna delighted her and which story about Hanna she found delightful. Temme & Verhoeven (2017) argue that affectedness can increase the acceptability of principle C violations particularly in German, meaning that higher coreference rates should be obtained with psych verbs than transitive verbs. However, in that case one may find that varying verb type across different conditions of the same item plays a role, and arguably, this would lead to greater variability across items than consistently using verbs with potential confounds. The next section discusses the experimental designs, in particular the control construction(s) that the supposed underlying principle C violation has been compared to in the existing studies.

2.3.3 Experimental designs

We now examine the experimental designs employed by previous researchers in more detail. Adger et al. (2017) did not collect data for PP modifiers functioning as adjuncts, only arguments, given that reports in the literature converge about the former but not the latter. All authors collected responses to surface violations of principle C, intended as a baseline as to what response a violation should elicit.³ Salzmann & Wierzba & Georgi (2023); Stockwell & Meltzer-Asscher & Sportiche (2021; 2022) additionally used items where only underlying c-command was manipulated without changing the linear order of the R-expression and the pronoun. Salzmann & Wierzba & Georgi (2023) varied the grammatical function of the extracted phrase, while Stockwell & Meltzer-Asscher & Sportiche (2021; 2022) used causative constructions such as which picture of Harry made him laugh, comparing them to regular transitives to achieve this distinction.

The divide between the groups arguing for and those arguing against the reconstruction of PP modifiers corresponds to the use of c-command as an experimental factor. Recall that Salzmann & Wierzba & Georgi (2023); Stockwell & Meltzer-Asscher & Sportiche (2021; 2022) argue that there is reconstruction while Adger et al. (2017); Bruening & Al Khalaf (2019) argue that there is not. In the former groups’ experiments, items without an underlying principle C violation allow for coreference more readily than ones where reconstruction leads to a violation. The experiments by Adger et al. (2017); Bruening & Al Khalaf (2019) base their arguments on the observation that supposed underlying violations are indicated to be possible much more frequently than surface violations of principle C, which, however, is confounded with the linear order of the pronoun and the R-expression (via controls in the case of Adger et al. 2017). The evidence provided by Stockwell & Meltzer-Asscher & Sportiche (2021; 2022); Salzmann & Wierzba & Georgi (2023), on the other hand, provides straightforward information on the relevance of c-command. The review of experimental designs in this section highlights that previous claims have been made on quite disparate grounds, making the experiments difficult to evaluate side by side. This is why interpreting the findings and placing them into context is particularly important. In what follows, we take a closer look at the different ways in which previous authors chose to do this.

2.3.4 Interpreting the results

To contextualize previous authors’ conclusions, it is crucial to understand how they interpret their results. Groups measuring proportions of responses unanimously report them to be below chance for the condition in which a principle C violation is predicted to occur, yet interpret this below-chance performance in different ways. The groups arguing against reconstruction, i.e. Adger et al. (2017) and Bruening & Al Khalaf (2019), rely on the observation that responses indicating the possibility of or the preference for coreference between the pronoun and the R-expression are well above zero. They argue that if there was a grammatical constraint, these kinds of results would be unexpected, especially because they found that responses were below 10% in items where a principle C violation without movement is involved. The authors therefore conclude that PP modifiers do not reconstruct based on a contrast between items with and without movement. Salzmann & Wierzba & Georgi (2023) also find coreference rates well above zero but below chance in conditions with PP modifiers functioning as arguments (35.9%). Crucially, however, they reach a different conclusion due to separating linear precedence from underlying c-command by comparing object and subject extraction (instead of surface vs. underlying violations of principle C, where a null result would indicate full reconstruction). Because coreference rates differ by roughly 15% between the conditions with and without an underlying principle C violation, and this effect is statistically significant, they conclude that there is reconstruction. They further argue against an argument-adjunct asymmetry in German because the exact same coreference rate was obtained with argument and adjunct PPs. We see, therefore, that similar coreference rates are interpreted in different ways depending on what comparisons research groups have included. Designs relying on a null effect, such as Adger et al. (2017) and Bruening & Al Khalaf (2019), are difficult to evaluate given that one has to rely on an arbitrary threshold-based reasoning (for a thorough discussion, see Salzmann & Wierzba & Georgi 2023).

Stockwell & Meltzer-Asscher & Sportiche (2021; 2022) collected judgments on a scale for each available reading rather than a binary coreference judgment. In their first experiment where the effect of distance was tested with PP arguments, they report that the coreferent reading scored a mean rating of 1.95, while the reading where the pronoun referred to an unnamed referent scored a mean rating of 5.67. They argue in favor of reconstruction because the rating for the unnamed referent increases if a violation is present. In their second experiment, the mean rating for the coreferent reading with PP arguments was 2.19 and 3.24 for PP adjuncts. Because this contrast is statistically significant, the authors argue that there is an argument-adjunct asymmetry in English. Recall that this group also isolated c-command from linear order by comparing causative and transitive structures. The experiment revealed a higher mean rating for the coreferent reading in the absence of a principle C violation, leading the authors to conclude that PP modifiers do reconstruct.

Recall that Stockwell & Meltzer-Asscher & Sportiche (2021; 2022) hypothesize that participants may have responded with preferences rather than possibilities. This is supported by the observation that the mean ratings of the two readings in each condition add up to roughly 8.0, indicating that an increase of one reading’s rating led to the decrease of the other’s rating. Notice also that the authors interpret an increased rating for the alternative referent as an indicator for a principle c violation rather than focusing on whether the rating for the embedded referent decreased. The definition of a principle C effect is the unavailability of coreference with the R-expression that is c-commanded by the pronoun, not the prominence of alternatives. The results of Stockwell & Meltzer-Asscher & Sportiche (2021; 2022) thus may be quite similar to those obtained by Bruening & Al Khalaf (2019), but the groups crucially differ in how they interpret them. For Stockwell & Meltzer-Asscher & Sportiche (2021; 2022), determining whether there is a principle C reconstruction effect is based on two components: the decrease of ratings for the named referent as well as the increase of ratings for the unnamed referent. That is, while mean ratings for the coreferent reading may look similar in the argument and adjunct conditions (reflected in the lack of a significant main effect of relation reported by Stockwell & Meltzer-Asscher & Sportiche 2022), it is crucially the larger increase of the mean rating for the unnamed referent that leads the authors to the conclusion that there is an argument-adjunct asymmetry (the significant interaction of response and relation). For Bruening & Al Khalaf (2019), and arguably also Adger et al. (2017), the possibility of coreference with the R-expression in the PP being well above zero is taken to indicate that there cannot be a syntactic violation. These differences in what researchers decide to focus on reveal a fundamental problem – syntactic theory makes categorical predictions, yet the data do not reflect them. It is up to the researcher to tease apart the confounds and identify the confounding factors that may lead to the observed results. The fact that in most studies, a threshold-based reasoning needs to be adopted to interpret the data creates problems due to the lack of objective parameters.

Running experiments is not only a welcome addition to introspective data, it is necessary if the introspective judgments diverge. However, if supposedly objective, experimental evidence seems to further fuel the debate rather than settle it, we need to investigate whether the experimental design and methods are causing this. The existing experiments, when compared, reveal a lot about what to consider when studying principle C reconstruction experimentally. However, they do not provide a definitive answer to whether principle C reconstruction exists. I came up with two experiments following up on the work by Salzmann & Wierzba & Georgi (2023), focusing on the experimental task and the presence of an alternative referent in the target item, while keeping the lexical content of the items consistent. By doing so, the novel data allow us to verify how much of the discrepancy between the results stems from actual properties of principle C reconstruction, and how much is indebted to methodological and item-related confounds.

3.2.1 Materials

In the first experiment, the 32 items from experiment 2 by Salzmann & Wierzba & Georgi (2023) were used, including only the conditions ‘object, argument, moved’ and ‘subject, argument, moved’.

Additionally, 24 target items from another experiment exploring a different research question were presented. Eight unambiguous fillers were also included. Before testing started, participants were guided through three training trials, including feedback explaining the logic of the task. Targets items were distributed in a 2x2 Latin square design.

3.2.2 Method

Participants were presented a single coreference judgment task per trial. Participants only had to decide whether the pronoun could refer to either of the referents, varying which referent the task was about across trials. Metalinguistic terms were avoided by repeating the sentence with the intended reading (cf. Salzmann & Wierzba & Georgi 2023). Each sentence was accompanied by a neutral context introducing both referents, aiming to mitigate the prominence of the matrix referent by having both referents present in the discourse before encountering them in the target item. Whether the wh-phrase was an object or a subject was encoded as the factor phrase. This factor was fully crossed with another factor, referent, encoding whether the task inquired about the matrix or the embedded referent, with the matrix conditions serving as a sanity check. In subject conditions, both the surface and the base position of the extracted phrase c-commands the pronoun, i.e. there should be no principle C violation regardless of reconstruction, and coreference between the embedded referent and the pronoun should always be possible. In object conditions, however, the surface position of the displaced phrase c-commands the pronoun, but under reconstruction, this relation is reversed, i.e. the pronoun c-commands the base position of the displaced phrase, yielding a principle C violation.

3.2.3 Participants

A total of 175 participants were recruited over the platform Prolific, with mean age 32.09, n = 87 identifying as male, n = 84 as female, and n = 4 as non-binary. Monetary compensation was provided for everyone who completed the experiment successfully. Participants were native speakers of German living and raised in Germany (76.3%), Austria (19.08%) or Switzerland (4.62%). The experimental conditions ‘object, matrix’ and ‘subject, matrix’ served as attention checks, with failure to indicate coreference with the matrix referent in more than 20% of cases leading to exclusion. This threshold was also implemented to assess whether participants understood the task and considered coreference possibilities rather than preferences. Based on this criterion, 25 participants were excluded.

3.2.4 Procedure

The experiment was set up through the platform L-Rex (Starschenko & Wierzba 2024). Participants were instructed to read the items carefully, but to decide based on their first impression. They were also encouraged to use the comment function in case they felt that the forced choice task alone did not fully express their judgment. The sentences were displayed simultaneously with the context, the latter in italics. The question was shown below the sentence with the answer options yes, indicating coreference between the respective referent and the pronoun, and no, indicating disjoint reference. The first block included the training items in non-randomized order, first showing an example of coreference, then disjoint reference, and an ambiguous sentence as the third example. Participants were explicitly guided through this training sequence and were given feedback indicating that yes should be chosen in cases of ambiguity. This was done to ensure that the experiment assessed coreference possibilities, not preferences. The second block contained target items, pseudofillers and fillers in pseudo-randomized order, such that two items from the same set of materials were never shown consecutively.

3.2.5 Hypotheses

Assuming that the mediocre proportions of coreference reported by Salzmann & Wierzba & Georgi (2023) are due to the complexity of the experimental task, and that their conclusions about reconstruction are correct, the majority of responses in the condition ‘object, embedded’ should be no. Responses indicating coreference with the referent in the sentence are expected to occur in 25% or less of the observations in ‘object, embedded’. Simultaneously, the majority of responses in the condition ‘subject, embedded’ should be yes, given that there is no principle C violation. Responses indicating the possibility of coreference are expected to occur in around 75% of the observations, indicating above chance performance. Neither floor nor ceiling effects were expected in either condition due to non-syntactic influences on coreference.⁵ Overall, simplifying the experimental task is expected to yield clearer results than found in previous work. This should manifest in a significant main effect of phrase if principle C reconstruction is a robust syntactic constraint based on c-command. On the other hand, if PP modifiers do not reconstruct, we expect no pronounced difference between the conditions ‘object, embedded’ and ‘subject, embedded’, meaning no significant effect of phrase.

3.2.6 Results

Coreference rates across conditions are illustrated in Figure 2. Participants chose the answer yes, indicating the possibility of coreference, in 19.4% of the observations in the condition ‘object, embedded’, and in only 26.4% of the observations in the condition ‘subject, embedded’. At the same time, coreference rates in conditions inquiring about the matrix referent are between 96.1% and 98.8%.

Figure 2: Coreference rates across conditions in experiment 1. Error bars indicate standard error.

Figure 3 visualizes the crucial information regarding how individual participants’ responses change from one condition to the other. Only 42.7% of participants show an effect in the predicted direction, while 57.3% show the opposite or no effect at all. Even among the 42.7%, the increase from object to subject condition is nowhere near the predictions of a syntactic constraint. There are barely cases moving from below chance to above chance, let alone indicating the clear possibility of the coreferent reading in the subject condition and the opposite in the object condition. The directionality in the data along with the visual cues is in line with the statistical models, or rather, the problem of fitting any. The data was analyzed in R (R Core Team 2024), modeled by a generalized linear mixed effects model using the glmer function with the family binomial (logit link) and the optimizer bobyqa (Bates et al. 2015). The model only included the conditions asking about the embedded referent, since the conditions inquiring about the matrix referent solely served as a sanity check. The dataset was thus reduced to the two levels of phrase as the conditions ‘object’ and ‘subject’. A conservative α-level of 0.05 was defined. The model included a random effects structure with varying intercepts and slopes for condition, for both participants and items (Barr et al. 2013). Factors were treatment contrast coded such that the intercept is the estimate of the baseline probability of yes-responses in the ‘subject’ condition. The estimated slope indicates the change in log-odds from ‘subject’ to ‘object’ condition.

Figure 3: Individual participants’ (n = 150) overall proportions of yes responses in experiment 1 by condition. Points on y-axis indicate respective proportion (ranging from 0 to 1 in steps of 0.125 based on eight observations per participant per condition), lines connect individual participants’ proportions in the condition ‘obj,emb’ to ‘subj,emb’. Color indicates whether the direction of the effect conforms to the predictions based on a principle C violation, thickness and opacity indicate how frequent the respective combination of proportions from the two conditions is in the data set.

This maximal model revealed a perfect negative correlation of the random intercepts and slopes: the higher the intercept of a participant, the lower their slope. Because the model needs to estimate two values per participant and these values have limited informativity due to the correlation, it is likely overfitting and therefore not providing reliable estimates for the fixed effects. Two more models were fitted. One excluded the correlation between the random effects, the other one reduced them to include estimates for random intercepts only. A Likelihood ratio test using the anova function revealed that the more complex model, despite its fitting problems, is superior to the models with a simplified random effects structure, indicated by a lower AIC and a significant p-value. Still, the estimates suggest that the maximal model is compensating for noise rather than identifying stable and interpretable effects. This is particularly evident from inspecting the log-transformed estimates and confidence intervals, which reveal that the model is estimating proportion of yes-responses to decrease from object to subject condition, and it likewise overestimates the intercepts. The log-transformed estimates and confidence intervals of the model including varying intercepts and slopes are reported below using the function ggemmeans (Lüdecke 2018), followed by the summary of the estimates from the maximal model in log odds in Table 2. Significance testing revealed no significant effect of phrase. Estimates for the random effects by participants are extremely high. The following discussion includes log-transformed estimates, see Table 3, with the estimate on the log scale as shown in the table given in parentheses. The intercept by participant, that is, the baseline response of individual participants, is estimated to vary by 0.9986 (6.60).

Given that we are dealing with proportions, that is, values between 0 and 1, this indicates that participants are estimated to vary along the entire possible range of responses. The corresponding slopes, i.e. the effect of the predictor for individual participants, are estimated to vary by 0.6295 (0.53). On the other hand, the baseline response by item is estimated to vary by 0.6341 (0.55), with slopes estimated to vary by 0.5374 (0.15). The model thus also attributes a fair amount of variability to the items. Notice that the estimates for the random effects, particularly by participant, are much higher than for the fixed effects. To evaluate how dominant the random effects truly are, a fourth generalized linear mixed effects model was fitted, this time completely omitting the fixed effect, estimating only the intercept and the random effects structure:

A likelihood ratio test comparing the generalized mixed effects model in (13) to the model in (14) revealed that the latter model, i.e. the one entirely excluding the effect of phrase, has the superior fit based on a lower AIC and p-value.⁶ The dominance of random effects by participant found in the model is in line with the data visualization in Figure 3.

3.2.7 Discussion

The evidence does not suggest that a principle C violation under reconstruction played a role in determining participants’ responses in this experiment. Generally, coreference rates are much lower than in the data collected by Salzmann & Wierzba & Georgi (2023), and in particular very low in the subject condition despite no syntactic violation inhibiting coreference. It may be the case that the presence of a matrix referent has a depressing effect on the coreference rate with the embedded referent. There are two potential reasons: on the one hand, participants may be choosing the embedded referent less frequently because the matrix referent is simply too prominent. It is mentioned first in the sentence, it is the subject, and neither of its positions is c-commanded by the pronoun. On the other hand, it may further be preferred over the embedded referent because it is both structurally and linearly more distant from the pronoun (cf. Adger et al. 2017; Stockwell & Meltzer-Asscher & Sportiche 2021). The overwhelming prominence of the matrix referent is supported by comments provided by 18 participants in the conditions inquiring about the embedded referent, some of which indicate ambiguity, others specifically reinforcing the interpretation regarding the matrix referent, which was not offered in the respective trials.

There may also be methodological reasons why coreference rates are generally low. The simplified task may be a poor fit to address the research question as it is. Since it is formulated as a forced choice task with the response options yes or no, participants are essentially asked to evaluate the truth value of only one reading of a potentially ambiguous sentence. This is not supported by the observation that the matrix referent is given a positive response in nearly all observations. However, one could argue that participants establish coreference with the matrix referent much more readily based on the aforementioned prominence factors, and simply do not consider the additional, weak possibility of the reading involving the embedded referent to be problematic for the truth value judgment regarding the matrix referent reading. Again, this is supported by participants’ comments explicitly reinforcing the matrix referent reading despite being asked about the embedded referent. Like in the experiments by Stockwell & Meltzer-Asscher & Sportiche (2021; 2022), participants may have responded based on preferences rather than possibilities.

Closer inspection of the data distribution among participants reveals that the overall coreference rates are the result of computing the mean over extreme inter-individual variability. This is particularly evident from how different statistical models fit the data. Recall that the maximal model, though indicated to be a better fit than a simplified version by a likelihood ratio test, (i) has severe fitting problems due to a perfect negative correlation for random intercepts and slopes, (ii) overestimates the intercepts in general, and (iii) estimates the predictor to have an effect in the opposite direction than indicated by the overall coreference rates in the observed data. Modeling under omission of the predictor revealed that a model based solely on the random effects structure fit the data best, indicating that the distribution of data is best explained in terms that completely ignore the information of underlying c-command and therefore, a principle C violation under reconstruction. Due to the additional confounds introduced by the design that are discussed above, the conclusion that principle C does not play a role at all may nevertheless be premature. Recall that participants were only asked about one out of two readings per trial, potentially making the someone else response dispreferred because the experimental task was poorly chosen altogether considering the fully intended ambiguity of the items.

To summarize, the experiment did not provide data illustrating the previously reported subject-object contrast more straightforwardly, it achieved quite the contrary. Although this may be indebted to the method itself, it is noteworthy that any potential effects traditionally associated with a principle C violation are apparently quite easily overridden, particularly in the subject condition where coreference is not inhibited by any syntactic factors. Under the assumption that there is no principle C reconstruction, a more plausible scenario would involve chance level results. The evidence so far seems to favor a view where relatively low coreference rates are not determined by an underlying principle C violation, but rather by the presence of a competing referent. This issue is explored in the second experiment, which is outlined in the following section.

3.3.1 Materials

In the second experiment, participants saw an interrogative sentence in each trial. The sentence contained a displaced NP modified by a PP containing the only referent in the clause, followed by a pronoun matching the referent (resembling items by Adger et al. 2017; Stockwell & Meltzer-Asscher & Sportiche 2021; 2022). The study followed a Latin square design.

As in experiment 1, 24 target items exploring a different research question were also presented. 12 unambiguous fillers were included, involving a referent and a morphosyntactically mismatched pronoun, making only the response someone else acceptable. Participants were guided through three training trials illustrating the task.

3.3.2 Method

Participants were again given a single forced choice task per trial. This time, they were asked what the presented sentence was about and had to decide between two readings. The task thus resembles the task in the experiments by Stockwell & Meltzer-Asscher & Sportiche (2021; 2022). Crucially, instead of two scales judging the naturalness of the offered readings, it was a simple forced choice task measuring preference between the only referent in the sentence and someone else.⁷ Again, metalinguistic terms were avoided by providing full sentences with the intended reading. The order in which the response options were shown was pseudo-randomized across trials. Sentences were not accompanied by individual context. Instead, participants were given the instruction that they should imagine being at a party and picking up snippets of conversations (Stockwell & Meltzer-Asscher & Sportiche 2021; 2022). Items only appeared in two conditions based on the grammatical function of the displaced wh-phrase, i.e. ‘subject’ and ‘object’. Like in experiment 1, both the surface and the base position of the extracted subject c-commands the object pronoun, and, if there is reconstruction, a principle C violation should only occur in the object condition.

3.3.3 Participants

A total of 64 participants have been recruited over the platform Prolific, with mean age 32.2, with n = 39 identifying as male, n = 24 identifying as female, and n = 1 identifying as non-binary. Monetary compensation was provided to everyone who successfully completed the experiment. Participants fulfilled the same criteria as in experiment 1 (4.69% from Austria, 92.19% from Germany, 3.12% from Switzerland). It was made sure that none of the participants from experiment 1 took part in experiment 2. Although testing different methods with the same participant pool is beneficial to reduce variability, due to data collection for both experiments taking place within a short time window, the priority was to avoid familiarization or adaptation effects from the first experiment affecting the results of the second experiment. Fillers contained a referent and a pronoun with mismatched ϕ-features. Participants who chose the reading indicating coreference between the two in at least 25% of the respective trials were excluded, leaving 60 participants in the sample.

3.3.4 Procedure

The experiment was set up using the platform L-Rex (Starschenko & Wierzba 2024). The instructions were the same as in experiment 1. The sentences were shown one by one, with the question Worum geht es hier? ‘What is this about?’ being shown below them. The answer options were repetitions of the event described in the sentence, as shown in (15) and (16).⁸ The first block included the training items in non-randomized order, showing two cases where the referent in the sentence could corefer with the pronoun and one case where it could not due to a morphosyntactic mismatch. The second block contained target items, pseudofillers and fillers in pseudo-randomized order, such that two items from the same set of materials were never shown consecutively.

3.3.5 Hypotheses

This experiment measures preferences in contrast to the experiment by Salzmann & Wierzba & Georgi (2023) and experiment 1 reported herein. This has important implications for the expected response rates. If the PP modifier containing the referent reconstructs, coreferent readings should be dispreferred in the object condition, with the embedded referent being chosen well below chance. In the subject condition, chance level performance is expected due to the lack of a syntactic violation. If there is no reconstruction, chance level performance is expected in the object condition, too. Because it is a single forced choice task between two referents, if the proportion to which the embedded referent is chosen increases, the proportion to which the unnamed referent is chosen automatically decreases. Chance level performance indicates no preference for either of the readings. Generally, given the lack of an alternative, higher proportions indicating coreference with the embedded referent are expected. If an increase is observed only in the subject condition, this would indicate an underlying principle C violation. if, however, participants choose the embedded referent above chance across conditions, this would indicate a general bias to resolve pronominal reference with any matching antecedent given in the discourse.

3.3.6 Results

Proportions of responses indicating coreference with the embedded referent across conditions are illustrated in Figure 4. Participants chose the answer indicating coreference with the embedded referent in 40.3% of the observations in the object condition. In the subject condition, the embedded referent was chosen in 54% of the observations, indicating no clear preference. There is no matrix referent in the sentence, meaning coreference with it cannot be assessed as a sanity check. Participants’ performance in the subject condition is in line with the lack of a principle C violation. Confronted with the two possible readings, participants show no clear preference for the embedded referent or the unnamed referent (someone else). In the object condition, participants show a slight preference for the option someone else, though this is not as strong as previously hypothesized.

Figure 4: Coreference rates across conditions in experiment 2. Error bars indicate standard error.

Figure 5 shows the distribution of data among participants. Given the omission of the matrix referent, participants each saw 16 items per condition, meaning that the individual proportion of responses indicating coreference with the embedded referent range from 0 to 1, increasing by 0.0625 at a time.

Figure 5: Individual participants’ (n = 60) overall proportions of responses indicating coreference with the embedded referent in experiment 2 by condition. Points on y-axis indicate respective proportion (ranging from 0 to 1 in steps of 0.0625 based on 16 observations per participant per condition), lines connect individual participants’ proportions in the condition ‘obj’ to ‘subj’. Color indicates whether the direction of the effect conforms to the predictions based on a principle C violation, thickness and opacity indicate how frequent the respective combination of proportions from the two conditions is in the data set.

Based on Figure 5, it is clear that participants show a vast degree of variability. There is no clustering among participants around chance level. Much rather, participants seem drawn toward the extremes. The majority of participants exhibit combinations of response proportions that are in line with the predictions of a principle C violation. However, pronounced slopes indicating a systematic preference for coreference in the subject but not the object condition and thus a syntactic violation, are extremely rare. 33.3% of participants show no effect at all or a reverse effect.

The data was analyzed in R (R Core Team 2024) using a generalized linear mixed effects model glmer with the family binomial (logit link) and the optimizer bobyqa (Bates et al. 2015) and a conservative α-level of 0.05. The model included a random effects structure with varying intercepts and slopes both per participants and items. The estimates from the maximal model specified in (17) are reported in Table 4.

Significance testing revealed a significant effect of phrase. As with the data from experiment 1, this model too had fitting problems, revealing a perfect negative correlation of random slopes and intercepts by item, but not by participant. The estimates for the random effects are quite large, i.e. the intercept by participant is estimated to vary by 0.9923 (4.86), while the accompanying slopes are estimated to vary by 0.6735 (0.72). By item, the intercept is estimated to vary by 0.6433 (0.59) and the slopes by 0.5049 (0.02). Model comparison with a simplified model excluding random slopes revealed that the more complex model was nevertheless superior based on a lower AIC and a significant p-value. However, this was again accompanied by large confidence intervals, a slope that is estimated to be much larger than the observed data, and a sign reversal. Upon suspicion that the model was relying on the random effects rather than the the fixed effect to derive the estimates, another generalized linear mixed effects model was fitted omitting the predictor phrase altogether (identical to the formula in (14)). However, a likelihood ratio test revealed that the model including the predictor was superior to the random effects only model (p = 1.316e-08), indicating that phrase is informative to the model and improves its fit.

The significance test using the maximal model, though possibly inflated, is therefore not entirely misleading. The log-transformed estimates from the maximal model including confidence intervals are summarized in Table 5.

3.3.7 Discussion

In this experiment, coreference rates increased compared to the previous studies reported in section 3.1 and 3.2, respectively. This increase is particularly dramatic because the experimental task requires a different interpretation of the results. While in the previous experiment, participants were given a forced choice task between yes and no concerning one interpretation of a sentence, experiment 2 forced participants to choose between two possible interpretations. In other words, while in prior experiments each referent could score a 100% coreference rate independently (within a single trial in the study by Salzmann & Wierzba & Georgi 2023 and across trials in experiment 1 reported herein), in experiment 2, a 100% choice rate of one referent automatically leads to a 0% choice rate of the other. Participants are at chance level in the subject condition. There is a statistically significant effect leading to 14% fewer responses indicating coreference with the embedded referent in the object condition compared to the subject condition. This shows that a principle C violation decreases the proportion of responses indicating coreference under reconstruction. The overall coreference rate of 40.3% also indicates that isolating c-command as an experimental factor is crucial, since a violation does not rule out the reading completely. Table 6 summarizes the coreference rates of experiments 1 and 2.

The simplification of the task was an attempt to approximate the setup used by Stockwell & Meltzer-Asscher & Sportiche (2021; 2022). The alternative referent, which intended to ease resolution of pronominal reference in prior experiments, was omitted due to the hypothesis that its presence was distracting from the embedded referent. This turned out to be correct, participants were much more willing to choose the embedded referent in this experiment, where they were given no salient alternative. This is indicative of the importance of non-syntactic factors, such as a trivial bias to assign reference to a referent already present in the discourse (cf. Gordon & Hendrick 1998). The effect of phrase is significant, indicating that participants’ behavior is at least partially informed by the presence or absence of an underlying principle C violation. The model indicating the significance of this effect nevertheless had severe fitting problems, manifesting in overestimation, large random effects, and a sign reversal. Further inspection of variability through visualizing individual participants’ proportions of responses by condition, see Figure 5, confirmed that while there is a tendency to respond according to principle C, participants are generally scattered all across the range of possible responses in both conditions. Inspecting individual participants’ data is imperative for understanding the implications of this experiment. It is not the case that participants are generally behaving close to chance level, or that there is a clear clustering. The patterns observed in these data are not merely noisy, they reflect that coreference resolution is guided by a number of factors. A principle C violation under reconstruction, while appearing to be one of them based on experiment 2, appears to play a variable role from participant to participant.

In sum, the experiment yielded a slightly more pronounced contrast between subjects and objects and a significant effect, providing evidence in favor of principle C reconstruction. This is further supported by individual participants’ slopes that show an increase in responses indicating coreference from object to subject condition. Consider that the magnitude of this effect does not compare to previous hypotheses about an underlying principle C violation ruling out coreference entirely. While the coreferent reading is chosen less given an underlying principle C violation than in the absence of such a violation, we observe a general bias to resolve coreference with whatever referent is given in the sentence. The principle C reconstruction effect does not hold universally across participants either. The subsequent section places these findings in a broader context.