PONE-D-24-16699Registered Report: Neural correlates of thematic role assignment for passives in Standard IndonesianPLOS ONE

Dear Dr. Jap,

Thank you for your continued patience throughout the review process. After a careful review of the manuscript and the detailed feedback provided by the reviewers, I have reached a decision regarding your submission. As you will notice, the reviewers’ opinions are somewhat divided. While Reviewer 2 is largely positive and suggests only minor revisions, Reviewer 1 has raised significant concerns, particularly regarding the study’s experimental design and the interpretation of the ERP data. Specifically, Reviewer 1 points out that the ERP obtained could reflect the processing of both the target word and the preceding word, which is considered as a critical limitation. Additionally, Reviewer 1 has suggested several areas where the clarity and detail within the introduction and discussion sections could be improved.

After considering both reviewers’ comments and my own assessment of the manuscript, I have decided to request a major revision. This decision is based on the following considerations:

1.  Given that the study was pre-registered, I believe it would be unreasonable to request significant changes to the experimental design at this stage.
2.  As far as I am aware, a stimulus onset asynchrony (SOA) of 600 ms is not uncommon in ERP studies of sentence processing. And in this study, the sentences in the active and passive conditions were matched, which should somewhat mitigate the problems of carry-over responses form the previous words.
3. I appreciate that this study focuses on Indonesian, an under-represented language in psycholinguistic research. This adds value to the manuscript, and I recognize that no study is without its limitations.

That said, it is crucial to address Reviewer 1’s concerns carefully. In your revision, please respond comprehensively to the points raised. If you believe certain issues cannot be fully resolved, please discuss them openly as limitations of the study.

Additionally, I would like to see further elaboration on your decision to adopt a p-level of 0.1 for the statistical analysis. While the manuscript currently states that “the rationale for using a cluster threshold of p < 0.1 was to make the test more sensitive to weak, sustained effects,” there is growing concern within the scientific community about the risk of false positives and the replicability of findings. In your revision, I encourage you to provide a more detailed justification for how you balanced the aim of increasing statistical power with the need to minimize Type I errors.

I look forward to your revised submission. After receiving your revision, I will make the final decision as quickly as I can. Please do not hesitate to contact me if you have any questions.

Please submit your revised manuscript by Nov 01 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org . When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols . Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols .

We look forward to receiving your revised manuscript.

Kind regards,

Yiu-Kei Tsang

Academic Editor

PLOS ONE

Journal requirements:  

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please include a complete copy of PLOS’ questionnaire on inclusivity in global research in your revised manuscript. Our policy for research in this area aims to improve transparency in the reporting of research performed outside of researchers’ own country or community. The policy applies to researchers who have travelled to a different country to conduct research, research with Indigenous populations or their lands, and research on cultural artefacts. The questionnaire can also be requested at the journal’s discretion for any other submissions, even if these conditions are not met.  Please find more information on the policy and a link to download a blank copy of the questionnaire here: https://journals.plos.org/plosone/s/best-practices-in-research-reporting. Please upload a completed version of your questionnaire as Supporting Information when you resubmit your manuscript.

3. When completing the data availability statement of the submission form, you indicated that you will make your data available on acceptance. We strongly recommend all authors decide on a data sharing plan before acceptance, as the process can be lengthy and hold up publication timelines. Please note that, though access restrictions are acceptable now, your entire data will need to be made freely accessible if your manuscript is accepted for publication. This policy applies to all data except where public deposition would breach compliance with the protocol approved by your research ethics board. If you are unable to adhere to our open data policy, please kindly revise your statement to explain your reasoning and we will seek the editor's input on an exemption. Please be assured that, once you have provided your new statement, the assessment of your exemption will not hold up the peer review process.

Additional Editor Comments (if provided):

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Does the manuscript adhere to the experimental procedures and analyses described in the Registered Report Protocol?

If the manuscript reports any deviations from the planned experimental procedures and analyses, those must be reasonable and adequately justified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

2. If the manuscript reports exploratory analyses or experimental procedures not outlined in the original Registered Report Protocol, are these reasonable, justified and methodologically sound?

A Registered Report may include valid exploratory analyses not previously outlined in the Registered Report Protocol, as long as they are described as such.

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Are the conclusions supported by the data and do they address the research question presented in the Registered Report Protocol?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. The conclusions must be drawn appropriately based on the research question(s) outlined in the Registered Report Protocol and on the data presented.

Reviewer #1: Partly

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: This manuscript presents an EEG study on the parsing of thematic roles during sentence comprehension in Standard Indonesian. It is very exciting to read such a study that expands our view beyond the languages that we read about all the time, having the potential to contribute to the diversification of psycho- and neurolinguistics. Unfortunately, there are a number of points that will need to be addressed to be able to publish this manuscript (if they can be addressed). The data and scripts were not accessible for review.

Main points:

* I will start with the most severe point, which I consider a fundamental flaw in the procedure: The visual presentation time of the words was 500 ms, with 100 ms blank screen between words. This means that each word can be processed for 600 ms (500 + 100) before the next word occurs. This already includes the 100 ms blank screen which goes in hand with visually evoked ERPs from the luminosity change. This means that (i) the epochs of -200 to 1000 ms relative to word onset include 100 ms of the previous word (-100 to -200 ms) as well as 400 ms of the next word (600 to 1000 ms) and (ii) the 500-700 ms window where a positivity is being found is entirely where the word is not being displayed anymore and also includes 100 ms of the next word. In addition, 500 ms may be a quite short presentation time, given that Indonesian words (esp. the verbs) are probably longer than average words in English. I am thus not confident whether the results are even interpretable, especially in terms of a P600-like effect.

* The presentation of results is extremely lacking. ERPs from only one electrode are shown and topographic maps are only shown for the 500-700 ms of NP2. ERP plots for all electrodes as well as topographic maps for the whole epochs (e.g. in 100 ms or 50 ms steps) need to be shown for all critical words. Otherwise, the readers have no way to assess the results in any meaningful way.

* The manuscript lacks in embedding in currrent theoretical discussions on sentence comprehension. There is no discussion of incremental processing theories from which predictions could be derived. The predictions of the experiment and the reasons why passives (in Indonesian) should lead to reanalysis responses are not clear. Actually, given the high frequency of agentless passives (which is mentioned for the first time on the second-to-last page!) could predict that disambiguation towards agents would require analysis. In addition, any discussion is missing on where the expectations for potential effects/mechanisms of thematic role assignment in incremental comprehension are coming from, particularly the agent preference (cf. works by Bornkessel-Schlesewsky and also 10.1371/journal.pone.0132819 and 10.1016/j.jml.2008.02.003). This becomes particularly apparent on p. 23 where the predictions are being presented. For the first tome a transitivity expectation is being mentioned and also that the agent is the default role is not being motivated. An integration in the current discussion on the need for more cross-linguistic studies is also missing. Overall, the impression arises that the current literature was not well integrated into the manuscript.

* There have been recent studies that exactly addressed the question that the manuscript claims to be untackled, namely the role of frequency in thematic role processing (DOIs: 10.1162/nol_a_00121, 10.1111/cogs.13340, 10.1080/23273798.2023.2250023). These studies used corpus- and Large-Language-Model-derived probabilities and surprisal measures to explore how frequency and general processing heuristics interact in several languages. The current manuscript falls severely short in it's attempts to do the same, compared with these publications. These studies (and further ones) also consistently found N400 effects for disambiguation towards patient readings, which is being interpreted as the detection of a prediction error. These previous studies, however, do not invalidate the current attempt, but they should be discussed and put in relation to the experiment reported here.

* The operationalisation of frequency unfortunately does not cover the relevant aspects for the sentence comprehension processes under investigation. First, a lot of emphasis is being put on passives being more frequent than in English. But if actives are still more frequent overall, then this does not resolve the frequency problem. In principle, based on statistical learning and experiences with their language, comprehenders would also be sensitive to a very small imbalance of structural frequencies. Second, the way of controlling for frequency by determining the verbs' biases to occur as actuves or passives does not capture the relevant aspects of frequency fo incremental role processing. When the first NP is encountered, the parser will assign a role to it (generally assumed to be the agent, based on frequency *and* the agent preference). At this point in time, when the role expectation is being formed, nothing about the verb is known. Thus, what would capture the frequency aspect better would be the surprisal or the cloze probability of an active or passive verb following th specific nouns or of the specific verbs with these specific forms following the nouns.

* What exactly is the disambiguating region in the current sentences? I first thought it would be the verb, but in the discussion when agentless passives are being mentioned, it sounds like only the combination of verb and NP2 fully disambiguates the role of NP1. If that is the case, then this could partially explain why there is no clear ERP pattern the verb. At the same time, the information in the verb (both the lexical verb and the active/passive marking) must contribute to forming an interpretation of the sentence (as it is also being described on p. 11).

Other points:

# Abstract #

The abstract mentioines that previous studies did not yet identify a unique correlate of thematic role processing. Is there any reason that there has to be an ERP that is *unique* to thematic role processing? In the context of seeing language processing as a cognitive skill, possibly relying on domain-general processes, this view does not become clear to me.

What does "syntactically more complicated" mean? Is there a more accurate way to describe passives?

The abstract also says that "the non-canonical word order is common" in Standard Indonesian. If it is common (and as it turns out in the discussion, even more frequent), why would the passive then be non-canonical? Just because the word order in English is canonically SVO, this does not have any automatic consequences for other languages. Each language should rather be treated on its own terms. Currently, Standard Indonesian is being treated as if it is just kind of like English, while it's system of actives and passives is quite different.

# Introduction #

Throughout the manuscript, it does not become very clear what level of linguistic analysis this is about. The terminology switches between thematic roles, subject and object and canonical vs. non-canonical word orders. I propose to reframe everything in terms of parsing (predicting) thematic/semantic roles, making the other distinctions more subordinate. This also applies to Table 1; is this paper about subject-object or about agent-patient (they ofter overlap, but not always)? I recommend replacing "SVO" with "agent-verb-patient" throughout the manuscript, also to avoid pressing Indonesian into the procrustean bed of English grammar terms.

The claim that there is very little research on sentences without anomalies (p. 4) seems to not be right. There are tons of studies with non-violated sentences.

On p.4 at the discussion of example 1, one could also argue that "the man" is the agent of the main clause verb (admitted) and it is rather the realisation that there is a relative clause that leads to processing difficulties. Could that be another possible explanation for this example?

The case marking in the glosses for example 2 needs to be on the determiner or on the whole NP, for that matter, but not on the noun.

Why is the sentence in 5a described as canonical (p. 10) when it is actually a less frequent structure than 5b?

The abbreviation SI needs to be introduced in the main text before being used.

A discussion of the functional role of P600 and N400 in sentence comprehension would be useful to situate the current experiment in the literature and to interpret the findings. Why is it expected that disambiguation towards a passive should elicit a P600? Relevant papers are (among others): 10.1080/01690960902965951, 10.1111/cogs.12461, 10.3389/fnhum.2013.00758, 10.1111/psyp.13351, 10.1371/journal.pone.0257430, 10.1097/00001756-200110290-00048, 10.3389/fpsyg.2019.00298

# Method #

It is hard to believe that the largest study in the literature on sentence comprehension has 25 participants. Haupt et al. (2008), for example had 40 participants (10.1016/j.jml.2008.02.003).

In the discussion about what baseline to use, the paper by Alday on "How much baseline correction do we need in ERP research?" (10.1111/psyp.13451) is missing. Not only because it is more recent but also because it provides a convincing solution for the problem.

# Materials #

The use of inanimate NPs would have been beneficial, too, because in combination with the verb, these could have been easier to disambiguate.

What are the units in Table 4?

Figure 1 should connect the dots of each verb in the Active and Passive columns so that the reader can see which points on each side belong together. In ggpplot, this can be done with geom_path().

# Procedure #

The comprehension questions should have been randomly distributed over all trials. By just presenting them every 3-8 trials, participants knew that after they answered a comprehension question, they would not have to answer another one for at least 2 more trials. How could that have affected their comprehension strategies? What if they zoned out for the next 2 sentences?

Asking comprehension questions only for a part of the critical trials also means that only for a part of those there is information about comprehension success.

The comprehension data should be presented, at least descriptively.

p. 19: Was the average of the mastoids used as the reference electrode?

Restricting the removal of independent components to only 4, when there are 64 electrodes, appears to be unnecessarily conservative.

# EEG recording and preprocessing #

The Bad Channels (2) criterion says that every participant with a cluster of bad electrodes would be excluded. Just as food for thought, would this also have been the case if in the last trial of the experiment, such a cluster emerges that all the previous data would be excluded? Some kind of time criterion for this would have been good. (I know that it didn't apply here, but for future studies.)

# Statistical analysis #

I know that this is a registered report, so the analyses were approved beforehand. However, cluster-based permutation approaches by virtue of being univariate tests don't allow to include (a) by-item variation, which is usually also high in language comprehension tasks (just look at the spread of your acceptability ratings) and (b) frequency as a predictor. Regression-based approaches would allow to do that.

Even though the frequency classes were matched for active and passive, there is still considerable spread as can be seen in Figures 1 and 2. Sassenhagen and Alday show that controlling for nuisance variables by running a stastical significance test beforehand actually does not achieve this goal (10.1016/j.bandl.2016.08.001). As an attempt to *partially* cover this problem, the data for each participant could be split into 2 bins (based on Figure 2) using the uvar attribute in FieldTrip's permutation tests (https://www.fieldtriptoolbox.org/faq/how_can_i_use_the_ivar_uvar_wvar_and_cvar_options_to_precisely_control_the_permutations/).

# Predictions #

Prediction 3 (a lack of ERP correlates) cannot be tested in the current framework because frequentist statistics cannot distinguish between genuine null effects and a lack of statistical power.

p. 24: While the study by Jackson et al. is very similar to the current study, other studies such as 10.1016/j.jml.2008.02.003 and 10.1371/journal.pone.0132819 are also very similar.

What does it mean that "verbs in Basque and Japanese provide no additional thematic information" (p. 24) given that the verbs provide the event the sentence is being about and also determine the argument structure, so basically disambiguate the roles if it has not been done?

# Results #

On p. 26, it is mentioned that a difference is not significant on the verb (p = .087) but on p. 22 it is mentioned that the threshold for signficance is set to p < .1.

Reviewer #2: The authors may add a conclusion section.

"...P600 time window which are maximally..." in line four, page 15 should probably be " ... P600 time window which is maximally...

"... the participants are similar to ..." in line seven, page 15 should probably be "... the participants is similar to ..."

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean? ). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy .

Reviewer #1: No

Reviewer #2: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/ . PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org . Please note that Supporting Information files do not need this step.

<p>Registered Report: Neural correlates of thematic role assignment for passives in Standard Indonesian

PONE-D-24-16699R1

Dear Dr. Jap,

Thank you for submitting the revised version of your manuscript. After carefully reading the manuscript myself, I am pleased to inform you that your manuscript has been accepted for publication without the need for an additional round of review. This decision is based on the following reasons:

1. Although Reviewer 1 previously expressed concerns regarding the methodology, I recognize the importance of adhering to the pre-registered protocol. You have made every effort to follow the approved protocol, and any deviations have been clearly documented.

2. Your revised manuscript provides adequate justification for the chosen methodological details.

3. Reviewer 2 expressed satisfaction with your manuscript.

Your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager®  and clicking the ‘Update My Information' link at the top of the page. If you have any questions relating to publication charges, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Yiu-Kei Tsang

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments: