PONE-D-24-48310Measuring individual semantic networks: A simulation studyPLOS ONE

Dear Dr. Aeschbach,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by May 08 2025 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Bruno Alejandro Mesz, Ph.D.

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at 

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and 

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please note that funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form. Please remove any funding-related text from the manuscript. 

3. We note that you have indicated that there are restrictions to data sharing for this study. For studies involving human research participant data or other sensitive data, we encourage authors to share de-identified or anonymized data. However, when data cannot be publicly shared for ethical reasons, we allow authors to make their data sets available upon request. For information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions. 

Before we proceed with your manuscript, please address the following prompts:

a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially identifying or sensitive patient information, data are owned by a third-party organization, etc.) and who has imposed them (e.g., a Research Ethics Committee or Institutional Review Board, etc.). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent.

b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. Please see http://www.bmj.com/content/340/bmj.c181.long for guidelines on how to de-identify and prepare clinical data for publication. For a list of recommended repositories, please see https://journals.plos.org/plosone/s/recommended-repositories. You also have the option of uploading the data as Supporting Information files, but we would recommend depositing data directly to a data repository if possible.

Please update your Data Availability statement in the submission form accordingly.

4. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Additional Editor Comments:

Two experts in the field have carefully reviewed the manuscript entitled “Measuring individual semantic networks: A simulation study“. You can find their comments below. They both had very positive comments on the manuscript but also requested clarifications and elaborations of some parts. 

In light of these reviews, I am requesting a minor revision and resubmission, in which you will need to respond to each point in each review. 

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Reviewer background: statistics and linguistics

Summary: The work is a simulation study that aims to address limitations in measuring individual differences in semantic networks. They focus on constructing individual semantic networks from two behavioral paradigms: free associations and relatedness judgement tasks. Their goal is to identify certain design configurations of these semantic networks that makes comparisons between individual networks accurate and meaningful. They start by building individual ground-truth networks from a vector space model, and then they generate behavioral data from which they construct many different configurations of inferred individual semantic networks. They then compare the inferred networks with the ground truth networks to assess bias, resolution, and generalizability of network measurements. These criteria allow the researchers to identify the design configurations in which it is suitable/unsuitable to compare individual semantic networks in a meaningful way.

Overall impression: The paper is well-written and easy to understand. It is a very useful study for researchers who work with individual semantic networks, as it can contribute to more methodologically sound investigations. The results are interesting and can easily be applied. Overall it is a great paper that I think will be of great interest to a specific group of researchers.

Suggestions: Some parts of the introduction could benefit from more details. Also, some of the methodology is somewhat unclear and could benefit from more detailed explanations. I provide more specific suggestions below.

Lines 25-26: “Recent work suggests, however, that this approach may be subject to significant limitations [9–12].” What kinds of limitations?

Lines 61-71: I believe this paragraph could benefit from significant elaboration. This paragraph is essentially the main motivation for the study. I think you should offer more details about the limitations you aim to investigate. Describe the biases, how they emerge, the types of problems they can cause, and give examples. Do the same for resolution and generalizability. By elaborating on these three potential limitations, the reader is more prepared to understand the analyses that you do in the simulation study.

Section “Generating individual ground-truth networks”: You describe how the networks are created, but it could be helpful to the reader to elaborate on the meaning of these ground-truth networks. What do they represent? Does each network represent a hypothetical ground-truth mental-lexicon of an individual person? (This is how I understood it). It could be helpful to provide a summary table of the statistics of the 250 networks (nodes, edges, density, clustering).

Section “Simulating behavioral data”: This part is somewhat unclear to me. Where are the edge weights coming from in equations 1 and 2? Are they coming from the fully connected vector space model ground-truth network? This is what I understood. Also, here it could be helpful to discuss what this data represents. It represents essentially “sampling” from the ground-truth mental lexicons of an individual person, and that sampling is done by asking questions in a behavioral experiment. Therefore the methods you describe to generate these data (free associations and relatedness judgement) are simulations of this sampling (This is how I understand it).

Section “Design configurations”: At the end of this section, I think it would be useful to summarize the number of total configurations you have, which you don’t do until later, in the paragraph immediately before the Results. Here, a summary table could be helpful as well.

Section “Evaluating network recovery”: This part is unclear to me. How are the comparisons between inferred networks and ground-truth networks made? Is one inferred network compared to one ground-truth network? What exactly are M and M-hat? What measures are they? Are they calculated on individual networks, one inferred network of a given configuration and one ground truth sub-network reduced to match that configuration?

Nice paper!!

Reviewer #2: Summary

- This research investigates the methodological challenges in measuring individual semantic networks through behavioural paradigms (of free associations and relatedness judgments). Using comprehensive recovery simulations, the authors evaluate how different study design parameters affect the bias, resolution and generalisability of semantic network measurements. The study certainly makes a significant contribution by systematically assessing the conditions under which semantic networks can be reliably inferred from behavioural data.

Strengths

- The methodology used is rigorous, examining multiple design parameters, with any shortcomings that become apparent reading through being clearly addressed in the limitations and future work. There are also practical implications for future empirical work, making this a useful reference work in the field.

- The evaluation of network measures is comprehensive, although given the part focus on broad, narrow and mixed cue sets, it is regrettable as a reader that remaining matters such as network diameter could not be included in this work.

- The insights on the generalisability of these network measures is a particularly impactful consideration, especially to identify the limitations of previous works.

Issues

- Page 4, Section “Generating individualized ground-truth networks” – There should be a simple justification/validation for the choice of parameter range of `p` and `r` to make obvious there are no complex underlying reasons.

- Page 5, Given that it is the basis of the synthesised responses and further experiments, the cognitive plausibility of the retrieval mechanisms used in the simulations could benefit from stronger theoretical grounding, or perhaps elaboration of the references to justify it. However, this is considered in the discussion, especially mentioning the potentially disparate nature of individual semantic retrieval processes. It could be deemed a necessary simplicity for, as written controllability, when considering that including methodology from Human Modelling/Computational Rationality fields would make this work overly complex.

- Page 9, Figure 3. Resolution caption has an error (assumed typographical) in the range for teal tiles “0 ≥ r < .5” -> “0 <= r < .5”.

Nitpicks (These do not need to be addressed)

- The discussion could address potential limitations of using fastText embeddings as ground truth.

- There was one sentence thought a little vague: “the networks’ average strength and clustering were `largely` independent” perhaps could be more concretely communicated, but it’s easily understood so no need to change.

PLOS ONE 7 Criteria

- Original Research. ✓

- Results not published elsewhere. ✓

- The simulation methodology is rigorous and well-documented. Statistical analyses are appropriate. ✓

- Conclusions supported by data. ✓

- The writing is clear and professional throughout. ✓

- No ethical concerns as this is a simulation study. ✓

- GitHub repository with code and data links for reproduction. ✓

An exceptionally well written paper. The suggested revisions are only to strengthen an already solid piece of work.

Overall recommendation: Accept with Minor Revisions

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Katherine Elizabeth Abramski

Reviewer #2: Yes: Owen G.W. Saunders

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Measuring individual semantic networks: A simulation study

PONE-D-24-48310R1

Dear Dr. Aeschbach,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager® and clicking the ‘Update My Information' link at the top of the page. If you have any questions relating to publication charges, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Bruno Alejandro Mesz, Ph.D.

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #2: (No Response)

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #2: No

**********