PONE-D-25-05136As Time Goes By: Long-term Retention of Economics SkillsPLOS ONE

Dear Dr. McKee,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Both reviewers believed that work on this issue is valuable, but have several comments that you will either need to address or refute in your response.  In addition, you will need to discuss more carefully your sample and any factors that might affect the external validity of your findings.  For example, you identify students for follow-up surveys based on students in courses with multiple students who participated in the initial assessment.  Are all these classes economics courses?  If not, please clarify.  If so, please discuss how this may bias your results.  Furthermore, there is little information on how the courses are taught and what is emphasized in the courses.  Are multiple choice exams used in the courses? Do the courses stress the use of graphs and equations or is the intro course a more issues-oriented course?  More broadly, how does the fact that there is only one instructor teaching these classes affect external validity?  Could this be construed as a study of the impact of one instructor's pedagogical choices?

Please submit your revised manuscript by May 16 2025 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Ann L. Owen

Academic Editor

PLOS ONE

Journal Requirements:

1. When submitting your revision, we need you to address these additional requirements.

Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please provide additional details regarding participant consent. In the ethics statement in the Methods and online submission information, please ensure that you have specified (1) whether consent was informed and (2) what type you obtained (for instance, written or verbal, and if verbal, how it was documented and witnessed). If your study included minors, state whether you obtained consent from parents or guardians. If the need for consent was waived by the ethics committee, please include this information.

If you are reporting a retrospective study of medical records or archived samples, please ensure that you have discussed whether all data were fully anonymized before you accessed them and/or whether the IRB or ethics committee waived the requirement for informed consent. If patients provided informed written consent to have data from their medical records used in research, please include this information.

3. Thank you for stating the following financial disclosure: [This paper is based upon work supported by the National Science Foundation under Grant 2021094, and any opinions, findings, and conclusions or recommendations expressed in this paper are those of the authors and do not necessarily reflect the views of the National Science Foundation.]. 

Please state what role the funders took in the study.  If the funders had no role, please state: ""The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.""

If this statement is not correct you must amend it as needed.

Please include this amended Role of Funder statement in your cover letter; we will change the online submission form on your behalf.

4. When completing the data availability statement of the submission form, you indicated that you will make your data available on acceptance. We strongly recommend all authors decide on a data sharing plan before acceptance, as the process can be lengthy and hold up publication timelines. Please note that, though access restrictions are acceptable now, your entire data will need to be made freely accessible if your manuscript is accepted for publication. This policy applies to all data except where public deposition would breach compliance with the protocol approved by your research ethics board. If you are unable to adhere to our open data policy, please kindly revise your statement to explain your reasoning and we will seek the editor's input on an exemption. Please be assured that, once you have provided your new statement, the assessment of your exemption will not hold up the peer review process.

5. Your ethics statement should only appear in the Methods section of your manuscript. If your ethics statement is written in any section besides the Methods, please move it to the Methods section and delete it from any other section. Please ensure that your ethics statement is included in your manuscript, as the ethics statement entered into the online submission form will not be published alongside your manuscript.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: 1. The literature review seems to not consider economics articles on retention. Juttler (2020, Feb) in this journal or Cosgrove and Olitsky (SEJ, 2015) which although for high school is still relevant.

2. The next to last paragraph says that a limitation of other studies is not considering the role of some characteristics. You say this is helpful. Expand on why.

3. Target sample is misleading. Isn't the entire class the target? The description of the samples was difficult to follow.

4. Do you know how much time the spent the first time the completed the assessment?

5. Better explain the normalization. what are the new mean and s.d.?

6. You say MESA scores declined substantially. Substantial is subjective. The percentage decline suggests a 1.8 item decline on average. My interpretation is that is substantial retention.

7. Is it retention? Suppose only a 3 item quiz and initially the student gets 2 of 3 correct. On the retake they get 1 of 3 correct, but the one they got correct is the one they initially got wrong. You would say they retained half of what the initially knew, but in fact they retained nothing.

8. Is IESA asking about MRS and MRTS? Should we expect students to remember that? In micro you learn of sunk costs, that can come up in life (same with supply and demand).

9. Do you have enough observations to report a regression for Spring 2023 of the intermediate sample?

10. The entire discussion section is your unsubstantiated opinion of why results look that way. For example, you have no idea why female students have different scores. I would argue for deleting this section.

Reviewer #2: The aim of the study is to capture what students retain in the very long run AND what determines that retention.

The paper provides evidence that retention of course skills is associated with some demographic characteristics and prior preparation for the course. It goes beyond existing studies on long-term learning by exploring such factors.

The study is well executed but there is a need to provide a more thorough explanation in some places of the limits of the measures used. More details are provided below.

Good context is provided 1) on some factors influencing short-term retention in economics and other disciplines, and 2) on long term learning from literature in disciplines other than economics, where such literature is sparse. The dearth of studies examining long-term learning in economics underscores the value of this study.

The samples used pool data from three cohorts for each course, giving a reasonable sample sizes for each of the courses (with fixed effects included for each cohort).

The study doesn’t raise any ethical concerns: it is reported as being IRB exempt, student consent was obtained in advance and data were de-identified.

P3: line 77-79: Be explicit about how the differences in course teaching method and modality are captured in the cohort fixed effect

P3: line 89 and P7: line 180: Can you say more about why the reader should be confident that 30 (or 31 for intermediate) multiple choice questions are a reasonable measure of student learning?

P3: line 99 (and equivalent for intermediate): What can you say about the external validity of the findings (beyond your own institution)?

P6: lines 124-126: Provide further explanation about how information on “courses that used their economic knowledge” and jobs where “they applied their economic knowledge” was gathered. Was this based on the student’s own interpretation of what using/applying economic knowledge meant? Was economic knowledge defined, or confined specifically to microeconomic knowledge? The specifics of the additional course might matter – e.g. upper level applied micro course.

The results are presented separately for intro and intermediate micro, implying that these samples are separate. Might some of the students in the (earlier) intro cohorts also be included in the intermediate sample? If so, does that/how might that matter?

P10: In discussing the results, how do you distinguish between retention of learning from intro micro and *re-learning* concepts in another economics course?

P13: Line 320 and 325: Do you have evidence to support the purported gender difference in study habits? If so, provide detail. If not, it would be better to remove purely speculative material.

P13: Line 342: This depends on what is being captured in the “jobs that apply economics concepts”. (See earlier comment about defining this more fully.)

Some of these questions are likely not answerable in a definitive way: the complexities in interpreting the results should be explicitly acknowledged.

The variables used are by their nature imperfect proxies. The measures used seem reasonable – but the paper should discuss the measurement complexities. (e.g. retention v relearning, what is “using economic knowledge” in a job, what about “using economic knowledge” in other parts of life etc.?)

The paper tackles a very difficult question – and adds to our knowledge.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

As Time Goes By: Long-term Retention of Economics Skills

PONE-D-25-05136R1

Dear Dr. McKee,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager® and clicking the ‘Update My Information' link at the top of the page. For questions related to billing, please contact billing support.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Ann L. Owen

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewer #1:

Reviewer #2:

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Thank you for addressing my concerns. Any remaining concerns are beyond the scope of this manuscript.

Reviewer #2: (No Response)

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

**********