PONE-D-21-34137Scientific and Engineering Practices Aligned with NGSS Standards in the Performance of Secondary Stage Physics TeachersPLOS ONE

Dear Dr. Alsalamat,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Feb 20 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Yiming Tang, Ph.D.

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at 

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and 

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. In your Data Availability statement, you have not specified where the minimal data set underlying the results described in your manuscript can be found. PLOS defines a study's minimal data set as the underlying data used to reach the conclusions drawn in the manuscript and any additional data required to replicate the reported study findings in their entirety. All PLOS journals require that the minimal data set be made fully available. For more information about our data policy, please see http://journals.plos.org/plosone/s/data-availability.

"Upon re-submitting your revised manuscript, please upload your study’s minimal underlying data set as either Supporting Information files or to a stable, public repository and include the relevant URLs, DOIs, or accession numbers within your revised cover letter. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories. Any potentially identifying patient information must be fully anonymized.

Important: If there are ethical or legal restrictions to sharing your data publicly, please explain these restrictions in detail. Please see our guidelines for more information on what we consider unacceptable restrictions to publicly sharing data: http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions. Note that it is not acceptable for the authors to be the sole named individuals responsible for ensuring data access.

We will update your Data Availability statement to reflect the information you provide in your cover letter.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: No

Reviewer #2: Partly

Reviewer #3: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: No

Reviewer #2: Yes

Reviewer #3: No

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: Yes

Reviewer #3: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: No

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: This article employed a survey methodology to examine self-reporting of adherence to Next Generation Science Standards (NGSS) by secondary education physics teachers in Saudi Arabia. While the authors demonstrate a clear interest in furthering science education by exploring the NGSS, a variety of issues prevent the reviewer from recommending this article for publication. These include:

Overview/Statement of the Problem/Significance

- The sentence "Many institutions catered for developing the field of teaching and learning

science, relying on such standards" does not have a clear meaning and should be rewritten to clarify the subject, verb, and object of the sentence.

- Early in the manuscript, the authors need to clarify that NGSS stands for "Next Generation Science Standards."

- The introductory paragraph consists mainly of the authors' assertions without a strong evidentiary base. If an overview of the standards is warranted here, it should be grounded in evidence-based claims from reputable sources.

- The citation for the Next Generation Science Standards documents does not match the information provided on the NGSS website itself regarding copyright and trademark: https://www.nextgenscience.org/trademark-and-copyright

- Assertions such as "Because students’ objectives for learning sciences in general and physics in particular have changed,

science teachers of all levels need to adapt themselves to these new standards in order to achieve an actual change in

their teaching practices. Therefore, educators see that teachers’ qualifying programs should be developed to make

teachers concentrate more on actual practice and to help them amend their practices." (p. 2) appear throughout the manuscript and lack references and citations to support them. Because of this pattern, many of the premises that follow from these assumptions appear unfounded.

- Despite claims by the authors, a goal of the NGSS "to make students behave and practice in a scholarly manner to gain skills of engineering design which enable them to conduct research and to solve problems they encounter throughout science study or in their actual life" appears nowhere in the standards documents. For reference: https://www.nextgenscience.org/faqs

- The Eight Science and Engineering practices described on page 3 are written in gender-biased language (deictically referring to "student" as "he/him/his." Further, typeface choices such as setting certain words in boldface are distracting to the reviewer.

Review of Literature

- There is no clear section that reviews literature; therefore, it is unclear how the current manuscript is building on or extending prior work in this field.

Methodology

- While the study refers to a "mixed approach" to research, the author should clarify whether this was a mixed methods design (and if so, how), or whether there were simply multiple methods or multiple sources of data.

- Consent procedures should specify whether oral or written consent was provided by participants.

- Study limitations are not discussed with sufficient detail. A much clearer explication of the sampling frame is necessary to evaluate the procedures of random and purposive selection. A check for validity was referenced, although there is no clear discussion of whether this is for construct, content, face, or criterion validity. Further, despite the mentioning of an exploratory survey, no statistical calculations or data are provided to support the authors claims of validity and reliability. In the case of qualitative trustworthiness, there is no mention whatsoever of credibility, transferability, dependability, confirmability, or authenticity.

- There are no clearly grounded definitions for the metric of "degree of availability," and thus, it appears to be an arbitrary construction of the authors. If any rating scale is to be used, it should be one that is grounded in prior evidence and tested for validity and reliability through established statistical analysis.

- There is no clear discussion of how qualitative interview data was analyzed, nor of how the conclusions were reached.

- Given the above statistical issues, the ANOVA does not appear to offer valid or reliable results.

With a much clearer discussion of the overall study design, sampling methodology, calculations of validity and reliability, qualitative trustworthiness, and evidence-based definition of the term "degree of availability," the study may be eventually become suitable for publication. However, in its current state, this reviewer believes it does not meet the standards of the journal.

Reviewer #2: 1. PLOS uses “Vancouver” style, as outlined in the ICMJE sample references. Please modify it.

2. I suggest that the author move the section "Study limits and limitations" to the paragraph "Conclusion".

3. The source of the basis for the evaluation criteria of the questionnaire explored?

4. Do these eight open-ended questions have relevant literature basis?

5. The problem of demographic variables was not seen in the research.

6. The teacher’s seniority is a continuous variable. Why the F test is used in this study, please add the author's explanation.

Reviewer #3: Abstract

1. There is no problem statement.

2. It is not stated what questionnaire was used.

3. The implications of the findings are not clearly stated in the abstract

Introduction

1. NGSS is the first word mentioned on page 1, but it is an acronym and it is not shown what it stands for.

2. The introduction and theoretical background is rather long. The format of having subtopics “significance of the study, terms of the study are from the dissertation format.

3. There are many standards mentioned such as NSTA, NSF, AAAS but what is the purpose for and how is it different from the NCSS? What is the change and adaptation that needs to be done? This needs to be more specific.

4. There were 8 practices outlined on page 3. These practices should be refenced. And literature can be added to build upon it as it seems to form the theoretical background of the study.

5. On page 4: “Educators and those concerned with NGSS see that physics teachers should possess such standards in general” Do teachers “possess standards?

6. In the research questions there is mention of degree of availability ie “What is the degree of availability pertaining scientific and engineering practices that cope with NGSS in the performance of physics teachers at the secondary stage? What exactly does this term mean?

7. Is there any hypothesis for the study.

8. On page 5, it was stated that this is a mixed method approach. Is there a refence for the framework of this mixed method study?

9. Who exactly are the participants? If they are teachers, why do they need supervisors? Or are these preservice teachers?

10. On page 6, it was written: “After determining objectives of the questionnaire, the researcher checked the official website of NGSS (https:www.nextgenescience.org), modern education literatures, besides surveying scientific and engineering practices relevant to teachers of physics at the secondary stage as presented in (Malkawi & Rababah, 2018 ; Hang & Srisawasdi, 2021 ; Alshyab, 2019 ; Aboathrah, 2019 ; Brownstein & Horvath, 2016 ; Qablan, 2016 ; Fulcher, 2014 ; Stuart et al., 2021 ; Duschl & Bybee, 2014 ; Kawasaki, 2015), and ..

a. The writing of in-text citation above is incorrect.

b. The validation of the instrument is also not clear.

11. The presentation of the findings is not well done. eg. on page 8: The item of teachers’ scientific and engineering practices might be elaborated on as follows:

•Asking questions and defining problems

Table 2. presents results pertaining this type of practice.

12. The second RQ on page 19: Does that degree of availability in performance of those teachers vary in

accordance with qualification, or years of experience?", was analysed using t-tests. However, the variables seem to be of nominal and ration data respectively. Is t-test the correct means of analysis.

13. What are the implications to the study. And the recommendations should be made more specifically.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Earl Aguilera

Reviewer #2: No

Reviewer #3: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Attachments

Attachment

Submitted filename: PONE-D-21-34137_reviewer D.pdf

PONE-D-21-34137R1Scientific and engineering practices aligned with the NGSS in the performance of secondary stage physics teachersPLOS ONE

Dear Dr. Alsalamat,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Jun 13 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Yiming Tang, Ph.D.

Academic Editor

PLOS ONE

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #2: All comments have been addressed

Reviewer #3: (No Response)

Reviewer #4: All comments have been addressed

Reviewer #5: (No Response)

Reviewer #6: (No Response)

Reviewer #7: All comments have been addressed

Reviewer #8: (No Response)

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #2: Yes

Reviewer #3: No

Reviewer #4: Yes

Reviewer #5: (No Response)

Reviewer #6: No

Reviewer #7: Yes

Reviewer #8: Partly

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: Yes

Reviewer #5: (No Response)

Reviewer #6: No

Reviewer #7: I Don't Know

Reviewer #8: (No Response)

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: No

Reviewer #5: No

Reviewer #6: Yes

Reviewer #7: Yes

Reviewer #8: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #2: Yes

Reviewer #3: No

Reviewer #4: Yes

Reviewer #5: Yes

Reviewer #6: No

Reviewer #7: Yes

Reviewer #8: No

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #2: The content has been amended to better reflect the work conducted.

The author has made a clear description of the source of the questionnaire.

Reviewer #3: 1. Abstract does not indicate the need for the study.

2. On page 2. The problem statement is not clear. You have mentioned that science teachers of all levels need to adapt to the new standards. However, you have not shown evidence on why this is needed. So, what is the problem? Is it a difficulty in designing appropriate training? Or do we need to have a change of attitude? This is not clear.

3. On page 3, you have stated the practices in NGSS.

a. However, it is not clear why and how you have applied these practices. The practices are the standards the students need to achieve, so is there a difference with the teaching standards or the standards for teachers?

b. Similarly, in the problem statement on page 4, it is not clear if the standards are for teachers or students,

4. On page 4, first paragraph, it is stated: “Because previous studies didn’t directly…However, there was no reference to the studies.

5. On page 4, the research questions refer to the “degree of availability”. This is not defined. The research questions are not clear and do not seem to be aligned with the statement of the problem.

6. In the significance of the study, it was mentioned that methods of assessment of teachers could be beneficial for supervisors. However, does the study allow for this as assessment is not being done.

7. On page 5, the terms of the study are stated. This reads like a thesis, and not like a journal article.

8. The study was limited to a sample from Taif city. What is the rationale for selecting teachers in Taif, and not other cities, or even rural areas?

9. The NGSS website was mentioned several times. Why was the website used and not an official document?

10. The questionnaire that was used for the study (page 6), is it a new questionnaire designed for the study or one which is already available?

11. When interviews were used, it would be valuable to see the interview protocols and what was asked. It was also not clear how the interview data was analysed. Qualitative data needs to be shown with evidences such as transcripts of the interview. This seemed to be lacking.

12. The limitations of the study needs to be mentioned. This would enable suggestions for futre studies.

The article needs to be proofread to ensure the language is of quality

Reviewer #4: Congratulations, I have seen you have performed all the changes requested in the previous review. My only question is "where is the data availability statement?" I haven't been able to find it. I highly suggest you to use an open repository for your data such as OSF or similar to upload data from your research and making it available without restrictions. You should include this information in your manuscript as indicated by PLOS policies.

Reviewer #5: Title:

This title is vague. I could not identify the research objective through this title; is it an analysis of teachers' observations, an analysis of teacher programs, or just a survey study?

I think the study deals with the extent to which teachers adopt scientific and engineering practices in their teaching from their point of view.

Aligning means that the study analyzed teacher programs based on teachers' actual practices. Therefore, this study has nothing to do with aligning

Language:

All manuscript needs proof reading.

Abstract:

The method is not clear in the abstract.

Introduction:

There are no paragraphs related to the objective of the study

Questions:

The method is not clear in the abstract

Method:

Description of the instruments, the way of calculated the values related to the criterion of judgement such as intervals and therefore, this affected the assignment of the levels in all tables.

Results:

The criterion of judgement was as follows

degree of accessibility?

I think it is the level or degree of practice from their point of view.

Discussion:

The discussion is too short. You should use a separate section for the discussion and make it richer and more organized. Too short and does not do justice to the study's richness.

References:

Minor corrections are needed for the references.

Reviewer #6: Comment 1: I am sorry to say that this manuscript is confusing and, at times, was disjointed and unclear. For example, after thoroughly reading this manuscript more than once, I still do not know if the items of the questionnaire (indicators) were constructed by the author or taken from sources. Nevertheless, I hope my comments make more sense by explicitly stating my understanding of this work.

Comment 2: Simply stated, the author of this study, based on the NGSS website and several studies, identified 8 variables, each corresponding to a standard of teaching science to secondary school students. The author wrote (I assume) several Likert-scale statements (items) to, presumably, represent or address or measure each variable. Variables 1 through 8 are measured by 5, 6, 4, 7, 5, 7, 5, 5 items, respectively. Therefore, this is an instrument that measures 8 variables based on 44 Likert 5-point scale items. The author calls the variables "practices" and the items "indicators."

Comment 3: Then, an exploratory sample of 15 physics teachers was used to provide validity and reliability statistics. It is unclear if these teachers were also part of the study sample.

Comment 4: Then, this instrument was completed by 49 randomly selected secondary school teachers. It is not clear how they were randomly selected. How many teachers were asked, or how many questionnaires were sent? How many teachers said no, or what was the questionnaire return rate? The mean and standard deviation were calculated to represent each variable based on each variable's item scores. In addition, criteria based on the range of mean were defined: Very weak (<1.80), very weak (1.8-2.60), medium (2.60-3.40), high (3.40-4.20), very high (>4.20). The author did not provide justification for these cut-off scores.

Comment 5: Then, each teacher's total score on all 8 variables in this instrument was used to identify differences between teachers who hold graduate degrees and those who hold bachelor's degrees. A simple t-test of independent samples was used to declare significant differences between the two groups.

Comment 6: Then, the teachers were divided into three groups based on their reported years of experience (less than 5 years, 5-10 years, and more than 10 years), and a one-way ANOVA was used to compare the mean scores on this instrument of the three experience groups. No justification is given for grouping teachers' experience based on 5 years intervals. Are there other grouping intervals that might have produced significant differences? If a linear relationship exists between years of experience and scores on the instrument, examining the correlation between the two is more beneficial.

Comment 7: Finally, 6 teachers' supervisors were picked and interviewed to answer 8 open-ended questions about the presence of the scientific and engineering practices (variables) in the performance of the physics teachers. The supervisors' responses to the open-ended questions were compared with the teachers' responses on the instrument. Again, these supervisors were specifically selected, implying they know the teachers, which means no anonymity. Is this the case, and if so, why are these details not explicitly stated?

Comment 8: Clearly, this instrument is central to this study, and therefore, all the rules regarding test construction and statistical matrices apply. Each of the 8 variables/practices is a construct defined by the item loadings. However, these item loadings are hypothesized, not tested. While the author appears to have tried to provide the statistical rationale for using this instrument, it is far from sufficient. I believe that the major flaw in this study is the instrument itself. This instrument needs to be validated and its statistical properties identified. The first step is to administer this instrument to some 120 science teachers (using the 15 subjects per variable rule of thumb), conduct a thorough reliability and validity analysis, conduct factor analysis to determine item loading and appropriateness of items and variables, and revise as necessary. Only then may this instrument be used to make such inferences.

Comment 9: The structure and the presentation of the manuscript are much more consistent with what is normally required for a thesis or a dissertation than for research articles for publication. PLOS ONE provides what it deems to be acceptable structure and sequence. I recommend the following sequence: Introduction, Methods (procedure, participants, analysis), Results, Discussion, Limitations, and Conclusions.

Comment 10: There are too many punctuation and grammatical errors to list. Mixed tense in one paragraph is confusing (for example, 2nd paragraph on page 2 of the manuscript uses past tense at the beginning and then switches to present tense in the last sentence leaving the impression that the first several sentences are about elements that are no longer applicable, which is not the case). Also, serval sentences would benefit from re-writing them more clearly (for example, 2nd line on page 2, "teaching appeared such as…," 2nd paragraph on page 2, "need to adapt themselves to these new…." 3rd paragraph on page 2, "they encounter while studying science or in real life." And "i.e. whoever teaches science, in addition to skills of the engineer who solves problems." Another issue is the overuse of quotation marks (for example, 3rd paragraph on page 2, "can be described as "cognitive, discursive, and social activities" that can be …" What is so special about those words that dictate the use of quotation marks? If these words are special and verbatim from a reference, then a reference should immediately follow the quotation. I recommend proofreading this manuscript and using a professional copy editor.

Comment 11: Many paragraphs before methodology are unnecessary or can be trimmed to one or two sentences. For example, the study problem and the significance of the study can be easily combined, shortened, and embedded at the end of the introduction. The terms of the study can be summarized and written more concisely. There is no reason to justify using the qualitative method when one has qualitative data. The qualitative method used and how one interprets the finding are important, but not the justification unless it is unusual to use such methods.

Comment 12: Reading paragraph 4 on pages 2 and 3 gives the impression that the author constructed the eight variables (practices). But reading the terms of study scientific and engineering practices on page 4 reveals the author adopted these variables from the listed sources. This is confusing, unnecessary, repetitive, and internally inconsistent. Therefore, these two paragraphs need to be summarized and combined into one.

Comment 13: Everything before the methodology needs to be re-written under the introduction heading with more focus on what is being studied, why it is being studied, and what others have said about what is being studied. I believe the whole section needs to be well thought out and re-written.

Comment 14: The presentation of results needs improvements. The table should be formatted differently. The table columns should be mean (SD), degree of presence should include the range of the mean and the designation, and rank.

Comment 15: It is unclear why a table for each practice (variable) is shown. Such a presentation might be appropriate if the work addresses the validation and the statistical properties of an instrument, but not when the work is focused on the mean of each variable (practice). Table 1 would be sufficient for this type of work where means are compared.

Comment 16: Also, the author does not seem to distinguish between results and discussion. The results section should be devoted to what the author finds. The discussion section should discuss the results and compare them to what others have found.

Comment 17: The limitations section needs to be well thought out and more clearly written and articulated.

Reviewer #7: 1. Introduction and literature review : first paragraph (1-4) who? “ explained that education ….), you need to write (1-4) at the end of the sentence.

2. The study limits section to be transferred to the end of discussion

3. The results and discussion parts are mixed.

4. Table 3 showed that general mean stated as weak while in the text stated as medium.

Reviewer #8: (No Response)

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #2: No

Reviewer #3: No

Reviewer #4: No

Reviewer #5: No

Reviewer #6: Yes: Dr. Ali M. AL-Asadi

Reviewer #7: No

Reviewer #8: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Attachments

Attachment

Submitted filename: PONE-D-21-34137_reviewer.pdf with comments.pdf

Attachment

Submitted filename: Coment on PLOS ONE review research.pdf

Attachment

Submitted filename: renamed_5e89d.docx

Attachment

Submitted filename: Reviewer reports.docx

PONE-D-21-34137R2Scientific and engineering practices aligned with the NGSS in the performance of secondary stage physics teachersPLOS ONE

Dear Dr. Alsalamat,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Aug 01 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Yiming Tang, Ph.D.

Academic Editor

PLOS ONE

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #4: All comments have been addressed

Reviewer #6: (No Response)

Reviewer #7: (No Response)

Reviewer #8: (No Response)

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #4: Yes

Reviewer #6: No

Reviewer #7: Yes

Reviewer #8: Partly

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #4: Yes

Reviewer #6: No

Reviewer #7: I Don't Know

Reviewer #8: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #4: No

Reviewer #6: Yes

Reviewer #7: Yes

Reviewer #8: (No Response)

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #4: Yes

Reviewer #6: Yes

Reviewer #7: No

Reviewer #8: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #4: Everything is ok. I see you have addressed the previous comments suggested by the reviewers. The only concern I have is your data availability statement. You indicate that "data will be available on request", but that is not enough. There are lots of free data repositories such as OSF that allow you to share your data with the world. Please upload your data to a repository and share the link before publication.

Reviewer #6: I am sorry that the author has not adequately addressed my comments. Although the author indicated that some sentences were modified, the modifications are insufficient. I do not believe that the central issue of establishing the author-constructed instrument's reliability and the validity has been conducted properly. For example, stating that "The characteristics of the psychometric instruments have been confirmed" is insufficient to address this matter. Also, the author's justification that "Since the manuscript presented the results of each scientific and engineering practice separately, this requires the presentation and discussion of each result directly" is insufficient.

Overall, there need to be significant changes to the content, the style, and the presentation. Unfortunately, the manuscript still reads like an abbreviated thesis, and the concerns I raised have not been thoroughly or adequately addressed.

Thank you.

Reviewer #7: The first paragraph in introduction need punctuation, the paper need language editing before publishing.

the result and discussion sections should be separated and not mixed

Reviewer #8: For the manuscript titled " Scientific and engineering practices aligned with the NGSS in the performance of secondary stage physics teachers ", there are some points that were not addressed and need to be revised; especially, in relation to the literature of the study, research methodology and conclusion as following:

Point 1: In terms of the aims of the study that were mentioned in the abstract. the literature of this study still has not covered all these aims and questions, and there are no modifications that were made in the literature.

Point 2: under the (Methodology): researcher still did not specify which one of the mixed methods approaches was applied in this study. There is no mention which one did the researcher start with it before the other (questionnaire or interviews)?. And what the kind of mixed methods that was applied in this study Sequential Explanatory Design or Sequential Exploratory Design.

Point 3: Under the (Study Limits and Limitations), researcher did not clarify which one of the three major dimensions of the NGSS was chosen and why. In addition, researcher did not explain or mention about these three major dimensions in the literature review.

Finally: Under the (Conclusion): there were no modifications that were made by researcher. Conclusion needs to be rewritten again. it should be a summary for only the ideas that have strong finial impression. Also, all the limitations under the conclusion need to be moved to the section that was moved by the researcher before the conclusion. In addition, all the information from (…the results revealed that the teachers did...) needs to be placed under new section called Implications of this study.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #4: No

Reviewer #6: Yes: Ali M. AL-Asadi

Reviewer #7: No

Reviewer #8: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PONE-D-21-34137R3Scientific and engineering practices aligned with the NGSS in the performance of secondary stage physics teachersPLOS ONE

Dear Dr. Alsalamat,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Sep 25 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Yiming Tang, Ph.D.

Academic Editor

PLOS ONE

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #4: All comments have been addressed

Reviewer #6: (No Response)

Reviewer #8: (No Response)

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #4: Yes

Reviewer #6: Partly

Reviewer #8: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #4: Yes

Reviewer #6: No

Reviewer #8: I Don't Know

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #4: Yes

Reviewer #6: Yes

Reviewer #8: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #4: Yes

Reviewer #6: Yes

Reviewer #8: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #4: Congratulations, the manuscript looks suitable for publication to me now. I am happy to see you have performed the changes that were suggested in the previous reviews.

Reviewer #6: I will start by thanking the author for making the changes and reorganizing the manuscript. It reads much better now. However, I still have one major concern, which I shall outline in detail.

My analysis, critique, and review are based on my understanding of this study. Therefore, briefly stating my understanding is vital to understanding my critique.

1. The author identified eight scientific and Engineering practices based on NGSS: asking questions, developing and using models, planning and investigating, analyzing and interpreting data, involvement with proofs and evidence, interpretations and solutions design, obtaining, evaluating and communicating information, and using mathematics and computational thinking.

1. Then, the author devised a questionnaire comprising 44 indicators that measure these eight practices on a Likert’s 5-point scale (number of indicators per practice ranged from 4 to 7).

2. The questionnaire was then administered to a sample of 15 teachers, and based on their scores, internal consistency and Cronbach’s Alpha coefficients were calculated to “verify” the validity and reliability of the instrument.

3. Next, the author administered the questionnaire to 49 secondary school physics teachers to identify the degree to which the performance of physics teachers is aligned with NGSS scientific and engineering practices and if teachers’ qualifications and years of experience affect the degree to which teachers’ performance and the NGSS practices are aligned.

If this is correct, then basically, the author is trying to do two things at the same time: (1) constructing an instrument (questionnaire) that measures the eight practices and (2) using this instrument to (a) measure the degree to which the teaching is aligned with NGSS practices and (b) detect differences in adherence to NGSS practices based on qualifications and years of experience.

4. Firstly, the eight practices and the 44 indicators are not separate dimensions. They are the same in that the 44 indicators reflect (or measure) the eight practices. Therefore, statements such as, “The questionnaire comprised (8) practices and (44) indicators” are inaccurate and should read “The questionnaire comprised of 44 indicators representing (or measuring) eight practices.” Also, bracketing the number of practices and indicators is confusing and unnecessary.

5. Secondly and most importantly, the issue of reliability and validity. As I have indicated in my first review of this manuscript, a serious issue exists in establishing the validity and reliability of this instrument. It does not matter whether it is a questionnaire or an instrument; it is still a test that must go through the proper test construction process, including adequate validity and reliability procedures. There is no need to establish reliability and validity when constructing a questionnaire that asks for the sex and age of participants and their preference for a flavour of ice cream. But, in a case where multiple dimensions (or factors) define a construct (or constructs), establishing reliability and validity properly is of utmost importance, even in low-stakes testing.

6. Validity reflects the instrument’s ability to measure what it purports to measure. Reliability reflects the consistency of the instrument’s measurements. An instrument cannot be valid unless it is reliable, but a reliable instrument may not be valid. The author provided a range of internal consistency and Cronbach’s Alpha based on a very small sample of 15 teachers. The sample size is simply too small to establish reliability and validity for an instrument comprising 44 items measuring eight dimensions. Moreover, the small sample size notwithstanding, these two statistics may show that the instrument is reliable, but there is nothing about its validity.

7. Therefore, in the final analysis, the first part of this study is related to test construction, which was not conducted properly. The second part of the study is procedurally acceptable, but its results are based on an instrument that lacks validity and reliability, or at least its reliability and validity are questionable.

8. It is rather strange that this significant limitation is not indicated and included in the limitation section. Furthermore, the limitation section is relatively thin.

9. The second paragraph of the conclusion section repeats what is briefly mentioned in the limitation. The entire paragraph should be moved to the limitation section.

Obviously, a lot of work has gone into building this instrument, and it may be made a standard instrument to test adherence to NGSS practices providing proper validity and reliability studies. I suggest the author continues this work and develops this instrument to its full potential.

Whether to publish this manuscript or not is in the hands of the journal editors. To adequately address my concern, the test construction process needs to be followed. Data must be collected on a larger sample, followed by proper reliability and validity analyses. Alternatively, explicit statements must be made to emphasize this instrument's lack of (or questionable) reliability and validity and the caution and limitation that must be exercised when interpreting these results.

Reviewer #8: For the manuscript titled " Scientific and engineering practices aligned with the NGSS in the performance of secondary stage physics teachers ", there still some limitations under the section of “Conclusion”. All these limitations under the conclusion need to be moved to the “study limits and limitations” and make them concise.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #4: No

Reviewer #6: Yes: Dr. Ali M. AL-Asadi

Reviewer #8: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Scientific and engineering practices aligned with the NGSS in the performance of secondary stage physics teachers

PONE-D-21-34137R4

Dear Dr. Alsalamat,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Yiming Tang, Ph.D.

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #6: All comments have been addressed

Reviewer #8: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #6: Yes

Reviewer #8: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #6: Yes

Reviewer #8: N/A

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #6: Yes

Reviewer #8: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #6: Yes

Reviewer #8: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #6: (No Response)

Reviewer #8: (No Response)

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #6: Yes: Dr. Ali M. AL-Asadi

Reviewer #8: No

**********