Introducing a blocked procedure in nonparametric CD-CAT

Jiahui Zhang; Yuqing Yuan; Ziying Qiu; Feng Li

doi:10.1371/journal.pone.0312747

Peer Review History

Original SubmissionMay 29, 2024
18 Jul 2024 Decision Letter - Peida Zhan, Editor PONE-D-24-21730Introducing a Blocked Procedure in Nonparametric CD-CATPLOS ONE Dear Dr. Li, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. 1. Respond to reviewers' requests on a case-by-case basis and reflect them in revised manuscript. 2. Respond to one of my additional question. Please submit your revised manuscript by Sep 01 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript: A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Peida Zhan Academic Editor PLOS ONE Journal Requirements: 1. When submitting your revision, we need you to address these additional requirements. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf 2. Please note that PLOS ONE has specific guidelines on code sharing for submissions in which author-generated code underpins the findings in the manuscript. In these cases, we expect all author-generated code to be made available without restrictions upon publication of the work. Please review our guidelines at https://journals.plos.org/plosone/s/materials-and-software-sharing#loc-sharing-code and ensure that your code is shared in a way that follows best practice and facilitates reproducibility and reuse. 3. In the online submission form, you indicated that "The data underlying the results presented in the study are available from the correpondence author." All PLOS journals now require all data underlying the findings described in their manuscript to be freely available to other researchers, either 1. In a public repository, 2. Within the manuscript itself, or 3. Uploaded as supplementary information. This policy applies to all data except where public deposition would breach compliance with the protocol approved by your research ethics board. If your data cannot be made publicly available for ethical or legal reasons (e.g., public availability would compromise patient privacy), please explain your reasons on resubmission and your exemption request will be escalated for approval. 4. When completing the data availability statement of the submission form, you indicated that you will make your data available on acceptance. We strongly recommend all authors decide on a data sharing plan before acceptance, as the process can be lengthy and hold up publication timelines. Please note that, though access restrictions are acceptable now, your entire data will need to be made freely accessible if your manuscript is accepted for publication. This policy applies to all data except where public deposition would breach compliance with the protocol approved by your research ethics board. If you are unable to adhere to our open data policy, please kindly revise your statement to explain your reasoning and we will seek the editor's input on an exemption. Please be assured that, once you have provided your new statement, the assessment of your exemption will not hold up the peer review process. 5. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice. Additional Editor Comments: After the review by two professional reviewers and my personal reading, I think that this study has a certain degree of innovation on the topic of nonparametric CDCAT. On the whole, I agree with the comments of the two reviewers; meanwhile, I would like to add one question that the authors would like to add in the revised manuscript: the problem of item bank construction of non-parametric CDCAT, i.e., does non-parametric CDCAT need to consider the quality of the items in the construction of item banks, and if it needs to consider the quality of the items, how to construct a high-quality item bank without using a model? [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Yes ******** 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: Yes ****** 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ****** 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: No ****** 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: 1. The authors explored two versions of each blocked procedure in the simulation study, can the authors explain the reasons? 2. Can the authors list the detailed steps of the blocked NPS and GNPS methods in the main text? 3. how to determine the number of items that each q-vector type needed in the first step for the constrained blocked GNPS (lines 208-210) 4. The reason of the simulation design should be explained. 5. During the data generation, only six attribute profiles were generated, the rests were discarded, the authors should explain this setting. 6. The authors should explain the purpose and reason of constraining the q-vector of blocked items. Did this situation (i.e., only those items with different q-vectors would be selected into the block) would be occurred in empirical study? 7. During the simulation two (lines 375-385), which methods (package(s)) were used to calibrate item parameters? The original EM algorithm? or the Bayes Modal Estimation method (Ma & Jiang, 2021). Meanwhile, how many people were used to calibrate the items? Ma, W., & Jiang, Z. (2021). Estimating cognitive diagnosis models in small samples: Bayes modal estimation and monotonic constraints. Applied Psychological Measurement, 45(2), 95-111. 8. During the calibration of item parameters, how to calibrate the item parameters? all 1240 items were assembled to create a single test, then calibrated using small samples (e.g., 100)? Is that realistic? Can the authors adopted the matrix sampling technique to calibrate the item parameters? 9. Can the authors explain why the classification accuracy rate decreased as the block size increased? 10. The authors information were missing for the 18th reference in the "references" section Reviewer #2: This study intends to incorporate a blocked procedure into nonparametric CD-CAT to allow item review and answer modification while maintaining classification accuracy. The results showed that the blocked procedure with increasing the block size reduced pattern classification; however, this reduction decreased as item quality and test length increased. This study is beneficial for implication; however, the following particular questions must be addressed in order for merit to be published in PLOS ONE. 1. The simulation design used a parametric CD-CAT design, with some uncommon settings. Can the authors explain on the rationale for doing such a design, as well as the relationship to a non-parametric CD-CAT and more explanations of the simulation design? The explanations and specifications can assist the reader in understanding the distinction between non-parametric and parametric CD-CATs, conducting a non-parametric CD-CAT with proper simulation settings, and drawing the simulation results or implications. Such as, a big item pool (1240 items) is employed, which is rare in small-scale or classroom contexts. Was the sample size of 200 subjects for each studied attribute mastery pattern large enough to make an inference? Why were just the DNIA and A-CDM models used? The rationale for setting up the three levels of test lengths, etc. 2. The authors recommended that the blocked NPS is more suited for the DINA model and the blocked GNPS for the A-CDM model. The empirical data may be collected from different types of CDMs; could the authors provide further explanations / advices on when and how to utilize a suitable item selection method for a non-parametric CD-CAT when various CDMs are used. For example, when using a non-parametric CD-CAT with the DINO model, which item selection strategy should be used? 3. The results of the non-parametric CD-CAT were not included in Table 3. 4. On pages 24 and 29, the authors mentioned that “The violation of NPS's underlying assumptions when generating response data from the A-CDM model resulted in relatively low overall pattern classification accuracy for NPS.” and “When the response data better conform to the A-CDM model assumptions, using GNPS is more effective.” It is hard to follow if a reader is not familiar with NPS/GNPS and A-CDM. 5. Lines 451-454 on page 28 need to be reorganized. Based on Tables 2 and 4, the DINA model produced a classification rate of .8 when the item quality was high for both blocked NPS and GNPS, as well as when the item quality was low and the test length was lengthy for the blocked NPS. While the A-CDM model yielded a classification rate of .8 with the high-quality item pool for the blocked GNPS and a high item quality with the longer test for the blocked NPS. 6. Before submitting the manuscript, please check for citations (e.g., introducing Q-matrix and attribute mastery pattern), grammatical errors, and coherence (e.g., missing notations for Equation 3, the simulation data generated by the DINA and A-CDM models but GNPS used the weights based on the DINA and DINO models on page 17). ****** 6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: Yes: Chia-Ling Hsu ******** [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. https://doi.org/10.1371/journal.pone.0312747.r001
Revision 1
2 Oct 2024 Author Response Dear Editor, We sincerely appreciate the time and effort you and the reviewers have invested in evaluating our manuscript. We have carefully considered all the valuable comments and suggestions provided, and have made comprehensive revisions to the paper. Below is a summary of the major modifications: �Detailed steps and the rationale behind the proposed procedures have been added. �The Methods section has been expanded to clarify the rationale for the simulation plan. �The tables in the Results section have been reorganized to improve clarity. �The Discussion section has been strengthened to address the reviewers’ concerns more thoroughly. �We have conducted a comprehensive grammar check throughout the manuscript. All identified grammatical errors have been corrected to ensure that the text is clear, concise, and professional. We would like to thank you for your question regarding the construction of item banks in non-parametric CD-CAT. We appreciate the opportunity to clarify this aspect of our study. Compared to parametric approaches, non-parametric CD-CAT indeed places relatively lower demands on item quality. In non-parametric methods, the reliance on strict model assumptions is reduced, which allows for greater flexibility in item selection. As a result, non-parametric methods can effectively function even when items exhibit lower levels of discrimination. However, it is important to emphasize that the fundamental criteria of item quality remain crucial, regardless of whether a parametric or non-parametric model is used. For example, key aspects such as content validity and the clarity of item wording are universally applicable. To construct a high-quality item bank without relying on a specific measurement model, we suggest focusing on general principles of test development. This includes ensuring that the items comprehensively cover the targeted skills or attributes and that they are of appropriate difficulty for the population being tested. Furthermore, empirical analyses, such as item-total correlations, can be employed to identify and remove poorly functioning items. These steps help in maintaining the overall quality of the item bank, even in the absence of model-based item calibration. In response to your comment, we have revised and supplemented the relevant section of the Discussion in our manuscript (see pp. 36-37). We have expanded our discussion on item bank construction, addressing the specific considerations for non-parametric CD-CAT. This includes the challenges of maintaining item quality and the strategies for constructing a robust item bank without relying on parametric models. We hope that this response and the corresponding revisions to the manuscript address your concerns effectively. Please find attached our detailed responses to the reviewers' comments, along with an outline of the revisions made. All modiﬁcations are highlighted in the revised manuscript. We greatly appreciate your continued consideration of our work and the constructive feedback provided. Sincerely, Authors Response to Reviewer 1 We appreciate the time and eﬀort that you have devoted to giving us insightful suggestions and comments. We thoroughly revised our paper according to the suggestions and comments from you and the other reviewer. Below, we have arranged our responses below in the same sequence as your original comments/suggestions and outline changes made to the paper according to your comments/suggestions. All modiﬁcations are highlighted in the revised manuscript. 1.The authors explored two versions of each blocked procedure in the simulation study, can the authors explain the reasons? Reply: Our decision to explore multiple versions of the blocked procedure is grounded in the work of Kaplan and de la Torre (2020), who proposed different versions of the blocked parametric CD-CAT, including unconstrained, constrained, and hybrid approaches. We provided a detailed explanation of the rationale behind the constrained version in the revised manuscript (pp. 13-14): “The constrained version of the blocked procedure aims to make the q-vectors of items within the same block as distinct as possible. The rationale behind introducing the constrained version lies in its potential improvement on the performance of the adaptive testing procedure. Previous research in parametric methods indicated that administering items with the same q-vector repeatedly did not provide additional information for improving measurement precision [29]. Kaplan and de la Torre introduced the constrained version to their parametric blocked procedure to enhance the diagnostic power of the test by avoiding redundancy and ensuring a broader coverage of the attribute space [11]. ” 2.Can the authors list the detailed steps of the blocked NPS and GNPS methods in the main text? Reply: "The proposed blocked procedures" section has been reorganized and largely rewritten to enhance clarity, with a detailed description of the steps involved in the blocked NPS and GNPS now provided in the main text. Furthermore, Fig 1 has been incorporated to visually represent the unconstrained and constrained versions of the blocked nonparametric CD-CAT procedure. 3.“how to determine the number of items that each q-vector type needed in the first step for the constrained blocked GNPS (lines 208-210)” Reply: We apologize for the confusion caused by the original wording. This section has been revised to improve clarity. Since the number of q-vector types is generally larger than the block size, q-vector types are randomly selected in the constrained blocked GNPS. Each chosen q-vector type contributes one item to form the block, meaning that the number of items that each q-vector type needed is either one or zero. The revised manuscript now reads, “In the constrained version of the blocked GNPS, since the number of distinct q-vector types M (e.g., M=7 when examining three independent attributes) typically exceeds the block size , only one item from each q-vector type is required. The items with the highest discrimination power (defined in Equation 3) within each q-vector type are identified, and the top items from this subset are selected to form the block.” (p. 14) 4.The reason of the simulation design should be explained. Reply: We have rewritten the Method section to provide explanation of the simulation design in the revised manuscript on pages 16-19. 5.During the data generation, only six attribute profiles were generated, the rests were discarded, the authors should explain this setting. Reply: We regret any confusion caused by the original writing. This section "Examinee generation" has been revised to improve clarity (pp. 21-22). The decision to focus on a subset of six attribute profiles is based on the methodology outlined by Kaplan et al. (2015), who employed a similar approach to design a more efficient simulation study. The six selected attribute profiles—ranging from no mastery to full mastery—represent key stages in attribute mastery: , ，，，，and . Since the attributes are assumed to be independent in our study, all attribute mastery patterns involving the same number of mastered attributes can be considered equivalent in terms of their simulation outcomes. By selecting one representative profile for each level of mastery (i.e., the number of attributes mastered), we can capture the essential dynamics of the attribute space without generating and analyzing all possible profiles. This more efficient simulation design allows us to significantly reduce computational time and resources while maintaining the generalizability of our findings. 6.The authors should explain the purpose and reason of constraining the q-vector of blocked items. Did this situation (i.e., only those items with different q-vectors would be selected into the block) would be occurred in empirical study? Reply: This is related to the first comment. Additionally, we have strengthened the discussion regarding the empirical study： “The constrained version of the blocked procedure is also worth considering, as it can maintain or even enhance classification accuracy under certain conditions by constraining the q-vectors of blocked items. Since items with identical q-vectors assess the same content or skills, incorporating a variety of item types within a block is beneficial to avoid redundancy and reduce examinee boredom.”(p. 36) 7.During the simulation two (lines 375-385), which methods (package(s)) were used to calibrate item parameters? The original EM algorithm? or the Bayes Modal Estimation method (Ma & Jiang, 2021). Meanwhile, how many people were used to calibrate the items? Ma, W., & Jiang, Z. (2021). Estimating cognitive diagnosis models in small samples: Bayes modal estimation and monotonic constraints. Applied Psychological Measurement, 45(2), 95-111. Reply: We acknowledge the reference provided (Ma & Jiang, 2021). This new method enhances estimation accuracy, particularly when dealing with small sample sizes (≤100). As the sample size and test length increase, the differences between the new method and other methods become less significant. Since our focus is not on improving the calibration accuracy of parametric methods, we chose the conventional EM algorithm for simplicity. For our purpose, the use of parametric methods with small calibration samples was included to ensure a comprehensive comparison to underline the advantage of the non-parametric method over the parametric method with substantial calibration errors. We have clarified in the revised manuscript: “To demonstrate the impact of small calibration samples on the performance of the parametric methods, the calibration sample size () was set to 100 [8,10].” (p. 22) “The R package GDINA was used for item calibration in the parametric item selection strategies, specifically for the DINA and A-CDM models [32], using the marginal maximum likelihood estimation (MMLE) method with the Expectation-Maximization (EM) algorithm.” (p. 20) We have strengthened the discussion to inform readers of the existence of new method that better calibrate items in small sample size scenarios: “First, the nonparametric approach was introduced to address challenges associated with parametric methods, particularly in scenarios involving small calibration samples. However, recent advancements in parametric methods have yielded techniques that enhance estimation accuracy, even with small sample sizes (≤100) [36]. Future research should incorporate these developments when comparing nonparametric and parametric approaches.” (pp. 37-38) 8.During the calibration of item parameters, how to calibrate the item parameters? all 1240 items were assembled to create a single test, then calibrated using small samples (e.g., 100)? Is that realistic? Can the authors adopted the matrix sampling technique to calibrate the item parameters? Reply: Thank you for your insightful comment. As this is a simulation study, it is indeed feasible to administer all 1,240 items to a calibration sample of 100 examinees during the calibration of item parameters. While this approach would be highly unrealistic in a real-world setting due to practical constraints, it is appropriate within the context of a simulation study, where the goal is to showcase the impact of calibration sample sizes on the parametric methods. In real-world applications, a matrix sampling technique would indeed be a more practical and realistic option for calibrating item parameters. However, the exploration of such techniques is beyond the scope of the current study, which is focused on the theoretical aspects of the proposed method. 9.Can the authors explain why the classification accuracy rate decreased as the block size increased? Reply: Thank you for highlighting this important issue. We have accordingly expanded and strengthened the relevant section of the discussion. It is important to note that a block size of 1 corresponds to the traditional item-level CD-CAT administration, where the test is highly adaptive, selecting the most informative item for each examinee at each stage. However, as the block size increases, the test becomes less adaptive, akin to the difference between Multi-Stage Testing (MST) and item-level CAT. In a blocked CD-CAT, multiple items are selected at once, reducing the ability to tailor each item to the examinee’s evolving ability estimate. This reduced adaptivity is a key reason why classification accuracy tends to decrease as block size increases. Kaplan and de la Torre (2020) observed similar findings, particularly under conditions where items were of low quality and the test lengths were short or medium. Ultimately, the choice of block size involves a trade-off between adaptivity and other practical considerations, such as allowing students to review and change their answers. As Kaplan and de la Torre (2020) suggest, practitioners must determine the tolerable level of loss in classification accuracy when deciding on the appropriate block size for their specific context. We have expanded our discussion on the decreased accuracy with increased block size (see p. 34-35). 10.The authors information were missing for the 18th reference in the "references" section Reply: Thank you for pointing out this. We apologize for this oversight and will ensure that the correct author details are included in the revised manuscript. Yan D, Von Davier AA, Lewis C. Computerized multistage testing: Theory and applications. New York: CRC Press; 2014.https://doi.org/10.1201/b16858 We are grateful for your careful attention to detail and the chance to refine our manuscript. We believe these revisions have greatly enhanced its clarity and coherence. Thank you again for your insightful feedback. We are confident that the manuscript is now stronger and more suitable for publication. Response to Reviewer 2 We appreciate the time and eﬀort that you have devoted to giving us insightful suggestions and comments. We thoroughly revised our paper according to the suggestions and comments from you and the other reviewer. Below, we have arranged our responses below in the same sequence as your original comments/suggestions and outline changes made to the paper according to your comments/suggestions. All modiﬁcations are highlighted in the revised manuscript. 1.The simulation design used a parametric CD-CAT design, with some uncommon settings. Can the authors explain on the rationale for doing such a design, as well as the relationship to a non-parametric CD-CAT and more explanations of the simulation design? The explanations and specifications can assist the reader in understanding the distinction between non-parametric and parametric CD-CATs, conducting a non-parametric CD-CAT with proper simulation settings, and drawing the simulation results or implications. Such as, a big item pool (1240 items) is employed, which is rare in small-scale or classroom contexts. Was the sample size of 200 subjects for each studied attribute mastery pattern large enough to make an inference? Why were just the DNIA and A-CDM models used? The rationale for setting up the three levels of test lengths, etc. a)The simulation design used a parametric CD-CAT design, with some uncommon settings. Can the authors explain on the rationale for doing such a design, as well as the relationship to a non-parametric CD-CAT and more explanations of the simulation design? Reply: This important point resonates with the fourth comment from the first reviewer. We have provided an explanation of the simulation design in the revised manuscript on pages 16-19. b)big item pool (1240 items) is employed, which is rare in small-scale or classroom contexts. Reply: Thanks for bringing this out. The use of a large item pool (1240 items) in our simulation study was intentional to isolate the effects of the proposed method from those of item pool size. While we recognize that such a large item pool may be uncommon in small-scale or classroom contexts, the goal was to avoid confounding the results with the limitations that a smaller item pool could introduce. Our method is designed to work with smaller item pools as well; however, in scenarios where the it Attachments Attachment Submitted filename: Response to reviewers.docx https://doi.org/10.1371/journal.pone.0312747.r002
9 Oct 2024 Decision Letter - Peida Zhan, Editor PONE-D-24-21730R1Introducing a Blocked Procedure in Nonparametric CD-CATPLOS ONE Dear Dr. Li, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. ============================== After modification, I think the author has responded well to the questions of the two reviewers, and I think this paper meets the publication requirements.In addition, it is suggested that the author further supplement the relevant literature on non-parametric CDA in the introduction, such as:(1)Doi: 10.16719/j.cnki.1671-6981.20230627；(2)Doi: 10.1007/S11336-009-9125-0 ============================== Please submit your revised manuscript by Nov 23 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript: A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Peida Zhan Academic Editor PLOS ONE Journal Requirements: Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice. Additional Editor Comments: After modification, I think the author has responded well to the questions of the two reviewers, and I think this paper meets the publication requirements.In addition, it is suggested that the author further supplement the relevant literature on non-parametric CDA in the introduction, such as:(1)Doi: 10.16719/j.cnki.1671-6981.20230627；(2)Doi: 10.1007/S11336-009-9125-0 [Note: HTML markup is below. Please do not edit.] [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. https://doi.org/10.1371/journal.pone.0312747.r003
Revision 2
10 Oct 2024 Author Response In this revision, we have enhanced the introduction by including relevant literature on nonparametric methods, as recommended by the editor. We would like to add the following funding information: "This research was supported by the National Key R&D Program of China (2021YFC3340801). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript." However, we could not find anywhere to enter this funding information in the submission system. Attachments Attachment Submitted filename: Response to reviewers R2.docx https://doi.org/10.1371/journal.pone.0312747.r004
14 Oct 2024 Decision Letter - Peida Zhan, Editor Introducing a Blocked Procedure in Nonparametric CD-CAT PONE-D-24-21730R2 Dear Dr. Li, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager® and clicking the ‘Update My Information' link at the top of the page. If you have any questions relating to publication charges, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, Peida Zhan Academic Editor PLOS ONE Additional Editor Comments (optional): Based on my review, the revised manuscript meets the requirements for publication and is recommended for publication. Good work. https://doi.org/10.1371/journal.pone.0312747.r005
Formally Accepted
17 Oct 2024 Acceptance Letter - Peida Zhan, Editor PONE-D-24-21730R2 PLOS ONE Dear Dr. Li, I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now being handed over to our production team. At this stage, our production department will prepare your paper for publication. This includes ensuring the following: * All references, tables, and figures are properly cited * All relevant supporting information is included in the manuscript submission, * There are no issues that prevent the paper from being properly typeset If revisions are needed, the production department will contact you directly to resolve them. If no revisions are needed, you will receive an email when the publication date has been set. At this time, we do not offer pre-publication proofs to authors during production of the accepted work. Please keep in mind that we are working through a large volume of accepted articles, so please give us a few weeks to review your paper and let you know the next and final steps. Lastly, if your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. If we can help with anything else, please email us at customercare@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Dr. Peida Zhan Academic Editor PLOS ONE https://doi.org/10.1371/journal.pone.0312747.r006

Open letter on the publication of peer review reports

PLOS recognizes the benefits of transparency in the peer review process. Therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. Reviewers remain anonymous, unless they choose to reveal their names.

We encourage other journals to join us in this initiative. We hope that our action inspires the community, including researchers, research funders, and research institutions, to recognize the benefits of published peer review reports for all parts of the research system.

Learn more at ASAPbio .