Computational thinking in university students: The role of fluid intelligence and visuospatial ability

Gabor Aranyi; Kristof Kovacs; Ferenc Kemény; Orsolya Pachner; Balázs Klein; Eszter P. Remete

doi:10.1371/journal.pone.0309412

Peer Review History

Original SubmissionFebruary 1, 2024
22 May 2024 Decision Letter - Liliana G Ciobanu, Editor PONE-D-24-03519Computational Thinking in University Students: The Role of Fluid Intelligence and Visuospatial AbilityPLOS ONE Dear Dr. Aranyi, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please submit your revised manuscript by Jul 06 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript: A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Liliana G Ciobanu Academic Editor PLOS ONE Journal Requirements: 1. When submitting your revision, we need you to address these additional requirements. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf 2. Thank you for stating the following financial disclosure: Gabor Aranyi received funding by the National Research, Development and Innovation Office of Hungary (NKFIH): Grant OTKA-FK-143095 project entitled ‘Advancing the Concept of Computational Thinking’. Kristof Kovacs received funding by the National Research, Development and Innovation Office of Hungary: Grant FK-21-138971, by the János Bolyai Research Scholarship of the Hungarian Academy of Sciences and by the ÚNKP-23-5 New National Excellence Program of the Ministry for Innovation and Technology from the source of the National Research, Development and Innovation Fund. Please state what role the funders took in the study. If the funders had no role, please state: "The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript." If this statement is not correct you must amend it as needed. Please include this amended Role of Funder statement in your cover letter; we will change the online submission form on your behalf. 3. Thank you for stating the following in the Competing Interests section: Balázs Klein is the executive director of Testar Ltd. The paper reflects the views of the scientists and not the company. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest. We note that one or more of the authors are employed by a commercial company: Testar Ltd. a. Please provide an amended Funding Statement declaring this commercial affiliation, as well as a statement regarding the Role of Funders in your study. If the funding organization did not play a role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript and only provided financial support in the form of authors' salaries and/or research materials, please review your statements relating to the author contributions, and ensure you have specifically and accurately indicated the role(s) that these authors had in your study. You can update author roles in the Author Contributions section of the online submission form. Please also include the following statement within your amended Funding Statement. “The funder provided support in the form of salaries for authors, but did not have any additional role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript. The specific roles of these authors are articulated in the ‘author contributions’ section.” If your commercial affiliation did play a role in your study, please state and explain this role within your updated Funding Statement. b. Please also provide an updated Competing Interests Statement declaring this commercial affiliation along with any other relevant declarations relating to employment, consultancy, patents, products in development, or marketed products, etc. Within your Competing Interests Statement, please confirm that this commercial affiliation does not alter your adherence to all PLOS ONE policies on sharing data and materials by including the following statement: "This does not alter our adherence to PLOS ONE policies on sharing data and materials.” (as detailed online in our guide for authors http://journals.plos.org/plosone/s/competing-interests) . If this adherence statement is not accurate and there are restrictions on sharing of data and/or materials, please state these. Please note that we cannot proceed with consideration of your article until this information has been declared. Please include both an updated Funding Statement and Competing Interests Statement in your cover letter. We will change the online submission form on your behalf. 4. In the online submission form, you indicated that The data underlying the results presented in the study are available on request from the corresponding author (aranyi.gabor@ppk.elte.hu). All PLOS journals now require all data underlying the findings described in their manuscript to be freely available to other researchers, either a. In a public repository, b. Within the manuscript itself, or c. Uploaded as supplementary information. This policy applies to all data except where public deposition would breach compliance with the protocol approved by your research ethics board. If your data cannot be made publicly available for ethical or legal reasons (e.g., public availability would compromise patient privacy), please explain your reasons on resubmission and your exemption request will be escalated for approval. 5. Your ethics statement should only appear in the Methods section of your manuscript. If your ethics statement is written in any section besides the Methods, please move it to the Methods section and delete it from any other section. Please ensure that your ethics statement is included in your manuscript, as the ethics statement entered into the online submission form will not be published alongside your manuscript. [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Partly ******** 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: I Don't Know ****** 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes ****** 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes ****** 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: Computational Thinking in University Students: The Role of Fluid Intelligence and Visuospatial Ability The study examines gender differences in computational thinking and how they are mediated by gender differences in visuo-spatial skills. While I’ve enjoyed reading this paper, I had problems in understanding which was the specific focus of the study. The title and part of the introduction suggests that the focus is on the role of fluid intelligence and visuo.spatial skills, but when the reader comes to the HPs section and the analyses, the focus becomes unclear: Is it about the cognitive factors explaining performance in CT among university students unfamiliar with programming? In this case, gender should be a control/subject variable. Is the paper about the existence of gender-related differences in computational thinking and their relation with gender differences in visuo.spatial skills (as it seems from most of the analyses)? If so, this should be clarified from the beginning, and fluid and crystallized intelligence should just be control/covaried variables. My general impression in reading the paper, and especially the HPs and analyses sections, was that the authors had problems in clarifying the specific role/nature of the study variables: which are the independent, dependent, mediation and control variables of the study? Morevoer, not all the analyses follow up from the RQs of the study and the literature review. While the mediational role of visuo.spatial skills in CT is supported by prior literature, the reason why the authors are assuming a mediation/moderation role of fluid intelligence explaining gender differences in CT is less clear. Additionally, some analyses seem to follow from revisited? HPs rather than the original HPs of the study: for instance, the analyses testing whether gender differences in fluid intelligence are accounted for by gender differences in visuospatial ability (SPOT) (p. 18) I did not understand why did the authors consider revisited HPs or what they do mean with “revisited”? Are they ex-post HPs? Sample. Although the authors recognize they have considered a convenience sample of psychology students, given that the prevalence of boys and girls enrolled in psychology classes is typically unbalanced, especially in some psychology classes, they could have involved more courses or classes of psychology to increase the number of boys in their sample (only 29 young men, versus 68 young women). Procedure. For all standardized measures used, reliability/validity should be reported ****** 6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No ******** [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. https://doi.org/10.1371/journal.pone.0309412.r001
Revision 1
3 Aug 2024 Author Response Response to reviewers We are grateful for Reviewer #1 for the useful comments. Addressing these comments has helped us to substantially improve the manuscript. The Reviewer’s comments are reproduced verbatim below with our answers inserted where appropriate. --- Reviewer #1: Computational Thinking in University Students: The Role of Fluid Intelligence and Visuospatial Ability The study examines gender differences in computational thinking and how they are mediated by gender differences in visuo-spatial skills. While I’ve enjoyed reading this paper, I had problems in understanding which was the specific focus of the study. The title and part of the introduction suggests that the focus is on the role of fluid intelligence and visuo.spatial skills, but when the reader comes to the HPs section and the analyses, the focus becomes unclear: Is it about the cognitive factors explaining performance in CT among university students unfamiliar with programming? In this case, gender should be a control/subject variable. Is the paper about the existence of gender-related differences in computational thinking and their relation with gender differences in visuo.spatial skills (as it seems from most of the analyses)? If so, this should be clarified from the beginning, and fluid and crystallized intelligence should just be control/covaried variables. With thanks to the reviewer, we clarified in the manuscript that the focus remains on the role of cognitive abilities, as in the title. However, since there is strong evidence in the literature for gender differences in both visuospatial ability and CT performance, gender is controlled in the analyses. We added an extra paragraph to the Background sub-section to emphasize the focus of the research: The present research investigates the relationship between CT performance, general intelligence, and visuospatial ability in students in higher education. The following sections provide an overview of CT measurement and the relevant research addressing the role of visuospatial ability and general intelligence in driving CT performance. We argue that a better understanding of CT and its relationship with well-established cognitive factors is supported by controlling for the effects of potential confounds which we identify in previous literature, such as participants’ age, their programming experience, and widely-reported gender differences in visuospatial ability and in CT performance. We made some further changes in the Introduction section, and the last paragraph of the updated Results section contains the relative contributions of these variables to CT performance, emphasizing the role of fluid intelligence. We also refocused the paper by explicitly stating the research question related to the relationships between CT and cognitive abilities: RQ2: How is computational thinking related to visuospatial ability, fluid intelligence, and crystallized intelligence in adults unfamiliar with programming? My general impression in reading the paper, and especially the HPs and analyses sections, was that the authors had problems in clarifying the specific role/nature of the study variables: which are the independent, dependent, mediation and control variables of the study? We confirm that the focus of the paper is on the role of fluid intelligence and visuospatial ability on CT. However, due to gender differences in both visuospatial ability and CT widely reported in the literature, as well as to correlations expected between the cognitive constructs (visuospatial ability, fluid IQ, and crystallized IQ) discussed in the positive-manifold and process-overlap arguments, we argue in the manuscript that gender as a confounding factor needs to be controlled. With thanks to the reviewer, we have made changes in the manuscript to clarify the focus (see above comment). We have also made changes to the Research Questions and Hypotheses section to highlight the role of gender as control variable: Gender is treated as a control variable in the present study, the effects of which on other study variables are considered when examining the relationship between CT and cognitive factors. Thus, H1 is formulated to test if gender differences in CT often reported in younger participants can be observed in programming-naïve university students, while H2 aims at testing gender differences in visuospatial ability often reported in the cognitive literature. Morevoer, not all the analyses follow up from the RQs of the study and the literature review. While the mediational role of visuo.spatial skills in CT is supported by prior literature, the reason why the authors are assuming a mediation/moderation role of fluid intelligence explaining gender differences in CT is less clear. With thanks to the reviewer, we note that no a priori hypothesis is related to the mediating effect of fluid intelligence between gender and CT. We made changes to the Research Questions and Hypotheses section related to the mediation hypothesis (H6) to clarify. We have also dropped RQ3 (the previous RQ2 is now RQ3, because a new RQ2 was added to explicitly address the relationships between CT and cognitive factors). Although a direct path from gender to CT is specified based on the simple association between gender and SAM, this is only used to test if the association between the two are spurious (see the updated H4 report and the comment below), which is supported by our results. Conversely, despite the large correlation between SAM and SPOT, fluid intelligence does not explain gender differences in visuospatial ability (see the updated H2 report). Additionally, some analyses seem to follow from revisited? HPs rather than the original HPs of the study: for instance, the analyses testing whether gender differences in fluid intelligence are accounted for by gender differences in visuospatial ability (SPOT) (p. 18) Indeed, this analysis association was not hypothesized, but we used the path model to follow up the simple association between gender and fluid intelligence reported in tables 2 and 3; considering visuospatial ability showed that gender and fluid intelligence were spuriously associated. We clarified this in the updated section related to H4. I did not understand why did the authors consider revisited HPs or what they do mean with “revisited”? Are they ex-post HPs? With thanks to the reviewer, we substantially reorganized the Results section of the manuscript. We now report hypothesis tests in one place, following the presentation of descriptive statistics, tables reporting simple associations, and the path model. Hypothesis tests are now reported in the Hypothesis tests and Indirect Effects sub-section, while the summary tables of simple associations are kept in the Descriptive Analysis and Simple Associations sub-section. Sample. Although the authors recognize they have considered a convenience sample of psychology students, given that the prevalence of boys and girls enrolled in psychology classes is typically unbalanced, especially in some psychology classes, they could have involved more courses or classes of psychology to increase the number of boys in their sample (only 29 young men, versus 68 young women). We agree with the reviewer that larger samples are desirable in general. As our protocol concentrated on controlled data collection in an individual and supervised setting consisting of two sessions, we aimed at achieving the required sample size to test a priori hypotheses informed by the effect sizes reported in the reviewed literature. In other words, we aimed to maximize data quality over quantity. We discuss in the Limitations and Future Work section that future studies aiming at population-based inferences should recruit a larger sample. We made additions to the Participants and Limitations and Future Work sections to expand on this. Procedure. For all standardized measures used, reliability/validity should be reported Thank you for pointing out that no information was provided about the tests’ accuracy. However, since the adaptive tests rely on item response theory (IRT), the classical notion of test reliability cannot be applied. This is because unlike in classical tests where accuracy is a test property represented in the reliability statistic, under IRT the error margin (standard error, SE) of each theta value is estimated. We added the mean SE for each adaptive test to Table 1. Additionally, since classical reliability values can be approximated from IRT-based SE values, we have calculated these values and include them in the main text: The computerized adaptive tests are based on item response theory (IRT), where the conception of accuracy is different from the concept of reliability in classical test theory. That is, under the IRT framework it is the (item or test) information function that represents precision so that information is the inverse of the items’ or the test’s accuracy: the more information the test or the items provide, the smaller the error margin of the measurement. Under IRT models the SE (standard error) of each ability (theta) score can be calculated; the higher the information the smaller the SE. We report the mean SEs obtained for each computerized adaptive test in Table 1. Classical reliability coefficients can be approximated from SE values obtained in IRT [66, 67]. The reliability values estimated from the SE values that appear in Table 1 are .92, .95, and .97 for SPOT, SAM, and NoVo, respectively. References to papers that report the psychometric properties and the validity of SAM and NoVo have been added to the Materials and Methods section (321-2., 331-2.). Unfortunately, validity information for the SPOT test is not yet available. For the validity of the computational thinking test, we refer to [14]. Unfortunately, validity information for the Hungarian version of CTt is not yet available. Additionally, a detailed check of our manuscript revealed that the constants in Table 4 were erroneously recorded (all other parameters were correctly displayed). Therefore, we have replaced Table 4 in the updated manuscript; the table now contains the correct values. We have also changed a misleading abbreviation in Table 5 (we now use the test’s name, SPOT, instead of VSA for visuospatial ability). Attachments Attachment Submitted filename: Response_to_review.docx https://doi.org/10.1371/journal.pone.0309412.r002
13 Aug 2024 Decision Letter - Liliana G Ciobanu, Editor Computational Thinking in University Students: The Role of Fluid Intelligence and Visuospatial Ability PONE-D-24-03519R1 Dear Dr. Aranyi, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager® and clicking the ‘Update My Information' link at the top of the page. If you have any questions relating to publication charges, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, Liliana G Ciobanu Academic Editor PLOS ONE Additional Editor Comments (optional): Reviewers' comments: https://doi.org/10.1371/journal.pone.0309412.r003
Formally Accepted
23 Aug 2024 Acceptance Letter - Liliana G Ciobanu, Editor PONE-D-24-03519R1 PLOS ONE Dear Dr. Aranyi, I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now being handed over to our production team. At this stage, our production department will prepare your paper for publication. This includes ensuring the following: * All references, tables, and figures are properly cited * All relevant supporting information is included in the manuscript submission, * There are no issues that prevent the paper from being properly typeset If revisions are needed, the production department will contact you directly to resolve them. If no revisions are needed, you will receive an email when the publication date has been set. At this time, we do not offer pre-publication proofs to authors during production of the accepted work. Please keep in mind that we are working through a large volume of accepted articles, so please give us a few weeks to review your paper and let you know the next and final steps. Lastly, if your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. If we can help with anything else, please email us at customercare@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Dr. Liliana G Ciobanu Academic Editor PLOS ONE https://doi.org/10.1371/journal.pone.0309412.r004

Open letter on the publication of peer review reports

PLOS recognizes the benefits of transparency in the peer review process. Therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. Reviewers remain anonymous, unless they choose to reveal their names.

We encourage other journals to join us in this initiative. We hope that our action inspires the community, including researchers, research funders, and research institutions, to recognize the benefits of published peer review reports for all parts of the research system.

Learn more at ASAPbio .