PONE-D-21-09587

Affective states in digital game-based learning: thematic evolution and social network analysis

PLOS ONE

Dear Dr. Zou,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. I have invited experts in the field and they have raised concerns regarding the methods,In particular, the choice of the methods should better explained, grounded in literature and allow a reproducible research.

Please submit your revised manuscript by Jun 03 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Mohammed Saqr, Ph.D

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2.  Please note that board membership at PLOS should be declared in your competing interest statement.

Please confirm that this does not alter your adherence to all PLOS ONE policies on sharing data and materials, by including the following statement: "This does not alter our adherence to  PLOS ONE policies on sharing data and materials.” (as detailed online in our guide for authors http://journals.plos.org/plosone/s/competing-interests).  If there are restrictions on sharing of data and/or materials, please state these. Please note that we cannot proceed with consideration of your article until this information has been declared.

Please include your updated Competing Interests statement in your cover letter; we will change the online submission form on your behalf.

Please know it is PLOS ONE policy for corresponding authors to declare, on behalf of all authors, all potential competing interests for the purposes of transparency. PLOS defines a competing interest as anything that interferes with, or could reasonably be perceived as interfering with, the full and objective presentation, peer review, editorial decision-making, or publication of research or non-research articles submitted to one of the journals. Competing interests can be financial or non-financial, professional, or personal. Competing interests can arise in relationship to an organization or another person. Please follow this link to our website for more details on competing interests: http://journals.plos.org/plosone/s/competing-interests

3. Thank you for stating the following financial disclosure:

"The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript."

At this time, please address the following queries:

1. Please clarify the sources of funding (financial or material support) for your study. List the grants or organizations that supported your study, including funding received from your institution.
2. State what role the funders took in the study. If the funders had no role in your study, please state: “The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.”
3. If any authors received a salary from any of your funders, please state which authors and which funders.
4. If you did not receive any funding for this study, please state: “The authors received no specific funding for this work.”

Please include your amended statements within your cover letter; we will change the online submission form on your behalf.

4.  We note that you have indicated that data from this study are available upon request. PLOS only allows data to be available upon request if there are legal or ethical restrictions on sharing data publicly. For more information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions.

In your revised cover letter, please address the following prompts:

a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially sensitive information, data are owned by a third-party organization, etc.) and who has imposed them (e.g., an ethics committee). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent.

b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings as either Supporting Information files or to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories.

We will update your Data Availability statement on your behalf to reflect the information you provide.

5. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: No

Reviewer #2: I Don't Know

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: No

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: I. Summary of the research and overall impression

Title: ‘Affective states in digital game-based learning: thematic evolution and social network analysis’

I’d like to thank the authors for their interesting submission. The study the manuscript reports on addresses an important aspect of learning, i.e., virtual learning that is nowadays related to DGBL. The goal is to synthesize how emotions are addressed in DGBL literature. The paper provides an overview of learners’ affective status in DGBL based on 393 research articles. In principle, this article could be of interest to the general readership of the Plos One Journal. However, there are some major issues that need to be addressed.

• The authors should better explain why their review is important, in light of previous research and of questions yet to be answered within the field.

• They should make a substantial, complete and exhaustive literature review (i.e., theoretical background).

• Based on this previous literature, the authors could formulate hypothesis related to the links that are expected between emotion and DGBL.

• Also the scope of the study is not always clear. For instance, the aim of the paper is to link emotions and DGBL, but within the results we can see that the links between autism - which is not an emotion - and DGBL are also tested.

• The method section should be developed more (e.g., statistical analysis).

• With a complete theoretical background (and maybe some hypothesis clearly formulated), the authors would be able to indicate which results align with previous literature and to explain why other do not correspond to what was found in previous studies.

• The authors need to rework their manuscript in order to put the different elements where they belong.

II. Specific comments

a) Abstract

• The problem statement should be made more explicit.

• ‘learners’ emotions is increasingly important in understanding learners’ learning experiences within digital game-based learning (DGBL)’ It would be useful to explain why and how it is important (one or two sentences).

• The authors summarize the main research question and key findings but the abstract is quite long and could beneficiate from deleting some unnecessary details.

b) Introduction

• The links between paragraphs are not always clear to me (e.g., between ‘Understanding students’ affective status during learning’ and ‘evidence show digital games potentials to nurture and sustain high levels of learning motivation and engagement’). I have the impression that sometime, steps are missing between the ideas that are developed.

• It would be helpful to clearly state from the start that the study is limited to virtual learning.

• Also, on all students or limited to high-school students, university students …?

• It would be helpful to give a clear definition of what are ‘topics’ for DGBL

• The ‘what’ is addressed: ‘The goal is to synthesize how emotions are addressed in DGBL literature’

• The ‘why’ is partially missing: it is not clear from the introduction, but I think that working on links between paragraphs might help better explaining why the paper is important? The authors should explain why their study matters and put the research in context.

• The ‘how’ is missing : The authors should briefly present how they intend to answer their research questions

• The way the manuscript is organized is missing

c) Literature review (i.e., theoretical background)

• What is contextual focus (row 86)?

• Engagement is an emotion? I don’t quite understand the links between rows 83-86 and 87-91?

• ‘For example, a close correlation between an emotion-related topic A and an analytical technology-related topic B means that B is popularly adopted to study issues related to A’. Could you give a concrete example? This might help understanding better why your review is important.

• The literature review is too narrow. I think the authors should deepen what’s have been done in previous research and that legitimates their research

• ‘We divided the whole period into three sub-periods each consisting of two years’: why this particular division?

• Rows 114-128: why within the literature review section?

d) Methods

• The emotions, educational games, gaming approaches, methodologies, concepts and different types of platforms supporting DGBL should be defined (within a theoretical section instead of the method section?)

• Search query: the authors have to specify on what search engine(s). If I understand correctly, the search queries are combinations of the different terms? (e.g., emotion+game+learn; affect+game+learn …). It should be explained better.

• Natural Language Toolkit: the authors should explain it in a few sentences.

• The topics in four categories (i.e., digital games, affective status, applications and devices supporting DGBL, and analytical technologies): the authors should explain from where the four categories come: from previous literature or were they identified by them (based on the 393 research articles)?

• Rows 183-198: within a theoretical section (or the literature review section) instead of the method section?

• Before speaking about nodes and their size, the authors should introduce SNA and define what a network is, then explain how they use SNA in the context of their research.

• It is not very clear (stated) that SNA is also applied on emotions. But I think that it is because the term ‘topic(s)’ is used to represent different things, and as reader, I’m not always sure of what topic(s) refer(s).

e) Results

• Thematic correlation: too poorly developed, the reader has to look at the figures (6 to 9) to (try) understanding the results.

o The results are explained later within the discussion section � they have to be removed within the results section.

• Also, some terms appear in some figures, but I don’t understand why, since it was not explained earlier that these terms would be tested in the correlations (e.g., elementary school, high school, flow, engineering education, autism …)

f) Discussion and conclusions

• Rows 271-272: ‘we more specifically identified detailed aspects of concepts such as emotions related to specific subject learning (e.g., computer anxiety and English anxiety) and social emotions’. Unless I’m wrong, it was not previously indicated that emotions related to subject learning would be investigated?

• The major problem encountered within the discussion is that since the literature review was poorly documented, and since no hypothesis formulated for the expected correlations between constructs, the reader is not able to know if the results support previous literature, if they bring something new to the field, support the conclusions …

• I’m also under the impression that a substantial part of the literature review (i.e., theoretical background) - which needs to be addressed in the beginning of the paper - is included within the discussion.

• Again, scope of the study?? ‘Mood/affective disorders’ section?

• If I’m correct, the authors do not discuss any limitations of their study.

g) Statistical analysis

• It would be nice to see some data descriptive analyses (e.g., a table with the means, standard deviation, median, Kurtosis and Skewness …)

• Between what and what exactly were computed the correlations? The co-occurrences? Why working on the co-occurrences and not the occurrences? The authors should explain how they proceeded to the correlation analysis within the method section.

• For the correlations, did the authors use Pearson? Spearman? Did they verify the normality of the data in order to choose the more reliable coefficient?

• Are the correlations significant? The authors need a table with the p-values for each correlation set.

h) Data and supporting information

• I’m not entirely sure that the data provide enough evidence for the authors’ conclusions

• The authors did not provide a sufficient amount of data and information for other researchers to recreate the analyses

i) Figures and tables

• Figures 2 to 5: too dense and small, it is difficult to read. Also, for figure 3, two topics are represented within the same figure. Why? Because it is a little bit confusing.

j) Minor issues

• Make sure using same terms to not confuse the reader.

e.g., (abstract): ‘Specifically, thematic evolution analysis was conducted to explore topic evolution concerning four different groups (i.e., games, emotions, applications, and analytical techniques) in the corpus. Social network analysis was used to explore the relation between topics to identify their correlations’: if I understood correctly, groups = topics? If yes, authors should use the same term. If no, it should be explained better.

• The sentences are quite long and difficult to understand (e.g., ‘With a focus on learners’ engagement during DGBL, Abdul Jabbar and Felicia systematically analyzed game design features promoting engagement and learning in DGBL based on 91 articles during 2003–2013, identifying complex gaming elements impacting gameplay engagement and learning experiences in DGBL at both cognitive and emotional levels’)

• Present and past tenses : not always used correctly (e.g., for the research questions)

• The article is not presented in an intelligible fashion nor is written in standard English. Please let the manuscript be proofread before resubmission.

• References: APA style? e.g.:

Halloluwa T, Vyas D, Usoof H, Hewagamage KP. Gamification for development: a case of collaborative learning in Sri Lankan primary schools. Pers Ubiquitous Comput. 2018; 22(2):391–407.

Instead of: Halloluwa, T., Vyas, D., Usoof, H., & Hewagamage, K. P. (2018). Gamification for development: a case of collaborative learning in Sri Lankan primary schools. Personal and Ubiquitous Computing, 22(2), 391-407.

Reviewer #2: The authors present a thematic evolution of DGBL research using SNA methods. It is an interesting article that examines the increasingly central topic of educational games. Moreover, the authors highlight some of the most relevant areas for further research.

I do have, however, some concerns regarding the methods of the article.

1. The authors used the code "TS" from Web of Science for the search. This term includes Title, Abstract, Author keywords, and Keywords+. The latter are automatically added by the database. This is not a problem since the authors manually checked the articles. However, it is not clear if for the analysis they used only Author keywords, also Keywords+, or also Abstract or Title terms.

2. The authors should describe in further detail how the DGBL-related terms for the search were selected. How did they come up with "affect", "feeling", "anxiety", etc.?

3. The authors should provide more information of the search to allow reproducibility of results, such as if they analyzed only journal articles, or also conference papers, book chapters, etc. They should also state if they filtered by year, by language, etc.

3. The authors should describe if and how keywords were combined or cleaned before the analysis. For instance, combining singular and plural keywords is very common, as well as synonyms. However, in Fig. 6, for instance, words like engage and engagement appear separately. Do the authors think they could/should be combined to better capture the relation between terms?

4. In Figs. 2, 3, 4, the author should specify how the graphs were constructed. A legend should indicate what the size of the circles represent (number of ocurrences of each keyword, I assume) and the color. Also, the authors need to explain the criteria for adding a keyword under a specific year. Was there a specific threshold for the keyword frequency in order it to consider it popular for a given year?

5. In Figs. 6, 7, 8, 9, why were co-ocurrences of 2, 3 or 4 specifically plotted? In my opinion, an overview of the whole network would be very interesting and even more so if a yearly network was constructed.

6. The authors should explicitly point to the results when discussing the findings of the article. For example, when talking about the co-ocurrence of topics, they should point towards the figure that allows them to establish connections among terms. The authors should discuss the meaning of the different colors representing the different groups of keywords that are commonly together as per the co-ocurrence network.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Affective states in digital game-based learning: thematic evolution and social network analysis

PONE-D-21-09587R1

Dear Dr. Zou,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Mohammed Saqr, Ph.D

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #2: The authors have addressed most of my comments. I believe the quality of the manuscript has improved considerably and it can be published in its current form.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #2: No