https://orcid.org/0000-0002-2424-6510
Figures
Abstract
Background
One of the most recent concerns of this pandemic regards the role of schools reopening in disease transmission, as well as the impact of keeping schools closed. While school reopening seems critical for the education and mental health of children, adolescents, and adults, so far the literature has not systematically reached a consensus whether to recommend the return to schools in a way that would be safe for students and staff.
Objective
To synthesize and critically evaluate the scientific evidence on the potential risk of accelerating the Coronavirus Disease 2019 (COVID-19) pandemic among children, adolescents, young adults, and adults with school reopening.
Methods
This systematic review and meta-analysis protocol was elaborated following the PRISMA-P. We will include all observational study designs, which report on the potential risk of accelerating the COVID-2019 pandemic with school reopening. Electronic databases included were MEDLINE/PubMed, Cochrane Library, EMBASE, Web of Science, SCOPUS and CNKI. Additional sources will be also retrieved, including Clinical trials.gov-NIH, The British Library, Pro Quest Dissertations Database, Public Health Gray Literature Sources and Health Evidence, Google Scholar, and pre-prints [medRXiv]. No restriction to language or date will be used as search strategy. In an independently manner, two investigators will select studies, perform data extraction, as well as perform a critical appraisal of the risk of bias and overall quality of the selected observational studies, based on their designs. The heterogeneity among the studies will be assessed using the I2 statistic test. According to the results of this test, we will verify whether a meta-analysis is feasible. If feasibility is confirmed, a random-effect model analysis will be carried out. For data analysis, the calculation of the pooled effect estimates will consider a 95% CI and alpha will be set in 0.05 using the R statistical software, v.4.0.4. In addition, we will rate the certainty of evidence based on Cochrane methods and in accordance with the Grading of Recommendations Assessment, Development and Evaluation (GRADE).
Expected results
This systematic review and meta-analysis will provide better insights into safety in the return to school in the context of the COVID-2019 pandemic, at a time when vaccination advances unevenly in several countries around the world. Hence, consistent data and robust evidence will be provided to help decision-makers and stakeholders in the current pandemic scenario.
Citation: Lopes-Júnior LC, Siqueira PC, Maciel ELN (2021) School reopening and risks accelerating the COVID-19 pandemic: A systematic review and meta-analysis protocol. PLoS ONE 16(11): e0260189. https://doi.org/10.1371/journal.pone.0260189
Editor: Muhammad Shahzad Aslam, Xiamen University - Malaysia Campus, MALAYSIA
Received: August 16, 2021; Accepted: November 2, 2021; Published: November 17, 2021
Copyright: © 2021 Lopes-Júnior et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: The author(s) received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
1. Introduction
Globally, a total of 192,284,207 confirmed cases of the Coronavirus Disease 2019 (COVID-19) caused by the new coronavirus (2019-nCoV), and 4,136,518 deaths have occurred to date, according to the report by the World Health Organization (WHO) on July 23, 2021. Additionally, a total of 3,568,861,733 vaccine doses have been administered so far [1].
It should be highlighted that in response to the 2019-nCoV, more than 100 countries had imposed national school closures by March 2020 to reduce the transmission rate [2]. Data available from 29 countries have shown that the proportion of children among COVID-19 cases varies from 0.3% to 13.8% [3]. Furthermore, 2019-nCoV infection in children is often asymptomatic, and most cases result in mild disease and better prognosis compared to adults [4–6]. However, data on the extent of the spread of 2019-nCoV among children/adolescents in the household setting, including transmission to elderly people, for example, who are at increased risk for severe disease, are still scarce [7,8].
An observational retrospective study comprising camp attendees and 526 household contacts, which assessed the secondary transmission and associated factors, has shown that the 2019-nCoV transmission from school-aged children/adolescents to household members led to the hospitalization of adults with secondary cases of COVID-19. In addition, considering the households where transmission occurred, 50% of the household contacts were infected [9].
Although many schools were closed by March 2020, several countries worldwide have now reopened schools for in-person teaching [10]. In fact, insufficient evidence is available to support school reopening in what regards the COVID-19 transmission rate and the harms to children/adolescents [11]. A study comprising 12,000,000,000 adults in the United Kingdom (UK) has found no statistical significance regarding the risk of death from the COVID-19 in households with or without children [12,13].
Additionally, a recent infection survey using molecular tests via polymerase chain reaction [14,15] showed that around 0.5–1% of children have a positive result. Also, earlier studies, including data from five countries [16–20], where all individuals were tested regardless of symptoms, found that transmission rates were considered low, mainly among elementary school children [16–20].
Data from ecological studies showed an association between in-person teaching and the speed as well as the extent of the 2019-nCoV community transmission [21–23], although these findings were not uniform across studies [24]. Despite the occurrence of several outbreaks in schools [25–27], studies of outbreaks in places other than schools and school-like settings have pointed out that, when mitigation measures were in effect, transmission within schools was considered low; hence, infection rates represent that of the surrounding community [28,29].
Indeed, one of the most recent concerns of this pandemic is the role of schools in the transmission, as well as the impact of keeping schools closed. The exact risk for 2019-nCoV infection posed to individuals and household contacts, as well as communities, by in-person teaching in the schools is still an ongoing heated debate. While a safe reopening of schools seems feasible if mitigation measures are followed, there is a lack of robust evidence to support this return to school [30]. Modelling data undertaken by UK universities suggest that at least 30,000 more deaths from COVID-19 are estimated due to the reopening of school settings [31].
Evidence suggests that younger children (<10 y.o.) are less susceptible to infection when exposed to the virus [32]. Nonetheless, it has not been well established yet whether they are less likely to transmit the virus once infected [33,34] or whether this low susceptibility is compensated by the increased number of face-to-face contacts at school [35]. Recent evidence from a survey held in the United State (US) has demonstrated an increased risk of COVID-19 in what regarding outcomes among respondents living with a child who attends in-person teaching in school settings [30].
A recent living meta-analysis that investigated the extent of 2019-nCoV transmission in in-person teaching pointed out that overall infection attack rate and 2019-nCoV positivity rate in the school settings are low. Particularly, lower infection attack rate (IAR) and positivity rates were reported in students compared to school staff [5].
While school reopening seems pivotal to the education and mental health of children/adolescents/adults, so far, the literature has not systematically agreed on whether to recommend the return of students and staff to school in a safe manner [31]. To fully reopening schools in the setting of high community transmission, especially in view of the circulation of new variants of the new coronavirus, such as the delta variant, and with the very uneven speed of vaccination in the countries, it is essential to gather and synthesize the evidence-based studies to analyze this intervention in the pandemic scenario carefully. In this regard, gathering evidence on how 2019-nCoV transmission occurs in in-person teaching could support decision making on appropriate and safe school closure or school reopening based on robust evidence. In this study, we will aim to synthesize and evaluate the evidence on the potential risk of accelerating the COVID-19 pandemic in children, adolescents, young adults, and adults with school reopening.
2. Methods
This protocol has been elaborated following PRISMA-P [36]. Additionally, the protocol was registered with PROSPERO/UK (registration ID: CRD42021265283). The systematic review and meta-analysis will also be reported based on MOOSE guidelines [37] added to the PRISMA 2020 statement [38].
Search strategy
The search strategy will be performed in six electronic bibliographic databases: MEDLINE/PubMed (from 1947 to August 30, 2021), EMBASE (Excerpta Medica dataBASE) (from 1973 to August 30, 2021), Cochrane Library (from 1991 to August 30, 2021), Web of Science (from 1985 to August 30, 2021), SCOPUS (from 2004 to August 30, 2021), and CNKI (from 1996 to August 30, 2021). Additional sources will be also searched, including Clinical trials.gov-NIH, The British Library, Pro Quest Dissertations Database, Public Health Gray Literature, Google Scholar, and pre-prints for Health Sciences [medRXiv]. No restriction to language or date will be employed in the search strategy. Furthermore, we will also scrutinize the reference lists of articles searched to seek additional studies [39]. The PECOS (Population, Exposure/Comparison/Outcomes/Study Design) acronym [40] was used to elaborate our research question: “Which evidence is available from observational studies on school reopening, and does this have an impact on the transmission rate of Coronavirus Disease 2019 pandemic among children, adolescents, young adults, and adults?”, as depicted in Table 1.
Moreover, the EndNote™ will be used to organize as well as manage all the studies retrieved. Study selection will be carried out by two independent researchers (LCLJ and ELNM) using the Rayyan™ application, as an auxiliary tool for archiving, organizing, and selecting articles. First, we will screen for controlled descriptors, for instance, MeSH terms, Emtree terms, and their synonyms, as well as keywords will be identified. The Boolean operators “AND” and “OR” will be employed to combine the descriptors [39,41,42]. The preliminary pilot search strategy combining MeSH terms, synonyms (entry terms) as well as keywords that will be used in MEDLINE/PubMed is shown in Table 2.
Eligibility criteria
In this study, we will include all observational study designs, i.e., cross-sectional, cohort, case-control, and ecological studies. Handsearching will be carried out in the reference lists and in the gray literature seeking additional studies. Moreover, no language and date restrictions will be employed in the search strategy.
Study selection
First, all the records scrutinized from the six databases will be imported in the EndNote™. Afterwards, the duplicate articles will be removed. Two independent researchers (LCLJ and PCS) will search and screen the article records by titles and abstract using the Rayyan™ application. After the initial screening, the full text of the studies retrieved will be assessed for inclusion/exclusion by two independent researchers using Rayyan™ application. Disagreements in selected studies will be resolved by a third reviewer (ELNM). A flowchart will summarize the study selection process in line with the PRISMA 2020 statement [38] (Fig 1).
Data extraction
Reviews or qualitative studies will be excluded in this systematic review. In addition, primary studies that did not meet the inclusion criteria will be excluded according to the reasons for exclusion. The two review authors (LCLJ and PCS) will perform data extraction for each included study based on forms published previously [39,41–44]. The expected completion date for this systematic review is September 30, 2021, due to the urgency to bring this evidence to light, in the current scenario in which the delta variant is spreading rapidly in several countries.
The same two reviewers (LCLJ and PCS) will perform data extraction independently. Information to be extracted includes: a) identification of the study and objectives; b) study population and baseline characteristics; c) type of exposure; d) study methodology; e) recruitment methods; f) times of measurement; g) follow-up; h) outcomes; i) main findings; j) clinical and epidemiological significance; and k) conclusions. S1 File shows the standardized form created for data extraction.
Critical appraisal of included studies
Since this systematic review will include observational studies of different designs, the quality assessment tools should be appropriate and specific for each design. In this respect, Newcastle-Ottawa Scale (NOS) [45] will be used for evaluating the internal validity and risk of bias for cohort studies. For case-control studies, The Critical Appraisal Skills Programme (CASP) tool [46] will be employed. Regarding the cross-sectional studies, we will assess them using The Agency for Healthcare Research and Quality (AHRQ) tool [47]. In addition, the Methodological Index for Non-Randomized Studies-MINORS will be used for other study designs [48]. Two reviewers (LCLJ and ELNM) will perform the methodological assessment independently.
Data synthesis
Study characteristics will be summarized and presented in tables. Heterogeneity among studies will be measured by the I2 statistic to estimate the percentage of variation across studies, ranging from 0% to 100% [49,50], and its interpretation is: a) I2 = 0%–40% indicates low heterogeneity; b) I2 = 30%–60% shows moderate heterogeneity; c) I2 = 50%–90% represents a substantial heterogeneity; and d) I2 = 75%–100% indicates high heterogeneity [50,51].
Moreover, the subgroup analysis will be performed using a a random-effect model analysis adjusted for age, ethnicity, and sex, level of education, and non-pharmacological measures to mitigate the spread of the virus. In addition, will be carried out to explore the heterogeneity across studies. According to the I2 statistic, we will determine whether a meta-analysis is feasible [43,44,51]. For data analysis, the calculation of the pooled effect estimates will consider a 95% CI and alpha set in 0.05 using the R statistical software v. 4.0.4. Besides, we will rate the certainty of evidence based on Cochrane methods and in accordance with the Grading of Recommendations Assessment, Development and Evaluation (GRADE) [52]. The evaluation of the quality of evidence in the evaluated studies will be independently performed in a paired manner by 2 reviewers (LCLJ and PCS). Disagreements will be addressed by a third reviewer (ELNM).
Ethical aspects and dissemination plans
Because this is a systematic review protocol, and since our analysis will only include previously published data, the Institutional Review Board approval was not applicable. Moreover, the systematic review and meta-analysis will be reported following the PRISMA 2020 statement. Additionally, any amendments to this protocol will be registered. Regarding the dissemination plans, we intend to disseminate the results via peer-reviewed publication and via presentations in international conferences.
3. Limitations and strengths of this study
Two main potential limitations include the predominance of cross-sectional studies that might limit the generalizability of the results, mainly with regards to direction of cause-effect, given the study design; and the risk of type-1 error (arising from the bias in participants recruiting, confounding factors, subgroup, and sensitivity analyses of the potential studies).
However, the systematic review will provide an up-to-date synthesis of the estimation of potential risk of school reopening to accelerate the COVID-19 pandemic. Hence, a meta-analysis with reliable estimates of the prevalence rate and potential risk factors for the return to school will be provided. Finally, the results of this systematic review and meta-analysis will address this gap in the literature and support decision makers with evidence-based data to be considered in this complex context.
4. Conclusion
This systematic review and meta-analysis, to the best of our knowledge, will be the first to critically evaluate the scientific evidence and estimate the potential risk of school reopening to accelerate the COVID-19 pandemic among children, adolescents, young adults, and adults. Hence, we will provide better insights into safety in the return to school in the pandemic context, at a time when vaccination advances unevenly in several countries worldwide. Finally, we believe that this study will provide consistent evidence that will aid the decision makers and stakeholders in the current pandemic scenario.
Supporting information
S1 File. Data extraction form based on previous publications [39,41–44].
https://doi.org/10.1371/journal.pone.0260189.s002
(DOCX)
References
- 1.
WHO. WHO Coronavirus Disease (COVID-19) Dashboard. Available from: https://covid19.who.int.
- 2. Viner RM, Russell SJ, Croker H, Packer J, Ward J, Stansfield C, et al. School closure and management practices during coronavirus outbreaks including COVID-19: a rapid systematic review. Lancet Child Adolesc Health. 2020;4:397–404. pmid:32272089.
- 3. Li X, Xu W, Dozier M, He Y, Kirolos A, Theodoratou E. The role of children in the transmission of SARS-CoV2: updated rapid review. J Glob Health. 2020;10:021101. pmid:33312511.
- 4. Dong Y, Mo X, Hu Y, et al. Epidemiology of COVID-19 among children in china. Pediatrics 2020;145(6):e20200702. pmid:32179660.
- 5. Xu W, Li X, Dozier M, He Y, Kirolos A, Lang Z, et al. What is the evidence for transmission of COVID-19 by children in schools? A living systematic review. J Glob Health. 2020 Dec;10(2):021104. pmid:33437465.
- 6. Ludvigsson JF. Systematic review of COVID-19 in children shows milder cases and a better prognosis than adults. Acta Pediatr. 2020;109:1088–95. pmid:32202343.
- 7. CDC COVID-19 Response Team. Severe outcomes among patients with coronavirus disease 2019 (COVID-19)—United States, February 12–March 16, 2020. MMWR Morb Mortal Wkly Rep 2020;69(12):343–6. pmid:32214079.
- 8. Chu VT, Yousaf AR, Chang K, Schwartz NG, McDaniel CJ, Lee SH, et al. Household Transmission of SARS-CoV-2 from Children and Adolescents. N Engl J Med. 2021 Jul 21. pmid:34289272.
- 9. He X, Lau EHY, Wu P, Deng X, Wang J, et al. Temporal dynamics in viral shedding and transmissibility of COVID-19. Nat Med 2020;26:672–675. pmid:32296168.
- 10.
Royal Society DELVE Initiative. Balancing the Risks of Pupils Returning to Schools. Available from: https://rs-delve.github.io/reports/2020/07/24/balancing-the-risk-of-pupils-returning-to-schools.html#fn:49.
- 11. Buonsenso D, Roland D, De Rose C, Vásquez-Hoyos P, Ramly B, Chakakala-Chaziya JN, et al. Schools Closures During the COVID-19 Pandemic: A Catastrophic Global Situation. Pediatr Infect Dis J. 2021 Apr 1;40(4):e146–e150. pmid:33464019.
- 12. Forbes H, Morton CE, Bacon S, McDonald HI, Minassian C, Brown JP, et al. Association between living with children and outcomes from covid-19: OpenSAFELY cohort study of 12 million adults in England. BMJ. 2021 Mar 18;372:n628. pmid:33737413.
- 13. Lewis SJ, Munro APS, Smith GD, Pollock AM. Closing schools is not evidence based and harms children. BMJ. 2021 Feb 23;372:n521. pmid:33622685.
- 14.
Office for National Statistics. Coronavirus (COVID-19) infection survey, UK: 19 February 2021. Available from: https://www.ons.gov.uk/peoplepopulationandcommunity/healthandsocialcare/conditionsanddiseases/bulletins/coronaviruscovid19infectionsurveypilot/latest.
- 15. Riley S, Walters C, Wang H, Eals O, Haw D, Ainslie K, et al. REACT-1 round 9 interim report: downward trend of SARS-CoV-2 in England in February 2021 but still at high prevalence. Available from: https://spiral.imperial.ac.uk/handle/10044/1/86126.
- 16. Macartney K, Quinn HE, Pillsbury AJ, Koirala A, Deng L, Winkler N. Transmission of SARS-CoV-2 in Australian educational settings: a prospective cohort study. Lancet Child Adolesc Health 2020;4(11):807–16. pmid:32758454.
- 17. Brandal LT, Ofitserova TS, Meijerink H, Rykkvin R, Lund HM, Hungnes O, et al. Minimal transmission of SARS-CoV-2 from paediatric COVID-19 cases in primary schools, Norway, August to November 2020. Euro Surveill 2021;26:2002011. pmid:33413743.
- 18. Ulyte A, Radtke T, Abela IA, Haile SR, Berger C, Huber M, et al. Clustering and longitudinal change in SARS-CoV-2 seroprevalence in school children in the canton of Zurich, Switzerland: prospective cohort study of 55 schools. BMJ 2021; 372:n616. pmid:33731327.
- 19. Larosa E, Djuric O, Cassinadri M, Cilloni S, Bisaccia E, Vicentini M, et al. Secondary transmission of COVID-19 in preschool and school settings in northern Italy after their reopening in September 2020: a population-based study. Euro Surveill 2020;25:2001911. pmid:33303065.
- 20. Theuring S, Thielecke M, van Loon W, Hommes F, Hulso C, von der Haar , et al. SARS-CoV-2 infection and transmission in school settings during the second wave in Berlin, Germany: a cross-sectional study. medRxiv 2021.01.27.21250517 [Preprint.] pmid:34448448
- 21. Li Y, Campbell H, Kulkarni D, Harpur A, Nundy M, Wang X, et al. The temporal association of introducing and lifting non-pharmaceutical interventions with the time-varying reproduction number (R) of SARS-CoV-2: A modelling study across 131 countries. Lancet Infect Dis 2021;21: 193–202. pmid:33729915.
- 22. Flaxman S, Mishra S, Gandy A, Unwin HJT, Mellan TA, Coupland H, et al. Estimating the effects of non-pharmaceutical interventions on COVID-19 in Europe. Nature 2020; 584:257–261. pmid:32512579.
- 23. Yang B, Huang AT, Garcia-Carreras B, Hart WE, Staid A, Hitchings MDT, et al. Effect of specific non-pharmaceutical intervention policies on SARS-CoV-2 transmission in the counties of the United States. medRxiv 2020. [Preprint].
- 24. Harris DN, Ziedan E, Hanig S. The Effects of School Reopenings on COVID-19 Hospitalizations. National Center for Research on Education Access and Choice (REACH) 2021. Available from: https://www.reachcentered.org/publications/the-effects-of-school-reopenings-on-covid-19-hospitalizations.
- 25. Stein-Zamir C, Abramson N, Shoob H, Libal E, Bitan M, Cardash T, et al. A large COVID-19 outbreak in a high school 10 days after schools’ reopening, Israel, May 2020. Euro Surveill 2020; 25: 2001352. pmid:32720636.
- 26. Lopes AS, Hill M, Antezano J, Vilven D, Rutner T, Bogdanow L, et al. Transmission Dynamics of COVID-19 Outbreaks Associated with Child Care Facilities—Salt Lake City, Utah, April-July 2020. MMWR Morb Mortal Wkly Rep 2020; 69: 1319–1323. pmid:32941418
- 27. Szablewski CM, Chang KT, Brown MM, Chu VT, Yousaf AR, Anyalechi N, et al.SARS-CoV-2 Transmission and Infection Among Attendees of an Overnight Camp—Georgia, June 2020. MMWR Morb Mortal Wkly Rep 2020; 69:1023–1025. pmid:32759921
- 28. Ismail SA, Saliba V, Bernal JL, Ramsay ME, Ladhani SN. SARS-CoV-2 infection and transmission in educational settings: A prospective, cross-sectional analysis of infection clusters and outbreaks in England. Lancet Infect Dis 2021; 21: 344–353. pmid:33306981.
- 29. Falk A, Benda A, Falk P, Steffen S, Wallace Z, Høeg TB. COVID-19 Cases and Transmission in 17 K-12 Schools—Wood County, Wisconsin, August 31-November 29, 2020. MMWR Morb Mortal Wkly Rep 2021; 70: 136–140. pmid:33507890
- 30. Lessler J, Grabowski MK, Grantz KH, Badillo-Goicoechea E, Metcalf CJE, Lupton-Smith C, et al. Household COVID-19 risk and in-person schooling. Science 2021 Jun 4; 372(6546):1092–1097. Epub 2021 Apr 29. pmid:33927057.
- 31. Gurdasani D, Alwan NA, Greenhalgh T, Hyde Z, Johnson L, McKee M, et al. School reopening without robust COVID-19 mitigation risks accelerating the pandemic. Lancet 2021 Mar 27;397(10280):1177–1178. Epub 2021 Mar 10. pmid:33713595.
- 32. Viner RM, Mytton OT, Bonell C, Melendez-Torres GJ, Ward J, Hudson L, et al. Susceptibility to SARS-CoV-2 Infection Among Children and Adolescents Compared With Adults: A Systematic Review and Meta-analysis. JAMA Pediatr 2021; 175: 143–156. pmid:32975552.
- 33. Flasche S, Edmunds WJ. The role of schools and school-aged children in SARS-CoV-2 transmission. Lancet Infect Dis 2021; 21: 298–299. pmid:33306982.
- 34. Bi Q, Lessler J, Eckerle I, Lauer SA, Kaiser L, Vuilleumier N, et al. SEROCoV-POP Study Group, Household Transmission of SARS-CoV-2: Insights from a Population-based Serological Survey. medRxiv 2020.11.04.20225573 [Preprint]. 16 January 2021.doi.org/10.1101/2020.11.04.20225573.
- 35. Zhang J, Litvinova M, Liang Y, Wang Y, Wang W, Zhao S, et al. Changes in contact patterns shape the dynamics of the COVID-19 outbreak in China. Science 2020; 368: 1481–1486 (2020). pmid:32350060.
- 36. Shamseer L, Moher D, Clarke M, Ghersi D, Liberati A, Pettierew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ 2015; 349: g7647–g7647. pmid:25555855.
- 37. Stroup DF, Berlin JA, Morton SC. Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis Of Observational Studies in Epidemiology (MOOSE) group. JAMA 2000; 283 (15):2008–2012. pmid:10789670.
- 38. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021 Mar 29;372(71). pmid:33782057.
- 39. Lopes-Júnior LC, Bomfim E, Olson K, Neves ET, Silveira DSC, Nunes MDR, et al. Effectiveness of hospital clowns for symptom management in paediatrics: systematic review of randomised and non-randomised controlled trials. BMJ. 2020 Dec 16;371:m4290. pmid:33328164.
- 40. Morgan RL, Whaley P, Thayer KA, Schünemann HJ. Identifying the PECO: A framework for formulating good questions to explore the association of environmental and other exposures with health outcomes. Environ Int 2018;121(Pt 1):1027–1031. Epub 2018 Aug 27. pmid:30166065.
- 41. Gonçalves CA, Lopes-Júnior LC, Nampo FK, Zilly A, Mayer PCM, Pereira-da-Silva G. Safety, efficacy and immunogenicity of therapeutic vaccines in the treatment of patients with high-grade cervical intraepithelial neoplasia associated with human papillomavirus: a systematic review protocol. BMJ Open 2019 Jul 17;9(7):e026975. pmid:31320349.
- 42. Lopes-Júnior LC, Rosa MADRP, Lima RAG. Psychological and Psychiatric Outcomes Following PICU Admission: A Systematic Review of Cohort Studies. Pediatr Crit Care Med. 2018 Jan;19(1):e58–e67. pmid:29189670.
- 43. Lopes-Júnior LC, Bomfim E, Silveira DSCD, Pessanha RM, Schuab SIPC, Lima RAG. Effectiveness of mass testing for control of COVID-19: a systematic review protocol. BMJ Open 2020 Aug 26;10(8):e040413. pmid:32847926.
- 44. Silva Junior FJGD Sales JCES, Monteiro CFS Costa APC, Campos LRB Miranda PIG, et al. Impact of COVID-19 pandemic on mental health of young people and adults: a systematic review protocol of observational studies. BMJ Open 2020 Jul 1;10(7):e039426. pmid:32611746.
- 45. Wells GA, Shea B, O’Connell D, Peterson J, Welch V, Losos M,et al. The Newcastle-Ottawa scale (NOS) for assessing the quality of nonrandomized studies in meta-analysis. The Ottwa Hospital Research Institute. Available from: www.ohri.ca/programs/clinical_epidemiology/oxford.asp.
- 46.
Critical Appraisal Skills Programme. CASP (Case Control Study) Checklist [online] Available from: https://casp-uk.net/casp-tools-checklists/. Accessed: Date Accessed in July 24, 2021.
- 47.
Agency for Healthcare Research and Quality. Methods guide for effectiveness and comparative effectiveness reviews; AHRQ Publication 10(14)-EHC063-EF. Rockville, MD: Agency for Healthcare Research and Quality;2014.
- 48. Slim K, Nini E, Forestier D, K. wiatkowski F, Panis Y, Chipponi J. Methodological index for non-randomized studies (minors): development and validation of a new instrument. ANZ J Surg 2003;73 (9):712–6. pmid:12956787.
- 49. Higgins JPT,Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat Med jun 2002; 21(11): 1539–1558. pmid:12111919.
- 50. Higgins JPT, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in metaanalyses. BMJ sep 2003;327(7414): 557–60. pmid:12958120.
- 51. Fokoua-Maxime CD, Lontchi-Yimagou E, Cheuffa-Karel TE, Tchato-Yann TL, Pierre-Choukem S. Prevalence of asymptomatic or "silent" myocardial ischemia in diabetic patients: Protocol for a systematic review and meta-analysis. PLoS One 2021 Jun 10;16(6):e0252511. pmid:34111136.
- 52. Balshem H, Helfand M, Schünemann HJ, et al. GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol 2011; 64: 401–406. pmid:21208779.