Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Changing risk factors for developing SARS-CoV-2 infection from Delta to Omicron

Abstract

Background

One of the few studies to estimate infection risk with SARS-CoV-2 in the general population was the UK Office of National Statistics Infection Survey. This survey provided data that allowed us to describe and interpret apparent risk factors for testing positive for SARS-CoV-2 in a period when variants and COVID-19 controls experienced large changes.

Method

The ONS published estimates of likelihood of individuals testing positive in two week monitoring periods between 21st November 2021 and 7th May 2022, relating this positivity to social and behavioural factors. We applied meta-regression to these estimates of likelihood of testing positive to determine whether the monitored potential risk factors remained constant during the pandemic.

Results

Some risk factors had consistent relationship with risk of infection (always protective or always linked to higher risk, throughout monitoring period). Other risk factors had variable relationship with risk of infection, with changes seeming to especially correlate with the emergence of Omicron BA.2 dominance. These variable factors were mask-wearing habits, history of foreign travel, household size, working status (retired or not) and contact with children or persons age over 70.

Conclusion

Relevance of some risk factors to likelihood of testing positive for SARS-CoV-2 may relate to reinfection risk, variant infectiousness and status of social distancing regulations.

Introduction

Since the start of the COVID-19 pandemic in December 2019 there have been hundreds if not thousands of publications seeking to address the risk factors for deleterious COVID-19 outcomes. However, relatively few of those publications were well designed to identify risk factors for infection, whether or not symptomatic, rather than risk factors for symptomatic infection, hospitalisation and deaths. One of the more powerful studies was based on the UK Biobank cohort which analysed a cohort of almost a quarter of a million people. In that study, non-modifiable risk factors for confirmed infection in March to May 2020 (when testing in the UK was limited to persons with substantial medical need or occupational risk) included male sex, black ethnicity and socioeconomic deprivation [1]. Another comprehensive government agency early review of the evidence into disparities in risk and outcomes from COVID-19 infections in patients detected between February and April 2020 inclusive [2] found that age and ethnicity were major drivers of infection and severe outcome. These early pandemic studies and many other COVID-19 risk studies overwhelmingly related to patients with symptomatic illness if not actual severe disease.

As far as we are aware, the only community-based studies of infection as opposed to symptomatic infection that also reported on behavioural risk factors is the coronavirus infection survey in the UK conducted by the UK Office of National Statistics (CIS ONS). This was a cohort study that aimed to sample up to 150,000 individuals over the age of 2 years every fortnight [3]. The CIS ONS was the least biased epidemic tracker because it was designed to collect data on all cases, including very mild, pre-symptomatic and asymptomatic cases [4]. As such, the CIS ONS gave the most complete picture of infection spread at any one time, and risk of infection, at any severity.

ONS published its estimates of prevalence and incidence of COVID-19 weekly, the last report was in March 2023 (ONS 2023b). In addition, ONS published risk factor analyses in its “characteristics report” once or twice a month from June 2021 to November 2022 [5]. These analyses include estimates of risk by during two-week periods for amongst other things prior vaccination and infection, occupational factors, age and gender, overseas travel, and wearing of face coverings. Risk factors were presented for each of subsequent two-week periods enabling determination of association between specific factors and case status.

We report a study to determine whether risk factors for infection changed after the emergence of the omicron variant. We explored whether risk factors for infection remained the same as they were prior to the emergence of Omicron or if they changed. In addition, we wanted to know whether risk avoidance behaviours, such as mask wearing, continued to be associated with reduced risk.

Methods

All data included in this study come from publicly available analyses produced by the UK Office of National Statistics (ONS) and were published online [5]. The ONS covid survey recruited about 200,000 people and took throat swabs for COVID-19 using rt PCR every two weeks. Basic prevalence estimates were published every week and for a limited time period, the ONS published its COVID-19 characteristics analyses where they reported risk factors for testing positive in each two-week period. The outcome variable in the ONS analyses was whether or not someone tested positive and did not distinguish between whether someone had symptoms or not. The data for the analyses presented here were published on 25th May 2022 [5]. After that date, analyses changed to presenting data on risk factors for reinfection only. The analyses used here presented estimated likelihood of testing positive along with the standard error for various risk factors in sequential fortnightly periods between 21st November 2021 and 7th May 2022, as this was the period when ONS published the relevant analyses. These likelihoods were calculated by ONS from models that controlled for age, region, sex, ethnicity, deprivation, household size, multi-generational household, and urban/rural classification as well as vaccination history and history of prior infection. The UK Census as undertaken by ONS defines a mutigenerational household as “Households where people from across more than two generations of the same family live together. This includes households with grandparents and grandchildren whether or not the intervening generation also live in the household” [6].

The primary data is not publicly available, and our analyses were done on the published summary data for each fortnightly time period. We consider separately the likelihood of testing positive over the whole period, as well as the likelihood of testing positive in any single report period (n = 12 in the monitoring period). Because we do not have access to the primary data, we used meta-regression, with the metareg tool in STATA 17.0. Initial analyses were the random effects pooled likelihood of testing positive for each risk factor over the included time periods. Then an analysis for trend was undertaken by adding period number (one to 12) as a fixed effect to the model. Those risk factors with a significant trend analysis were then used in fir tree graphs generated using STATSDirect 3.6. Significance threshold was set at p < 0.05.

We discuss the results with respect to concurrent dominant variant and government epidemic control policies. The time period covered by these analyses include the final few weeks when the Delta variant was dominant in the UK (until 19 December 2021), after which Omicron variant BA.1 was dominant until 1 March 2022 and Omicron variant BA.2 was dominant until 15 June 2022 [7]. The introduction and easing of non-pharmaceutical control measures varied between the four devolved administrations that comprise the UK, but the data presented here were most influenced by concurrent controls regime in England (the most populous administrative region). The end of all COVID-19 controls in England was announced on 21 Feb 2022 [8], which stated that the legal requirement to self-isolate stopped on 24 Feb and that guidelines to self-isolate as well as free testing would cease on 1 April 2022. Because this is a secondary analysis of published aggregate data, ethical approval was not required.

Results

The meta-regression analyses are shown in Table 1. Likelihoods of testing positive allowing for selected risk factors in specific periods are shown in Figs 14. S1-S29 Figs in S1 Appendix show the pattern for those risk factors for which trend was not statistically significant at the p = 0.05 level. In this monitoring period, the risk factors not associated with any likelihood of testing positive and for which there was no significant trend were work in health or social care (p = 0.848), work in a patient facing role (p = 0.534), work in a care or nursing home (p = 0.468) and work in contact with others (p = 0.066). Variables that were associated with difference in the likelihood of testing positive but there was no significant change (p > 0.005 for trend over time) over the study period were:

thumbnail
Fig 1. Risk of testing positive for SARS-CoV-2 by household size compared to households with just a single person.

a. 2 people in household, b. 3 people in household, c. 4 people in household, d. 5 or more people in household.

https://doi.org/10.1371/journal.pone.0299714.g001

thumbnail
Fig 2. Risk of testing positive for SARS-CoV-2, with respect to ages of other household members.

a. Adults living with someone age 16 years or younger, b. Adults aged under 70 years and living with someone age 70 years or older.

https://doi.org/10.1371/journal.pone.0299714.g002

thumbnail
Fig 3. Risk of testing positive for SARS-CoV-2 when never wearing a face covering, compared to always wearing one.

a. School children in enclosed spaces, b. School children in school, c. Adults in enclosed spaces, d. Adults at school or at work.

https://doi.org/10.1371/journal.pone.0299714.g003

thumbnail
Fig 4. Risk of testing positive by rurality, retirement status or recent travel history.

a. Living in urban city or town compared to major urban area, b. Being retired compared to being employed and going to work, c. Travelled abroad in recent 28 days compared to no foreign travel in last 28 days.

https://doi.org/10.1371/journal.pone.0299714.g004

  • Gender, with females consistently less likely to test positive (likelihood -0.051, p < 0.001)
  • Being an ethnic minority was linked to less positivity (likelihood -0.119, p = 0.012)
  • People from a multi-generational household were less likely to test positive (likelihood -0.055, p = 0.025)
  • People were less likely to test positive if in the previous 28 days they had contact with a care home (likelihood –0.126, p = 0.008) or a hospital (likelihood -0.176, p < 0.001)
  • Tobacco smokers were less likely to test positive (likelihood -0.170, p < 0.001)
  • Living in a house with someone who was disabled was associated with lower risk (likelihood –0.126, p < 0.001)
  • Being disabled meant lower risk of having infection (likelihood -0.075, p = 0.005)
  • Increasing numbers of physical contacts and socially distanced contacts in the previous 7 days were also associated with increased risk. For physical contacts risk plateaued about 10 such contacts and for socially distanced contacts these plateaued about 20 such contacts.

Other risk factors for infection were not as simple to describe and were even variable during the monitoring period. There was a complex relationship between household size and positivity. People living with others, especially two or more others, had consistent increased risk of testing positive (Table 1, likelihood of testing positive increased with household size, p always < 0.001). Figs 14 show the likelihood of testing positive for selected risk factors with confidence intervals, in each of the 12 two week periods. Between the figures on each row in the figures is a chart indicating changing proportions of tested samples in each of the Delta and Omicron variants over the concurrent time period, earliest at top to latest at bottom. This may facilitate interpretation about how changes in variants changed or not with each risk factor. Fig 1 focuses on household size, with likelihoods shown compared to singleton households. The likelihoods of testing positive generally decreased for households with 3 or more members after 12 Feb 2022, but the likelihood of testing positive increased for households with size = 2 in the period from January to May 2022. The greatest decrease in likelihood of testing positive was for the largest households (n = 5+). By the end of the study period people in larger households (4 and above) had negligibly greater risk than people living in singleton households.

Although overall there was no difference in risk between people who lived with or did not live with someone aged ≤ 16 years (likelihood = 0.035, p = 0.504), there was evidence of significant change (trend over time p = 0.015). Fig 2A shows that there was a notable negative association with risk from about end February onwards. People aged under 70 years who lived with someone 70 years or older had overall lower likelihood of testing positive (likelihood = -0.097, p = 0.034), and this relationship also had a significant change over time (p = 0.021). However, detailed analysis (Fig 2B) is that those sharing households with older adults were actually only at reduced risk for the first half of the study period. By about mid-February there was little effect on risk of infection.

Compared to always wearing face coverings, school-aged children who never wore face coverings had an insignificantly higher likelihood of testing positive (Table 1: likelihood was 0.028 in enclosed spaces, p = 0.533; likelihood was 0.051 at school, p = 0.462). However, both of these comparisons had significance for trend over time. Fig 3A and 3B show that the likelihood of testing positive for never-wearers was higher (significantly so) in the early monitoring period (before end February 2022) in both settings, and much lower in the March-May period (although not usually significantly different from the always-wearers at p < 0.05).

Compared to always wearing face-coverings, adults had higher risk of testing positive (Table 1: 0.146 in enclosed spaces, p = 0.019; 0.077 at school or work, p = 0.019). Again, both of these comparisons had significance for trend over time. Fig 4C and 4D show that never wearing a face covering was much more strongly linked to positivity before early-mid January, and by end May not wearing a face covering in adults had no effect on risk for adults in these settings.

There was no evidence for trend over time for likelihood of infection in rural towns or villages (p > 0.005 in Table 1), but change over time was indicated for minor urban areas compared to major urban areas (p = 0.005). Compared to living in a major urban area, people in smaller urban locations had lower risk of testing positive during the first half of the monitored period (Fig 4A), and then greater risk of positivity which coincided with the emergence of BA.2 dominance and announcements about lifting of all COVID-19 restrictions on or around 1 March 2022.

Being retired was associated with reduced risk (Table 1, likelihood -0.127, p < 0.001) compared to those in work overall, but any protective effect had disappeared by 27 Feb 2022 (Fig 4B), which coincided with the start of the second Omicron wave.

Travel abroad was not associated any increased risk before 2 January, but after Omicron became dominant, travel abroad had a substantially increased association with risk of infection (Fig 4C), likelihood = 0.45 or 0.54 in January 2022. This elevated likelihood of testing positive linked to travel abroad subsequently declined somewhat in the Feb-May 2022 period.

Discussion

We have shown that effect of many of the posited risk or protective factors for COVID-19 infection varied during the course of the pandemic in the UK. Some factors that were associated with lower risk for testing positive during the Delta and Omicron BA.1 periods (eg., such as being retired or wearing face coverings) were no longer associated with any risk difference or even an increased risk during the second Omicron wave and after it was announced that restrictions would be lifted. Some factors that were associated in our data with increased risk of testing positive early on, such as living with someone under age 16 or children wearing face-coverings, were associated with much reduced or even no risk later on. Multi-generational households had no time trend change in risk of infection, although size of household did have a changing relationship with risk of infection over time; this suggests that multi-generational households are not reliably also the largest households. Some risk factors such as working in health or social care or in contact with others, were often found to be important in the first year of the pandemic [by July 2020: 9, 10], but were not associated with overall higher or changing risk of infection in our study period. Being an ethnic minority was strongly associated with increased risk in the first few months of the UK epidemic [by May 2020: 11], but was associated with lower risk and no significant trend change during our full monitoring period.

We are not the first investigators to find that risk factors for COVID-19 changed during the course of a within-country epidemic. In a large case-controlled study of symptomatic people with confirmed SARS-CoV-2 positivity in the USA, Hansen and colleagues (2022) [12] noted that reported contact with a known case was highly associated with infection during the Delta period but much less so for Omicron. Unlike our analysis, that study did not include any behavioural characteristics. In a study using international datasets, Sharma and colleagues (2021) [13] found that many of the non-pharmaceutical interventions that were effective in early 2020, such as school closures and gathering bans, were less effective in the winter 2020/21 wave.

The finding that wearing face coverings ceased being protective after the first Omicron wave is worthy of further discussion. It is possible that behavioural changes drove this change in part, as wearing face coverings became less expected after the announcements that COVID-19 restrictions were being lifted. Physical contact rates in all settings likely rose sharply in wake of the announcements about the plans for “Living with COVID-19”. However, behavioural changes may not explain all changes in risk factor association with infection. Biological changes such as increased infectivity with new variants or waxing and waning immunity in the population seem likely to also cause differences in risk or protective factors. The balance of evidence is that wearing face coverings reduces transmission of respiratory infections in community settings and did reduce transmission of COVID-19 (Chou et al. 2023). The question, however, is by how much. Systematic review of pre-pandemic evidence [14] and analysis of original survey data during the COVID-19 pandemic [15] both indicated that mask wearing could or did reduce transmission of SARS-CoV-2 by about 19%. But these conclusions were derived mainly from data prior to the emergence of Omicron variants.

From the data presented here, prior to Omicron BA.2, never wearing a mask was associated with an increased risk of around 30% in adults and 10% in children. But by the second Omicron wave (mid to late February 2022 onwards) there was no protective effect from mask wearing in adults and possibly an increased risk of infection in children. The need for children to wear masks in school has been a topic of considerable debate. Budzyn and colleagues [16] reported that in the summer of 2021, those US counties that did not have school mask mandates in place reported a greater increase in paediatric COVID-19 infections compared to those that did. However, this study only covered the first two weeks of the school term. In an analysis of the same data source but including data from many more counties and for a longer period of time. Chandra and Høeg (2022) [17] found that by the ninth week after school start, any association between mask mandates in schools and reduced risk of infection had disappeared. They also concluded that once confounding factors such as poverty and a Social Vulnerability Index were included, counties imposing mask mandates saw more paediatric infections.

The question remains what could be the mechanism(s) that help to explain why protective interventions that were associated with decreased risk early in the pandemic become less protective or even associated with increased risk. Similarly, we can ask why did some variables that were associated with increased risk early in the pandemic cease to be associated with risk of infection later.

We propose that two biological mechanisms were relevant. The first hypothesis is that because Omicron was more infectious than previous variants [18], interventions that had previously been able to suppress the effective reproduction number to keep it close to 1.0 were no longer as effective, thereby increasing risk of infection in circumstances where previously risk was relatively low. Our finding that the risk associated with international travel increased substantially after the appearance of Omicron supports this hypothesis. However, we note that the change in risk associated with foreign travel applies to a period when international travel was rapidly becoming easier (pandemic travel restrictions were being eased). This change in opportunity may have correlated with changes in other risk factors and behaviours not recorded. In any event, the first hypothesis does not seem likely to explain all of our findings, such as the shift from reduced to increased risk associated with wearing face coverings in school.

The second hypothesis is linked to immunity from prior infection. People who accept greater risk are more likely to have been infected and so more likely to be immune later in the pandemic and at lower risk of infection. Early in the pandemic most published epidemic models of the COVID-19 pandemic were of the SIR or SEIR form (Susceptible, Exposed, Infected and Recovered) [19]. However, these models perform badly after the first wave for infections like COVID-19 where immunity to reinfection is not long lasting [20]. As is now well known, COVID-19 infection and immunisation only provides immunity against further infection for a relatively limited period, of the order of just a few months [21]. A more appropriate epidemic model for COVID-19 at least after the first wave is the SEIRS model (Susceptible, Exposed, Infected, Recovered and Susceptible) [20]. In the SEIRS model, an epidemic in a previously naive population will pass through a series of diminishing waves until the infection becomes endemic. The COVID-19 pandemic has generally followed this path [22]. As the epidemic transitions to become endemic the drivers of infection change and then a major driver of infection rates is the rate at which immunity is lost. At this point, interventions that would have had a major impact on slowing transmission early in the epidemic phase have less influence. That non pharmaceutical interventions have value primarily in the early stages of a pandemic is something that has been known for some time. Their value lies largely in delaying most people’s infections until a suitable vaccine becomes available [23].

Ideally, we would have used the primary individual level data and undertaken some form of longitudinal panel or multilevel mixed effects regression analysis. However, we only had access to summary data for each time period. Thus, the data were effectively clustered by time period [24]. Calculating summary data for each cluster followed by standard analysis of that summary data is an acceptable approach [23]. The main weakness of this approach is that just using the central estimate does not take account of the precision of each summary value [23]. Meta-analysis is an approach that combines the results from different studies whilst accounting for the precision of each study [25]. Thus, our use of meta-regression allowed us to use the summary data from each time period whilst adjusting for the precision of each estimate.

In conclusion we have shown that risk factors for infection and effectiveness of public health interventions are not fixed but sometimes, and may commonly, change during the course of a pandemic. It is plausible and seems most likely that at least partial explanations for such changes are increased immunity in people at high risk of infection as the epidemic progressed, and the rate at which such immunity is lost or maintained by repeat exposure.

Supporting information

Acknowledgments

We acknowledge the essential contribution from the Office of National Statistics.

References

  1. 1. Ho FK, Celis-Morales CA, Gray SR, Katikireddi SV, Niedzwiedz CL, Hastie C, et al. Modifiable and non-modifiable risk factors for COVID-19, and comparison to risk factors for influenza and pneumonia: results from a UK Biobank prospective cohort study. BMJ Open. 2020;10(11):e040402. pmid:33444201
  2. 2. Public Health England. Disparities in the risk and outcomes of COVID-19 2020. Available from: https://www.gov.uk/government/publications/covid-19-review-of-disparities-in-risks-and-outcomes.
  3. 3. Office for National Statistics. Coronavirus (COVID-19) Infection Survey: England 2023 [updated Feb 10; cited 2023 Mar 9]. Available from: https://www.ons.gov.uk/peoplepopulationandcommunity/healthandsocialcare/conditionsanddiseases/datasets/coronaviruscovid19infectionsurveydata.
  4. 4. Brainard J, Lake IR, Morbey R, Jones NR, Elliot AJ, Hunter PR. Comparison of UK surveillance systems for monitoring COVID-19: Lessons for disease surveillance. The Lancet Public Health. 2023 forthcoming.
  5. 5. Office for National Statistics. Coronavirus (COVID-19) Infection Survey, characteristics of people testing positive for COVID-19, UK: 20 July 2022 2022 [updated Jul 20; cited 2023 Apr 18]. Available from: https://www.ons.gov.uk/peoplepopulationandcommunity/healthandsocialcare/conditionsanddiseases/bulletins/coronaviruscovid19infectionsurveycharacteristicsofpeopletestingpositiveforcovid19uk/20july2022.
  6. 6. ONS. Census 2021 dictionary: Office for National Statistics; 2021 [cited 2024 Jan 29]. Available from: https://www.ons.gov.uk/census/census2021dictionary.
  7. 7. Office for National Statistics. Coronavirus (COVID-19) latest insights: Infections: ONS; 2023 [updated Mar 30; cited 2023 Sep 27]. Available from: https://www.ons.gov.uk/peoplepopulationandcommunity/healthandsocialcare/conditionsanddiseases/articles/coronaviruscovid19latestinsights/infections.
  8. 8. GOV.UK. PM statement on living with COVID: 21 February 2022 2022 [updated Feb 21; cited 2023 Oct 2]. Available from: https://www.gov.uk/government/speeches/pm-statement-on-living-with-covid-21-february-2022.
  9. 9. Nguyen LH, Drew DA, Graham MS, Joshi AD, Guo C-G, Ma W, et al. Risk of COVID-19 among front-line health-care workers and the general community: a prospective cohort study. The Lancet Public Health. 2020;5(9):e475–e83. pmid:32745512
  10. 10. Mutambudzi M, Niedzwiedz C, Macdonald EB, Leyland A, Mair F, Anderson J, et al. Occupation and risk of severe COVID-19: prospective cohort study of 120 075 UK Biobank participants. Occupational and environmental medicine. 2021;78(5):307–14.
  11. 11. Niedzwiedz CL, O’Donnell CA, Jani BD, Demou E, Ho FK, Celis-Morales C, et al. Ethnic and socioeconomic differences in SARS-CoV-2 infection: prospective cohort study using UK Biobank. BMC medicine. 2020;18(1):1–14.
  12. 12. Hansen C, Perofsky AC, Burstein R, Famulare M, Boyle S, Prentice R, et al. Trends in risk factors and symptoms associated with SARS-CoV-2 and rhinovirus test positivity in King County, Washington, June 2020 to July 2022. JAMA Network Open. 2022;5(12):e2245861–e. pmid:36484987
  13. 13. Sharma M, Mindermann S, Rogers-Smith C, Leech G, Snodin B, Ahuja J, et al. Understanding the effectiveness of government interventions against the resurgence of COVID-19 in Europe. Nature Communications. 2021;12(1):5820. pmid:34611158
  14. 14. Brainard J, Jones NR, Lake IR, Hooper L, Hunter PR. Community use of face masks and similar barriers to prevent respiratory illness such as COVID-19: a rapid scoping review. Eurosurveillance. 2020;25(49):2000725. pmid:33303066
  15. 15. Leech G, Rogers-Smith C, Monrad JT, Sandbrink JB, Snodin B, Zinkov R, et al. Mask wearing in community settings reduces SARS-CoV-2 transmission. Proceedings of the National Academy of Sciences. 2022;119(23):e2119266119. pmid:35639701
  16. 16. Budzyn SE, Panaggio MJ, Parks SE, Papazian M, Magid J, Eng M, et al. Pediatric COVID-19 cases in counties with and without school mask requirements—United States, July 1–September 4, 2021. Morbidity and Mortality Weekly Report. 2021;70(39):1377. pmid:34591829
  17. 17. Chandra A, Høeg TB. Lack of correlation between school mask mandates and paediatric COVID-19 cases in a large cohort. Journal of Infection. 2022;85(6):671–5. pmid:36183909
  18. 18. Liu Y, Rocklöv J. The effective reproductive number of the Omicron variant of SARS-CoV-2 is several times relative to Delta. Journal of Travel Medicine. 2022;29(3):taac037. pmid:35262737
  19. 19. Shankar S, Mohakuda SS, Kumar A, Nazneen P, Yadav AK, Chatterjee K, et al. Systematic review of predictive mathematical models of COVID-19 epidemic. Medical Journal Armed Forces India. 2021;77:S385–S92. pmid:34334908
  20. 20. Bjørnstad ON, Shea K, Krzywinski M, Altman N. The SEIRS model for infectious disease dynamics. Nature Methods. 2020;17(6):557–9. pmid:32499633
  21. 21. Bobrovitz N, Ware H, Ma X, Li Z, Hosseini R, Cao C, et al. Protective effectiveness of previous SARS-CoV-2 infection and hybrid immunity against the omicron variant and severe disease: a systematic review and meta-regression. The Lancet Infectious Diseases. 2023. pmid:36681084
  22. 22. Schellekens P. COVID waves: Europe and US compared: Pandem-IC.com; 2023 [updated Jun 26; cited 2023 Sep 27]. Available from: https://pandem-ic.com/covid-waves-europe-and-us-compared/.
  23. 23. World Health Organization. Nonpharmaceutical interventions for pandemic influenza, national and community measures. Emerging Infectious Diseases. 2006;12(1):88. pmid:16494723
  24. 24. Kirkwood BR, Sterne JA. Essential medical statistics. 2nd ed: John Wiley & Sons; 2010.
  25. 25. Borenstein M, Hedges LV, Higgins JP, Rothstein HR. Introduction to Meta-analysis: John Wiley & Sons; 2021.