Figures
Abstract
Healthcare has been one of the most affected sectors during the coronavirus disease 2019 (COVID-19) pandemic. The utilization of related services for non-COVID-19 diseases fell dramatically following the point at which the virus broke out; however, little is known about whether this observed decline in healthcare use was due to voluntary behaviors or enforced measures. This paper quantifies the spontaneous change in healthcare utilization during the pandemic. We utilize a county-by-week-level dataset from Taiwan’s National Health Insurance (NHI) record, covering the entire Taiwanese population, and a difference-in-differences design. Our results indicate that even if there were no human mobility restrictions or supply-side constraints, people voluntarily reduced their demand for healthcare, due to fears of contagion, or COVID-related precautionary behaviors. We find that the number of outpatient visits (inpatient admissions) decreased by 19% (10%) during the pandemic period (February to May 2020). Furthermore, the demand response of healthcare for Influenza-like illness (ILI) was much greater and more persistent than for non-ILI, thereby suggesting that the substantial decline in accessing healthcare was induced by positive public health externality of prevention measures for COVID-19. Finally, we find that the demand for healthcare services did not get back to the pre-pandemic baseline, even when there were no local coronavirus cases for 253 consecutive days (mid-April to December 2020) in Taiwan.
Citation: Tsai Y-Y, Yang T-T (2022) Measuring voluntary responses in healthcare utilization during the COVID-19 pandemic: Evidence from Taiwan. PLoS ONE 17(12): e0271810. https://doi.org/10.1371/journal.pone.0271810
Editor: M. Sohel Rahman, Bangladesh University of Engineering and Technology, BANGLADESH
Received: December 20, 2021; Accepted: July 7, 2022; Published: December 8, 2022
Copyright: © 2022 Tsai, Yang. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data and code used in this paper is available at: https://github.com/yungyutsai/Taiwan_COVID19_Healthcare_Utilization.
Funding: The author(s) received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
COVID-19 has raged through most countries around the world. As of April 2022, it caused 494 million confirmed cases and 6.16 million deaths [1]. The pandemic has changed many aspects of people’s lives and had negative impacts on macroeconomic activities [2–6], household consumption [7, 8], and the labor market [9–11]. Particularly, COVID-19 is a public health crisis, and so healthcare systems have been severely affected during the pandemic. Recent studies have shown that there have been large declines in healthcare utilization for non-COVID-19 diseases [12–16]. However, it is not clear whether this observed decline is due to voluntary behaviors or to other inevitable issues, such as government restrictions on mobility [12, 14, 17] or the availability of health resources [18–20].
Measuring the voluntary response in healthcare utilization during the pandemic has important policy implications. First, the pandemic significantly strained the capacity of healthcare systems. To free up medical sources for COVID-19 patients, healthcare providers had to restrict or delay the use of services not related to COVID-19 [21]. Since such restrictions might also impede the use of some essential medical care and negatively affect people’s health, it raises a question as to what extent the government or healthcare providers have to do so. Previous studies suggest that people change their behaviors voluntarily to reduce the chance of contracting diseases [22–24]. If the spontaneous response is substantial, the government could achieve a similar outcome by implementing policies with fewer restrictions/costs. Second, compared to policy-induced behavior, voluntary responses could be more persistent. Therefore, understanding voluntary change in healthcare use can help us evaluate the possible impact of disease outbreaks on people’s health behaviors after the pandemic.
This paper fills the gap by examining the effect of the COVID-19 outbreak and pandemic responses on the voluntary demand for non-COVID-19 healthcare. We utilize a difference-in-differences (DID) design and a 2014–2020 county-by-week-level dataset from Taiwan’s National Health Insurance (NHI), covering the entire population in Taiwan. This study aims to identify the impact of the COVID-19 outbreak on healthcare demand. Given that epidemic diseases (especially influenza) are usually seasonal, simply comparing the change in health utilization before and after the pandemic would not provide a valid estimate of the causal effect of the pandemic. Therefore, we utilize the pattern of health utilization in the years without the pandemic as a comparison group, in order to account for any seasonal pattern associated with the health utilization. The DID method estimates the impact of the intervention (the COVID-19 outbreak) by quantifying the differences in changes in outcomes before and after the intervention across the treated (the year 2020) and untreated units (the year 2014–2019). Specifically, we examine whether healthcare utilization during the pandemic (2020) varied compared with corresponding weeks in previous years (2014–2019), after controlling for the county-specific trend in demand for healthcare (e.g., county-by-week fixed effects and county-by-year fixed effects).
The case of Taiwan is well-suited for this analysis—for three important reasons. First, as of the end of 2020, Taiwan had only experienced seven deaths and 799 COVID-19 cases. The total population of Taiwan is around 23 million, so the country’s infection rate is very low. Moreover, no severe local transmissions had occurred in 2020. Fig 1 displays the number of new confirmed cases for each week of 2020. The first confirmed COVID-19 case was announced in the 4th week of 2020 (i.e., the week of January 21st). The dark color represents the number of new local confirmed cases in a given week, and the gray color means the weekly number of new non-local cases. Fewer than 10% of confirmed cases are local ones, and there have been no new confirmed local cases from the 17th to 51st week of 2020 (i.e., April 12th to December 22nd, 2020). So the pandemic had a very limited impact on healthcare capacity in the country, thereby helping us rule out unmet demand due to supply-side factors.
This figure displays weekly numbers of newly confirmed COVID-19 cases in Taiwan. The dark color represents the number of new local confirmed cases in a given week, and the gray color means weekly numbers of new imported cases. Data source comes from TCDC.
Second, Taiwan did not implement any lockdown or self-isolation policies in 2020; the Taiwanese, for their part, carried on with their normal lives alongside relatively looser regulations. Therefore, our estimated change in demand for healthcare can persuasively represent the voluntary response to the COVID-19 pandemic rather than government restrictions on activity/mobility. Furthermore, Taiwan had a consecutive 253 days of no local cases from April 12th to the end of 2020 [25]. Basically, Taiwanese people returned to normal life during this period. This experience gives us a unique chance to examine whether the behavioral change in healthcare utilization can persist in a “COVID-free” period. Though Taiwan experienced a second-wave outbreak of local transmissions from May 15th to the end of June 2021, we do not include the 2021 period in our analysis.
Finally, Taiwan’s NHI is a compulsory single-payer system in which everyone has to enroll; thus, NHI data cover the population-wide healthcare utilization of both outpatient care and inpatient care. This feature allows us to investigate the pandemic effect on different types of healthcare use. Specifically, in this paper, we identify the impact on both the Influenza-like illness (ILI) diseases and non-ILI diseases.
This paper stands apart from the previous literature in the following ways. First, we contribute to the fast-growing body of literature analyzing the impacts of the COVID-19 pandemic on healthcare systems. Several recent studies indicate that there was a large decline in healthcare utilization during the pandemic period in the US [12–16], UK [26], Spain [27], and Italy [28]. Our paper provides novel evidence by showing the patterns in health utilization when there are no government restrictions on human mobility (e.g., lockdowns or stay-at-home orders) and no supply-side constraints (e.g., inadequate healthcare resources for non-COVID-19 patients).
Second, the spontaneous response to COVID-19 risk broadly relates to the “prevalence response” in the economic epidemiology literature [29–34]. This study is also related to the body of theoretical and empirical literature on voluntary avoidance behavior [35–38]. For example, several recent studies have found that people’s voluntary response plays an important role in what decisions they make in terms of mobility, social distancing, and mask-wearing during the pandemic [23, 39–43]. We contribute to this stream of the literature by quantifying the voluntary avoidance of healthcare utilization during the pandemic.
Third, this paper is related to the literature on the health effects of non-pharmaceutical interventions (NPIs). Most of the recent research points out that such interventions can indeed effectively reduce COVID-19 transmission [44–48] and might lead to unintended health benefits [49, 50]. For example, Feng et al. found that COVID-19 outbreaks and corresponding NPIs (e.g. lockdown or shelter-in-place order) substantially reduced influenza activity in China and US [50]. However, Taiwan did not implement such high-cost NPIs against COVID-19. Instead, it used relatively low-cost ones, such as wearing face masks and washing hands and relied on individual distancing efforts. We contribute to this stream of literature by showing whether the interventions based on individual responsibility can still achieve similar health benefits by reducing the incidence of non-COVID-19 diseases.
The primary aim of this paper is to examine the voluntary response in health demand during the COVID-19 outbreak. In addition, this paper also utilizes the unique “COVID-free” period in Taiwan during the second half of 2020 to examine whether and to what extent the spontaneous response persists in the post-pandemic period. Furthermore, this paper compares the heterogeneous impact of outpatient and inpatient care as well as the ILI and non-ILI diseases to decompose the potential mechanisms of the pandemic’s impact on health demand responses.
Materials and methods
Data
Our healthcare utilization data originates from the Taiwan National ILI Disease Statistics System, accessed via the Taiwan Center for Disease Control’s (TCDC) Open Data Portal [51]. This database holds NHI claim data, covering almost the entire population’s healthcare utilization. In order to investigate the outbreak of selected infectious diseases in a timely manner, the TCDC provides the public with weekly data on the numbers of outpatient visits and inpatient admissions by county, age group, and category of selected infectious diseases. Note that the definition of “week” in this database follows the World Health Organization (WHO)’s definition, which always begins on a Sunday and ends on a Saturday, but does not definitely start from January 1st.
To construct our outcome variables—the incidence rate of outpatient visits/inpatient admissions per 100,000 population for specific types of diseases—we divide the number of outpatient visits and inpatient admissions by the population of each corresponding county per year. Population information comes from the population statistics database provided by the Department of Household Registration (DHR), Ministry of Interior (MOI), Taiwan [52]. Population data are measured on a monthly basis. We use the month of the end day of each week to link the weekly day on health utilization with the monthly population data.
In our estimated sample, we also include time-varying covariates, such as county-level demographic and weather variables, that may affect health utilization in each county. We acquire demographic data, such as age structure, sex ratio, and educational attainment, from the MOI statistics portal [53]. The above demographic variables are measured on an annual basis. We use the year of the end day of each week to link the weekly day on health utilization with the annual data. We retrieve daily weather information from the Central Weather Bureau’s (CWB) observation data inquiry system [54] in order to calculate the weekly average temperature and rainfall for each county.
Sample
The estimated sample is at the weekly-county level. The sample period is from 2014 to 2020, and we use data from the first to the 52th week. Thus, the estimated sample includes 22 counties × 52 weeks × 7 years (2014 to 2020), leading to a sample size of 8,008. Furthermore, in order to examine the mechanisms behind COVID-19 effects, we categorize diseases into ILI (include influenza, non-COVID-19 pneumonia, and acute upper respiratory infections) and non-ILI.
Table 1 displays summary statistics for the outcome variables and covariates during the pre-outbreak period (i.e., the first three weeks of a year) and the post-outbreak period (i.e., the 4th to 52nd weeks of a year) in the treated year (i.e., 2020) and untreated years (i.e., 2014–2019).
Empirical specifications
The empirical challenge in identifying the causal effect of the COVID-19 outbreak is that the health utilization changes according to the seasonal patterns and holidays over a year. For example, as shown in Table 1, in the years 2014–2019, the outpatient visits rates for ILI diseases were around 2.7 thousand visits per 100 thousand people in the first three weeks. However, the visit rate dropped to 2.2 thousand visits per 100 thousand people in the fourth week and to the end of the year. This change reflects that the winter season is usually the peak of ILI diseases. Therefore, simply comparing the change in health utilization before and after the COVID-19 outbreak in 2020 might have been confounded by the disease’s seasonality.
We apply a difference-in-differences (DID) design to overcome the empirical challenges. Since the first COVID-19 case in Taiwan was reported on January 21st 2020 (i.e., the 4th week of a year), inspired by previous studies [55–57], we use 2020 as the treated year and define the 1st to 3rd weeks and 4th to 52th weeks of the year as the pre-outbreak and post-outbreak periods, respectively. To control for the seasonal pattern of healthcare demand unrelated to the COVID-19 outbreak, we use 2014–2019 as untreated years to construct the counterfactual trend of health utilization in 2020.
We first use a conventional DID design to examine the average effects of the COVID-19 outbreak on healthcare utilization, following which we extend the design to a multiple period DID and an event study design, to investigate further the dynamic trajectory of COVID-19 effects.
Conventional DID.
In order to obtain average effect of COVID-19 outbreak on healthcare utilization, we estimate the following DID specification:
(1)
Our estimation is implemented at the weekly-county level. Hidt represents the outcomes of interest, namely, numbers of outpatient visits (inpatient admissions) per 100,000 population in county i in week d of year t. We focus on three measures of healthcare utilization: 1) visits/admissions for all diseases; 2) visits/admissions for ILI diseases; and 3) visits/admissions for non-ILI diseases. Y2020 is a dummy for the treated year that takes one if the observation is in 2020, and zero otherwise. Postd is a dummy indicating the post-outbreak period (after the 4th week of a year). We include the county fixed effect λi to control for any time-invariant confounding factors at the county level. The week-of-the-year fixed effect ηd controls for seasonal patterns in healthcare utilization at the national level within a year and year fixed effect θt controls for the general trend in healthcare utilization over time. To account for any county-specific seasonal patterns or health shocks, we include the county-by-week fixed effect (λi×ηd) and the county-by-year fixed effect (λi×θt). Xidt refers to a set of covariates, including various holiday dummies (e.g., the Lunar New Year week) and county-level variables, such as age structure, sex ratio, educational attainment, average temperature, average rainfall, and county-specific linear time trend (constructed as the number of weeks from the first week of 2014). We estimate the Eq (1) using a Poisson regression since the outcome variable is weekly counts of outpatient visits/inpatient admissions. We perform our analysis with Stata 16 through the package ppmlhdfe [58].
Throughout our analysis, we use the multiway clustering approach proposed by Cameron, Gelbach, and Miller [59] to calculate standard errors clustered at both the year-week and the county levels. We adopt the multiway clustered standard error due to two concerns. First, there are potential within-group correlations of errors associated with both the county cluster and the year-week cluster. We apply the clustered standard error approach to account for this correlation pattern. Secondly, there is an overdispersion issue in our outcome variables. As shown in Table 1, the standard deviations of the outpatient visits/inpatient admissions are far exceed the squared root of the mean. Overdispersion problems in count data are sometimes caused by clustering [60]. We observe in our data that any variances in outcomes variables within the county cluster and the year-week cluster are smaller than the variances for the whole sample. Therefore, we apply the multiway clustered standard error to account for the overdispersion problems due to clustering. Finally, we conduct robustness checks on the estimates by computing the standard errors at different cluster levels.
Finally, all regressions are weighted by the monthly population size of a county. As shown in Table 1, a fairly variation in population size across counties exists. Specifically, the population size ranges from 12 thousand to 4 million. To obtain estimates that represent the population average, we weighted all of our estimations with population size. In a latter section, we also report the results on unweighted regressions as a robustness check.
The key variable used for identification in the Eq (1) is an interaction term between an indicator for the treated year Y2020 and a dummy for the post-outbreak period Postd. The coefficients of interest are γ0, measuring the difference in healthcare utilization before and after the COVID-19 outbreak in 2020 (i.e., the treated year), relative to the difference in the corresponding periods for 2014–2019 (i.e., the untreated years). γ0 can represent COVID-19 effects on healthcare utilization if the common trend assumption holds. That is, in the absence of the COVID-19 outbreak, the weekly trend in healthcare utilization should be similar in the treated and the untreated years. We examine this assumption by using the DID event study design and a set of placebo tests.
Multiple period DID.
In 2020, Taiwan had no local COVID-19 cases from April 12th to the end of 2020 (i.e., around 253 days), so the government relaxed preventive measures for COVID-19 on June 7th (i.e., the 24th week of 2020) and suggested that people could return to a normal life. Therefore, we divide the post-outbreak period into 1) pandemic period and 2) COVID-free period, and estimate the following multi-period DID specification:
(2)
Pandemicd denotes the dummy variable, which takes a value of 1 during the 4th to the 23rd weeks (the pandemic period). CovidFreed takes the value 1 during 24th to the 52nd weeks (COVID-free period). The coefficients of interest are γ1 (γ2), which measure changes in healthcare utilization during the pandemic period (COVID-free period) compared to the pre-outbreak period (the first three weeks of a year) in 2020, relative to the corresponding weeks in 2014–2019. If the COVID-19 effects faded out when there were no local cases, the γ2 should shrink to zero.
Event study.
In order to examine common trend assumption and outline the full dynamic trajectory of the COVID-19 effects, we implement an event study design by interacting the treated year dummy Y2020 with lead and lag time dummies Wd.
(3)
We use Wd, where d = −3, −1, 0, 1, 2, 3, ….46, 47, 48, to denote dummy variables for the weeks before and after the 4th week of a year. For example, W1 represents a dummy for the first week after the announcement of the first confirmed COVID-19 case. Note that we use the 2nd week of a year as the baseline week (i.e., d = −2).
The key variables used for identification in Eq (3) are a set of week dummies Wd interacted with the treated year dummy Y2020. The coefficients of interest are βd, measuring the difference in healthcare utilization between week d and the baseline week for 2020 (i.e., the treated year), relative to the difference for 2014–2019 (i.e., the untreated years). βd can represent the dynamic effects of the COVID-19 outbreak on healthcare utilization.
Results
Graphical evidence
Fig 2 displays trends in the utilization of outpatient care and inpatient care. The solid line represents the trend in 2020, and the dashed line denotes the average across 2014–2019 with a 95% confidence interval. Since the first confirmed COVID-19 case was announced in the 4th week of 2020, the vertical line in the graph separates that week from the following weeks. The vertical axis in Fig 2 stands for the percentage change in the number of outpatient visits (inpatient admissions) from the baseline week, which is the second week of each year.
A: Outpatient care: ILI diseases. B: Outpatient care: Non-ILI diseases. C: Inpatient care: ILI diseases. D: Inpatient care: Non-ILI diseases. Sample period is 2014–2020. The solid line represents the trend in 2020, and the dashed line denotes the average across 2014–2019 with a 95% confidence interval (error bar). Since the first confirmed COVID-19 case was announced in the 4th week of 2020 (i.e. the week of January 21st), the vertical line in the graph separates that week from the following weeks. The vertical axis of this figure stands for the percentage change in the number of outpatient visits (inpatient admissions) from the baseline weeks. We use the number of outpatient visits (inpatient admissions) in the second week of each year as the baseline.
Fig 2A shows the trend in outpatient visits for ILI diseases. As the flu season usually runs from October to April in Taiwan, numbers of visits for ILI diseases tend to be higher until April during 2014–2019 (see the dashed line). In a sharp deviation from the usual seasonal patterns of 2014–2019, the numbers of ILI visits fell by 30% to 65% after the COVID-19 outbreak, and the declining pattern was still persistent at the end of 2020 (see the solid line). For non-ILI diseases, Fig 2B shows that there was a large decline in visits for non-ILI diseases during the lunar new year (i.e., the 5th week of 2020) and a rebound after the holiday. This pattern can also be found in 2014–2019. The difference between 2020 and the previous years is that the average numbers of visits for non-ILI diseases in 2020 did not rebound back to the baseline level, and in fact they declined by around 15% until the middle of the year.
Fig 2C illustrates the evolution of inpatient utilization for ILI diseases. Compared to the baseline weeks, admissions for ILI diseases decreased by 15% to 60% after the virus outbreak in 2020, which is very different from the trends in 2014–2019. In contrast, Fig 2D suggests inpatient admissions for non-ILI diseases in 2020 largely followed patterns similar to those in 2014–2019 except that there was a small decline during the 6th week to the 13th week after the first COVID-19 case.
COVID-19 effects on healthcare utilization
Table 2 reports the DID estimates (i.e., the coefficient on Y2020 × Post of Eq (1)) in the form of incidence rate ratios (IRR). We display the raw estimated coefficients in Table B1 of the S2 File. The first four columns in Table 2 display results for outpatient care. We gradually include covariates to test the sensitivity of the results. Estimates across the specifications are fairly independent of the introduction of different sets of covariates and fixed effects. Our preferred specification is in Column (4) of Table 2, which includes a full set of covariates. The estimate in Column (4) of Panel A suggests that compared to the same weeks in 2014–2019, total outpatient visits during the post-outbreak period in 2020 significantly decreased by 13% (IRR = 0.87, 95% confidence interval [CI] 0.84 to 0.90). Furthermore, the estimates in Column (4) of Panels B and C show that ILI visits saw a much larger decline (i.e., 38% decrease, IRR = 0.62) than non-ILI visits (i.e., 10% decrease, IRR = 0.9).
Columns (5) to (8) display the DID estimates for inpatient care. Column (8) of Table 2 provides our preferred estimates. The estimate in Column (8) of Panel A indicates that the COVID-19 outbreak reduced total inpatient admissions by 4% (IRR = 0.96, 95% CI 0.90 to 1.01). This estimate is only one-third of the decline in outpatient care. The estimates in Column (8) of Panels B and C show the results for ILI admissions and non-ILI admissions, respectively. Compared to the trend in previous years, the number of ILI admissions declined by 40% (IRR = 0.6, 95% CI 0.54 to 0.68) during the pandemic period in 2020 (see Panel B). In contrast, there was only a negligible decrease in inpatient admissions for non-ILI diseases (see Panel C).
COVID-19 effects during and after the pandemic
The first four columns of Table 3 illustrate the results for outpatient care. We report the IRR here and include the estimated coefficients in Table B2 in the S2 File. Again, we gradually include covariates to examine the robustness of our estimates to various specifications. The DID estimates in all specifications are fairly stable. The estimate in Column (4) of Panel A suggests that compared to the same weeks in 2014–2019, total outpatient visits during the pandemic period in 2020 significantly decreased by 19% (IRR = 0.81, 95% CI 0.78 to 0.85). In addition, we find that the reduction in outpatient utilization persisted during the COVID-free period, but the estimate shrank to a 9% (IRR = 0.91, 95% CI 0.88 to 0.94) decline. The estimates in Panel B indicate that ILI visits experienced a large decline in both the pandemic period (44% decrease, IRR = 0.56) and the COVID-free period (33% decrease, IRR = 0.67). However, we find that outpatient visits for non-ILI diseases saw a 15% drop (IRR = 0.85) during the pandemic period and rebounded to a 6% reduction (IRR = 0.94) during the COVID-free period (see Panel C).
The last four columns of Table 3 show the results for inpatient care. Column (8) of Table 3 is our preferred specification. Panel A indicates that the COVID-19 outbreak did indeed significantly reduce total inpatient admissions by 10% (IRR = 0.9, 95% CI 0.84 to 0.95) in the pandemic period.
In contrast, the COVID-19 impact disappeared during the period when Taiwan had no local COVID-19 cases (IRR is close to 1 and statistically insignificant). Interestingly, the estimates in Panel B suggest that the decline in inpatient utilization for ILI diseases continued during both the pandemic period (41% decrease, IRR = 0.59) and the COVID-free period (39% decrease, IRR = 0.61). Similar to the results in Panel A, inpatient admissions for non-ILI diseased only declined during the pandemic period (8% decline, IRR = 0.92) but rebounded to pre-pandemic level in the period without local virus cases.
Dynamic effects of COVID-19 on healthcare utilization
Fig 3 highlights the IRR of estimated βd in Eq (3), which measures the dynamic effect of the COVID-19 outbreak on healthcare utilization, and the corresponding 95% confidence intervals. We displays the estimated coefficients of the event study model in Fig B1 in the S2 File. In Fig 3, the horizontal axis denotes the number of weeks from the COVID-19 outbreak (i.e., the 4th week in a year). The top (bottom) panel of Fig 3 displays the results for outpatient (inpatient) care.
A: Outpatient care: ILI diseases. B: Outpatient care: Non-ILI diseases. C: Inpatient care: ILI diseases. D: Inpatient care: Non-ILI diseases. This figure shows the IRR of the estimated βd in Eq (3). The dashed lines represent the corresponding 95% confidence intervals. The horizontal axis denotes the number of weeks from the COVID-19 outbreak (i.e., the 4th week in a year). The top (bottom) panel displays the results for outpatient (inpatient) care by disease type. Sample size is 8,008, and the sample period is 2014–2020.
Three key insights emerge from the figures. First, estimated IRR for the first three weeks of a year (i.e., d = −3, −2, −1) in all figures are close to one, suggesting that trends in the numbers of outpatient visits/inpatient admissions between the treated year (i.e., 2020) and the untreated years (i.e., 2014–2019) were in parallel before the COVID-19 outbreak. Therefore, the common trend assumption of our DID design is valid.
Second, Fig 3A indicates that the size of the reduction in visits for ILI diseases is very large. The COVID-19 outbreak reduced the utilization of outpatient care for ILI diseases by about 50% (IRR is around 0.5) within the first four weeks of the pandemic, and these effects then persisted, thereby suggesting that ILI disease visits still declined by at least 40% (IRR is around 0.6) at the end of the sample period. For visits in relation to non-ILI diseases, Fig 3B suggests that the reduction in outpatient use is relatively smaller. The number of visits for non-ILI diseases declined by 20% (IRR is around 0.8) in the 4th week after the first case was reported and rebound to pre-pandemic level in late June (i.e., the 23rd week after first COVID-19 case), because there were no local COVID-19 cases in Taiwan for around two months.
Third, Fig 3C suggests that the number of inpatient admissions for ILI diseases decreased by about 40% (IRR = 0.6) in the 4th week after the announcement of the first COVID-19 case. Consistent with outpatient care, the reduction in ILI diseases admissions never rebounded to the pre-pandemic level until the end of 2020. Fig 3D indicates that inpatient admissions for non-ILI diseases significantly declined by nearly 20% (IRR is around 0.8) when the number of COVID-19 cases accumulated quickly and reached peak (i.e., the 6th week to the 13th week after the first COVID-19 case). Interestingly, the COVID-19-induced decline in the number of admissions for non-ILI diseases dropped immediately to zero when Taiwan began to have no local case (i.e., the 14th week after the first COVID-19 case).
Placebo test and robustness checks
In this section, we first implement a series of placebo tests by excluding observations in 2020 and only using the 2014–2019 sample. Following previous studies [56, 61, 62], we randomly select one year as the pseudo “treated year” for each county and estimate Eq (1). We repeat the above procedures 1,000 times to obtain the distribution of placebo estimates. Fig 4 compares the real estimate with these placebo estimates. Our results suggest that for the outpatient care of all diseases and inpatient care of ILI diseases, the real estimates are way below the placebo ones (see Fig 4A–4C). In sum, this placebo test indicates that the significant estimates in Table 2 should be treated as causal and are not just findings made by chance.
A: Outpatient care: ILI diseases. B: Outpatient care: Non-ILI diseases. C: Inpatient care: ILI diseases. D: Inpatient care: Non-ILI diseases. We use the 2014–2019 sample and randomly select one year as the pseudo “treated year” in each county and estimate Eq (1). We repeat the above procedures 1,000 times to obtain the distribution of placebo estimates. This figure compares our real estimate with these placebo estimates. The vertical axis displays the relative frequency of the estimates. The gray bars denote the placebo estimates, and the red dashed lines denote the real ones.
We perform the same placebo tests for our multi-period DID design (Eq (2)) and event-study analysis (Eq (3)). Fig 5 displays the results for Eq (2). The red and blue dashed lines denote the real estimates for COVID-19 effects during the pandemic period and COVID-free period, respectively. The placebo test verifies that all significant estimates in Table 3 are not the result of randomness. Fig 6 illustrates the results for Eq (3). The red lines show real estimates, and the gray lines denote 1,000 placebo ones. Again, the falsification test confirms that significant estimates in the event-study analysis are unlikely to be chance findings.
A: Outpatient care: ILI diseases. B: Outpatient care: Non-ILI diseases. C: Inpatient care: ILI diseases. D: Inpatient care: Non-ILI diseases. We use the 2014–2019 sample and randomly select one year as the pseudo “treated year” in each county and estimate Eq (2). We repeat the above procedures 1,000 times to obtain the distribution of placebo estimates. This figure compares our real estimate with these placebo estimates. The vertical axis displays the relative frequency of the estimates. The red (blue) dashed line denotes the real estimates for COVID-19 effects during the pandemic period (COVID-free period). The red (blue) bars denote the placebo estimates for COVID-19 effects during the pandemic period (COVID-free period).
A: Outpatient care: ILI diseases. B: Outpatient care: Non-ILI diseases. C: Inpatient care: ILI diseases. D: Inpatient care: Non-ILI diseases. We use the 2014–2019 sample and randomly select one year as the pseudo “treated year” in each county and estimate Eq (3). We repeat the above procedures 1,000 times to obtain the distribution of placebo estimates. This figure compares our real estimates with these placebo estimates. The gray lines denote the placebo estimates, and the red dots and blue dashed lines denote the real estimates and the corresponding 95% confidence intervals.
Next, in the S2 File, we conduct several robustness checks, using various specifications. First, we calculate standard errors based on different clustering levels, in order to examine the robustness of statistical inference. In our main specification, we cluster the standard error at both the year-week and the county levels. We also conduct statistical hypothesis tests using standard errors clustered on the county or year-week levels, respectively (see Tables B3 and B4 in S2 File). We find that the statistical significance of the estimates is robust to the standard errors clustered at different levels. Second, Tables B5 and B6 in S2 File show the estimates based on unweighted Poisson regression (Eqs (1) and (2)). Our main results are robust to this change.
Discussion
Interpretation of the results
So far, we have found that the COVID-19 outbreak was associated with a substantial reduction in both outpatient and inpatient utilization. In addition, the healthcare utilization for ILI diseases experienced a much larger decline than for non-ILI diseases. Further, negative impacts peaked during the pandemic period, and then began to rebound when there were no local cases. Since COVID-19 had limited impacts on Taiwan’s healthcare system, and the government did not implement any mobility-restricted policy or close health facilities, the reduction in healthcare utilization is unlikely to have been caused by issues related to human mobility restrictions or healthcare supply.
Therefore, the decline in demand for healthcare during the pandemic period is likely to be voluntary responses, which are mixed with two effects, namely the fear effect and the prevention effect. First, people may have reduced healthcare utilization due to the fear of COVID-19 infection. As hospitals are usually a place with a high risk of contracting diseases, patients might postpone or cancel their visits if receiving medical treatment is neither necessary nor urgent: we call this mechanism the “fear effect.” Second, since ILI diseases (e.g., flu or other forms of pneumonia) shares many similarities with COVID-19 in terms of disease presentation and the ways of transmission, COVID-19 prevention measures, such as wearing a face mask, hand-washing, and social distancing, might have had an unintended effect by reducing the transmission of ILI diseases. Thus, the demand for healthcare could have decreased due to an improvement in health status. This particular mechanism is called the “prevention effect.”
In our main analysis, we find that the COVID-19 outbreak caused different impacts on healthcare utilization for ILI diseases and non-ILI diseases. Furthermore, the negative impact of the COVID-19 outbreak was most severe during the pandemic period and faded out when there was very little risk of the local spread of COVID-19 in Taiwan. Such differences help us understand the mechanisms behind healthcare demand responses to the COVID-19 outbreak. If the decline in healthcare utilization were mainly driven by the fear effect, we should expect that the COVID-19 outbreak would have had less of a negative impact on the utilization of inpatient care (i.e., more essential healthcare) than of outpatient care (i.e., less essential healthcare). In addition, the demand response to the fear of contracting COVID-19 might have disappeared when the risk of catching the virus was low.
Our results indicate that the demand response of healthcare for non-ILI diseases could have been induced by the fear effect of contracting COVID-19. For non-ILI diseases, the COVID-19 outbreak led to a 13% reduction in outpatient visits but almost no impact on inpatient admissions (i.e., insignificant 4% decrease). Furthermore, we find that the negative effect of the COVID-19 outbreak on the utilization of both outpatient care and inpatient care for non-ILI diseases faded out when no new local COVID-19 cases were reported in Taiwan. Interestingly, the event study analysis indicates that the demand response of inpatient care vanished earlier than for outpatient care (i.e., the 14th week vs. the 23rd week after the first COVID-19 case).
On the other hand, if the reduction in healthcare demand was mainly driven by the effect of COIVD-19 preventive measures, we should expect that the utilization of both outpatient and inpatient care declined similarly because these measures basically helped stop the spread of ILI diseases. In addition, the negative effect should be sizable and persistent since Taiwanese people maintain these healthy habits during the year 2020. For example, Fig A4 of the S1 File shows that the proportion of people who wear a face mask in public spaces remained at over 80% throughout the whole of 2020.
We do find that there were large declines in both outpatient visits and inpatient admissions for ILI diseases after COVID-19 outbreak and the negative effects continued even during the period when Taiwan had no local COVID-19 case. Moreover, if precautions against coronavirus indeed led to a decline in ILI-related healthcare use, we should also find corresponding evidence on improvements in health status, such as a decline in ILI mortality.
Fig 7 displays the percentage changes in the weekly number of ILI-related deaths per 100,000 population from the baseline mean (i.e., the average outcome of the second weeks in each year). The solid line represents the trend of ILI-related mortality rate in 2020. The dashed line represents the 6-year average of the outcome during 2014–2019 and the corresponding 95% confidence interval. Again, the vertical line in the graph denotes the 4th week of a year. Our results suggest that the trend for the ILI-related mortality rate in 2020 is an outlier compared to the same period in the previous 6 years. The percentage change from the baseline mean for the ILI-related mortality rate fell by 10% to 30% after the COVID-19 outbreak. This result is consistent with the findings in the recent literature on the unintended health benefits of NPIs against COVID-19 [49, 50].
This figure displays the percentage change in the weekly number of ILI-related deaths per 100,000 population from the baseline mean (i.e., the average outcome of the second weeks in each year). The solid line represents the trend of the ILI-related mortality rate in 2020. The dashed line represents the 6-year average of the ILI-related mortality rate during 2014–2019 and the corresponding 95% confidence interval (error bar). The vertical line in the graph denotes the 4th week of a year.
Implications for the observed decline in healthcare utilization
Our results indicate that outpatient visits and inpatient admissions fell by 19% and 10%, respectively, during the pandemic period. For ILI diseases, the declines in outpatient visits and inpatient admissions were even larger (i.e. more than 40% decrease). Compared to estimates in recent studies, we find that the voluntary response is substantial. For example, Birkmeyer et al. found that hospital admissions for non-COVID-19 diseases decreased by more than 20% from February to April 2020 in the US [13]. In addition, their results suggest that admissions for ILI diseases, such as urinary tract infection and pneumonia, fell by 40% to 50%.
Given the low risk of contracting COVID-19 in Taiwan, we believe our estimates could serve as a “lower bound” for voluntary healthcare utilization responses in other countries. This implies that we may treat voluntary behavior as a major reason for the observed decline in healthcare utilization. Furthermore, our results indicate that the demand for healthcare services did not get back to the pre-pandemic baseline, even after the pandemic died away in Taiwan, thereby suggesting that the COVID-19 outbreak might have had (and be having) a long-term impact on people’s health behaviors.
Implications for healthcare expenditure
Using our results, we can provide the estimated effect of the COVID-19 outbreak on NHI healthcare expenditure. Note that the average expense per outpatient visit and per inpatient admission are around 1,387 NT$ (i.e., 49.5 US$) and 63,249 NT$ (i.e., 2,258.8 US$), respectively. Based on the above information, our DID estimates from column (4) and (8) in Panel A of Table 3 suggest that the COVID-19 outbreak could have “saved” the NHI around 52.4 billion NT$ (i.e., 1.9 billion US$), which accounts for 7.3% of the annual NHI budget.
Conclusion
This paper examines the effects of the COVID-19 outbreak on voluntary demand for healthcare in Taiwan. By comparing with the same period in the years before the COVID-19 outbreak (i.e. 2014–2019), we find a large decline in health utilization after the outbreak of COVID-19. On average, the number of outpatient visits and inpatient admissions decreased by 19% and 10% during the pandemic period, respectively. Even in the period with no local coronavirus cases, outpatient visits still remains a 9% reduction. In addition, the demand response of healthcare for ILI diseases was much larger and persistent than for non-ILI diseases. Finally, we also find that the outbreak induced a 10% to 30% decline in the ILI-mortality rate.
There are two important limitations of this paper. First, our data source only provides total outpatient visits/inpatient admissions and selected infectious diseases. Due to this limitation, we cannot analyze the impact of COVID-19 on other types of healthcare, such as mental health [63]. It is possible that future researches will explore such issues after individual-level NHI claim data have been released. Second, our research design cannot untangle which prevention measures avoided the spread of ILI diseases and then reduced demand for associated healthcare. However, as universal masking has been the key element in Taiwan’s pandemic response, we believe that this practice should play an important role in this regard. Nevertheless, more studies are needed to understand the effect of different prevention measures on health utilization and health.
Supporting information
S1 File. Appendix A: Taiwan’s response to the COVID-19 pandemic.
https://doi.org/10.1371/journal.pone.0271810.s001
(ZIP)
Acknowledgments
We thank participants of the COVID-19 study group at the Institute of Economics, Academia Sinica for their valuable comments.
References
- 1.
Hannah R, Mathieu E, Rodés-Guirao L, Appel C, Giattino C, Ortiz-Ospina E, et al. Coronavirus pandemic (COVID-19). Our World in Data. 2020. Available from: https://ourworldindata.org/coronavirus.
- 2.
Atkeson A. Lockdowns and GDP: Is there a Tradeoff? Working Paper. 2020. Available from: https://drive.google.com/file/d/1uS7n-7zq5gfSgrL3S0HByExmpq4Bn3oh/view.
- 3.
Baker SR, Bloom N, Davis SJ, Kost KJ, Sammon MC, Viratyosin T. The unprecedented stock market impact of COVID-19. National Bureau of Economic Research Working Paper. 2020. Available from: https://www.nber.org/papers/w26945.
- 4.
Eichenbaum M. Bridging the COVID-19 recession. The Don Patinkin Lecture. 2020.
- 5.
Guerrieri V, Lorenzoni G, Straub L, Werning I. Macroeconomic implications of COVID-19: Can negative supply shocks cause demand shortages? National Bureau of Economic Research Working Paper. 2020. Available from: https://www.nber.org/papers/w26918.
- 6. Altig D, Baker S, Barrero JM, Bloom N, Bunn P, Chen S, et al. Economic uncertainty before and during the COVID-19 pandemic. Journal of Public Economics. 2020;191:1–13. pmid:32921841
- 7.
Baker SR, Farrokhnia RA, Meyer S, Pagel M, Yannelis C. How does household spending respond to an epidemic? Consumption during the 2020 COVID-19 pandemic. National Bureau of Economic Research Working Paper. 2020. Available from: https://www.nber.org/papers/w26949.
- 8.
Cicala S, Holland SP, Mansur ET, Muller NZ, Yates AJ. Expected health effects of reduced air pollution from COVID-19 social distancing. National Bureau of Economic Research Working Paper. 2020. Available from: https://www.nber.org/papers/w27135.
- 9.
Beland LP, Brodeur A, Wright T. COVID-19, stay-at-home orders and employment: Evidence from CPS data. IZA Discussion Paper No13282. 2020. Available from: https://www.iza.org/publications/dp/13282/covid-19-stay-at-home-orders-and-employment-evidence-from-cps-data.
- 10.
Rojas FL, Jiang X, Montenovo L, Simon KI, Weinberg BA, Wing C. Is the cure worse than the problem itself? immediate labor market effects of COVID-19 case rates and school closures in the US. National Bureau of Economic Research Working Paper. 2020. Available from: https://www.nber.org/papers/w27127.
- 11. Forsythe E, Kahn LB, Lange F, Wiczer D. Labor demand in the time of COVID-19: Evidence from vacancy postings and UI claims. Journal of Public Economics. 2020;189:1–7. pmid:32834178
- 12.
Ziedan E, Simon KI, Wing C. Effects of state COVID-19 closure policy on non-COVID-19 health care utilization. National Bureau of Economic Research Working Paper. 2020. Available from: https://www.nber.org/papers/w27621.
- 13. Birkmeyer JD, Barnato A, Birkmeyer N, Bessler R, Skinner J. The impact of the COVID-19 pandemic on hospital admissions in the United States: Study examines trends in US hospital admissions during the COVID-19 pandemic. Health Affairs. 2020;39(11):2010–2017. pmid:32970495
- 14. Chatterji P, Li Y. Effects of the COVID-19 pandemic on outpatient providers in the United States. Medical Care. 2021;59(1):58–61. pmid:33136711
- 15.
Mehrotra A, et al. The Impact of the COVID-19 Pandemic on Outpatient Visits: A Rebound Emerges. To the Point. 2020;.
- 16. Hartnett KP, Kite-Powell A, DeVies J, Coletta MA, Boehmer TK, Adjemian J, et al. Impact of the COVID-19 pandemic on emergency department visits—United States, January 1, 2019–May 30, 2020. Morbidity and Mortality Weekly Report. 2020;69(23):699–704. pmid:32525856
- 17. Sands LP, Albert SM, Suitor JJ. Understanding and addressing older adults’ needs during COVID-19. Innovation in Aging. 2020;4(3):1–3. pmid:32613086
- 18. Søreide K, Hallet J, Matthews J, Schnitzbauer A, Line P, Lai P, et al. Immediate and long-term impact of the COVID-19 pandemic on delivery of surgical services. The British Journal of Surgery. 2020;107(10):1250–61. pmid:32350857
- 19.
Hamel L, Kearney A, Kirzinger A, et al. KFF Health tracking poll-May 2020: Impact of coronavirus on personal health, economic and food security, and medicaid. San Francisco, CA: The Henry J. Kaiser Family Foundation; 2020.
- 20. Rossman H, Meir T, Somer J, Shilo S, Gutman R, Arie AB, et al. Hospital load and increased COVID-19 related mortality in Israel. Nature Communications. 2021;12(1):1–7. pmid:33771988
- 21. Yang H, Sürer Ö, Duque D, Morton DP, Singh B, Fox SJ, et al. Design of COVID-19 staged alert systems to ensure healthcare capacity with minimal closures. Nature Communications. 2021;12(1):1–7. pmid:34145252
- 22. Watanabe T, Yabu T. Japan’s voluntary lockdown: further evidence based on age-specific mobile location data. The Japanese Economic Review. 2021;72(3):333–370. pmid:34177343
- 23. Yan Y, Malik AA, Bayham J, Fenichel EP, Couzens C, Omer SB. Measuring voluntary and policy-induced social distancing behavior during the COVID-19 pandemic. Proceedings of the National Academy of Sciences. 2021;118(16). pmid:33820846
- 24. Cuthbertson H. Filling the gap: Grassroots mask making in California during the Coronavirus pandemic. Journal of Emergency Management (Weston, Mass). 2021;18(7):119–126. pmid:34723353
- 25.
Wang C, Ellis S. Taiwan achieves record 200 days with no local virus cases. Time [Internet] 2020 Oct 29. [cited 2022 Mar 24]. Available from: https://time.com/5905129/taiwan-coronavirus-record/.
- 26. Abbas MJ, Kronenberg G, McBride M, Chari D, Alam F, Mukaetova-Ladinska E, et al. The early impact of the COVID-19 pandemic on acute care mental health services. Psychiatric Services. 2021;72(3):242–246. pmid:33076794
- 27. Gómez-Ramiro M, Fico G, Anmella G, Vázquez M, Sagué-Vilavella M, Hidalgo-Mazzei D, et al. Changing trends in psychiatric emergency service admissions during the COVID-19 outbreak: Report from a worldwide epicentre. Journal of Affective Disorders. 2021;282:26–32. pmid:33387743
- 28. Borrelli E, Grosso D, Vella G, Sacconi R, Querques L, Zucchiatti I, et al. Impact of COVID-19 on outpatient visits and intravitreal treatments in a referral retina unit: let’s be ready for a plausible “rebound effect”. Graefe’s Archive for Clinical and Experimental Ophthalmology. 2020;258(12):2655–2660. pmid:32960319
- 29. Ahituv A, Hotz VJ, Philipson T. The responsiveness of the demand for condoms to the local prevalence of AIDS. Journal of Human Resources. 1996;31(4):869–97.
- 30. Gersovitz M, Hammer JS. Infectious diseases, public policy, and the marriage of economics and epidemiology. The World Bank Research Observer. 2003;18(2):129–57.
- 31. Lakdawalla D, Sood N, Goldman D. HIV breakthroughs and risky sexual behavior. The Quarterly Journal of Economics. 2006;121(3):1063–1102.
- 32. Paula ÁD, Shapira G, Todd PE. How beliefs about HIV status affect risky behaviors: Evidence from Malawi. Journal of Applied Econometrics. 2014;29(6):944–64.
- 33. Delavande A, Kohler HP. The impact of HIV testing on subjective expectations and risky behavior in Malawi. Demography. 2012;49(3):1011–36. pmid:22744765
- 34. Bennett D, Chiang CF, Malani A. Learning during a crisis: The SARS epidemic in Taiwan. Journal of Development Economics. 2015;112:1–18. pmid:32287877
- 35. Perra N, Balcan D, Gonçalves B, Vespignani A. Towards a characterization of behavior-disease models. PLoS ONE. 2011;6(8):e23084. pmid:21826228
- 36. Funk S, Salathé M, Jansen VA. Modelling the influence of human behaviour on the spread of infectious diseases: a review. Journal of the Royal Society Interface. 2010;7(50):1247–1256. pmid:20504800
- 37. Rubin GJ, Amlôt R, Page L, Wessely S. Public perceptions, anxiety, and behaviour change in relation to the swine flu outbreak: cross sectional telephone survey. British Medical Journal. 2009;339:1–8.
- 38. Bayham J, Kuminoff NV, Gunn Q, Fenichel EP. Measured voluntary avoidance behaviour during the 2009 A/H1N1 epidemic. Proceedings of the Royal Society B: Biological Sciences. 2015;282(1818):20150814. pmid:26511046
- 39. Gupta S, Simon K, Wing C. Mandated and voluntary social distancing during the COVID-19 epidemic. Brookings Papers on Economic Activity. 2020;2020(2):269–326.
- 40.
Chudik A, Pesaran MH, Rebucci A. Voluntary and mandatory social distancing: Evidence on COVID-19 exposure rates from Chinese provinces and selected countries. National Bureau of Economic Research Working Paper. 2020. Available from: https://www.nber.org/papers/w27039.
- 41. Farboodi M, Jarosch G, Shimer R. Internal and external effects of social distancing in a pandemic. Journal of Economic Theory. 2021;196:105293.
- 42.
Allcott H, Boxell L, Conway JC, Ferguson BA, Gentzkow M, Goldman B. What explains temporal and geographic Variation in the early US coronavirus pandemic? National Bureau of Economic Research Working Paper. 2020. Available from: https://www.nber.org/papers/w27965.
- 43.
Herby J. A first literature review: Lockdowns only had a small effect on COVID-19. Available from: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3764553.
- 44.
Lin Z, Meissner CM. Health vs. wealth? public health policies and the economy during COVID-19. National Bureau of Economic Research Working Paper. 2020. Available from: https://www.nber.org/papers/w27099.
- 45. Chernozhukov V, Kasaha H, Schrimpf P. Causal impact of masks, policies, behavior on early COVID-19 pandemic in the US. Journal of Econometrics. 2021;220(1):23–62. pmid:33100476
- 46. Mitze T, Kosfeld R, Rode J, Wälde K. Face masks considerably reduce COVID-19 cases in Germany: A synthetic control method approach. Proceedings of the National Academy of Sciences. 2020;117(51):32293–301.
- 47. Abouk R, Heydari B. The immediate effect of COVID-19 policies on social distancing behavior in the United States. Public Health Reports. 2021 Mar;136(2):245–52. pmid:33400622
- 48.
Dave DM, Friedson AI, Matsuzawa K, Sabia JJ, Safford S. Were urban cowboys enough to control COVID-19? Local shelter-in-place orders and coronavirus case growth. National Bureau of Economic Research Working Paper. 2020. Available from: https://www.nber.org/papers/w27229.
- 49.
Qi J, Zhang D, Zhang X, Yin P, Liu J, Pan Y, et al. Do lockdowns bring about additional mortality benefits or costs? Evidence based on death records from 300 million Chinese people. 2020. Available from: https://www.medrxiv.org/content/10.1101/2020.08.28.20183699v3.
- 50. Feng L, Zhang T, Wang Q, Xie Y, Peng Z, Zheng J, et al. Impact of COVID-19 outbreaks and interventions on influenza in China and the United States. Nature Communications. 2021;12(1):1–8. pmid:34059675
- 51.
Taiwan Centers for Disease Control. Taiwan CDC open data portal [Internet]. 2022 [updated 2022 Mar 24; cited 2022 Mar 24]. Available from: https://data.cdc.gov.tw.
- 52.
Department of Household Registration, Ministry of Interior. Population demographics statistics [Internet]. 2022 [updated 2022 Mar 24; cited 2022 Mar 24]. Available from: https://www.ris.gov.tw/app/portal/346.
- 53.
Ministry of Interior. Statistics inquiry website [Internet]. 2022 [updated 2022 Mar 24; cited 2022 Mar 24]. Available from: https://statis.moi.gov.tw/micst/stmain.jsp?sys=100.
- 54.
Central Weather Bureau. CWB Observation Data Inquire System [Internet]. 2022 [updated 2022 Mar 24; cited 2022 Mar 24]. Available from: https://e-service.cwb.gov.tw/HistoryDataQuery/index.jsp.
- 55. Berniell I, Facchini G. COVID-19 lockdown and domestic violence: Evidence from internet-search behavior in 11 countries. European Economic Review. 2021;136:103775. pmid:35721306
- 56. Tanaka T, Okamoto S. Increase in suicide following an initial decline during the COVID-19 pandemic in Japan. Nature Human Behaviour. 2021;5:229–238. pmid:33452498
- 57. Chang H, Chang K, Fan E. The intended and unintended effects of drunk driving policies. Oxford Bulletin of Economics and Statistics. 2020;82(1):23–49.
- 58. Correia S, Guimarães P, Zylkin T. Fast Poisson estimation with high-dimensional fixed effects. The Stata Journal. 2020 Mar;20(1):95–115.
- 59. Cameron AC, Gelbach JB, Miller DL. Robust inference with multiway clustering. Journal of Business & Economic Statistics. 2012;29(2):238–49.
- 60. Poortema K. On modelling overdispersion of counts. Statistica Neerlandica. 1999;53(1):5–20.
- 61. Leslie E, Wilson R. Sheltering in place and domestic violence: Evidence from calls for service during COVID-19. Journal of Public Economics. 2020;189:104241. pmid:32834179
- 62. Heft-Neal S, Burney J, Bendavid E, Voss KK, Burke M. Dust pollution from the Sahara and African infant mortality. Nature Sustainability. 2020;3(10):863–871.
- 63. Hampshire A, Hellyer PJ, Soreq E, Mehta MA, Ioannidis K, Trender W, et al. Associations between dimensions of behaviour, personality traits, and mental-health during the COVID-19 pandemic in the United Kingdom. Nature Communications. 2021;12(1):1–15.