To control the COVID-19 outbreak in Japan, sports and entertainment events were canceled and schools were closed throughout Japan from February 26 through March 19. That policy has been designated as voluntary event cancellation and school closure (VECSC).
A simple susceptible–infected–recovered model was applied to data of patients with symptoms in Japan during January 14 through March 26. The respective reproduction numbers for periods before VECSC (R0), during VECSC (Re), and after VECSC (Ra) were estimated.
Results suggest R0 before VECSC as 2.534 [2.449, 2.598], Re during VECSC as 1.077 [0.948, 1.228], and Ra after VECSC as 4.455 [3.615, 5.255].
Citation: Sugishita Y, Kurita J, Sugawara T, Ohkusa Y (2020) Effects of voluntary event cancellation and school closure as countermeasures against COVID-19 outbreak in Japan. PLoS ONE 15(12): e0239455. https://doi.org/10.1371/journal.pone.0239455
Editor: Toshiyuki Ojima, Hamamatsu Ika Daigaku, JAPAN
Received: March 13, 2020; Accepted: September 7, 2020; Published: December 21, 2020
Copyright: © 2020 Sugishita et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Japan Ministry of Health, Labour and Welfare. Press Releases. (in Japanese) https://www.mhlw.go.jp/stf/houdou/houdou_list_202001.html.
Funding: The author(s) received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
The initial case of COVID-19 in Japan was an airline passenger returning from Wuhan, China on January 3, 2020. The Ministry of Health, Labour and Welfare (MHLW) in Japan would report 3,906 cases in Japan by April 6, 2020, including asymptomatic cases but pointedly excluding those from a large cruise ship: the Diamond Princess .
Sports and entertainment events were canceled in Japan for two weeks from February 26 through March 11 in compliance with a government advisory. It had extended until March 19. At that time, it was advised that small business and private meetings be cancelled voluntarily: the measure was not enacted as a law or enforced by authorities. Therefore, people were not arrested or cited even if they did not comply with this government advisory ; the effort depended entirely on the voluntary compliance of individuals. Moreover, this was the first measure necessitating voluntary event cancellation. Those characteristics of the measures complicate the ex ante prediction of the proportion of events that were cancelled and the extent to which contact among people was reduced. Moreover, as announced March 3, almost all schools were closed from the middle of March through spring vacation (early April) as a measure to control the spread of COVID-19. Even though young people can be infected and can transmit the virus to adults, school closure effects were questionable ex ante because schoolchildren were not regarded as the most susceptible age class for COVID-19 [3–5]. Therefore, these policies must necessarily be evaluated ex post. The policies are known collectively as voluntary event cancellation and school closure (VECSC). After the VECSC period, the VECSC measures ceased, but a tacit norm of cancellation of large events continued.
If the reproduction number under the VECSC environment was less than one, then the outbreak can be expected to have been contained. Alternatively, even if the reproduction number under them were markedly less than the reproduction number before VECSC but greater than one, one could have expected instead that the outbreak would be exacerbated.
This study was conducted to evaluate VECSC based on the epidemic curve because no evidence exists to reflect the number of cancelled events. The obtained result might be expected to contribute to a government decision to implement VECSC as a countermeasure in Japan.
We applied a simple deterministic susceptible–infected–recovered (SIR) model  to the epidemic curve in Japan for its population of 120 million. We assume an incubation period following the empirical distribution from the early stage of the outbreak. A person on the i-th day of incubation will move to a symptomatic or an asymptomatic state with probability of p(i)/Σj = i Lp(j), where p(i) represents the proportion of the i-th day on the empirical distribution of the incubation period. Also, L denotes the maximum length of the incubation period. Alternatively, a person moves to the i+1 day in the incubation state with one minus p(i)/Σj = iLp(j).
For the model, symptomatic and asymptomatic states are assumed to continue for seven days, followed by a transition to a recovery state with probability of one . This model ignores outcomes and necessary medical resources: states of death and hospitalization are not incorporated into the model.
Asymptomatic cases are not observable unless complete laboratory-based surveillance is performed. One exceptional study of a sample of elderly people indicated them as 3 cases among 23 . We checked its robustness for the proportion of asymptomatic cases assuming the proportion of 4 cases among 19 from pre-outbreak experiences of Japanese residents of Wuhan .
Infectivity by severely infected patients and by mildly infected patients was assumed to be equal. Moreover, we assumed for simplicity that asymptomatic cases have the same capability of infecting others as symptomatic cases have . Therefore, we verified the model robustness through sensitivity analysis of the infectiousness of asymptomatic cases, such as 50% assumed from simulation studies of influenza infections [9–13]. The distribution of infectiousness by symptomatic and asymptomatic cases was assumed to be 30% on the onset day, 20% on the following day, and 10% for the subsequent five days . Regarding its robustness, we also applied a uniform distribution for a week.
Because VECSC policies were administered during February 26–March 19, we divided the full data period into three sub-periods: before VECSC, during VECSC, and after VECSC. Their corresponding reproduction numbers are represented as R0, Re, and Ra. We modeled the change in the reproduction number based on the prior day’s reported number of persons susceptible, incubating, symptomatic, asymptomatic, and in a recovered state.
For this study, R0 was defined as the number of newly infected persons from one person infected over the infectious period when all persons were susceptible with no countermeasures. Also, Re is defined by the numbers of newly infected persons from one person infected over the infectious period when all persons were susceptible under VECSC. Finally, Ra is defined by the numbers of newly infected persons from one person infected during a time of countermeasures after the VECSC period. For estimation, the number of newly infected persons was calculated using R0 (Re or Ra) × the proportion of susceptible persons among the population × the number of persons with infectiousness weighted by their degree of infectiousness.
The values of R0, Re, and Ra were sought to fit the data to minimize the sum of absolute values of discrepancies among the bootstrapped epidemic curve and the fitted values. The estimated relative values of the three reproduction numbers were calculated using 10,000 iterations of bootstrapping for empirical distribution of the data for symptomatic patients. In this sense, although this model is deterministic at each bootstrapping iteration, its range can be estimated through 10,000 iterations.
The bootstrapping procedure we used has fully replicated bootstrapping for a fixed number of initial cases. There were N patients in the data, with numbering of the patients from the initial case to the last case. Initially, no patient was on the bootstrapped epidemic curve. If a random variable drawn from a uniform distribution of (0,1) was t included in the internal [i/(N-1), (i+1)/(N-1)], then we added one to the onset date of the i+1th patient to the bootstrapped epidemic curve. We replicated this procedure N-1 times. Thereby, we obtained the bootstrapped epidemic curve with N-1 patients. Finally, we added the initial patient, for whom the onset date was January 14, to the bootstrapped epidemic curve. Consequently, we obtained a bootstrapped epidemic curve with N patients starting from January 14.
We estimated the curve sequentially as described hereinafter. First, we estimated R0 as the best fit to bootstrapped data for the pre-VECSC period. Then, based on the obtained R0 and on the course of the outbreak before the VECSC period, we estimated Re as the best fit to bootstrapped data in the VECSC period. Finally, based on the obtained values of R0 and Re, we estimated Ra as the best fit to bootstrapped data for the post-VECSC period. At each step, reproduction numbers were grid searched in the interval of (0,10) by 0.001.
We applied some one-way sensitivity analyses in addition to the base case as explained above: the infectious power of asymptomatic cases was 50% of the power of symptomatic cases; the proportion of asymptomatic cases was 4 /19 of symptomatic cases; infectiousness and infectious patterns showed a uniform distribution for one week. We also estimated the parameters by minimizing the sum of the squared residuals or maximizing Poisson likelihood instead of the difference in absolute values and using the fitted incubation period as a log normal distribution instead of the empirical distribution.
Moreover, we examined the possibility of infection during two days before onset. Its expected number can be represented as Σi = 2L-1 p(i)/Σj = iLp(j) x(i-1,t) +Σi = 2L-1 (1- p(i)/Σj = iLp(j)) p(i+1)/Σj = i+1Lp(j) x(i-1,t)+ x(L-1,t-), where x(i,t) denotes the number of persons at the i-th incubation period on day t. The first term, Σi = 2L-1 p(i)/Σj = iLp(j) x(i-1,t), indicates the expected number of patients who do not show symptoms day t but who will show symptoms on day t+1, except for the third term. p(i)/Σj = iLp(j) indicates probability of showing symptoms among patients who did not show symptoms until i-1 th period but will show symptom on day t +1. Similarly, the second term of Σi = 2L-1 (1- p(i)/Σj = iLp(j)) p(i+1)/Σj = i+1Lp(j) x(i-1,t) indicates the expected number of patients who do not show symptoms on day t or t +1 but who will show symptoms on day t +2. The third term, x(L-1,t), represents patients who showed no symptom until the last day of the incubation period. Therefore, they will certainly show symptoms on the following day of the last day of incubation period. Their infectiousness was assumed to be the average of the first two days of the symptomatic period. To meet the definition of the reproduction number, infectiousness in the incubation and (a) symptomatic period were adjusted such that the sum of infectiousness from two days before onset to one week following onset would be one.
We also examined incorporation of the imported cases into the model. They were entered into the model as newly symptomatic cases with no history of incubation period when the imported cases were reported. The parameters in this model were estimated as minimizing the sum of the absolute difference among the number of patients including the imported cases and prediction.
We adopted 5% as the level for which we inferred significance. We used software (Matlab 2014a; The MathWorks Inc.) to code the model as explained above.
The numbers of symptomatic patients during January 14–March 26were published by the MHLW . The data were reported publicly from 46 prefectures and some cities. They were summarized in S1 File as of April 6. During this period, 1516 cases were recorded with onset dates. We excluded imported cases and cases representing infected persons from the Diamond Princess because they were presumed not to have been community-acquired in Japan.
All information used for this study has been published elsewhere . This study therefore poses no ethical issues.
Fig 1 depicts the empirical distribution of the incubation period among 125 cases for which the exposed date and onset date were published by MHLW. Its mode was six days. The average was 6.1 days. The maximum incubation period length was 19 days.
Bars shows distribution of incubation period among 91 cases whose exposed date and onset date were published by Ministry of Health, Labour and Welfare, Japan. Because the patients whose incubation was longer than 14 days were just 4%, they were included as bars on 14.
The value of R0 before introduction of VECSC was estimated as 2.534. Its range was [2.449, 2.598]. Also, Re during the VECSC period was estimated as 1.077 [0.948, 1.228]. For the period after VECSC, Ra was estimated as 4.455 [3.615, 5.255]. The null hypothesis that R0, Re, and Ra are equal was rejected.
Fig 2 depicts the observed epidemic curve and the predicted epidemic curve based on the estimated R0, Re, and Ra. Its goodness of fit is reasonably good. The small bump visible at around February 26 reflects overshooting caused by the sharp decline of the reproduction number from R0 to Re.
Bar indicates the observed epidemic curve and bold line indicates the predicted line based on the estimated reproduction numbers. Thin lines indicate range of the best fit line at each bootstrapping iteration. Between two vertical lines indicates voluntary event cancellation and school closure period.
One-way sensitivity analysis revealed that if the infectiousness of asymptomatic cases was 50% of symptomatic cases instead of 100%, then R0 would be estimated as 2.886 [2.801, 2.970]. Also, Re would be 1.203 [1.059, 1.343]; Ra would be 4.985 [4.134, 5.893].
If the proportion of asymptomatic cases were 4 cases divided by 19 of symptomatic cases instead of 3 cases divided by 32, then R0 would be estimated as 2.049 [2.403, 2.557], Re would be 1.091 [0.972, 1.237], and Ra would be 4.428 [3.627, 5.049]. If the infectious pattern formed a uniform distribution for a week instead of the base case, then R0 would be estimated as 2.534 [2.449, 2.598], Re would be 1.077 [0.948, 1.228], and Ra would be 4.455 [3.615, 5.255]. If we introduce infectiousness in the incubation period, then R0 would be estimated as 2.877 [2.765, 2.980], Re would be 1.278 [1.085, 1.478], and Ra would be 4.687 [3.621, 5.000].
When estimating the parameters by minimizing the sum of the squared residuals instead of the absolute difference, R0 is estimated as 2.391 [2.505, 2.609], Re is 1.124 [0.935, 1.295], and Ra is 4.409 [3.168, 5.341]. In the case of using Poisson likelihood for evaluation, the parameters were estimated as 2.007 [1.957, 2.058], 1.141 [1.030, 1.237], and 3.255 [2.697, 3.922]. However, when estimating the parameters using a log normal distribution with mean of 1.66 and standard deviation of 0.558 truncated as 19 days as the incubation period instead of the empirical distribution, R0 was estimated as 2.2660 [2.179, 2.838], Re as 1.116 [0.970, 1.277], and Ra as 3.574 [2.897, 4.338].
We found 217 imported cases in this period, representing 14.3% of the community-acquired COVID-19 patients. When we incorporated the imported cases into the model, R0 was estimated as 2.502 [2.412, 2.577], Re was 1.113 [0.978, 1.298], and Ra was 4.311 [3.603, 5.103]. Because the ranges of Re and Ra were overlapped in all cases, these factors might not greatly affect both parameters, at least in the region with parameters and distribution that are considered reasonable. The estimated R0 was the most sensitive. When using Poisson likelihood for evaluation, R0 was significantly lower than in the base case. Moreover, infectiousness before onset raised R0 significantly. In other scenarios, the ranges of R0 were overlapped.
We applied a simple deterministic SIR model including asymptomatic cases. A deterministic model might be simpler but more appropriate than a stochastic model for the study including changing parameters and policy evaluation .
An earlier study [15–17] estimated R0 for COVID-19 as 2.24–3.58 in Wuhan. Our R0 obtained for the period before VECSC was similar. However, an earlier other study  found the average number of secondary cases infected by a single primary case found in contact tracing in Japan as 0.6 before the VECSC period, with 80% of patients found to have infected no one. If the average number of secondarily infected persons was less than one, then the outbreak might have stalled on its own. Given such an eventuality, no countermeasures would have been required. However, based on results of a study uploaded to the site of Japanese Public Health Association, health authorities in the public health sector or local government were urged to conduct contact tracing to detect clusters .
Infectiousness two days before symptom onset has been described in the literature [7, 20]. We applied it as sensitivity analysis. It showed slightly higher reproduction numbers than the base model, although the estimated Re and Ra were found to have a non-significant difference from the base case because a longer period of infectiousness implies fewer infected people on a day when the reproduction number does not change. However, the epidemic curve was given. Therefore, the estimated reproduction number probably increased by extending the period of infectiousness from seven days to nine days.
Moreover, when using Poisson likelihood for evaluation, R0 alone was found to be significantly lower than in the base case. Although the difference was not large, it might engender some difference attributable to selection of the evaluation function.
Among children and younger adults, the proportion of asymptomatic infected people might be larger than among elderly people, but that point remains uncertain. Lower susceptibility of children [3–5] might imply a higher proportion of asymptomatic cases in children. In this sense, the finding of 3 cases divided by 23 among elderly people might be the lower bound of the proportion for the general population. Overall, sensitivity analysis led to similar estimation with the base case. Even though significant difference was found through comparison with the base case only in R0, differences among them were not large, quantitatively speaking. Therefore, one can infer that the obtained results are robust.
The average incubation period applied in an earlier study was 5.1 days , but it was 6.1 days for the present study. However, the latter number based on the epidemic curve considered in the present study is certainly consistent with the data. It might reflect lower severity among patients in Japan than in other countries such as the US. The estimated value of the reproduction number probably would be smaller if we were to adopt a slightly shorter incubation period, which is a similar result to that found with sensitivity analysis for infectiousness before onset.
Many weeks have passed since the end of the study period. Therefore, the reporting lag almost disappeared. If timely estimation, meaning estimation using data from a day prior, was necessary, then we used an adjusted delay for reported data [8, 22, 23].
Instead of using the minimized sum of squared residuals, such as maximum likelihood estimation based on a normal distribution, we used the minimized sum of the absolute values of discrepancies for the bootstrapped epidemic curve and the fitted values. In general, minimization of the sum of the absolute values is more robust than minimization of the sum of squares because the absolute value is less sensitive than squared values to the effects of outliers [24–26]. The daily epidemic curve sometimes exhibits spikes according to the day of the week, and for other reasons. Especially, few patients per day were reported during the early stage of the outbreak. Therefore, spikes might be quite large. These spikes are probably outliers. They might overly affect the estimator. For this reason, we prefer minimization of the sum of the absolute values to minimization of the sum of squared values when analyzing daily data in the earlier stage of the outbreak.
Moreover, we chose non-parametric approaches using actual data in preference to parametric approaches assuming a particular distribution. Such an assumption might affect results through miss-specification. Therefore, we prefer to use the actual distribution of incubation periods or epidemic curves with no unnecessary or restrictive assumption. Sensitivity analysis using minimum least squares of errors or using Poisson likelihood for maximization showed very similar results to those obtained with the base case. Although R0 and Ra were slightly higher, Re were slightly lower. That result might indicate that some outlier in the pre-VECSC and post-VECSC period pulls the estimator in sensitivity analysis.
The present study has some limitations. The first is that even though we evaluated VECSC, the respective effects of voluntary event cancellation and school closure cannot be distinguished. To distinguish their respective effects, one would have to develop a model including several age classes. School closure mainly affects contact patterns among schoolchildren; voluntary event cancellation mainly affects patterns among adults. Therefore, studies of those respective age groups might elucidate the separate effects of these policies. Such studies stand as challenges for our future study.
The second point is under ascertainment. Although the epidemic curves of COVID-19 in all countries are subject to under ascertainment, it might be very difficult to evaluate the degree to which they are affected. It might bias the estimation result.
A third point is a lack of estimation of the outcomes, such as dead or severe cases or necessary medical resources for the care of COVID-19 patients. We particularly examined how policies affect the reproduction number under the effects of imposed countermeasures. Therefore, we ignored prediction of the entire course of the outbreak and its outcomes such as the number of deaths. Nevertheless, outcomes are expected to be a primary concern for modelling. Moreover, the collapse of medical services can be expected to engender worse outcomes even if the reproduction number remains unchanged. Prediction of the effects of severe policies including lockdowns is anticipated as a challenge to be addressed in our future research.
Results have demonstrated that VECSC can reduce the infectiousness of COVID-19 considerably: approximately to one. However, the figure is probably greater than one. Outbreaks might continue for a long time. Therefore, lockdown policies are expected to be as effective as VECSC if they are executed carefully. After VECSC, the reproduction number escalated again beyond that before VECSC. Similar phenomena might be observed after a lockdown. It is our earnest hope that results of the present study can contribute to governmental policy-making related to lockdown measures and other countermeasures used to combat the spread and destruction of COVID-19.
The present study was based on the authors’ opinions. Neither the results nor implications reflect any stance or policy of professionally affiliated bodies.
We acknowledge the great efforts of all staff at public health centers, medical institutions, and other facilities who are fighting the spread and destruction associated with COVID-19.
- 1. Ministry of Health, Labour and Welfare, Japan. Press Releases; 2020. Available from: https://www.mhlw.go.jp/stf/newpage_10723.html (in Japanese) [accessed on April 10, 2020]
- 2. Japan Times on February 29; 2020. Available from: https://www.japantimes.co.jp/news/2020/02/29/national/politics-diplomacy/japan-shinzo-abe-coronavirus-2/#.Xr4T9meP604 [accessed on May 15, 2020]
- 3. Lee PI, Hu YL, Chen PY, Huang YC, Hsueh PR. Are children less susceptible to COVID-19? J Microbiol Immunol Infect. 2020;533:371–2.
- 4. Kelvin A, Halperin S. COVID-19 in children: the link in the transmission chain. Lancet Infect Dis. 2020;20(6):633–4. pmid:32220651
- 5. Qiu H, Wu J, Liang H, Yunling L, Song Q, Chen D. Clinical and epidemiological features of 36 children with coronavirus disease 2019 (COVID-19) in Zhejiang, China: an observational cohort study. Lancet Infect Dis. 2020;2:689–96.
- 6. Ohkusa Y, Sugawara T, Taniguchi K, Okabe N. Real-time estimation and prediction for pandemic A/H1N1(2009) in Japan. J Infect Chemother. 2011;17:468–72. pmid:21387184
- 7. Kimball A, Hatfield KM, Arons M, James A, Taylor J, Spicer K, et al. Asymptomatic and Presymptomatic SARS-CoV-2 Infections in Residents of a Long-Term Care Skilled Nursing Facility—King County, Washington, March 2020. Morb Mortal Wkly Rep. 2020;69:377–81. pmid:32240128.
- 8. Kurita J, Sugawara T, OhkusaY. Forecast of the COVID-19 outbreak and effects of self-restraint in going out in Tokyo, Japan. medRxiv. 2020. Available from: https://doi.org/10.1101/2020.04.02.20051490
- 9. Ferguson NM, Cummings DA, Cauchemez S, Fraser C, Riley S, Meeyai A, et al. Strategies for containing an emerging influenza pandemic in Southeast Asia. Nature. 2005; 437:209–14. pmid:16079797
- 10. Longini IM Jr, Nizam A, Xu S, Ungchusak K, Hanshaoworakul W, Cummings DA, et al. Containing Pandemic Influenza at the Source. Science. 2005; 309:1083–7. pmid:16079251
- 11. Germann TC, Kadau L, Longini IM Jr, Macken CA. Mitigation strategies for pandemic influenza in the United States. Proc Natl Acad Sci USA. 2006;103: 5935–40. pmid:16585506
- 12. Ferguson NM, Cummings DA, Fraser C, Cajka JC, Cooley PC, Burke DS. Strategies for mitigating an influenza pandemic. Nature. 2006; 442: 448–52. pmid:16642006
- 13. Ohkusa Y, Sugawara T. Simulation Model of Pandemic influenza in the Whole of Japan. Jpn J Infect Dis. 2009; 62:98–106. pmid:19305048
- 14. Britton T. Stochastic epidemic models: a survey; 2009. Available from: https://arxiv.org/pdf/0910.4443. [accessed on May 16]
- 15. Zhao S, Lin Q, Ran J, Musa SS, Yang G, Wang W, et al. Preliminary Estimation of the Basic Reproduction Number of Novel Coronavirus (2019-nCoV) in China, From 2019 to 2020: A Data-Driven Analysis in the Early Phase of the Outbreak. Int J Infect Dis. 2020;92:214–7. pmid:32007643
- 16. Liu Y, Gayle AA, Wilder-Smith A, Rockly J. The reproductive number of COVID-19 is higher compared to SARS coronavirus. J Travel Med. 2020;27:taaa021. pmid:32052846
- 17. Lai CC, Shih TP, Ko WC, Tang HJ, Hsueh PR. Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) and Coronavirus disease-2019 (COVID-19): The Epidemic and the Challenges. Int J Antimicrob Agents. 2020;55:105924. pmid:32081636
- 18. Nishiura H, Oshitani H, Kobayashi T, Saito T, Sunagawa T, Matsui T, et al. Closed environments facilitate secondary transmission of coronavirus disease 2019 (COVID-19). medRxiv. 2020. Available from: https://doi.org/10.1101/2020.02.28.20029272.
- 19. Oshitani H. Concept of the Counter Measure for COVID-19; 2020. Available from: https://www.jsph.jp/covid/files/gainen_0402.pdf. (in Japanese) [accessed on My 16, 2020]
- 20. He X, Lau EHY, Wu P, Deng X, Wang J, Hao X, et al. Temporal dynamics in viral shedding and transmissibility of COVID-19. Nat Med. 2020;26:672–5. pmid:32296168
- 21. Lauer SA, Grantz KH, Bi Q, Jones FK, Zheng Q, Meredith HR, et al. The Incubation Period of Coronavirus Disease 2019 (COVID-19) From Publicly Reported Confirmed Cases: Estimation and Application. Annals of Internal Medicine. 2020;172:577–82. pmid:32150748
- 22. Höhle M, an der Heiden M. Bayesian nowcasting during the STEC O104: H4 outbreak in Germany, 2011. Biometrics. 2014;70:993–1002. pmid:24930473
- 23. Glockner S, Krause G, Hohle M. Now-casting the COVID-19 epidemic: The use case of Japan, March 2020. medRxiv. 2020. Available from: https://doi.org/10.1101/2020.03.18.20037473.
- 24. Narula SC, Wellington JF. The Minimum Sum of Absolute Errors Regression: A State of the Art Survey. International Statistical Review. 1982;50(3):317–26.
- 25. Phillips RF. Least absolute deviations estimation via the EM algorithm. Statistics and Computing. 2002;12: 281–5.
- 26. Siemsen E, Bollen KA. Least Absolute Deviation Estimation in Structural Equation Modeling. Sociological Methods & Research. 2007;36: 227–65.