On November 24th 2005, the Government of England and Wales removed regulatory restrictions on the times at which licensed premises could sell alcohol. This study tests availability theory by treating the implementation of Licensing Act (2003) as a natural experiment in alcohol policy.
An interrupted time series design was employed to estimate the Act’s immediate and delayed impact on violence in the City of Manchester (Population 464,200). We collected police recorded rates of violence, robbery, and total crime between the 1st of February 2004 and the 31st of December 2007. Events were aggregated by week, yielding a total of 204 observations (95 pre-, and 109 post-intervention). Secondary analysis examined changes in daily patterns of violence. Pre- and post-intervention events were separated into four three-hour segments 18∶00–20∶59, 21∶00–23.59, 00∶00–02∶59, 03∶00–05∶59.
Analysis found no evidence that the Licensing Act (2003) affected the overall volume of violence. However, analyses of night-time violence found a gradual and permanent shift of weekend violence into later parts of the night. The results estimated an initial increase of 27.5% between 03∶00 to 06∶00 (ω = 0.2433, 95% CI = 0.06, 0.42), which increased to 36% by the end of the study period (δ = −0.897, 95% CI = −1.02, −0.77).
This study found no evidence that a national policy increasing the physical availability of alcohol affected the overall volume of violence. There was, however, evidence suggesting that the policy may be associated with changes to patterns of violence in the early morning (3 a.m. to 6 a.m.).
Citation: Humphreys DK, Eisner MP, Wiebe DJ (2013) Evaluating the Impact of Flexible Alcohol Trading Hours on Violence: An Interrupted Time Series Analysis. PLoS ONE 8(2): e55581. doi:10.1371/journal.pone.0055581
Editor: John E. Mendelson, California Pacific Medicial Center Research Institute, United States of America
Received: September 26, 2012; Accepted: January 3, 2013; Published: February 15, 2013
Copyright: © 2013 Humphreys et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: The authors are grateful for support from Economic and Social Research Council (ESRC) who funded a 1+3 Doctoral Studentship grant. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Violence and aggressive behaviour has a well established association with alcohol consumption , , . This relationship has been shown to be causal in some laboratory settings , but in the natural environment, alcohol-related violence is thought to be mediated by a complex web of personality, cultural and situational factors , . Throughout history, societies have developed regulatory controls that restrict the times and places at which alcohol can be sold, in an attempt to reduce social disorder . These measures are consistent with ‘availability theory’, proposing that the greater availability of alcohol in the population, the greater the prevalence of problems stemming from alcohol consumption . In recent years, many western governments have relaxed restrictions on the physical availability of alcohol  and, in some cases, implemented policies that increase public access to alcohol . Sudden changes to regulations governing when and where alcohol can be sold can provide a unique opportunity to study the relationship between alcohol availability and violence .
At the turn of the millennium, the growing prevalence of alcohol related problems led to increased public and political concern in England and Wales. A Government report estimated the annual cost of alcohol related harm to exceed twenty billion pounds, with 60% (£12 billion) being attributed to the costs of alcohol-related crime and disorder . At around the same time, a Home Office White Paper (‘Time for Reform’) was published recommending radical changes to regulatory framework governing alcohol availability . In 2003 these proposals were accepted and formalised in The Licensing Act (2003) (hereafter ‘the Act’), which was designed to release the leisure trade from unnecessary bureaucracy, to instil a safer drinking culture, and to reduce crime and disorder .
Contrary to traditional restrictive controls, the Act proposed to reduce crime by removing restraints, rather than by adding them. The rationale used to justify these changes evolved from three reports observing overcrowding and other late-night bottlenecks (e.g. at food outlets and taxi ranks) at fixed closing times (i.e. 23∶00 and 02∶00), which were believed to fuel violence and disorder , , . These reports suggested that removing fixed closing times would help to stagger crowd dispersal from licensed premises, reducing violent behaviour as a consequence . In November 2005 the Act was implemented, removing restrictions on trading hours for alcohol outlets, thus potentially increasing the physical availability of alcohol. Under the principles of availability theory, a significant increase in the availability of alcohol should lead to increased rates of physical and social harm. This study uses the Act as an opportunity to evaluate a government policy that proposed to reduce violence by increasing the availability of alcohol. To do this we examined whether and how trends in violence changed in a large city in northern England after implementation of the Act.
Effects of Increased On-premise Trading Hours on Violent Crime: Previous Research
Several reviews have summarised the impact of changes to on-premise trading hours on alcohol-related harm , , , . Overall, empirical evidence supporting the availability hypothesis seems particularly strong for road traffic harm and excessive alcohol consumption . However, fewer studies have evaluated the impact of extended trading hours on violent behaviour.
A recent study found only six studies evaluating the impact of trading hour extensions on violent behaviour, some of which offer support for the availability hypothesis . In the same study, researchers predicted a 17% increase in violent assault for every one hour of extended opening in a sample of Norwegian cities . A study conducted in Australia found a statistically significant increase in violent assault around premises with extended trading permits , and a 31% increase in emergency room attendances on weekends was observed in Iceland when restrictions on closing hours were removed . Alternatively, however, a British study that examined the impact of a one hour extension in trading times found no significant impact on reported violence . On the whole, the evidence from current reviews offers some support for the hypothesis that increased hours of sale result in greater violent harm, however the overall picture lacks consistency .
The review also included studies examining the impact of the Licensing Act (2003) on violence . However, their synthesis included only three of the ten evaluations available to date. To examine the results of the evidence in its entirety, we have summarised all studies examining the impact of on-premise availability extensions (see Table S1). Four studies found no significant change to patterns of violence following the implementation of the Act , , , . In line with availability theory, three studies reported statistically significant increases in violence , , . Conversely, three studies found reductions in violence, contradicting availability theory (whilst offering partial support for the Government rationale) , , . In addition, several studies found empirical evidence of changes to the hourly distribution of violence in the post-intervention period. Consistent with findings from international literature , , these studies show patterns of violence spreading later into the evening , , , .
To date, there are no randomised controlled trials of changes to licensed trading times, therefore the evidence base relies heavily on opportunistic studies from several natural experiments around the world. The lack of consistency within the literature may be attributed to various methodological problems inherent within complex alcohol policy evaluations , , . These studies tend to employ weak methodological designs that lack suitable control conditions. Even in cases where comparison conditions have been incorporated, the proximity of treatment and control areas make contamination a problem. Furthermore, many studies are retrospective and therefore unable to conduct process evaluations detailing the implementation of new measures. Until now, only two studies have employed more robust non-randomised designs such as interrupted time series modelling, neither of which were applied to the Licensing Act , . In the absence of prospectively designed randomised control studies, more robust nonrandomised studies are required to help generate evidence.
Materials and Methods
We treated the removal of trading hour restrictions that resulted from the Act as a natural experiment in order to test two competing hypotheses: first, that flexible trading hours would lead to a reduction in levels of violence (as predicted by the Labour Government ); and secondly, that flexible trading hours would increase levels of violence (as predicted by availability theory ). Each hypothesis was tested using an interrupted time series (ITS) design, which has been recommended as a suitable evaluation design in cases where it is difficult to find appropriate control conditions , , , , . Using a period of time as the unit of analysis, ITS designs use multiple measures of a pre- and post-intervention outcome variable to estimate intervention effects.
We obtained recorded crime incident data from Greater Manchester Police between 1st of February 2004 to the 31st of December 2007. In order to generate measures comparable to other evaluations , , , , we recoded crime into categories defined by the 2008 Home Office counting rules . The data included detailed information on the date and time of the incident, which made it possible to aggregate ‘violence against the person’ to weekly units; this was used as the primary dependent variable. The long study period enabled us to generate a series of 204 weekly time points (95 pre- and 109 post-intervention), which exceed minimum recommended sample size (i.e 50 observational units) for ARIMA impacts assessments , , . We separated our analyses by weekday (Sunday 12∶00 p.m. to Friday 11∶59 a.m.) and weekends (Friday 12∶00 p.m. to Sunday 11∶59 a.m.) due to well known differences in routine patterns of public alcohol consumption , . Furthermore, in order to examine changes to the temporal distribution of violence we performed separate analyses on individual time segments. We separatedq night-time hours into four segments: 18∶00–20∶59, 21∶00–23.59, 00∶00–02∶59, 03∶00–05∶59.
A common limitation of natural experimental studies is the failure to construct plausible counterfactual conditions, thus making it difficult to take account of historical confounding factors . This is a problem common to studies examining policy interventions which are implemented simultaneously across a population . A partial solution to this problem is to include non-equivalent dependent variables , , –variables not expected to respond to the intervention, but exposed to the same historical validity threats . In this study we used two non-equivalent variables to control for confounding factors, these were: robbery, and total crime. Although it is possible that many types of criminal behaviour could be associated with alcohol consumption, studies of drug use in English and Welsh arrestees calculated attribution fractions for alcohol in robbery (13%) and total crime (22%), which were far lower than for violence (37%) . To our knowledge there is no evidence that robbery or total crime rates are associated with changes in closing hours to the same extent as violent assault. Variations in these types of crime are therefore less likely to change as a result of the Act, but are potentially responsive to cyclical factors that may confound the Act’s impact on violence. The annual frequencies for these variables are presented in Table 1.
In an attempt to identify any confounding events occurring during the study period that might serve as plausible alternative explanations for our findings ,  (i.e. historical threats to validity), we researched local events that occurred between 2004 and 2008 and found two such factors. The first was the Smoking Ban of 2007, which prohibited smoking within licensed premises, causing individuals to crowd in streets. The second was a series of short-term Alcohol Misuse Enforcement Campaigns (AMECs), during which extra police and local authority enforcement resources targeted alcohol related disorder. Each was included within our analytical structure.
We tested our hypothesis using interrupted autoregressive integrated moving average (ARIMA) models , , . For each time series, conventional methods were used to identify the nature of the time series. Logarithmic transformations and differencing was applied when necessary to achieve time series that were normally distributed and stationary in level and variance. Autocorrelation functions (ACF) and partial autocorrelation functions (PACF) of the time series and of the residuals from each ARIMA model were used to identify evidence of seasonality and to test model fit.
Transfer functions were then incorporated into the ARIMA models to perform the impact assessment component of the analysis. Each time series was modelled using three types of transfer functions to test for evidence that the date of the licensing act was associated with either an abrupt permanent, a gradual permanent, or an abrupt but temporary impact on violence (or other crime) , . We used this strategy because a process evaluation (reported elsewhere, ) found considerable variation in the application of extended trading hours across the study area. These analyses found that most (63%) premises changed their hours of trade on the implementation date. However, a smaller proportion (15.6%) changed trading times throughout the post-implementation period, potentially delaying the onset of effects. Due to this, the Act’s effect could be abrupt or gradual, and therefore it was necessary to estimate each of the transfer functions to determine the most accurate model.
The implementation of the Act, which was introduced at week 96 of the study period (November 24th, 2005), was represented as a dummy variable (or step function) coded 0 through week 95 and 1 thereafter. As recommended in other studies , , we also modelled the intervention onset for the 95th and 97th weeks to account for time that might have elapsed before practices were changed. For brevity, these models are only reported where significant effects are detected.
The zero-order transfer function estimates the impact (ω) at the onset of the intervention. Therefore, if the Act had an abrupt one-dimensional effect, the zero-order transfer function would be an adequate model. However, it may be the case that although trading hour restrictions were lifted instantly following the Act’s implementation, intervention effects may not have been immediately observable. Because it is not clear that an abrupt permanent effect should be expected, two additional transfer functions were used to explore alternatives.
This model estimates the change at onset (ω) as well as a rate value (δ) enabling the identification of dynamic impacts. Using the rate parameter, the asymptotic change (ω/1−δ) can be calculated (i.e. the magnitude of change at the end of the study period). The rate parameter (δ) alone indicates how quickly the overall change was achieved. A value of zero means change was achieved instantly following the onset of an intervention; a value of 1 indicates that the impact was diffused slowly throughout the study period.
As the third alternative for each time series, the Act was modelled as a first-order transfer function applied to a differenced (pulse) intervention variable, thereby testing for the possibility of an abrupt, temporary (1.3) effect:(1.3)
Table 1 reports the incidence of violence, robbery, and total crime annually over the four-year study period. The weekly time series of violence shown in Figure 1(a) does not reveal any visual evidence of a distinct disruption to the violence trend following the implementation of the Act in November 2005. When subjected to more rigorous impact assessment, the ARIMA results presented in Table 2 confirm an absence of any significant effects. Estimates using a zero-order transfer function found a non-significant increase of 12% (ω = 0.781, 95% CI = −0.06, 0.17) in violence at the onset of the intervention period (percent changes are calculated using (eωzt –1)100). Furthermore, there was no evidence of a gradual-permanent, or abrupt-temporary effect from models (b) and (c). In these analyses and also in the analyses reported below, parameter estimates derived through maximum likelihood and exact algorithms were indistinguishable.
Weekly trends in (A) violence, (B) robbery, and (C) total crime, between 2004 and 2008 in the City of Manchester.
We extended the search for possible omitted control variables by analysing the impact of the Act on crimes for which it was not expected to have an impact. This is an important validation step as it examines whether some unobserved third variable might have operated on crimes other than violence, hence masking the effect of the intervention. However, Figure 1 (B–C) and Table 3 reveal no evidence of any external shock having an impacted on total crime (−0.09%, ω = 0.001, 95% CI = −0.05, 0.05), or robbery (−8%, ω = −0.084, 95% CI = −0.21, 0.04).
Then, to extend the analysis of the time series of violence, we incorporated two policy interventions that may confound the estimation of the Act’s impact: the smoking ban, and a series of AMECs. The smoking ban was entered into the model as a step variable, whereas the AMECs were entered as a series of pulse variables occurring throughout the pre- and post-implementation period. These policy changes occurred close to the change in opening hours. Their omission could therefore bias the estimation of the intervention effects in question. When added to the violence model, neither of these variables were associated with changes to violence (or other crime), and the introduction of these variables had no impact on the model estimates. Similarly, no changes to model estimates occurred when these variables were introduced to the models for total crime, robbery, or criminal damage.
Impact on Late Night Patterns of Violence
A number of previous studies found evidence of increased violence later in the evening, which could have numerous implications for services that provide support for the night-time economy (i.e. police, emergency services, transport) , , , , . In these analyses the aim was to evaluate whether more subtle changes to night-time patterns of violence were observable following the Act’s implementation. This was done by disaggregating night-time patterns of violence into smaller three-hour segments, and conducting ARIMA intervention analysis for each segment.
The results are presented in Table 3 and Figure 2. For weekdays, ARIMA modelling found no significant changes to violence throughout the evening, and therefore we have not tabulated these results. The findings for weekends are largely supportive of the previous findings and show no statistically significant changes in violence between 6 p.m. and 3 a.m. However, consistent with several other evaluations , , , we found a statistically significant increase in early morning violence between 3 a.m. and 6 a.m.
Weekly trends in (A) violence between 3 a.m. and 6 a.m., and (B) the proportion of asymptotic change in violence in the post-intervention period.
Specifically, Figure 2(A) shows a series of spikes in the weeks immediately after the Act’s implementation, followed by a subtle increase in the average level of violence. The apparent increase is confirmed in Table 3, where the model for a first-order transfer function shows a significant gradual and permanent impact on violence. The results estimate an initial increase of 27.5% (ω = 0.2433, 95% CI = 0.06, 0.42) at the onset of the intervention. However, the asymptotic change parameter shows that the level of violence continues to increase to a logged rate of 2.36 (asymptotic change = ω/(1−δ))–a 36% increase overall. The significant rate parameter (δ = −0.897, 95% CI = −1.02, −0.77) indicates that the asymptotic level of violence was reached gradually during the post-implementation period. Specifically, this suggests that an increase of 36% overall would predict that the implementation of the Act was ultimately responsible for an additional 3 violent incidents per week between 3 a.m. and 6 a.m. A graph depicting the rate of the impact is presented in Figure 2 (B), showing that the impact was much steeper in the initial weeks following the implementation, but remained significantly higher than pre-implementation levels thereafter.
This study used a natural experimental design to analyse the impact of the removal of trading restrictions on violent behaviour in the City of Manchester. Consistent with other evaluations , , , , , our findings show no significant changes to the overall level of violence following the implementation of the Licensing Act (2003). Analysis of night-time violence found evidence of a statistically significant 36% increase in violence between 3 a.m. and 6 a.m. These findings add to a growing body of evidence showing that extensions in hourly sales of alcohol may contribute to small yet meaningful changes in late night violence.
These findings also suggested that the Act’s impact was not abrupt, but gradual and permanent. The validity of this finding is strengthened when we consider the findings of a previous study  which showed that the Act’s implementation followed a pattern of diffusion similar to that shown in Figure 2(b). Taken together, these findings offer further evidence to reject the government-proposed hypothesis that removing restrictions on trading times would lead to decreased levels of violence. Whilst critics of the Act will argue that the absence of reduced violence is unsurprising in light of the extensive literature associating availability to increased harm , , it is somewhat surprising that greater increases in violence were not observed in this or several other evaluations of the Act (Table S1). Such counterintuitive findings should prompt further investigation of why greater alcohol availability leads to increased violence in some contexts and not in others .
Like other studies evaluating the impact of the Licensing Act, this study has a number of strengths and limitations. The ARIMA model approach is arguably a more rigorous method of assessing policy impact than other approaches that have evaluated the Act to date. In this study we explicitly modelled other policy changes that we believe could have had an impact on violence rates during the study period, although it possible that unknown and unmeasured confounding may still exist. We also investigated potential model misspecification by examining whether any change occurred in crime indicators where no impact of the Licensing Act was to be expected. In the absence of overall effects, we also investigated whether more subtle changes to night-time violence were evident. Unlike many previous evaluations, this study was adequately powered to estimate immediate and delayed impacts of the Act.
Power was enhanced in our study by using time series aggregated by week. Relative to monthly time series that we created for comparison to explore this issue, the weekly time series reported above had proportionately better model fits and, hence, supported more powerful hypothesis tests. For example, whereas our weekly time series of late-night violence (3 a.m. to 6 a.m.) suggested that the Act had a gradual permanent impact as evidenced by statistically significant parameter estimates for the transfer function (Table 3), a monthly aggregation of the same data yielded a statistically significant denominator estimate but a numerator estimate that was small in comparison (ω = 0.0475) and included the null value (95% CI = −0.07, 0.12). Also the Q statistic, used to gauge model residuals for randomness at 24 lags, was 18.1 for the monthly time series but 12.0 for the weekly time series. Ultimately then, reliance on monthly data would have led to a different, less defensible conclusion.
The reliance on police-recorded crime data means that these results are susceptible to the ‘dark figure’ of unreported crime , . In order to address this, it would have been desirable to supplement the data with injury records from accident and emergency departments, but these data were unavailable. Furthermore, these analyses investigate the impact of extended trading on a macro-level and do not attempt to isolate the relationship between changes to closing times and changes to local rates of violence. As previous studies have shown , the implementation of the extended trading hours differed across cities, meaning that smaller geographic units are likely to have experienced different doses of the intervention. It is conceivable that violence increased (or decreased) in areas where extended trading increased, and reduced in areas where trading contracted, thus averaging out effects when examined at the macro level.
A further limitation of this study is its generalisability. Under ideal circumstances, it would have been desirable to design a prospective multisite study, collecting detailed population data on alcohol consumption patterns, licensed trading times and both self-reported and recorded rates of violence and injury. However, given the nature of the Act’s implementation , the considerable resources this would require, and the lack of detailed routine data, such a design was deemed infeasible. Complex policy interventions like this are rarely able to deal with all threats to external validity , and in many cases, generalisable causal inference will rely on the replication of natural experiments and the synthesis of this combined evidence . By increasing attention to the rigor and internal validity of this evaluation, this study provides further evidence that the Act failed to reduce violence (as was suggested by the Labour government), and may have contributed to additional problems by spreading violence later into the early hours of the morning.
The findings from this and other evaluations of the Act have implications both for policymakers and the research community. These findings serve as a reminder that preventive policy may not always have its anticipated affect on behaviour, and when conceived without proper attention to available scientific evidence, may even cause harmful side effects. These findings suggest some evidence of negative side effects on early morning violence resulting from the Act’s implementation. However, we still know very little about whether an absence of dramatic effects on overall violence is evident in other alcohol-related harms. Future research needs to move beyond black-box evaluative designs by investigating the impact of policy on exposure (e.g. alcohol availability) as well as the impact of exposure on multiple indicators of physical and social harm. In England and Wales, a vast improvement in the provision of routine data on alcohol availability and consumption is required to further our understanding of the relationship between alcohol availability, alcohol misuse and its related harms.
Summary of Evaluations of Extended Trading. This table provides a summary of the findings of 13 evaluations of interventions extending the availability of alcohol.
Conceived and designed the experiments: DKH MPE. Performed the experiments: DKH MPE DJW. Analyzed the data: DKH MPE DJW. Contributed reagents/materials/analysis tools: DKH MPE DJW. Wrote the paper: DKH MPE DJW.
- 1. Pernanen K (1991) Alcohol in Human Violence. New York: Guilford Publications.
- 2. Bushman BJ, Cooper HM (1990) Effects of alcohol on human aggression: An intergrative research review. Psychological Bulletin 107: 341.
- 3. Raistrick D, Hodgson R, Ritson B, editors (1999) Tackling alcohol together: The evidence base for a UK alcohol policy. London: Free Association Books.
- 4. Bushman BJ (1997) Effects of alcohol on human aggression: Validity of proposed experiments. In: Galanter M, editor. Recent Developments in Alcoholism: Alcoholism and Violence. New York: Plenum Press. 227–243.
- 5. Lipsey MW, Wilson DB, Cohen MA, Derzon JH (1997) Is there a causal relationship between alcohol use and violence? In: Galanter M, editor. Recent developments in alcoholism. New York: Plenum Press. 245–282.
- 6. Graham K, Leonard KE, Room R, Wild TC, Pihl RO, et al. (1998) Current directions in research on understanding and preventing intoxicated aggression. Addiction 93: 659–676.
- 7. Babor T, Caetano R, Casswell S, Edwards G, Giesbrecht N, et al.. (2010) Alcohol: no ordinary commodity, research and public policy.
- 8. Stockwell T, Gruenewald P (2003) Controls on the Physical Availability of Alcohol. In: Heather N, Stockwell T, editors. The Essential Handbook of Treatment and Prevention of Alcohol Problems. Chichester: John Wiley & Sons Ltd. 213.
- 9. Hadfield P, editor (2009) Nightlife and Crime: Social Order and Governance in International Perspective. Oxford: Oxford University Press.
- 10. Popova S, Giesbrecht N, Bekmuradov D, Patra J (2009) Hours and days of sale and density of alcohol outlets: impacts on alcohol consumption and damage: a systematic review. Alcohol and Alcoholism 44: 500.
- 11. Campbell DT (1969) Reforms as experiments. American Psychologist 24: 409–429.
- 12. The Cabinet Office Strategy Unit (2003) Alcohol misuse: how much does it cost. Prime Minister's Strategy Unit, London.
- 13. Home Office (2000) Time for reform: Proposals for the modernisation of our licensing laws (White paper). London: Home Office.
- 14. Humphreys DK, Eisner MP (2010) Evaluating a natural experiment in alcohol policy: The Licensing Act (2003) and the requirement for attention to implementation. Criminology & Public Policy 9: 41–67.
- 15. Hope T (1986) Liquor Licensing and Crime Prevention. Home Office Research and Planning Unit Research Bulletin 20: 5–8.
- 16. Tuck M (1989) Drinking and disorder: a study of non-metropolitan violence. London: Home Research & Planning Unit 116 p.
- 17. Marsh P, Kibby K (1992) Drinking and public disorder: Portman Group.
- 18. Stockwell T, Chikritzhs T (2009) Do relaxed trading hours for bars and clubs mean more relaxed drinking? A review of international research on the impacts of changes to permitted hours of drinking. Crime Prevention & Community Safety 11: 153–170.
- 19. Middleton JC, Hahn RA, Kuzara JL, Elder R, Brewer R, et al. (2010) Effectiveness of policies maintaining or restricting days of alcohol sales on excessive alcohol consumption and related harms. American journal of preventive medicine 39: 575–589.
- 20. Carpenter C, Dobkin C (2010) Alcohol regulation and crime: University of Chicago Press.
- 21. Rossow I, Norström T (2012) The impact of small changes in bar closing hours on violence. The Norwegian experience from 18 cities. Addiction 107: 530–537.
- 22. Chikritzhs T, Stockwell T (2002) The Impact of Later Trading Hours for Australian Public Houses (Hotels) on Levels of Violence. Journal of Alcohol Studies: 591–599.
- 23. Ragnarsdottir P, Kjartansdottir A, Davidsdottir S (2002) Effect of extended alcohol serving-hours in Reykjavik. In: Room R, editor. The effects of Nordic alcohol policies: What happens to drinking and harm when alcohol controls change. Helsinki: NAD Publication.
- 24. Duffy JC, Pinot De Moira AC (1996) Changes in licensing law in England and Wales and indicators of alcohol-related problems. Addiction Research & Theory 4: 245–271.
- 25. Newton A, Armitage A, Hirschfield A, Monchuk L, Rogerson M, et al.. (2008) Evaluation of Licensing Act: Measuring Crime and Disorder in and around Licensed Premises. Huddersfield: University of Huddersfield Applied Criminology Centre.
- 26. Durnford A, Perkins T, Perry J (2008) An evaluation of alcohol attendances to an inner city emergency department before and after the introduction of the UK Licensing Act 2003. BMC public health 8: 379.
- 27. Babb P (2007) Violent Crime, Disorder and Criminal Damage Since the Introduction of the Licensing Act 2003. London: Home Office.
- 28. Pike S, O'Shea J, Lovbakke J (2008) Early experiences of the Licensing Act 2003 in the east of England and Yorkshire and the Humber regions. London: The Home Office.
- 29. Newton A, Sarker SJ, Pahal GS, Bergh Evd, Young C (2007) Impact of the new UK licensing law on emergency hospital attendances: a cohort study. Emergency Medical Journal: 532–534.
- 30. Jones LA, Goodacre S (2010) Effect of 24-h alcohol licensing on emergency departments: the South Yorkshire experience. Emergency Medicine Journal 27: 688–691.
- 31. Peirce BH, Boyle AA (2011) How has the Licensing Act (2003) changed the epidemiology of assaults presenting to a Cambridgeshire emergency department? Before and after study. European Journal of Emergency Medicine 18: 351.
- 32. Bellis MA, Anderson Z, Hughes K, Messages K (2006) Effects of the Alcohol Misuse Enforcement Campaigns and the Licensing Act 2003 on Violence: A preliminary assessment of A & E attendances in Wirral. Liverpool: North-west Regional Alcohol Strategic Group. Centre for Public Health, Liverpool John Moores University.
- 33. Kirby S, Hewitt L (2011) The impact of the Licensing Act 2003 on drinking habits, offences of crime and disorder, and policing in England's newest city. Safer Communities 10: 31–38.
- 34. El-Maaytah M, Smith SF, Jerjes W, Upile T, Petrie A, et al. (2008) The effect of the new “24 hour alcohol licensing law” on the incidence of facial trauma in London. British Journal of Oral and Maxillofacial Surgery 46: 460–463.
- 35. Shadish W, Cook T, Campbell D (2002) Experimental and quasi-experimental designs for generalized causal inference. Boston Houghton-Mifflin.
- 36. Cook TD, Campbell DT (1979) Quasi-experimentation: Design & Analysis Issues for Field Settings. Chicago: Rand McNally & Co.
- 37. McCleary R, Hay R, Meidinger EE, McDowall D, Land KC (1980) Applied time series analysis for the social sciences: Sage Publications Beverly Hills, CA.
- 38. McDowall D, McCleary R, Meidinger EE, Hay RA (1980) Interrupted time series analysis. Thousand Oaks, Califronia: SAGE Publications Inc.
- 39. Hough M, Hunter G (2008) The 2003 Licensing Act's impact on crime and disorder: An evaluation. Criminology and Criminal Justice 8: 239.
- 40. Home Office (2008) Home Office counting rules for recorded crime. London: Home Office.
- 41. Tabachnick BG, Fidell LS (2006) Chapter 18: Time Series Analysis. In: Tabachnick B, Fidell L, editors. Using Multivariate Statistics. 5th ed. Boston: Pearson International Edition.
- 42. McCain LJ, McCleary R (1979) The statistical analysis of the simple interrupted time series quasi-experiment. In: Cook TD, Campbell DT, editors. Quasi-experimentation: Design & Analysis Issues for Field Settings. Chicago: Rand McNally & Co.
- 43. Wikstrom P (1990) Urban crime, criminals, and victims: The Swedish experience in an Anglo-American comparative perspective. New York: Springer-Verlag Publishing.
- 44. Eisner M, Wikström P (1999) Violent crime in the urban community: A comparison of Stockholm and Basel. European Journal on Criminal Policy and Research 7: 427–442.
- 45. Mercer SL, DeVinney BJ, Fine LJ, Green LW, Dougherty D (2007) Study Designs for Effectiveness and Translation Research: Identifying Trade-offs. American journal of preventive medicine 33: 139–154.e132.
- 46. Ross HL (1973) Law, Science, and Accidents: The British Road Safety Act of 1967. The Journal of Legal Studies 2: 1–78.
- 47. Coryn CLS, Hobson KA (2011) Using nonequivalent dependent variables to reduce internal validity threats in quasi-experiments: Rationale, history, and examples from practice. New Directions for Evaluation 2011: 31–39.
- 48. The Cabinet Office Strategy Unit (2003) Alcohol misuse: how much does it cost - Technical Annex. London: The Cabinet Office.
- 49. Cochran WG, Kempthorne O, Tukey JW (2006) Observational studies: Iowa State University.
- 50. Box GEP, Taio GC (1975) Intervention analysis with applications to economic and environmental problems. Journal of the American Statistical Association 70: 70–79.
- 51. Cryer JD, Chan KS (2008) Time series analysis: with applications in R. New York: Springer Verlag.
- 52. Romelsjö A (2010) Effects of Changes in Availability of Alcohol: Unexpected Results Can Stimulate Theory Development and Research. Alcohol and Alcoholism 45: 468–469.
- 53. Coleman C, Moynihan J (1996) Understanding crime data: Haunted by the dark figure. Buckingham: Open University Press.
- 54. Biderman AD, Reiss AJ (1967) On Exploring the “Dark Figure” of Crime. The ANNALS of the American Academy of Political and Social Science 374: 1–15.
- 55. Humphreys DK (2010) Do flexible opening hours reduce violent crime? An evaluation of the Licensing Act (2003). Cambridge: University of Cambridge.
- 56. Craig P, Cooper C, Gunnell D, Haw S, Lawson K, et al.. (2011) Using natural experiments to evaluation population health interventions: guidance for producers and users of evidence. London: Medical Research Council.