RQ1.

Did police.uk crime statistics affect property prices? We test this indirectly using snap-point data only.

RQ2.

What is the effect of a one-unit increase in crime around a house (as reported by crime maps) on its selling price? We test this directly using geomasking errors.

We have the following hypotheses:

H1.

The null hypothesis related to RQ1 is that police.uk did not affect house prices. This was the position of the Home Office and some property analysts in 2011 following criticisms of the website [7]. Our hypothesis is that police.uk lowered house prices in high crime areas. This was the position of other property analysts and estate agents in 2011 [4].

H2.

The null hypothesis related to RQ2 is that the number of crimes shown on police.uk will not decrease house prices. This was the position of the Home Office [7]. Our hypothesis is that an increase in the number of crimes shown on police.uk’s crime map in an area should lower house prices. This is in line with economic theory that perceptions of high crime make houses less desirable.

The objective of this study protocol is to specify the research plan ahead of data collection/ access. For RQ1, we will use public domain data only and examine evidence from all 43 police forces in England and Wales. RQ2 cannot be answered without access to the original geocoded crime data used by police.uk in their online crime maps. To this end, we have gained the support of the South Yorkshire Police (henceforth SYP) Force to access and use their geocoded crimes and incidence data.

Estimators (RQ1).

From the DAG, at T = 0 we can see that the statistical correlation between Y and M_s is simply due to confounding. If the police.uk affected house prices, then this correlation will change at T = 1. Therefore we can answer RQ1 by checking the statistical relationship between house prices and the number of potential nearby snap-points M_s.

Estimator 1A: Non-parametric. Calculate the correlation (i.e. spearman’s rank) between Y and M_s at T = 0 and T = 1 separately. Under the null hypothesis the correlation would be identical for both time periods. We can use bootstrapping or permutation tests to get standard errors.

Estimator 1B: Parametric. Assuming that a linear functional form is close enough approximation, estimate the following model using OLS:

Where β denotes parameters to be estimated from the data. The interaction term denotes a change in the statistical relationship between M_s and Y over time. Under the null hypothesis that police.uk had no effect, we would expect to be zero.

Estimator 1C: Parametric time series. Assume that a linear functional form is close enough approximation. Then redefine T to be an index number for year (e.g. T = 2011 if the year is 2011). For every year, estimate the following model using OLS:

The statistical association between Y and M_s () may change over time. If that change was consistent (e.g. greater effect sizes due to inflation) then, for every year before 2011, we can fit the following model using OLS:

We can use the model to estimate . This is what would have been if police.uk did not launch in 2011. Under the null hypothesis:

Estimators (RQ2).

To answer RQ2, we can directly estimate the effect of the information shown on its website C_g by exploiting the discrepancies in C_g, which is what the public observes, and the actual crime count C_r. By comparing areas with the same underlying crime levels (C_r) but different crime counts as shown on police.uk, we can estimate the causal effect of C_g on house prices.

This is only possible if either police.uk’s had no effect on house prices outside its website (U→Y) or police.uk only affected the impact of other crime-related factors on house prices. For instance, if police.uk affected the credibility of our sources of crime information like word of mouth or caused the shutdown of other crime maps. In this instance, we assume that this is entirely captured by the causal relationship C_r→Y and that other pathways U→Y are unchanged.

Under such conditions, we can compare prices for houses sold in areas with the same underlying crime levels (C_r) but different crimes counts (C_g) as shown on police.uk. From the DAG, the observed statistical relationship between C_g and Y conditional on C_r is a function of both confounding and collider bias (C_r is a collider). If C_g had no effect on house prices then this statistical relationship should remain exactly the same at T = 0 and T = 1.

Estimator 2A: Non-parametric. Check E(Y|C_g, C_r, T): the expected value of Y given C_g, C_r and T. Since C_g and C_r are continuous counts, we will have to band these into categories (based on say quartiles). For example, a house is in band B₁ if its values of C_g and C_r fall in the lowest quartile.

For each band, calculate difference mean Y over time:

Under the null hypothesis, this value should be equal to zero. Given the large number of potential bands, we may have to correct for multiple tests (e.g. Bonferroni correction).

Estimator 2B: Semi-parametric using matching. Match each case (i.e. house sold) in T = 1 to cases in T = 0 based on C_r and C_g. Discard unmatched cases. Ideally, the remaining data in T = 0 and T = 1 should have identical distributions of C_r and C_g (i.e. P(C_g, C_r|T) = P(C_g, C_r)).

Assuming approximate linearity, estimate the following using OLS:

Under the null hypothesis that C_g has no effect, is equal to zero. is the estimate for the impact of a one-unit increase in C_g on house prices. We can also replace C_r with dummy variables for C_r quartiles.

Alternative estimators.

We can further extend estimators 1B, 1C, and 2B using areal fixed effects. We will use lower super output area (LSOA) fixed effects. This is because public information at levels lower than LSOA is rare due to small area estimation and privacy issues. For example, the much-used index of multiple deprivation only reports deprivation at the LSOA level. We are almost certain that the older crime maps did not contain information at a lowest geographical scale than LSOA (with one potential exception for West Yorkshire). In addition, fixed effects control for time-variant confounders. As we mentioned previously, the major sources of confounding are due to static urban features and arise due to how police.uk created the snap-point list.

For example, the alternative estimator with fixed effects (FE) for 1B would be:

Where F_LSOA are the LSOA fixed effects. The quantity of interest (i.e. ) and all other things remain unchanged. Standard errors have to be adjusted for fixed effects.

Instead of using the log of house prices as our outcomes Y, we can try to use house prices without any transformations. This is only a useful estimator if inflation between T = 0 and T = 1 is negligible. Else if inflation is non-negligible but consistent, then estimator 1C (i.e. time series) will remain unbiased.

Another alternate estimator uses the relationship between crimes shown on police.uk C_g and Y to answer RQ1. Replace M_s with C_g in estimators 1A - 1C and change the relevant quantities of interest. Everything else remains the same. There are data limitations based on how many years of geocoded data (before 2011) have been archived by police forces.

For estimator 1C, which is relevant in case of trends over time, we can explore more elaborate time series models with different techniques for identifying structural breaks.

Our statistical tests are designed to reject the null hypothesis. Failure to reject the null hypothesis does not mean that the null hypothesis is true (i.e. police.uk did not affect house prices). An alternative approach is to calculate the probability that the true effect size is higher than some substantial amount. For example, if we can check the probability that a standard deviation change in C_g causes an increase in house prices higher than 2%.

Sensitivity and robustness tests.

From the DAG in Fig 2, we can infer most of our key assumptions, which are:

1. The effect of confounders on crimes shown on police.uk does not change over time P(C_g|U,T) = P(C_g|U)
2. The effect of confounders on potential nearby snaps does not change over time P(M_s|U,T) = P(M_s|U)
3. The distribution of confounders do not change over time P(U|T) = P(U)
4. The distribution of selling prices did not change for other reasons. To account for rising average house prices, we can demean Y (or add an intercept term in regression models).
   For RQ2, we need the additional assumption:
5. Between T = 0 and T = 1, any change in the effect of U on Y is entirely mediated by the real crime count C_r.

Many of these assumptions involve U: common causes of M_s, C_g and Y. Confounders can be split into two groups: observable and unobservable. Observable confounders are factors the research team has information about. In contrast, unobserved confounders are factors that the research team has no information on either because that information is inaccessible or the team is unaware of their existence.

For observable confounders, we can check for changes over time (assumptions 1–3) by comparing univariate and multivariate statistics between T = 0 and T = 1. For assumption 1, we can use an F-test comparing linear models:

For demonstration, we specify U as a continuous variable, but it can be categorical. Under the null model, the relationship between U and C_g remains the same over time (except for a scale shift accounted for by T). Under the alternative model, changes in the relationship between U and C_g are modelled as an interaction term. If an F-test rejects the null hypothesis that null model and the alternative model are equivalent, then assumption 1 is not credible. We test assumption 2 in the same way. To test assumption 3, we can use either a Fisher’s exact test or a Kurskal-Wallis test, depending on whether U is continuous or categorical.

In theory, any variable can be a confounder (unless proven otherwise). However, the most important confounders are the three inputs that determine what is shown on police.uk crime maps (see S2 File):

• the location that is queried on the crime map. In our case, this is the area around a property.
• the secret snap-point database
• police force recorded crime data

In our study, the research team can set the snap-point database used in the control periods (satisfying assumptions 1–3). For the location of houses and crimes, we can use geocoded information for tests (e.g. coordinates, higher areal units such as neighbourhoods). We can also use areal characteristics such as deprivation or access to amenities (see data section).

To test for changes over time for unobserved confounders, we can do a pre-intervention test. First, we find another period T = −1 (e.g. year 2009) before T = 0 where police.uk did not launch. Then we check that the same assumptions are met (e.g. P(U|T = −1) = P(U|T = 0)). Then we check if our estimators (e.g. 1A - 1C) give the expected result of no effects for years before any intervention took place.

We have no statistical way to test the assumption that between T = 0 and T = 1, any change in the effect of U on Y will be entirely mediated by the real crime count C_r. There may be other mediating pathways that do not travel along U→C_r→Y. For example. Md is a mediator whose effect on house prices Y differs between T = 1 and $T = 0 $. If there is a single causal pathway between Md and Y that does not lie along the causal pathway C_r→M→Y then this would violate our assumptions.

We do not expect spatial autocorrelation to affect our results greatly. However, we will test for spatial autocorrelation (i.e. checking Moran’s I) and adjust estimates accordingly.

Finally, we can resort to investigative work to uncover evidence that may refute any of our assumptions. In particular, we will look for other ways that the police.uk crime maps may have affected house prices in non-crime related ways.

Data sources / datasets used.

The HM Land Registry Price Paid dataset is a publicly available dataset of properties sold in England and Wales since 1995. The dataset contains information on price sold, address, house type and other features of the property sold. The dataset excludes certain types of transactions, such as inheritance and discounted transactions (e.g. discounted sales of social housing under the ‘Right to Buy’ scheme). The dataset can be accessed at: https://www.gov.uk/government/collections/price-paid-data. The coordinates of a property are derived from the coordinates of its postcode as recorded in the ONS National Statistics Postcode Lookup [24].

Archival data from police.uk are publicly available from the police.uk data site [21]. Other information, such as police force boundaries, are also contained on the website. We use the earliest archival extract of police.uk which contains data on crimes from December 2010 to December 2013. In general, police.uk keeps excellent documentation on archival data and changes made to its website and data manipulation. We also cross-referenced the historical police.uk website using the Wayback machine, which is an archive of websites.

For SYP crime data, we will use the same data source sent to the Home Office and ultimately processed by police.uk. We expect data from at least the year 2010 to be available.

We infer the snaps used by police.uk from the unique crime locations shown on police.uk during these periods covered by versions one and two (see S1 and S3 Files). In our inferred dataset, we have 734,000 snaps in version 2, roughly 96% of all the snaps in use by police.uk during this period. The inferred snaps are much lower in version one (~462k); we do not know how many snaps were used in this version. Ideally, we would like to use the real snap-point database to mitigate against all measurement errors. However, for our estimators, it only matters that the causal relationship between our inferred snap-points and confounders remains constant during the period under study.

To test the plausibility of the research design, we use a variety of data sources to check for confounders. We aim to use a kitchen-sink approach: assume that any variable can be a confounder and test it all. Potential confounders include:

• Food agency standards ratings. This is point data that contain information on business type, address, latitude and longitude alongside the most recent hygiene rating of an establishment
• Radon readings per 1km square grid. This is dataset is publicly available.
• Aggregated data (LSOA) on households from the British Household panel
• Access to Healthy Assets and Hazards. This is data from the Consumer Data Research Centre (CDRC). These are areal (LSOA) measures of how ‘healthy’ a neighbourhood including the mean distance to:–retail environments (e.g. fast food outlets)–health services (e.g. GP offices, hospitals etc)–physical environment (e.g. green space)–air quality
• Levels of homeownership in the area (output area, lower super output area)

Data access restrictions.

Police force data is not directly available for research. Researchers need to contact and negotiate access with individual police forces. Other data sources mentioned are publicly available or else require a free registered account. In the case of Ordinance Survey product (OS), researchers may need to purchase data from OS.

We inferred the master list of snap-points from public domain data from data.police.uk [22]. The inferred list cannot be used for reverse geomasking (i.e. to reveal the exact location of crimes and identify victims). We have explained exactly how to recreate this list using public data (see S3 File) and have shared our code on Github (https://github.com/MengLeZhang/crimeMaps-preReg-code).

Variables/ codebook.

The main variables used are:

police.uk variables (points data)

• Date of offence/ incident (truncated to year and month only)
• Home Office Offence Code
• Latitude and Longitude (WGS84, to be converted to Easting and Northing OSGB36)

SYP Police force variables (to be confirmed)

• Date of offence/ incident
• Home Office Offence Code
• Easting and Northing (OSGB36)
• Other contexts (e.g. free text information about location)

Land registry price paid data

• Date of transfer. Date when the sale was completed, as stated on the transfer deed.
• Price paid (in GBP)
• Postcode (joined to coordinates of postcode centroid via the ONS master postcode lookup)
• Type of property (e.g. Flats, Detached housing etc)
• Old/New. Whether a property was newly built or an established residential building
• Freehold or Leasehold Price Paid Data (PPD) category. Relates to type of price paid data and data recording changes over time. We use category A which forms the bulk of the dataset and is available from 1995. Category B transactions are only recorded since October 2013. These include transfers under a power of sale/repossessions and buy-to-lets.

For every residential property sold, we can derive the following variables:

• number of potential snap-points nearby (based on a particular snap list version)
• number of crimes and incidents nearby in the past three months (police force recorded).
• number of crimes and incidents nearby in the past three months (police.uk recorded)
• number of crimes and incidents nearby in the past three months of available police.uk data.

There is a lag in police.uk data, homebuyers buying in February can only access data up to January (and maybe even less recently than that).

Data on crimes in the prior three months is chosen based on Braakman’s research [14]. Nearby is defined as within 150, 300 or 500m; our preferred distance is 150 because the lowest level of points data on police.uk appears at a specific zoom level. At that zoom level, the scope of the interactive map on police.uk roughly covers a 300m by 300m square.

Data quality issues.

From speaking to SYP, there are some data quality issues in the raw police geocoded crime and incidents data. First, some crimes and incidents will have no locations recorded, or locations are misrecorded. For instance, incidents with unknown or ambiguous locations are often recorded as taking place within police stations. Second, the data received by police.uk each month is a snapshot of police systems. The police continually update these records (e.g., omitting duplicate incidents), but these updates will not be reflected on public crime maps. For RQ2, our police records will be more up-to-date than those used to produce the police.uk crime maps in the past. The extent of these errors is unlikely to affect our results.

Aside from these issues, there are no missing values in our data. Where missing values exist, we will perform list-wise deletion (i.e. get rid of cases with missing fields). Public domain data on housing and crimes already have undergone data cleaning and error checks by their respective data owners. We will conduct checks on the SYP data. This is in addition to any data cleaning already done by the police.

To check that we can replicate police.uk’s crime maps, we have cross-referenced statistics from our inferred snap-point list with the statistics from the real snap-point data (see S3 File). We will also use the raw police data to check that we can recreate the public crime data from 2011–2013.