Jails and criminal justice in New York City

In the United States, prisons incarcerate individuals who have committed serious crimes and have long sentences. In contrast, jails play multiple roles in the justice system. The primary purpose of a jail is to provide temporary detainment to pre-trial defendants, either to ensure their attendance at trial or to mitigate the immediate threat they pose to public safety. But depending on the municipality, a jail may also contain locally sentenced individuals serving short sentences, state-sentenced individuals awaiting transfer to a long-term state prison, parole or probation violators awaiting a hearing, or detainees of federal law enforcement agencies [5]. As a result, while US prisons are usually segregated by gender, age, and offense level, a US jail holds a widely diverse population in all three categories. Researchers have long acknowledged the role of jail population growth in the meteoric rise of the national incarceration rate over the last 40 years [6]. During this time, the heterogeneous nature of the country’s decentralized, municipal jail system has made jail populations difficult to analyze, manage, and reform [7].

In 2015, the need for sweeping reform to, specifically, the jail system of New York City became more stark in the wake of Kalief Browder’s suicide. Browder, then sixteen years old, had been arrested for allegedly stealing a backpack in 2010, detained for three years in New York City’s Rikers Island jail awaiting a trial date, and then released due to a lack of evidence. Due to the persistent abuse and violent treatment that he received from correctional officers, Browder’s mental health had quickly deteriorated, and during his three years in jail, he had committed multiple acts of self-harm, including at least one suicide attempt [8]. “Being home is way better than being in jail,” Browder told a journalist from The New Yorker, a year after his release, and six months before his suicide. “But in my mind right now I feel like I’m still in jail, because I’m still feeling the side effects from what happened in there…I feel like I was robbed of my happiness” [9].

Browder’s story, as well as hundreds of other accounts of systemic abuse and injustice in the Rikers Island jail [10, 11], inspired criminal justice advocates to fight for change. In October 2019, the city council approved an $8 billion plan to close Rikers and replace it with new, smaller facilities scattered throughout the city [12]. These facilities would have a combined capacity of 3,300, which would require a reduction in jail population of over 50%. Thus, by necessity, jail population reduction became a priority for the de Blasio administration.

To accomplish this goal, jail reformers urged the state to enact a longtime policy objective: bail reform. The United States justice system’s reliance on pretrial bail has been widely criticized as a mechanism of jailing people simply for being poor. It has also come under focus as a driver of racial inequity, because a disproportionate number of defendants in New York City who cannot pay bail are Black or Hispanic [13]. In July 2018, New York City’s Independent Commission on Criminal Justice and Incarceration Reform wrote that “Money bail is the preeminent driver of the jail population in New York City.” As of May 2018, nearly 75% of individuals in jail were pretrial, with the vast majority there simply because they could not afford bail [14]. The Commission found that an expanded supervised release program, under which defendants would be allowed to remain in the community as long as they submitted to frequent monitoring by city-employed social workers, would lower jail populations while successfully ensuring that defendants attended their court dates.

New York State helped expedite this reform in March, 2019 by abolishing the use of cash bail for all defendants, save those charged with certain violent felonies. This bail reform law went into effect January 1, 2020, but judges began applying it retroactively to individuals already in jail for not posting bail in November, 2019 [15]. As a result, the city’s jail population dropped from approximately 7,000 at the beginning of November to just over 5,300 at the beginning of February, 2020 ––– a reduction of nearly 25%. Three months later, however, Governor Andrew Cuomo slightly rolled back the reform by allowing bail to be used for defendants charged with an additional list of 25 offenses [16].

Jails and infectious disease

As compared to the United States’ general population, the incarcerated population experiences higher disease contagion because they are in a congregate setting, and because there is an over-representation of individuals from racial and socioeconomic groups vulnerable to poor health outcomes [17]. The spread of illness in jails, specifically, is of interest to public health researchers because it has significant influence on spread in the surrounding communities, as most individuals in jails are discharged within a few weeks [18]. In particular, people incarcerated across the country [19] and in New York City [12] have contracted COVID-19 at a dramatically higher rate than the general population. Reducing jail populations has had a demonstrable effect on reducing spread of COVID-19 both within jails [20, 21] and in the community [22].

Our study

Given the ongoing debates about criminal justice reform, and given the continuing worldwide struggle to find effective public policies to curtail the spread of COVID-19, the study of New York City’s jail discharge policy in March 2020 is a pressing matter. The rest of this paper is organized as follows.

In Methods, we present our procedure for gathering, reconciling, and cleaning data that we obtain from the city’s Open Data Portal. Our data set consists of nearly 350,000 records, each of which describes an incarceration episode.

In Analysis and Results, we report our findings. The city jail population did decrease substantially from March 23—29, 2020, following de Blasio’s policy charge. We find that being discharged during this focus week is associated with a lower probability of readmission as compared to the same calendar week in previous years. Furthermore, comparing the individuals discharged during the focus week of 2020 to those discharged during the same calendar week in previous years, we find that the former group was, on average, slightly older than the latter group, although the difference is not large. Additionally, the individuals in the former group had spent substantially longer in jail than those in the latter group.

With the release of long-serving individuals demonstrated to be feasible, and keeping in mind that the jail system is intended for short-term incarceration, in Policy Simulation, we examine how the jail population would have looked over the past six years had caps in incarceration been in place. With a cap of one year, the system would experience a 15% decrease in incarceration. With a cap of 100 days, the reduction would be over 50%.

Because our results are only as accurate as New York City’s public-facing jail data, in Data Recommendations, we discuss numerous challenges with this data and suggest improvements. These improvements would address issues related to the incarcerated individual’s age, gender, race, and more.

Finally, in Discussion and Conclusions, we discuss policy implications of our work, highlight some opportunities and challenges posed by incarceration caps, and suggest key areas for reform. One such reform might involve identifying and discharging low-risk individuals sooner in general, which might be feasible given the de Blasio administration’s actions during the early stages of COVID-19.

Data acquisition

The Inmate Admissions data set (ADS) and Inmate Discharges data set (DDS) files are updated at the start of each month. We downloaded both in.csv format on January 9, 2020, after their January 1 update. Upon download, ADS had 335,491 records and DDS had 339,910 records. Both files cover the time period 2014 through 2020, and both contain the following fields:

• ID number, a unique identifier for each incarcerated individual,
• date and time of admission into jail system,
• date and time of discharge from jail system,
• race
• gender
• a code describing the individual’s status within the justice system, and
• a code corresponding to New York state law that specifies the most significant charge associated with the incarceration.

Additionally, DDS provides the age of each individual at discharge.

Data reconciliation

ADS and DDS provide similar information and have identical columns, with the exception that DDS includes age at discharge time. Crucially, though, the two files are not identical. For example, any individuals admitted prior to January 1, 2014 but discharged afterwards would appear in DDS but not in ADS. Conversely, any individuals who had been admitted but were not discharged prior to January 1, 2020 would appear in ADS but not in DDS. Still, beyond these explicable discrepancies, we find: records that are present in one but not both files; records that are in both files but with some fields missing data in one or both files; and records that are in both files but with conflicting information. Additionally, within each data set, we find records that are near-duplicates but that disagree on a subset of variables. We recognize four categories of records that must be removed prior to analysis.

1. (1) Records within ADS for which all information is an exact duplicate of another record. We remove one such record.
2. (2) Records within DDS that conflict with other, nearly identical DDS records. These records contain matching ID and admission date-time information with conflicting discharge date-time information. We remove 667 such records.
3. (3) Records for which the discharge date-time is prior to the admission date-time. We remove one such record each from ADS and DDS.
4. (4) Records within DDS that conflict with records in ADS. These records contain matching ID and admissions date-time information and conflicting discharge date-time information. We remove 11 such records.

Data merging

After the data reconciliation process, we have 335,478 ADS records and 339,231 DDS records, accounting for 99.9% of the originally downloaded data. We now accept certain of these records into our final data set, attempting to match records across the two data sets when feasible. For matching, we attempt to match on three key identifiers: ID number, admission date-time, and discharge date-time.

Altogether, we accept seven different categories of records into our final data set for analysis.

1. (1) Records that match across ADS and DDS based on ID, admission date-time, and discharge date-time. There are 277,117 such records.
2. (2) Records in ADS with a missing discharge date-time that can be recovered via matching with DDS based on ID and admission date-time. There are 51,285 such records.
3. (3) Records in DDS that are absent from ADS because their admission date-time is prior to 2014. There are 10,723 such records.
4. (4) Records in DDS that are absent from ADS, but that have admission date-time 2014 or later. There are 106 such records. We presume that their absence from ADS is a clerical error.
5. (5) Records in ADS that are missing from DDS despite having discharge date-time information. There are 27 such records. We presume their absence from DDS is a clerical error.
6. (6) Records in ADS that have missing discharge date-time information and that are found in the Current Inmates data set. There are 4,814 such records.
7. (7) Records in the Current Inmates data set that are found neither in ADS nor DDS. There are 253 such records.

At this stage, our data set has 344,325 records. This data set has no internal conflicts on key identifiers (ID, admission date-time, and discharge date-time) and it has complete information in those variables except for cases of missing discharge date-times for currently incarcerated individuals. In Data Cleaning, below, we will remove a small number of additional records.

Data cleaning

Though we have eliminated conflicts on key matching variables, there remain conflicts on demographic variables. For instance, there may be an individual who is designated as Black in ADS and Asian in DDS. Another individual may be listed as male for one record in ADS and may have missing gender information for another record in ADS.

We resolve race demographics in the following manner. For each unique ID number, we examine all data that we have for that individual’s race, both (potentially) across multiple incarceration episodes and (potentially) across data gleaned from ADS and DDS. In cases of disagreement, we remove all missing values and take the mode of the remaining data as the individual’s race. If there is no unique mode either because all data were missing or because there is a tie in the data, we assign NA to represent missing data. This procedure resolves race information for 13,621 records corresponding to 10,941 individuals. We perform a similar procedure for gender, which resolves 938 records for 801 individuals. It is possible that demographic discrepancies are due to changes in identity, for example, gender presentation for transgender individuals. However, because the data does not explicitly address transgender identity, we follow the procedure above. In Discussion and Conclusions, we make recommendations for improving data practices to make them more humane and to mitigate erasure of transgender individuals and other marginalized groups.

We also resolve discrepancies in status code and top charge code. Here, comparisons across records for the same individual are not relevant, as each incarceration episode may differ on these variables. However, for records that were matched across ADS and DDS, these variables should, but do not always, match. For each inconsistency, we check whether it is due to missing information or to conflicting information. For missing information, we simply accept the information that we do have. For direct conflicts, we set the final value of the variable to be NA to represent missing data. Overall, we find 21,299 explicit conflicts for status code and 3,269 for top charge code.

Next, we clean age at discharge. The distribution has a long tail of age up to 93 years old, and then a gap, with 45 individuals recorded with ages 117 or older. For these 45 records, we set age to NA. For the remaining data, there are inconsistencies in age information among many individuals with multiple incarcerations. For example, an individual discharged in 2014 may show an age of 43 at time of discharge, and may show an age of 44 for an incarceration that took place from 2019—2020. To remedy these inconsistencies, we examine all individuals with multiple incarcerations and we impose three different sets of changes.

1. (1) For 220,299 records comprising 63,552 individuals, there is inconsistent age data. That is to say, each of these individuals has multiple incarceration records which imply different birth dates, calculated as the difference between their discharge dates and ages at discharge. For these individuals, we calculate a birth date that minimizes the sum of the squared differences between our imputed birth date and the implied birth dates. Using this imputed birth date and the individual’s discharge dates, we calculate imputed ages at discharge. Our imputed ages differ from the original raw values by less than one year on average.
2. (2) For 422 records comprising 420 individuals, age data is missing. We recover it by using the same process as in (1) to compute a theoretical birth date from records that do have age data and come from the same individuals.
3. (3) For 13,147 records comprising 2,111 individuals, we attempt to apply the same computation described above. For each of these individuals, age adjusts by over five years for at least one record. We assume the age data is corrupt and set it to NA for all records.

We also examine the date-time of admission and date-time of discharge to create a derived variable, the duration of incarceration. For currently incarcerated individuals, we use the date-time of data acquisition instead of the date-time of discharge, since the latter does not exist. There are four cases we deal with.

1. (1) For 108 records, the duration of incarceration exceeds five years. Because the city corrections system is for short-term incarceration, we presume these records contain clerical errors in date-time of admission and/or discharge. Without accurate knowledge of admission/discharge information, we cannot analyze these records and so we exclude them. These eliminated 108 records would have made up less than 0.1% of the records in our final data set, and so we do not expect the elimination of these outliers to have a substantial impact on our conclusions.
2. (2) For 64 records, discharge occurs on the day prior to admission. We assume these are clerical errors, and we modify the discharge date-time to be 11:59 p.m. on the day of admission. For these records we count the duration of incarceration as one-half day.
3. (3) For 2218 records, discharge occurs before admission but on the same day. Similar to the case above, we modify the discharge time to be 11:59 and count the duration of the incarceration as one-half day.
4. (4) For the remaining 341,935 records, we subtract the date of discharge from the date of admission to obtain the duration of incarceration in days.

Final data set

Our final data set is available at http://bit.ly/qside-nyc-jail-data. It comprises 344,217 incarceration records in the New York City jail system for individuals admitted and/or discharged from 2014 through 2020. We now enumerate the fields in the data set and provide some contextual and summary information about them.

ID number. As mentioned previously, ID is a unique identifier. There are 163,077 IDs in our data, indicating multiple incarcerations for some individuals. The median number of incarceration episodes is one and the mean is just over two.

Race. Perhaps surprisingly, in the ADS and DDS raw data that we download from the city’s data portal, the only values for race are Asian, Black, and Unknown, which we code as NA. The data disallows meaningful analysis of race. We discuss this issue further in Discussion and Conclusions. We find that 1.5% of incarceration episodes are known to be associated with an Asian individual and 54.0% with a Black individual. The remaining 44.5% of incarceration episodes are associated with individuals of unknown race (which could include Asian and Black).

Gender. The city codes gender as binary using the terms “female” and “male.” We find that 9.0% of incarceration episodes are associated with female individuals and 90.8% with male individuals. The remaining 0.2% of episodes have missing gender information. The median number of incarcerations is one for both males and for females, and the mean number is 1.8 for females and 2.2 for males. In Discussion and Conclusions, we make several recommendations for inclusive practices pertaining to gender data.

Age at discharge. We have age at discharge for 94.7% of records in our data set. Excluding records for ongoing incarcerations (which have no discharge date), we have age for 96.1% of our data. The mean age is 37 years old.

Admission date-time. The corrections system records the incarcerated individual’s date and time of admission. All records in our data set have an admission date-time.

Discharge date-time. The corrections system records the incarcerated individual’s date and time of discharge. For 1.5% of records in our data set, there is no discharge date-time. These records represent incarcerations that were ongoing at the time of data collection.

Duration. As mentioned, we derive this variable by taking the difference in discharge and admission dates for each episode. For the 25,522 records for which admission and discharge are on the same day, we count the duration as one-half day. Overall, the mean is 64 days, with 66 days for males and 45 days for females.

Status. Associated with each incarceration episode is a code that describes the individual’s status within the corrections system. Explanations of each status code can be found in [24].

Top charge code, description and class. For each incarceration episode, the city’s data provides the legal charge that is most responsible for the incarceration. This charge appears as an alphanumeric code that refers to a section of state law. From this charge code, we extract from the New York state laws website [25] a description of the crime and the class of the charge. For example, one top charge code in the data set is 140.20. This crime is “burglary in the third degree” and state law specifies that it is a Class D Felony.

Scrape date. We have recorded the date we downloaded raw data, namely, January 9, 2021 for all records.

It is worth contrasting our data set with another resource, the Vera Institute for Justice’s JailVizNYC [24], an online, public-facing, interactive tool for exploring New York City jail data. We conjecture that JailVizNYC is built by scraping the city’s Daily Inmates in Custody data set each day. As a result, this data set is richer and more granular than the one we have constructed. Unfortunately, the Vera Institute’s historical data only covers the time period June 25, 2017 through November 11, 2019, and thus cannot be used for studying incarceration during COVID. Additionally, it appears that the underlying data set is not readily available to the public. However, for our study, [24] was useful as a validation of our own data for the dates that are shared between the two.

Duration of incarceration

We provide results about duration of incarceration in Fig 2. Panel A shows a histogram over the entire data set. Because the frequencies in the histogram vary over many orders of magnitude, we use a pseudo-log scale on the vertical. Panel B shows, via box plots, the distributions of duration, separated by year. The distribution for 2020 appears to have a substantially higher median value as compared to other years. To test whether duration was statistically significantly different in 2020, we perform a Kolmogorov-Smirnov (KS) test. More specifically, we test the null hypothesis that the cumulative distribution function (CDF) of durations of incarceration for individuals discharged during the focus week, 2020 does not lie below that of the CDF for those discharged during the focus week in all previous years (combined). The alternative hypothesis is that the 2020 distribution lies below the distribution for previous years. Panel C displays the two CDFs. The value of the KS test statistic is D = 0.98 with p-value p < 0.001. Therefore, we reject the null hypothesis and conclude that the CDF of durations for 2020 lies below and to the right of the CDF for previous years. That is to say, we conclude that for March 23—29 discharges, the individuals discharged in 2020 tended to have incarcerations of longer duration than individuals in previous years.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Duration of New York City incarceration episodes, 2014–2020.

(A) Histogram of all episodes. Because the frequencies vary over many orders of magnitude, we use a pseudo-log scale on the vertical. (B) Distributions of duration for individuals discharged March 23—29, separated by year and shown as box plots. The distribution for 2020 appears to have a substantially higher median value. (C) Cumulative distribution function for the data in panel B, with years 2014—2019 combined. From a Kolmogorov-Smirnov test (see text) we conclude that for March 23—29 discharges, the individuals discharged in 2020 tended to have served more time than individuals in previous years.

https://doi.org/10.1371/journal.pone.0262255.g002

Age at discharge

We provide results about age at discharge in Fig 3. Panel A shows a histogram over the entire data set. Because the frequencies in the histogram vary over many orders of magnitude, we use a pseudo-log scale on the vertical. Panel B shows, via box plots, the distributions of age, separated by year. To test whether age was statistically significantly different in 2020, we perform a KS test. More specifically, we test the null hypothesis that the CDF of age of individuals discharged during the focus week, 2020 does not lie below that of the CDF for those discharged the same week in all other years (combined). Panel C displays the two CDFs. The value of the KS test statistic is D = 0.068 with p-value p < 0.01. Therefore, we reject the null hypothesis and conclude that the CDF of age for 2020 lies below and to the right of the CDF for previous years. That is to say, we conclude that for March 23—29 discharges, the individuals discharged in 2020 tended to be older on average. The difference is, as stated, statistically significant. The magnitude of the difference is modest, with the median value for 2020 approximately two years older than for previous years.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Age at discharge for New York City incarceration episodes, 2014–2020.

(A) Histogram for all episodes. Because the frequencies vary over many orders of magnitude, we use a pseudo-log scale on the vertical. (B) Distributions of age for individuals discharged March 23—29, separated by year and shown as box plots. (C) Cumulative distribution function for the data in panel B, with years 2014—2019 combined. From a Kolmogorov-Smirnov test (see text) we conclude that for March 23—29 discharges, individuals discharged in 2020 tended to be older than in previous years, with a modest median diference of approximately two years.

https://doi.org/10.1371/journal.pone.0262255.g003

Readmission

As we have mentioned, during the focus week of 2020, Mayor de Blasio ordered the release of incarcerated individuals who were at low risk of reoffending. Because there is no definitive way to know whether an individual has committed a crime, we take readmission to jail as a proxy for reoffense and study readmission within our data. Specifically, we fit a logistic regression model for a dichotomous outcome variable of whether or not a person discharged during a given calendar year was readmitted in the same calendar year.

In designing the regression, there are several important factors to keep in mind. First, the reason we use a calendar year time horizon is to make a fair comparison between years. The target of our study is March 23—29, 2020, and our data set has coverage only through the end of 2020. Second, and relatedly, individuals released later in the year would of course have less time to commit a crime and be apprehended than those released earlier in the year. To control for this effect, we add an indicator fixed effect for release during our focus week, March 23—29. Third, it is possible that the year of release could have an impact on readmission. For example, perhaps 2020 had lower readmission rates overall than other years, not just for those individuals released during our focus week. To control for this, we add an indicator fixed effect for release during 2020. Finally, our target of estimation is the effect on readmission of being released specifically during March 23—29, 2020. Thus, we also include an interaction effect between the two aforementioned fixed effects. This allows us to estimate the association between release during the focus week of 2020 and readmission, while controlling both for the marginal effects of our focus week and of 2020.

We show the results of our logistic regression estimation in Table 1. Key results are as follows. First, we observe an estimate of 0.473 associated with the focus week covariate. This positive and statistically significant effect suggests a positive association between release during March 23—29 and the log-odds of readmission in the same calendar year, controlling for all other variables in the model. Second, we observe an estimate of −0.542 associated with the 2020 covariate. This negative estimate is also statistically significant, and suggests a negative association between release during 2020 and the log-odds of readmission in the same calendar year, controlling for all other variables in the model. Lastly, our statistically significant estimate of −0.260 for the interaction effect suggests that, after controlling for the marginal effects of release during the focus week and release during 2020, release specifically during March 23—29, 2020 has a negative association with the log-odds of readmission. In other words, all else being equal, our model would predict a lower probability of readmission for an individual released during our focus week in 2020.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Logistic regression model for readmission.

As a proxy for reoffense, we use a dichotomous outcome variable indicating whether or not an individual discharged during a given calendar year was readmitted in the same calendar year. The model includes four parameters: an intercept; an indicator fixed effect for release during the week of March 23—29; an indicator fixed effect for release during 2020; and an interaction effect for release specifically during March 23—29, 2020.

https://doi.org/10.1371/journal.pone.0262255.t001

In order to assess the robustness of our conclusions, we show the results of a second logistic regression model in Table 2. In this second model, we include a fixed effect for incarceration duration. It is possible, for example, that the negative estimate of the parameter associated with focus week in the first model arises because people who are incarcerated longer are less likely to be readmitted. When we incorporate duration into our model, the estimate of the interaction effect becomes slightly less negative, increasing to −0.228, but it remains statistically significant.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Logistic regression model for readmission with additional parameter.

As a proxy for reoffense, we use a dichotomous outcome variable indicating whether or not an individual discharged during a given calendar year was readmitted in the same calendar year. The model includes five parameters: an intercept; an indicator fixed effect for release during the week of March 23—29; an indicator fixed effect for release during 2020; an interaction effect for release specifically during March 23—29, 2020; and a fixed effect for incarceration duration. The model is similar to that of Table 1 but adds the duration fixed effect.

https://doi.org/10.1371/journal.pone.0262255.t002

Crucially, the interpretation of these parameter estimates is not causal. That is to say, neither our model nor our conclusions suggest that release during the focus week of 2020 caused inmates to be readmitted at a lower rate. Instead, our model merely predicts a lower probability of readmission for these individuals. This lower probability could come from a number of factors, including a causal effect, exogenous factors, or randomness in the data. Nonetheless, our models do help identify associations that are present in the data.