Sainath R. Iyer
Pre-trial risk assessment instruments (RAIs) – assessments that help judges make “smarter” bail decisions by quantifying a defendant’s risk of flight and/or threat to public safety – are a relatively new phenomenon. There is fierce debate regarding their role and utility; specifically, how they affect incarceration rates, racial disparities in the criminal justice system, and public safety. At the very least, in some jurisdictions, some have helped reduce the jail population, although others, arguably, at the risk of entrenching racial bias. Given the endemic racial bias in the criminal justice system, this is an important concern, and makes transparency in the development, design, and implementation of RAIs critical.
New York City provides pre-trial assessment services through the NYC Criminal Justice Agency (CJA). This includes the use of a pre-trial RAI that assigns a score to defendants appearing before judges at arraignment. During the period between a defendant’s arrest and their first court appearance, which must be within 24 hours, he or she is ideally interviewed by a CJA representative using questions from the RAI. A “yes” or “no” to each question is assigned a point value, and the sum of all the values – colloquially known as the “CJA score” – is used to provide a recommendation of whether the defendant should be released on their own recognizance or held because they are a moderate or high risk of failing to appear for future court dates.
While this may seem effective in theory, a 2014 assessment of all pre-trial services across New York State suggest that the NYC tool and similar mechanisms are unevenly applied and produce inconsistent results. Seventy-eight percent of responding jurisdictions noted that they used a pre-trial risk assessment, yet 47 percent reported that no defendants are actually interviewed prior to arraignment. In those situations when a score is created, only 55 percent of respondents estimated that a release recommendation corresponded with the assessed level of risk 75 percent or more of the time. Moreover, in spite of stating that supervision is prioritized by risk level, over half of the jurisdictions surveyed reported that they have low or very low risk defendants supervised. Indeed, only 41 percent of programs reported that release decisions were consistent with pre-trial recommendations 75 percent or more of the time.
There are many possible explanations for the state of pre-trial services across New York and the role of RAIs in genuinely informing release decisions. My personal (albeit, anecdotal) experience as a legal intern in public defender offices in NYC, specifically in Brooklyn, Manhattan, and the Bronx, highlighted the ways in which RAIs can be used and abused. Prosecutors seem to cite a defendant’s CJA score when it suits their argument to hold a client and ignore it when it doesn’t. Defense attorneys do the same in arguing for release. Judges sometimes cite it as an afterthought when making their release decision. From informal conversations with attorneys and other experts in the field, the scores that many RAIs provide – in NYC, across New York State, and elsewhere – are not viewed as very reliable or useful measures of risk. To be sure, the specter of racial bias informing every aspect of the assessment process only reinforces distrust of such evaluations.
Thus, perhaps unsurprisingly, NYC is in the process of designing a new pre-trial risk assessment tool. Given that it is in the early design phase, virtually no information is available yet about what the tool will look like. There are plenty of ideas the designers could, and hopefully will, consider as they build the new RAI. One in particular – well-regarded for its efficiency and evidence-based approach – is the Public Safety Assessment (PSA) by the Laura and John Arnold Foundation. The PSA is a scientifically validated risk assessment tool that employs an algorithm to score a defendant based on ten objective (and its backers argue, race-neutral) factors found to be highly predictive of a defendant’s behavior. Given its early success, the PSA has enjoyed adoption by 38 jurisdictions. Indeed, even some of those deeply concerned with eliminating racial bias appear to support its approach.
Like many, I believe we must do whatever we can to radically reduce the reprehensible amount of people we incarcerate. Well-tested and validated RAIs may be a part of the solution. However, advocates of the new generation of RAIs must face an uncomfortable truth: a data-driven approach to assessing risk can minimize bias, but potentially at the risk of normalizing racially-corrupted data. Race-neutral factors in assessing risk only work insofar as the data used is race-neutral. That, unfortunately, is impossible to achieve if traditional data sources, such as prior convictions and age at time of arrest, play an outsized role in risk assessment. Let’s hope that the designers behind New York’s newest pre-trial tool keep those concerns front and center as they move forward in their process.
Sainath R. Iyer is a Quorum Editor and a J.D. candidate, Class of 2018, at N.Y.U. School of Law.