Two years ago, Demontez Campbell, who goes by Tez, was arrested for third-degree burglary stemming from what he says was a domestic dispute at his house in Hardin County, Kentucky. As he was being processed, a pretrial officer told him that he had been evaluated by the state’s risk assessment algorithm, a criminal justice reform tool intended to release more people before trial without bail.
Campbell, who was 29 at the time, had a record of theft and drug-related charges, though none at that point were violent. “On paper, it looked as if I was just a real hardened criminal because of my past mistakes,” he said. Though he lived in the house in Hardin County where he was arrested, Campbell is from Louisville, about an hour’s drive away. The disparity led authorities to consider him a flight risk. The combination of factors — his age, history, and lack of an official local address — resulted in a moderate risk rating, according to the algorithm, which in Campbell’s case meant that he would have to pay cash bail to be released.
“There’s no hands to help, there’s just hands pointing at you. I felt like I was being convicted before even being convicted.”
The judge set bail at $15,000. “They’re looking at your charge and looking at your past charges and deciding, ‘That’s what he’s going to do forever,’” Campbell said. At the time, Campbell worked at Subway and had to pay child support every week on top of his other bills. “You might as well have set it at a million, because I’m not going to be able to make that,” he said. On top of the financial strain, Campbell also said he had post-traumatic stress disorder, depression, and anxiety, as well as substance use issues, which he said was part of why he got tangled up with the law in the first place.
He spent six months in jail, but faced with family obligations and the uncertainty of how much longer he’d have to spend behind bars awaiting trial, he decided to plead guilty. He was given five years of probation.
“There’s no hands to help, there’s just hands pointing at you,” he said of the risk assessment that helped put him behind bars instead of sending him home. “I felt like I was being convicted before even being convicted.”
In 2011, as part of a criminal justice reform package meant to reduce incarceration and its related expenditures, Kentucky became among the first states to require judges to use a risk assessment as part of their pretrial decisions. Similar risk assessments have since proliferated across the country in recent years, as jurisdictions that have decided cash bail is unjust — and potentially unconstitutional — have had to grapple with the circumstances in which to detain people who are too poor to come up with the money.
Risk assessments can give the veneer of a more scientific approach than a judge’s discretion, while dampening critics who argue that getting rid of cash bail allows dangerous people to be released. A large amount of past court data is run through a system that determines what factors appear to correlate with higher rates of rearrest or failure to appear for later court dates. Each jurisdiction decides what factors to weigh more heavily, and the algorithm creates a matrix to determine whether someone is low risk, moderate risk, or high risk. Jurisdictions then decide how the scores get used; in most places, they are given to a judge to consider when deciding whether and how to release someone.
As community bail funds across the country received an outpouring of donations to release thousands of people arrested during anti-police brutality protests in the wake of George Floyd’s killing, the underlying inequities of bail reform measures are getting a closer look.
Criminal justice reform advocates have raised the alarm about risk assessments, arguing that the tools are based on questionable data that cannot accurately predict someone’s actions after they are released and often reproduce racial disparities already in place in the criminal justice system. What little success risk assessments have achieved has been undercut by judges who appear to ignore recommendations and legislators attempting to exacerbate their defects. The very idea that we can accurately predict future crime is doubtful.
In many jurisdictions, risk assessment has become “a totally political, manipulated, secret process,” said Pilar Weiss, director of the Community Justice Exchange, a criminal justice reform advocacy group. “They’re these flawed, racist, classist tools that purport to be based on science that are how the system is making decisions about people’s freedoms and their liberties.”
In the months after risk assessments were first made mandatory in Kentucky, there was a 13-percentage point increase in people released without having to post bail, according to a 2018 study. The effect was greater for those who were deemed to be low or moderate risk by the tool.
But that effect quickly wore off. “Judges were going back to their prior ways,” said Megan Stevenson, a law professor at George Mason University and author of the study. By the beginning of 2016, more than half of the initial positive impact had disappeared. Ashley Spalding, research director at the Kentucky Center for Economic Policy, also noticed the tendency of judges to ignore the results. “They’re overriding the findings of the risk assessment tool,” she said. “In practice, we’re seeing that it is often disregarded.” There is no oversight for how judges use the scores and no penalties for ignoring them.
There are also disparities by race. A 2019 study found that Kentucky judges are more likely to override moderate risk scores in order to set cash bail for Black defendants than for white defendants. “Judges see the moderate risk label, and for white defendants, moderate risk was interpreted as low risk, and for Black defendants, it was interpreted as a signal of higher risk,” Stevenson said.
The scores often reflect inaccurate judgments because the information the tool processes is incomplete. For example, Kentucky’s assessment deems people without a permanent address as high risk, presuming that they will fail to appear for subsequent court dates. But according to Shameka Parrish-Wright, the Kentucky site manager for the Bail Project, her homeless clients who don’t have addresses return to court with the organization’s assistance. “The algorithm wouldn’t tell you that,” she said. “Statistical risk is something we combat every day.”
“The statistical predictions are dehumanizing. They rob people of their individual stories.”
Yet Kentucky has frequently been held up as the exemplar for risk assessments: In 2016, 67 current and former district and state attorneys described the state’s program as “very effective.” After originally using an in-house risk assessment, in 2013 Kentucky became the first state to pilot the Public Safety Assessment, developed by the Laura and John Arnold Foundation, a charitable investment firm. The PSA is now the most widely used among states and counties that conduct risk assessments — despite the fact that there was no effect on how many people in Kentucky were released or on racial disparities compared to when the state was using its own tool.
Parrish-Wright also objects to risk assessments because they reduce human beings to a number. “The statistical predictions are dehumanizing,” she said. “They rob people of their individual stories.”
Imagine risk assessments as rudimentary versions of PreCrime, the system used to incarcerate people before they commit murders in the Steven Spielberg sci-fi film “Minority Report.” The idea is that by looking at a defendant’s past, an algorithm will be able to accurately predict whether they will pose any threat to public safety or fail to return to court if released.
Risk assessments “make it seem like predictions of future violence are much more accurate than they can be,” said Colin Doyle, a staff attorney at Harvard Law School’s criminal justice policy program. The system in “Minority Report” relied on clairvoyants to predict crimes, but judges in the real world can’t see into the future. So the assessments end up amounting to “pretty wild speculation,” said Doyle.
Because committing a crime is such a rare occurrence, statistical correlations in the data aren’t strong enough to make accurate predictions about a single person. “They’re really good for aggregate policymaking decisions,” said University of Georgia law professor Sandra Mayson, such as what kinds of defendants can simply be released without seeing a judge. “They are completely incapable of identifying a significant enough risk in an individual case to warrant preventative detention.”
Most people who are labeled high risk won’t actually be rearrested pretrial. In Cook County, Illinois, for example, 99 percent of defendants deemed high risk by the Public Safety Assessment but released before trial weren’t charged with new violent crimes — virtually the same share of those with moderate or low scores.
Meanwhile, the data that risk assessment tools are built on constitute “garbage in, garbage out,” Doyle said. Creating the purportedly predictive assessments is accomplished by feeding in as much court data as possible into an algorithm. The designers then look at what kinds of characteristics correlate with increased chances of rearrest or failure to show up for court.
The underlying data, though, is already steeped in racial bias: Arrests, charges, bail amounts, and sentencing are all more harshly meted out against Black people, when compared with white people. Then there is the quality of the data itself: It is often riddled with errors because it is compiled as administrative data by clerks and other nonscientists, as opposed to being collected for the purpose of creating a sophisticated tool.
What’s more, the data does not establish causal relationships; instead, it is simply correlative. The process therefore leads to reliance on some odd factors. Colorado’s homemade tool takes into consideration whether someone has a cellphone and whether or not they own or rent their homes. A tool in use in Rockland County, Ohio, included a homegrown personality quiz, with questions about family issues and feelings of anger.
Even prior arrests are a weak indicator. Someone who is arrested is still considered innocent until proven guilty, and not all arrests lead to conviction. “Arrest is not something you do,” Mayson said. “Arrest is something the state does to you.”
Missed court dates don’t necessarily mean that someone was fleeing and could have a range of explanations: They could have been given the wrong date, may have moved, or were in a hospital or jail at the time of their hearing. Work, child care, or unaddressed mental health or substance use issues can also present barriers to getting to court.
“If you are making incarceration decisions solely on the basis of risk alone, what you’d want to do is lock up all the teenagers and young adults and let them out when they turn 30.”
Age is the factor most strongly associated with committing crime, but putting outsize weight on it in a risk assessment would yield impractical directives. “If you are making incarceration decisions solely on the basis of risk alone, what you’d want to do is lock up all the teenagers and young adults and let them out when they turn 30,” said Stevenson, of George Mason University.
While there are many features of risk assessments that are public, there is much in the process that remains opaque. When Hannah Sassaman, policy director at the social justice organization Media Mobilizing Project, began trying to put together a database of how all the risk assessments across the country operate, the courts would decline her requests for information. Journalists didn’t seem to have any more luck. In some jurisdictions, it wasn’t even clear who to call, or no information was published online.
Many of the people who are subject to these algorithms may not have any idea how they were evaluated, on what grounds, and what the recommendation was. In some places, the scores aren’t even given to the defendant’s lawyer or read in open court. “If we can’t find out, how can people who are being judged in a hearing know what the risk assessment said about them?” Sassaman said.
Even defendants who are deemed low or moderate risk may face increased surveillance as a condition of their release, such as electronic monitoring, home confinement, or regular check-ins with a parole officer, said Sassaman. Those who can’t comply with those restrictions risk rearrest and having to go through the risk assessment process all over again.
Given the evidence that risk assessments have failed to make inroads in ending bail, many criminal justice reform advocates want to institute a different system entirely: one in which the vast majority of people are released before trial and those who prosecutors want to detain get a full hearing before being jailed.
In 2018, over 100 organizations signed onto a letter led by the Leadership Conference on Civil and Human Rights outlining concerns with risk assessments and six recommendations to lessen the harm they can cause, including a call for “adversarial hearings” before someone is detained or given conditions of supervision.
In most bail hearings, defendants are represented by attorneys who barely get to know their name and the proceedings are over in a matter of minutes. An adversarial hearing would allow the defense to have legal representation who could conduct discovery and present evidence and witnesses.
“The simplest mechanism for getting better pretrial outcomes is procedure,” Doyle said. “If a prosecutor and a judge are going to incarcerate someone pretrial, you have to have a substantial hearing and make the convincing case of that person’s dangerousness.”
“It makes it taxing to incarcerate someone,” Doyle added, rather than imprisonment being “the default decision.”
To facilitate those hearings, more people would need to be released first. “Ninety-five percent of the people who are coming into contact with the legal system at the pretrial phase can be safely released,” said Sakira Cook, justice reform program director at the Leadership Conference. Some may need to be released with conditions, she said, but those should only be to ensure their return to court or to protect public safety — which she defined narrowly as mitigating a likely threat to a known person in the near future.
In response to the advocates’ letter, the Arnold Foundation released a statement agreeing with many of the letter’s principles. This past February, the Pretrial Justice Institute, which had previously endorsed the use of risk assessments, said in a statement that the tools could no longer be part of its reform efforts.
“I’m doing all of these building things, positive things that on paper my pretrial assessment probably stated I wasn’t going to do.”
However, in Kentucky there appears to be little appetite for fixing the risk assessment; rather, lawmakers are pushing to further entrench its flaws. Last February, a bill was introduced in the legislature to mandate that people with high risk scores be detained pretrial and make release for those with low scores no longer obligatory. But it never got voted out of committee, effectively killing the bill for the meantime.
“There is probably not a lot of low-hanging fruit,” Stevenson said, “in terms of being able to lower incarceration rates without increasing crime.” She added, “It’s not as simple as, just adopt a risk assessment and great things will happen.”
Things have changed for Tez Campbell, though. After he was released from jail, he started going to Alcoholics Anonymous and later entered a treatment center. While there, he missed one of his court dates and was issued a bench warrant and a bond of $1,500. He reached out to the Kentucky Bail Project, which paid the bond. “It saved my life,” he said. “I felt like I had somebody on my side.”
Today he has his own apartment and a steady job as a construction flagger. He is also leading a life skills workshop at the Bail Project. Eventually, he wants to go back to school to become a social worker and possibly do drug addiction counseling. “I’m doing all of these building things,” he said, “positive things that on paper my pretrial assessment probably stated I wasn’t going to do.”