Part of a series on
Algocracy
Examples
AI in government COMPAS Cybersyn DAO Toeslagenaffaire IT-backed authoritarianism Ofqual exam results algorithm OIA PMPs Predictive policing Robodebt Smart city SyRI
v t e

Correctional Offender Management Profiling for Alternative Sanctions (COMPAS)^[1] is a case management and decision support tool developed and owned by Northpointe (now Equivant) used by U.S. courts to assess the likelihood of a defendant becoming a recidivist.^[2][3]

COMPAS has been used by the U.S. states of New York, Wisconsin, California, Florida's Broward County, and other jurisdictions.^[4]

Risk assessment

The COMPAS software uses an algorithm to assess potential recidivism risk. Northpointe created risk scales for general and violent recidivism, and for pretrial misconduct. According to the COMPAS Practitioner's Guide, the scales were designed using behavioral and psychological constructs "of very high relevance to recidivism and criminal careers."^[5]

Pretrial release risk scale

Pretrial risk is a measure of the potential for an individual to fail to appear and/or to commit new felonies while on release. According to the research that informed the creation of the scale, "current charges, pending charges, prior arrest history, previous pretrial failure, residential stability, employment status, community ties, and substance abuse" are the most significant indicators affecting pretrial risk scores.^[5]

General recidivism scale

The General recidivism scale is designed to predict new offenses upon release, and after the COMPAS assessment is given. The scale uses an individual's criminal history and associates, drug involvement, and indications of juvenile delinquency.^[6]

Violent recidivism scale

The violent recidivism score is meant to predict violent offenses following release. The scale uses data or indicators that include a person's "history of violence, history of non-compliance, vocational/educational problems, the person’s age-at-intake and the person’s age-at-first-arrest."^[7]

The violent recidivism risk scale is calculated as follows:

${\displaystyle s=a(-w)+a_{\text{first))(-w)+h_{\text{violence))w+v_{\text{edu))w+h_{\text{nc))w}$

where ${\displaystyle s}$ is the violent recidivism risk score, ${\displaystyle w}$ is a weight multiplier, ${\displaystyle a}$ is current age, ${\displaystyle a_{\text{first))}$ is the age at first arrest, ${\displaystyle h_{\text{violence))}$ is the history of violence, ${\displaystyle v_{\text{edu))}$ is vocational education scale, and ${\displaystyle h_{\text{nc))}$ is history of noncompliance. The weight, ${\displaystyle w}$, is "determined by the strength of the item’s relationship to person offense recidivism that we observed in our study data."^[8]

Critiques and legal rulings

Interventions of AI and algorithms in the court are usually motivated by cognitive biases such as hungry judge effect.^[9]

In July 2016, the Wisconsin Supreme Court ruled that COMPAS risk scores can be considered by judges during sentencing, but there must be warnings given to the scores to represent the tool's "limitations and cautions."^[4]

A general critique of the use of proprietary software such as COMPAS is that since the algorithms it uses are trade secrets, they cannot be examined by the public and affected parties which may be a violation of due process. Additionally, simple, transparent and more interpretable algorithms (such as linear regression) have been shown to perform predictions approximately as well as the COMPAS algorithm.^[10][11][12]

Another general criticism of machine-learning based algorithms is since they are data-dependent if the data are biased, the software will likely yield biased results.^[13]

Specifically, COMPAS risk assessments have been argued to violate 14th Amendment Equal Protection rights on the basis of race, since the algorithms are argued to be racially discriminatory, to result in disparate treatment, and to not be narrowly tailored.^[14]

Accuracy

In 2016, Julia Angwin was co-author of a ProPublica investigation of the algorithm.^[15] The team found that “blacks are almost twice as likely as whites to be labeled a higher risk but not actually re-offend,” whereas COMPAS “makes the opposite mistake among whites: They are much more likely than blacks to be labeled lower-risk but go on to commit other crimes.”^[15][10][16] They also found that only 20 percent of people predicted to commit violent crimes actually went on to do so.^[15]

In a letter, Northpointe criticized ProPublica’s methodology and stated that: “[The company] does not agree that the results of your analysis, or the claims being made based upon that analysis, are correct or that they accurately reflect the outcomes from the application of the model.”^[15]

Another team at the Community Resources for Justice, a criminal justice think tank, published a rebuttal of the investigation's findings.^[17] Among several objections, the CRJ rebuttal concluded that the Propublica's results: "contradict several comprehensive existing studies concluding that actuarial risk can be predicted free of racial and/or gender bias."^[17]

A subsequent study has shown that COMPAS software is somewhat more accurate than individuals with little or no criminal justice expertise, yet less accurate than groups of such individuals.^[18] They found that: "On average, they got the right answer 63 percent of their time, and the group’s accuracy rose to 67 percent if their answers were pooled. COMPAS, by contrast, has an accuracy of 65 percent."^[10]. Researchers from the University of Houston found that COMPAS does not conform to group fairness criteria and produces various kinds of unfair outcomes across sex- and race-based demographic groups.^[19]