Articles
Bench & Bar of Minnesota is the official publication of the Minnesota State Bar Association.

Understanding ‘Risk Assessment’ Tools

What they are and the role they play in the criminal justice system: a primer

The concept of “risk assessment” in the justice system has proliferated alongside other terms such as “evidence-based practices,” “risk algorithms,” “big data,” and “machine learning.” While these terms and topics are related, they are not all synonymous. Not all risk assessments are created with the same algorithms or use advanced machine learning techniques. And while many criminal justice agencies like probation departments and treatment programs are moving toward evidence-based practices, using a risk assessment does not guarantee the agency has implemented evidence-based practices. The goal of this article is to define and explain risk assessment, summarize the research supporting risk assessment for correctional agencies, and explain the limitations and practical concerns that arise when risk assessment is used in the criminal justice system.

What is risk assessment and how are risk assessment tools created?

In the context of criminal justice, risk assessment generally refers to the measurement of someone’s likelihood of reoffending.1 However, “risk” is not necessarily linked to offense seriousness. A person who was convicted of a serious offense like murder might present a low risk to reoffend, and conversely, someone who was convicted of committing a low-level offense like theft might be highly likely to reoffend in the future. “Risk” refers to the potential harm posed to the community according to a person’s chances of committing a new offense—any offense—in the future.

Risk to reoffend is measured with a risk assessment tool. Generally these tools take the form of a questionnaire that is completed by the probation officer or other trained assessor either by interviewing the probationer, reviewing the probationer’s file, or both. Each item listed in the questionnaire is a risk factor associated with predicting recidivism. The officer or assessor will complete the questionnaire, scoring each item in accord with specific criteria outlined in a scoring guide. The score is then totaled, and the total score is associated with a risk level, such as low, moderate, or high risk. The risk assessment tool used by federal probation, the Post Conviction Risk Assessment (PCRA), includes 30 items, 15 of which are scored out.2 The Level of Service Case Management Inventory (LS/CMI), a tool used by community corrections agencies in Minnesota, follows a similar process and includes 43 items.3 

Risk assessment tools are developed through the statistical analysis of a large number of cases to identify significant correlates of recidivism—factors that have a statistically significant relationship with recidivism.4 Because large data sets and statistical analysis underlay the development of these risk assessment tools, you might sometimes hear the terms “big data” or “machine learning” to describe them. While risk assessments are developed using large datasets to identify predictors of recidivism, their development can involve selecting risk factors through correlations, analyzing regression models, or employing more advanced, machine learning techniques based on computer science. Not all of these methodologies are the same. 

But despite these differences in how the tools are created, the overriding point remains the same: When a large dataset is used to create a risk assessment tool, it produces a statistically significant improvement in estimating risk. We make more accurate predictions using these tools than when we use our own unstructured, clinical judgment.5 The analysis of a large number of cases can identify common characteristics across people, jurisdictions, situations, and systems more reliably than one person working from a single vantage point: their own position, their own agency, their own professional experience. Using data rather than perceptions can also serve to remove bias, and the algorithms used to develop prediction models allow us to detect patterns more objectively. Even in other fields, research has supported the improvement in prediction that a data-based tool can make compared to unstructured decision-making alone.6 

How do risk assessments inform evidence-based practices in corrections?

Today, risk assessment tools used in corrections often measure both risk to reoffend and “criminogenic” needs (a term for dynamic risk factors that predict recidivism and can be changed).7  While older risk assessment tools measured only static risk factors like prior criminal history,8 newer tools include dynamic factors9 such as employment, substance abuse, and involvement with antisocial peers. The following section explains how targeting criminogenic needs can reduce someone’s risk and therefore, reduce the likelihood that person will commit a future offense. 

Risk, need, and responsivity principles

Risk and needs assessment allows correctional agencies to target supervision and programming in a way that can reduce recidivism among the people they supervise. An extensive amount of research over the past three decades has identified three clear principles for reducing recidivism rates in supervised populations: risk, need, and responsivity.10

The first principle, risk, includes a few components. One is that risk to reoffend can be predicted, and supervision intensity should be assigned according to risk.11 Minnesota community corrections agencies use the LS/CMI to identify the risk level of the offenders they supervise. Higher risk individuals receive more supervision and report more frequently to their probation officers.12 Similarly, treatment programs should calibrate programming intensity to risk by requiring high risk individuals to engage in more services. 

For low risk offenders, conversely, we want to provide minimal interventions. We do not want to mix them with high risk offenders because doing so can actually increase their chances of committing a new crime by providing more exposure to criminal behaviors and techniques.13 Further, the very things that make someone low risk—such as having employment or family support and noncriminal friends—can be disrupted when excessive conditions and requirements are placed on them. If a low risk person has a job and is involved in their community, requiring them to report every week could jeopardize their employment or their ability to report as required. Risk assessments are crucial tools in helping correctional agencies adhere to the risk principle, because they are more accurate at sorting people.

The second principle is need, which refers to “criminogenic needs,” or those characteristics, traits, problems, or issues that directly relate to the individual’s likelihood to commit another crime. In the LS/CMI, criminogenic needs are categorized into seven areas: self-control/impulsivity issues; employment/education; substance abuse; peers/associates; thinking/values/attitudes; leisure/recreation, and family/marital.14 When corrections staff are able to target particular criminogenic needs, they are directly addressing factors that will reduce someone’s risk.15 

The needs information can inform the conditions that should be ordered at the time of sentencing. For example, if someone scored high in the area of substance abuse, the probation officer would want to refer that individual for substance abuse services, and would recommend that the court impose a special condition of probation requiring completion of substance abuse treatment. If that same person also scored high on the domain of companions, the officer would use supervision techniques to monitor the probationer’s associates while also working with that person to develop more positive relationships with noncriminal people in the community. The key for correctional agencies seeking to reduce risk is to use a tool that identifies the needs related to reoffending and then provide services and interventions to address those needs.

Finally, the last principle is responsivity, an important component of the research on risk and needs. One part of the responsivity principle focuses on how treatment is delivered, since there is significant evidence showing that interventions for offenders are most effective when delivered through cognitive-behavioral methods.16 The other part seeks to anticipate barriers that may prevent individuals from succeeding in treatment, even though these barriers might not be related directly to a person’s propensity to commit crime.17 Someone who does not have a car to drive to treatment, for example, may not complete it. So while transportation is not directly responsible for their criminal behavior or substance abuse, it presents a barrier to addressing their substance abuse issues. 

It is important to stress that risk assessment tools must be accompanied by professional judgment.18 The professional experiences of people working in the system are still important in applying the principles of risk, need, and responsivity. Sometimes decision-makers and practitioners choose to override a risk assessment or diverge from assessment recommendations; they recognize that risk assessment tools do not account for all situations. 

What are the limitations and issues concerning risk assessment?

Despite the research supporting the use of risk assessment for correctional agencies, judges and attorneys should be aware that there are limits to and concerns about their usefulness in other areas of the criminal justice system. 

Risk assessment tools use actuarial methods, or group data, to predict risk.

Risk assessment tools provide a total risk score and place people into a risk level according to their likelihood of committing a new offense, just as insurance companies use actuarial practices to create insurance rates. Neither can predict human behavior with certainty. Just because a particular person is a male teenager—to cite one particularly high-risk pool—it does not follow that he is going to have more accidents; it means that male teenagers as a group have more accidents. Similarly, a risk assessment may classify someone as low risk who nonetheless goes on to reoffend in the future. It is the low risk group as a whole that is less likely to reoffend in comparison to the other groups. While risk assessments create an improvement in predicting risk, the risk levels produced do not mean certainty at the individual level. 

Risk assessment tools require effective implementation.

Many of the risk assessment tools in common use require that the people administering the tool be trained and certified to use them. This is because the tool is only as reliable as the person scoring the tool according to a scoring guide protocol. There are always issues surrounding implementation. Some staff may vary in their expertise in using a tool, so agencies should provide ongoing training and coaching for staff. Agencies should also audit files and review data regularly to ensure staff are using the tool as the developers intended and according to the scoring criteria. Agencies may also struggle with issues like organizational culture and buy-in, creating situations where risk assessment tools are scored but not applied in agency decision-making. Sometimes we may find that a tool does not predict well—but we should remain mindful that the issue may reside with how the tool was implemented in that agency and not the tool itself.19

Risk assessment tools must be validated.

It is important to study the effectiveness of a risk assessment tool on the population for which it will be used. This is commonly referred to as a validation study, where a risk assessment is studied to determine how well it predicts risk in a jurisdiction or agency that is different from where the tool was originally created. The LS/CMI that is used in Minnesota was developed by Canadian researchers, but evaluators have reviewed the ability of the LS/CMI to predict recidivism in the U.S.20 For agencies that develop their own risk assessment tool or use an understudied tool, it is important to evaluate how well the tool predicts recidivism in the local population. We want to make sure that there is not something unique to that population, agency, or time period that impacts prediction. And it is important for researchers to provide transparency on the evaluations of these tools. Results of their validation studies should be shared publicly, and the tools should be reviewed by external researchers.

Risk assessments should be used for the purpose for which they were designed.

Not all risk assessment tools are the same. This article has focused on tools such as the LS/CMI and PCRA, which are used by correctional agencies for community supervision case management purposes. But there are other risk assessment tools that may be used to make decisions at other points in the life of a criminal case or to predict other types of behavior. For example, pre-trial risk assessment is designed to aid the judge in determining whether or not a defendant should be released on bail. A pre-trial risk assessment tool predicts someone’s likelihood of committing a new offense or failing to appear while on pre-trial release. This type of tool is based on factors that are legally permissible to obtain at this stage and that predict both of those outcomes, whereas a risk assessment tool like the LS/CMI might not predict these outcomes at the pretrial stage, and should not be used for this purpose.21 

A related question is whether risk assessment should be used at the sentencing stage. In Minnesota, the LS/CMI is often completed at the time of the presentencing investigation (PSI), so does that mean that it should be used by the judge at sentencing? On the one hand, the LS/CMI and other risk assessment tools like it were not designed for the purposes of deciding sentences, especially where other considerations such as the severity of the offense, the harm inflicted on society, deterrence, or incapacitation may be paramount. However, because the tool identifies risk and needs, it can be helpful to the court in setting appropriate conditions of probation.22 For this reason, some jurisdictions have included the risk assessment score in the presentence report as an additional piece of information available for the judge to review.23 And in fact, the two other states that have considered this issue directly—Indiana and Wisconsin—have concurred, holding that risk assessment tools may not be used by the court to determine whether the offender should be incarcerated or supervised in the community or to determine the length of the sentence, but generally may be used to determine the terms and conditions of probation.24 

There is concern that risk assessment tools might perpetuate racial disparity.

Some critics, including former Attorney General Eric Holder, have argued that risk assessment might perpetuate racial disparities already prevalent throughout the criminal justice system.25 There is concern that risk assessment could contribute to disparity at sentencing: Risk assessments include criminal history as a factor in the tool, which—owing to potential systemic bias in policing and prosecution—might elevate risk scores for offenders who are black.26 A study examining the role of criminal history and racial bias acknowledges a complicated relationship between race, criminal history, and recidivism and suggests that criminal history might have more of an impact when risk assessment is considered at sentencing rather than other stages.27 

While more research is being conducted in this area, thus far the research suggests that risk assessments do not present evidence of racial bias.28 These studies have found that when comparing the predictive abilities of risk assessment on blacks as compared to whites, the tools predicted recidivism with similar accuracy for both groups.29 One salient caveat here involves using risk assessment for the purposes for which it was designed.30 Even Holder did not dispute the use of risk assessment for case management purposes; instead, he was concerned about using risk assessment to determine the type and severity of sentence. Thus, if there are concerns about perpetuating racial disparity, it may not be the risk assessment itself, but rather how risk assessment is used in the criminal justice system, that should be driving our concern. 

Conclusion

Risk assessment is critical for correctional agencies tasked with reducing recidivism. Risk assessment allows them to prioritize resources objectively so as to promote reductions in recidivism. It has taken decades of research, training, and evaluation for correctional agencies to implement the tools and resources needed to reduce reoffending and reduce the likelihood of offenders returning to the criminal justice system. Risk assessment is an improvement over professional judgment alone, and it can be helpful to the court in setting the conditions of probation, but it must be implemented well and should be used in conjunction with the stage of the system for which it was designed. As research continues in this area, it is important that researchers engage in dialogue with other stakeholders about risk assessment to further understand its potential uses and limitations in the criminal justice system.

ERIN HARBINSON, Ph.D., is a research fellow for the Robina Institute of Criminal Law and Criminal Justice at the University of Minnesota Law School. She conducts research in the areas of risk assessment, community supervision, criminal justice policy, and white-collar crime. Erin has experience conducting trainings for correctional staff and working with correctional agencies to implement risk assessment tools. Please contact her for any questions or resources on risk assessment. 

 

Notes

1 James Bonta, Offender Risk Assessment: Guidelines for Selection and Use, 29 Criminal Justice and Behavior 355 (2002).

2 Christopher T. Lowenkamp, James L. Johnson, et al., The Federal Post Conviction Risk Assessment (PCRA): A Construction and Validation Study, 10 Psychological Services 87 (2013).

3 D. A. Andrews., James Bonta, & J. Stephen Wormith. The Level of Service/Case Management Inventory (LS/CMI). Toronto, Canada: Multi-Health Systems (2004).

4 Bonta, supra note 1.

5 Stefanía Ægisdóttir, Michael J. White, et al., The Meta-Analysis of Clinical Judgment Project: Fifty-Six Years of Accumulated Research on Clinical Versus Statistical Prediction, 34 The Counseling Psychologist 341 (2006).

6 Id.

7 Bonta, supra note 1.

8 D. A. Andrews, James Bonta, & J. Stephen Wormith, The Recent Past and Near Future of Risk and/or Need Assessment, 52 Crime & Delinquency 7, (2006).

9 Paul Gendreau, Tracy Little, & Claire Goggin., A Meta-Analysis of the Predictors of Adult Recidivism: What Works!, 34 Criminology 575 (1996).

10 Paula Smith, Paul Gendreau, & Kristin Swartz, Validating the Principles of Effective Intervention: A Systematic Review of the Contributions of Meta-Analysis in the Field of Corrections, 4 Victims & Offenders 148 (2009).

11 D.A. Andrews, James Bonta, and Robert D. Hoge, Classification for Effective Rehabilitation: Rediscovering Psychology, 17 Criminal Justice and Behavior 19 (1990).

12 MN D.O.C., 201.022, Adult Probation and Supervised Release Supervision Standards (2016).

13 Christopher T. Lowenkamp & Edward J. Latessa, Understanding the risk principle: How and why correctional interventions can harm low risk offenders. In National Institute of Corrections (Series Ed.), Topics in Community Corrections: Assessment issues for managers 3(2004).

14 D. A. Andrews and James Bonta, The Psychology of Criminal Conduct (5th ed. 2010).

15 Id.

16 Nancy A. Landenberger & Mark W. Lipsey, The Positive Effects of Cognitive-Behavioral Programs for Offenders: A Meta-Analysis of Factors Associated with Effective Treatment, 1 Journal of Experimental Criminology 451 (2005). 

17 Andrews, supra note 15.

18 D. A. Andrews, Recidivism is Predictable and Can Be Influenced, 1 Forum on Corrections Research, Risk Assessment and Prediction 11 (1989).

19 Anthony W. Flores, Christopher T. Lowenkamp, et al., Predicting Outcome with the Level of Service Inventory – Revised: The Importance of Implementation Integrity, 34 Journal of Criminal Justice 523 (2006).

20 Andrews, supra note 3.

21 See how the authors argued the importance of using risk assessment tools at the appropriate stage they are designed to predict: Anthony W. Flores, Kristen Bechtel, & Chrstopher T. Lowenkamp, False Positives, False Negatives, and False Analyses: A Rejoinder to “Machine Bias: There’s Software Used Across the Country to Predict Future Criminals. And It’s Biased Against Blacks”, 80 Federal Probation 38 (2016).

22 Pamela M. Casey, Roger K. Warren, & Jennifer K. Elek, Using Offender Risk and Needs Assessment at Sentencing: Guidance for Courts from a National Working Group. National Center for State Courts. (2011).

23 Id.

24 Wisconsin v. Loomis, 881 N.W.2d 749 (Wis. 2016), cert. denied 2015 WL 5446731 (Sep. 17, 2015) (No. 2015AP157-CR); Malenchik v. State, 928 N.E.2d 564 (Ind. 2010).

25 The United States Department of Justice, Justice News, 2013, available at https://www.justice.gov/opa/speech/attorney-general-eric-holder-delivers-remarks-annual-meeting-american-bar-associations

26 See, for example, Jesse Jannetta, Justin Breaux, and Helen Ho, Examining Racial and Ethnic Disparities in Probation Revocation: Summary Findings and Implications from a Multisite Study, Urban Institute (2014).

27 Jennifer L. Skeem and Christopher T. Lowenkamp, Risk, Race, and Recidivism: Predictive Bias and Disparate Impact, 54 Criminology 680 (2016).

28Anthony W. Flores, Kristen Bechtel, & Chrstopher T. Lowenkamp, False Positives, False Negatives, and False Analyses: A Rejoinder to “Machine Bias: There’s Software Used Across the Country to Predict Future Criminals. And It’s Biased Against Blacks”, 80 Federal Probation 38 (2016); Skeem, supra note 28.

29 Id.

30 Id.

One Comment


  1. Landon Ascheman
    Aug 06, 2018

    A link to Minnesota’s new update, scheduled to roll out Dec 1st.

    http://www.mncourts.gov/GovernmentPartners/Pretrial-Release-Initiative.aspx

Leave a Reply

Articles by Issue

Articles by Subject