The risk of negative feedback loops in some weighted measures of crime harm

Abstract

Analyses of crime based upon aggregate counts of different crime types have restricted value, because they count all crime types equally irrespective of the harm caused. In response to this problem, a series of weighted measures of crime harm have been proposed. In this short contribution, we contend that the use of some crime harm metrics to inform police deployment practices has the potential to reinforce ethnic disparities in the criminal justice system through the creation of unintended negative feedback loops. We focus our analysis on the Cambridge Crime Harm Index and the Office for National Statistics (ONS) Crime Severity Score, the preeminent crime harm indexes in England and Wales. We conclude that the ONS Crime Severity Score, which is based on mean sentencing outcomes, does give cause for concern in some contexts. There is currently no evidence that the Cambridge Crime Harm Index, based on sentencing guidelines, presents the same problems.

Introduction

The last two decades have seen the development of weighted crime harm indexes in jurisdictions around the world (Curtis-Ham, 2022: 180–182). These are intended to provide a more meaningful measure of police-recorded crime than typical crime counts, which count all crimes equally regardless of impact (Sherman et al., 2016). The case of England and Wales illustrates the relevance of considering crime harm. This jurisdiction has seen an overall decline in crime since the early 1990s (Farrell et al., 2014). Despite this, the emergence of complex problems (e.g. online child sexual exploitation, human trafficking, modern slavery), increased demand around longstanding issues (e.g. domestic abuse, homelessness, mental health problems), and reductions in police funding and staff (Home Office, 2022: 9), have placed significant pressure on the police. Weighted crime metrics can therefore assist by targeting scarce police resources on the people causing, and the places experiencing, the greatest harm.

In this paper, however, we highlight a potential problem that could arise from the uncritical application of some weighted crime metrics, namely, the risk of exacerbating the ethnic inequalities detected in the criminal justice system. To our knowledge, this issue has thus far gone unnoticed in the growing literature on this topic. After defining key terms, we begin by noting the global development of these tools and discussing two principal methods of calculating crime weights. We then turn our attention to England and Wales, and the Cambridge Crime Harm Index (CHI) and the Office for National Statistics (ONS) Crime Severity Score (CSS), noting key similarities and differences between these tools. In the main body of this short article, we outline the process whereby some metrics could exacerbate existing ethnic disparities in the criminal justice system. We conclude by encouraging critical reflection upon the construction and use of weighted measures of crime harm.

A note on terminology

In England and Wales, court sentences are required to reflect the ‘seriousness’ (or severity) of the offence. Assessments of seriousness encompass both the culpability of the offender and the ‘harm’ caused (or at risk of being caused) by the offence (Sentencing Guidelines Council, 2004). In other words, measures of seriousness (or severity) include but are not limited to the harm caused to victims and communities. To be clear, then, seriousness (or severity) on the one hand, and harm on the other, are not synonymous.

Background

Recent years have seen the emergence of methods for weighting different offence types to reflect the harm caused by different offences and shape the police response. Most weighted measures of crime operate in one of two ways. The first approach, utilised in countries including Canada, Denmark, Hungary, Japan and various American states, uses sentencing guidelines to calculate the weights attached to different crimes: the greater the (minimum, maximum or median) sentence, the greater the weight. The second approach, used in Australia, Ireland, Italy, New Zealand, and Sweden, often in the absence of official sentencing guidelines, bases the weights on actual sentences (see Bland & Ariel, 2020: 63–82; Curtis-Ham, 2022; Ruitenberg & Ruiter, 2022).¹

Key developments in England and Wales

Taking the first approach, Sherman et al. (2016) advanced the Cambridge CHI, which uses the custodial ‘starting point’ in the Sentencing Council’s Sentencing Guidelines to weight different crimes. More harmful crimes are expected to have higher ‘starting points’ producing greater weights. Bland and Ariel (2020: 70 − 1) describe the Cambridge CHI as the ‘catalyst for a relative explosion’ in similar tools in jurisdictions including Denmark, Sweden, Australia and New Zealand.

The UK’s ONS advanced an alternative weighted measure of police-recorded crime to supplement traditional crime counts (ONS, 2016). The ONS CSS is intended ‘to reflect the relative harm of an offence to society and the likely demands on the police’ (p. 2). In contrast to the Cambridge CHI, the ONS CSS takes the second approach, originally weighting offences based on mean sentencing outcomes over five years ending December 2015 (p. 4). This was chosen as an ‘objective measure, reflecting how society views crimes differently … based on legislation set by Parliament on behalf of the public’ (p. 4).

The same but different

Both the Cambridge CHI and the ONS CSS offer weighted measures of police-recorded crime. However, Ashby (2017) found that they produced markedly different estimates of harm for similar crimes, leading to different crime harm totals for areas, and cautioned against using them interchangeably. He concluded that both are imperfect but preferable to aggregate crime counts, and ‘there does not appear to be any reason to prefer one measure over the other’ (p. 449).

Others take a different view. Bland and Ariel (2020) questioned whether the ONS CSS weightings, which are based on sentencing decisions that reflect myriad considerations beyond the harm caused by the offence (e.g. criminal history, personal mitigating factors, guilty plea discounts, etc.), provide a reliable measure of harm (p. 76–77). Similarly, Sherman et al. (2020) contended that the inclusion of offender-related factors makes the ONS CSS an unreliable measure of crime harm (Sherman et al. 2020).

Racial disparities and weighted crime metrics

The Statistics on Ethnicity and the Criminal Justice System 2022 (Ministry of Justice, 2024) show the over-representation of minority ethnic groups at many stages of the criminal justice system compared to their representation within the general population. The over-representation of minority ethnic groups in the raw (or ‘unadjusted’) data is not, however, evidence of differential and discriminatory treatment, but could have legitimate explanations (e.g. variations in offence type or criminal history by ethnicity) (p. 8).

A growing body of research from England and Wales has gone beyond the raw data, however, and after controlling for factors that could legitimately affect outcomes, found evidence of ethnic disparities at key stages of the criminal justice system. For example, Uhrig (2016) found that despite self-reports showing that members of minority ethnic groups are less likely to commit crime than white people, arrest rates were markedly higher for Black and Mixed Heritage males and females, and for young Black males (p. 12). A recent study of demographic disparities in charging decisions in England and Wales found that after controlling for sex, age and offence type, minority ethnic suspects were significantly more likely to be charged with an offence than white British suspects (CPS 2023). When being tried at the Crown Court, minority ethnic defendants were more likely than white defendants to plead ‘not guilty’ (Uhrig, 2016: 28), which (for those ultimately found guilty) limits the prospect of receiving a sentence reduction because of a guilty plea.

Additionally, several studies have found evidence of unjustified ethnic disparities at the sentencing stage. Isaac (2020) examined the impact of an offender’s sex and ethnicity on sentences imposed by the Crown Court for drug offences. Black, Asian and ‘other’ offenders were more likely to receive an immediate custodial sentence for drug supply offences than white offenders, whilst Asian offenders received longer custodial sentences than white offenders (p. 4). Hopkins et al. (2016) explored the relationship between ethnicity and custodial sentences in the Crown Court, and found that the odds of imprisonment for indictable offences were higher for Black, Asian and Chinese offenders than white offenders (p. 1). This finding varied by offence type, and was particularly marked in relation to drugs offences, where the odds of a custodial sentence for Black, Asian and Minority Ethnic offenders was more than twice the odds for white defendants (p. 9).

Criminal justice outcomes are shaped by myriad factors that are not all recorded in administrative datasets (e.g. offenders’ culpability, or genuine remorse). Thus, it is impossible to control for all factors that could legitimately affect outcomes. Although these studies failed to control for all relevant case characteristics, Pina-Sánchez et al. (2022) used sensitivity analysis to demonstrate that the ethnic disparities were too strong to be explained away by unmeasured confounders.

Thus, there is growing evidence of unjustified racial disparities at different stages of the criminal justice system in England and Wales. Further, theories of cumulative disadvantage suggest that rather than being episodic, disadvantage may build as people progress through the criminal justice system (Kurlycheck & Johnson 2019). Figure 1 illustrates how ethnicity (Eth) might affect the ONS CSS (top) and the Cambridge CHI (bottom).

Fig. 1

Causal graph indicating the criminal justice decision points present in the configuration of the ONS CSS (above) and the Cambridge CHI (below). Solid lines represent effects documented in the literature. Dashed lines represent effects not proven in the literature

The top diagram illustrates how the main causal mechanisms linked to ethnic disparities in criminal justice outcomes (i.e. arrest [Arr], charge [Chr], conviction [Con], guilty plea [GP], and sentencing [Stc]) feed into one another, and ultimately into the ONS CSS. The bottom diagram concerns the relationship between ethnicity, the sentencing guidelines [SG] and the Cambridge CHI.

Ethnicity and the use of weighted metrics to target people and places

The consequences of our observations are significant for crime harm indexes like the ONS CSS that utilise mean sentencing outcomes to derive offence weights. The ranking of crimes by harm caused may be distorted as a result of unwarranted disparities in the criminal justice system, with crime types where ethnic minorities are heavily represented appearing more severe than where they are not. Drug offences provide an example. In 2022, 33% of adult males convicted of drug offences in England and Wales were from Black, Asian, Mixed Heritage or ‘other’ ethnic groups and 67% were white (Ministry of Justice, 2023). Hopkins et al. (2016) found that ethnicity had a ‘strong effect’ on the odds of imprisonment for minority ethnic offenders for drug crime (see also Isaac, 2020). Our contention, then, is that the large proportion of minority ethnic offenders within this crime type may have inflated the ONS crime severity weighting for drug crimes.²

As noted above, the original ONS CSS weights were based on mean sentences from a five-year period ending in December 2015. Since then the proportion of offenders convicted of drug offences who were white has fallen year-on-year, from 72% in 2015 to 67% in 2022 (Ministry of Justice, 2023). Further, research by Pina-Sánchez et al. (2019) found an overall 39% increase in sentence severity between 1999 and 2018 for ‘indictable only’ offences including drug offences. Calver (2024) has documented this process of sentence inflation with more recent data. If the severity metric for drug crime was recalculated using more recent sentencing data, it would be further inflated by these trends.³

Our concern, then, is that the use of mean-sentence-based metrics to inform police deployment practices risks further exacerbating ethnic disparities. In the worst-case scenario, there is the potential to create a harmful feedback loop, whereby minority ethnic individuals and places with large minority ethnic populations receive increased police attention on account of (high) harm scores fuelled by racial bias, generating more offences to feed into future weighting exercises, and potentially reinforcing the existing distortions that contributed to sentencing disparities.⁴

In contrast, the Cambridge CHI weightings are based on the custodial ‘starting points’ for offences as stated in the Sentencing Guidelines (SG). If the ‘starting points’ for offence types disproportionately linked to ethnic minorities were longer than those for comparable offences disproportionately linked to white British offenders, unwarranted ethnic disparities might also affect the Cambridge CHI, as illustrated in the bottom diagram in Fig. 1. There is no evidence to suggest this is the case, however. A report entitled Equality and Diversity in the Work of the Sentencing Council, which presents the findings of research to examine whether the Council’s work might cause variations in outcomes for different groups, found nothing that supports this hypothesis (Chen et al., 2023).⁵

Discussion and conclusion

Weighted measures are an alternative to aggregate crime counts. They have the potential to assist police to target scarce resources on high-harm offenders, and the areas most affected by crime harm, and afford a picture of local and national trends in crime harm over time. The Crime Survey for England and Wales estimated that levels of victimisation are highest amongst Black, Asian, Mixed Heritage, and other minority ethnic groups compared to the white group (ONS, 2021), which suggests that targeted crime prevention strategies could be of particular benefit to these communities. At the same time, this article suggests that the use of metrics based on mean sentences could have unintended and undesirable consequences.

In some jurisdictions, sentence-based CHIs have been constructed to minimise the impact of offender culpability. For example, first-time offenders’ median sentences were used in Australia (House & Neyroud, 2018) and estimated starting point sentences (based on the 15th percentile of actual sentences) were deployed in New Zealand (Curtis-Ham & Walton, 2017) to mitigate the effect of criminal history on CHI calculations. As Fig. 1 shows, however, criminal antecedents are not the only offender-related determinant of sentencing outcomes, nor the only one influenced by ‘race effects’. Whether racial bias fuels the marked over-representation of Indigenous people in Australasian criminal justice systems is a longstanding concern (O’Brien, 2021), and Australian research found that even responses to first-time juvenile offenders varied by Indigenous status (Papalia et al., 2019). Thus, even indexes that use actual or proxy starting point sentences for first time offenders to estimate harm, when used to target people and places, could amplify ethnic disparities.

Since its original release, the ONS CSS has been updated. The current tool uses sentencing data from a five-year period ending December 2019 to calculate offence weights (ONS, 2023). Whilst the derivation of the tool remains the same, this will not resolve the problems raised in this article. As noted above, for offences where increases in ethnic disproportionality have been accompanied by increases in sentence severity, the problem may even be exacerbated. That the ONS CSS weight for trafficking in controlled drugs has increased from 513 in the original index to 667 in the latest version supports our argument.

The solution is not to abandon weighted measures, but to tackle upstream ethnic disparities. While they persist, we would encourage policy makers and practitioners in England and Wales (and beyond) to think carefully about the construction of weighted measures of crime harm and the contexts in which they are deployed. Whilst sentence-based indexes could be used to counter racial-bias, e.g. by using them to identify individuals and communities for preventative or therapeutic interventions, we contend that their use to target police resources risks exacerbating ethnic disparities.