Beyond crime rates and community surveys: a new approach to police accountability and performance measurement

Abstract

In this conceptual piece, we argue that the current approach to police performance measurement typically based on the use of traditional police metrics has failed to achieve the desired results and that a different strategy is required. Traditional police metrics have a narrow focus on crime and the police response to it. They provide little information on how well police organizations are performing. Importantly, traditional police metrics do not incorporate input from police stakeholders in goal identification, nor do they use specifically designed indicators to assess progress towards achieving these goals. Following an analysis of the criticisms levelled at the use of traditional police metrics, and subsequent attempts to address these issues, we argue that a networked governance approach represents a more promising foundation for undertaking police organizational performance assessment. Such an approach would engage stakeholders more directly in goal identification and performance assessment, and potentially lead to more successful, responsive and accountable policing.

Introduction

In the early 1990s, a ‘businesslike’ approach to public administration known as New Public Management (NPM), began to take root in the United Kingdom, the United States, and many other western nations (Hartley 2005; Hough 2010; Micheli and Neely 2010). The impetus for NPM came, in part, from growing concerns over mounting budget deficits and the desire by governments to gain greater control over public sector organizations like the police (den Heyer 2011). The impact of these policies and the changes they created in police organizations cannot be overstated. As Reiner (2013:171) notes, the reforms aimed at policing in the United Kingdom, including the use of performance measurement, have been the most substantial in the past 50 years “and, arguably in the nearly two centuries since the establishment of the modern British police”. And, despite a shift in performance metrics to alternative such as public confidence through procedural justice, many police organizations continue to experience pressures to demonstrate accountability.

In this conceptual paper, we consider the growing emphasis on performance measurement in policing with a particular focus on the Canadian context. We explore the criticisms levelled at the use of traditional police metrics for performance assessment, then consider some of the alternatives that have been proposed for overcoming these criticisms including a shift to examining police legitimacy through procedural justice. Finally, we outline what we believe to be a more fruitful and meaningful approach to police organizational performance measurement—one that builds on well-established governance principles that offer a promising way of addressing police organizational performance concerns, while prioritizing police legitimacy and accountability. In this way, we outline a path forward for policing, that shifts organizational innovation from ‘episodic’ efforts to ‘systematic’ change (Goldstein 2018, 3).

History and background

Police organizations were originally exempted from the types of budget cuts, austerity measures and other changes imposed on public sector organizations as a result of NPM-inspired reforms because they were considered an ‘essential service’ (Butterfield et al. 2004). This situation began to change, however, in the mid-2000s as the global recession put increasing pressure on governmental budgets (Edwards and Skidmore 2006). For example, in Canada, the per capita cost of policing has risen steadily since the 1950s. Accounting for inflation, Canadian policing budgets have grown more than 40% since 1986 reaching $14.7 billion in 2016 (approximately $315 per capita and 26% higher than in 2006) (Conor 2017).

Concerns over policing costs prompted the Canadian federal government to launch an “economics of policing” initiative in 2012. This initiative focused on finding ways of delivering more ‘efficient and effective’ policing services while reducing policing costs (Drummond 2012; Leuprecht 2014; Di Matteo 2014; Public Safety Canada 2018).

Considerations of the costs of policing are clearly important since they represent a significant expenditure for municipal, provincial and federal levels of government in Canada. However, a more nuanced discussion about these costs is required given public expectations of the police, the increasing complexity of police work, and the greater time required to adequately perform policing tasks considering the legal requirements under which the police currently operate (e.g. regulations regarding the handling of domestic conflict calls). As well, there has been growing public demand for greater police accountability and legitimacy in Canada, as a result of a series of high-profile incidents involving the police, and the development of the National Inquiry into Missing and Murdered Indigenous Women and Children, and the Black Lives Matter Movement (Lucchesi and Echo-Hawk 2019; Reiti 2016). These developments have put policing high on the public policy agenda in Canada in recent years as has been the case in many liberal democracies.

Police performance assessment and the use of traditional police metrics

The current context of policing in Canada includes a growing emphasis on police performance measurement frameworks by police organizations (ICURS 2014). Historically, policy makers and police leaders have typically used existing police metrics, such as crime rates and clearance rates, to assess police performance. Indeed, the Uniform Crime Reports (UCR) were developed in the early 1900s specifically for this purpose. However, traditional police metrics such as the UCR have a very narrow focus on crime and the police response to it (which the UCR also measures poorly) while providing little information on how well police organizations are run (Maguire and Uchida 2000). Nor does the UCR provide any detail on the unique context of similar criminal incidents (e.g. one assault can be very different from another and require different police response). Furthermore, clearance rates do not provide any indication about the type of service provided by the police beyond whether an arrest was made, or the case was solved.

The four main types of information reported in existing police metrics include: crime rates, clearance rates, response times and productivity or workload statistics (e.g. number of arrests, citations, stops and checks, etc.) (Collier 2001; Fielding and Innes 2006; Maguire 2003; Sparrow 2015). These metrics do not alone address police goals, nor do they include specific indicators designed to track progress towards goal achievement. Implicitly, police organizations are understood to be, “doing a good job” in relation to how well they perform on the above-mentioned traditional police metrics, rather than what some might consider legitimate police goals. While the police have some ability to impact the rise and fall of official crime statistics, most of these changes are a result of indirect police behaviour. For example, different attending officers may record an incident as more or less severe depending on a number of factors. If this is done over time it could impact crime rates. However, traditional police metrics continue to be used despite ongoing criticisms including questions about what is being measured, what is being missed, how the data are collected, and how other data sources (like victim surveys) compare with these metrics (Maguire and Uchida 2000; Wells et al. 2005). Four general critiques of traditional police metrics are outlined below.

The first critique is that they are methodologically unsound. Critics claim that traditional police metrics like crime rates are ill defined, poorly specified, and often do not capture the actual amount of crime that occurs in a given area (Rein and Winship 1999). Reporting practices have received particular attention with respect to the methodological criticisms levelled at traditional police metrics. The research on reporting practices suggests that high reporting rates are often linked to a variety of factors (Fielding and Innes 2006). For example, increasing the number of police officers on the street could lead to higher crime rates because there are more police available to witness and report crime, rather than having to rely on the public to make these reports (Rein and Winship 1999). Additionally, crime rates can decrease if citizens fail to report crimes because they do not trust their police (Reiner 2010). Even crimes that have relatively high reporting rates, such as auto-theft and homicide, have been shown to be more correlated with an increase in security and a reduction in criminal opportunities, than the performance of the local police organization (Hodgkinson et al. 2016; Farrell et al. 2018).

A second critique notes that traditional police metrics are too narrow. They tend to focus on particular kinds of crime (e.g. public order, interpersonal violence, property crime) while failing to adequately address arguably more serious crime categories (e.g. white-collar crime, corporate crime, fraud, or cybercrime) that are often difficult or expensive to investigate, go undetected by victims, or are not reported to avoid damage to the affected organization’s reputation (Mailley et al. 2008; Sommer 2004).

Traditional police metrics are also too narrow because they fail to address important aspects of police work (Maguire 2003). For example, up to 70% of police calls for service are not crime related (McFee and Taylor 2014; Di Matteo 2014), with many being in response to mental health issues (Coleman and Cotton 2014) or other non-criminal social problems like addictions, poverty, and homelessness (Goldstein 2018). These non-criminal social problems go unrecorded by traditional police metrics, but account for a large proportion of what the police actually do in the course of carrying out their normal duties. As well, traditional police metrics fail to account for the time devoted to administrative and operational tasks such as report writing, attending court, attending hospitals, and directing traffic (Leuprecht 2014; Murphy 2004).

Traditional police metrics also place relatively little emphasis on crime prevention and other practices that can enhance community safety and improve police-stakeholder relationships (Wells et al. 2005). Ironically, excluding these activities from the metrics used to assess police organizational performance penalizes police organizations that are working proactively to prevent crime and enhance community safety (Collier 2001; Feltham and Xie 1994).

A third critique of traditional police metrics focuses specifically on the way they are used to influence public perception. Some observers note that in order to demonstrate significant declines, crime rates need to be high in the first place. These critics claim that it is difficult to reduce crime rates below a certain level (i.e. a statistical floor effect), and while attempting to do so may be politically expedient, it wastes valuable police resources on a pursuit that cannot succeed (Sparrow 2015).

Additionally, crime rates have been declining internationally for more than two decades and these declines cannot be explained by individual police service strategies (Farrell 2013). In fact, the police actually have limited ability to influence the crime rate with some critics arguing that they play a largely symbolic role that primarily serves to reassure the public (see Manning 2003). This does not mean that reducing crime is not a valuable policing goal but needs to be considered as one of the several contributions made by modern policing that is worth measuring.

The fourth critique focuses on the negative consequences associated with the use of performance metrics themselves. For example, some observers argue that an excessive emphasis on crime control metrics such as arrest rates, can lead to violations of civil liberties or ignoring community satisfaction with the police (Moore and Poethig 1999). This is problematic, since, according to procedural justice researchers, public confidence in the police has been found to be an important predictor of overall feelings of community safety as well as compliance with the law (Shilston 2011; Tyler and Murphy 2011).

Others have suggested that overemphasizing performance measurement can lead some officers to ‘game the system’. This can take various forms, including failing to report negative incidents and manipulating the data to make it look like performance is good even if it is not (Courty and Marschke 2004; Espeland and Sauder 2007). Some have suggested that gaming the system can become institutionalized to the point that the main goal of reporting and recording practices becomes production of the expected results rather than accuracy (Loveday 2000). As Patrick (2009) points out, statistics can and will be manipulated when the stakes are high.

Importantly, traditional police metrics fail to tell policy makers and stakeholders very much about what the police are actually doing or how well a police organization is operating. Critics have argued that traditional police metrics more accurately reflect the activities of offenders than those of the police (Hoogenboezem and Hoogenboezem 2005).

In response to the wide-ranging criticisms levelled at traditional police metrics, numerous authors have suggested a variety of alternatives including developing and using better indicators, qualitative measures, or conducting public/staff satisfaction surveys (Shilston 2011; Wells et al. 2005; Rosenbaum et al. 2015). The policing literature is replete with new models and other alternatives to current police performance measurement schemes as many police services move away from traditional performance metrics or have discarded these metrics altogether. Barlage et al. (2014) suggest that performance measurement should focus on objective versus subjective measures by using a multi-trait/multi-method (testing response bias across different response groups while testing the validity of the measures) approach. In England and Wales, the PEEL (Police Effectiveness, Efficiency and Legitimacy) program brings together numerous measures of police activity to provide the public with a standard to evaluate police performance (HMIC 2017). Barton and Beynon (2011) introduced the PROMETHEE ranking technique that uses secondary data on sanction detection levels to improve police performance in a specific area. And others are attempting to identify better measures and indicators (Guilfoyle 2015), measure additional factors such as city-level attributes (Sharp and Johnson 2009), and use citizen surveys to evaluate police-citizen encounters and public satisfaction (Shilston 2011; Rosenbaum et al. 2015).

While the proposed alternatives to traditional police metrics may provide better indicators or ways of measuring, they do not resolve the fundamental challenges inherent in the use of traditional police metrics for assessing police organizational performance (Maguire and Uchida 2000). An important consideration for those concerned about this issue is that existing police metrics were not specifically designed to address the questions inherent in determining how well a police organization operates and fulfils its role in community safety. For example, in the current context there are few measures of what might indicate that progress has been made towards achieving stated policing goals.¹ This is remarkable because what the police are doing about crime may actually be antithetical to what governments and the public desire.

Rethinking performance metrics according to organizational goals

We believe that a different approach to police organizational performance measurement is required: one that is more attuned to the expectations of local stakeholders, including governance and oversight bodies, as well as community members. An important starting point would require acknowledging the need for greater input and involvement by local stakeholders in discussions regarding the role of the police in a democratic society, realistic expectations for those undertaking these roles, and a consideration of the expectations of the people being served (Goldstein 2018; Sparrow 2015; Moore and Braga 2003). Appropriate organizational performance assessment frameworks, specific to the police service, could then be developed based on well-established evaluation principles (goal identification, logic models, tailored indicators, regular data collection/analysis, ameliorative action based on evidence, etc.).

A growing body of literature provides important insights into what an alternative approach might look like. Newman (2001), for example, discusses a model that shifts governance away from current practices towards a more networked form, in which stakeholders are directly involved in the identification of organizational goals and how these are to be pursued (expectations about police behaviour). In this way, networked governance can provide stakeholders with the voice they require to help shape police service policies and practices, so their needs and expectations can be more adequately addressed (Fung 2009; Bryson et al. 2014).

Networked governance has been gaining traction in community safety and public service management. For example, in Vancouver, Canada, a collaborative model of decision making was used to transform homelessness policy and allot funding to mutually agreed upon services (Doberstein 2016). This model demonstrated that networked governance is far more advantageous than current governance approaches, highlighting in particular, the importance of the ‘process’ of networked governance.

The policing literature includes other examples of a networked approach to police governance and how this might work at the organizational level. Moore and Braga (2003), for example, assert that stakeholders should be involved in measuring how the actions of the police contribute to the safety of their communities. An example of this is demonstrated in the Nexus project in Australia where participating police organizations work with communities to determine what they want their police organizations to achieve in relation to community safety, and what their vision of policing is for the 21st century (Shearing and Marks 2011). Similarly, “Measuring what Matters”, a federally sponsored project in the US, includes stakeholder engagement as a means of identifying public expectations of the police regarding community safety (Moore 2002; Moore and Braga 2003; Sparrow 2015).

If such an approach were to be used more broadly in policing, police leaders would have to work more closely with stakeholders, be more responsive to their input and guidance, and be more flexible and facilitative in their leadership roles (Moore 2000; Hartley 2005). At the same time, such an approach would require stakeholders to be more engaged and willing to participate in co-producing the outcomes they desire. In addition to community members, we argue it is important to engage institutional stakeholders who are informed about the issues at hand (Simmons 2007). These institutional stakeholders should be drawn from the partners who comprise the community safety constellation and have a stake in working towards safe and healthy communities (see Caputo and McIntyre 2015). These include health care providers, educators, social service providers, and even private security firms (Shearing and Marks 2011). Finally, to address issues of conflict and consensus, unbiased and experienced facilitators would contribute to the governance process (Morley and Trist 1993; Hodgkinson 2018).

A networked governance approach would address the issue of performance measurement in a way that is appropriate for police organizations in the 21st century (see Council of Canadian Academies 2014). It would lead to goal setting for the police that is:

• a. Deliberate and intentional because it reflects the results of a collaborative process and negotiated agreements;

• b. Aligned with what a police organization can actually do;

• c. Aligned with stakeholder expectations; and,

• d. Linked to a suitable reference point for after-the-fact evaluation;

This type of approach would place the interaction between the police and their stakeholders at the centre of the oversight and accountability process. The police would be required to keep their stakeholders informed of the progress they are making toward jointly determined goals through the use of indicators that were also jointly identified. Based on an assessment of these performance measures, changes could be made as necessary to help a police organization become more successful going forward.

We acknowledge that this proposal is a complex task. Police organizations have several stakeholders (outlined above) who often differ in their expectations of police. We do not believe that consensus around these goals is a possibility or even a worthwhile pursuit. In fact, some argue that conflict is necessary to the networked governance process (O’Leary and Vij 2012). Rather, we argue that the process of networked governance is meant to create context specific metrics, that measure the ability of police to address locally identified needs and problems while acknowledging that these issues are subjective to local needs and experiences.

While the proposed approach calls for the collaborative setting of strategic goals, we recognize that operational decisions about how to achieve these goals should be left to police professionals. The distinction between strategic goals and operational decisions is an important one because it helps to clarify the role of the various participants in the community safety enterprise. For example, while police will continue to respond to calls for service (violence-related calls would continue to take priority), how they frame these issues, and strategize their response will be co-determined and in line with mutually agreed upon priorities, such as those created by organizations like the American Bar Association’s Standards for the Urban Police Function (ABA 2018).

A networked governance approach goes further than requiring the police to seek information or input from the community about its concerns or priorities as is done in the existing community consultation process used by most police organizations. Rather, the goals that emerge in networked governance should be co-determined by police leadership and their stakeholders such as governance or oversight bodies, institutional partners, and other community representatives (McKnight and Kretzmann 1996; Hodgkinson and Saville 2018).² In doing so, police response becomes imperative rather than an elective that can be easily ignored by police leaders or to which they can merely pay lip service (Klockars 1988). Additionally, these goals are implemented at the organizational level, contributing to ‘systematic’ rather than episodic changes in policing and community safety (Goldstein 2018). These goals then become the standard that stakeholders can use to assess the performance of their police organizations and this process will contribute to greater transparency as well as police accountability and legitimacy.

Conclusions

The introduction of police reforms based on NPM-inspired policies has had a dramatic impact on police organizations. This has been most evident in the increasing emphasis placed on police organizational performance measurement including the narrowly conceived performance imperatives exemplified by the rubric of efficiency and effectiveness and “value for money”. We submit that police performance measurement conceived of in this way fails to achieve its objectives while missing much of the public’s concerns related to police legitimacy and accountability. Further, the reliance on existing traditional police metrics has privileged crime control over other policing functions even though this accounts for between a quarter and a third of all calls for service received by the police. Clearly a much broader focus is required for police performance assessment if the myriad concerns related to community safety are to be addressed.

Our view is that a more fulsome approach to assessing the performance of police organizations is required—one this is more “bottom-up” and organic in nature. That is, it should be based on the collaborative identification of policing goals arrived at by a closer working relationship between the police and their primary stakeholders. Moreover, once such goals are identified, appropriate indicators should be created for assessing whether progress is being made towards achieving them. Questions about police organizational performance would then revolve around the collection and sharing of the requisite information on an ongoing basis among police leaders and their stakeholders. Assessments of the performance of a police organization could then be made based on whether and to what extent it is achieving its stated goals. In this way, the assessment of police organizational performance would be based on evidence that specifically addresses this issue rather than on the expedient use of narrow crime-based and organizational metrics (work load statistics) that are readily at hand. In the proposed approach, police organizational performance would be assessed against the backdrop of broader community safety goals while increasing the potential for more democratic, responsive and accountable policing organizations.