The specter of performance metrics

If you go to the homepage of the journal in which this review is published, you will easily spot a link called “Journal Metrics,” just below the journal title and the names of the editors. Clicking on it will lead you to a page that lists various scores, including the journals' impact factor and acceptance rate. Evidently, these numbers “mean” something: otherwise, they would not be placed the way they are. They offer orientation to authors and readers and function as indicators of the journals' prestige and purported “value” compared to other journals. At the same time, however, we know somehow that these scores aren't everything and we shouldn't follow them blindly. And yet, this is only one instance of a much wider phenomenon: we encounter performance metrics, such as the impact factor, in many different areas of our lives, from restaurant ratings, to personal fitness scores, or the growth rates of national economies. For anthropologists Cris Shore and Susan Wright, the rise of performance metrics such as the impact factor is a symptom of one of the defining social features of our time: the belief that such metrics help to increase social goods such as transparency or effectiveness and the according practice of measuring almost everything and everyone according to the logic of financial bookkeeping. The questions that lie at the heart of Shore's and Wright's new book Audit Culture are (a) what is driving this massive expansion of performance metrics into virtually all areas of life in the last decades and (b) what are the effects of this expansion on people, organizations and society more broadly. Do metrics, as their creators and proponents often proclaim, really lead to more transparency, accountability, and effectiveness? In tackling these questions, the authors cover broad empirical ground, ranging from the rise of the “Big Four” accounting firms and the growing influence of a technocratic managerial class on public policy, to the restructuring of public health, education, or social work following the introduction of quantitative performance measurement. They take a decidedly critical approach to the proliferation of metrics such as ratings, rankings, and indicators and focus on their detrimental and socially harmful—or what the authors call: perverse—effects. Central to their argument is the assumption of a mutually constitutive link between the rise of performance metrics and the expansion of neoliberalism as a form of capitalist governmentality that entails the erosion of the public sector and the marketization of services formerly provided by the state. In this political context, numbers serve as a means of control and discipline “at a distance” (e.g., p. 23). The book builds on previous work by the authors, who have each done extensive research on performance metrics in various fields since the 1990s and have contributed influential and widely-cited articles to the social scientific discussion (e.g., 1999, 2015). Audit Culture adopts a similar conceptual approach to their previous studies. The main difference compared to earlier contributions lies in its attempt to generalize the approach and to extend its scope both across scales (from the individual to the global) and societal sectors (corporations, transnational governance, military, education, health), aiming to weave together insights from all of these heterogeneous sites into the common theme that has largely been present since their earlier works. Apart from the authors' own work, the book can be seen in the context of a literature that has investigated the social effects of the increasing quantification in contemporary society, such as Power's The Audit Society (1999), Espeland and Sauder's Engines of Anxiety (2016), Merry's The Seductions of Quantification (2016), or Mau's The Metric Society (2019). In this context, Audit Culture shares the skeptical, de-naturalizing approach towards quantification of the other studies, while having a stronger ambition of generalizing it as a macro-feature of contemporary society, thus being more aligned with Mau's book in this regard. The book's central argument revolves around the fundamental link between the expansion of performance metrics and new forms of domination and power relations that this expansion brings about. Audit culture, according to the authors, is essentially a new “rationality of governance and a corresponding set of dispositions and practices” (p. 4) that pervades contemporary society and undermines its foundational virtues like professional judgment or democratic accountability. It instills a managerial logic in settings that were previously governed by rules serving their “core function” (e.g., p. 172): be it educating children or protecting them from abuse, healing people suffering from illnesses, or doing valuable academic research. According to Shore and Wright, the implementation of metrics as the main anchor of judgments of performance chips away at the ability of these actors to adequately fulfill their real tasks. Instead, it subjects them to an abstract system of permanent evaluation and locks them into competitive cycles with their peers. On the conceptual level, the book's central notion is domination by surveillance, drawing on Foucauldian concepts of disciplinary power and the image of the panopticon. Through quantification, sites of practice across the spectrum are subjected to “control at a distance” and can be integrated into ever more centralized systems of governance. Without being explicitly mentioned by the authors, this figure of argumentation also echoes Habermas' “colonization of the lifeworld”—thesis, as it suggests the image of intrusion of an economic logic into social contexts that have previously functioned according to their own logics. One of the contentions made by Shore and Wright particularly stands out to illustrate this notion: referring to Zuboff (2019), they argue that the critique of China's social credit system as an illiberal surveillance dystopia, which is often made by Western observers, is actually hypocritical, since analogous systems of surveillance also exist in the Western world, namely in the activities of “surveillance capitalist firms” (p. 159), such as Google, Facebook, or Microsoft. One thing the reader may notice throughout the book is a strong focus on scandals as the empirical “flesh” of the book. To name a few examples: the cover-up of the brake malfunction at Toyota (Chapter 1), the cases of fraud, corruption and private enrichment through the academy schools in the UK (Chapter 2), the “revolving door”—phenomenon in which public officials join consultancy firms to advise corporations on how to handle legislation they themselves have helped to write (Chapter 3), the case of “policy washing” through OECD assessments used by Danish education officials (Chapter 4), or the blatant failure of medical services at the British National Health Service (Chapter 5), due to an obligation of meeting certain statistical targets. While all of these cases are clearly worthy of condemnation on their own—as scandals, they already imply their own moral lesson—the question of how well they support the general argument can be raised. The authors' intention to expose the “perverse” (e.g., p. 5, 20) effects of audit culture is explicit and clear. What seems, however, less clear is the link of all these effects to the same underlying principle, that is, quantification. For some of the instances taken up by the authors, the relation to the book's main theme seems clear, such as gaming strategies by academic researchers to boost their publication count. In other cases, like Toyota and Volkswagen, or the revolving door, the relationship to the expansion of performance metrics seems less plausible to me. Moreover, I wonder if the point the authors wish to make—that we should limit the role these metrics play in our society, as they are often socially harmful—becomes more compelling by way of using outright scandals as empirical illustrations. I, for one, would have found it more convincing to use nuanced empirical material that also shows dissonances and ambiguities in the phenomenon studied. That is, I believe that the argument would have benefitted from attending to the effects that performance metrics have on routines, day-to-day practices and the variations between these effects in different domains. Further questions can be raised about the geographic and cultural scope of the argument. I have found it somewhat striking that the book does not contain any explicit qualifiers regarding the predominant focus on the UK as the main political-geographic reference of the study: the examples are, with a few exceptions, exclusively taken from the UK context. However, the generalizing language used by the authors suggests that the expansion of audit culture encompasses the entire world. This is supported by formulations like “Few countries better illustrate this assemblage of auditing and marketization better than the United Kingdom, where the rise of audit culture is most pronounced (…)” (p. 89, emphasis mine), which appears to imply that it also exists elsewhere. This can be read as a justification for the empirical focus on the United Kingdom, but it is not a systematic reflection of it. This raises questions of generalizability with regards to the geographic dimension: what accounts for the applicability of the claim to other places beyond the United Kingdom, or beyond high-income countries? Even if the same type of calculative rationality exists in say, Bolivia or Sri Lanka, one would suspect that it takes on very different shapes as it interacts with the particular histories of those places. To borrow Monika Krause's (2021) recent formulation, what accounts for using the UK as a “model case” for quantification and what are the limitations of such an approach? This is not in contradiction with the authors' argument, they may be correct after all. Yet a more explicit reflection about the diversity of geographic contexts and the geographic limits of the argument would be expected in a study like this one. Taken together, these two points are also indicative of a broader point of discussion: the question whether the occurrences of performance metrics in different fields and across scales are indeed articulations of the same basic principle. I would like to raise two challenges to the argument that performance metrics always reinforce a neoliberal agenda: (1) rankings as a means of comparing merit arguably have a longer and more varied history than the narrative presented in Audit Culture suggests. As historical-sociological studies have shown, the emergence of rankings in different societal fields like academia, the arts, or sports not only dates back to the late nineteenth century (i.e., long before neoliberalism as a political ideology took hold), but there have also been significant differences in the ways these fields have dealt with and reacted to them (Ringel & Werron, 2020). In the case of academia, in which performance metrics have traditionally played an ambiguous role, an important contribution to their development has been made by academics themselves (Wilbers & Brankovic, 2023), which is often overlooked by critics. This is not to say that affinities between neoliberal modes of governance and performance metrics do not exist—but to take their entanglement as given, instead of treating it as an empirically open question, potentially limits our empirical sensitivity to the manifold ways in which numbers are used, appropriated, subverted and even rejected in different social settings. (2) Performance metrics are also used in counter-hegemonic projects, and some studies have highlighted the emergence of “transformative indicators” in gender politics (McLeod, 2024) or “revolutionary benchmarks” in tax justice activism (Seabrooke & Wigan, 2015). Overall, this indicates the ideological plasticity of numerical representations of the world and cautions us against too quickly subsuming them under the all-encompassing concept of neoliberal discipline and power. In the cases that the authors study, the perverse effects of quantification are evident, and exposing them seems equally important from both a scholarly and a political perspective. Audit Culture undoubtedly sharpens our understanding of its effects in different areas of society. Crucially, the work of de-naturalizing and dismantling the inevitable ideological character inherent to any performance metric, as well as highlighting its potentially transformative and unintended effects, remains as important as ever. In this regard, Audit Culture makes a strong and convincing case and offers wide-ranging empirical insights. Considering the overarching argument of the book, the strong emphasis that the authors put on the inherent link between quantification and neoliberal governance appears as a point for productive and controversial discussion in the future. The author declares no conflicts of interest.