A deeply flawed study of police shootings, published in an influential journal, has been retracted by its authors. This is a positive step for the crucial debate on police violence because pundits continue to use this baseless article to dismiss concerns about racial bias in policing.
Despite the social unrest after George Floyd's death — and despite polls finding that most Americans believe the incident reflects a broader societal problem — unscientific denialism about the possibility of police racism remains prevalent among some partisans, up to and including the president. Indeed, on Tuesday, President Donald Trump echoed a prominent misleading argument — that "more white people" are killed by police than Black people, a statement offered as a rebuttal of discriminatory police violence but one that is true only because there are so many more white than Black people in the United States.
In the Wall Street Journal on June 2, an opinion article headlined "The myth of systemic police racism" argued that the "charge of systemic police bias was wrong during the Obama years and remains so today." Like many others making this case, the piece cited an article published last year in the Proceedings of the National Academy of Sciences (PNAS), by researchers at Michigan State University and the University of Maryland, who concluded, "We did not find evidence for anti-Black or anti-Hispanic disparity in police use of force across all shootings, and, if anything, found anti-White disparities." Before its retraction, the study received widespread, and largely unquestioning, coverage by news outlets across the political spectrum.
But the study was fundamentally flawed, and now the authors have admitted as much — which is why they took the extraordinary step of withdrawing it. It's important to grasp how the paper went wrong, because some people, including Manhattan Institute fellow Heather Mac Donald, the author of that Wall Street Journal opinion piece, continue to claim it was retracted only because it had become politically controversial ("I Cited Their Study, So They Disavowed It"). The authors deny this explicitly.
What did this debunked study do? Drawing on new databases assembled by the Washington Post and Guardian newspapers, the study focused on a tiny, but important, fraction of police-civilian encounters: more than 900 fatal police shootings in 2015. Of the killings, 501 involved white people, 245 involved Black people and 171 involved Latinos. The authors gathered additional information on the race, sex and experience of the officers involved. The study promised to answer two questions: which groups of civilians were more likely to be shot by police, and which groups of officers were more likely to shoot them.
But the analysis went wrong from the start. To begin to measure racial bias in police killings, careful researchers must ask: How often do officers use fatal force out of all encounters between minority civilians and the police? They should then compare this with the same analysis for white civilians, accounting for relevant differences between minority and white encounters.
That's not what the paper did. Instead, it looked only at fatal encounters and asked, in average circumstances, which group of civilians appears more often among victims? In other words, they analyzed how often fatally shot civilians were Black and Hispanic. But they confused this with a much more important question: how often Black and Hispanic civilians are fatally shot. It's a basic statistical error (violating a centuries-old tenet of statistical analysis called Bayes' theorem). These quantities can differ enormously: When officers encounter many more white civilians (due to white people's majority status, for example), the proportion of killings involving Black civilians can be small, even if encounters with Black civilians are more likely to end in shootings.
The study indeed found, unsurprisingly, that white shooting victims outnumbered Black and Hispanic victims in various circumstances. The five authors wrote that "in the typical shooting … a person fatally shot by police was 6.67 times less likely … to be Black than White and 3.33 times less likely … to be Hispanic than White" and concluded: "Thus, in the typical shooting, we did not find evidence of anti-Black or anti-Hispanic disparity." Put another way, the authors mistakenly claimed to find no evidence of racial bias simply because, among "typical" fatal shootings, there were more white civilians than minorities — thereby committing the same logical fallacy as Trump.