Federal surplus military equipment distributed to local law enforcement agencies doesn’t reduce crime as earlier studies have claimed because those studies were based on unreliable, flawed data, an Emory University political science paper published today shows.
The scholarly peer-reviewed paper, “Counterevidence of Crime-Reduction Effects from Federal Grants of Military Equipment to Local Police,” was released by the London-based journal Nature Human Behaviour.
“This is a cautionary tale about the importance of oversight. The most important thing for policy makers and the public to know is that you can’t justify giving surplus military equipment to police departments on the grounds it will lead to a reduction in crime,” says Tom Clark, Charles Howard Candler Professor of Political Science at Emory.
“There is no evidence for that. You can’t claim this program is important because it reduces crime.
“If you are going to engage in policy-making experiments, it is important to include resources and requirements for reporting so that policy analysts can study whether the policy is working,” Clark continues.
Their findings are a culmination of several years of study by a political science team at the Politics of Policing Lab at Emory University. Lab co-directors Clark, Michael Leo Owens and Adam Glynn — along with political science graduate students Elisha Cohen, Kaylyn Jackson Schiff and Anna Gunderson, now Louisiana State University assistant professor of political science — collaboratively conducted the research and co-authored the article.
“I don’t think we have ever seen the scrutiny of police on such a national level like what we are seeing right now,” says Owens, associate professor of political science at Emory. “And I think in particular, in light of the protests, and how many police appeared at them looking like soldiers and carrying weapons that looked like what soldiers would carry and some of the vehicles which might have been surplus military equipment, it really forces one to be clear about the claims that are made about the transfer of military equipment, particularly arms and vehicles and surveillance equipment, to police.
“We want to have faith and confidence in the social science that is out there in light of that,” Owens says.
Replication of previous studies finds data flaws
The conclusions of the Emory scholars came down to two things: data and replication of two prominent studies. The original 2014 data utilized in those previous studies came from the federal 1033 Program, which distributes surplus military equipment to local law enforcement agencies.
That data wasn’t complete and accurate enough to make claims about crime reduction, the Emory researchers found. And newer, more complete data still doesn’t support the assertion that surplus military equipment provided to local law enforcement agencies reduces crime, they argue.
“When we looked at the data and ran the replications, nothing looked like the results being cited by the Trump administration. We spent a year trying to diagnose the problem,” says Gunderson, first author of the paper, who was part of the research team as a graduate student at Emory.
The earlier studies, published in 2017 by the American Economic Association, claimed local law enforcement agencies with surplus military equipment from the 1033 Program reduced crime. Surplus military equipment can include everything from rifles and armored vehicles to gym equipment and office supplies.
At the time, those two studies garnered a lot attention from both the media and policy makers because of the claims of deep crime reduction. Both studies based their analysis on federal data released as part of a Freedom of Information Act (FOIA) request by National Public Radio in 2014.
About 30 years ago, as part of the war on drugs, the U.S. government’s Department of Defense through the Defense Logistics Agency (DLA) began issuing surplus military equipment (SME) — including weapons, vehicles and attire — on a limited basis to local law enforcement agencies. Congress later expanded the program.
As a result of the 2014 killing of Michael Brown in Ferguson, Missouri, and subsequent protests against police violence and militarization, the Obama administration placed limits on the kinds of surplus military equipment local law enforcement agencies could receive. The Obama rules also required better record keeping: Law enforcement agencies were required to do quarterly SME inventories and routine audits.
In 2017, the findings of the two American Economic Association papers took on increased relevance because they were touted by the Trump administration as justification to restore access of certain surplus military equipment — primarily bayonets and trucks — to law enforcement agencies that had been previously prohibited.
When Emory scholars read the studies, they noticed statistical flaws in the analysis. They set out to rigorously test those two previous studies’ claims by replicating them. They utilized the same 2014 NPR data and applied the studies’ same methods of analysis.
What immediately got the attention of the Emory scholars was that the studies were doing analysis at the county level, not the municipal level (i.e., the individual jurisdictions of cities). So, there wasn’t a way to directly compare which local agencies received SME and their specific crime rates. That’s because the federal government only reported the 1033 Program data at the county level.
“The problem with doing analysis at the county level is that it is possible crime is being reduced for the urban agencies while surplus military equipment is being requested by suburban agencies. If you do the analysis at the county level, it will appear that surplus military equipment is causing the reduction in crime but that can’t be happening because (crime rates) are going down in a different location from the surplus military equipment,” says Glynn, Winship Distinguished Research professor and associate professor of political science and quantitative methods and theory.
The Emory researchers replicated the previous studies and came up with the original results based on the flawed data. But they went further by drawing on more recent, more complete agency-level data from the DLA, and generated new results.
That new data was a result of the Obama administration’s regulations requiring more accurate records including routine audits of police departments requiring quarterly inventories. This provided an opportunity to study the quarterly locations of the SME at the level of local law enforcement agencies.
It was only after Emory used the new, agency-level data in analysis that they determined the SME didn’t reduce crime.
“It crystalizes so many of the concerns and claims both pro and con about policing in the U.S. It raises the matter of funding the police and how do we provide resources to the police — through money or giving them equipment. It raises the matter of police militarization — that the police look and act like they are soldiers at war against citizens,” Owens says. “And it raises questions about efficiency — costs and benefits.
“It raises too questions about taxpayer dollars and why is there so much surplus equipment,” he notes. “Plus, it raises the question of whether there are ways to have greater civilian oversight of police and the things they do that are tied to the 1033 Program.”
The Emory team also learned the data has other significant problems. They found discrepancies and missing information in the records being kept by the federal government.
All the 2014 data showed was what had been shipped and where. There was no information about what happened once an agency received it — whether it was transferred to another agency, broken, destroyed, returned to the federal government or actually used by the department or agency.
Today, Nature Human Behaviour is simultaneously publishing another paper from the University of Michigan that shows results similar to the Emory study. Combined, they suggest previous studies justifying giving surplus military equipment to local law enforcement agencies are based on unreliable data and draw incorrect conclusions.
“I think it is exciting and rewarding largely because the work we do is very much focused on our own internal audiences and talking to other academics,” Clark says. “This is the kind of paper where real world policies were issued and research has been used to make decisions at the U.S. federal level. I hope it will have a greater chance to be seen and noticed by relevant actors in the political world and media.”