New Light on the Accuracy of the CIA’s Predator Drone Campaign in Pakistan

Widely-cited reports of the inaccuracy and disproportionality of civilian to militant deaths in the CIA’s ongoing Predator drone campaign against the Taliban and al-Qaeda in Pakistan are grossly misleading. The most detailed database compiled to date, assembled by the authors of this article, indicates (among other important findings) that the strikes have not only been impressively accurate, but have achieved and maintained a greater proportionality than either ground operations in the area or targeting campaigns elsewhere. [1]

This finding is striking because highly critical reports over the last year, emanating in particular from the Pakistani press, have impugned both the accuracy of the CIA’s drone strikes in the tribal areas of that country and the proportionality of the civilian collateral damage they cause. In April 2009, for example, the Pakistani daily The News published an article by terrorism expert Amir Mir reporting Predator strikes had killed only 14 high value al-Qaeda targets but were responsible for 687 civilian fatalities – a 1:49 ratio of terrorist to civilians (The News [Islamabad], April 10, 2009; see also Terrorism Monitor, February 19).  To put it another way, Mir’s report suggested that 98.14% of fatalities associated with the Predator strikes were civilians. On February 1 of this year, Mir added that in January 2010 alone 123 Pakistani civilians had been killed in ten errant CIA drone strikes, while only three al-Qaeda targets had been eliminated (The News, February 1).  These shocking statistics were picked up and widely reported in the Western press (New York Times, May 16). Along similar lines another Pakistani daily, Dawn, reported in January 2010 that “of the 44 Predator strikes carried out by U.S. drones in the tribal areas of Pakistan over the past 12 months, only five were able to hit their actual targets, killing five key al-Qaeda and Taliban leaders, but at the cost of over 700 innocent lives. For each al-Qaeda and Taliban terrorist killed by the American drones, 140 civilian Pakistanis also had to die” (Dawn [Karachi], January 2). Such reports have reinforced the notion that drone strikes are not only inaccurate, but seem to kill innocent civilians in wildly disproportionate numbers. However, even a cursory investigation of The News and Dawn’s own reports of Predator strikes on a case by case basis reveals that the great majority of fatalities are reported as “militants” or “suspected militants.” These discrepancies highlight the need for a thorough, independent and verifiable investigation of the reported toll of the U.S. drone campaign.

Methodology

The authors of this article have compiled a database over the last year that draws extensively on Pakistani newspapers (in their English language versions), and Western newspapers of record (primarily the New York Times and Washington Post).  Only cases in which it was possible to compare multiple independent reports of drone strikes have been included.  Where reported numbers of fatalities differed, we have favored the most detailed and updated account, always using low-end estimates of suspected militants slain. [2] All children under 13 and women were assumed to be civilian, along with all of those specifically identified as civilians, bystanders or locals uninvolved in the fighting. Where it was impossible to determine whether a person killed was properly categorized as a suspected militant or civilian, we assigned them to the category of “unknowns.”  By systematically applying these simple rules to the universe of available information, we have assembled what we believe to be the most comprehensive and credible database of Predator drone strikes in this operational region currently available.

The Findings

According to our database, as of June 19, 2010, there have been a total of 144 confirmed CIA drone strikes in Pakistan, killing a total of 1,372 people.  Of those killed, only 68 (or 4.95%) could be clearly identified as civilians, while 1,098 (or 80%) were reported to be “militants” or “suspected militants” (see Figure 4).  As these terms are used somewhat interchangeably by the Pakistani press, we simply classified all of them as “suspected militants.”  This category of suspected militants includes 50 high value targets – that is, al-Qaeda and Taliban leaders, whether local commanders or senior militant chiefs.  The status of the remaining 206 (or 15% of) individuals killed in drone strikes could not be ascertained, and consequently they were assigned to the category “unknown.” The inclusion of this indeterminate category is admittedly frustrating but unavoidable given the limited and sometimes contradictory reports emanating from the inaccessible tribal areas.  It is important to stress, however, that even if every single “unknown” is assumed to in fact be a civilian, the vast majority of fatalities would remain suspected militants rather than civilians – indeed, by more than a 4:1 ratio. [3] On the more precise count of civilians (leaving “unknowns” aside), we found an even more imbalanced ratio of approximately 16.5 suspected militant fatalities for each civilian death. [4] Equally striking, we found a 1.36 to 1 (or close to 1 to 1) ratio of civilians to high value target fatalities (in stark contrast with Mir’s 49 to 1 report).  Finally, in contrast to Mir’s report of 123 civilian casualties in January 2010 (with only 3 al-Qaeda targets killed), we found 0 civilians, 85 suspected militants and 16 unknowns killed in that month.

We also wanted to be careful to address any concerns that Western papers, including those of record like the New York Times and the Washington Post, might be underreporting civilian casualties, and that by relying at times on their stories we were introducing a downward bias into that element of our data.  We therefore ran a second analysis, applying the same categories and criteria solely to the Pakistani news sources (specifically, Dawn, The Daily Times and The News).  The results were even more striking.  We found reports of 1,061 suspected militants killed, 48 civilians, and 251 unknowns, for a ratio of 22.1:1:5.2.  Although some ambiguity is suggested by the slightly higher number of unknowns, the lower absolute number of civilians in the Pakistani data along with the higher proportion of suspected militants to civilians indicates that, if anything, leading Western news sources are leaning towards over-reporting the number of civilian casualties and underreporting suspected militants killed, at least in relation to representative local news sources.  At any rate, we take this result based solely on Pakistani data to reinforce our main finding of a surprisingly high reported rate of suspected militant fatalities to civilians, particularly in the light of a number of widely circulated stories sharply to the contrary.  

Indeed, even our main finding of a 16.5:1 suspected militant to civilian fatality ratio significantly exceeds that recently reported on CNN for the period of 2006 to 2009 by Peter Bergen and Katherine Tiedemann, who suggested that around two-thirds of those killed in the campaign were suspected militants. [5]  Some (although not all) of the variance here can be accounted for in a difference of methodology: where Bergen and Tiedemann appear to assume that all of those who are not clearly reported as suspected militants must be civilians, we allowed that in some cases the available data is insufficient to be confident one way or the other.  We also focused on the most updated and credible reports, while Bergen and Tiedemann simply compiled all reports into loose ranges of possible fatality levels.  The result of our more rigorous and comprehensive approach is a substantially more lopsided ratio in favor of suspected militant fatalities.

Our data also revealed that despite a substantial intensification of the Predator strikes starting in 2008 and accelerating through 2009 into 2010, and the broadening of target categories to include low level Pakistani Taliban, the ratio of suspected militant to civilian fatalities has remained steadily high and has gradually (if unevenly) improved. [6] After incremental increases in attacks from one in 2004 and three in 2005, 2006 and 2007, strikes escalated drastically to 33 in 2008, 54 in 2009 and 30 in the first three months of 2010 alone (See Figure 3).  Still, far from showing a reduction of accuracy as the campaign has accelerated, our data shows that the ratio of suspected militant to civilian deaths has improved from the approximate 6:1 and 7.8:1 ratios that characterized 2004 and 2006 respectively, to 13:1 in 2005 and 14.067:1 in 2008, peaking in 2007 and 2010 (up to June 19) when no confirmed civilian deaths were reported (see Figure 2). [7]

Moreover, the campaign’s overall ratio of suspected militant to civilian fatalities appears to be substantially better than both that of ground operations in the region undertaken by the Pakistani Army and of non-drone operations executed in the area by U.S. forces, as indicated in Figure 1 below (columns 1-4).  It has also greatly exceeded the efficiency of the Israeli targeted-killing campaign conducted in the West Bank and Gaza in response to the second Intifada, according to figures collected by B’Tselem, a well-established human rights organization active in the area (column 5). [8] Not surprisingly in light of the foregoing, the CIA drone campaign has also bettered the most oft-cited proportionality statistic for armed conflict in general at the end of the twentieth century (column 6).  Indeed, as Figure 1 suggests, the closest contender in terms of militant to civilian proportionality is the reported impact of the Pakistani Army’s Swat offensive in 2007, which was just over one quarter the ratio in terms of suspected militant to civilian deaths. At the other extreme, the most often cited statistic for armed conflict in general at the end of the twentieth century is less than 1/150th the ratio.

Conclusion

One conclusion that can be confidently drawn from this brief analysis of our database is that the available evidence on the CIA’s Predator campaign suggests that it is neither inefficient nor disproportionate in terms of civilian casualties, at least in relation to alternative means of conducting hostilities and/or other recent targeting campaigns for which credible numbers are available. This conclusion does not, of course, resolve the ongoing debate over the use of Predator drones. Other objections are certainly being raised, perhaps most interestingly that their use may make going to war too easy, and thus result in a proliferation of armed conflict. [9] We hope, however, that it does move the debate forward by shedding a more balanced light on the numbers of civilian casualties reported by the Pakistani media.

 
Notes:

1. The database is a comprised of news stories available at the time of its compilation and is subject to ongoing amendment as new information becomes available.
2. So, for example, if a story says most of the dead were suspected militants, we have counted only half plus one; if a story says three to four militants were killed, we have counted three.
3. The exact ration is 3.6981132:1.
4. The exact ratio is 19.215682745:1.
5. Peter Bergen and Katherine Tiedemann, “Pakistan Drone War Takes a Toll on Militants,” CNN, edition.cnn.com/2009/OPINION/10/29/bergen.drone.war/. Bergen and Tiedemann estimate 615 militants and 292 civilians; for Bergen and Tiedemann’s full dataset, see counterterrorism.newamerica.net/drones.
6. Adam Entous, “CIA Drones Hit Wider Range of Targets in Pakistan,” ABC News, May 5, 2010, abcnews.go.com/International/wireStory.
7. The ratios here do not include “unknowns.”
8. It must be cautioned that B’Tselem uses the categories of “target” and “non-target” fatalities, as indicated in the cells of Figure 1.  We are assuming, for the purposes of comparison, that the “targets” category gives at least a baseline account of suspected militants, while the “non-target” category represents a high estimate of civilian casualties.  It is of course possible that some “non-targets” are in fact “suspected militants,” in which case the Israeli targeting campaign has been more accurate than our ratio suggests.  It is impossible, based on the data gathered by B’Tselem to determine exactly how significant this distortion is, and our policy is to resolve grey areas in favor of higher accounts of civilian casualties.   It seems extremely unlikely, however, that any distortion would be significant enough to raise a 1.529:1 ratio into the area of a 19.21:1 ratio.  Our finding that the CIA campaign appears to be more accurate therefore remains secure.
9. For example, Peter W. Singer, Wired for War: the Robotics Revolution and Conflict in the 21st Century (New York: Penguin Press, 2009), p. 316.

*Fricker/Plaw/Williams Drone Campaign Database
1. Based on data gathered by the Institute for Conflict Management (http://www.satp.org/).
2. Based on B’Tselem Statistics (http://www.btselem.org/English/Statistics/Casualties.asp).
3. See, for example, Mark Osiel, The End of Reciprocity (Cambridge: Cambridge University Press, 2009), p. 143; Mary Kaldor, New and Old Wars (Stanford: Stanford University Press, 2001), p. 8. A recent data set, offering an avowedly conservative assessment of civilian deaths in armed conflict around the world, gives a significantly different assessment.  Taking Lacina and Gleditsch’s (2005) dataset for battle-deaths as a basis for comparison, Kristina Eck and Lisa Hultman (2007) find that over the 1989-2004 period, battle deaths exceeded deaths form one-sided violence (a rough indicator of civilian casualties) by a ratio of two to one – Kristine Eck and Lisa Hultman, “One-Sided Violence Against Civilians in War: Insights from new Fatality Data,” Journal of Peace Research 44(2), 2007, p. 241; and Bethany Lacina and Nils Gleditsch, “Monitoring Trends in Global Combat: A New Dataset of Battle Deaths,” European Journal of Population 21 (2/3), p. 145-165.  Still, even on Eck and Hultman’s conservative estimate, the apparent proportionality of Predator drone strikes exceeds the norm for combatant to civilian proportionality for 1989-2004 by a factor of more than ten to one.