I’ve received a torrent of comments about my earlier post about The Lucifer Effect (ok, one comment and an email) and so I’ve figured that I should expand upon the subject, particularly, as the findings might relate to intelligence analysis.
First of all, let’s look at how situational factors might might influence the act of intelligence analysis using an obvious example. It’s estimated that upwards of 70% of the classified intelligence budget at the federal level goes towards private intelligence contractors. These individuals and companies are (and have been) making huge gobs of money since 9/11. It doesn’t take a rocket scientist to figure out that the government contract gravy train will stop if the threat from the “Global Jihadist/fascist/communist/Chavezist/vast right wing conspiracy” network disappears. So, just how likely to you think individuals and companies are going to be (even on an unconscious level) to provide assessments which indicate the threat to the United States is decreasing or low? That would be essentially like sending a memo to your boss explaining how your job is redundant and if they fired you they could realize big savings with no loss of productivity. It just ain’t gonna happen.
Even if it was so obvious that the threat was so small that it could not be denied (and let’s face it, that would be quite a feat…, apparently it’s possible for some people to believe that Cuba poses an existential threat to the U.S.) you could be rest assured that these contractors would have a long list of new threats that desperately require not only all the resources currently under contract but a host of new ones as well (all available at a very reasonable price, if you act now).
There’s an even more prevalent instance of how situational factors can negatively affect intelligence products. Every agency has its own culture and set of assumptions. Challenging those assumptions risks having your analysis marginalized or even damage to your career. Paul Pillar wrote about these issues in his article about how intelligence was manipulated to lead us into the Iraq war:
Well before March 2003, intelligence analysts and their managers knew that the United States was heading for war with Iraq. It was clear that the Bush administration would frown on or ignore analysis that called into question a decision to go to war and welcome analysis that supported such a decision. Intelligence analysts — for whom attention, especially favorable attention, from policymakers is a measure of success — felt a strong wind consistently blowing in one direction. The desire to bend with such a wind is natural and strong, even if unconscious.
A clearer form of politicization is the inconsistent review of analysis: reports that conform to policy preferences have an easier time making it through the gauntlet of coordination and approval than ones that do not. (Every piece of intelligence analysis reflects not only the judgments of the analysts most directly involved in writing it, but also the concurrence of those who cover related topics and the review, editing, and remanding of it by several levels of supervisors, from branch chiefs to senior executives.) The Silberman-Robb commission noted such inconsistencies in the Iraq case but chalked it up to bad management. The commission failed to address exactly why managers were inconsistent: they wanted to avoid the unpleasantness of laying unwelcome analysis on a policymaker’s desk.
As Pillar and Zimbardo both point out (although in different ways), an even more subtle form of bias is the imposition of norms or standards that take on the equivalent of physical laws. Those norms and standards make it difficult to do ‘macroanalysis’ of the system and even more difficult for such analysis (if done at all) to be accepted and acted upon. For example, in law enforcement circles the common response to a criminal problem is suppression through the use of (or threat of) arrest and imprisonment. The underlying assumption is that everyone (including criminals) make their decisions based upon cost-benefit analysis and the threat of arrest trumps virtually benefit. If crime continues (assuming that’s the benchmark used to determine success), it’s not that the underlying assumption is wrong, rather it’s that the penalty isn’t stiff enough or we aren’t arresting enough people. But what if that’s not how people make decisions? Or, if what is motivating criminals is more basic (a la Maslow) that the fear of imprisonment?
That might prompt some analysis to look for alternate ways to deter crime but would require everyone to shift their perspective from the established current one that says “You fight crime by locking people up.” That could be a scary idea for people who’ve made their career based upon that idea. When ideas like that pervade an organization it’s difficult for metaanalysis to crack that mindset and so analysts are left with little option but feeding the organizational delusion. From Pillar:
As any competent pollster can attest, how a question is framed helps determine the answer…On any given subject, the intelligence community faces what is in effect a field of rocks, and it lacks the resources to turn over every one to see what threats to national security may lurk underneath. In an unpoliticized environment, intelligence officers decide which rocks to turn over based on past patterns and their own judgments. But when policymakers repeatedly urge the intelligence community to turn over only certain rocks, the process becomes biased. The community responds by concentrating its resources on those rocks, eventually producing a body of reporting and analysis that, thanks to quantity and emphasis, leaves the impression that what lies under those same rocks is a bigger part of the problem than it really is.
Not particularly encouraging, I know. If anyone has an idea of how to overcome such obstacles, I’ll buy ya a beer.