Tag Archives: critical thinking

The HORROR (of intelligence analysis)

Yeah, there are plenty of places on the internet where you can find horror themed blog posts (especially today) and perhaps not so many (but still plenty) that talk about intelligence issues.  But where, gentle readers, other than right here will you find a blog that can seamlessly merge the two as Dr. Frankenstein sewed together the body parts of the dead?

If you’re a long time reader of TwShiloh (and have too much time on your hands) you may remember way back in October of 2007 (there must be something about the season that makes me return to this subject) when I talked about an idea I had for using the John Carpenter movie The Thing as a vehicle for teaching various aspects of critical thinking and structured analytical techniques.

Well, I’m very pleased to announce that I finally had a bit of time and put my money where my mouth was, developing such a course.  While it still needs a group (or two) of guinea pigs to test the concept I think my draft project exceeded even my original optimistic vision.

My list of learning objectives include:

  • Recognize how structuring benefits the work of an analyst
  • Explain how biases interfere with analysis
  • Discuss how structured analytic techniques assist in defeating biases and overcoming mind-sets
  • Discuss the utility of and demonstrate proficiency in structured analytic techniques to include:
  • Brief an audience about the results of a hypothesis generation/testing process that that disseminates results of research and analysis

As I watched the movie (again and again throughout the week) I became more and more convinced that various forms of media could serve as useful source material for teaching aspects of analysis.

Intelligence analysts in law enforcement – observations. UPDATED.

I just spent a few days with a bunch of civilian intelligence analysts and that plus a number of other recent events have prompted some observations on the state of the field.

This particular group came from a number of agencies spread across two states and I have to admit I was blown away by their motivation and drive.  While, generally, analysts remain an introverted bunch (even I, gentle reader am an INTJ) their work ethic seemed to overpower that trait and didn’t dawdle when there was work to be done, even in groups.  While they were much more quite than comparable groups of law enforcement officers I’ve worked with before they were more task focused by orders of magnitude.  You know you’ve got an engaged bunch when you have to tell them to go home three times!  The next time you hear someone talk about lazy government workers, tell them to suck it.

They also had a great deal of interest and pride both in their craft generally and in their specific fields.  These are people who are chomping at the bit to do their work.  It’s stuff like this that really keeps me going in this field.

That’s good because (yeah, thanks, bring on the buzzkill. eds), I’m convinced (based on observations and reports from others) there remain deep structural problems in the field of law enforcement intelligence.  Let me count them (well, some of them anyway):

  1. I keep hoping to come across analysts that actually have some sort of ‘seat at the table’ and to be quite honest at this point I’d even be happy if it was the kiddy table out in the kitchen with all the 4 year olds making jokes about poo.  Analysts are simply not brought in to discuss serious issues in which they should be central players.  I’m talking about issues of hiring, training, and career progression where (and for once I’m not exaggerating) I don’t think I can think of more than three or four analysts I’ve met over the past eight years that have been brought in to actually have a serious voice on these issues.  There are even fewer incidents of analysts being allowed to have a say in things like analytical focus or resource allocations.  And just forget about the idea of analysts actually supervising in analytical shops.  I’m not saying these things never happen, just that they’re so rare we might as well consider them mythical creatures.
    • Given that most state and local agencies are facing big budget problems with many departments considering laying off police officers, does it make sense to pay cops to not conduct investigations, enforce the law and arrest bad guys and, instead, assign them tasks for which they are rarely trained in and, usually, have little interest or capability to do?  Does it make sense to take those people with training in intelligence work and perpetually keep them at the bottom of the food chain?
    • This makes sense only if you consider the following:
      • These supervisory positions represent promotional opportunities for agencies and you always take care of your tribe first.  No matter how much they may like their civilian employees, they simply aren’t on the right side of that thin blue line and if push comes to shove, you take care of your tribe first.
      • Further riffing off the tribal theme…There’s an abundance of literature and anecdotal evidence that civilian analysts are secondary or peripheral players (or, as I recently heard to my annoyance, ‘cop-lite’).  Given the recent emphasis on things like ‘Intelligence Led Policing’ where intelligence is supposed to be the prime mover in operations, this attitude is simple cultural chauvinism.  There’s unease at letting ‘out-group’ members get access to information or resources.  Who knows what they’ll do with them?  Can they be counted on to further the ‘in-group’ interests?
        • (A brief vignette) I once sat in on a meeting, the only civilian in a room of law enforcement officers discussing some procedural issues.  When the possibility arose that civilian analysts might have access to investigative information, you would have thought wikileaks just published the nuclear launch codes.  Bizarre scenarios began floating around about analysts being sleepers for criminal groups, selling confidential information, etc.  I then pointed to the dearth of evidence of ‘dirty analysts’ and recommended we asked internal affairs to generate some numbers of investigations of analysts compared to cops.  For some reason, there didn’t seem to be much enthusiasm for following this up with hard numbers.  While the point was made, analysts still were barred from accessing important information.
        • Think I’m being too cynical?  Allow me to recommend part 2 of last week’s This American Life (ignore the first part which is about a jerk who can’t understand why publicly threatening to shoot people would possibly get him in trouble).  It’s laden with interesting items regarding what happens when you rely too much on poor metrics, question the status quo and if such problems are the result of a ‘few bad apples’ or a rotten barrel which spoils most of what you put into it, regardless of how nice the apples were when you started out.
  2. I still think Fusion Centers do more harm than good.  Since they were created without a great deal of forethought, their numerous flaws just continue to get bigger and more obvious.  Agencies which  run them (usually law enforcement entities) have limited experience dealing with ways to measure success meaningfully (whether that’s intentional or accidental, I’ll leave for you to ponder) and so nobody has really bothered to figure out what these centers should do and how you should measure their effectiveness.  As a result, they generally figure out what the easiest things are to measure and then reverse engineer metrics and mission statements around those.  Therefore, things like number of bulletins published, database checks made, events hosted, or agencies contacted are used, giving a priority to activity rather than progress or effectiveness.
  3. This deemphasizes analysis which is a tricky thing to measure or evaluate easily.  I dig, trust me, I do.  This sort of thing is hard to do but it doesn’t mean you don’t even try.  So, rather than bother with figuring out how to build a system where you ‘fuse’ intelligence into functional analytical products, fusion centers give information sharing primacy because it’s much easier to measure.  Now, don’t get me wrong.  Information sharing is very important but it’s not intelligence and it’s not going to prevent intelligence failures.
    • Many years ago, Richards Heuer wrote that intelligence failures were failures of analysis rather than collection.  I’m not sure I agree completely with that (unless you use the term ‘analysis’ broadly to include a leadership component that actively obstructs the work on analysts) but he was definitely on to something.  You wouldn’t know that however, by examining how fusion centers operate.  Rather than spending their efforts on enhancing their analytical capabilities, they spend all their time on the collection side.
    • There’s a manic frenzy to get access to more and more sources of information (what a shock, yet another simple metric you can throw around to demonstrate how ‘cutting edge’ you are) and give any Fusion Center shill a chance to pitch his/her agency and within the first two minutes you can bet you’ll hear them proclaim how many databases they have access to and advertise to anyone who will listen that they’re a ‘one stop shop’ for your agency when it comes to sorting through the numerous datasets out there.  That’s great and could be a really powerful tool but problem is the people they assign to answer all those queries, chained to those databases, are the analysts who are supposed to be doing, like, analysis.
    • To resurrect a tired cliché, everyone is so obsessed with making sure we’ve got all the dots, nobody is spending any time trying to connect them.
  4. I’m torn about intelligence professional organizations.  I know a number of people in them that I really respect and like but, as I’ve written before, I can’t help thinking they’re just enablers for bad practises and cover for agencies that engage in the trade of smoke and mirrors rather than intelligence analysis.  In short, they just don’t have any balls.  Now, I understand the argument that we’re talking about big cultural changes and we need to gently and gradually lead everyone by the hand, build consensus and we’ll eventually get to the promised land.  I don’t think it’s true but I understand it.  But if that’s our plan let’s be honest about it and tell the taxpayers ‘Yeah, you know that whole 9/11 thing?  Well, we’re going to fix our intelligence systems to prevent that sort of thing but we don’t want to be too pushy so we’re going to wait another decade or two until people who are uncomfortable with change can ride out their careers without too much stress and then we’ll start fixing things.’

All of these things occasionally make me swerve into the territory of crisis of faith and wonder if talking about intelligence analysis in law enforcement is futile.  I still don’t know the answer but suspect it lies in waiting for the analytical community to develop its ‘revolutionary consciousness’.

A las Barricadas!

Update:  Ask and ye shall receive!  I received an email from someone (who I was kind of hoping would take the bait since I knew he/she was an exception to what I was taking about) who said:

My analysis shop is run by analysts.  I’ve never worked in any other shop so I don’t know what it would be like for this not to be the case.  Our shop is set up with a “director” who basically just handles grants, supplies, HR stuff, etc., and then a managing analyst, with 3 civil service analysts, and then 6 contract analysts.  We’re actually hiring 2 contract analysts and the managing analyst is basically taking care of the whole thing, choosing who to interview and hire.

I think the key is that our shop basically started as one guy attached to homicide, and then another guy who worked with narcotics, and then it grew to 3, and then 4, until it exploded to 10.  And because it initially grew so slowly and was working with the most sensitive stuff, and also because the chiefs really liked what we were doing, they basically allow us to run our own stuff.  My sense is that this is not how most analytical shops develop.
Still, no idea how to measure our success.  Maybe in the # of proactive successful investigations?  Or crimes solved with analyst help?  The problem is that if we trumpet our success then people won’t want to work with us as much.  Part of our success is because we let the investigators and narcotics cops take all the credit (which is why they like us).

I’ve heard of success in analysis shops before but the times I’ve been able to delve into their details (and I can’t speak to the situation above) it’s been attributable to a particularly competent and influential person.  The concern (well, at least mine) is that there aren’t sufficient processes in place to maintain that success.  Time will tell…

And, thanks to my ghost contributor.  I don’t consider this endeavor futile.  If I did I wouldn’t stay in the field.

Critical thinking for everyone

I just stumbled upon a podcast/website by Kevin deLaplante who’s building a promising looking catalog of work revolving around critical thinking.  I’ve sampled the first three episodes of his video podcast in which he explains the value of critical thinking and the parameters of his project.  His goal is to produce 5-10 minutes lectures focusing on one idea at a time.  The podcast is now firmly ensconced in my queue.  Good sound quality, fair production values (thus far, essentially a PowerPoint presentation…no shiny objects or fluttering animation) but well worth your consideration.

He also offers other videos, some of which are free but the primary goal is to encourage you to purchase a membership to access a great deal more original content.  I haven’t had a chance to check these out yet, but I have high hopes.

I’ve felt that critical thinking courses for intelligence analysts can be a bit too subject focused and miss the underlying messages and concepts.  It becomes harder (in my opinion) when students feel they have an extensive understanding of the subject material that the instructor is using to demonstrate the practice of critical thinking.  In those cases it becomes easy to fall prey to cognitive biases and jump to conclusions, even when claiming use critical thinking techniques.  Therefore, a non-national security/law enforcement themed class on critical thinking is generally superior (although, you have to do more up front work when it comes to highlighting the relevance of what you’re teaching).

Anyway, give it a looksee…

Analysis of Competing Hypothesis

Kristan over at Sources and Methods has a post up about the efficacy of ACH.  I’m a huge fan of the system and actually used the freeware version to help Mrs. TwShiloh and me in deciding on a house to buy.

We were trying to juggle multiple factors on multiple properties.  As you know from reading Richards Heuer, you’ll know that the human mind can really only track 7-9 things at one time and attempting to do more just results in all sorts of cognitive train wrecks. We pllugged in the properties with a whole list of criteria (from the important to the trivial), weighted them and ranked them.  When it was all set and done one house that we thought we loved dropped to the bottom of the pack and we realized that we had gotten swept away by one or two really attractive features and blotted out the many, many problems such a house would give us.

So, I was pleased to read that ACH actually does have some utility.

The results were in favor of ACH in terms of both forecasting accuracy and bias.  In Drew’s words, “The findings of the experiment suggest ACH can improve estimative accuracy, is highly effective at mitigating some cognitive phenomena such as confirmation bias, and is almost certain to encourage analysts to use more information and apply it more appropriately.”

That’s doubly good since it looks like ACH is about to get another boost through another freeware release of a new, improved version.

For three years, Matthew Burton has been trying to get a simple, useful software tool into the hands of analysts at the Central Intelligence Agency. For three years, haggling over the code’s intellectual property rights has kept the software from going anywhere near Langley. So now, Burton’s releasing it — free to the public, and under an open source license.

This is really exciting and I’m anxious to see it when it’s finally released.  Burton’s improvement is that his ACH version will be collaborative, allowing multiple analysts to add evidence and share views. 

Kvick Tänkare

Ta-Nehisi has a great post about the case of the D.C. detective pulling a gun on a bunch of people throwing snowballs.  The detective is claiming that he thought an angry mob of  anarchists were threatening him and he feared for his life and so pulled his gun.  Yeah…cause anarchists attack a whole lot of cops…and snowball them to death.  Maybe…maybe, he’d have a case if the G20 was meeting at the time and he was in the area of protests.  But no, Baylor was in civilian clothes and a civilian vehicle.  There was no way he’d have been identified as a police officer.  That means Baylor had to be under the assumption that these ‘anarchists’ were on some sort of racial attack.  Yeah…cause as widespread as anarchists attacking police officers is, it’s even more common for them to lynch random black people.

As Coates sums up:

…it’s good to know that Detective Baylor won’t be, like, fired or anything. Wouldn’t want a cop who feels endangered by snowballs to be bounced off the force. The rough streets of D.C. need men with that kind of mettle.

If you are the sort of person that worries about the carbon footprint of food, don’t feel guilty about bananas.

Bananas are a great food for anyone who cares about their carbon footprint. For just 80g of CO2e you get a whole lot of nutrition: 140 calories as well as stacks of vitamin C, vitamin B6, potassium and dietary fibre. All in all, a fantastic component of a low-carbon diet.

• They are grown in natural sunlight, which means that no energy-intensive hot-housing is required.

• They keep well, so although they are often grown thousands of miles from the end consumer, they are transported by boats, which per kilo of freight transported emit only 1% as much CO2 as planes do.

• There is hardly any packaging, if any, because they provide their own. (You might sometimes see a bunch in a light plastic bag or wrapper, but this probably pays for itself carbon-wise by reducing the chance of customers ruining the fruit when they try to split a bunch.)

“You get to play with very large toys in the oil industry.”  A great explanation of the science behind the BP oil spill.  (h/t Phronesisaical)

I was never a big fan of Andrew Sullivan’s ‘View from your window’ (probably because he refused to publish my own, most excellent entry) but he’s now got a contest where you guess the location of the shot.  It’s quite amazing to read the logic people use to figure out where the picture was taken.  I only wish they published all the guesses.  Great critical thinking exercise.

Help an analyst out…

From Sources and Methods a fledgling analyst is working on his thesis and needs some help.  He’s put together an on line exercise together and needs as many people as possible to participate.  If you’re convinced that the goons in the CIA and FBI miss all the big clues and you’ve got a foolproof method of divination or you just want to help a guy out, give it a go.

Mulder Thesis Experiment.

Intelligence Analysis 101 (part 2)

(Here’s part 1 if you’re joining us late)

Ok…pens down.  So, you’ll remember this picture:And the questions associated with it were:  Do you agree with the interpretation of the ad?  Why or why not?  Do you agree with the analysis?  Why or why not?  If not, please provide an alternate explanation.  What recommendation(s) do you make to your boss?

The interpretation of the ad was totally off base.  In fact, the ‘analysis’ provided was so bad I’d argue it belongs in a hall of fame somewhere.  At least George W. Bush had ‘Curveball’.  He might have had no credibility but at least it was something.  The interpretations that came in the alert had nothing.  Now, let’s talk about how one arrives at that conclusion (that’s the good stuff).  As an aside, if you’re teaching entry level analysts (or anyone for that matter) this can be a pretty good way to introduce the workings of Analysis of Competing Hypothesis.

  1. The ad itself.  The cost of a full page ad in both the NY Times and the Washington Post will run into tens of thousands of dollars (perhaps with a total price tag into six figures).  I don’t know if you’ve seen animal rights activists but they aren’t exactly awash in money.  Assuming they pooled their money or had a rich benefactor though, there are still problems with the ad itself.
    1. Show me the money! You probably aren’t going to be able to pay for this ad space anonymously and with cash so the people taking it out are going to leave a paper trail.  Groups like ALF derive their strength from their anonymity and if they were going to give it up it would probably have to be for a pretty big payoff.
    2. The picture.  The iconography is all wrong.  Google animal liberation front images and you will indeed see pictures of guys in balaclavas but almost without fail they’re shown with animals.  The menacing garb is not the focus of the imagery.  The focus is on the act of ‘liberating’ those animals.  So, they balance the scary image of the dude in camo and face mask by having him coddle a cute bunny, usually holding it and looking at it in a way to trigger images of parents holding an infant.  Whoever made this ad would be the same people who argue that the Mona Lisa is a great Italian landscape picture…oh, and I think there’s some chick in it.  They’re clearly trying to raise the fear factor by putting a guy who looks like a terrorist in the image but I don’t think that’s not how the ALF would picture themselves.
    3. The jacket.  I’m torn on the jacket.  Either it was a sloppy mistake (you aren’t going to find many animal liberation advocates wearing leather – and that’s an understatement) or it was a subtle ‘fuck you’ to the activists.  I guess it could be pleather though…
  2. The message
    1. Who’s the target demographic?  If the point of the ad is to intimidate the business community (well talk about messaging problems below), why not place the ad in the Wall Street Journal or Business Week?  Who are these people targeting?
    2. What are they trying to get their demographic to do?  Are we to believe they’re going to spend a huge wad of cash to get people to go to a website?  That’s it?
    3. There’s NO WAY anyone sympathetic to the cause of ‘animal liberation’ would ever, EVER describe the activities of LSRI as ‘vital pharmaceutical research’. Really, you’re more likely to hear that the local B’nai B’rith is planning a lecture series on the valuable research into humanitarianism done at concentration camps.
    4. People tend not to refer to themselves in the third person.  Hence, if the ad was done by animal rights activists, this sentence wouldn’t read this way:  NYSE employees were reportedly threatened by animal rights activists whose campaigns had already targeted businesses connected to LSRI. But instead would read something like:  ‘In addition to targeting businesses connected to LSRI, we’ve also been targeting NYSE employees.’
    5. While not complementary to the NYSE, this isn’t exactly a revolutionary calling to free the animals or protest animal testing.  The message calls the NYSE cowards for bowing to pressure from animal liberation activists.  It doesn’t even work as gloating.  It only makes sense if you are unhappy with the fact that the NYSE caved into pressure, which one can assume, would not include SHAC or other animal liberation advocates.

I won’t get into a detailed tear down of the analysis since that part was fabricated by me to provide a composite example of the types of alerts and warnings commonly used.  I will say that there’s nothing in there that would have been out of place or unusual at that time in that ‘analysis’.  But allow to highlight some serious concerns with this sort of ‘analysis’ in general terms.

  1. You have no idea how the analyst came to their conclusion.  Blanket statements without supporting facts loaded with unexamined assumptions.
  2. Mealy-mouthed qualifications.  Ye old ‘we have no specific information…’ has become the nervous tic of the intelligence community.  It’s the refuge of people who want to cover their ass in the event something goes wrong and absolves them from having to do real intelligence work and follow through.  It’s especially maddening when seen (as it usually is) at the bottom of a warning threatening the end of civilization and life on earth.  “Warning:  If three dozen nuclear weapons were detonated in the largest U.S. cities simultaneously, there could be catastrophic social, economic and humanitarian consequences!!!  Terrorists have indicated their intent to acquire such weapons and all agencies should remain on high alert!!!  Oh..by the way, we have no indication that anyone is actually close to acquiring such a weapon.  We just don’t want to get blamed for not warning you in case some evil overlord has been planning something like this in his hidden cave complex with the mole people.”

This was not a difficult nut to crack.  Really, I would consider this something an entry level analyst should be able to work through on their own.  If you’ve got an intermediate level analyst (more than a couple years on the job) who has difficulty with this, they need to see a career counselor immediately.

I’ve been accused of being a bit cynical regarding the progress (or lack thereof) of the capabilities of intelligence analysis within the law enforcement community and this has probably more than a little to do with that.  As I stated in Part 1, I used this as a training exercise and was taken aback at how poorly people did at this.  Preconceived notions about who the ‘bad guys’ are and a slavish (really quite inexplicable) deference to everything produced by other law enforcement agencies meant that it was difficult to even get people to consider that there might be alternate explanations.  When pushed, there seemed to be a tendency to try to retain as much as possible of the original analysis, and only grudgingly working at the edges.

Finally, it also highlighted another serious flaw in these sorts of ‘alerts’.  There is little in the way of procedure and almost nothing in terms of incentives to get agencies to issue corrections.  Corrections are a permanent and official record of a screw up which can be used to undermine your reputation and authority.  My observation is that the preferred way to deal with errors like this is to just ignore them.  Usually they’ll fade away without a whisper and you still get to use the alert as ‘proof’ that you’re providing cutting edge intelligence (We issued 500 intelligence alerts to our partner agencies the past fiscal year!).  Of course, the problem is that once you release that bogus information it keeps bouncing around, getting recycled, absorbed into cognitive frameworks throughout the community.  Until you see as much effort into putting out credible, reliable analysis and correcting mistakes as you do in fussing over the quantity of information being put out, intelligence analysis will struggle for relevance and credibility.

Intelligence analysis 101

I was going through my documents and came across this practical exercise I gave a couple of years ago to some analysts and, given some work I’m doing for a class I’m taking, figured I’d post it here.  It’s based on a real event but the names and details have been changed to protect the guilty.

You shouldn’t require too much in the way of experience or background in the law enforcement, homeland security, or intelligence fields.  When I ran this scenario I didn’t allow any internet searching or additional information since it was primarily a critical thinking exercise and I recommend you not search specifically for this ad as it might alter your answer and you wouldn’t have had access to that information at the time (although, it’s not like I can stop you…).  I’ve included a few links in the scenario that would probably be most helpful.

If you’d like, post your answers in the comments section and I’ll follow up with the ‘answer’ and my thoughts in a day or two.

Situation: You work as an analyst for a law enforcement/homeland security agency.  Your boss sends you the following picture with an attached alert from an agency similar to yours (but with a lot more money and a good reputation for being ‘on the ball’).

The alert says:

This full page ad appeared in today’s NY Times and the Washington Post.  The text says:  ‘On September 7th, 2005, the New York Stock Exchange was scheduled to add Life Sciences Research Inc. (LSRI) to the big board.  Fifteen minutes before trading opened, NYSE officials changed their mind.  LSRI is involved with vital pharmaceutical research that requires the use of animals.  NYSE employees were reportedly threatened by animal rights activists whose campaigns had already targeted businesses connected to LSRI.  In March, six of the campaign’s leaders were convicted of federal terrorism charges but the NYSE is still running scared.  Find out more at NYSEHostage.com

Analysis: We interpret this ad to be a message by animal rights extremist groups (such as the Animal Liberation Front or SHAC) indicating their determination to step up attacks against the pharmaceutical and financial industries in our area.  All agencies should be on alert for indicators of targeting such businesses.  While we have no specific information regarding threats at this time, law enforcement should exhibit caution as such groups have a history of engaging in illegal activity, including  physical violence.

Your boss turns to you and asks what your opinion is of the article and analysis.  Do you agree with the interpretation of the ad?  Why or why not?  Do you agree with the analysis?  Why or why not?  If not, please provide an alternate explanation.  What recommendation(s) do you make to your boss?

As a note, don’t bother trying to go to NYSEHostage.com, the link is now dead.


Imperial Secrets part 2

A continuation of my previous post about Patrick Kelley’s Imperial Secrets:

One of the things worthy of a mention separate from the main post was Kelley’s idea that physical structures can have an effect on information accessibility.  Now, in Kelley’s work he describes how the Ottoman Empire dealt with this problem (the restricted avenu0es of information in the Sultan’s court was counterbalanced by the informal information exchanges which came from the harem).

So, consider the following hypothetical situation:

Imagine an organization designed to be a clearing house of intelligence information.  Let’s say a definition of its function is something like:

…a collaborative effort of two of more agencies that provide resources, expertise, and/or information to the center with the goal of maximizing the ability to detect, prevent, investigate, apprehend, and respond to criminal and terrorist activity.  Components of a [this agency] include the intelligence process, where information is collected, integrated, evaluated, analyzed, and disseminated. Other components include public safety, homeland security, private sector, and critical infrastructure.

Now picture this agency is located in a physically restricted space.  Proper identification is required to enter the same area as the agency.  Further security checks (and a more specific credential requirements) are required to enter the actual building where this agency is and people who don’t work in the building (regardless of their credentials) require escort.

Finally, this is not a ‘classified’ facility as defined by the national intelligence community.  No information is passed around that exceeds the ‘sensitive but unclassified‘ level.

So, given that, how likely does it seem that information would flow into that facility from individuals?  How easy might it be for people who work in such a facility to absorb information from the ‘outside world’?

Probably not so much since they’ve essentially been hermetically sealed from the outside world.

‘But wait!’ I hear you say.  ‘This is the 21st century.  People don’t need to physically interact.  People don’t need to physically go anywhere to acquire information.   The concept of ‘ground truth’ is highly overrated.’

So, let me add that in addition to concern for physical security there’s also an extreme concern over digital security and therefore, access to ‘teh internets’ is severely restricted.  What is unrestricted is messages from partner agencies (so long as they’re sent in the proper format and through the proper systems).

Such a system is going to give you (maybe) an abundance of one type of information but is unlikely to get you the sort of information you might want if you’re tasked with preventing strategic surprise.  In other words, don’t count too much on the ‘detect and prevent’ aspects of your goals but you could probably do a decent job of ‘responding’ and ‘disseminating’.

But…one wonders how much thought goes into the effects of physical structure on information accessibility and if there’s a conscious decision made to set up such centers this way (hypothetically speaking, of course).

Imperial Secrets

I’ve just finished ‘Imperial Secrets:  Remapping the Mind of Empire’ by Patrick A. Kelley.  It’s available as a free download from the National Defense Intelligence College and well worth your time.  In fact, I’m inclined to say it’s one of the best books I’ve read about intelligence analysis in a very long time.  If Richards Heuer’s Psychology of Intelligence Analysis can be thought of as an examination of analysis from an internal perspective, then Kelley can be thought of examining the subject from the external.

Heuer examines how neurobiology, psychology and cognitive biases all influence how an analyst searches for information and interprets it once she has it.  Kelley, on the other hand, examines how the preferences and biases of an ’empire’ (more on the definition of that in a moment) influence not only analysts but consumers of intelligence.  He identifies the purpose of his work as addressing three questions:

How do power and knowledge interact? How do marginal actors and slippery knowledge mediate this interaction? And is there a distinctly imperial way of knowing?  184

As you might guess from those questions, this is an epistemological study designed to look at the limits of what we can know, bounded within the limits of our circumstances.  As such, it is not filled with ‘practical’ tips for analysts (‘No more than 3 bullets per PowerPoint slide!  Don’t cross lines on your link chart!’) but rather it presents cases in which allow analysts today see those boundaries that restrict them and their customers.  Too often, those boundaries are so invisible to us that we don’t even realize they exist.  But they do and ignoring the information outside our universe of information is what puts us at risk of becoming victims of strategic surprise.

Empires are always at an information deficit—telling more than they hear—and the deficit over time becomes associated with a lost capacity to listen. That quality of listening is key to generating truly alternative analysis, if as we might guess it takes surprising approaches to anticipate surprise.

Kelley uses three historical examples which he says share some of the same challenges and circumstances as the United States does today:  Rome in the first two centuries of the Principate, the Ottoman Empire of the 16th to 18th centuries and the British Empire from the 18th to 20th centuries.  In explaining why the United States in the twenty first century should look back centuries to other nations to explain our current situation he says:

…our relation to the world has changed beyond that of a traditional nation-state—albeit a profoundly powerful one—our attempts at intelligence reform indicate our interest in answering traditional questions, just in a faster and more accurate fashion. Better interagency cooperation, sharing with partners, broader and more rapid dissemination will ultimately result in more “actionable intelligence.” Not surprisingly, the catch-phrase implies a specific type of action, especially for military audiences.

…there will never be a “VE” or “VJ” day when all the insurgents and terrorists are captured and killed; the letters of capitulation signed; and the vast war-time machines of actionable intelligence dismembered, dissolved and disbanded. Success, rather, will come with an open-ended enterprise to identify ideas, values, understandings and movements that threaten an international order built around a specific set of legal principles and economic interests. It is an intelligence challenge more akin to that facing Rome in 150 CE, Istanbul in 1600 or London in 1800, than it is to that confronting Washington, DC in 1941. (pg 2)

And here a word about his use of the word ’empire’ or ‘imperial’.  While I was reading this, the one complaint I had was that I couldn’t find a definition to these terms put forth.  By the end of the book it’s clear that his intention all along and upon reflection I think that was a good choice.  I would argue that the term definition could be applied even more broadly than Kelley would have imagined when he wrote this.  So, allow me to present this chart and explanation of his in which he explains his theory of how entities interact with information.

As imperial formations and associated political power expand from the inchoate and fractured nomad condition, information availability increases with more of the world entering a single social space. It also becomes increasingly accessible as communities merge and morph, and individuals are able to circulate and contest definitions of identity. In short, power and knowledge grow together; they are mutually enhancing. At the apogee of imperial power, this hybrid and experimental condition begins to take on more of the formal attributes of the state. Information availability continues to increase, but it becomes progressively less accessible as only one frame of reference defines legitimacy, only one perspective constitutes truth, and only one network of transmission seeks to control the passage of information. In short, from this point on, power increasingly inhibits knowledge. pg 183

There’s no reason why this needs to be confined to actual empires…I suspect the same explanation could be applied to business, (non-empire) government or other organizations.  A company moving from small entrepreneurial to hot start up to cumbersome multinational following the same trajectory as the Romans or you can look at the newspaper industry which arguably can trace its decline to the inability to identify the value of ‘marginal actors’ and ‘slippery knowledge’ (via public media and information) or a way to monetize them.  I’d also argue that virtually all government entities in the U.S. are at the far right side of the graph within their sphere of influence as they attempt to bind their corner of the universe within their bureaucratic rules.

Knowledge is indeed power, but in a relation of mutual influence rather than direct equality. Power influences the kinds of questions asked and on what topics; it shapes the kinds of answers possible and how they are expressed. Knowledge, too, influences power; and philosophical issues about the nature of reality and understanding can shape how power is realized. pg 3

As the British empire in India expanded, it encompassed an increasing expanse of human and physical terrain which it did not fundamentally understand, prompting the series of “information panics” which characterized the British experience in the sub-continent. pg. 30

Information panics.  Think about how many of those we’ve had in the past, leaving aside the question of terrorism.  The RIAA has been an almost permanent freak out since the introduction of Napster.  Concerns about predators on the internet.  Youth sub cultures (goth, hip-hop, metal).  Immigrants.  Religion.  As all of these issues have been absorbed in our ‘imperial’ sphere of interest they’ve generally been ignored until some trigger (either real or contrived) caused them to leap to the public consciousness and lead to a bunch of flapping about, chicken little style.  Some of these are overblown and others (going back to terrorism – the rise of violent Islamism) aren’t but what these all have in common is that they generally weren’t within the scope of what the ‘imperial power’ considered to be valuable knowledge.

British attempts to rationalize Hindu and Islamic law in India, for example, fundamentally contorted the material at hand in order to make it intelligible for officials operating from an English common-law background. Their consequent understanding misread what their subjects experienced and expected, with far-reaching ramifications for Anglo-Indian relations and the experience of religiously defined identity in South Asia. pg 10

This is a concept I still don’t think we fundamentally understand.  Before the invasion of Iraq, conventional wisdom was that Saddam’s refusal to allow wide ranging weapons inspectors in was proof that he was hiding stockpiles of WMD.  It wasn’t until later that it became clear that at least part of his motivation in keeping the existence of a WMD program ambiguous was to deter Iranian hostility.  The mind boggles at the number of times we’ve made errors both tactical and strategic because we don’t understand that not everyone shares the same reality we do.  For Americans, I suspect that even the majority of them that claim to be religious don’t experience the concept of God in the same way that people might in the NWFP or in Ghana, for example.

My most recent posting provides a perfect example, as we moved operations to a new embassy complex shortly after my arrival. This facility—based on a standard model common to all new embassy construction worldwide—is modern, sanitary, safe from bombs and earthquakes, and plausibly attractive, depending on one’s aesthetic; all attributes we would presumably like associated with America. As a practical matter, however, its prominent fortress-like appearance at the top of the hill has inspired local rumors regarding the 800 Marines secretly sequestered inside and the CIA rendition facility in the basement, and prompted my driver and cook to ask “Why does America want to take over Nepal?” pg. 95

But that’s only one half of the equation.

The language and image of empire is universally visible and available to its nominal subjects, while the reverse is not usually the case. Josephus, a Jewish priest, can write a generally acceptable history and enter the informal canon of Roman literature. Moreover, discounting divine inspiration, his prophecy of Vespasian’s ascension suggests a savvy understanding of Roman politics. It is far less likely, were the attempt ever undertaken, that a Roman noble could enter the discourse of the Midrashim (various collections of Jewish commentary on Scripture, especially prevalent in the 2d Century CE). Similarly, Osama bin Laden and his ilk are far more ready and able to post videos to the Internet—to enter the imperial discourse—than are U.S. interlocutors suited to interact with restive tribes in Pakistan’s North West Frontier Province via Urdu poetry.  pg. 157

In issues closer to home, think about activists (like the ad hoc ‘group’ that crushed Nestle recently).  These people were understand the information flowing from Nestle yet the company was clearly unable to process information coming the other way until it was too late and they had to take a road trip into Fail Land (here’s a presentation of the whole timeline which I normally wouldn’t link to but it was done on Prezi which I find to be an interesting alternative to linear presentation systems like PowerPoint).

In intelligence systems (regardless if its national, state or local) it’s still common practice to only value information obtained via traditional sources.  The idea of open source intelligence having real, practical value is not widely held even if it’s generally understood that you have to say it’s important.  But yet again, this is a problem of prioritizing the wrong things.

…modern intelligence analysis tends to attribute undue value to information obtained via “sensitive sources and methods”—i.e. the value of its degree of protection, its dearness to the seller—rather than information which meets purchaser requirements.  pg. 194

So, instead of how important the information is to address collection requirements we determine the value of the information on how hard it was to get.  Even when we’re talking about information others are attempting to hide, that doesn’t always translate into information we might need.

Here we need to expand upon the old cliche that we’re spending too much time on technical intelligence collection (IMINT, SIGINT, MASINT) and not enough on human intelligence.  But it might even be more than that.  Even if we hire a ton of additional agents and analysts and dedicate ourselves to finding out what’s going on in people’s heads will we get where we want?  Will be we able to get to, understand and pass along the importance of the ‘slippery knowledge’ if our people are all corn fed, middle Americans forced to operate in organizations which, like the Ottoman Empire, “privileged a certain world view and prioritized a certain kind of knowledge”. I’d remind you that in the wake of the Hasan shootings there were calls to forbid Muslims from serving in the military.  There are numerous stories of the difficulties in recruiting people with the cultural/ethnic/linguistic backgrounds which we might want because those same backgrounds don’t fit within the perception of what’s trusted or normal.  An agent who becomes fluent in Dari through extensive schooling is good.  One who grew up in Central Asia and not only knows the language but can effortlessly understand the cultural context of what goes on there is priceless.  And yet…try to do a background check of someone who grew up in Charikar (or places even further afield).

…this restriction in sense perception with the structural winnowing of imperial architecture both material and political, and the amount of information arriving at the policy-making end rapidly approaches nil.  Whether driven by considerations of status or force protection, the blinding and deafening effects of isolation are the same—…pg. 94

But, maybe even that doesn’t go far enough.  Kelley seems to argue that trying to capture ‘slippery knowledge’ within the guidelines of highly structured organizations may just not be possible.  Instead, what may be required is the reliance on other types of networks.

Truly alternative information networks—the places where surprises come from—may circulate among these “oppositional structures” and associations of more genuinely state-separated “civil society” like modern incarnations in the liberation theology movements of Latin America or the exploding Christian Pentecostal churches of sub-Saharan Africa in the 1980s. pg. 68

So, what are the characteristics of success?

Organizations that thrive will likely not be those that seek information dominance (a chimera if ever there was one), but those that provide innovative perspectives and connections. In practice, this means there is a place in strategic intelligence analysis for topics as apparently irrelevant as women’s literacy rates and rural access to irrigation systems.  193

The point is that frequently sound analysis isn’t about paring away the extraneous to reveal the “bottom line” or to identify the “center of gravity.” Rather, more fruitful results may oft en open up from exploring marginal aspects of context, picking a single stray thread and pulling until the fabric unravels, or tugging at a single weed until an entire underground network of roots emerges from the earth.  71

Really a superb read.  Check it out.