I’ve just completed the final portion of the Intermediate Intelligence Analyst course that I’ve been writing about and after thinking about parts 3 and 4 am ready to disgorge my final judgment.
If you just want to cut to the chase: This is a great course. Maybe the best general analytical course I’ve seen. It’s got some teething problems but none that can’t be overcome. If I were running an analytical shop, this course would be part of every analysts’ career development plan
Ok, onto the details
Module 3 was designed “to improve the writing skills of intelligence analysts. This will be accomplished by providing the learners with an understanding of the writing process and the strategies that drive that process. Learners will engage in numerous problem-solving exercises and activities, in addition to engaging in numerous practice writings. The instructor will provide the learner with meaningful feedback that will improve the learner’s writing skills.”
The same problems with the course management software I mentioned in the first module were back with a vengeance. They used the ANGEL software which seems to be fairly widely represented in the educational field so I’m hesitant to say the software was bad but I’m not sure how to account for the numerous log on problems, clunky and slow posting performance, limited formatting options and maddeningly absurd email system. If there was one point of agreement across all the students (and, I suspect, among the staff as well) it was the universal dislike of the course management software.
That being said, even with good software I think there were some problems with the distance learning portion of the course. Distance learning is difficult in the best of circumstances and given the limited classroom time (about 7 total days) and the need to establish and emphasize good teamwork and collaboration the class missed an opportunity to use the social capabilities of the software to begin that team building. The course web page became a vehicle for submitting assignments and not much else, encouraging students to hop on the course website, upload their assignment and get the hell out of there as quickly as possible.
I’ve taken a number of distance learning courses of widely varying quality and the ones that have succeeded have had a teacher who keeps students engaged and communicating on the discussion boards/email system. Ideally students will do this on their own and sometimes they do, but a facilitator can’t count on that and should be prepared to prod the class to get them to speak up.
My recommendation:
Make discussions on the bulletin boards a required part of the coursework. One way I’ve seen it work is where the teacher will ask one (or more) questions per week and all students are required to respond to the teacher as well as at least two comments to other students in a meaningful way (which means “I totally agree!” wouldn’t count). For this course specifically, this greatly increases the amount of writing available from each student for evaluation without placing an undo burden on the students. They don’t need to write ‘War and Peace’, they just need to submit a well thought out answer and use appropriate citation rules. I’ve seen quite good discussions go on long, long after grading was done between students because their interest buttons were pushed. Perhaps this would be a good place to discuss some ‘big picture’ issues and let the fur fly: Why haven’t we had a major attack on U.S. soil since 9/11? Is there a meaningful difference between crime analysts and intelligence analysts and what should their roles be in law enforcement analysis? Should analysts focus on tactical or strategic products? The list could go on and on.
The module had two ‘big’ (big for a professional development course, positively puny for a graduate level course) assignments. The first was to read a series of articles from The Economist about the War on Terror and designed to “enhance the learner’s cognitive ability to write an intelligence product that demonstrates comprehension of ” audience, purpose, form and the collection and collation steps of the intelligence process.
The instructions went on to say:
Applying the considerations described in the lecture material, craft the documents to be appropriate for the different readers. Be prepared to defend you decisions regarding type and level of content, degree of analysis, general organization, level of vocabulary, and word choice…This exercise will focus on the ability of analysts to collect and identify significant facts from an assigned source and communicate the results in a written product for later evaluation and analysis…You are not “interpreting” the article, nor are you “arguing” for or against its point of view. You are summarizing the article, and your summary should be accurate and free from personal bias.
Now, this assignment didn’t have anything to do with the collection phase of the intelligence process since we were told what information source to use and there was no room for bringing in outside sources of information.
I was torn on this assignment since I hated the articles (which were superficial, poorly imagined and gave the impression that some editor just had a bunch of stuff lying around and decided to cram it into a ’special report’) but enjoyed the assignment. The length of the source articles precluded (in my opinion) any attempt to summarize each component and rather forced the writer to look for underlying themes which connected the scattered ideas and concepts (in other words, we were asked to do the work that the writers should have done). In order to do that however, you do need to be able to place the source document in some context and need to have at least a rudimentary understanding of current events in Pakistan, Afghanistan, Iraq, as well as European approaches to terrorism and current theories of terrorist dynamics in general. If you don’t have that this assignment becomes much more difficult. Likewise, if you are fairly well versed in these areas you can write this pretty easily.
My recommendation:
Use different source material. Ideally, pick a subject that won’t allow students to ‘game’ the system if they come to the table with prior knowledge about a subject. The purpose of this block is writing to different audiences and so it shouldn’t really matter what students are writing about. A subject that isn’t law enforcement or terrorism related should put students on a level playing field and therefore make it a bit easier to see how they are at the skills of summarizing, audience analysis, and writing skills. This would require the course managers to explain this concept explicitly to students so they can understand why they’re being asked to write about…the dangers of Astro-turf (or whatever) instead of what they do for a living.
The second assignment was to create an intelligence assessment which would have real value for the students’ agency. This would both serve to provide the agency with something of value for sending their people to school.
This brought out one problem with the course that I regarded as only relatively minor until this point. In the student guide, each block of instruction and assignment was given a time limit. The final writing assignment was slated for four hours. I assumed that those limits were created for a purpose, namely to encourage analysts to organize their thoughts and produce a product under some pretty serious time constraints. So, I took four hours and researched, wrote and submitted my paper for evaluation on the day it was due.
And I waited.
Nine days later I received an email addressed to the class which said ‘If you haven’t finished your assessment yet, please try to have it done in the next seven days.’
Now, had I known I had two weeks to research, write, and revise the assessment I would have ended up with a very different product. You can have a valuable training experience either way you go but they really should have been more clear about this issue.
My recommendation:
Determine what training value you want students to take away from the assessment process and tailor the assignment to address it.
Using the time constraint forces analysts to quickly conduct decide what information they think they can get, how to organize it and how to characterize their work to their audience. In addition to the product itself, analysts should be asked/mentored in the process of how to compress the analytical/writing process.
An assessment with a longer time frame between assignment and submission could have analysts demonstrate the various steps of the analytical and writing process. In this scenario, I’d recommend breaking down the project into multiple mini-assignments to verify understanding and competence. Steps might include:
- Develop your hypothesis
- Create a collection plan
- Apply analytical techniques
- Write your outline
- Write draft
- Revise
- Finalize
Each one of those steps could be an assignment that needs to be completed before being allowed to move on.
Module 4 consisted of two main portions: a capstone exercise designed to merge the variety of skills taught in the earlier modules and a briefing of our assessments that we completed in Module 3.
The Capstone exercise was quite fun and well done. I don’t want to reveal any spoilers but it centered around potential terrorist activity and we played various fictional fusion centers trying to understand what, if anything, was going on. It wasn’t totally realistic since at a fairly early stage the federal authorities would have taken over an investigation like this but I’m not sure that was a serious flaw. It occurred to me that there were numerous points during the exercise that the class would have benefited from a brief pause to discuss how things were progressing and how might things work in the ‘real world’.
One specific example revolves around information sharing and collaboration. Our four groups were in different rooms and could contact each other via email or (more commonly) by walking down the hall. A lot of confusion was avoided by walking 10 meters. It might have been nice to have the analysts really think about how things would have changed if our fusion centers were separated by hundreds of miles or if we had institutional and/or personal biases which would have discouraged information sharing.
Also, each group was given free reign in terms of how to organize themselves, their information and their workload. It would have been nice to discuss the thinking that did (or didn’t) go into that decision making process. Was it thought or or just happen? Was there a leader or group consensus? Which worked better in which situations?
The briefings went well and I don’t have much to say about them. They set up a system for peers to evaluate each other that seemed to work OK.
So…that’s it. The course administrators were top notch and professional throughout. Their dedication to the course and the class were clearly evident and they frequently exhibited more patience than I could muster.
Kudos to the Sacramento Regional Office of Homeland Security for a job well done.
