I was corresponding with an analyst going through the first module of the training program I reviewed earlier this year and the issue of peer review came up. It remains difficult to manage since most people (either in class or at work) don’t want to make waves, and perhaps more importantly, don’t want to antagonize others and there aren’t really any perceived consequences to letting less than great analysis slip through the cracks.
There’s a feeling that it’s best to take it easy on peer review and not give the other fellow a hard time but that really misses the point. If we assume that our work a) influences the allocation of precious resources and personnel and c) has an impact upon public safety then why would we tolerate shoddy work just to not make waves? It would be like police officers searching subjects where one doesn’t do a thorough job. Should the other officer just let it go because he doesn’t want to be a stickler? Of course not.
Of course, if you’ve read even a small sampling of this blog, dear readers, you’ll know that I’m highly dubious of both a) and b) above. Still, I’m a big believer that analysts should both know how to do such a review as well as (and perhaps even more importantly) know what to do when they get one. Hey, I get just as defensive as the next guy when someone gives me back some of my work marked up along with questions about my superficial and slapdash analysis but I’ve since learned that in virtually every case such criticism has resulted in a stronger final product…even when my reviewer is a totally off base bone head.