Social media, intelligence analysis and deception

Courtesy of a reader comes this piece from Social media is the ‘new’ thing in many circles and while I am amazed at the prodigious amount of creativity it has unleashed from the minds of the masses, it isn’t likely to be the panacea some think it will be in terms of identifying trends and combing through masses of information quickly.

The fabulous city of gold that our modern conquistadors are searching for is based on the legend of the Wisdom of Crowds. That legend may or may not have a basis in truth but it’s sunk into the consciousness of some to a degree that isn’t really warranted at this point. Part of its appeal is the realization that subject matter experts can be so myopic that they fail to see dramatic changes in their environment . Getting a broad range of opinions from people of widely varying backgrounds, so the thinking goes, can reduce that bias.

There are aspects of this argument that I find highly appealing and suspect there is real value there. But (and it’s a big but) I also believe that differentiating between real gold and pyrite involves a lot of work establishing parameters and processes to avoid swapping one set of biases for another. I’ve written about my experience with prediction markets already and think there’s a real danger there of mistaking the ‘wisdom of crowds’ for the ‘regurgitation of the average opinion of the punditry class’. Unfortunately, it’s very easy to get swept up in the infatuation of the whole concept without understanding what’s required in order to get a reasonable chance of success from your efforts.

Case in point: In the financial industry, there is money to be made by making transactions just a few seconds faster than everyone else. In order to squeeze out every ounce of potential, some companies have developed ‘algorithmic trading’. This (as I understand it) hands off trading decisions over to software. When certain criteria are met, an order to buy or sell is put through. That whole process can occur without human oversite. Some companies have begun to incorporate things like news feeds and twitter streams into their software. Set certain parameters like “buy when company X releases a statement which contains the words ‘profits exceed’ and you’re off to the races.

As data mining gets more refined, there’s no real end to this sort of thing. I suppose there may come a day when someone develops an algorithim which follows the facebook pages of the children and spouses of Federal Reserve members. An increase in posts from them talk about mom or dad being ‘snappy and grumpy’ might indicate bad economic news is coming down the pike.

Sounds cool, right? A brave new world in which we can not only sort through the incredible amount of data which is being created now but can actually derive understanding from it. Easy. Once we tweak the right algorithms we won’t even need analysts anymore. The great computer can tell us what’s going to happen.

That, of course, requires someone to create the right algorithims. Good luck with that.

We’ve seen a couple of (potentially) catostrophic failures from algorithmic traiding which should give us pause before going too far down this road:

In September of 2008, Google News posted as current a six-year-old article about United Airlines filing for bankruptcy; as a result the value of UAL stock dropped by 75% but recovered as the error was spotted.

Looks like we don’t need a guy in a cool leather jacket to break the stock market.

Now, that was an accident. Imagine the possibilities if someone is intentionally attempting to deceive the market (or the intelligence community). So, combine that with….

…this post where a bunch of wily Swedes, hoping to learn about how information quality is evaluated as it passes through the interwebs, demonstrated how an anonymous (unverified and incorrect) piece of information can quickly spread throughout the public consiousness and outpace the truth.

They did this by posting a picture of a (fake) screw on Reddit with a cryptic comment about it being associated with Apple. The implication being that Apple was about to build its hardware so that users could no longer access the guts of their gadgets (note:  some users care about this sort of stuff).

The story was picked up by a variety of tech reporters and bloggers, the majority of who clearly stated that the information was unconfirmed and generally treated the story with skepticism.

Then, the interesting thing happened.

Commenters and those who reported second hand on the screw increasingly dropped the ‘unconfirmed’ portion of the story and treated the story as fact, leading to speculation and adding assumptions upon assumptions.

Now imagine a campaign intended to deceive those trading algorithms.  Hit upon the right piece of information and you’re not just one lone voice in the vast information universe but you can get picked up, retweeted, rebroadcast, commented upon with ever greater speculation as everyone scrambles to get page views.  Sure, eventually things will get sorted out but remember, we’re talking about making profits based on price swings that can occur over just a few moments.

And that, of course, assumes you care about profits.  Perhaps you just want to demonstrate the fragility of the system.  You want the market to go through wild swings just to create the impression that someone (maybe the 1%, maybe that junior vice-president just down the street) is rigging the system to screw you.

The possibilities are endless my friends.

So, what does this mean for intelligence analysis?  Well, hopefully you can see but a reliance on social media to determine trends or ‘do the analysis’ can be easily deceived, either intentionally or accidentally.  For every real trend that can be uncovered by tapping into the wisdom of crowds there are a dozen Kony2012, LOLCats and other flotsam and jetsam that will be forgotten with the next sunrise.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s