Stripmining Twitter and Facebook for Security, Part 1

See what the Pentagon is doing? 

I wonder how much doing that would cost the UK? More than £1.8bn, I’ll bet.

All to help us not be surprised (again) next time a major geopolitical or terrorist incident occurs (again) and to keep us safe from evil (again); of course monitoring social media and intercepting phonecalls is the answer (again) because more monitoring and interception is always the answer (again).

The Defense Department wants new computer tools to analyze mounds of unstructured text, blogs and tweets as part of a coordinated push to help military analysts predict the future and make decisions faster.

The search is part of the Office of Naval Research’s “Data to Decisions” program, a series of three-to-10-year initiatives that will address the volume of information that threatens to overwhelm planners in the digital age, contract databases indicate. The goal is to build an open source system that can unite various tools that collect, manage and draw relationships between data sets.

In a program announcement, ONR is calling for computer algorithms to predict events, fuse different forms of information and offer context on unfolding events. The office expects to spend $500,000 each year in funding. “The Department of Defense recognizes the potential for text analytics to play a vital role in future capabilities that inform timely and accurate situational awareness in time-constrained, uncertain, and complex environments,” the tender reads.

Defense is seeking ways to predict the future by monitoring Twitter, blogs and news, and determining the “frequency of contacts between nodes or clusters.” As networks grow larger and more complex, researchers have found it harder to monitor group behavior. ONR also wants researchers to discover networks that could be hidden within networks, and how information and money flows through a community.

Officials also want tools that fuse and assimilate multiple, incomplete data sets on agriculture, weather, terrain, demographics and economic indicators to find patterns. ONR is especially interested in ways to comb text-based information to provide more nuanced views of how groups, such as terrorists, operate by extrapolating the “stated values and beliefs that motivated behaviors of interest,” “community structure and clusters of social networks” and the level of “emotional support expressed towards topics or persons.”

The office also seeks better technologies for machine translation and processing — translating physical characters or sounds — into one machine-readable language.

via Can you predict the future by reading Twitter? The Pentagon thinks maybe. –

3 Replies to “Stripmining Twitter and Facebook for Security, Part 1”

  1. Oh dear; looks like augury just came back into fashion, but in a really expensive form.

    This is a fine example of human perception screwing up, again. First, just because we seem to have an amazing facility for pattern-matching, we reckon we can get a machine to do it at least as well – and to spot the “right” patterns, without being able to interrogate its decision processes (see eg 4.1.2, Biased Data Sets, in Priddy and Keller, “Artificial Neural Networks: An Introduction”).

    There’s also the matter of thresholds. As I once took the opportunity to say to a (now retired) senior person in the Security Service, “the problem with mining data for terrorism is that if you look hard enough you’ll always find it, even where there aren’t any terrorists”.

    A couple of years back, some nice people in SIS’ Human Behaviour unit released a paper, the conclusion of which is that you can’t profile for terrorism; couple this with the famous “6 degrees of separation” and the cell pseudo-structure used by almost every non-lone-wolf, non-government terrorist organisation since the IRA, and it’s pretty obvious that such a project is doomed from the outset.

    The real shame is that all that money could be spent, arguably more productively, on training and retaining officers involved in traditional tradecraft, and equipping hard-strapped forensics labs to keep up with conventional demands.

Leave a Reply

Your email address will not be published. Required fields are marked *