One recent morning on the way to his office in Palo Alto, Calif., software engineer Bob McGrew made some quick, rather innocuous stops. He slipped into a Starbucks and grabbed a coffee — a regular, with room for cream. He swiped his credit card and continued to a nearby ATM. A couple of punches on the keypad, and out came $100 and a receipt.
McGrew works for Palantir Technologies, a Silicon Valley company that, among other things, sells software to the FBI and CIA. The software is meant to help the government find patterns among terrorists — by spotting clues in everything from phone calls to the kind of trail McGrew left on his way to work.
So, imagine for a moment that instead of being a software engineer, McGrew was a terrorist. In that case, his simple morning errands could be seen in a more sinister light. And he would have left a data trail in at least three places: at Starbucks, at his credit card company and at his bank. If he were a suspect, the government might well have scooped up that information as a way to track him down or see with whom he is associating.
Intelligence officials have been hoping for some time that vacuuming up vast amounts of information and putting it into a computer would uncover some sort of discernable terrorist pattern. The technique, known as data mining, is controversial because information on the innocent, as well as potential terrorists, ends up in the same database. Now it is increasingly unclear whether data mining will ever really work because terrorists don't appear to have predictive patterns.
"We don't even have enough of a data set to get a good pattern of 'What does a terrorist look like?' " says Fred Cate of Indiana University's Center for Applied Cybersecurity Research. "And terrorists, of course, are constantly changing their patterns because, quite simply, they don't want to get caught."
That's why they use one-time cell phone numbers and drop-box addresses.
"There had been, over the past seven years, this sense that if you collect more and more data and put it into a powerful enough computer, shake it and bake it the right way you'll come up with the unknowns" — terrorists who aren't yet on law enforcement's radar screens — says Jim Dempsey, the executive director of the Center for Democracy and Technology, a privacy group in San Francisco.
"I think, and other people who are more technically adept than I think, that's really a fool's errand."
A Software Solution?
"Terrorists are entrepreneurial, and that's what makes fighting terrorism tricky," says Alex Karp, the CEO of Palantir Technologies. His company, named for the all-seeing stones in the Lord of the Rings series, sells software that fuses vast amounts of disparate data together and then allows it to be searched for connections. Using Palantir, intelligence analysts can look for links using phone and tax records, as well as the kind of data trails McGrew left behind with his morning errands.
Intelligence information basically comes in two forms — structured and unstructured data. Structured data have fields, like a spreadsheet. It is relatively easy to search. Unstructured data, which is the form most intelligence information comes in, are like notes in a reporter's notebook. There is a tip here, a phone call there. Palantir can search both kinds of data simultaneously. That's one of the reasons why the FBI, CIA and New York Police Department, among others, have all recently started using the software.
How It Works
The program works a bit like the Six Degrees of Kevin Bacon game. You can start with a suspect or a terrorist organization, and the software chews through a bunch of databases looking for common threads.
The software is more Mac than PC. There are cartoon-looking icons. The searches are meant to be intuitive. There are buttons to click on. You could, for example, start with a name or address and have the software search across platforms to see who may be connected to either one of those pieces of information. Click on a map icon, and the software puts the names on a Google map to show where they live. Did the suspects travel to the same places? Another search function would allow analysts to search that as well.
Intelligence agencies are moving toward using analysis software for their data-mined information because just hoping that some terrorist pattern emerges, at this point, seems like a dim light on a far-off field. Late last year, the National Academy of Sciences' National Research Council said so little is known about how terrorists operate, analysts couldn't spot a terrorist pattern even if there were one.
"The notion that if we just put billions into data mining, we'll be safe, I don't hear anyone advocating that any longer," Cate says.