From a Silicon Valley office strewn with bean-bag chairs, a group of twenty-something software engineers is building an unlikely following of terrorist hunters at U.S. spy agencies.
One of the latest entrants into the government spy-services marketplace, Palantir Technologies has designed what many intelligence analysts say is the most effective tool to date to investigate terrorist networks. The software's main advance is a user-friendly search tool that can scan multiple data sources at once, something previous search tools couldn't do. That means an analyst who is following a tip about a planned terror attack, for example, can more quickly and easily unearth connections among suspects, money transfers, phone calls and previous attacks around the globe.
Employee Nick Miyate demonstrated the red light and bubble machine that turns on whenever an engineer fixes a software error, or "build break."
Palantir's software has helped root out terrorist financing networks, revealed new trends in roadside bomb attacks, and uncovered details of Syrian suicide bombing networks in Iraq, according to current and former U.S. officials familiar with the events. It has also foiled a Pakistani suicide bombing plot on Western targets and discovered a spy infiltration of an allied government. It is now being used by the Central Intelligence Agency, the Pentagon and the Federal Bureau of Investigation.
Yet Palantir -- which takes its name from the "seeing stones" in the "Lord of the Rings" series -- remains an outlier among government security contractors. It rejected advice to hire retired generals to curry favor with the agencies and hired young government analysts frustrated by working with slow-footed technology. The company's founders knew little about intelligence gathering when they started out. Instead, they went on a fact-finding mission, working with analysts to build the product from scratch.
"We were very naive. We just thought this was a cool idea," says Palantir's 41-year-old chief executive Alexander Karp, whose usual dress is a track-suit jacket, blue jeans, and red leather sneakers. "I underestimated how difficult it would be."
Technology like Palantir's is increasingly important to spies confronting an information explosion, where terrorists can hide communications in vast data streams on the Internet. Intelligence agencies are struggling to identify and monitor such information -- and quickly send relevant data to the analysts who need it. U.S. officials say the software is also crucial as the country steps up its offensive in difficult theaters like Afghanistan. There, Palantir's software is now being used to analyze constantly shifting tribal dynamics and distinguish potential allies from enemies, according to current and former counterterrorism officials familiar with the work.
"It's a new way of war fighting," says former Assistant Secretary of Defense Mary Beth Long. While there are many good systems, Ms. Long says, with Palantir's software "you can actually point to examples where it was pretty clear that lives were saved."
Palantir's chief rivals are I2 Inc., a 20-year-old software company with offices in McLean, Va., and a handful of defense contractors who have been building software for intelligence agencies for years. I2's general manager, Todd Drake, dismisses his upstart competitor as "the new sexy thing," saying that Palantir won't be able to make lasting inroads in a government market that prizes the stability of established companies. Palantir CEO Mr. Karp says such criticism doesn't trouble him. He says the company is already expanding rapidly.
Palantir's roots date back to 2000, when Mr. Karp returned to the U.S. after living for years in Frankfurt, where he earned his doctorate in German social philosophy and discovered a talent for investing. He reconnected with a buddy from Stanford Law School, Peter Thiel, the billionaire founder of online payment company PayPal.
Palantir may look like a typical Silicon Valley start-up, with free food and the usual comforts to make work more like home. But with clients like the FBI and CIA, it's far from the usual software company.
In 2003, Mr. Thiel pitched an idea to Mr. Karp: Could they build software that would uncover terror networks using the approach PayPal had devised to fight Russian cybercriminals?
PayPal's software could make connections between fraudulent payments that on the surface seemed unrelated. By following such leads, PayPal was able to identify suspect customers and uncover cybercrime networks. The company saw a tenfold decrease in fraud losses after it launched the software, while many competitors struggled to beat back cheaters.
Mr. Thiel wanted to design software to tackle terrorism because at the time, he says, the government's response to issues like airport security was increasingly "nightmarish." The two launched Palantir in 2004 with three other investors, but they attracted little interest from venture-capital firms. The company's $30 million start-up costs were largely bankrolled by Mr. Thiel and his own venture-capital fund.
They modeled Palantir's culture on Google's, with catered meals of ahi tuna and a free-form 24-hour workplace wired so 16 people can play the Halo video game. The kitchen is stocked by request with such items as Pepto Bismol and glass bottles of Mexican Coca Cola sweetened with sugar not corn syrup. The company recently hosted its own battle of the bands.
One of the venture firms that rejected Palantir's overtures steered the company to In-Q-Tel, a nonprofit venture-capital firm established by the CIA a decade ago to tap innovation that could be used for intelligence work. As Silicon Valley's venture funding dries up, In-Q-Tel says it has seen a surge of requests from start-ups in the last year or so, many of which now see the government as an alternate money stream.
In-Q-Tel invested about $2 million in Palantir and provided a critical entreé to the CIA and other agencies. For his first spy meeting in 2005, Mr. Karp shed his track suit for a sports coat. He arrived at an agency -- he won't say which one -- and was immediately "freaked out" by security officers guarding the building with guns. In a windowless, code-locked room, he introduced himself to the first official he met: "Hi, I'm Alex Karp," Mr. Karp said, offering his hand. No response. "I didn't know you really don't ask their names," he says now.
Mr. Karp showed the group a prototype. The software was similar to PayPal's fraud-detection system. But instead of identifying and connecting cyber criminals, it focused on two hypothetical terror suspects and followed their activities, including travel and money transfers.
After the demo, he was peppered with skeptical questions: Is anyone at your company cleared to work with classified information? Have you ever worked with intelligence agencies? Do you have senior advisers who have worked with intelligence agencies? Do you have a sales force that is cleared to work with classified information? The answer every time: no.
But the group was sufficiently intrigued by the demo, and In-Q-Tel arranged for Palantir engineers to meet directly with intelligence analysts, to help build a comprehensive search tool from scratch.
Every other week for about two years, the engineers returned to Washington with a revised product, based on analysts' requests. The approach won over a number of tech-savvy younger analysts who asked their bosses to adopt the software.
Spy agencies like the CIA and military intelligence organizations have hundreds of databases each, most of which aren't linked up. A single database might contain reports from field agents or lists of known terrorists or companies thought to be financing terrorism. To conduct an investigation, analysts have to query individual databases separately, then try to make sense of the data -- frequently with pen and paper.
With many of the existing search tools, analysts also can't access some files on terrorist suspects or other threats because a bit of data in the file is classified at a level higher than they are allowed to see. That is a problem, because making connections among new clues and existing data is a key to foiling terrorist plots. Among the missed opportunities cited by post-9/11 investigations were the failure to see that five of the 19 hijackers used the same phone number as ringleader Mohammad Atta to book their airline tickets, two used the same frequent-flier number, and five used two common addresses to make their reservations.
Palantir's software plugs these gaps by using a "tagging" technique similar to that used by the search functions on most Web sites. Palantir tags, or categorizes, every bit of data separately, whether it be a first name, a last name or a phone number. That means if only one piece of data in a file is classified top-secret, an analyst with a lower level clearance can still see the rest of the data. It also allows analysts to quickly tag information themselves as it arrives in the form of field reports from spies overseas, and to see who else in the agency is doing similar research so they can share their findings.
By connecting different databases, analysts can start making new links. Someone could see, for example, that one terrorist suspect flagged in one database has been living at the same address as the cousin of another suspect whose information is in another database, and that the two men flew to the same city after money was transferred to a particular bank account.
Some analysts say Palantir's strength is helping analysts draw inferences when confronted with an enormous amount of disparate data. Palantir's tool is getting a thumbs-up from officers using it. "It is much simpler to understand the results of inquiries, and provides more in-depth database links then the current programs in use by the Army today," says Captain James King, an Army intelligence officer.
A handful of agencies have adopted Palantir's software for specific projects. The Pentagon recently used it to track patterns in roadside bomb deployment. Officials say analysts were able to connect two reports and conclude that garage-door openers were being used as remote detonators and soldiers on the ground had a new device to look for.
Analysts at West Point recently used Palantir's software to map evidence of Syrian suicide-bombing networks buried within nearly 700 al Qaeda documents, including hundreds of personnel records that the military recovered in Iraq. The analysts did an initial sweep of the data without the Palantir tool and assembled a report on foreign fighters in Iraq who were paying Syrian middlemen to send over suicide bombers.
A second analysis with Palantir uncovered more details of the Syrian networks, including profiles of their top coordinators, which led analysts to conclude there wasn't one Syrian network, but many. Analysts identified key facilitators, how much they charged people who wanted to become suicide bombers, and where many of the fighters came from. Fighters from Saudi Arabia, for example, paid the most -- $1,088 -- for the opportunity to become suicide bombers.
Such details helped local law enforcement break up some of the rings, said one U.S. official familiar with the work. It also revealed the extent to which al Qaeda was relying on mercenary smuggling networks, rather than true believers, to get suicide bombers into Iraq.
In the past two years, Palantir's work in Washington has expanded from eight pilot programs to more than 50 projects, executives say. The Australian government is now a client, and the NSA is eyeing Palantir, as is the U.K., current and former government officials say.
The company expects to turn a profit on its government work this year -- it recently started working with financial companies, but says it is too early to see any profits from that yet -- and for revenues to reach $100 million within the next two years. Palantir also maintains a pro-bono roster. It examined the cyber attacks on the central Asian country of Georgia last year, and earlier this year helped Canadian researchers uncover a cyberspying operation on the Dalai Lama. The company is now working with a nonprofit investigative group in Washington to resolve open questions in the 2002 murder of The Wall Street Journal reporter Daniel Pearl.
In 2007, Mr. Karp hired his first intelligence-agency alum, David Worn, to open a Washington office. Mr. Worn says he was among the younger agency analysts who felt trapped in an outdated system.
As he builds up the East Coast office, which now employs 20 people, Mr. Worn says that the company is still figuring out "how to live in those two worlds" of Silicon Valley and Washington. One thing that does seem to help: He and his colleagues make frequent trips to Palo Alto to make sure they don't lose "the vibe of the Shire," the home of the hobbits from Lord of the Rings.