CIA Predicts a Big Data Payoff, and Soon

predictive analytics

It’s been one year since the CIA established the Directorate for Digital Innovation—the first new major directorate stood up by the agency in more than 50 years.

The goal of the new directorate is to integrate emerging technological capabilities more closely with the traditional roles of intelligence collection, analysis, production, and dissemination. But one small part of this new endeavor is beginning to pay off in a big way.

The CIA recently stood up an “anticipatory intelligence unit” that leverages emerging capabilities in big data and computing power to help the agency’s analysts stay ahead of developing situations around the world.

“I don’t call it predictive analysis because that implies a precision that we certainly can’t reach,” said Andrew Hallman, deputy director of Digital Innovation at the CIA, speaking last month at the Third Ethos and Profession of Intelligence Conference in Washington, D.C. “There are strides being made not only the growing diversity and variety of data, but the analytics and the computing power to begin to not only get meaning from that data…and give a little bit more edge with the application of streaming analytics to analysts to at least get them a little bit more ahead of the problem.”

CIA Hallman
Andrew Hallman, deputy director of Digital Innovation at the CIA, speaking Sept. 20 at the Third Ethos and Profession of Intelligence Conference in Washington, D.C. (Photo: GWU Center for Cyber and Homeland Security)

But how much ahead of the world’s many complex socio-economic and political problems can the CIA expect to get? According to Hallman, who recently spoke at an event hosted by Government Executive and NextGov, the agency has been able to anticipate social and societal unrest as much as much as three to five days in advance.

Surprisingly, the CIA’s new Directorate of Digital Innovation has been watching and learning from the growing advertising technology market. According to Chris Darby, president and CEO of the CIA’s venture capital firm, In-Q-Tel, ad-tech firms are generating identity intelligence in real time and at a granular level that many in the intelligence community are barred from collecting.

“These companies are intelligence agencies on their own,” said Darby, who also spoke at the Third Ethos and Profession of Intelligence Conference sponsored by the CIA and George Washington University’s Center for Cyber and Homeland Security.

“We’re trying to take the best technologies that are being deployed to these commercial entities and transfer it into the [intelligence] community. And they’re really good at it. They understand the notion of big data, they understand deep learning, and the algorithms that are necessary to pinpoint behavior,” Darby said. “To me, it’s almost a negative. What keeps me up at night is the notion that the commercial sector is going to be shaping individuals or digital tribes on their own, for their own benefit.”

“The old saw in the intelligence community is that the IC is great at secrets and terrible at mysteries,” said Chris Inglis, the former deputy director of the NSA. “In the future, there are going to be fewer and fewer secrets as physical phenomena and the spoken word have fewer places to hide as the sensors and the analytic power essentially reveals all things almost concurrent with the execution of those things.”

Big data will be front and center at MeriTalk’s 2016 Big Data Brainstorm Nov. 16 at the Newseum in Washington, D.C.

Dan Verton
About Dan Verton
MeriTalk Executive Editor Dan Verton is a veteran journalist and winner of the First Place Jesse H. Neal National Business Journalism Award for Best News Reporting -- the highest award in the nation for business/trade journalism. Dan earned a Master's Degree in Journalism and Public Affairs from American University in Washington, D.C., and has spent the last 20 years in the nation's capital reporting on government, enterprise technology, policy and national cybersecurity. He’s also a former intelligence officer in the United States Marine Corps, has authored three books on cybersecurity, and has testified on critical infrastructure protection before both House and Senate committees.
One Comment
  1. Anonymous | - Reply
    The future software to analyze and make sense of all this corporate data mining of private information that has been getting sniffed and permanently stored will emerge in unpredictable formats seeking to expand previously un-dreamt-of fortresses of knowledge currently hidden in all that data. This is not necessarily a good thing, even for the parties that win the race and get to control all these databases - and by implication - all of us. - CMH

Leave a Reply