February 10, 2006
Total Information Awareness on the Comeback Trail
Of course, that's not what it's called. Anymore. The vast data mining attempt by the government was killed by Congress in 2003 after the project leaked to the press. Something about privacy concerns apparently piqued congress into acting against it.
Now, some three years later we find a report in the Christian Science Monitor about ADVISE, Analysis, Dissemination, Visualization, Insight, and Semantic Enhancement. The program received $50 million in funding within the Department of Homeland Security. The purpose of ADVISE is to use data mining to foil terrorism. It does it by linking transactional information such as blog postings (anyone listening to this?) and email to other information held in government databases. So says the CSM at least. The government isn't talking, and when it does it seems to cough a lot.
Here's a quote from the story that sets out the scope of the project, and it's truly staggering:
What sets ADVISE apart is its scope. It would collect a vast array of corporate and public online information - from financial records to CNN news stories - and cross-reference it against US intelligence and law-enforcement records. The system would then store it as "entities" - linked data about people, places, things, organizations, and events, according to a report summarizing a 2004 DHS conference in Alexandria, Va. The storage requirements alone are huge - enough to retain information about 1 quadrillion entities, the report estimated. If each entity were a penny, they would collectively form a cube a half-mile high - roughly double the height of the Empire State Building.
A quadrillion, by the way, is one thousand trillion, or a thousand million million, or 10 to the 15th power. According the the U.S. Census, there are 298,081,786 Americans and 6,496,915,442 total humans on the planet.
Critics of the system note that there are no privacy safeguards in the data mining system, or at least none in public awareness. The story also discusses some of the techniques the system uses, including a component called Starlight which sets data out in 3D and other graphical patterns. This can show linkages through social behavior analysis in ways that traditional techniques may not.
Other questions not answered include what happens to the data, how long is it stored (likely forever) and when will the government use it to go after non-terrorists. After all, with a system such as this in place, it might be easier to track all those allegedly shady transactions by Enron. Would that information be available to prosecutors?
The information the government got from Yahoo and MSN and would like to get from Google would go very nicely in here to feed in more data. But then, the government really wants that information to defend COPA. You wouldn't think that was a ruse? Nah.
February 10, 2006 | Permalink
TrackBack URL for this entry:
Listed below are links to weblogs that reference Total Information Awareness on the Comeback Trail: