Hancock programming language

Alex Dang

After the September 11, 2001 terrorist attacks, the United States Congress expanded the surveillance capabilities of the Federal Bureau of Investigation (FBI) and the National Security Agency (NSA). 

Empowered by the USA PATRIOT Act, the FBI began to access and collect, without warrants, the private data of many U.S. persons, ostensibly for the purpose of counterterrorism. The PATRIOT Act expanded the bureau’s ability to collect telecommunications data—like billing records, customer information, or call records—to issue national security letters (NSLs). The use of NSLs by the FBI had first been permitted by Congress in 1986 for investigatory purposes; the process to obtain them required approval from the director of the FBI or someone who he designated. In other words, getting an NSL was pretty difficult for an FBI Agent. Imagine having to get approval from the CEO of your company just to get some information! It would probably make you think twice before making such a request.

But after 9/11, after the PATRIOT Act had allowed all FBI Special Agents who were in charge of a field of office (56 total) to approve NSL requests, the frequency and volume of letter requests became enormous—from 2003 to 2005, FBI agents were granted a staggering 140,000 NSL requests on 24,000 U.S. persons. Not only that, but the FBI could hold on to these records indefinitely, allowing the bureau to construct an electronic database of surveillance data on their own citizens. Even if a federal agency has the best intentions at heart, it should be concerning to all Americans when surveillance databases are constructed which consist of not just the “bad guys,” but everyday private citizens too. 

Those who were issued NSL requests couldn’t even speak out about them, not even to their lawyers, because NSLs came with gag orders, forcing their silence. Gag orders might help with keeping investigations as confidential as possible, but when they assist in domestic surveillance, citizens should at least have the ability to know that a tab is being kept on them. Otherwise, helping the government without being able to push back on their actions assumes that the government’s actions are always justified (which would be an incredibly naive thing to assume). 

While it may seem like the FBI imposed itself on holders of data, particularly telecommunications companies, the relationship wasn’t so one-sided. In fact, telecommunications giants like AT&T and Verizon cooperated with requests for transactional records from the NSA. Telecoms have been quite chummy with federal agencies, including the Department of Homeland Security, working together on national cybersecurity issues and information sharing. 

AT&T and Verizon were both sued in 2007 in federal court for aiding the NSA in its quest to access the telephone records of billions of Americans (again, supposedly for counterterrorism measures). While the FBI was handing out NSLs like they were promotional flyers—You get one, Verizon! Where do you think you’re going, BellSouth? You get one too!—the NSA had its own domestic surveillance operation, the secret Terrorist Surveillance Program, which aimed to find terrorist suspects and sympathizers. The NSA intercepted communications, leaving and entering the United States, screening them for possible terrorist links.

The innovations telecom companies were developing were used for surveillance purposes. Take, for example, Hancock, a programming language developed by researchers at AT&T Labs in 1998 to analyze data streams. The language, as it was designed, had nothing to do with domestic surveillance—it was intended to improve the efficiency of data mining, which extracts new knowledge from massive amounts of data. Specifically, AT&T used Hancock to analyze telecommunications data from its entire long-distance network, which included around 300 million records from 100 million customer accounts every day. Hancock, which could be used to detect social networks among customers, was also used by AT&T to detect instances of fraud and for marketing.

In an article for Wired, published in 2007, Ryan Singel claimed that Hancock had created the concept of “communities of interest”—those social networks previously discussed—which the FBI took interest in. At the time of the article’s publishing, it had been reported that the FBI was making warrantless requests for “communities of interest” records from telecom companies. This is only a conjecture from one person, and without knowing whether or not these record requests were granted, not much can be definitively made about the relationship between Hancock and FBI domestic surveillance. But if we’ve learned anything about the close relationships telecoms have had with federal agencies in the past (and present), it wouldn’t be a stretch to assume that AT&T, charged with a sense of duty to help the FBI’s cause, complied with the requests.

Brian: Great draft! I thought your blogpost was very informative and contained a plethora of information. However, my suggestion to make this blogpost more accessible for all audiences would be to be a little more informal in your long paragraphs by including your personal opinions and analysis for the information stated. I think that would make your blogpost informative and interesting to read.

Bibliography:

  1. Bendix, William and Paul J. Quirk. 2016. “Deliberating Surveillance Policy: Congress, the FBI, and the Abuse of National Security Letters.” Journal of Policy History 28(3):447–69.

  2. Busch, Nathan E., and Austen D. Givens. 2012. "Public-Private Partnerships in Homeland Security: Opportunities and Challenges." Homeland Security Affairs 8(1):1-25.

  3. Cortes, Corinna, Kathleen Fisher, Daryl Pregibon, Anne Rogers, and Frederick Smith. 2004. “Hancock: A Language for Analyzing Transactional Data Streams.” ACM 26(2):301–38.

  4. Singel, Ryan. 2007. “AT&T Invents Programming Language for Mass Surveillance.” Wired. Retrieved November 8, 2020.

  5. Smyth, Padharic. 2000. "Data Mining: Data Analysis on a Grand Scale?" Statistical Methods in Medical Research 9(4):309-27.

  6. Theoharis, Athan. 2016. "Expanding U.S. Surveillance Powers: The Costs of Secrecy." Journal of Policy History : JPH 28(3):515-534.