It was a powerful piece of technology created for an important customer. The Medusa system, named after the mythical Greek monster with snakes instead of hair, had one main purpose: to vacuum up vast quantities of internet data at an astonishing speed.

The technology was designed by Endace, a little-known New Zealand company. And the important customer was the British electronic eavesdropping agency, Government Communications Headquarters, or GCHQ.

Dozens of internal documents and emails from Endace, obtained by The Intercept and reported in cooperation with Television New Zealand, reveal the firm’s key role helping governments across the world harvest vast amounts of information on people’s private emails, online chats, social media conversations, and internet browsing histories.

The leaked files, which were provided by a source through SecureDrop, show that Endace listed a Moroccan security agency implicated in torture as one of its customers. They also indicate that the company sold its surveillance gear to more than half a dozen other government agencies, including in the United States, Israel, Denmark, Australia, Canada, Spain, and India.

Some of Endace’s largest sales in recent years, however, were to the United Kingdom’s GCHQ, which purchased a variety of “data acquisition” systems and “probes” that it used to covertly monitor internet traffic.

Documents from the National Security Agency whistleblower Edward Snowden, previously disclosed by The Intercept, have shown how GCHQ dramatically expanded its online surveillance between 2009 and 2012. The newly obtained Endace documents add to those revelations, shining light for the first time on the vital role played by the private sector in enabling the spying.

Stuart Wilson, Endace’s CEO, declined to answer questions for this story. Wilson said in a statement that Endace’s technology “generates significant export revenue for New Zealand and builds important technical capability for our country.” He added: “Our commercial technology is used by customers worldwide … who rely on network recording to protect their critical infrastructure and data from cybercriminals, terrorists, and state-sponsored cybersecurity threats.”

4943588831_1f9d1924d2_o

Former Endace Director Ian Graham, right, meets New Zealand Prime Minister John Key in 2010.

Photo: NZNationalParty/Flickr

Endace says it manufactures technology that allows its clients to “monitor, intercept and capture 100% of traffic on networks.” The Auckland-based company’s motto is “power to see all” and its logo is an eye.

The company’s origins can be traced back to Waikato University in Hamilton, New Zealand. There, in 1994, a team of professors and researchers began developing network monitoring technology using university resources. A central aim of the project was to find ways to measure different kinds of data on the internet, which was at that time only just beginning to take off. Within a few years, the academics’ efforts proved successful; they had managed to invent pioneering network monitoring tools. By 2001, the group behind the research started commercializing the technology — and Endace was formed.

Today, Endace presents itself publicly as focused on providing technology that helps companies and governments keep their networks secure. But in the past decade, it has quietly entered into a burgeoning global spy industry that is worth in excess of an estimated $5 billion annually.

In 2007, Endace representatives promoted their technology at a huge surveillance technology trade show in Dubai that was attended by dozens of government agencies from across the world. Endace’s advertising brochures from the show, which described the company’s products and promoted the need for greater state surveillance, were published by WikiLeaks in 2013.

One Endace brochure explained how the company’s technology could help clients “monitor all network traffic inexpensively.” It noted that telecommunications networks carry many types of information: Skype calls, videos, emails, and instant message chats. “These networks provide rich intelligence for law enforcement,” the brochure stated, “IF they can be accessed securely and with high precision.”

Germany Mecklenburg-Western Pomerania Baltic Sea - undersea cable to Hiddensee island

Workers lay undersea cables near Hiddensee Island, Germany.

Photo: Ullstein Bild/Getty Images

The United Kingdom’s geographic location — situated between North America, mainland Europe, and the Middle East — made it a good market for Endace.

Many major international undersea data cables cross British territory, and according to top-secret documents from Snowden, as much as 25 percent of all the world’s internet traffic flows through the U.K. The country’s spies have worked to exploit this, with GCHQ tapping into as many of the cables as it can, sifting through huge volumes of emails, instant messages, social media interactions, and web browsing records as they are being transmitted across the internet.

As of 2009, GCHQ’s surveillance of undersea cables was well underway. The agency was measuring the amount of traffic it monitored in tens of gigabits per second (10Gs) — the equivalent in data of about 1 million average-sized emails every minute. The electronic eavesdropping agency was tapping into 87 different 10Gs capacity cables and funneling the collected data into its processing systems for analysis.

By March 2011, GCHQ’s aim was to tap into 415 of the 10Gs cables, and its longer-term goal was to “grow our internet access to 800 10Gs.” The agency wanted to build what it described as the largest covert surveillance apparatus in the world. And in an effort to fulfill that plan, it turned to Endace’s technology.

Leaked documents and emails from Endace, obtained by The Intercept, lay out a series of deals the company made with GCHQ to help it broaden its mass surveillance capabilities. A confidential February 2010 Endace statement of work for GCHQ, for instance, outlined a £245,000 ($299,500) deal to upgrade “monitoring solutions” for the British agency that were designed to intercept large amounts of internet traffic and send it into “memory holes” — repositories used to store the data.

The agency wanted to build the largest covert surveillance apparatus in the world.

Between November 2010 and March 2011, GCHQ purchased more technology from Endace, including specialized surveillance technology built for “FGA only,” a code name the company often uses in its internal documents to refer to GCHQ; it stands for “friendly government agency.”

A November 2010 company document said that “FGA” had an order of 20 systems scheduled for delivery in March 2011. Each system was equipped with two “data acquisition” cards capable of intercepting 20Gs of internet traffic. The total capacity of the order would enable GCHQ to monitor a massive amount of data — the equivalent of being able to download 3,750 high-definition movies every minute, or 2.5 billion average-sized emails an hour.

Endace added in the document that “a potential for 300-500 systems over the next two to three years is being discussed” and noted that it was soon anticipating another order of “30-40 additional systems.” Indeed, the following month a new $167,940 purchase order for 27 more systems arrived, and the items were swiftly dispatched for delivery to GCHQ’s headquarters in Cheltenham, England.

The records of the Endace sales are confirmed by internal GCHQ documents, provided by Snowden, which describe the company’s data capture devices being used as part of mass surveillance programs. GCHQ documents from 2010 and 2011 repeatedly mention the Endace products while discussing the capture of “internet-derived” data to extract information about people’s usage of services such as Gmail, Hotmail, WhatsApp, and Facebook.

GCHQ declined to comment for this story.

endance-product

An Endace diagram depicts a custom data capture system built for GCHQ.

Photo: Endance

Throughout the summer of 2011, at Endace’s offices in Auckland, New Zealand, the orders from GCHQ were continuing to flow in. Meanwhile, the company’s engineers were busy turning their sights to new technology that could vastly increase surveillance capability. Endace was developing a powerful new product for GCHQ called Medusa: interception equipment that could capture internet traffic at up to 100 gigabits per second.

Medusa was first logged in Endace’s sales systems in September 2011. Endace staff produced weekly status reports about their progress and updated GCHQ at biweekly review meetings. By November 18, 2011, the first version of Medusa arrived at GCHQ. “FGA are very pleased with the prototypes we delivered last week,” Endace noted.

Apparently after testing the Medusa prototype, GCHQ requested some refinements. One feature the agency wanted was called “Separate MAC insertion by IP type.” This suggests the British agency may have sought the ability to target individuals by searching internet traffic for the built-in hardware address of their computers, routers, or phones.

Notably, the Medusa status reports reveal that Endace was using taxpayers’ money to develop the new equipment for GCHQ. They state that the Medusa system was being built for “FGA” with funding from the Foundation of Research Science and Technology, the body that handed out New Zealand government research grants.

In 2010, Endace received two grants totaling $11.1 million. A public announcement for the first grant — issued in July 2010 — said the funding was for “50% of the cost of a series of substantial product developments over the next two years,” but did not say what the products were nor who they were for.

A New Zealand government spokesperson told The Intercept that he could not immediately give a “definitive” answer on whether the funding body had known Endace would use the grants to develop surveillance technology for GCHQ, but said it was “highly unlikely Endace would have provided that information, as they were under no obligation to do so.”

Endace has never publicly disclosed any of its work with GCHQ, likely because it is subject to strict confidentiality agreements. In one contract obtained by The Intercept, GCHQ states that Endace staff are bound to the U.K.’s Official Secrets Act, a sweeping law that can be used to prosecute and imprison people who disclose classified information. GCHQ warned Endace that it must not “make any press announcements or publicize the contract or any part thereof in any way.”

The back of two satellite antennae at GCHQ's surveillance base in Bude, England. (Photo By: Education Images/UIG via Getty Images)

The back of two satellite antennae at GCHQ’s surveillance base in Bude, England.

Photo: Education Images/UIG/Getty Images

Endace’s leaked client lists show three main categories of customers: governments, telecommunications companies, and finance companies.

The government clients appear to be mostly intelligence agencies. A 2008 Endace customer list included: GCHQ; the Canadian and Australian defense departments (where their electronic spy agencies are located); a U.S. government contractor called Rep-Tron Systems Group, located in Baltimore, Maryland; and Morocco’s domestic surveillance agency, the DGST.

Other Endace customer lists contained in the leaked trove include the U.S. Army and the U.S. Navy’s Space and Naval Warfare Systems Command, called SPAWAR; the Israeli Ministry of Defense (home of its Unit 8200 electronic spy agency); the government of India, the Spanish Ministry of Defense; and Denmark’s Defense Intelligence Service.

Endace’s apparent dealings with the Moroccan agency, the DGST, are particularly controversial. Moroccan authorities have been persistently accused over more than five decades of committing a range of severe human rights abuses.

In Morocco, digital surveillance is intimately linked with repression of peaceful dissent.

Amnesty International, in a 2015 report, specifically singled out the DGST agency as a key perpetrator of recent abuses, accusing it of detaining people incommunicado and using brutal torture methods that included beatings, electric shocks, sexual violence, simulated drowning, drugging, mock executions, and food and sleep deprivation.

Sirine Rached, Amnesty’s North Africa researcher, told The Intercept that sales of surveillance technology to Morocco raised major concerns.

“In Morocco, digital surveillance is intimately linked with repression of peaceful dissent — people who are peacefully protesting or criticizing the authorities face intimidation, arrest, unfair trials, and sometimes imprisonment,” said Rached. “We fear that the more that these surveillance tools are sold [to Moroccan agencies], the more we will see human rights abuses, especially in relation to freedom of expression and information.”

Endace declined to comment on its dealings with Morocco. Stuart Wilson, Endace’s CEO, claimed in a statement that he had to keep details about the company’s customers confidential in order to help them “battle cyberthreats and breaches.”

An Endace “data acquisition and generation” card, used to monitor networks.

Photo: Endace

Alongside its government clients, Endace has many major corporate customers.

Endace’s sales lists include finance industry giants such as Morgan Stanley, Reuters, and Bank of America. Endace’s website says it provides financial companies with its monitoring technology to help “high-frequency traders to monitor, measure, and analyze critical network environments.”

In addition, Endace sells its equipment to some of the world’s largest telecommunications companies, among them AT&T, AOL, Verizon, Sprint, Cogent Communications, Telstra, Belgacom, Swisscom, Deutsche Telekom, Telena Italy, Vastech South Africa, and France Telecom.

Some of these companies may use the Endace equipment for checking the security of their networks. But a key strand of Endace’s business involves providing technology for telecommunications firms that enables law enforcement and intelligence agencies to intercept the messages and data of phone and internet users.

A company product strategy document from 2010 said that Endace had “seen early success” providing a Lawful Intercept product to the major U.S. telco and internet company Sprint Corporation.

All telcos and internet companies in the U.S., Europe, New Zealand, and a number of other countries are required by law to have “intercept capable” equipment on their networks. When police or spy agencies want private data about a customer (with or without a warrant, depending on the country), it can be extracted easily.

When installed on a network, Endace’s surveillance equipment can be used to perform targeted monitoring of individual people, but it can also be used to enable dragnet spying.

In one of the leaked Endace documents obtained by The Intercept — under a section titled “customer user stories” — the company describes a situation in which a government agency has obtained “the encryption keys for a well-known program.” An Endace surveillance “probe,” the document suggests, could help the government agency “unencrypt all packets sent by this program on a large network in the last 24 hours.”

Once the data has been decrypted, the agency will be able to “look for the text string ‘Domino’s Pizza,’” Endace joked, “as they have information suggesting this is the favorite pizza of international terrorists.”

———
Documents published with this article: