General security

XKeyscore: NSA’s Surveillance Program

Introduction

The former contractor for NSA, Edward Snowden, became famous for revealing PRISM, a confidential mass surveillance program run by the U.S. agencies to eavesdrop on any electronic media. The whole world realized that Big Brother is real and, yes, he is watching you. It is not a fiction any longer.

So what is XKeyscore, exactly?

It's a tool that can scour all things from the Internet for surveillance purposes. The purpose of XKeyscore is to allow analysts to search the metadata as well as the contents of email and other Internet activity, such as browser history. Analysts can also search by name, telephone number, IP address, keywords, and the language in which the Internet activity was conducted or the type of browser used.

According to the slides published, XKeyscore is:

DNI (digital network intelligence) exploitation system/analytic framework
Performs strong (e.g., email) and soft (e.g., content) selection. In an interview in June, Edward Snowden elaborated on his statement about being able to read any individual's email if he had their email address. He said the claim was based in part on the email search capabilities of XKeyscore, which Snowden says he was authorized to use while working as a Booz Allen contractor for the NSA.

One of the top-secret documents describe how the program searches within the "bodies of emails, web pages and documents," including the "To, From, CC, BCC lines" and "Contact-Us" pages on websites.
Provides real-time activity

Beyond emails, the XKeyscore system allows analysts to monitor a virtually unlimited array of other Internet activities, including those within social media. The DNI presenter enables an analyst using this tool to read the content of Facebook chats or private messages.
"Rolling buffer" of ~3 days of ALL unfiltered data seen by XKeyscore:
Stores full-take data at the collection site, indexed by meta-data
Provides a series of viewers for common data types
Federated query system—one query scans all sites

Performs full-take, allowing analysts to find targets that were previously unknown by mining metadata.

Revealing

When and Where?

The program's existence was most publicly revealed in July 2013 by Edward Snowden in The Sydney Morning Herald and O Globo newspapers.

What Was Snowden's Statement?

"I, sitting at my desk, (can) wiretap anyone, from you or your accountant, to a federal judge or even the president, if I had a personal email address." The Guardian (June 10).

Legal vs. Technical Restrictions

The FISA Amendments Act (2008) requires a warrant for targeting U.S. individuals, but NSA analysts are permitted to intercept the communications of such individuals without a warrant if they are in contact with one of the NSA's foreign targets.

The ACLU's deputy legal director, Jameel Jaffer, told The Guardian that national security officials expressly said that a primary purpose of the new law was to enable them to collect large amounts of communications by Americans without individual warrants.

The following example is provided by one XKeyscore document showing a NSA target in Tehran communicating with people in Frankfurt, Amsterdam, and New York.

The working

What can it do?

According to The Washington Post and Marc Ambinder, editor of The Week, XKeyscore is a data retrieval system that consists of a series of interfaces, backend databases, software, and servers that select certain type of metadata that NSA has already collected, using other methods from wide range of different sources.

F6, which is the SCS or special collection service, operating from a U.S embassy or consulate overseas.
FORNSAT, which means "foreign satellite collection," refers to intercepts from satellites that process data used by other countries.
The special source operations, or the SSO, is a branch of NSA that taps cables, finds microwave paths, and otherwise collects data not generated by F6 or foreign satellites.

A training slide on page 24 of the National Security Agency's 2008 presentation on the program, states it quite baldly:

Show me all exploitable machines in country X
- Fingerprints from TAO (tailored access operations) are loaded into XKeyscore's application/fingerprinted engine.
- Data is tagged and databased.
- No strong-selector.
- Complex Boolean tasking and regular expression required.

According to Ars Technica's Sean Gallagher, "the vulnerability fingerprints are added to serve as a filtering criteria for XKeyscore's app engines, comprised of a worldwide distributed cluster of Linux servers attached to the NSA's Internet backbone tap points."

He explains how this could give the NSA a toehold of surveillance in various countries:

"This could allow the NSA to search broadly for systems within countries such as China or Iran by watching for the network traffic that comes from them through national firewalls, at which point the NSA could exploit those machines to have a presence within those networks."

The slides explain how XKeyscore can track encrypted virtual private networks (VPN) sessions and their participants, can capture metadata on who's using PGP (pretty good privacy) in email or who is encrypting their Word documents, which later can be decrypted.

It keeps all trapped Internet traffic for three days, but the metadata is kept for up to 30 days. This one-month duration allows the authorities the time to trace the identity of those who created the documents.

Further capabilities include Google Maps and web searches. It also has the ability to track the authorship and source of a particular document.

Location

Where is XKeyscore located?

It works with the help of over 700 servers based in U.S. and allied military and other facilities as well as U.S. embassies and consulates in several dozen countries.

Data Collection and Storage

How much data is being collected?

The quantity of communications accessible through programs such as Xkeyscore is staggeringly large. One NSA report from 2007 estimated that there were 850 billion "call events" collected and stored in the NSA database and close to 150 billion Internet records. Each day, the document says, 1 – 2 billion records were added.

A 2010 Washington Post article reported that "everyday, collection systems at the [NSA] intercept and store 1.7 billion emails, phone records and other type of communications."

How long is the data stored?

The XKeyscore system is collecting a vast amount of Internet data that can be stored only for short periods of time. Content remains on the system for three to five days. The metadata on the other hand is stored for 30 days.

One document explains: "At some sites, the amount of data we receive per day (20 plus terabytes) can be only stored for as little as 24 hours."

In 2012, there were at least 41 billion records collected and stored in XKeyscore for a single 30 days of duration.

What was NSA's response?

In a statement to The Guardian, the NSA said: "NSA's activities are focused and specifically deployed against—and only against—legitimate foreign intelligence targets in response to requirements that our leaders need for information necessary to protect our nation and its interests.

"XKeyscore is used as a part of NSA's lawful foreign signals intelligence collection system.

"Allegations of widespread, unchecked analyst access to NSA collection data are simply not true. Access to XKeyscore, as well as all of NSA's analytic tools, is limited to only those personnel who require access for their assigned tasks … In addition, there are multiple technical, manual, and supervisory checks and balances within the system to prevent deliberate misuse from occurring."

"Every search by an NSA analyst is fully auditable, to ensure that they are proper and within the law.

"These types of programs allow us to collect the information that enables us to perform our missions successfully—to defend the nation and to protect U.S. and allied troops abroad."

What's more?

Many more aspects of the XKeyscore are not revealed in depth, such as with which countries the U.S. shares information, how the program contends with encrypted contents, the evolving of the program in recent years.

Despite all this, the government claims some real benefits, such as 300 terrorists captured by 2008.

What should you learn next?

From SOC Analyst to Secure Coder to Security Manager — our team of experts has 12 free training plans to help you hit your goals. Get your free copy now.

Get Your Plan

References:

Posted: September 9, 2013

Bhavesh Naik

View Profile

Bhavesh is a Certified Ethical Hacker and Security + certified. Currently, he is pursuing Final year BSc in Information Technology. The upcoming goal for him is CISSP. He is interested in cyber forensics, pentesting, information security. He wishes to do his Masters in Information Security in upcoming years.

XKeyscore: NSA’s Surveillance Program

Get certified and advance your career