For the past year and a half, the FBI has been using a secret scoring system to judge the likelihood that someone will carry out a violent attack. The survey, called “Indicators of Mobilization to Violence,” assigns points based on factors like religious observance, travel history, financial transactions, and physical appearance.

A copy of the survey in a classified FBI document was obtained by The Intercept.

The survey itself consists of 48 questions that agents are supposed to answer about terror suspects as part of an investigation. The document also includes a set of frequently asked questions about the scoring system aimed at agents working counterterrorism investigations.

According to the document, the program was rolled out in the fall of 2015.

Many of the questions in the survey have obvious relevance to whether someone is plotting a violent act, such as whether the person has access to weapons, has trained or fought overseas, or has tried to acquire bomb-making materials.

But some of the activities the survey flags are more ordinary. They include using encryption or masking internet browsing, showing an interest in previous bombings, or “participating in activities that simulate military or operational environments (e.g. paint ball, airsoft, laser tag, shooting ranges, camping/survival trips, etc.).”

Other questions are far more subjective and deal with the individual’s emotional state. For example, the survey asks if the person has changed their appearance or habits, has “experienced a recent personal loss or humiliation,” or has a history of mental health problems or substance abuse.

Still other questions gauge ideological leanings. “Does the Subject hold a belief or ideology that supports the use of violence?” Or, “Has the Subject become more extreme in their beliefs?”

While the survey is, in theory, meant to be universal, it contains several references to Islamic terminology and seems to be focused on concerns about Muslim terrorism.

Sahar Aziz, a law professor at Texas A&M University and expert on programs aimed at countering violent extremism, told The Intercept that some of the questions are subject to bias by FBI agents.

“Any effort to create ideological prototypes of people that are presumed to be susceptible to violence can have a chilling effect on constitutionally protected free speech,” Aziz said. “When government agents are screening people based on ideological beliefs or lifestyles that are not directly associated with criminal activity, it casts a wide net, and that can create serious consequences for targeted communities.”

Trying to understand the factors behind terrorism is understandable, but the FBI survey illustrates the perils of trying to use past examples to predict and prevent violent attacks.

“Set the civil liberties issues aside for a moment, and this says that we are grasping at straws to try to figure out what the factors involved in radicalization are,” said Karen Greenberg, the director of Fordham Law School’s Center on National Security. “They are so individualized. How much will algorithms really tell us about the threat?”

The FBI in recent years has taken an aggressive stance in sting operations designed to catch potential terrorists. These and other government efforts under the banner of “countering violent extremism” have been criticized by lawyers and activists who say law enforcement is looking for terror threats in the wrong places, painting Islamic and immigrant communities as uniquely susceptible to violence. The Trump administration has indicated it will make this direction explicit, changing the name of the program to specify Islamic extremism and dropping efforts aimed at other threats such as those from white supremacist groups.

0-100

The FBI document explains that agents working counterterrorism cases are now prompted to fill in the survey through the FBI’s case management system, and to keep their answers updated as they learn new information.

Based on the answers that agents put in, the survey generates several scores on a scale of 0-100, which show “the subject’s level of mobilization or likelihood of carrying out a violent act,” and “the subject’s likely level of radicalization or internal commitment to violent ideology.”

The score is calculated via a “statistical methodology” — not described in detail — that compares the case with historical counterterrorism subjects, “both those that mobilized to violence and those that did not.”

The survey stresses several times that high scores “do not provide conclusive evidence that a subject will take violent action,” and that they should be used for comparison only.

However, it also says that the score could be used to “assist decision-makers with determining where to focus available resources.” The document does not further explain how the survey would be used by law enforcement, and if it could lead to escalating an investigation.

Reached for comment, the FBI did not answer specific questions about the document but said that “these surveys are but one part of our internal assessment process in counterterrorism cases.” 

The document notes that the FBI doesn’t consider the score “discoverable” at trial, because “it is derived from evidence that already exists within a court.” Defense lawyers interviewed by The Intercept said that it is up to judges to decide whether something must be provided to the defense in criminal proceedings. If the score could somehow be considered exculpatory, then it may well have to be turned over, said Joshua Dratel, a prominent national security lawyer.

Dratel and other lawyers interviewed by The Intercept were unaware of the FBI scoring system and speculated that, because of its recent implementation, it may not yet be a factor in terror prosecutions currently making their way through the courts.

The scoring system is the product of the Americas Fusion Cell, part of the counterterrorism division at FBI headquarters. The algorithm appears to have grown out of an analytic effort to survey agents about their casework in order to gain insights into radicalization. The Intercept previously reported on one such survey, from 2012, which found that U.S. military operations were the most commonly cited motivation for “homegrown” violent extremists.

The document notes that anonymized data from the survey will be shared with intelligence partners in the United Kingdom, Canada, Australia, and New Zealand (the so-called “Five Eyes Alliance”) and that those countries would share results from their own, similar initiatives.

The document states that over time, “the dataset will become larger and the weighting of questions will become more accurate.”

Others are less confident about the possibility of singling out factors that lead to radicalization. A recent study published by the National Consortium for the Study of Terrorism and Responses to Terrorism looked at psychological, emotional, and material factors at work in the cases of about 1,400 individuals it identified as “violent and non-violent extremists,” both Islamic and from the far right and left.

Using statistical methodology, the report found that having a criminal history and belonging to a small, close-knit “clique” of other extremists were the factors most closely associated with turning violent. But it concluded that there were “myriad causes of radicalization.”

Michael Jensen, one of the report’s authors, told The Intercept that the likelihood of developing universal criteria for predicting extremist violence was slim, and that efforts to operationalize such models today would probably have poor results. “The factors that come out over and over again as risk factors for radicalization are things that are present in the lives of millions of people who will never radicalize,” Jensen said. “There’s just not going to be a simple checklist of warning signs for future violence.”

Wadie Said, a law professor at the University of South Carolina and author of the book “Crimes of Terror,” about federal terrorism prosecutions, was even more negative on the prospect of such indicators.

“The idea of protecting people from violence is understandable. But how do you do that without falling into a trap of stereotypes and assumptions?” he said. “The idea that radicalization can be rendered into bite-size scores and equations under a pseudo-scientific name is deeply problematic.”

Top photo: An aide to Attorney General John Ashcroft holds a binder that outlines new guidelines for FBI agents investigating terrorist activity in the United States.