DYNAMIC PAGE -- HIGHEST POSSIBLE CLASSIFICATION IS
TOP SECRET // SI / TK // REL TO USA AUS CAN GBR NZL
(U) My Comment
FROM: Cindy Farkus
NSA/CSS Senior Intelligence Authority and Assistant Deputy Director for
Analysis & Production (S2)
Run Date: 06/08/2006
(U//FOUO) Cindy Farkus addresses analysts' concerns regarding the inclusion of COMMENTs in
SIGINT reports:
(U//FOUO) I have been closely following the initiative
is leading to include more
insights into the SIGINT reporting. (See yesterday's article .) The intelligence community and
NSA will have to be more proactive in providing facts and insights to protect the nation today
and for the future. In fact, I believe that including more insights into reporting and providing
more assessments from our analyst experts is one of the most important things we can do to
improve and better serve our customers.
(U//FOUO) It is so important, that it is one of the very first things outlined in the A&P strategy
that we will roll out in the next few weeks. In the meantime, I thought I would provide some
comments of my own to issues raised from the recent SID today articles about COMMENTs.
(U//FOUO) Issue: Measuring success by "counting comments" will lead to a rash of empty,
meaningless COMMENTS that worsen the quality of our reports.
(U//FOUO) Cindy's COMMENT:
Absolutely! Let me be clear: S2 will NOT measure its success or progress by the number of
COMMENTs that are included in our product reports. I agree that doing so would be a disaster. It
would be like pretending to carry an umbrella in a rainstorm - it might make you feel better, but
you are still going to get wet. The true measure of our success is the degree to which we enable
our nation's leaders and warfighters to make decisions that lead to positive outcomes for the
nation. Sometimes that will entail reporting "only the facts." But in most cases, we can better
enable those decisions by including insight, perspective, context, and the "so what" factor by
including COMMENTs, collateral, or constructing our reports so that the implications of the data
are readily apparent.
(U//FOUO) Issue: Reality is that some customers want comments and some don't.
(U//FOUO) Cindy's Comment:
In general, this is indeed true. Certainly the Commands tend to want comments and insights.
After all, these are military leaders who are used to hearing many inputs and making decisions
based on them, even when some conflict with each other. Other customers - Department of
State and Commerce come to mind - seem more interested in just the unique information we
may have access to. Regardless, it is incumbent on us -- you, your team, and your management
-- to know who wants what and to check frequently that we have it right.
(U//FOUO) Ultimately, we are trying to develop a dialogue with the customer by providing
comments with some insight as to why the report answers their need. As I see it, we will follow a
continuum where well-written reports with "so what" facts begin to be complemented
consistently with our contextual insights. This raises the bar for everyone, and customers will
demand that more insights and expertise be shared through reporting vehicles like assessment
reports. Facts will always be part of reporting, but reports featuring expertise, like assessment
reports, are still underdeveloped.
will be working the details of this issue with the
division chiefs and the specific answer may vary by target.
(U//FOUO) Issue: What happens when analysts are wrong? How do we manage the damage to
NSA's credibility?
(U//FOUO) Cindy's Comment:
I encourage every analyst to be confident enough in his or her expertise and tradecraft to
include appropriately marked insights in reports. NSA offers more than facts, we offer SIGINT
expertise in our target areas. Our reputation will not be hurt if analysts provide appropriately
marked insights; in fact, it can only be enhanced. There will be times when legitimate
differences of opinion exist, and that's ok. Remember, the 9-11 commission report encourages
us to develop and share differing points of view.
(U//FOUO) There will also be times when people are wrong. But we have to take the long view.
Those mistakes will prompt a very good discussion and many people will learn from that
situation. Everyone makes mistakes; we want to be viewed as an organization that accepts risks
but also one that deals with problems once they are recognized. This approach can only enhance
our credibility in the long run.
(U//FOUO) Issue: What about junior, less experienced analysts providing erroneous "insight"
based on partial knowledge?
(U//FOUO) Cindy's Comment:
One thing we must keep in mind is the need to convey how sure we are of the insights contained
in the COMMENTs. So, for example, junior personnel who don't have deep target knowledge
may want to use qualifiers ("possible" or "probable") to convey a level of uncertainty about the
conclusions reached. As noted in
's last column , Colin Powell once said, " Tell me
what you know, tell me what you don't know, tell me what you think; always distinguish which
is which. "
(U//FOUO) Furthermore, over time, any gaps in experience among our analysts can be filled
through the process of working with editors and collaborating with the customer. We all learn
from this dialogue. We also need to realize that while a great deal of experience certainly
provides a "mature" perspective, those with less experience bring a fresh look and often pull us
in the right direction. We hire, train, and employ very smart people and it is often said that they
are NSA's best assets; lets begin to value the diversity of experience.
(U//FOUO) Issue: How will NSA measure success in a way that ensures a value-added product
if we begin to report more than the facts? Will P3's be changed to reflect and reward individuals
who do this?
(U//FOUO) Cindy's Comment:
Providing more insights from our SIGINT target experts is critical to our mission. Making generic
changes will not positively affect the quality of our products - no one wants quality to suffer or
time wasted on "checking a box." The magic formula for how many well-written reports, reports
with comments, assessment reports, etc. does not exist - nor will it. A cookie-cutter approach
will not work for all customers or targets. However, everyone recognizes that changes need to
be made to the performance evaluation tools in order to effect permanent change. We need to
be flexible as we consider how we measure this effort on a group and individual level. As we
begin to form recommendations, we will share them with you. If you have any suggestions in the
meantime, please send them to
.
(U//FOUO) Feedback: Will training be developed to support the changes you are
recommending?
(U//FOUO) Cindy's Comment:
This effort (to increase the level of commentary in our reporting) is being coordinated with the
training folks, the Reporting Board and a team of S2 senior analysts, so any issues or
opportunities can be integrated into training.
(U//FOUO) Feedback: Will the entire process be reviewed with an eye toward changing those
editing procedures and policies that hinder the sharing of insights?
(U//FOUO) Cindy's Comment:
We are looking at all of the components to the process of reporting so that more insights are
included in what we provide to customers. This is a big job and critical to the value we add to
the Intelligence Community. Not only must we deliver the SIGINT facts, but our customers also
demand our expertise. This is one of the reasons I asked you to include more insights and
comments into reports over the next few months - it will help us uncover the strengths and
weaknesses of the process and the customer feedback loop. Like anything, change will not occur
overnight and we will keep apprised of what we learn.
"(U//FOUO) SIDtoday articles may not be republished or reposted outside NSANet
without the consent of S0121 (DL sid_comms)."
DYNAMIC PAGE -- HIGHEST POSSIBLE CLASSIFICATION IS
TOP SECRET // SI / TK // REL TO USA AUS CAN GBR NZL
DERIVED FROM: NSA/CSSM 1-52, DATED 08 JAN 2007 DECLASSIFY ON: 20320108