Documents
Next Generation Events
Sep. 25, 2015
TOP SECRET STRAP 1
Next Generation Events
TOP SECRET STRAP 1
23 March 2009
TOP SECRET STRAP 1
Next Generation Events
TOP SECRET STRAP 1
23 March 2009
TOP SECRET STRAP 1
What is NGE?
Systems like HAUSTORIUM reaching ingest capacity
–
But scale and variety both increasing
5-Eyes also far-apart on “metadata” requirements, need to get closer together
The Answer?
NGE: A multi-stage project that tackles a series of the problems, at increasing scale, and with
increasing collaboration
CLASSIFCATION
29 September 2009
"This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure
requests to GCHQ on 01242 221491 x30306 (non-sec) or email infoleg@gchq."
TOP SECRET STRAP 1
What is NGE?
Systems like HAUSTORIUM reaching ingest capacity
–
But scale and variety both increasing
5-Eyes also far-apart on “metadata” requirements, need to get closer together
The Answer?
NGE: A multi-stage project that tackles a series of the problems, at increasing scale, and with
increasing collaboration
CLASSIFCATION
29 September 2009
"This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure
requests to GCHQ on 01242 221491 x30306 (non-sec) or email infoleg@gchq."
TOP SECRET STRAP 1
Next Gen Events: High-level Plan
53 x 10G
New technologies (particularly from JCE) incorporated into solution as they are de-risked / proven
CLASSIFCATION
29 September 2009
We Are
Hereis exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure
"This information
requests to GCHQ on 01242 221491 x30306 (non-sec) or email infoleg@gchq."
TOP SECRET STRAP 1
Next Gen Events: High-level Plan
53 x 10G
New technologies (particularly from JCE) incorporated into solution as they are de-risked / proven
CLASSIFCATION
29 September 2009
We Are
Hereis exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure
"This information
requests to GCHQ on 01242 221491 x30306 (non-sec) or email infoleg@gchq."
TOP SECRET STRAP 1
NGE: The Last Three Months
Sharing Enriched Metadata (HARBOUR PILOT)
–
Moving towards metadata standards across 5-Eyes
–
Invisible to GCHQ analysts
Internet Profiling (BLAZING SADDLES)
–
Taking ICTR ideas on how to process Events at scale, and scale even more
–
Required significant effort on End-to-End Sigint process
CLASSIFCATION
29 September 2009
"This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure
requests to GCHQ on 01242 221491 x30306 (non-sec) or email infoleg@gchq."
TOP SECRET STRAP 1
NGE: The Last Three Months
Sharing Enriched Metadata (HARBOUR PILOT)
–
Moving towards metadata standards across 5-Eyes
–
Invisible to GCHQ analysts
Internet Profiling (BLAZING SADDLES)
–
Taking ICTR ideas on how to process Events at scale, and scale even more
–
Required significant effort on End-to-End Sigint process
CLASSIFCATION
29 September 2009
"This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure
requests to GCHQ on 01242 221491 x30306 (non-sec) or email infoleg@gchq."
TOP SECRET STRAP 1
Plug 1 - Internet Profiling: The BLAZING SADDLES Delivery
What It Does:
–
Takes 8 ICTR QFD’s and scales them for up to 100 x 10G bearers
–
Allows the analyst to see large amounts of a targets online activity
–
Metadata – MUTANT BROTH, AUTO ASSOC, KARMA POLICE, SOCIAL ANIMAL, INFINITE
MONKEYS, HRMAP
–
Content – MEMORY HOLE, MARBLED GECKO
Why You Care:
–
Want to know alternate online accounts?
–
Quickly build up a picture of someone’s online MO and interests?
–
Identify for further exploitation (with other techniques) a targets network/machines?
–
Success across IP/X – CP, SIMMER, Mumbai, G20 – and ask around in your IPT!
How You Get Access:
–
Currently instigating corporate process (based on C2C skill level)
–
Interim – see your Tech Director or Tech Ex
CLASSIFCATION
29 September 2009
"This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure
requests to GCHQ on 01242 221491 x30306 (non-sec) or email infoleg@gchq."
TOP SECRET STRAP 1
Plug 1 - Internet Profiling: The BLAZING SADDLES Delivery
What It Does:
–
Takes 8 ICTR QFD’s and scales them for up to 100 x 10G bearers
–
Allows the analyst to see large amounts of a targets online activity
–
Metadata – MUTANT BROTH, AUTO ASSOC, KARMA POLICE, SOCIAL ANIMAL, INFINITE
MONKEYS, HRMAP
–
Content – MEMORY HOLE, MARBLED GECKO
Why You Care:
–
Want to know alternate online accounts?
–
Quickly build up a picture of someone’s online MO and interests?
–
Identify for further exploitation (with other techniques) a targets network/machines?
–
Success across IP/X – CP, SIMMER, Mumbai, G20 – and ask around in your IPT!
How You Get Access:
–
Currently instigating corporate process (based on C2C skill level)
–
Interim – see your Tech Director or Tech Ex
CLASSIFCATION
29 September 2009
"This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure
requests to GCHQ on 01242 221491 x30306 (non-sec) or email infoleg@gchq."
TOP SECRET STRAP 1
NGE: The Next Three Months
ROCK RIDGE
–
Continuing QFD roll-out
• SAMUEL PEPYS
• CAFFEINE HIT
–
Sharing some QFD’s with (initially) NSA
Converged Events
–
Ensuring we don’t perpetuate the C2C/Telephony divide
–
Specific QFD’s that enhance our ability to exploit converged
• Evolved MUTANT BROTH
• LAUGHING HYENA
–
Exit strategy for SALAMANCA/HAUSTORIUM
CLOUD Experiments at Bude
–
JCE and TINT
• Developing/testing technologies for later in the roadmap
ICTR (and others!) continue to develop new ideas
CLASSIFCATION
29 September 2009
"This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure
requests to GCHQ on 01242 221491 x30306 (non-sec) or email infoleg@gchq."
TOP SECRET STRAP 1
NGE: The Next Three Months
ROCK RIDGE
–
Continuing QFD roll-out
• SAMUEL PEPYS
• CAFFEINE HIT
–
Sharing some QFD’s with (initially) NSA
Converged Events
–
Ensuring we don’t perpetuate the C2C/Telephony divide
–
Specific QFD’s that enhance our ability to exploit converged
• Evolved MUTANT BROTH
• LAUGHING HYENA
–
Exit strategy for SALAMANCA/HAUSTORIUM
CLOUD Experiments at Bude
–
JCE and TINT
• Developing/testing technologies for later in the roadmap
ICTR (and others!) continue to develop new ideas
CLASSIFCATION
29 September 2009
"This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure
requests to GCHQ on 01242 221491 x30306 (non-sec) or email infoleg@gchq."
TOP SECRET STRAP 1
NGE: And After That?
Capability Development Workspace
–
Bulk datamining capability
–
Use existing sources, and new cloud capabilities
Large-scale contact chaining
–
MOAG – but anyone can create
–
Using both GCHQ and NSA datastores
MO/Profiling based discovery
–
Always been the goal for events-led analysis
–
Dependent on technological advancements, but looking good
Events/Content Fusion & Visualisation
–
Seamless navigation between Events and Content
–
Making sure we continue the MONTE VISTA/LOOKING GLASS ideas
CLASSIFCATION
29 September 2009
"This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure
requests to GCHQ on 01242 221491 x30306 (non-sec) or email infoleg@gchq."
TOP SECRET STRAP 1
NGE: And After That?
Capability Development Workspace
–
Bulk datamining capability
–
Use existing sources, and new cloud capabilities
Large-scale contact chaining
–
MOAG – but anyone can create
–
Using both GCHQ and NSA datastores
MO/Profiling based discovery
–
Always been the goal for events-led analysis
–
Dependent on technological advancements, but looking good
Events/Content Fusion & Visualisation
–
Seamless navigation between Events and Content
–
Making sure we continue the MONTE VISTA/LOOKING GLASS ideas
CLASSIFCATION
29 September 2009
"This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure
requests to GCHQ on 01242 221491 x30306 (non-sec) or email infoleg@gchq."
TOP SECRET STRAP 1
Next Generation Content..?
Not yet…but thinking and delivery is happening
–
TIPC using TDI’s
–
Expand XKS use
–
Trial new ways of collecting/processing content (TINT)
CLASSIFCATION
29 September 2009
"This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure
requests to GCHQ on 01242 221491 x30306 (non-sec) or email infoleg@gchq."
TOP SECRET STRAP 1
Next Generation Content..?
Not yet…but thinking and delivery is happening
–
TIPC using TDI’s
–
Expand XKS use
–
Trial new ways of collecting/processing content (TINT)
CLASSIFCATION
29 September 2009
"This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure
requests to GCHQ on 01242 221491 x30306 (non-sec) or email infoleg@gchq."
TOP SECRET STRAP 1
Plug 2: TIPC Expansion
What It Does
–
Full client IP stream collection triggered by known selector
–
Expanded to STM-64 environment as well as STM-1/4
–
Now triggered by TDI’s, not just gmail, yahoo and maktoob
Why You Care
–
Unique Intelligence material that can’t be strong selected – web visits/searches etc
–
Find new protocols used by targets – you, tech trends, T development
–
Contextless
–
New dictionary – old one completely erased
How Do You Get Access?
–
Talk to your C2C Tech Ex – they are running pre-requisite briefings as there are some
dangers…(full IIB!)
CLASSIFCATION
29 September 2009
"This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure
requests to GCHQ on 01242 221491 x30306 (non-sec) or email infoleg@gchq."
TOP SECRET STRAP 1
Plug 2: TIPC Expansion
What It Does
–
Full client IP stream collection triggered by known selector
–
Expanded to STM-64 environment as well as STM-1/4
–
Now triggered by TDI’s, not just gmail, yahoo and maktoob
Why You Care
–
Unique Intelligence material that can’t be strong selected – web visits/searches etc
–
Find new protocols used by targets – you, tech trends, T development
–
Contextless
–
New dictionary – old one completely erased
How Do You Get Access?
–
Talk to your C2C Tech Ex – they are running pre-requisite briefings as there are some
dangers…(full IIB!)
CLASSIFCATION
29 September 2009
"This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure
requests to GCHQ on 01242 221491 x30306 (non-sec) or email infoleg@gchq."
TOP SECRET STRAP 1
Plug 3: XKS & TINT @ Bude Experiments
What It Will Do:
–
Promotion from XKS to IIB
–
Integration into LOOKING GLASS
–
Connection to Native File Viewer (FUME CUPBOARD)
–
Continuing to work on the NSA data access issue
–
The TINT@Bude Experiments Attempt To:
•
Re-sessionise everything
•
Tag traffic, based on
•strong selector/ geography/ application
•contextual fingerprints:
•
•
Extract metadata in bulk
Retain a 3-day rolling buffer of ‘interesting’ content
•for retrospective/protocol/network/analysis
•for refining fingerprints/selectors
•
Do this on 20 x 10G’s!
Why You Care:
–
Packet processing approach misses stuff
–
Strong selection only
–
Too much data retained is unused (97% unviewed)
–
Promote only the good stuff to long-term storage
–
Aim: to automatically promote to long term storage
When Do You Get Access?
–
New XKS capabilities will be rolled out to GCHQ KS’s when available
–
TINT PUT in place, but experimental, not operational use only
CLASSIFCATION
29 September 2009
"This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure
requests to GCHQ on 01242 221491 x30306 (non-sec) or email infoleg@gchq."
TOP SECRET STRAP 1
Plug 3: XKS & TINT @ Bude Experiments
What It Will Do:
–
Promotion from XKS to IIB
–
Integration into LOOKING GLASS
–
Connection to Native File Viewer (FUME CUPBOARD)
–
Continuing to work on the NSA data access issue
–
The TINT@Bude Experiments Attempt To:
•
Re-sessionise everything
•
Tag traffic, based on
•strong selector/ geography/ application
•contextual fingerprints:
•
•
Extract metadata in bulk
Retain a 3-day rolling buffer of ‘interesting’ content
•for retrospective/protocol/network/analysis
•for refining fingerprints/selectors
•
Do this on 20 x 10G’s!
Why You Care:
–
Packet processing approach misses stuff
–
Strong selection only
–
Too much data retained is unused (97% unviewed)
–
Promote only the good stuff to long-term storage
–
Aim: to automatically promote to long term storage
When Do You Get Access?
–
New XKS capabilities will be rolled out to GCHQ KS’s when available
–
TINT PUT in place, but experimental, not operational use only
CLASSIFCATION
29 September 2009
"This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure
requests to GCHQ on 01242 221491 x30306 (non-sec) or email infoleg@gchq."