Documents
DEEPDIVE Readme
Jul. 1 2015 — 9:51 a.m.

CDN FIDENTIAL
DEEPDIVE
Configuration Read Me
Dvervlew
The purppse pfthis dpcument is to prcuvide prpcedures tci cpn?gure an HE server as a
server. can be defined as featuring a filter in pfthe traditipnal HE
prpcesspr {hack?end). It is a Federated Eluervr svstem that has a rolling Iauffer pf all unfiltered data
prpcessed lav Dne queryr scans all sites.
has two distinct functipns. The Front End ingests varipus input tvpes leg, .pcap,
.sff, Ethernet, sdh, and packets}, sessipnizes the data and the data tn the Back End. The
hackend can alsp ingest different input types and uses tcucils such as packet_sp atter, sks_xfip [a Fast
sessipnizer), letlDr?lEll), defrag, and sptf_putput.
The Back End strung lie?mail} and [ccuntent] selectian and provides real?time
tipping. It uses GEN ESIS AppidfFingerprints which are updated hpurlv to all accessible field sites. An
appid identifies a specific and details pfa sessicun. Fingerprints flag sessipns that meet speci?c
criteria.
DEEPDIVE Data?flpw
Data packets ?enter? frcint?end, are prpcessed and are fullvr sessicunized hefcire
being passed tn the back?end. The data is then analvzed, prpcessed and released pr stpred as the
mission dictates.
{Cl
DEEPDIVE
H-s
Frcunt Encl Back: End
Pll'lgll-l? Metadata
Packet Dictic-narv . .
upluglns
I
Cuntent
i
I
Packers In: HJHEc-s-vans sh
?x
cuter
Esplatter ::-c:anner
sFip
(Cl
HIKE Deep Dive can be configured at each site depending cm the priprities pf vcuur
missipn.
1
CDN FIDENTIAL

CDN FIDENTIAL
Configuring DEEPDWE for MINUTEMAN in sks.con?g
Use these configurations if your front?end system is outputting SDTF packets only to an HHS
E. If this is the case, then follow these steps to con?gure DEEPDWE for the MINUTEMAN
program only software has been installed:
1.
2.
5.
(U) Logon as the user oper.
(UHFDUDJ At the command line from within any directory, type yi and then press
Enter. The sitscanfia ?le will open.
(UHFDUDJ In the Signalr Acquisition configuration section of sitsconfia, confirm:
a. igoal_aoc[ui itioo_eoa]ole yes By setting this option to yes,
signal_acquisition processes and any associated con?gurations will be added to
pro c_res ou rc es .
b. signal_aom.lisitioo_oo_tnaster yes This createsa
on the Master.
c. haye_protnoter false This indicates no promoteris configured forthe system.
d. splatter_hosts [master_hostname]
In this case, master_hostoatne is the actual hostname ofthe Master seryer. Setting
sp latte c_hosts equal to ma ster_ho stostne indicates that the master is the only
back?end host to receiye the SDTF ?le ie.g., sks?l, sks?l, etc.}.
In the Jisianai acquisition} section of sitsconfia, type:
sigad
froot_eod_ooly False
In this case, the commas separate three options:
I :loase] sigacl :This creates a
process on each host in the HHS cluster and configures each to the U5 SIGAD (HE) that
is carrying the data.
important: Dn each host, do not forget to change masts c_hostoarne to
the appropriate Master seryer hostname.
I Config ignal_aoc1'l_1 isitioo . Sets the configuration ?le to
I front eocl_ooly False Indicates the host will act as both a front?end and a
ba ck?en_d host.
(UHFDUDJ Type :wa and then press Enter to saye and exit sitscanfia. You will now
configure signaLacauisitian. confia.
2
CDN FIDENTIAL

Completing the Con?guration of DEEPDWE for MINUTEMAN
CON FIDENTIAL
To complete the configuration of DEEPDWE for MINUTEMAN, be sure to configure
sign ol_ ocouis in on. con fig
1. At the command lino from within any directory, type oa? oonfig and then press
Enter. This will take you to
2. Dpen signoLocouisitioncon?g, or create a filo by that name if it does not already
exist. This ?lo will be used to configure soyeral front?end processes for ingesting, sossionizing,
and roassombling data. Each process is described in tho following table.
Front-End Processes
What h?s Called What ft Does What It Means
Ingests packets (from ?les, from If it?s a packet stream, it can
Pocket Splotter the network, from a capture probably be fed into a
card} in a yarioty of formats. DEEPDWE.
EH Fast reassembly of and Efjp?i?: 5:332:53km a
uoryleoo streams??. . .
keopfdrop decIsIon.
Heassembly of streams from less
common protocol stacks.
f'lter'i'g ?f intelligently
reassembled sessIons, based on
Promoter ch oosos the most useful
keyword, country code or . .
. . . traffic for retention.
Enough content ayailable to
Dofrog Fully rebuilds sessions? do full decodingfdocument
descent at the Back End
*up too 255i: limit
?up to a 113MB limit
Note: In this Read Me, we will not address the Promoter.
3. In the signoLocouisition.config, typefedit the following configurations for the
processes identified in step 2:
El.
??otato_topio po_otato ?y ?o 4
,ioCritioal=Truo,aoRoot=Truo
?i ?t ootf
Kfip, ?f
COIN FIDEHTIAL
3

CDN FIDENTIAL
c. Mettle_tomalloo, ?f
d. ?o 5D40,oount=4
(UHFDUDJ Type :wa and then press Enter to saye and exit signoLoco-uisition.config.
(UHFDUDJ Perform the following commands only after making changes to both
sign oLocouisitioncon?g and skscon?g:
(UHFDUDJ At the command prompt, type aka setup pro ceases and then press Enter.
This will create signal_acquisition_base on each host in the cluster.
(UHFDUDJ At the command prompt, type aka proo start and then press Enter. This will
start the newly created processes.
rumoqu Configuring DEEPDWE for
Use these configurations if your front?end system is a and outputting
packets, packet bundles, and sessions to an HHS DEEPDIVE. lfthis is the case, then follow these steps to
configure DEEPDWE for
1.
2.
(U) Logon as the user oper.
(UHFDUG) At the command line from within any directory, type vi oonfig and then press
Enter. The xkscon?g file will open.
(UHFDUD) In the #[signoi {Inquisition} section of set the following configurations:
a. a ignal_aocpji a ition_ena]ole yes By setting this option to ye a,
aignal_aomjisition processes and associated configurations will be added to
proo_resouroes.
b. aignal_aomjisition_on_1naster no This willnot createa
aignal_aoc1uiaition_]oaae on the Master.
c. naye_promoter fal se This indicates no promoter is configured for the system.
d.
4
CDN FIDENTIAL

CDN FIDENTIAL
In the case, the comma separates two options:
I oaeenotation=c1ynarnio This configures the multiple
process on all the hosts in the cluster
I Confioi This setsthe
configuration file to
important: If it does net already exist, you must create and configure
gene ri o_pao]cet_to_buncile . oonfig. See laelow, Configuring
for configuration insructions.
4. (UHFDUD) Type :wa and then press Enter to saye and exit skscon?g.
Configuring generic _poci:et_ bundiecon?'g
Configuring DEEPDWE for FURNSAT also requires that you setup the file:
1. (UHFDUDJ At the command line from within any directory, type ea? oonfig and then press
Enter. This will take you to
2. (UHFDUDJ ?pen generic_pocket_bundie.config, or create a ?le by that name if it does not
already exist.
3. (UHFDUDJ In the genericjockeLbuc-die.config, typefedit the following configurations for the
processes identified in step 2:
a. SUBS ?o ?e
b. KkS_Efip, ?f
c.
d. EDBQ ?o
4. (UHFDUDJ Type :wa and then press Enter to saye and exit generic_pocket_bmdie.config.
5
CDN FIDENTIAL

CDN FIDENTIAL
Additional Processes
Run these additional processes only after making changes to the configurations in
sits. con
1. (UHFDUDJ At the command prompt, type eke re yne puen_eonfig and press Enter.
This sets pushes configuration changes out to the slayes.
2. (UHFDUDJ At the command prompt, type eke eetup pro ee ee and press Enter. This
creates the eignal_aec1uie ition_]oaee process.
3. (UHFDUDJ At the command prompt, type aka proe etart and press Enter . This will
ensure all of the running processes pick up any configuration changes.
5
CDN FIDENTIAL