Documents
Full Log vs HTTP
Jul. 1 2015 — 9:51 a.m.
TCIP T0 USA, AUS, CAN, GER, NZLH20291123
11 June 2009
-
TOP SECRETHCOMINTHREL TO USA, nus, CAN, GER, NZLH20291123
TU USA, E. @339 Ml.
.Full Log - Pros f?i
The Full Log search gives you access to a_H
DNI sessions collected by X-KEYSCORE
- Data is indexed by the basic meta-data like
IP Address, Country Codes Port,
Casenotation, Application lD/Fingerprints
etc
If you?re only interested in content, Full Log
will give you access to everything
TDP SEGHETHCUMIHTHRELTD USA, AUS, CAN, GER, NZL
To USA, Aus, ?rm. GEL
Co
I However, in most cases there will be too
many results in XKS to look through every
piece of content by hand
To be more efficient, it?s important to utilize
the meta-data contained in the other search
forms (E-mail Addresses, HTTP Activity,
Extracted Files, Document Meta-data etc.)
TDP SEGHETHCUMIHTHRELTD USA, AUS, CAN, GER, NZL
To USA, Age, ?rm, GEL
Activity
. HTTP Activity is essentially all web-based
activity from a user?s internet browser (with
some exceptions)
It includes, web-surfing, Internet Searching
(like Google), Mapping Website (Google
Earth/Maps) etc.
Most of this data will not contain a strong
selector like E-mail address
TDP SEGHETHCUMIHTHRELTD USA, AUS, CAN, GER, NZL
SECRETHCUMINTHHEL TU USA, AUS, CAN, GER, NZL
HTTP Activity
HTTP activity comes in two types:
cnn.cem Server
Client-te-Server
?requests?
Server-te-Client
?responses?
TDP SEGHETHCUMIHTHRELTD USA, AUS, CAN, GER, NZL
SECRETHCUMINTHHEL TU USA, AUS, CAN, GER, NZL
HTTP Activity Client-to?SerV
GET nl . start=
Accept:
A: I: apt?Language
I -
User?Agent Hazillaf?.? (compatible; HSIE Windows NT 5.1;
HDSE:
Eache? an EU max?3 a e=l
Connectinn: Ee-a?1 -
E?Elue?uat?via
Hn?t UHL Path LIHL ?rga
Hearth
Search Terma Language: Elrnwaer ma
mLJEharraf an Mozillam? (compatible; MSIE Window NT 5.1;
Heferer
CDDME
TDP SEGHETHCUMIHTHRELTD USA, AUS, CAN, GER, NZL
SECRETHCUMINTHHEL TU USA, AUS, CAN, GEIR, MEL
"ve r ap
Full log contains basic information on every
single DNI session XKS processes.
- HTTP activity contains more detailed
information on the subset of that data which
is web-based (aka port 80 ?internet
browser? traffic)
TDP SEGHETHCUMIHTHRELTD USA, AUS, CAN, GER, NZL
TU USA, AUS, CAN. GER.- NZL I [El
How the Search Forms Fit Togej; mm
-
of all DNI sessions collected
.ll IIHTTP Activity
TDP SEGHETHCUMIHTHRELTD USA, AUS, CAN, GER, NZL
,Example
TU USA, AUS, CAKE-ER,
1 i I
-
Analysis of 14 May Internet session of
based target started in MARINA
TH i
20030514 1323533
2013130514 1324163
2013130514 1324193
201330514 13233-43
200130514 1323-433
2013130514 1335133
20030514 1335223
UHEEJITI-I USEFLA
.Qhatma?. cam?msnp as 51: 011:3
manned]. cam?msnp 355123-011?
mic-1311131]. camimsnp as 31:: 011:1?
mica-?nal. camimsnp as 312:: art?
.Qhatma?. cam?msnp as 51: 011:3
mic-Maj]. minimally-as 31:42:11?
TDP SEGHETHCUMIHTHRELTD USA, AUS, CAN, GER,
UHEELE
119-?
119.?
119.?
in
.153 lagged in (11an
Ragga-:1 111 {1111]
at: lagged in (1111)
3+ lagged in {1111}
.153 lagged in (11an
~42 lagged in (1111)
NZL
SECRETHCUMINTHHEL TU USA, ALIS, CAN, GEIR, MEL
"Understand what is behind}t HP
Ensure Activity on IP can be associated with
Target
Understand IP usage Dynamic/Static
- Research IP using Foxtrail/NKB
Is it a Proxy, DVBLAN, Dial-Up, DSL, etc
- Is it Client to Server or Server to Client
- Still not sure? User Activity pull for 5 minute
period on Foreign IP
Derived Frem: NEAICSSM 1-52
Dated:
0n:
TDP SEGHETHCUMIHTHRELTD USA, AUS, CAN, GER, NZL
To use, AUS, CAN. GER.-
. I
HTTP with a Proxy
1-
Requeet sent to Prexy Request sent to Server
1.. E.
Response sent to User Response sent to Proxy
- Performance: Proxy can cache responses for static pages
- Censorship: Proxy can filter traffic
- Security: Proxy can look for malware
- Access-Control: Proxy can control access to restricted content
Proxy can be run by
- a user
- an ISP
a web-hosting company
- a content-delivery network Akamai)
TDP SEGHETHCUMIHTHRELTD USA, AUS, CAN, GER, NZL
SECRETHCUMINTHHEL TU USA, AUS, CAN, GER, NZL
Proxies on the Internet
Web?Servers
Shed-lived eenneetiene Shert-lived eenneetiene Lung-lived cennectiene
Single-user Multiple-users multiplexed Multiple-users multiplexed
Web_sewer Web?Servers
Tl It'll i?l ?lii
OD'rec't'ConneCt Mlxed'Gateway National-Level Proxy
I
Caghe
rexy-te-Proxy
User-to-Prexy
1M JLUMM
TDP SEGHETHCUMIHTHREL TD USA, 13
Mamet
Mamet
Creme-{tie
bait-6mm
bib-6mm
bib-6mm
tr-eE-eE-eE-e
pee?6mm
Mamet:
heme-em
bib-6mm
whet-{be
mE?ebee-e
gee-(eat!
SECRETHCUMINTHHEL TU USA, ALIS, CAN, GER, NZL
Example #1
The analyst then did an HTTP activity qu
find all web surfing from that IP address within
the same rough timeframe.
El {jl tit-M
Search: HTTP Activity
alert
Black?arw Quart,? Mama:
Gaul?U 3 PH baa-ad LE Ltaal?. 1:3? CT
I: Juatificatian: target
Eatagarar [it'll
Cellular [JP-ll
DEED Datatirna: Custarn t-t Start: panama-14 'El 1a:aa Staci: panama-14 14:15 $1 mil
Dacurnarrt l't-latadata
- Dacumarrl Tagging
Ernail ?3 ?ddrag?: Either
Extract-ad Filaa Addreaa: TD
- Full Lag [JP-ll part: Ham
HTTP
lltE F'araar Part: TD
IRIS Cafe Gaalacatian
Lagina and
lt-licraplugin
l_l hi 1'
TDP SEGHETHCUMIHTHRELTD USA, AUS, CAN, GER, NZL
SECRETHCUMINTHHEL TU USA, AUS, CAN, GER, NZL
. 14 May Strange HTTP Activit
HTTP meta-data indicated strange web-bed
activity
Hnat LIHL F'ath
Jilrf?s?ruites
Elmwaar
GET In?atascaclestams? 1
East: ?fasemice??M?dmsdende
Germanic-11: KEEP-MWE, TE
TE: trailers, de?ate, gap, campress
Us Er-?tgant: BPT-HTIPCHEHHU.4-C1EF
Cache-Cantr?: ?a-cacha
Pragma; na?cache
TDP SEGHETHCUMIHTHRELTD USA, AUS, CAN, GER, NZL
SECRETHCUMINTHHEL TU USA, ALIS, CAN, GER, NZL
14 May Strange HTTP Activit]
Indications from the HTTP activity
Elm-weer
Note the strange User
GET
Heat; dresdende
Cements-11: Keep-Alive, TE
Cache-Central: rte-cache
Pragma: Ila?cache
TDP SEGHETHCUMIHTHRELTD USA, AUS, CAN, GER, NZL
TU USA, AUS, CAN. GERpossible anonyi
Open Source research indicated that this user
agent was indicative of multi-cast traffic. A
likely tip off that this was some type of
anonymizer
GET
Heat: inf-es Ef'ii'ill tie
Came :tie?: Heep-MaeCache-Central: Ila-cache
Pragma:
TDP SEGHETHCUMIHTHRELTD USA, AUS, CAN, GER, NZL
TU USA, E. @339 Ml.
Results led to Full Ugh; Query
The two tu-dresdende requests were the only
HTTP activity seen within that timeframe, but
given the open source research suggesting that
the user agent was an anoynmizing proxy a full
log query was ran to identify all other traffic
originating from that same IP address during
the same time
Datetirne End
Host UHL Path
:inf?aaruicaa
TDP SEGHETHCUMIHTHRELTD USA, AUS, CAN, GER, NZL
.Ftill Log
Detetirne an.
4 13433]?
13:43:35
Results
Fm
TDIP
1415
1513
1491
1134
1134
To USA, Alia, ?rmFor Application Type
EHII web
5544 unknown
5544 unknown
Application
litter-yet
httpi'get
unlizn?wnften
unknown-ten
Note the two HTTP activity (port 80) sessions were
seen, but in addition there was one SSL (port 443)
session and two unknown port 6544 sessions
TDP SEGHETHCUMIHTHRELTD USA, AUS, CAN, GER, NZL
TU USA, AUS, CARI-GER
tiFu'IILog Results
.e
Te Fm F'er Te F't Applieetien Type ?pplieetien Date Length Length
?Ir-11.- 1134 5544 uni-tnewnften
He- 14i._ 1m sen unitinewniteu eat-ea
0f the unknown port 6544 traffic, the data length
of the sessions indicated that a significant amount
of data was leaving the Pakistan IP used by our
target
TDP SEGHETHCUMIHTHRELTD USA, AUS, CAN, GER, NZL
TU USA, AUS, CAN. GER.- NZL I [El
Full Log Results
The content appeared unreadable. Further
analysis by CES and open source research
showed that the content was
[lil'll [Iiiallulajllr Raw Data DNI Format 1
Services 1
Ne available ter thie type at elata. eerrelirrg it te arrether eewiee fer a Ijetter view- Eelew ie an attempt te
diaplap the ae plain teat.
reverie.
"are
genera-?ea f'?rtr" ewee [l?u?w
em?pegr??e?a?a [ea-1: 2112'
.E?rkc?fa'? ?ea-eL-Eaa e3 ewe 1-H rave ea area! ?eat-1:. ea
ewe-pee
el page ?edema: pie anaemetereep ?ne
pane-ea [temp??r [rt-?r amateur-area" ?a?r?e zae ?ee aura-e
e?av?eaa??l?ey-?e ?r??mae?am?aaea?-ea
ear?tee I appear
TDP SEGHETHCUMIHTHRELTD USA, AUS, CAN, GER, NZL
TU USA. AUS, CAN.GER,
Results led to Full Lo?puery
- While we were ultimately unable to identify what
was underneath the 150K of traffic,
we were at the least able to identify that our
target was using an anonymizing service to
mask a portion of his Internet activity
SERI EERIAL: *3f??f5131?9??9
German TD UEA,
TD USA, During the it. 15; and 18 hay
sessicns an the telenhcne, the userisl were using a free
German?hasea
presumably tc mask the
scurce at Internet traffic. Use cf the prcay
primarily curing the times in which
_Ehctmail.ccm anti. _@yahcc.ccm uere accessed.
CDHHENT: TD DEA, The German?hasea
is a Java iJle. which was develcped
hy the Technical University cf Dresden as a free and cnen
scarce tccl. The functicns in a manner
similar tc The Dnicn Renter netucrh. Given his
in ccmputer science and netucrhing; it is net
surprising that - t-rculcl use an tc
secure his Internet activity.
TDP SEGHETHCUMIHTHRELTD USA, AUS, CAN, GER, NZL
To USA, nus, em. @3139 ML
0
If the Full Log query gave us the HTTP traffic in
addition to the other non web based traffic, why
don?t we only use the Full Log query?
- Because the meta-data options in the full log table
are limited
Dototirno End 1-.
4
13:43:15
43:49?2?.
Fm To
Hi-
F'or To Aooliootion Type
4495 ?fi
4134 5544 unknown
4434
443 notworl-iz?norgmtion
119- 55-5-
web
5544 unknown
TDP SEGHETHCUMIHTHRELTD USA, AUS, CAN, GER, NZL
Aooliootion
lrth1-?et
httoiget
unknown?on
unknown-ion
TU USA, AUS, CAN, GER
Example #2 i
-
Starting with MARINA results of a 20 May.
Internet session of an Iran based target
Ts 1r USEEJD ACTIVITY UttEEj mam
20090520 05921392, l?gg?di? {Email}
@1392 used
magma] 0921303 lagged?'t {email}
20090520 05921302, used
TDP SEGHETHCUMIHTHRELTD USA, AUS, CAN, GER, NZL
SECRETHCUMINTHHEL TU USA, AUS, CAN, GER, NZL
Example #2
The analyst then did a full log query based off
the IP X-Forwarded-IP pair
El El ?l-M
end 'Isi'mii-l? Metedete
. sales I
. BIEEHEEFW Query Heme: Iren_terget
- ENE
- Cell Lege
- CetegC-r'll DNI
- Celluler Eihll
-
.
- Deeurnel'lt iu'ietegete
- Deeull'lel'lt Teggil'lg
Iranian ER Address need he g?yeg
Juetiti Ceti en: ?1 lit-He?ll
Cuslsm Start: etsls: Bi 11:ue if ti
Email Addresses Client: ID [I-Fewerded-Ferjl:
.
Full Leg erll ugemame:
.
Il?lfCI: Iii. [Field Builder]
IHE Pereer
- leC
. Legirle end
5- - Metegete gddregg; TE.
Frern
TDP SEGHETHCUMIHTHRELTD USA, AUS, CAN, GER, NZL
Example
IO I
I
a. -I
'1l
SECRETHCUMINTHHEL To USA.- em ML
#2
Full Log table contains the standard DNI meta-data
with some but not all information from other plug-ins
included (ie. Username from User Activity and
Application Info contains some HTTP activity)
Us er name
?application Into
rn1.ali:.facelmoii .c-tun'ii'n'
. I
-
5.
le FrnCitH
IFI:
gmailaml IF:
IF:
?l?i'itliljtIFE
IFI:
IFI:
IFI:
IFE
TEHHAH
TEHHAH
TEHHAH
TEHHAH
TEHHAH
TEHHAH
TEHHAH
TEHHAH
TEHHAH
TEHHAH
TEHHAH
TEHHAH
TEHHAH
TEHHAH
TEHHAH
TEHHAH
TIJC TI: EttaGEE
L?ll?? l-I
FHAHHFLIRT
FHQFIHFUHI
FHAHHFUHT
HMETEHDAM
FHAHHFUFLT
FHAHHFUFLT
L?ll??
Datetime
111:1?:1!
Datetirne End
'l ?:1-9:15
1
1 ?:16
1 mil-3:54
1 1 :33
amt-9 435-??
1?:1 2:12
435-!?
1?:12:?5
1
1
1 2:?9
1
1 ?:24
TDP SEGHETHCUMIHTHRELTD USA, AUS, CAN, GER, NZL
Fm
A
'Iu'l
'h'l
E13.
213.
213.
213.
E13.
213.
195._ 41595
195._ 3441]!?
195._ 344%
To
TU USA, AUS, CAN, GER
Example #2 i .
- The analyst wanted to know if the From lP X-
Forwarded-For IP pair was representing a single
computer or if there were multiple users on multiple
computers in this data.
- Full log only provides the bare minimum meta-data to
make this determination
ID Detetirne .Etpplieetien lnfe
1 I121 :45
FITI TU Fm TD ?pplieetien T'fpe ?pplieetien Date Length Length
EU meil meilhwelemeilhrehee 1 443 1
211- 55.55.125.215 55455 55 3452 5522
TDP SEGHETHCUMIHTHRELTD USA, AUS, CAN, GER, NZL
To USA, ALIS, 90TH. 2512120122
#2 fl?
i
Example
MARINA provided this information:
T0 0. 0021120 2110212 0221221 0:521:02: 0:002:12
200005201021052 pram-5021;; 102320111 212- 012122514051211202110022001022
20000520 1021052 15220100215; 1023201111 212- 01212251005211202110020001022
200505201021452 0520520110 1521152301 012122214052112521002000222
20020520 1022122 -52.:20002: 0520520110 102152551
20000520 1022122 -20200022 registered 0201 22
20020520 1022122 laggeaarramm?; 215--
- The Yahoo and Facebook activity came from the same
proxy IP and the same X-Forwarded-For-IP and
around the same time but was it from the same
computer?
TDP SEGHETHCUMIHTHRELTD USA, AUS, CAN, GER, NZL
SECRETHCUMINTHHEL TU USA, AUS, CAN, GER, NZL
I HTTP Activity uery
Let?s query that same date time range an
and XFF IP pair in the HTTP Activity query to
see what we get
El {jl sit-M
Alert
l2;
CHE
Cell Luge
Search: HT Activity-
Query Ha me: iren_terget_http
catag?rlf Iranian eddress used by
Dr." Justificatien: target
2: Elie-3n
Dee
Document Meta-date
Tagging b: Ferwerded Fer: 1EE.1EE.3E.1
Email
Eetreeteci Fiiee re me ?r
2: Full Leg [p TD
Activity
litE F'ereer
r:E lFtI: Cafe Genlneetien
Legine end
Micreplugin it-ietedete
hi 1'
Detetime: Cuetem Start: Enema-HE Step: 14:15 till
TDP SEGHETHCUMIHTHRELTD USA, AUS, CAN, GER, NZL
TU USA, AUS, CAKE-ER,
{Now view the HTTP Actix?z [teem
We saw this meta-data in the Full Log results:
ID Datetirne J. Applieatien Info
3 21 :45
1
lF' TD lF' F'Di't TD Applieetien Type Applieetien Date Length Length
213- 2ee_ ee mail mailhvelerneilffehee 1445 19:33
211? eel- 55435 at] sex-mill se?ialif?eelaeeh 34112 3922
And then these three fields are among the
unique (and valuable) fields only found in the
HTTP activity table:
Ceeltie Heferer Elreweer
tflL-l?fewidth?EE-Eteue? li-lezillai?? While-we; Ll; 'i-itl'indewe NT 5.1; en-US;
detr=1 2421 .faeeheel-teemlifrarf li-leIillaIEJZI While-we; Ll; NT 5.2; en-US;
TDP SEGHETHCUMIHTHRELTD USA, AUS, CAN, GER, NZL
To USA, ALIS, gin. gamma;
Of interest, note the differences between the
two user agents
Browser
Mozillaf?? upwind-awe NT 5.1 in-US; Ei Firefox??d
Mozillaf?? (Windows; Windows NT 5.2; Ei Firefox??d
This indicates different versions of Windows, so
unless they did an upgrade within the 1 minute
difference of activity, there were at least two
different computers behind that Proxy and XFF
IP pair
TDP SEGHETHCUMIHTHRELTD USA, AUS, CAN, GER, NZL
To us?, HEEL.
the story}
no u:
I 1 Lug; w.
n: *nw
Cali: .
(l 1.1
.
-.I: v. ..
r'.ul
You should be use both the HTTP activity and
Full Log queries to help discover everything
your target does when he?s online
HTTP Activity will give you great meta-data for
quick analysis of ?web-based? (port 80) activity
But not all DNI is done through an Internet
Browsers, so it?s important to look at the Full
Log query results for indications of the use of
other applications
TDP SEGHETHCUMIHTHRELTD USA, AUS, CAN, GER, NZL
SECRETHCUMINTHHEL TU USA, AUS, CAN, GER, NZL
Moral of the story
The lVlulti-Search page gives you the ability to search full log and
HTTP activity based on an IP address at the same time
El-Sl?llassis
El
534' Simply enter In an IP address choose any or all
.3 ?nd-dresses I 5
Ere: sees roles (Ie. fromltolef) and then choose what
Usernams
Elama?gic ?rm
Eel-ea
EEllasHElerry
ECHE IP address: HE.
ICall Legs
2: Cat-sense,f DNI v" Frarn
?is-antler DNI IF. EDIE: H. TH
Sissa Passwards
2: ENE at H-Fnrwarded-Far
Basement l'i-lstadata
Dimmers Tagging User Activity
- Phene Number Estraeter
Email addresses Search .
E: Extracted HES Elnall addresses
Extracted Files
2: FullLag DHI nativity-
EHTTP same? Full Lag
IHE F'arser Prim?
IRE: Cafe Gsalasatian
Z: Lagins and Password:
Misraplugin l'i-lstadata
TDP SEGHETHCUMIHTHRELTD USA, AUS, CAN, GER, NZL
SECRETHCUMINTHHEL TU USA, AUS, CAN, GER, NZL
Moral of the story
It will submit the multiple searches at the same time, you can
either View the results separately or View them as a merged
table
My Recent Results
Help Actiens 1* FILTERS:
Eistetime Sulsrnilted Heme Status Hum Results Query.f Type
till: in 15 met.? finished full_led
sit in men.r finished 4'55
TDP SEGHETHCUMIHTHRELTD USA, AUS, CAN, GER, NZL