Name: Franck Guénichot Answer 1: 00:25:00:fe:07:c4 Answer 2: AppleTV/2.4 Answer 3a: h Answer 3b: ha Answer 3c: hac Answer 3d: hack Answer 4: Hackers Answer 5: http://a227.v.phobos.apple.com/us/r1000/008/Video/62/bd/1b/mzm.plqacyqb..640x278.h264lc.d2.p.m4v Answer 6: Sneakers Answer 7: $9.99 Answer 8: iknowyourewatchingme Description: Network Forensics Puzzle Contest #3 =================================== Challenge by Sherri Davidoff & Jonathan Ham Solution by Franck GUENICHOT (franck.guenichot[at]orange[dot]fr) Tools Used: _ TShark 1.2.2 _ macfinder (custom ruby script giving IP/MAC bindings from a pcap file.) (require packetfu) _ httpdumper (custom ruby script that can display and dump HTTP conversations) (require packetfu and terminal-table) _ plist.rb (custom ruby script that can display informations extracted from Apple Property-List 1.0 XML documents) Recommandations for using these tools: macfinder and httpdumper rely on the wonderful ruby lib: packetfu. Significant performances improvements have been done by its author in version 0.3.1. (8x faster) So, You SHOULD use the last version of packetfu, eg: at least, packetfu 0.3.1 The Challenge ============= """ Ann and Mr. X have set up their new base of operations. While waiting for the extradition paperwork to go through, you and your team of investigators covertly monitor her activity. Recently, Ann got a brand new AppleTV, and configured it with the static IP address 192.168.1.10. Here is the packet capture with her latest activity. You are the forensic investigator. Your mission is to find out what Ann searched for, build a profile of her interests, and recover evidence including: 1. What is the MAC address of Ann’s AppleTV? 2. What User-Agent string did Ann’s AppleTV use in HTTP requests? 3. What were Ann’s first four search terms on the AppleTV (all incremental searches count)? 4. What was the title of the first movie Ann clicked on? 5. What was the full URL to the movie trailer (defined by -Y΄preview-url‘)? 6. What was the title of the second movie Ann clicked on? 7. What was the price to buy it (defined by ΄price-display‘)? 8. What was the last full term Ann searched for? Here is your evidence file: http://forensicscontest.com/contest03/evidence03.pcap MD5 (evidence03.pcap) = f8a01fbe84ef960d7cbd793e0c52a6c9 """ Quick Answers: ============== 1. What is the MAC address of Ann’s AppleTV? A: 00:25:00:fe:07:c4 2. What User-Agent string did Ann’s AppleTV use in HTTP requests? A: AppleTV/2.4 3. What were Ann’s first four search terms on the AppleTV (all incremental searches count)? A: h A: ha A: hac A: hack 4. What was the title of the first movie Ann clicked on? A: Hackers 5. What was the full URL to the movie trailer (defined by -Y΄preview-url‘)? A: http://a227.v.phobos.apple.com/us/r1000/008/Video/62/bd/1b/mzm.plqacyqb..640x278.h264lc.d2.p.m4v 6. What was the title of the second movie Ann clicked on? A: Sneakers 7. What was the price to buy it (defined by ΄price-display‘)? A: $9.99 8. What was the last full term Ann searched for? A: iknowyourewatchingme Detailled Answers ================= As usual, we have to verify the evidence file integrity: franck@ODIN:~/Analysis/Sources/Puzzle_3$ md5sum evidence03.pcap f8a01fbe84ef960d7cbd793e0c52a6c9 evidence03.pcap Ok, we're good to go ! First,I used to look at the protocol hierarchy stats given by tshark to take a first look at a pcap file: franck@ODIN:~/Analysis/Sources/Puzzle_3$ tshark -r evidence03.pcap -qz io,phs =================================================================== Protocol Hierarchy Statistics Filter: frame frame frames:1778 bytes:1508750 eth frames:1778 bytes:1508750 ip frames:1778 bytes:1508750 udp frames:28 bytes:6102 dns frames:28 bytes:6102 tcp frames:1750 bytes:1502648 http frames:167 bytes:93189 image-gif frames:33 bytes:21202 xml frames:18 bytes:20852 tcp.segments frames:65 bytes:46469 http frames:65 bytes:46469 xml frames:17 bytes:11732 image-jfif frames:48 bytes:34737 =================================================================== Hum, I bet we'll have to work with HTTP and some XML documents/data ! Let's continue... We know that Ann has recently aquired an AppleTV and has configured it with a static IP address: 192.168.1.10 For some obvious reason, we have to know the mac (or hardware) address of Ann's new HDTV box. Tshark could help for this task: ############################################################################################################################ franck@ODIN:~/Analysis/Sources/Puzzle_3$ tshark -r evidence03.pcap -R "ip.src==192.168.1.10" -Tfields -e "eth.src" |uniq 00:25:00:fe:07:c4 ############################################################################################################################ well, i've also coded a small ruby script for this task: macfinder.rb. Here's the help screen: franck@ODIN:~/Analysis/Sources/Puzzle_3$ ./macfinder.rb macfinder version 0.1 Copyright (C) 2009 Franck GUENICHOT macfinder comes with ABSOLUTELY NO WARRANTY; This is free software, and you are welcome to redistribute it under certain conditions. (GPL v3) Usage: macfinder [options] -i, --ip Display Mac address for the given IP address only (4-digit decimal dot notation form) -v, --version Display version information -h, --help Display this screen Without any switch, macfinder.rb displays all the source IP/MAC address found in the pcap file: franck@ODIN:~/Analysis/Sources/Puzzle_3$ ./macfinder.rb evidence03.pcap Listing all Mac Address found ! IP: 8.18.65.10 | Mac: 00:23:69:ad:57:7b IP: 8.18.65.32 | Mac: 00:23:69:ad:57:7b IP: 8.18.65.88 | Mac: 00:23:69:ad:57:7b IP: 8.18.65.89 | Mac: 00:23:69:ad:57:7b IP: 8.18.65.67 | Mac: 00:23:69:ad:57:7b IP: 8.18.65.58 | Mac: 00:23:69:ad:57:7b IP: 8.18.65.82 | Mac: 00:23:69:ad:57:7b IP: 8.18.65.27 | Mac: 00:23:69:ad:57:7b IP: 192.168.1.10 | Mac: 00:25:00:fe:07:c4 IP: 4.2.2.1 | Mac: 00:23:69:ad:57:7b IP: 66.235.132.121 | Mac: 00:23:69:ad:57:7b From the listing above we can easily find Ann's AppleTV mac address. But to be less verbose, and because we know the IP address, we can use the -i switch to display only the interresting MAC. franck@ODIN:~/Analysis/Sources/Puzzle_3$ ./macfinder.rb -i 192.168.1.10 evidence03.pcap Mac: 00:25:00:fe:07:c4 And Voila ! Going Deeper Part I : HTTP ============ Now, we have to go deeper in the pcap file to analyse Ann's networking activity and particularly her AppleTV network conversations. Tshark let us know that we'll have to deal with HTTP (and maybe XML documents, later), so I wrote a specialized tools to facilitate the investigation: httpdumper httpdumper basically displays informations about HTTP conversations. The HTTP protocol is a Request/Response protocol meaning that a client makes a request to a server with HTTP request messages and the server answers with HTTP response messages. httpdumper handles this mechanism and displays these conversations in an easy to understand manner. Some terminology: An HTTP conversation, for httpdumper, is the set of all REQUEST/RESPONSE HTTP messages involving the same 2 hosts and tcp ports. An HTTP flow is an unidirectionnal flow of http data (eg: client to server (request) or server to client (response) Here's the help screen: franck@ODIN:~/Analysis/Sources/Puzzle_3$ ./httpdumper -h httpdumper version 0.1 Copyright (C) 2010 Franck GUENICHOT httpdumper comes with ABSOLUTELY NO WARRANTY; This is free software, and you are welcome to redistribute it under certain conditions. (GPL v3) Usage: httpdumper [options] -r -r, --read Read the given pcap file [REQUIRED] -c, --conversation # List only flows for conversation # -f, --flow # List only flow # --with-headers For Display ONLY -d, --dump Dump the selected conversation or flow -p, --port Define custom HTTP port -s, --stats type,[val1],[val2] Displays statistics Valid options: Request stats: request,[requester_ip],[requested_host] URI list: uri,[requester_ip],[target_hostname] -v, --version Display version information -h, --help Display this screen httpdumper is only a passive (lightweight) analysis tool, it needs a file in entry, so -r options is required to launch this tool. The default output (without any options) displays all the HTTP conversations found in the given pcap file. Let's do it ! franck@ODIN:~/Analysis/Sources/Puzzle_3$ ./httpdumper -r evidence03.pcap Reading file evidence03.pcap Parsing packets... 1778 packets read in 4.385 sec. Found 20 HTTP conversation(s) +--------------------+--------------------------------------------+-----------------+---------+----------+-------------------+ | Conversation Index | Hosts | HTTP Flow count | Request | Response | Cumulative length | +--------------------+--------------------------------------------+-----------------+---------+----------+-------------------+ | 0 | 192.168.1.10:49163 < - > 8.18.65.67:80 | 2 | 1 | 1 | 16551 | | 1 | 192.168.1.10:49164 < - > 66.235.132.121:80 | 2 | 1 | 1 | 43 | | 2 | 192.168.1.10:49165 < - > 8.18.65.32:80 | 8 | 4 | 4 | 22453 | | 3 | 192.168.1.10:49166 < - > 66.235.132.121:80 | 8 | 4 | 4 | 172 | | 4 | 192.168.1.10:49167 < - > 8.18.65.58:80 | 20 | 10 | 10 | 161118 | | 5 | 192.168.1.10:49168 < - > 8.18.65.67:80 | 4 | 2 | 2 | 3157 | | 6 | 192.168.1.10:49169 < - > 66.235.132.121:80 | 2 | 1 | 1 | 43 | | 7 | 192.168.1.10:49170 < - > 8.18.65.82:80 | 44 | 22 | 22 | 675124 | | 8 | 192.168.1.10:49171 < - > 8.18.65.27:80 | 6 | 3 | 3 | 13582 | | 9 | 192.168.1.10:49172 < - > 66.235.132.121:80 | 6 | 3 | 3 | 129 | | 10 | 192.168.1.10:49173 < - > 8.18.65.27:80 | 8 | 4 | 4 | 12744 | | 11 | 192.168.1.10:49174 < - > 66.235.132.121:80 | 2 | 1 | 1 | 43 | | 12 | 192.168.1.10:49175 < - > 66.235.132.121:80 | 6 | 3 | 3 | 129 | | 13 | 192.168.1.10:49176 < - > 8.18.65.67:80 | 4 | 2 | 2 | 3493 | | 14 | 192.168.1.10:49177 < - > 8.18.65.10:80 | 32 | 16 | 16 | 362826 | | 15 | 192.168.1.10:49178 < - > 66.235.132.121:80 | 2 | 1 | 1 | 43 | | 16 | 192.168.1.10:49179 < - > 8.18.65.88:80 | 20 | 10 | 10 | 5576 | | 17 | 192.168.1.10:49180 < - > 66.235.132.121:80 | 20 | 10 | 10 | 430 | | 18 | 192.168.1.10:49181 < - > 8.18.65.89:80 | 18 | 9 | 9 | 4861 | | 19 | 192.168.1.10:49182 < - > 66.235.132.121:80 | 18 | 9 | 9 | 387 | +--------------------+--------------------------------------------+-----------------+---------+----------+-------------------+ The table above show all the HTTP conversations found. (this kind of table is best viewed on large display) The flow count indicates the number of flows in each conversations Request and Response column, each displays the number of HTTP Request or HTTP response in each conversation Cumulative length: the length (in Bytes) of the HTTP Payloads (or HTTP message body) exchanged in each conversation. Note: this length takes only HTTP Response payloads into account. By now HTTP Request message body, if any, is not displayed (and not "dumpable") Quickly, we learn interresting infos: _ 20 HTTP conversations, all involving the same client (Ann's AppleTV) _ 7 of them are composed of 18+ flows _ Conversation #7 has the greater cumulative length (so, the largest HTTP payload) But all these informations aren't enough: to continue our investigation we need to go deeper. Let's try to answers Question #2: What User-Agent string did Ann’s AppleTV use in HTTP requests? According to RFC2616: "The User-Agent request-header field contains information about the user agent originating the request. This is for statistical purposes, the tracing of protocol violations, and automated recognition of user agents for the sake of tailoring responses to avoid particular user agent limitations." The User-Agent is just a string contained in the HTTP Request Message Header indicating which Software/Version is used to send the request. So to get this information, we have to display the http header of Ann's AppleTV HTTP request. httpdumper could help us for this task with the sub-options --with-header. Let's take conversation 0 with headers displayed: franck@ODIN:~/Analysis/Sources/Puzzle_3$ ./httpdumper -r evidence03.pcap -c0 --with-headers Reading file evidence03.pcap Parsing packets... 1778 packets read in 4.471 sec. Listing flows for conversation 0 with full http headers ---------------------------------------------------------------------- Flow Index: 0 192.168.1.10:49163 -> 8.18.65.67:80 REQUEST /WebObjects/MZStore.woa/wa/viewGrouping?id=39 0 ----------- HTTP HEADER ----------- Accept: */* Accept-Language: en Accept-Encoding: gzip, deflate Cookie: s_vi=[CS]v1|259C176A85010C29-6000010D80115D7F[CE] User-Agent: AppleTV/2.4 <------------------------------------------------------------ Look Here If-Modified-Since: Fri, 25 Dec 2009 04:42:31 GMT X-Apple-Store-Front: 143441-1,3 Connection: keep-alive Host: ax.itunes.apple.com ---------------------------------------------------------------------- Flow Index: 1 8.18.65.67:80 -> 192.168.1.10:49163 RESPONSE text/xml 16551 ----------- HTTP HEADER ----------- Last-Modified: Sun, 27 Dec 2009 14:24:32 GMT x-apple-lok-response-date: Sun Dec 27 06:27:25 PST 2009 Content-Encoding: gzip x-apple-lok-current-storefront: 143441-1,3 x-apple-application-site: CUP Content-Type: text/xml x-apple-lok-expire-date: Sun Dec 27 07:04:32 PST 2009 x-apple-lok-stor: memcached x-apple-max-age: 3600 x-apple-woa-inbound-url: /WebObjects/MZStore.woa/wa/viewGrouping?id=39 x-apple-application-instance: 6119 x-apple-lok-path: v0_1:MZStore/viewGrouping&id=39-143441-1,3,pc-3-Ak x-apple-aka-ttl: Generated Sun Dec 27 06:27:25 PST 2009, Expires Sun Dec 27 07:27:25 PST 2009, TTL 3600s x-apple-lok-ttl: Generated Sun Dec 27 06:24:32 PST 2009, Expires Sun Dec 27 07:04:32 PST 2009, TTL 2400s x-webobjects-loadaverage: 0 Content-Length: 16551 Expires: Mon, 28 Dec 2009 04:08:02 GMT Cache-Control: max-age=0, no-cache Pragma: no-cache Date: Mon, 28 Dec 2009 04:08:02 GMT Connection: keep-alive Vary: Accept-Encoding Vary: X-Apple-Store-Front X-Apple-Partner: origin.0 ---------------------------------------------------------------------- httpdumper "-c num" option is used to select only Conversation 0 flows, --with-headers tells httpdumper to display entirely the http header. Now, easily, we can gather what we are searching for by looking at Flow 0 header's field: User-Agent : AppleTV/2.4 It indicates that the Software installed on Ann's AppleTV is version 2.4 To answer #3, we have to understand the protocol used to make movies search and so on... Lets have another view of the HTTP traffic: Using -s options (or --stats) we could display the request statistics for a given client: franck@ODIN:~/Analysis/Sources/Puzzle_3$ ./httpdumper -r evidence03.pcap --stats request,192.168.1.10 Reading file evidence03.pcap Parsing packets... 1778 packets read in 4.402 sec. HTTP REQUEST STATISTICS ----------------------- +----------------------------+-------------------+---------------+ | Requested Host | Requested Host IP | Request Count | +----------------------------+-------------------+---------------+ | ax.search.itunes.apple.com | 8.18.65.32 | 30 | | a1.phobos.apple.com | 8.18.65.58 | 48 | | metrics.apple.com | 66.235.132.121 | 33 | | ax.itunes.apple.com | 8.18.65.67 | 5 | +----------------------------+-------------------+---------------+ Interresting, with the hostname of the servers in the table above, we could make some assumptions on their respective roles. I bet that ax.search.itunes.apple.com has something to do with searching something... Maybe metrics.apple.com is a kind of statistics gathering host. And for the two others we will look at them later. Let's display all the URI requested by 192.168.1.10 to ax.search.itunes.apple.com : (USE: --stats (or -s) uri,192.168.1.10,ax.search.itunes.apple.com franck@ODIN:~/Analysis/Sources/Puzzle_3$ ./httpdumper -r evidence03.pcap --stats uri,192.168.1.10,ax.search.itunes.apple.com Reading file evidence03.pcap Parsing packets... 1778 packets read in 4.362 sec. ---------------------------------- Listing URI requested by 192.168.1.10 ---------------------------------- ---------------------------------- Requested to ax.search.itunes.apple.com ---------------------------------- [conv: 2] [flow: 0] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=h [conv: 2] [flow: 2] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=ha [conv: 2] [flow: 4] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=hac [conv: 2] [flow: 6] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=hack [conv: 8] [flow: 0] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=s [conv: 8] [flow: 2] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=sn [conv: 8] [flow: 4] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=sne [conv: 10] [flow: 0] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=sneb [conv: 10] [flow: 2] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=snea [conv: 10] [flow: 4] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=sneak [conv: 10] [flow: 6] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=i [conv: 16] [flow: 0] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=ik [conv: 16] [flow: 2] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=ikn [conv: 16] [flow: 4] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=ikno [conv: 16] [flow: 6] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=iknow [conv: 16] [flow: 8] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=iknowy [conv: 16] [flow: 10] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=iknowyo [conv: 16] [flow: 12] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=iknowyou [conv: 16] [flow: 14] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=iknowyour [conv: 16] [flow: 16] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=iknowyoure [conv: 16] [flow: 18] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=iknowyourew [conv: 18] [flow: 0] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=iknowyourewa [conv: 18] [flow: 2] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=iknowyourewat [conv: 18] [flow: 4] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=iknowyourewatc [conv: 18] [flow: 6] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=iknowyourewatch [conv: 18] [flow: 8] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=iknowyourewatchi [conv: 18] [flow: 10] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=iknowyourewatchin [conv: 18] [flow: 12] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=iknowyourewatching [conv: 18] [flow: 14] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=iknowyourewatchingm [conv: 18] [flow: 16] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=iknowyourewatchingme My thoughts were right. The listing above displays all the research made by Ann on her AppleTV. So the first four search term were: 1 h 2 ha 3 hac 4 hack and the last full term search was: iknowyourewatchingme (answer question #8) he he :-) All these requests are within conversation 2 in httpdumper (look at the conv value in bracket on the left of the lines) franck@ODIN:~/Analysis/Sources/Puzzle_3$ ./httpdumper -r evidence03.pcap -c 2 Reading file evidence03.pcap Parsing packets... 1778 packets read in 4.439 sec. FLOWS TABLE +------------+-------------------------------------+-------------------+------------------------------------------------------------------+---------------------+ | Flow Index | Hosts | HTTP message type | HTTP Request or Content type | HTTP Content Length | +------------+-------------------------------------+-------------------+------------------------------------------------------------------+---------------------+ | 0 | 192.168.1.10:49165 -> 8.18.65.32:80 | REQUEST | /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=h | 0 | | 1 | 8.18.65.32:80 -> 192.168.1.10:49165 | RESPONSE | text/xml; | 10087 | | 2 | 192.168.1.10:49165 -> 8.18.65.32:80 | REQUEST | /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=ha | 0 | | 3 | 8.18.65.32:80 -> 192.168.1.10:49165 | RESPONSE | text/xml; | 10020 | | 4 | 192.168.1.10:49165 -> 8.18.65.32:80 | REQUEST | /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=hac | 0 | | 5 | 8.18.65.32:80 -> 192.168.1.10:49165 | RESPONSE | text/xml; | 1173 | | 6 | 192.168.1.10:49165 -> 8.18.65.32:80 | REQUEST | /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=hack | 0 | | 7 | 8.18.65.32:80 -> 192.168.1.10:49165 | RESPONSE | text/xml; | 1173 | +------------+-------------------------------------+-------------------+------------------------------------------------------------------+---------------------+ What do we learn with this view ? Each time, Ann's type a letter in maybe a kind of search engine, an HTTP request is sent to ax.search.itunes.apple.com. Then, the server answers with an XML document encapsulated in an HTTP response message. We will look at this kind of document later. [note : httpdumper can handle multi TCP segments HTTP response and reassemble all segments in one HTTP response if needed.] It is interresting to note that more accurate is the research, more reduced is the size of the XML document. (= less matching results) It seems that "hac" and "hack" lead to the same results. Ok, let's continue ! We want to know now what was the title of the first movie Ann clicked on. If Ann has clicked on a link, her AppleTV may have made an HTTP request. So, again lets display all the uri requested by 192.168.1.10, but this time without any hostname filter: (I've cut all the uri lists, because this output could be really verbose...) franck@ODIN:~/Analysis/Sources/Puzzle_3$ ./httpdumper -r evidence03.pcap --stats uri,192.168.1.10 Reading file evidence03.pcap Parsing packets... 1778 packets read in 4.434 sec. ---------------------------------- Listing URI requested by 192.168.1.10 ---------------------------------- ---------------------------------- Requested to ax.search.itunes.apple.com ---------------------------------- [conv: 2] [flow: 0] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=h [conv: 2] [flow: 2] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=ha [conv: 2] [flow: 4] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=hac [conv: 2] [flow: 6] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=hack ... ---------------------------------- Requested to a1.phobos.apple.com ---------------------------------- [conv: 4] [flow: 0] /us/r1000/038/Video/57/e5/af/mzi.hnmcsmdp.170x170-75.jpg [conv: 4] [flow: 2] /us/r1000/051/Features/a8/de/6e/dj.nofulnci.170x170-75.jpg [conv: 4] [flow: 4] /us/r1000/009/Video/f0/1e/ec/mzl.hhpbkslu.170x170-75.jpg [conv: 4] [flow: 6] /us/r1000/037/Features/71/b1/8c/dj.orlnvciu.170x170-75.jpg [conv: 4] [flow: 8] /us/r1000/032/Features/8e/dc/ca/dj.dzbaqgpw.170x170-75.jpg [conv: 4] [flow: 10] /us/r1000/015/Video/88/d3/62/mzi.xtsujktt.170x170-75.jpg [conv: 4] [flow: 12] /us/r1000/032/Music/f1/33/e0/mzi.kvyqgmsa.170x170-75.jpg [conv: 4] [flow: 14] /us/r1000/032/Music/5c/86/a2/mzi.dutwwfyg.170x170-75.jpg [conv: 4] [flow: 16] /us/r1000/032/Video/f0/48/dd/mzi.pizbdeal.170x170-75.jpg [conv: 4] [flow: 18] /us/r1000/026/Music/68/ec/04/mzi.aydemkgw.170x170-75.jpg [conv: 7] [flow: 0] /us/r1000/032/Video/f0/48/dd/mzi.pizbdeal.enc.jpg?downloadKey2=1265245618_f3a714a27ea9388f7c07104353e1d763 [conv: 7] [flow: 2] /us/r1000/038/Music/2e/10/15/mzi.qdnwlnpu.170x170-75.jpg [conv: 7] [flow: 4] /us/r1000/026/Music/36/fd/a8/mzi.xvqemsit.170x170-75.jpg [conv: 7] [flow: 6] /us/r1000/005/Video/87/fa/5e/mzi.fwmnbval.170x170-75.jpg [conv: 7] [flow: 8] /us/r1000/021/Video/2e/7e/94/mzi.uamugbjf.170x170-75.jpg [conv: 7] [flow: 10] /us/r1000/048/Video/bb/8d/f4/mzi.psezsery.170x170-75.jpg ... ---------------------------------- Requested to metrics.apple.com ---------------------------------- [conv: 1] [flow: 0] /b/ss/applesuperglobal/1/G.6--NS?pageName=US-Movies-Movies-33&pccr=true&h5=appleitmsnatv%2Cappleitmsustv&ch=Movies%20main&g=http%3A%2F%2Fax.itunes.apple.com%2FWebObjects%2FMZStore.woa%2Fwa%2FviewGrouping%3Fid%3D39 [conv: 3] [flow: 0] /b/ss/applesuperglobal/1/G.6--NS?pccr=true&ch=Movies-Search&g=http%3A%2F%2Fax.search.itunes.apple.com%2FWebObjects%2FMZSearch.woa%2Fwa%2FincrementalSearch%3Fmedia%3Dmovie%26q%3Dh&pageName=Movies-Search%20Hints-US&v2=h&h5=appleitmsnatv%2Cappleitmsustv&c2=h [conv: 3] [flow: 2] /b/ss/applesuperglobal/1/G.6--NS?pccr=true&ch=Movies-Search&g=http%3A%2F%2Fax.search.itunes.apple.com%2FWebObjects%2FMZSearch.woa%2Fwa%2FincrementalSearch%3Fmedia%3Dmovie%26q%3Dha&pageName=Movies-Search%20Hints-US&v2=ha&h5=appleitmsnatv%2Cappleitmsustv&c2=ha [conv: 3] [flow: 4] /b/ss/applesuperglobal/1/G.6--NS?pccr=true&ch=Movies-Search&g=http%3A%2F%2Fax.search.itunes.apple.com%2FWebObjects%2FMZSearch.woa%2Fwa%2FincrementalSearch%3Fmedia%3Dmovie%26q%3Dhac&pageName=Movies-Search%20Hints-US&v2=hac&h5=appleitmsnatv%2Cappleitmsustv&c2=hac [conv: 3] [flow: 6] /b/ss/applesuperglobal/1/G.6--NS?pccr=true&ch=Movies-Search&g=http%3A%2F%2Fax.search.itunes.apple.com%2FWebObjects%2FMZSearch.woa%2Fwa%2FincrementalSearch%3Fmedia%3Dmovie%26q%3Dhack&pageName=Movies-Search%20Hints-US&v2=hack&h5=appleitmsnatv%2Cappleitmsustv&c2=hack [conv: 6] [flow: 0] /b/ss/applesuperglobal/1/G.6--NS?pageName=Movie%20Page-US-Hackers-Iain%20Softley-333441649&pccr=true&h5=appleitmsnatv%2Cappleitmsustv&ch=Movie%20Page&g=http%3A%2F%2Fax.itunes.apple.com%2FWebObjects%2FMZStore.woa%2Fwa%2FviewMovie%3Fid%3D333441649%26s%3D143441 [conv: 9] [flow: 0] /b/ss/applesuperglobal/1/G.6--NS?pccr=true&ch=Movies-Search&g=http%3A%2F%2Fax.search.itunes.apple.com%2FWebObjects%2FMZSearch.woa%2Fwa%2FincrementalSearch%3Fmedia%3Dmovie%26q%3Ds&pageName=Movies-Search%20Hints-US&v2=s&h5=appleitmsnatv%2Cappleitmsustv&c2=s [conv: 9] [flow: 2] /b/ss/applesuperglobal/1/G.6--NS?pccr=true&ch=Movies-Search&g=http%3A%2F%2Fax.search.itunes.apple.com%2FWebObjects%2FMZSearch.woa%2Fwa%2FincrementalSearch%3Fmedia%3Dmovie%26q%3Dsn&pageName=Movies-Search%20Hints-US&v2=sn&h5=appleitmsnatv%2Cappleitmsustv&c2=sn [conv: 9] [flow: 4] /b/ss/applesuperglobal/1/G.6--NS?pccr=true&ch=Movies-Search&g=http%3A%2F%2Fax.search.itunes.apple.com%2FWebObjects%2FMZSearch.woa%2Fwa%2FincrementalSearch%3Fmedia%3Dmovie%26q%3Dsne&pageName=Movies-Search%20Hints-US&v2=sne&h5=appleitmsnatv%2Cappleitmsustv&c2=sne [conv: 11] [flow: 0] /b/ss/applesuperglobal/1/G.6--NS?pccr=true&ch=Movies-Search&g=http%3A%2F%2Fax.search.itunes.apple.com%2FWebObjects%2FMZSearch.woa%2Fwa%2FincrementalSearch%3Fmedia%3Dmovie%26q%3Dsneb&pageName=Movies-Search%20Hints-US&v2=sneb&h5=appleitmsnatv%2Cappleitmsustv&c2=sneb [conv: 12] [flow: 0] /b/ss/applesuperglobal/1/G.6--NS?pccr=true&ch=Movies-Search&g=http%3A%2F%2Fax.search.itunes.apple.com%2FWebObjects%2FMZSearch.woa%2Fwa%2FincrementalSearch%3Fmedia%3Dmovie%26q%3Dsnea&pageName=Movies-Search%20Hints-US&v2=snea&h5=appleitmsnatv%2Cappleitmsustv&c2=snea [conv: 12] [flow: 2] /b/ss/applesuperglobal/1/G.6--NS?pccr=true&ch=Movies-Search&g=http%3A%2F%2Fax.search.itunes.apple.com%2FWebObjects%2FMZSearch.woa%2Fwa%2FincrementalSearch%3Fmedia%3Dmovie%26q%3Dsneak&pageName=Movies-Search%20Hints-US&v2=sneak&h5=appleitmsnatv%2Cappleitmsustv&c2=sneak .... ---------------------------------- Requested to ax.itunes.apple.com ---------------------------------- [conv: 0] [flow: 0] /WebObjects/MZStore.woa/wa/viewGrouping?id=39 [conv: 5] [flow: 0] /WebObjects/MZStore.woa/wa/viewMovie?id=333441649&s=143441 [conv: 5] [flow: 2] /WebObjects/MZStore.woa/wa/relatedItemsShelf?ct-id=3&id=333441649&storeFrontId=143441&mt=6 [conv: 13] [flow: 0] /WebObjects/MZStore.woa/wa/viewMovie?id=283963264&s=143441 [conv: 13] [flow: 2] /WebObjects/MZStore.woa/wa/relatedItemsShelf?ct-id=3&id=283963264&storeFrontId=143441&mt=6 By analysing all the uri requested, we could note at least two very explicit ones, requested to ax.itunes.apple.com /WebObjects/MZStore.woa/wa/viewMovie?id=333441649&s=143441 (in conversation 5) and /WebObjects/MZStore.woa/wa/viewMovie?id=283963264&s=143441 (in conversation 13) Without any knowledge of the inner working, the meaning of the "viewMovie" command isn't hard to guess. So let's assume that the first movie selected by Ann was requested in conversation #5 flow #0. This lead us to think that Ann surely received this link in a response preceding conversation #5 and then clicked on it. We also know that the keyword "hack" was searched in conversation #2 and some XML documents have been received after. let's try to dump one of these documents, conversation 2 flow #7 seem to be a good candidate because it's the response received after the full "hack" term search. httpdumper gives you -d options to dump an HTTP response directly in a file (by now the filename is automatically generated) You could use -d with a flow or an entire conversation, in the last case all the HTTP response will be dumped. httpdumper can handle compressed content (Content-Encoding) and will decompress Gzipped content (only gzip or deflate by now) franck@ODIN:~/Analysis/Sources/Puzzle_3$ ./httpdumper -r evidence03.pcap -c 2 -f 7 -d Reading file evidence03.pcap Parsing packets... 1778 packets read in 4.529 sec. Dumping data to disk: 8.18.65.32_80-192.168.1.10_49165-743066.xml Inflating gzipped content We have now a file named 8.18.65.32_80-192.168.1.10_49165-743066.xml. ... CUT ... First information: the DOCTYPE line define the type of XML content described by the file: plist or PropertyList. With some research on the Internet, we could easily found informations of this kind of file. PropertyLists are used by Apple/itunes to describe media content, it is basically a collection of structures in which informations about the media are stored. In our case, these files describe the informations to be displayed on the TV screen, like the movies descriptions, actors, etc... matching the research. We have to go deeper, now, in the XML document itself to find more clues. Going Deeper Part II: the XML Document (Or plist parsing) --------------------------------------------------------- Well, in fact, it is fairly easy to find the answers of the remaining questions when you have dumped the good XML document to disk with httpdumper. You could simply use your favorite text editor and search through the XML the interresting informations about the movie. But, to facilitate this work, I wrote a minimalist tool called: plist.rb. plist.rb take an xml file in entry (a plist), reads it and parses it. Then it will display all the information contained in the xml plist. You could use grep, for example, to view only selected lines or attributes. Now, it seems to be a good idea to search in our dumped xml file a reference to the url with the viewMovie command. Let's do it franck@ODIN:~/Analysis/Sources/Puzzle_3$ ./plist.rb 8.18.65.32_80-192.168.1.10_49165-743066.xml |grep "viewMovie?id=333441649&s=143441" -A 5 -B 5 link-metadata: title: Hackers title2: Released 1998 url-page-type: list url: http://ax.itunes.apple.com/WebObjects/MZStore.woa/wa/viewMovie?id=333441649&s=143441 type: link box-height: 170 url: http://a1.phobos.apple.com/us/r1000/026/Music/68/ec/04/mzi.aydemkgw.170x170-75.jpg From the output above, we could view that our url is associated with a movie. And the title of this movie is : Hackers Now, we want the full URL to the movie trailer, and we know that this information is stored in a key named : preview-url Again with plist and grep: franck@ODIN:~/Analysis/Sources/Puzzle_3$ ./plist.rb 8.18.65.32_80-192.168.1.10_49165-743066.xml |grep preview-url franck@ODIN:~/Analysis/Sources/Puzzle_3$ Hum... it seems that this url is not in this plist file, so maybe we will find it in the xml file received by Ann after she has clicked on the movie link. So in conversation #5 Below are the step, that i took to find the answer: Listing the conv#5 flows: franck@ODIN:~/Analysis/Sources/Puzzle_3$ ./httpdumper -r evidence03.pcap -c 5 Reading file evidence03.pcap Parsing packets... 1778 packets read in 4.274 sec. FLOWS TABLE +------------+-------------------------------------+-------------------+--------------------------------------------------------------------------------------------+---------------------+ | Flow Index | Hosts | HTTP message type | HTTP Request or Content type | HTTP Content Length | +------------+-------------------------------------+-------------------+--------------------------------------------------------------------------------------------+---------------------+ | 0 | 192.168.1.10:49168 -> 8.18.65.67:80 | REQUEST | /WebObjects/MZStore.woa/wa/viewMovie?id=333441649&s=143441 | 0 | | 1 | 8.18.65.67:80 -> 192.168.1.10:49168 | RESPONSE | text/xml | 2278 | | 2 | 192.168.1.10:49168 -> 8.18.65.67:80 | REQUEST | /WebObjects/MZStore.woa/wa/relatedItemsShelf?ct-id=3&id=333441649&storeFrontId=143441&mt=6 | 0 | | 3 | 8.18.65.67:80 -> 192.168.1.10:49168 | RESPONSE | text/xml | 879 | +------------+-------------------------------------+-------------------+--------------------------------------------------------------------------------------------+---------------------+ Dumping flow 1, because it's the response to Ann's request in flow 0 franck@ODIN:~/Analysis/Sources/Puzzle_3$ ./httpdumper -r evidence03.pcap -c 5 -f 1 -d Reading file evidence03.pcap Parsing packets... 1778 packets read in 4.275 sec. Dumping data to disk: 8.18.65.67_80-192.168.1.10_49168-758883.xml Inflating gzipped content Now, requesting 'preview-url' in this new xml file: franck@ODIN:~/Analysis/Sources/Puzzle_3$ ./plist.rb 8.18.65.67_80-192.168.1.10_49168-758883.xml |grep preview-url preview-url: http://a227.v.phobos.apple.com/us/r1000/008/Video/62/bd/1b/mzm.plqacyqb..640x278.h264lc.d2.p.m4v preview-url: http://a227.v.phobos.apple.com/us/r1000/008/Video/62/bd/1b/mzm.plqacyqb..640x278.h264lc.d2.p.m4v preview-url: http://a227.v.phobos.apple.com/us/r1000/008/Video/62/bd/1b/mzm.plqacyqb..640x278.h264lc.d2.p.m4v preview-url: http://a227.v.phobos.apple.com/us/r1000/008/Video/62/bd/1b/mzm.plqacyqb..640x278.h264lc.d2.p.m4v preview-url: http://a227.v.phobos.apple.com/us/r1000/008/Video/62/bd/1b/mzm.plqacyqb..640x278.h264lc.d2.p.m4v preview-url: http://a227.v.phobos.apple.com/us/r1000/008/Video/62/bd/1b/mzm.plqacyqb..640x278.h264lc.d2.p.m4v Finally, a simple step to be sure: franck@ODIN:~/Analysis/Sources/Puzzle_3$ ./plist.rb 8.18.65.67_80-192.168.1.10_49168-758883.xml |grep title title: Hackers title2: 22 Ratings unmodified-title: Hackers item-title: Jonny Lee Miller item-title: Angelina Jolie item-title: Jesse Bradford item-title: Matthew Lillard item-title: Laurence Mason section-title: Actors item-title: Iain Softley section-title: Director item-title: Michael Peyser item-title: Ralph Winter section-title: Producers So the preview-url was: http://a227.v.phobos.apple.com/us/r1000/008/Video/62/bd/1b/mzm.plqacyqb..640x278.h264lc.d2.p.m4v As this is already a long explanation, I will just write the steps and screen ouputs for the last answers. Title of the second movie: franck@ODIN:~/Analysis/Sources/Puzzle_3$ ./httpdumper -r evidence03.pcap -c 13 Reading file evidence03.pcap Parsing packets... 1778 packets read in 4.267 sec. FLOWS TABLE +------------+-------------------------------------+-------------------+--------------------------------------------------------------------------------------------+---------------------+ | Flow Index | Hosts | HTTP message type | HTTP Request or Content type | HTTP Content Length | +------------+-------------------------------------+-------------------+--------------------------------------------------------------------------------------------+---------------------+ | 0 | 192.168.1.10:49176 -> 8.18.65.67:80 | REQUEST | /WebObjects/MZStore.woa/wa/viewMovie?id=283963264&s=143441 | 0 | | 1 | 8.18.65.67:80 -> 192.168.1.10:49176 | RESPONSE | text/xml | 2586 | | 2 | 192.168.1.10:49176 -> 8.18.65.67:80 | REQUEST | /WebObjects/MZStore.woa/wa/relatedItemsShelf?ct-id=3&id=283963264&storeFrontId=143441&mt=6 | 0 | | 3 | 8.18.65.67:80 -> 192.168.1.10:49176 | RESPONSE | text/xml | 907 | +------------+-------------------------------------+-------------------+--------------------------------------------------------------------------------------------+---------------------+ franck@ODIN:~/Analysis/Sources/Puzzle_3$ ./httpdumper -r evidence03.pcap -c 13 -f 1 -d Reading file evidence03.pcap Parsing packets... 1778 packets read in 4.208 sec. Dumping data to disk: 8.18.65.67_80-192.168.1.10_49176-879127.xml Inflating gzipped content franck@ODIN:~/Analysis/Sources/Puzzle_3$ ./plist.rb 8.18.65.67_80-192.168.1.10_49176-879127.xml |grep title title: Sneakers title2: 43 Ratings unmodified-title: Sneakers item-title: Robert Redford item-title: Dan Aykroyd item-title: Ben Kingsley item-title: Mary McDonnell item-title: River Phoenix item-title: Sidney Poitier section-title: Actors item-title: Phil Alden Robinson section-title: Director item-title: Lawrence Lasker item-title: Lindsley Parsons Jr. item-title: Walter F. Parkes section-title: Producers item-title: Phil Alden Robinson item-title: Walter F. Parkes item-title: Lawrence Lasker section-title: Screenwriters The title was : Sneakers What was the price to buy it: franck@ODIN:~/Analysis/Sources/Puzzle_3$ ./plist.rb 8.18.65.67_80-192.168.1.10_49176-879127.xml |grep price-display price-display: $9.99 rent-price-display: $2.99 price-display: $9.99 rent-price-display: $2.99 So the price was: $9.99 We have multiple matches here, anyway they are the same. But we could display more lines with grep to be sure: franck@ODIN:~/Analysis/Sources/Puzzle_3$ ./plist.rb 8.18.65.67_80-192.168.1.10_49176-879127.xml |grep price-display -A 5 -B 5 store-offers: STDQ: price: 9.99 duration: 7518393 size: 1236583962 price-display: $9.99 preview-url: http://a1738.v.phobos.apple.com/us/r1000/011/Video/7f/9d/ce/mzm.gbctwnmq..640x352.h264lc.D2.p.m4v buy-params: productType=V&salableAdamId=283963264&pricingParameters=STDQ&price=9990 action-display-name: Buy preview-duration: 164080.0 screen-format: widescreen -- action-display-name: Rent preview-duration: 164080.0 screen-format: widescreen rent-params: productType=V&salableAdamId=283963264&pricingParameters=SDVOD&price=2990&rental=1 rental-duration: 43200 rent-price-display: $2.99 type: movie English -- flavors: 4:640x480LC-128: price: 9.99 duration: 7518393 size: 1236583962 price-display: $9.99 preview-url: http://a1738.v.phobos.apple.com/us/r1000/011/Video/7f/9d/ce/mzm.gbctwnmq..640x352.h264lc.D2.p.m4v buy-params: productType=V&salableAdamId=283963264&pricingParameters=STDQ&price=9990 action-display-name: Buy preview-duration: 164080.0 screen-format: widescreen -- action-display-name: Rent preview-duration: 164080.0 screen-format: widescreen rent-params: productType=V&salableAdamId=283963264&pricingParameters=SDVOD&price=2990&rental=1 rental-duration: 43200 rent-price-display: $2.99 the keyword action-display-name confirms that the price to BUY it was $9.99. For the last questions, i've already answered it previously in this loooooooong document. Last Words: That was nice ! Jonathan, Sherri, please give us more challenge like this one ! Franck References: =========== TShark : http://www.wireshark.org/docs/man-pages/tshark.html Packetfu http://code.google.com/p/packetfu/ Terminal table http://vision-media.ca/resources/ruby/ruby-terminal-table-gem Ruby doc http://www.ruby-doc.org/ruby-1.9/index.html