Network Forensics Puzzle Contest

At last, the long-awaited Puzzle #3 winners! Thank you all for your terrific submissions, and your patience as we tested each one carefully. Congratulations to everyone who sent in the correct answers.

As always, we were tremendously impressed by the quality of the entries. We received a wide variety of creative, original submissions, including file carving tools, network-layer tools, HTTP, XML and Plist analysis tools, graphical tools, command-line tools, and more. It was very hard to narrow down a winner, and there were several production-quality tools which will now be covered in future SANS “Network Forensics” curriculum. Please check out all the Finalist submissions!

The winner is… Matt Sabourin, for his elegant tool, “findappletv.py“. Matt’s tool is simple to use. It parses a pcap and creates a report for each potential AppleTV client, containing “Search Terms Sent by Client,” “Movie Items Viewed by Client,” “Overview of Recognized Requests,” and more. It also creates an overview report for all clients. Each of these reports can easily be included in the appendix of a professional forensics report. We could definitely envision using this in a real forensics case to quickly summarize AppleTV usage information. Congratulations, Matt! Your AppleTV is on it’s way.

We’d also like to call attention to several other submissions (in no particular order):

Amar Yousif created two excellent tools: applejuice and gzippedNOT. Amar’s “gzippedNOT” parses gzipped content out of HTTP responses. This tool will be AWESOME for squid proxy analysis as well. “Applejuice” dumps out the list of search queries for each AppleTV IP address. “Applejuice” also wins the Best Name Award!

Richard Springs built two great tools: transmute.rb and scarabsieve.rb. Scarabsieve parses through any Webscarab-logged traffic, carves it all out, dumps it into a directory, and prints MD5 and SHA1 hashes for each carved file. This script alone is very useful for any WebScarab user. Richard also wrote “transmute.rb” to convert any pcap into the WebScarab log format so that scarabsieve can parse it. Wow! Nice work.

Sébastien Damaye built a tool called “pyHttpXtract.py” to extract all the files in the packet capture and list out the search requests. This tool even goes a step above and automatically creates a graphical web interface which you can scroll through to view all the files. He also submitted a companion tool, webObjects.py, which pulls AppleTV searches out of the packet capture and prints them out. Sébastien included a *fantastic* writeup which everybody should read. We were really impressed.

Franck Guénichot lived up to his reputation as network forensics hacker extraordinare with his excellent tool, “httpdumper.” This tool displays HTTP conversations, filters and dumps the contents (automatically decompressing gzipped content). Franck also submitted two handy tools, macfinder.rb, and plist.rb. Franck’s writeup is very thorough– definitely check it out for a great walk-through of the solutions.

Tom Samstag wrote a really cool tool, httpAnalyzer, which creates a graphical web interface that lets you browse through HTTP traffic. It includes MD5 and SHA1 hashes of each file contained in the packet capture. The interface is very user-friendly! Tom’s httpAnalyzer is easily extensible, and we hope we’ll see it again in future contests.(Note: When you load the page, httpAnalyzer makes a request to jQuery.com, apparently in order to get up-to-date jQuery Javascript library. If you are using it for forensics work, you’ll want to block outbound traffic.) Tom also wrote a very handy tool called “trafficAnalyzer.sh,” which analyzes a pcap and reports basic info such as a packet count, MAC addresses and IP addresses.

Lou Arminio built a Plist parser to analyze Apple plist files, as well as an HTTP analyzer called “httpparse”. On top of that, he created a great tool called pcaputil which analyzes TCP flows and carves files out of selected TCP flows and creates MD5sums. These are three handy little tools. Nice work!

Michael_Nijs built upon an open-source pcap analysis tool, read_pcap.py, adding the option to parse GET and POST requests and display the values of any parameter in the URL. We appreciated that he leveraged existing code and built a useful extension.

Alan Tu wrote a script, http_analysis.pl, which leverages tshark’s powerful HTTP dissection capability, outputs handy information to a file, and can also produce filtered pcaps. Alan also wrote an HTTP response extractor, http_rx.pl, and polished his TCP stream analysis tool, stream.pl. Check them out!

Wesley McGrew wrote an excellent tool, “atvsnarf.py,” which carves out plist files and creates a CSV file with useful information about AppleTV traffic from a pcap. The tool is very easy to use, and a great foundation for detailed forensic analysis. His writeup is outstanding, too– read about how he identified six request types from the pcap file, and incorporated these into atvsnarf.py’s output.

These tools are great! Thank you all for making your work available to the community. We hope you’ll continue to maintain and extend your code.

Many thanks to everyone who participated. We hope to see you guys in future contests.

Here are the answers for Puzzle #3. Big thanks to everyone who entered!

Just wanted to send a hint out for those of you who are out to win Ann’s AppleTV.

We’ve received lots of submissions with the correct answer, but to win the AppleTV, you’ll need to go a step beyond manual extraction with Wireshark or Network Miner. Imagine if you had a huge packet capture containing LOTS of AppleTV traffic. There’s no way you could extract that manually!

Can you build a tool that will automatically list each of the movies that a user previewed? Or all of the terms that Ann searched for? Carve out files transferred and their MD5sums? Even perhaps reconstruct what Ann saw on the AppleTV based on the traffic content?

To win the AppleTV, you’ll need to be creative and take things to a level beyond just manual extraction. (By the way, we suspect that the underlying traffic for the AppleTV is the same format as iTunes traffic.)

Submissions are due by the end of 2/1/10 (next Monday night). Good luck!!

Ann and Mr. X have set up their new base of operations. While waiting for the extradition paperwork to go through, you and your team of investigators covertly monitor her activity. Recently, Ann got a brand new AppleTV, and configured it with the static IP address 192.168.1.10. Here is the packet capture with her latest activity.

You are the forensic investigator. Your mission is to find out what Ann searched for, build a profile of her interests, and recover evidence including:

1. What is the MAC address of Ann’s AppleTV?
2. What User-Agent string did Ann’s AppleTV use in HTTP requests?
3. What were Ann’s first four search terms on the AppleTV (all incremental searches count)?
4. What was the title of the first movie Ann clicked on?
5. What was the full URL to the movie trailer (defined by “preview-url”)?
6. What was the title of the second movie Ann clicked on?
7. What was the price to buy it (defined by “price-display”)?
8. What was the last full term Ann searched for?

Prize: Ann’s AppleTV (of course!)

Deadline is 2/01/10 (11:59:59PM UTC-11) (In other words, if it’s still 2/01/10 anywhere in the world, you can submit your entry.)

Please use the Official Submission form to submit your answers. Here is your evidence file:
http://forensicscontest.com/contest03/evidence03.pcap
MD5 (evidence03.pcap) = f8a01fbe84ef960d7cbd793e0c52a6c9

The MOST ELEGANT solution wins. In the event of a tie, the entry submitted first will receive the prize. Coding is always encouraged. We love to see well-written, easy-to-use tools which automate even small sections of the evidence recovery. Graphical and command-line tools are all eligible. You are welcome to build upon the work of others, as long as their work has been released under a GPL license. (If it has been released under another free-software license, email us to confirm eligibility.) All responses should be submitted as plain text. Microsoft Word documents, PDFs, etc will NOT be reviewed.

Feel free to collaborate with other people and discuss ideas back and forth. You can even submit as a team (there will be only one prize). However, please do not publish the answers before the deadline, or you (and your team) will be automatically disqualified. Also, please understand that the contest materials are copyrighted and that we’re offering them publicly for the community to enjoy. You are welcome to publish full solutions after the deadline, but please use proper attributions and link back. If you are interested in using the contest materials for other purposes, just ask first.

Exceptional solutions may be incorporated into the SANS Network Forensics Toolkit. Authors agree that their code submissions will be freely published under the GPL license, in order to further the state of network forensics knowledge. Exceptional submissions may also be used as examples and tools in the Network Forensics class. All authors will receive full credit for their work.

Deadline is 2/01/10 (11:59:59PM UTC-11). Here’s the Official Submission form. Good luck!!

Copyright 2009, Lake Missoula Group, LLC. All rights reserved.

We were blown away by the quality of your submissions for Puzzle #2. There were many excellent, automated, well-documented solutions, including production-quality tools. Congratulations to everyone who submitted the correct answers, and a special thanks to all of you who pushed forward network forensics technology, either by writing your own tools or by improving those which already exist.

You sent in nearly 150 unique entries. After testing each entry for usability and functionality, we narrowed it down to 79 correct solutions, 15 Semifinalists, and 8 Finalists. After much debate we declared TWO (yes, two) winners, with different and complementary approaches:

Franck Guénichot and Jeremy Rossi

Both Franck and Jeremy will receive a Lenovo Ideapad S10-2, similar to the netbooks that will be distributed in SANS Sec558 classes.

Franck wrote two tools:
- smtpdump (home made ruby script to extract some smtp info from a pcap file)
- docxtract (home made ruby script to extract files from a docx package)

Franck’s smtpdump is an easy-to-use tool for analyzing SMTP traffic in pcap files. It can export emails and attachments, automatically generate MD5sums, and display SMTP-related information. You can narrow your search down to a specific flow, or extract information from the entire packet capture. The docxtract script extracts files from a Microsoft .docx file, and can take the MD5sum of each extracted item. We especially appreciated that both of Franck’s tools were very well documented and user-friendly.

Jeremy wrote a fantastically simple tool called findsmtpinfo.py. As he describes, the “script creates a report of the SMTP information, stores any emails in msg format, stores any attachments from the emails, decompresses them if they are a compressed format (zip, docx), checks MD5 hashes of all files including the msg files (and generates MD5 hash of output report).” The result? An easy-to-follow report with complete paths to the extracted files and corresponding MD5sums. The report itself is detailed enough to be used as an attachment to a real-world forensics report.

Franck and Jeremy’s tools, smtpdump and findsmtpinfo.py, complement each other well. They can be used individually or together as part of a real-world investigation. Smtpdump facilitates inspection and makes it easy to drill down on the SMTP traffic of interest. Once you have identified specific flows of interest, you can use findsmtpinfo.py to automatically generate a report and quickly extract all of the SMTP-related information, emails, attachments, etc.

Don’t miss the excellent tools and narratives by the eight Finalists. We’d like to specifically call attention to Erik Hjelmvik’s latest version of Network Miner, which he submitted as his entry. Erik extended Network Miner to include an SMTP parser and “Messages” tab. His GUI tool is both effective and very easy to use.

Amar Yousif (smtpcat), Jeff Jarmoc (smtpcat.rb), Kristinn Gudjonsson (smtp_anex), Richard Springs (carnivorous.rb) and Serge Gorbunov (smtpParser.py) each wrote their own excellent SMTP analysis and data extraction tools. Tom Samstag submitted patches for dsniff and mailsnarf which substantially improved their functionality, fixing dsniff’s SMTP authentication decoding and allowing mailsnarf to examine traffic on port 587. Alan Tu wrote a great walk-through using tshark’s new tcp.stream field to identify TCP streams, and created a script based on this to output data from the application layer of selected streams.

As before, what we considered “elegant” was the construction of some automated process for solving the puzzle which was easy to use, easy to understand, portable, and would easily be able to scale to much larger and more difficult problems.

We received a number of solutions which were almost, but not quite, correct. For example, several people submitted MD5sums and left out one or two digits, or submitted email addresses with a “1″ instead of an “l”. In forensics, exactness matters, and unfortunately being off-by-one is still not correct. If your name is not on the list of correct answers, please check your submission carefully. We appreciated *every* submission, and encourage you to try again next time!

Fifteen people were named Semifinalists because they contributed to an automated process that would significantly facilitate future investigations. Eight Finalists took this to a level beyond and created polished, novel solutions involving considerable amounts of scripting. Please take a look at each of their solutions as WE learned something from every one.

Thank you all for playing! Puzzle Contest #3 will be coming out soon…

The winners for Puzzle #2 will be announced tonight on PaulDotCom. The show starts at 7:30PM EST. We’ll have all the results posted here shortly thereafter. Talk to you soon!

Thank you all for your contest submissions! We received well over a hundred and we are busily reviewing them. In the meantime, here are the answers:

1. What is Ann’s email address?
Answer 1: sneakyg33k@aol.com

2. What is Ann’s email password?
Answer 2: 558r00lz

3. What is Ann’s secret lover’s email address?
Answer 3: mistersecretx@aol.com

4. What two items did Ann tell her secret lover to bring?
Answer 4: A fake passport and a bathing suit

5. What is the NAME of the attachment Ann sent to her secret lover?
Answer 5: secretrendezvous.docx

6. What is the MD5sum of the attachment Ann sent to her secret lover?
Answer 6: 9e423e11db88f01bbff81172839e1923

7. In what CITY and COUNTRY is their rendez-vous point?
Answer 7: Playa del Carmen, Mexico

8. What is the MD5sum of the image embedded in the document?
Answer 8: aadeace50997b1ba24b09ac2ef1940b7

The Contest #2 deadline has been EXTENDED to 11/22/09 (11:59:59PM UTC-11). That’s a whole extra week to polish up your code The winner will receive a Lenovo IdeaPad S10-2 – just like the free netbooks Sec558 students will get in Orlando.

We’re pleased to announce the PRIZE for Network Forensics Contest #2:

A Lenovo IdeaPad S10-2 – just like the free netbooks Sec558 students will get in Orlando!

The MOST ELEGANT solution wins. Deadline is 11/15/09 11/22/09. Good luck!!

After being released on bail, Ann Dercover disappears! Fortunately, investigators were carefully monitoring her network activity before she skipped town.

“We believe Ann may have communicated with her secret lover, Mr. X, before she left,” says the police chief. “The packet capture may contain clues to her whereabouts.”

You are the forensic investigator. Your mission is to figure out what Ann emailed, where she went, and recover evidence including:

1. What is Ann’s email address?
2. What is Ann’s email password?
3. What is Ann’s secret lover’s email address?
4. What two items did Ann tell her secret lover to bring?
5. What is the NAME of the attachment Ann sent to her secret lover?
6. What is the MD5sum of the attachment Ann sent to her secret lover?
7. In what CITY and COUNTRY is their rendez-vous point?
8. What is the MD5sum of the image embedded in the document?

Please use the Official Submission form to submit your answers. Prize TBD. Prize will be a Lenovo IdeaPad S10-2 – just like the free netbooks Sec558 students will get in Orlando.

Here is your evidence file:

http://forensicscontest.com/contest02/evidence02.pcap
MD5 (evidence02.pcap) = cfac149a49175ac8e89d5b5b5d69bad3

The MOST ELEGANT solution wins. In the event of a tie, the entry submitted first will receive the prize. Scripting is always encouraged. We love to see well-written, easy-to-use tools which automate even small sections of the evidence recovery. You are welcome to build upon the work of others, as long as their work has been released under a GPL license. (If it has been released under another free-software license, email us to confirm eligibility.) All responses should be submitted as plain text. Microsoft Word documents, PDFs, etc will NOT be reviewed.

Exceptional solutions may be incorporated into the SANS Network Forensics Toolkit. Authors agree that their code submissions will be freely published under the GPL license, in order to further the state of network forensics knowledge. Exceptional submissions may also be used as examples and tools in the Network Forensics class. All authors will receive full credit for their work.

Deadline is 11/15/09 11/22/09. Here’s the Official Submission form. Good luck!!