When upper management is notified of a data breach, they have to choose between:

a) Announcing publicly and in a timely manner, which would result in major reputational damage, financial drain, loss of business, and potentially huge lawsuits.

b) Keeping quiet and hoping that no one ever finds out (in which case, nothing happens).

Of course, usually upper management doesn’t find out at all. There is little incentive for IT staff to report compromises all the way up the chain, since it just makes them look bad. System administrators fear that if they detect a compromise on their own servers, managers will accuse them of doing a bad job. Also, the breaches have to be detected in the first place– and often security staff are overworked and have limited resources for tuning IDS or following up on alerts.

The bottom line is that no one is motivated to do a good job detecting and publishing breaches– not corporations, not upper management, not IT staff, and in many cases not even security teams themselves. Ethics can hardly compete against real financial incentives and fears for job security.

Don’t Companies Have to Report Breaches?

Many states have data breach notification laws, but these tend to have major loopholes. Importantly, they don’t provide clear guidelines for deciding whether a “security breach” happened. As a result, if an attacker destroys important evidence or if the company does not retain records that would explicitly prove inappropriate access, then the company will probably decide that they are not required to report. Customers affected never even hear that there was concern about a breach in the first place.

The assumption is that the data is secure unless there is explicit evidence which proves otherwise. This is backwards! When log retention creates a liability, companies have reduced incentive to collect or retain detailed records. If we assume the data is secure unless there is proof otherwise, then there is no reason for companies to work to retain evidence.

The irony is that companies with the worst security practices, who do not keep logs or configure IDS systems effectively, are the ones who get off scot-free because they do not collect or retain the evidence of a breach.

What about the proposed federal Data Accountability and Trust Act?
The Data Accountability and Trust Act which passed the US House of Representatives last month does nothing to address this loophole. It requires that “Any person engaged in interstate commerce that owns or possesses data in electronic form containing personal information shall, following the discovery of a breach of security of the system maintained by such person that contains such data…notify each individual…”

OK, so what is a “breach of security”?

“(1) BREACH OF SECURITY- The term `breach of security’ means unauthorized access to or acquisition of data in electronic form containing personal information.”

How do you decide if there has been “unauthorized access to or acquisition of data”? The bill does not provide any guidance. As long as the organization does not keep records which would *prove* that confidential data was accessed or exported, their legal counsel may advise them that they do not have to report. I am not a lawyer, but I have seen this happen repeatedly with respect to existing data breach regulations.

How Can We Fix This Loophole?
Here are some ideas:

• Assume insecurity. Companies should be able to produce access logs and records which confirm that the data has been kept safe, rather than vice versa. This will motivate companies to collect and retain access logs in much greater detail than they do now.
• Proactively audit large organizations that retain lots of personal data.
• Publish yearly certificates based on audit results, the same way health inspectors publish certificates for restaurants. This way the public can decide which companies to give our information to, based on how well they secure it.

Today, the vast majority of security breaches are never reported. When you examine the incentives and the myriad of holes which exist in reporting regulations, it’s easy to understand why. Detailed logging and monitoring practices result in greater liability. Reporting incidents to the public can lead to financial ruin. There’s little incentive for organizations to do a genuinely good job tracking access to confidential data.

In this backward system, it’s a wonder we hear about any breaches at all. The fact that we do hear about data breaches frequently should make you stop and think about the number that are *really* occurring, but are never detected, let alone reported. Speaking from experience, I can tell you that the data breaches you hear about are just the tip of the iceberg.

The MOST ELEGANT solution wins. Good luck!!

“After being released on bail, Ann Dercover disappears! Fortunately, investigators were carefully monitoring her network activity before she skipped town. “We believe Ann may have communicated with her secret lover, Mr. X, before she left,” says the police chief. “The packet capture may contain clues to her whereabouts.”

“YOU are the forensic investigator. Your mission is to figure out what Ann emailed, where she went, and recover evidence…”

http://forensicscontest.com/

Brought to you by the authors of SANS Sec558: Network Forensics:
Sherri Davidoff and Jonathan Ham

We asked you for the most elegant solution. It was possible to solve the puzzle with common tools such as Wireshark, and many people did. However, modern investigations often involve many gigabytes– if not terabytes– of packet data. In the real world, pointing and clicking doesn’t scale. Moreover, when you’re working with large amounts of data, processing time is extremely valuable. Small, fast tools are key.

What we considered “elegant” was the construction of some automated process for solving the puzzle which was easy to use, easy to understand, very portable, and would easily be able to scale to much larger and more difficult problems.

Five people were named Semifinalists because they created an automated process (ie scripting) to facilitate future investigations. Seven Finalists took this to a level beyond and created novel solutions involving considerable amounts of scripting. Please take a look at each of their solutions as WE learned something from every one.

The winner got fame, glory, and a free SANS On-Demand class (worth up to $3500), and the finalists each receive a Fiendish Japanese Pocket Puzzle from Thinkgeek.

We’ve created a dedicated web site, forensicscontest.com, with the solutions and winner. Check it out for the full solutions and names:

http://forensicscontest.com

Be sure to subscribe to the RSS feed. We’ll be posting more contests soon!

We received so many great contest entries, including custom-written tools, that we will be taking a few more days to finish testing all of the code we received. The winner will be announced on the PaulDotCom podcast next Thursday and posted on this site. Great job everybody! Stay tuned.

Remember, the MOST ELEGANT solution wins. We highly encourage coding and automated solutions. You are welcome to submit multiple solutions if you would like to continue to refine your work. Submissions will be accepted through September 10, 2009.

Anarchy-R-Us, Inc. suspects that one of their employees, Ann Dercover, is really a secret agent working for their competitor. Ann has access to the company’s prize asset, the secret recipe. Security staff are worried that Ann may try to leak the company’s secret recipe.

Security staff have been monitoring Ann’s activity for some time, but haven’t found anything suspicious– until now. Today an unexpected laptop briefly appeared on the company wireless network. Staff hypothesize it may have been someone in the parking lot, because no strangers were seen in the building. Ann’s computer, (192.168.1.158) sent IMs over the wireless network to this computer. The rogue laptop disappeared shortly thereafter.

“We have a packet capture of the activity,” said security staff, “but we can’t figure out what’s going on. Can you help?”

You are the forensic investigator. Your mission is to figure out who Ann was IM-ing, what she sent, and recover evidence including:

1. What is the name of Ann’s IM buddy?
2. What was the first comment in the captured IM conversation?
3. What is the name of the file Ann transferred?
4. What is the magic number of the file you want to extract (first four bytes)?
5. What was the MD5sum of the file?
6. What is the secret recipe?

Here is your evidence file:

http://philosecurity.org/558/contest_01/evidence.pcap
MD5 (evidence.pcap) = d187d77e18c84f6d72f5845edca833f5

The MOST ELEGANT solution wins. In the event of a tie, the entry submitted first will receive the prize. Scripting is always encouraged. All responses should be submitted as plain text files.

Exceptional solutions may be incorporated into the SANS Network Forensics Toolkit. Authors agree that their code submissions will be freely published under the GPL license, in order to further the state of network forensics knowledge. Exceptional submissions may also be used as examples and tools in the Network Forensics class. All authors will receive full credit for their work.

Email submissions to contest@philosecurity.org. Deadline is 9/10/09. Good luck!!

cheers,
Sherri

“No Hard Drive? No Problem!” SANS Network Forensics (Sec558)

A lot has been written about methods for “fingerprinting” systems with active scanning methods (eg. nmap). These of course require that the system be actively reachable, and that you don’t mind totally giving away your position with a very noisy scan (sort of like shooting a shotgun directly at a suspect to see if you can get him to look at you, in hopes that you’ll catch a glimpse of his face).

A lot has also been written about more covert ways of achieving the same goal, based on packets surreptitiously captured from the host of interest (a la p0f). This is certainly very cool, and can be inordinately useful…if you happen to have packet captures from the host of interest, or can begin to get them. (Either you were capturing its packets to begin with, or it’s still around to get packets from.)

But what if the system is long gone, never to return? Or what if you’re lucky enough to see it again, but for technological/logistical/legal reasons you can’t grab its packets? As we see in Sec558, all hope is not lost…

While most firewalls report only sparse information about the packets that they see (and perhaps reject), many of them at least include such information as the Time To Live (TTL) field. What a lot of forensic analysts don’t realize is that different operating systems choose different initial values for the TTL field. For example, current versions of Linux start with 64, and Windows with 128. So if you see a packet logged by a firewall with a TTL of 61, it’s a pretty good guess that it came from a Linux system 3 hops from the firewall. Of course it could be a Windows system 67 hops away, but which is more likely?

TTLs can be, and sometimes are, crafted. But when dealing with the 99% of packets whose headers aren’t crafted, this works like a charm. You can also correlate TTLs with other aspects of the network traffic logged by a firewall, such as source and destination port numbers, IP ID sequences, and such.

Here are three lines from an iptables firewall log. Can you guess what OS the client is running? How about the manufacturer?

Mar 24 12:13:13 192.168.1.10 kernel: [ 915.256256] FIREWALL:BLOCKEDIN=eth3 OUT= MAC=ff:ff:ff:ff:ff:ff:00:21:70:4d:4f:ae:08:00 SRC=192.168.1.170 DST=192.168.1.255 LEN=96 TOS=0×00 PREC=0×00 TTL=128 ID=61495 PROTO=UDP SPT=137 DPT=137 LEN=76 Mar 24 12:13:14 192.168.1.10 kernel: [ 916.006952] FIREWALL:BLOCKEDIN=eth3 OUT= MAC=ff:ff:ff:ff:ff:ff:00:21:70:4d:4f:ae:08:00 SRC=192.168.1.170 DST=192.168.1.255 LEN=96 TOS=0×00 PREC=0×00 TTL=128 ID=61496 PROTO=UDP SPT=137 DPT=137 LEN=76 Mar 24 12:13:14 192.168.1.10 kernel: [ 916.764653] FIREWALL:BLOCKEDIN=eth3 OUT= MAC=ff:ff:ff:ff:ff:ff:00:21:70:4d:4f:ae:08:00 SRC=192.168.1.170 DST=192.168.1.255 LEN=96 TOS=0×00 PREC=0×00 TTL=128 ID=61497 PROTO=UDP SPT=137 DPT=137 LEN=76

With a TTL of 128, this is probably a Windows system 0 hops away (meaning it has not traversed a router, so it is on the local segment). This is further supported by the UDP port 137 (NETBIOS) traffic, which is very common for Windows systems. The sequential IP IDs tend to corroborate this as well.

Based on the first three bytes of the MAC address (“00:21:70″), it’s probably a Dell.

Jonathan Ham is an independent security consultant and a SANS Certified Instructor, who teaches forensics and other tracks. When he goes to sleep at night, he counts packets as they leap through firewalls.
 

Chances are pretty good that you’re reading this page through a web proxy right now, especially if you’re in an enterprise environment. Web proxying and caching have become increasingly popular, for both filtering traffic and speeding up requests. Even consumer ISPs have latched onto the idea (sometimes using similar techniques to insert ads into pages as they are downloaded). That means your web surfing history is probably being recorded in a proxy log somewhere.

Web proxy and cache servers are untapped gold mines for forensic analysts. They often record the web browsing history for an entire organization, all rolled up into one directory. Web caching servers also contain copies of pages themselves, for a limited time.

This is great for forensic analysts (and not so hot from a privacy perspective). Investigators can examine web browsing histories for everyone in an organization all at once. Moreover, it’s possible to reconstruct web pages from the cache. Right now, investigators often simply visit web sites in order to see what they are. This has some serious drawbacks: first, there is no guarantee you’re seeing what the end user saw earlier; and second, your surfing now appears in the server’s activity logs. If the owner of the server is an attacker or suspect, you may well have just tipped them off. It’s much better to first examine the web cache to see what you can find stored locally.

To learn more, I installed Squid, a popular web proxy/cache server, on my lab network and dissected it. There are a number of tools out there that will reconstruct client browsing history, based the access logs. I really liked squidview (which has a Kismet-style interface) and sarg (HTML clickable).

What I didn’t find was public information or tools for reconstructing pages from the web cache. It’s definitely possible. The proxy cache, by its very nature, stores the pages you view on its local hard drive and may later serve those pages to you or someone else. The precise pages it stores and the length of time they are retained vary depending on the specific server configuration and usage.

As a forensic analyst, I wanted to recover those cached pages. I figured, if Squid could do it, so could I.

By changing Squid’s configuration to “offline” mode, you can use wget to extract some pages directly from the local cache. This is handy because it reconstructs the pages automatically, if they exist. However, I wanted to see what information was stored directly in the cache, and access associated headers and metadata.

Squid’s access log is straightforward: it’s essentially a text file which contains a list of client IP addresses and pages accessed. If you correlate these with DHCP and central authentication logs, you can potentially match web surfing activity to a particular network card or user.

The cache directory is far more mysterious. If you simply list the directory contents, here is what you will see:

$ ls
00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F swap.state

Daunting. That swap.state file is Squid’s database, which contains a record of every item in the cache. It’s a binary file. If you delete it while Squid isn’t running, Squid will actually re-create it the next time it starts up. (This is helpful if you’re trying to manually edit the Squid cache in order to create lab exercises for, oh, a new class on network forensics.)

Within each of those subdirectories are files such as these:

And each of those subdirectories contains files such as this:

Finally, each of those eight-character files contains- yes! – the pages actually cached by Squid. Here is an example. When you surf to a web page, Squid will add some metadata to the top, which includes the full URI and its MD5sum. Squid then stores this, along with the full HTTP reply (headers and body) as a file in one of these subdirectories. If the page is requested later, it can look it up in swap.state and fetch it.

Now let’s extract some content directly from the cache.

Let’s say we’re analyzing web traffic associated with 192.168.1.26. We come across the following entry in Squid’s access.log:

1239739309.653 377 192.168.1.26 TCP_MISS/200 30348 GET http://finickypenguin.files.wordpress.com/2007/10/1161451564593.jpg – DIRECT/72.233.69.12 image/jpeg

Interesting… What is this image? Let’s see if it’s in the cache.

We could analyze swap.state, but I created my own table of the URIs stored in Squid, along with their corresponding cache files. This was for two reasons: first, I didn’t have to rely on the accuracy of Squid’s database; and second, I’m a lazy bum and it’s pretty easy to do using a simple Bash script. The URI is stored near the beginning of each cached page, just after the MD5sum of the URI. If you grep for strings beginning with “http” in the first few lines of each cache file, you’ll find it.

Here’s that file we were looking for:
./00/03/0000036A    http://finickypenguin.files.wordpress.com/2007/10/1161451564593.jpg

Now let’s open up that cache file. Running strings on it, we see the following metadata and header info:

Lots of juicy info there. To extract the image itself, let’s open this up in a hex editor. I like to use “bless” on Ubuntu. JPEG images begin with “FFD8,” so extracting this content is fairly easy. Highlight everything before the magic number, click “Cut” and save as 0000036A-edited.jpg.

A quick check with “file” confirms that we got it right:
$ file 0000036A-edited.jpg
0000036A-edited.jpg: JPEG image data, JFIF standard 1.01

Now let’s open it up:

Looks pretty suspicious to me…
 

Forensic analysts traditionally focus on hard drive analysis, but hard drives are not always accessible, and they don’t tell the full story. Savvy investigators also include the network environment. Recently I’ve been co-authoring a class on Network Forensics (SANS Sec558), and I’ve been building lab networks, collecting evidence for the class exercises, and practicing network forensics in-depth. Here are a few ways that investigators reconstruct computer activity using network forensics.

Web Surfing: Many organizations use web proxies to improve web surfing performance. As it happens, web proxies maintain a log of web requests and even copies of web pages (for a limited period of time). Forensic investigators can use graphical tools such as Sarg to analyze web proxy logs and view a list of client’s browsing history. Investigators can even extract pages themselves from the web proxy cache with common tools such as wget.

By analyzing web proxy logs, investigators can reconstruct browsing history, view downloads, recover social networking information, identify external email accounts, and even obtain usernames and passwords (which are sometimes included in URLs). Web proxies result in faster web browsing and help lower network congestion. As a side effect, they also accumulate logs which record your web browsing history.

Laptop/Mobile Device Tracking: Investigators can identify a specific laptop even when its IP address changes, and even if it moves to a different network. The network card in your computer has a unique address assigned to it by the manufacturer called the Media Access Control (MAC) address. When you connect your laptop to a hotspot or company network, your network card broadcasts its unique MAC address, and the DHCP server then assigns an IP address to your network card.

Forensic analysts can use DHCP server logs to track which IP addresses belonged to which network cards at a given time. By default, your MAC address also reveals information about the manufacturer, so forensic analysts can infer whether your laptop contains, for example, an Apple-manufactured network interface.

There’s a catch: You can change your network card’s MAC address. It’s actually fairly easy to do, even though most people don’t bother. A MAC address is really about as reliable as using hair color to describe a suspect. Much of the time, it’s accurate, and it takes conscious effort to change– but with a little effort it can be modified. You can even change the MAC address so that it looks as though your network card was made by a different manufacturer. If someone purposefully changed their MAC address, that would make it harder to link network activity to their specific network card.

Logon History: In order to comply with regulations such as HIPAA (as well as standard security best practices), organizations frequently configure workstations to send records of events to central logging servers, which collect and store logs from many workstations all in one place. Typically, central logging servers store login times, failed login attempts, and commands that are executed with administrative privileges (the specifics vary depending on the organization). The workstation must be preconfigured to send logs to the central logging server. My favorite log analysis tool is Splunk. By analyzing a central logging server, forensic investigators can track when you logon (or when you mess up your password), when you run privileged commands, or take other noteworthy actions.

Network traffic: Forensic investigators who are monitoring active connections can collect all network traffic to and from a specific computer, without the user ever knowing. There are many ways to monitor network traffic. If investigators have the support of network administrators, then the administrators can simply set up a SPAN port on a router, mirror all traffic, and filter for interesting bits. Some companies do this all the time, filtering for malicious traffic, proprietary data, or even keywords that might indicate employees are angry. Wireless networks are even easier to analyze. Since wireless access points are hubs, every client can potentially capture traffic destined for any other system– or all systems. Tools such as Wireshark and tcpdump are useful for capturing network traffic (investigators: if you use tcpdump, remember to set the snaplength to zero for full packet contents).

Here are a few things forensic investigators can do with raw traffic captures:

• File carving: Investigators can actually carve files out of raw network traffic and reconstruct file transfers. If you upload a JPG to a web site, send an email attachment, or download an MP3, anyone who has captured your network traffic can reconstruct your file. Tools such as tcpxtract are helpful for this purpose. Investigators can also view images and other file formats in real time as they are transferred across the network, using tools like driftnet.
• Instant message reconstruction: If you’re not encrypting your instant messages, then they are quite easy to see as they travel across the network. One of my clients once half-jokingly said that he considered deploying a scrolling sign in the lunchroom which broadcast everybody’s IMs, in order to reduce the amount of IM usage.
• Email reconstruction: Emails are rarely encrypted as they traverse the network. Much like instant messages, the text is trivial to read. Investigators don’t even need to go to the trouble of reconstructing files: you can simply run “strings” on raw packet captures and dump the output to a file (I recommend always checking both ASCII and Unicode output). If you’re feeling more interactive, you can also view the raw traffic in a hex editor and read the ASCII output.
• Web surfing reconstruction: Perhaps your organization doesn’t have a proxy server, or the forensic investigator doesn’t have access to it. With access to captured traffic from your computer, investigators can extract your web browsing activity, full page content, and form submissions.

Forensics and privacy are two sides of the same coin. Both investigators and everyday citizens benefit from understanding the types of personal information that companies, hotspots and ISPs routinely store, and how activity can be tracked and reconstructed.

Check out our three-day class: SANS Sec558: Network Forensics, scheduled to run this June at SANSFIRE in Washington, DC. We’ll do lots of advanced, hands-on exercises in which we analyze a virtual network, and spend a full day working as investigative teams to solve a crime. Hope to see some of you there!

Sherri Davidoff

PGP-signed text: 2009-03-16 (current)

Did you like this article? Share it!

]]> http://philosecurity.org/2009/03/16/beyond-hard-drive-forensics/feed 1 http://philosecurity.org/2009/03/02/national-drug-intelligence-center-keeps-hashes http://philosecurity.org/2009/03/02/national-drug-intelligence-center-keeps-hashes#comments Mon, 02 Mar 2009 05:20:27 +0000 sherri http://philosecurity.org/?p=1087 The National Drug Intelligence Center has developed software called (ahem) “HashKeeper” “as its principal tool to expedite the analysis of electronic media.”

Hahahaha…..

Apparently, “HashKeeper is available free of charge.” Contact the National Drug Intelligence Center for more information.

National Drug Intelligence Center
c/o Mr. Steve Gironda
Telephone: 814-532-4987
E-mail: ndic.domex.request@usdoj.gov

Hat tip to John Masterson.

Sherri Davidoff

PGP-signed text: 2009-03-01 (current)

Did you like this article? Share it!

]]> http://philosecurity.org/2009/03/02/national-drug-intelligence-center-keeps-hashes/feed 3