Category Archives: Analysis

Detecting Heartbleed Traffic

The big news in the tech industry this week is The Heartbleed Bug, a vulnerability that affects a large portion of secure web sites on the Internet. I updated the Wireshark and WinPcap web sites on Monday (along with reissuing and revoking certificates) shortly after OS patches were released.

Our web sites are protected going forward, but what about the past? We have a Shark appliance in our environment but that leads to a challenge. We had about 350 GB of HTTPS on our network on Monday alone. This is just slightly too large to load into Wireshark.

Fortunately one of my coworkers (P.J. Malloy) came up with a BPF filter that matches Heartbleed traffic:

http://www.riverbed.com/blogs/Retroactively-detecting-a-prior-Heartbleed-exploitation-from-stored-packets-using-a-BPF-expression.html

Applying this filter directly on the Shark appliance gave me a much smaller number of packets which I could easily analyze in Wireshark. So far I haven’t found anything suspicious.

Troubleshooting the hidden dangers of TCP’s Nagle algorithm and delayed acknowledgement

As we all know, TCP/IP is a great protocol suite.  However, there are times when it can become the bottleneck.  This is especially true if you use TCP/IP for real time transactions where small data sizes are the norm (think financial institutions).  In this session, I’ll show you why Nagle algorithm and delayed acknowledgement was developed.  But more importantly, I’ll highlight the unintended consequences when the two features interact – badly – with each other.  After watching this session, you will be able to spot the hidden dangers of using TCP/IP for real time transactions.   Enjoy, and as always, I would really appreciate your feedback and suggestions. Here is the video:

And as always, any and all feedback and suggestion are welcome.  Thank you and Enjoy!

Hansang Bae

Antivirus Outbreak

Wednesday, August 4, 8:00 AM

We receive a phone call from someone complaining about “Wireshark Antivirus”. I take the call. The person on the other end isn’t able to provide many details other than that a program named “Wireshark Antivirus” is displaying some a shield and directing him to the cacetech.com web site.

This is new. We’ve been on the receiving end of a few false positives in the past but this is new. Some jackass is using our name do do harm.

This will not end well. Continue reading

T-Mobile: Clever or Insane?

I recently got an Android phone. After downloading the Android SDK I noticed that my cellular provider (T-Mobile) was doing something odd. According to the netcfg command they’re using 25.0.0.0/8 on their GPRS/EDGE network:

$ netcfg
lo       UP    127.0.0.1       255.0.0.0       0x00000049
dummy0   DOWN  0.0.0.0         0.0.0.0         0x00000082
rmnet0   UP    25.130.205.212  255.255.255.252 0x00001043
rmnet1   DOWN  0.0.0.0         0.0.0.0         0x00001002
rmnet2   DOWN  0.0.0.0         0.0.0.0         0x00001002
sit0     DOWN  0.0.0.0         0.0.0.0         0x00000080
ip6tnl0  DOWN  0.0.0.0         0.0.0.0         0x00000080

T-Mobile doesn’t own that netblock. The UK Ministry of Defence does. Why would they do such a thing? After all, RFC 1918 gives you three whole blocks (10.0.0.0/8, 172.16.0.0/12, and 192.168.0.0/16) to do with as you please. Straying from those on your private will damn you to an eternity of network flakiness and give your twisted pair cabling scurvy, right?

Why this is clever

According to several BGP looking glasses and figure 5 of Geoff Huston’s IPv4 Address Report the Ministry of Defence doesn’t advertise any routes for 25.0.0.0/8. That means that none of the 25.x.x.x addresses are being used on the public Internet. If you’re on a private network they’re effectively free for the taking. But still, why aren’t they using the officially-sanctioned RFC 1918 address?

My phone also has an 802.11 interface. Let’s take a look at netcfg’s output when I’m connected to T-Mobile’s network and my home network:

$ netcfg     
lo       UP    127.0.0.1       255.0.0.0       0x00000049
dummy0   DOWN  0.0.0.0         0.0.0.0         0x00000082
rmnet0   DOWN  25.130.205.212  255.255.255.252 0x00001002
rmnet1   DOWN  0.0.0.0         0.0.0.0         0x00001002
rmnet2   DOWN  0.0.0.0         0.0.0.0         0x00001002
sit0     DOWN  0.0.0.0         0.0.0.0         0x00000080
ip6tnl0  DOWN  0.0.0.0         0.0.0.0         0x00000080
eth0     UP    192.168.25.4    255.255.255.0   0x00001043

See the 192.168.25.4? That could just as easily be 10.0.0.4, 172.18.34.4, or any other RFC 1918 address. On many networks (particularly universities) it could even be a public address. T-Mobile has no way of predicting or controlling what happens on that interface. The 25.0.0.0/8 netblock has the following advantages:

  • It doesn’t overlap with any other network, public or private. Therefore you won’t get any routing confusion when the phone is connected on GPRS/EDGE and WiFi at the same time.
  • It’s not in public use. The next Facebook or Lolcats isn’t going to show up with a 25.x.x.x address, thereby causing routing confusion for your users.
  • Even if the UK MoD is handing out 25.x.x.x address over 802.11 they’re way over in the UK. It’s unlikely that my phone will be connected to the MoD and T-Mobile networks at the same time.

Why this is insane

IPv4 addresses are getting scarce. Who says the MoD won’t turn the 25.0.0.0/8 netblock over to RIPE or IANA next week? Even then my phone has to go through a proxy server on T-Mobile’s network so it’s probably not a huge deal.

Update

Just before publishing this I ran netcfg and my phone was using 14.64.186.160. The 14.0.0.0/8 netblock used to be reserved for public data networks but was allocated to APNIC earlier this month. I wonder what other questionable netblocks they’re using.

Troubleshooting A Slow Web Site

A couple of weeks ago we had a strange problem in the CACE Technologies World Domination Secret Lair. Loris was having trouble loading the Wireshark blog. It was working fine on my machine so I checked some of the other machines in the office. I found the same problem on one of the development machines. Loading the page in Firebug showed that the object requests were stalling out every 20 seconds or so:
firebug-slow-blog
I fired up Wireshark and captured the browser attempting to connect to the blog:
ipv6 failure
The problem was immediately obvious. The browser attempted to connect to the blog’s IPv6 address (2607:f0d0:2001:e:1::1), timed out, and connected to the blog’s IPv4 address (67.228.110.126).

As it turns out Loris and I ran some IPv6 tests a long time ago and we added unique local addresses to our machines (mine was fd00:cace::4) . Since each machine had an IPv6 address the TCP stack assumed that there was general IPv6 connectivity. However, our ISP is steadfastly ignoring IPv6 so end-to-end connectivity wasn’t there. After timing out the connection fell back to IPv4 and proceeded normally.

This sort of thing will likely crop up more and more often as we transition to IPv6. If either IPv4 or IPv6 connectivity fails, your applications will still have connectivity so the failure may not be immediately obvious. In this case Firefox stalled out but still managed to connect. If it used a connect-by-name socket API the failure wouldn’t have been evident at all.

Chrome OS

Today I followed the announcement of the new Chrome OS from Google and the acclaiming response it received. Am I the only one thinking: very simple, maybe TOO simple?

Since the beginning of the “browser as an OS” idea, I always wondered how I am supposed to run my favorite program, Wireshark, in a browser. Wireshark, of course, is an example, but many people do things with computers that don’t translate well into the cloud paradigm. Will they just be dismissed by OS manufacturers as a “nerdy minority”? Or will they need to stick with old uncool OSes to do their things? I’d like to be cool too, Google, but I’d still like to run Wireshark.

Another thing I feel weird about: Google is telling me not to worry about data loss, because from now on all my documents will be online. Nothing will need to be saved locally.
Actually, I like that my documents stay in my computer. Am I really the only one in the world? I’m not only talking about personal stuff like the love letters to my wife or the drunk pictures at the party last week. What about the trace files that I get all the time from customers? Will they have to go online too? Hmm, I’m sure my customers will love having them on Google’s server.

I can already hear the objection: “this OS is for netbooks, and netbooks are only used for simple things like going online, so this is perfect for them”. Actually, I love using Wireshark on a netbook! Together with an Airpcap adapter, it’s the perfect portable wireless troubleshooting station.
Why does small need to mean “limited”? Personally, I’d actually love to run Wireshark on my wireless-enabled IPod Touch! Ok, maybe I’m going too far…

http://googleblog.blogspot.com/2009/07/introducing-google-chrome-os.html

Polycom IP Phone Boot Analysis

We use Polycom SoundPoint IP phones here at the CACE Technologies World Domination Secret Lair. While troubleshooting a different problem recently I noticed that the phones do something funny. During the boot process they obtain an IP address via DHCP, similar to most PCs. A DHCP option tells the phones to fetch their configuration and firmware from an FTP server. The funny part is that some of the FTP transfers were failing even though the phones booted up just fine:

SIP firmware download failure

SIP firmware download failure

At packet 175 the phone starts to download sip.ld, the SIP firmware image. The transfer starts at packet 177, but the phone cuts the transfer short at packet 181. The phone then behaves as if nothing is wrong. What’s going on here? Continue reading