Troubleshooting the hidden dangers of TCP’s Nagle algorithm and delayed acknowledgement

As we all know, TCP/IP is a great protocol suite.  However, there are times when it can become the bottleneck.  This is especially true if you use TCP/IP for real time transactions where small data sizes are the norm (think financial institutions).  In this session, I’ll show you why Nagle algorithm and delayed acknowledgement was developed.  But more importantly, I’ll highlight the unintended consequences when the two features interact – badly – with each other.  After watching this session, you will be able to spot the hidden dangers of using TCP/IP for real time transactions.   Enjoy, and as always, I would really appreciate your feedback and suggestions. Here is the video:

And as always, any and all feedback and suggestion are welcome.  Thank you and Enjoy!

Hansang Bae

Wireshark Tutorial Series. Tips and tricks used by insiders and veterans

For those of you who have attended Sharkfest in the past, you already know that protocol analysis is near and dear to my heart. It’s also a field where experience and art still matter. As great as Wireshark is as a tool, it still takes coaxing by an analyst to ferret out root cause. And as networks and applications become more complex, keeping up will be challenging.

But the one thing that I noticed over the years is that people rush to install sniffers without really thinking about it. It’s almost as if people expect sniffers to magically spit out the root cause, served on a silver platter! In reality, it takes fair amount of protocol and application knowledge to truly bring a tool like Wireshark to bear.

I started posting to this blog so that I can help budding protocol analysts and perhaps show interesting tricks-of-the-trade to veteran users. To become good in this field, it takes a fair amount of practice. It takes practice to know how to capture the right data, where to capture the data, what filters to use, and how to interpret the data. So how do you go about getting started? First, you can watch the accompanying video/tutorial session (see below for the link.) Next, make sure you setup your Wireshark in a consistent manner – the video tutorial covers this.

Ever wonder how router jockeys like me can scroll through a “sho run” output so quickly? It’s because I’ve done it for so long that the eyes are trained to filter out unneeded information. That’s the key to training – knowing what to filter out so your brain can get to work on the important stuff. It turns out protocol analysis works the same way. You have to train your brain to filter out the noise. Setting up your Wireshark environment will go a long way to maximizing productivity.

There is no “right way” to setup Wireshark. There’s only “my way” and everyone else’s – by definition – is wrong! Some like destination address to be the first column just like in DOS Sniffer. Others prefer using Wireshark’s default order. Whatever your style is, make sure it’s consistent. And if you’re just starting out, perhaps you can benefit from my setup. Even Anthony Bourdain in his book “Kitchen Confidential” talks about “mise-en-place.” It’s a term used by chefs and signifies how the cooking stations are setup. It’s important because it makes them more productive. For the same reason, you need to develop your own Wireshark mise-en-place!

If you still have not modified the default layout of Wireshark, you’re definitely missing out. In the video, I’m going to help you setup Wireshark so that you can become more productive. And we’re going to embark on a journey where I show you all the secrets to protocol analysis. I’m like the “magicians’ tricks revealed” guy. I’m going to help make you a rock star – where protocol analysis is concerned – in your company. If you’re an industry veteran, don’t be alarmed. The first few sessions are geared towards beginners so they can catch up. After that, I promise you that we’ll be in the weeds!

Hope you enjoy it, and I’d love to hear your comments. You can reach me at [email protected]

Sharkfest ’12 Recap

The fifth annual Sharkfest wrapped up a couple of days ago. You are no doubt feeling terrible if you missed it. Don’t despair, you can find a retrospective at the Sharkfest web site. The events team tried to make sure each session was recorded so we should have videos to go along with the slides soon.

As usual I wanted to attend more presentations than physics allowed but unfortunately missed some, so I’m waiting for the retrospective videos along with everyone else. (Hansang Bae observed that this is one of the few conferences where presenters attend other sessions.) Here are some quick notes about the sessions I was able to attend:

Monday’s keynote was delivered by Cliff Stoll. He kept insisting that he didn’t have much to say while simultaneously delivering an informative and entertaining talk.

Steve Riley provided a lot of insights into cloud computing: how it’s changing computing and how it can best be utilized.

Loris Degioanni (aka my boss) gave a great presentation on the new platform API that some people have been working on here at Riverbed. I think this is going to be a must-have once people realize the cool and crazy ways that this can be leveraged.

Wireless was well-represented Thomas D’Otreppe taked about OpenWIPS-ng, his new wireless IPS/IDS project. Metageek’s Ryan Woodings talked about wireless below the packet level. Mike Kershaw and Michael Ossman talked about the development of the Kisbee and Ubertooth adapters, and how you too can design and implement real hardware devices.

Hansang Bae showed modern TCP stacks and network devices can collude to make connectivity really slow, and at a really low level.

Laura Chappell showed off a lot of the new features in Wireshark 1.8. I even learned something — you can type a color name (e.g. “bisque3”) into the color picker in the preferences. She also released the second edition of Wireshark Network Analysis. It’s even heavier than the first edition, which I learned first-hand.

I led one session this year, Ask The Experts. We ended up talking about ways to improve Wireshark and a few of the core developers and I discussed these ideas the next day.

We got logs of great feedback on the conference as well. Several people commented that we didn’t have as many devlopment-oriented sessions this year as we have in the past. Maybe we can add more next time.

Wireshark is a great and wondrous thing due to its community. The developers, users, educators, sponsors all play a big role in its development and evolution. I see small reminders of this every day but it becomes abundantly clear each year at Sharkfest.

Thanks to the attendees, presenters, and to the Riverbed events team for creating a wonderful experience. I can’t wait to see what next year’s Sharkfest has in store.

Wireshark and Pcap-ng

When Wireshark 1.8.0 is released in the next few months it will introduce two major features: the ability to capture from multiple interfaces at once and the ability to annotate packets. These features have been on the wishlist for years and they will make Wireshark much more useful. They both share dependency on pcap-ng.

Packet comment example

Packet comment example

Pcap-ng is Wireshark’s new default file format. It is more complex than its predecessor (pcap) but it’s also more flexible. Along with multiple interface types and annotations you can store host information, extended interface information, and much more. Wireshark has had basic support for pcap-ng for several years now and 1.8.0 will bring it to the forefront.

But there’s a catch. Suppose you sit down at a computer with Wireshark 1.7 or 1.8 installed. What file format will Wireshark use by default? Unfortunately the answer is “It depends.” If this is a fresh install of Wireshark on a new computer you’ll get pcap-ng files. If this is an upgrade from 1.6 or below you’ll get pcap files. If the preferences were changed or copied from another machine the default format will be affected as well.

There’s another catch. Suppose you’ve just saved a capture file and you want to read it using tcpdump, Cascade Pilot, Snort, or any of the dozens of other applications that use the classic pcap file format. What will happen? Again, it depends. If that application has been updated to support pcap-ng or if it’s using a recent version of libpcap it will probably work. Otherwise don’t hold your breath.

If you’re a Wireshark user you might want to try a recent build of Wireshark 1.7 to see the new features and new file format in action. If you’re a developer and your application reads pcap files you’re most likely in for some grief. For that I apologize. You should probably investigate adding pcap-ng support now before your users start complaining.

Despite the migration pains the switch to pcap-ng is long overdue and will make Wireshark more useful to more people.

Used Cars and Stub Installers

The Wireshark development team works hard to earn the respect of our users. This includes making sure that downloading and installing Wireshark is as easy and trouble-free as possible. Right now the vast majority of our users can go to, follow the big green arrows, and immediately download the appropriate Wireshark package for their platform.

For many years a number of third party sites have also offered Wireshark downloads. Typing “wireshark download” into your favorite search engine will turn up a bunch of them, usually just below links to These sites are popular and often provide valuable services such as reviews and malware prescreening. They also reside outside the Wireshark ecosystem — we don’t link to them and aren’t affiliated with any of them.

"This is the Cadillac of invasive toolbars at a Chevy price!"

Sometimes these sites abuse their relationship with their users. For example a few months ago started using a stub installer which tries to get you to install various toolbars and who-knows-what-else before it installs the package you ultimately want, much like a sleazy car salesman trying to bundle add-ons you don’t want or need.

This sort of bottom-feeding behavior is harmful to our user community and exploits the goodwill we have with our users. Brian Krebs and Gordon “Fyodor” Lyon describe the problem with much more depth and eloquence than I can.

I sent a request to to disable their stub installer for Wireshark. They complied, but there are dozens of other download sites. Trying to keep tabs on all of them would result in a never-ending game of Whac-A-Mole®.

On behalf of the Wireshark development team I promise to provide easily accessible, direct downloads of Wireshark from just as we always have. If you choose to download Wireshark somewhere else we can’t guarantee that the experience will be free of shenanigans so please be careful.

Sharkfest ’11 Recap

The fourth annual Sharkfest was held last week. If you missed it, don’t worry. We are busy uploading all of the presentations to

The conference started with a keynote by Dr. Steve McCanne, CTO of Riverbed. He described the history of BPF and how its optimizing compiler works. It was interesting to see all of the problems he ran into and the solutions he found. The presentation was easy to follow despite its technical level. (Steve created BPF, and co-wrote tcpdump and libpcap.)

The keynote set the tone for the rest of the conference, which featured a lot of talented speakers. Jeff Carrell’s IPv6 was so popular it had be repeated several times over the course of the conference. Lara Chappell, Hansang Bae, and Betty Dubois drew their usual huge crowds. The other presenters did great jobs as well.

People have great ideas for improving Wireshark. Plenty of these ideas and suggestions were heaped our way throughout the conference, especially during the last two sessions. I hope to spend some time this summer implementing some of them.

Wireshark is very much a community-owned project. It is where it is today because of the developer and user communities. Steve stressed this fact during his keynote and it was readily apparent throughout the conference. On behalf of the developer team, the Sharkfest organizers, and Riverbed I extend my heartfelt thanks to the Wireshark community. This continues to be an amazing adventure and I can’t wait to see what the future has in store.

We’re not Participating in World IPv6 Day. Mostly.

Tomorrow is World IPv6 Day, the largest full-frontal test of IPv6 to date. It is going to be a historic event. It’s also one in which will and won’t be participating.

In one sense every day is IPv6 day here and tomorrow will be just another day. Most of our web sites (anonsvn, ask, this blog, bugs, buildbot, sharkfest, and wiki) have been fully dual-stacked for some time. You can reach them over both IPv4 and IPv6 and so far it’s been working pretty well. The big exception to this is the main web site, which still only has an A record. We can add an AAAA record at any time, but I’ve been holding off doing so until well *after* World IPv6 Day.

My concern is that having an AAAA record in place for tomorrow will cause unnecessary problems. If anyone runs into trouble reaching dual-stacked sites I don’t want to impede their ability to troubleshoot the problem by making Wireshark difficult to download.

We’ll add the AAAA record for in a few weeks.

P.S. According to the SCM revision logs IPv6 support was introduced in Wireshark in 1998. Tomorrow’s test is long overdue.

Looking forward to Sharkfest ’11

I’ve been looking over the session schedule for Sharkfest ’11. Once again Janice and Sheri have created an event which guarantees a wealth of knowledge and insight for everyone attending.

What to expect

Sharkfest is small. This is on purpose. We limit the size of the conference in order to allow more one-on-one communication between the attendees and presenters.

It has a high knowledge density. Our strategy is to gather together a bunch of people who are excited about Wireshark and protocol analysis, and know what the heck they’re talking about. We do our best to make sure the presentations focus on usable information with a minimum of fluff.

How to get the most out of Sharkfest

Sharkfest is active, not passive. Mingle. Compare notes. Many of the attendees are Wireshark power users, but many are not. Everyone has something insightful to share. The worst thing you can do is keep to yourself.

For the past three years I’ve had the opportunity to witness the top people in protocol analysis exchanging and sharing ideas. I look forward to seeing the same thing this year. See you there.

Is there an RFC for hashtags?

On May 24, Steve McCanne, Loris Degioanni and I will answer questions sent to us on Twitter via #packetcap. We’ve given talks about the origins of tcpdump, BPF, WinPcap, and Wireshark live but this is the first time we’ll do so remotely and capped at a 140-byte MTU. It should be fun and informative and I hope you’ll join us.

Full details on the Riverbed blog.