For every Network Service
there's an equal and opposite Network Disservice

Nothing in networking comes for free, which is a fact many people seem to forget. Any time a network technology offers "guaranteed reliability" and other similar properties, you should ask what it is going to cost you, because it is going to cost you. It may cost you in terms of money, in terms of lower throughput (bandwidth), or in terms of higher delay, but one way or another it is going to cost you something. Nothing comes for free.

1. A guaranteed network service guarantees to be a low quality overpriced service

For example, one way that a network can guarantee never to lose a packet is to limit the users to a rate that is so slow that even if they all conspire to send data at the same instant, they can never overload the capacity of your network. Of course all the users in the world never do conspire to send data at the same instant, so your network is never operating close to its possible capacity, but that's the price of guarantees. ISDN works this way. A 100Mb/sec Ethernet card may let you send data at 100Mb/sec, but it doesn't guarantee it. Sometimes you may only be able to send 10Mb/sec or even as little as 1Mb/sec. ISDN on the other hand guarantees that as long as you are connected, you will always be able to send up to 0.064Mb/sec. Of course it also guarantees you will never be able to send more than 0.064Mb/sec per ISDN channel, which is still 15 times slower than an Ethernet operating at only 1% of capacity. The other drawback of ISDN's guarantee is that you have to pay for it all the time you're connected -- typically 1c per minute (60c per hour, $14 per day, $100 per week) -- even if you are not sending any data. You have to pay for it because even if you are not sending any data the network has reserved bandwidth for you, and no one else is allowed to use your "reserved" bandwidth so it sits idle until you "hang up" your connection.

The whole concept of "hanging up" a network connection is very strange to computer networking. How many people are expected to diligently disconnect their computer's ethernet cable whenever they're not using it? So why do the telephone companies charge by the minute for ISDN calls and expect users to diligently "hang up" their ISDN call whenever they're not using it? Is it because they just don't understand computer networking?

After you've paid all this money, ISDN still doesn't really guarantee anything. It guarantees that as long as you're connected you can send 0.064Mb/sec, but it doesn't guarantee that you'll *stay* connected. If you leave your ISDN line up for long enough, you'll find that eventually something always goes wrong and the connection spontaneously fails. If an Ethernet has a problem it might drop a packet. If the ISDN system has a problem it has no choice but to hang up your connection, because that's what it guarantees -- it guarantees all-or-nothing behaviour, with nothing in between. ISDN's guarantee just means that any minor problem has to become a total failure, because ISDN can't have minor failures. More on this in Part 3.

2. For every guarantee there's a corresponding refusal

Note that ISDN's guarantee only applies once you are connected. There is no guarantee that you will get connected when you place your call. Just like any other network the ISDN network has finite capacity. The difference is that even when it is under heavy load an Ethernet will always take your packet and try to deliver it if it can. The ISDN network won't even let you place the call unless it is absolutely 100% certain that it has so much spare capacity that it knows it can guarantee never to lose any data you might send. So in the end ISDN doesn't guarantee anything at all. If there's the slightest possibility that there might not be enough capacity to send your data, ISDN will refuse to even try.

You see, adding a guarantee doesn't magically increase the overall capacity of the network. The capacity of the network stays the same, but the effective usable capacity goes down because now the network has to refuse data any time there's even the slightest chance it might not be able to deliver it.

So, a guarantee doesn't improve the chance of your data getting through. In fact it reduces the chance, because now the network guarantees that in the questionable cases, it will refuse to even try.

3. For every Network Connection there's a corresponding Network Disconnection

It's easy to forget what a guarantee means. It's easy to think that a guarantee means an absolute assurance that something will work as promised. Of course it means nothing of the kind. When you buy a product, a one year guarantee does not mean that the product will work for one year -- it means that if it fails, the company will repair it, replace it, or refund your money.

A network guarantee means the same thing. If your call mysteriously hangs up (and they often do), then your network "guarantee" is of the "replace it" variety. If your call mysteriously hangs up then the telephone company guarantees that you can try dialing again to get a new call. (They don't guarantee it will connect -- the 'lines' might all be 'busy' -- see (2) above.) Most network-savvy computer users have autodialing scripts that automatically redial, so when the line goes down all they notice is a brief pause while it reconnects. If you need to have software that automatically copes with lost calls, what exactly is this "guarantee" providing for you, except really low bandwidth? If you need to have software that automatically copes with lost calls, why not go the whole hog and use proper network software like TCP, which automatically copes with lost packets? In fact, the funny thing is that most people are using their modems to send TCP packets, so they don't even need the "guarantee" that the connection-oriented phone call is supposed to provide.

Many people have disagreed with my comment that telephone calls often spontaneously hang up. Many people have an impression that telephone calls are, on the whole, pretty reliable. This is true for short calls. If you make a three minute voice call, or a thirty minute voice call, then the chance of you experiencing a failure of the telephone system during that time is quite small. However, think how long your office computer has been connected to the Ethernet. Is it three minutes? Is it thirty minutes? It's probably more like months or years. If you try to keep a telephone connection active for months or years you'll pretty quickly realize that it's hopelessly unreliable compared to computer networking technologies like Ethernet.

4. Modems that do compression make your connection slower, not faster.

Modern modems offer new 'features' like compression and error correction (ARQ). These 'features' don't come for free. They take time. With old 2400 baud modems, every character you sent to them would be immediately sent down the telephone line. Modern modems don't do that. Modern modems do compression. You can't compress one single character. You have to get a whole block of characters before you can apply compression techniques. All those characters you send, instead of being sent immediately, are stored up inside the modem until it has a block large enough to compress effectively. When they are finally sent, the transmission takes less time because the data has been compressed, but first it had to be delayed in order to make that possible. So, the transmission takes less time, but the total elapsed time between you sending the data and it being received at the other end is longer. That's a little ironic, no? Less is more. Faster is slower.

If you are solely concerned with the total amount of data you can send over the modem, then compression is a win. However, the actual time it takes any particular piece of that data to arrive is longer. Sometimes delay matters more than aggregate throughput.

Why does web browsing feel so slow on a 28.8 modem? Certainly low throughput is part of the problem, but poor latency is more responsible.

The typical round-trip delay between two machines on an Ethernet is less than a thousandth of a second. The typical round-trip delay over a modem connection, using many of today's popular modems, is a quarter of a second or more. Let's analyse how this affects your web browsing:

It has taken five round trips before your computer even gets to see the first pixel of the first image. That's more than a second after the user first clicked on the link.

If you think that one second doesn't make a noticable difference and I'm unreasonable to make such a fuss about it, consider the following: My Linux PC, with a direct Ethernet/FDDI connection to the Internet, displays entire Web pages in less than a second. Now, suppose in the future you buy yourself a DirecPC satellite dish, or an ADSL line, or a cable modem, or some other device that has a megabit/s, or a gigabit/s, or even a terabit/s of bandwidth. If those devices also have a quarter-second round-trip time (as current versions do), then you're still going to have a 1.25 second delay before the first pixel arrives. My Ethernet-connected PC will have finished downloading the page before yours even started. It won't matter how many terabit/s of bandwidth your device has, because the race will be over before you even get to use any of it. Great bandwidth alone, without a correspondingly good latency, is worthless.

It's true that sliding-window protocols like TCP are designed to cope with latency, and it's also true that companies like Sandcastle Inc. have developed very effective techniques to mask latency in interactive appliations. However, it's important to remember that while these techniques are clever solutions to an unfortunate problem, they do not make high latency network hardware desirable, acceptable or excusable. That would be like saying that because all cars have shock absorbers we'd prefer all our roads to be full of pot-holes. Shock absorbers make a rough ride smoother, but shock absorbers on a smooth road are even better.

Okay, the high delay has absolutely killed our start-up time, but now the connection is in place and the data is flowing, the compression will start to show some benefit, right? Well, actually no. What are the big files on a web server? Pictures and images. Pictures on Web servers are already compressed with JPEG or GIF. The modem can't compress data that's already compressed. Java applets can be big, but on Web servers they're already compressed into zip files. Video files are big, but they're always compressed with MPEG or QuickTime. The only thing left uncompressed is the text of the html files. These would benefit from compression, but the amount of data is so small compared to all the images that it doesn't make a noticable difference. If there was a significant benefit to be had compressing the text files, Web servers could do that compression too, and do it better than a modem can. The end result here is that anything that can be usefully compressed is already compressed, so all the compression features of the modem give you no benefit at all. They make the modem hardware more expensive, and they delay your data, slowing your network traffic down, but they give no benefit in return.

Most modems have modes that allow you to turn off the error correction and compression. Unfortunately, these modes get rid of the error correction and the compression but they don't get rid of the delay, so they don't help.

High latency-devices like modems are a problem for web browsing, but they're a complete disaster for more interactive software, like network games. It is currently extremely difficult to make a network game run over modems, and it is impossible to make it have the same interactive responsiveness that a typical single-player game has. This is never going to change until modem makers start worrying less about how many kilobits of compressed throughput their modems have and start worrying more about how many milliseconds of latency they add.

To put this into perspective, the current measured round-trip delay from Stanford to MIT over the Internet, a total round-trip distance of over 8640km, is 85ms. The round-trip delay from a home user to their Internet service provider, over a typical modem, is 250ms. If they want to send packets to their next-door neighbour, there's another 250ms delay for their neighbour's modem too, making a total round-trip delay of 500ms. I can talk to someone thousands of miles away using the Internet about six times faster than I can talk to my next-door neighbour using a modem and a telephone line. There's something very wrong there.

There's more discussion of this in It's the Latency, Stupid.


http://ResComp.Stanford.EDU/~cheshire