ADSL Bandwidth Management HOWTO - Dan Singletary (best romantic novels to read .txt) 📗
- Author: Dan Singletary
- Performer: -
Book online «ADSL Bandwidth Management HOWTO - Dan Singletary (best romantic novels to read .txt) 📗». Author Dan Singletary
| |
+---------------------------------------------------------------------------+
At this point we still haven't realized any change in the performance. We've
merely moved the FIFO queue from the ADSL modem to the router. In fact, with
Linux configured to a default queue size of 100 packets we've probably made
our problem worse at this point! But not for long...
Each neighbor class in an HTB queue can be assigned a priority. By placing
different types of traffic in different classes and then assigning these
classes different priorities, we can control the order in which packets are
dequeued and sent. HTB makes it possible to do this while still avoiding
starvation of any one class, since we're able to specify a minimum guaranteed
rate for each class. In addition to this, HTB allows for us to tell a class
that it may use any unused bandwidth from other classes up to a certain
ceiling.
Once we have our classes set up, we set up filters to place traffic in
classes. There are several ways to do this, but the method described in this
document uses the familiar iptables/ipchains to mark packets with an fwmark.
The filters place traffic into the classes of the HTB queue based on their
fwmark. This way, we're able to set up matching rules in iptables to send
certain types of traffic to certain classes.
3.3. Classifying Outbound Packets with iptables
+---------------------------------------------------------------------------+
| |
| Note: originally this document used ipchains to classify packets. The |
| newer iptables is now used. |
| |
+---------------------------------------------------------------------------+
The final step in configuring your router to give priority to interactive
traffic is to set up the firewall to define how traffic should be classified.
This is done by setting the packet's fwmark field.
Without getting into too much detail, here is a simplified description of how
outbound packets might be classified into 4 classes with the highest priority
class being 0x00:
Mark ALL packets as 0x03. This places all packets, by default, into the
lowest priority queue.
Mark ICMP packets as 0x00. We want ping to show the latency for the
highest priority packets.
Mark all packets that have a destination port 1024 or less as 0x01. This
gives priority to system services such as Telnet and SSH. FTP's control
port will also fall into this range however FTP data transfer takes place
on high ports and will remain in the 0x03 band.
Mark all packets that have a destination port of 25 (SMTP) as 0x03. If
someone sends an email with a large attachment we don't want it to swamp
interactive traffic.
Mark all packets that are going to a multi-player game server as 0x02.
This will give gamers low latency but will keep them from swamping out
the the system applications that require low latency.
Mark any "small" packets as 0x02. Outbound ACK packets from inbound
downloads should be sent promptly to assure efficient downloads. This is
possible using the iptables length module.
Obviously, this can be customized to fit your needs.
3.4. A few more tweaks...
There are two more things that you can do to improve your latency. First, you
can set the Maximum Transmittable Unit (mtu) to be lower than the default of
1500 bytes. Lowering this number will lower the average time you have to wait
to send a priority packet if there is already a full-sized low-priority
packet being sent. Lowering this number will also slightly decrease your
throughput because each packet contains at least 40 bytes worth of IP and TCP
header information.
The other thing you can do to improve latency even on your low-priority
traffic is to lower your queue length from the default of 100, which on an
ADSL line could take as much as 10 seconds to empty with a 1500 byte mtu.
3.5. Attempting to Throttle Inbound Traffic
By using the Intermediate Queuing Device (IMQ), we can run all incoming
packets through a queue in the same way that we queue outbound packets.
Packet priority is much simpler in this case. Since we can only (attempt to)
control inbound TCP traffic, we'll put all non-TCP traffic in the 0x00 class,
and all TCP traffic in the 0x01 class. We'll also place "small" TCP packets
in the 0x00 class since these are most likely ACK packets for outbound data
that has already been sent. We'll set up a standard FIFO queue on the 0x00
class, and we'll set up a Random Early Drop (RED) queue on the 0x01 class.
RED is better than a FIFO (tail-drop) queue at controlling TCP because it
will drop packets before the queue overflows in an attempt to slow down
transfers that look like they're about to get out of control. We'll also
rate-limit both classes to some maximum inbound rate which is less than your
true inbound speed over the ADSL modem.
3.5.1. Why Inbound Traffic Limiting isn't all That Good
We want to limit our inbound traffic to avoid filling up the queue at the
ISP, which can sometimes buffer as much as 5 seconds worth of data. The
problem is that currently the only way to limit inbound TCP traffic is to
drop perfectly good packets. These packets have already taking up some share
of bandwidth on the ADSL modem only to be dropped by the Linux box in an
effort to slow down future packets. These dropped packets will eventually be
retransmitted consuming more bandwidth. When we limit traffic, we are
limiting the rate of packets which we will accept into our network. Since the
actual inbound data rate is somewhere above this because of the packets we
drop, we'll actually have to limit our downstream to much lower than the
actual rate of the ADSL modem in order to assure low latency. In practice I
had to limit my 1.5mbit/s downstream ADSL to 700kbit/sec in order to keep the
latency acceptable with 5 concurrent downloads. The more TCP sessions you
have, the more bandwidth you'll waste with dropped packets, and the lower
you'll have to set your limit rate.
A much better way to control inbound TCP traffic would be TCP window
manipulation, but as of this writing there exists no (free) implementation of
it for Linux (that I know of...).
ImplementationNow with all of the explanation out of the way it's time to implement
bandwidth management with Linux.
4.1. Caveats
Limiting the actual rate of data sent to the DSL modem is not as simple as it
may seem. Most DSL modems are really just ethernet bridges that bridge data
back and forth between your linux box and the gateway at your ISP. Most DSL
modems use ATM as a link layer to send data. ATM sends data in cells that are
always 53 bytes long. 5 of these bytes are header information, leaving 48
bytes available for data. Even if you are sending 1 byte of data, an entire
53 bytes of bandwidth are consumed sent since ATM cells are always 53 bytes
long. This means that if you are sending a typical TCP ACK packet which
consists of 0 bytes data + 20 bytes TCP header + 20 bytes IP header + 18
bytes Ethernet header. In actuality, even though the ethernet packet you are
sending has only 40 bytes of payload (TCP and IP header), the minimum payload
for an Ethernet packet is 46 bytes of data, so the remaining 6 bytes are
padded with nulls. This means that the actual length of the Ethernet packet
plus header is 18 + 46 = 64 bytes. In order to send 64 bytes over ATM, you
have to send two ATM cells which consume 106 bytes of bandwidth. This means
for every TCP ACK packet, you're wasting 42 bytes of bandwidth. This would be
okay if Linux accounted for the encapsulation that the DSL modem uses, but
instead, Linux only accounts the TCP header, IP header, and 14 bytes of the
MAC address (Linux doesn't count the 4 bytes CRC since this is handled at the
hardware level). Linux doesn't count the minimum Ethernet packet size of 46
bytes, nor does it take into account the fixed ATM cell size.
What all of this means is that you'll have to limit your outbound bandwidth
to somewhat less than your true capacity (until we can figure out a packet
scheduler that can account for the various types of encapsulation being
used). You may find that you've figured out a good number to limit your
bandwidth to, but then you download a big file and the latency starts to
shoot up over 3 seconds. This is most likely because the bandwidth those
small ACK packets consume is being miscalculated by Linux.
I have been working on a solution to this problem for a few months and have
almost settled on a solution that I will soon release to the public for
further testing. The solution involves using a user-space queue instead of
linux's QoS to rate-limit packets. I've basically implemented a simple HTB
queue using linux user-space queues. This solution (so far) has been able to
regulate outbound traffic SO WELL that even during a massive bulk download
(several streams) and bulk upload (gnutella, several streams) the latency
PEAKS at 400ms over my nominal no-traffic latency of about 15ms. For more
information on this QoS method, subscribe to the email list for updates or
check back on updates to this HOWTO.
4.2. Script: myshaper
The following is a listing of the script which I use to control bandwidth on
my Linux router. It uses several of the concepts covered in the document.
Outbound traffic is placed into one of 7 queues depending on type. Inbound
traffic is placed into two queues with TCP packets being dropped first
(lowest priority) if the inbound data is over-rate. The rates given in this
script seem to work OK for my setup but your results may vary.
+---------------------------------------------------------------------------+
|
Comments (0)