Bufferbloat

[David Brownlee]

Very interesting blog article from Jim Gettys (apologies to those who
have already read it) regarding the size of network buffers breaking
TCP.

http://gettys.wordpress.com/2010/12/03/introducing-the-criminal-mastermind-bufferbloat/

I wonder how many people out there are running RED in ALTQ to
effectively mitigate this on their DSL links?

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[David Young]

Post by David Brownlee
Very interesting blog article from Jim Gettys (apologies to those who
have already read it) regarding the size of network buffers breaking
TCP.
http://gettys.wordpress.com/2010/12/03/introducing-the-criminal-mastermind-bufferbloat/
I wonder how many people out there are running RED in ALTQ to
effectively mitigate this on their DSL links?

I have been fighting bufferbloat at work, see
<http://mail-index.netbsd.org/tech-net/2009/03/19/msg001141.html>.

I'll add to my two hasty conclusions in that email, two more
conclusions:

3. The kernel ought to base its decision to drop or queue a packet based
on realistic transmission-time estimates for the queue.

4. For bus-mastering NICs, there should not be both a transmission queue
*and* a transmission ring, but only a transmission ring.

Dave

[Steven Bellovin]

Post by David Brownlee
Very interesting blog article from Jim Gettys (apologies to those who
have already read it) regarding the size of network buffers breaking
TCP.
http://gettys.wordpress.com/2010/12/03/introducing-the-criminal-mastermind-bufferbloat/
I wonder how many people out there are running RED in ALTQ to
effectively mitigate this on their DSL links?

I've been working with Jim on this issue -- there's no doubt that he's
correct. A lot of other networking people are looking at it, too, up
to and including Van Jacobson. Van has some modifications to RED that
were never published but make it work a lot better and *much* more
automatically. He's going to post them soon (http://gettys.wordpress.com/2011/01/06/a-committee-is-a-life-form-with-six-or-more-legs-and-no-brain-lazarus-long/);
it would be a very good idea if NetBSD would implement the new scheme.


--Steve Bellovin, http://www.cs.columbia.edu/~smb






--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Greg Troxel]

Post by David Young
4. For bus-mastering NICs, there should not be both a transmission queue
*and* a transmission ring, but only a transmission ring.

I would say instead that the total queued packets should be limited to
the plan, regardless of whether the queue is in host memory or already
pushed to the dma controller. There's merit in keeping the NIC's part
of the queue small if you have a low-latency queue.

It would be a good project to go over altq and figure out if it needs
help to be modern; someone commented privately to me (I think) that altq
wasn't smp-friendly.

[Thor Lancelot Simon]

Post by Greg Troxel
It would be a good project to go over altq and figure out if it needs
help to be modern; someone commented privately to me (I think) that altq
wasn't smp-friendly.

It needs a lot more than that. To the extent I've looked at it, I've
become increasingly persuaded that it's basically a toy. There's no
chance it can keep up with modern interface speeds.

Also, it basically cannot manage interface-private resources like
descriptor rings the way David wants, but letting a modern network
interface's ring stall is about the worst thing you can do in terms
of sustained performance.

ALTQ is too big, too "flexible" (in quotes because without something
like OpenBSD's pf integrated version, which slows it down even more,
it is not all that flexible!) and way too slow. It is demonstration
code, not something suitable for use on a modern high speed end host
or router.

I am aware that it works well when connecting fast local networks to
slow uplinks. But that is a very different task and also really a
matter of luck.

I think that a much less flexible implementation of some single effective
queueing discipline -- SFB perhaps, or this rumored autotuning RED
that Steve mentioned -- should be used to replace the current queueing
between the stack proper and the network interfaces. To handle the
issues relating to large descriptor rings (or no descriptor rings, on
other interface designs) an approach like the SCSI midlayer where
adapters announce their "opening" count to the stack might work.

*Maybe* then you'd get something that would have a chance in hell of
keeping up with 10Gb line rate on the fastest systems we support.

But anything like this means reworking every interface driver in the
system. I'd rate it somewhere between "very very unlikely" and
"never gonna happen".

[Mindaugas Rasiukevicius]

Post by Thor Lancelot Simon
ALTQ is too big, too "flexible" (in quotes because without something
like OpenBSD's pf integrated version, which slows it down even more,
it is not all that flexible!) and way too slow. It is demonstration
code, not something suitable for use on a modern high speed end host
or router.

I have to agree with this. The framework was a nice idea, but the code
base got very impractical - to the point where it might be reasonable to
design and implement something more compact (and better documented).

Also, for traffic shaping, I would say Linux-like HTB is a way to go.

Post by Thor Lancelot Simon
But anything like this means reworking every interface driver in the
system. I'd rate it somewhere between "very very unlikely" and
"never gonna happen".

Can be done gradually, in steps.