TCP socket buffers auto sizing

[Rui Paulo]

FreeBSD has done this:

<quote>
Auto sizing TCP socket buffers.

Normally the socket buffers are static (either derived from global
defaults or set with setsockopt) and do not adapt to real network
conditions. Two things happen: a) your socket buffers are too small
and you can't reach the full potential of the network between both
hosts; b) your socket buffers are too big and you waste a lot of
kernel memory for data just sitting around.

With automatic TCP send and receive socket buffers we can start with a
small buffer and quickly grow it in parallel with the TCP congestion
window to match real network conditions.

FreeBSD has a default 32K send socket buffer. This supports a maximal
transfer rate of only slightly more than 2Mbit/s on a 100ms RTT
trans-continental link. Or at 200ms just above 1Mbit/s. With TCP send
buffer auto scaling and the default values below it supports 20Mbit/s
at 100ms and 10Mbit/s at 200ms. That's an improvement of factor 10, or
1000%. For the receive side it looks slightly better with a default of
64K buffer size.

New sysctls are:
net.inet.tcp.sendbuf_auto=1 (enabled)
net.inet.tcp.sendbuf_inc=8192 (8K, step size)
net.inet.tcp.sendbuf_max=262144 (256K, growth limit)
net.inet.tcp.recvbuf_auto=1 (enabled)
net.inet.tcp.recvbuf_inc=16384 (16K, step size)
net.inet.tcp.recvbuf_max=262144 (256K, growth limit)

</quote>

Maybe we should too ?

--
Rui Paulo - http://fnop.net/~rpaulo/



--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Zafer Aydogan]

Definitely. It would be great.

Z.

Post by Rui Paulo
<quote>
Auto sizing TCP socket buffers.
Normally the socket buffers are static (either derived from global
defaults or set with setsockopt) and do not adapt to real network
conditions. Two things happen: a) your socket buffers are too small
and you can't reach the full potential of the network between both
hosts; b) your socket buffers are too big and you waste a lot of
kernel memory for data just sitting around.
With automatic TCP send and receive socket buffers we can start with a
small buffer and quickly grow it in parallel with the TCP congestion
window to match real network conditions.
FreeBSD has a default 32K send socket buffer. This supports a maximal
transfer rate of only slightly more than 2Mbit/s on a 100ms RTT
trans-continental link. Or at 200ms just above 1Mbit/s. With TCP send
buffer auto scaling and the default values below it supports 20Mbit/s
at 100ms and 10Mbit/s at 200ms. That's an improvement of factor 10, or
1000%. For the receive side it looks slightly better with a default of
64K buffer size.
net.inet.tcp.sendbuf_auto=1 (enabled)
net.inet.tcp.sendbuf_inc=8192 (8K, step size)
net.inet.tcp.sendbuf_max=262144 (256K, growth limit)
net.inet.tcp.recvbuf_auto=1 (enabled)
net.inet.tcp.recvbuf_inc=16384 (16K, step size)
net.inet.tcp.recvbuf_max=262144 (256K, growth limit)
</quote>
Maybe we should too ?
--
Rui Paulo - http://fnop.net/~rpaulo/

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[David Young]

Post by Rui Paulo
<quote>
Auto sizing TCP socket buffers.
</quote>
Maybe we should too ?

Yes.

Dave

[Christos Zoulas]

Post by Rui Paulo
<quote>
Auto sizing TCP socket buffers.
</quote>
Maybe we should too ?

Yes.
Dave

Definitely.

christos


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Matthias Scheler]

Post by Rui Paulo
Maybe we should too ?

Yes, please.

You don't even need a trans-atlantic 100ms RTT link. The default socket
buffer size doesn't work very well for a GbE LAN, too.

Kind regards

[Manuel Bouyer]

Post by Matthias Scheler

Post by Rui Paulo
Maybe we should too ?

Yes, please.
You don't even need a trans-atlantic 100ms RTT link. The default socket
buffer size doesn't work very well for a GbE LAN, too.

Or for ADSL2 links ...

[Steven M. Bellovin]

On Mon, 5 Feb 2007 17:59:37 +0100

Post by Manuel Bouyer

Post by Matthias Scheler

Post by Rui Paulo
Maybe we should too ?

Yes, please.
You don't even need a trans-atlantic 100ms RTT link. The default
socket buffer size doesn't work very well for a GbE LAN, too.

Or for ADSL2 links ...

I was impressed with how much performance different I got with ttcp on
a GigE LAN just by changing the buffer size.


--Steve Bellovin, http://www.cs.columbia.edu/~smb

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[David Young]

Post by Manuel Bouyer

Post by Matthias Scheler

Post by Rui Paulo
Maybe we should too ?

Yes, please.
You don't even need a trans-atlantic 100ms RTT link. The default socket
buffer size doesn't work very well for a GbE LAN, too.

Or for ADSL2 links ...

Or 802.11n.

Dave

[Matthias Scheler]

Post by Steven M. Bellovin
I was impressed with how much performance different I got with ttcp on
a GigE LAN just by changing the buffer size.

Yes, almost factor two:

ttcp-t: buflen=32768, nbuf=8192, align=16384/0, port=5001 tcp -> excalibur
ttcp-t: socket
ttcp-t: connect
ttcp-t: 268435456 bytes in 3.91 real seconds = 67127.17 KB/sec +++
ttcp-t: 8192 I/O calls, msec/call = 0.49, calls/sec = 2097.72
ttcp-t: -1.9user 0.2sys 0:03real 6% 0i+0d 0maxrss 0+80227pf 18791+3csw

ttcp-t: buflen=32768, nbuf=8192, align=16384/0, port=5001, sockbufsize=131072 tcp -> excalibur
ttcp-t: socket
ttcp-t: sndbuf
ttcp-t: connect
ttcp-t: 268435456 bytes in 2.36 real seconds = 111008.54 KB/sec +++
ttcp-t: 8192 I/O calls, msec/call = 0.30, calls/sec = 3469.02
ttcp-t: -1.9user 0.1sys 0:02real 5% 0i+0d 0maxrss 0+76397pf 16018+1csw

That's NetBSD-i386 3.1 sending to a Mac OS 10.4.8 (which either uses
automatic socket buffer sizing or has a higher default value).

Kind regards