Maximizing UDP rate

[Jan Danielsson]

Hello,

Let's say I want to receive packets through a uni-directional link at
a high rate with as little packet loss as possible. What sysctl knobs,
kernel config parameters and even hardware choices should I be looking at?

[Jan Danielsson]

Post by Jan Danielsson
Let's say I want to receive packets through a uni-directional link at
a high rate with as little packet loss as possible.

You don’t say what rate you are interested in. 1Gb/s? 100Mb/s? 10Gb/s? even higher? big packets? small packets?

It's 1Gb/s; but I'm sure that within a year I'll be asking the same
for 10Gb/s -- though I know that'll open up new interesting challenges
in the application, so that's a problem for later.

Packet size is a variable of transfer rate and how deterministic the
transfers are. If higher rates can be achieved with larger packets,
then larger packets will be used. Will larger packets affect how
deterministic transfers are? Then smaller packets will be used. (The
application has a configurable packet size, which will be tuned in for
each installation).

In addition to deterministic (high) transfer rates, very long uptimes
are high priority, which essentially means both battle tested hardware
_and_ drivers.

Post by Jan Danielsson
What sysctl knobs,
kernel config parameters and even hardware choices should I be looking at?

I don’t know much about sysctl and kernel configs off the top of my head, but I do know a bit about hardware and some software. Not a lot, but a bit.
If the hardware is something old that you found in a box hidden in a back room somewhere, then it won’t work. Old hardware that might look like it’s fast enough often generates a high interrupt rate and ends up not working. Motherboards with Intel 1Gb/s ethernets on them have worked well for me, but I don’t tend to run NetBSD on x86 bare metal these days, so my experience is limited.

We will be buying new hardware if needed, and unless the NICs are
made out of a solid diamond, the cost is secondary to achieving a high
(deterministic) transfer rate.

Prodding around in network drivers has been somewhat enlightening --
turns out there's plenty of buggy network hardware out there. :/

The hardware designers have been saying Intel is the way to go, so
they have been at the top of the list, but I'm wondering if there are
any specific drivers in NetBSD which are known to be exceptionally
stable (written once, used a lot without problems, rarely/never fixed
because there has been no need to).

If you can’t get the capture rate you want from native NetBSD then you may need to look at dpdk.org <http://dpdk.org/>. You will need to pay more attention to hardware compatibility if you need to use the DPDK.

This looks very relevant -- thank you. As implied above, specific
hardware requirements is not a problem.