Packet ordering

[Jan Danielsson]

Hello,

Where, technically, do packet orders get mixed up?

I know UDP packet ordering is not guaranteed. I have seen UDP
packets arrive in the wrong order when sent over the Internet. However,
I have never seen UDP packets arrive in the wrong order when sent
between two hosts on our LAN, and it made me wonder where they actually
happen. I obviously understand that it could simply be that I haven't
observed it, not that it doesn't happen -- but it seems pretty easy to
trigger them over the Internet.

I understand that it's conceptually wrong to assume packet ordering
regardless -- but I'm curious if it can happen anywhere in the NetBSD
network stack.

[Greg Troxel]

Post by Jan Danielsson
Where, technically, do packet orders get mixed up?
I know UDP packet ordering is not guaranteed. I have seen UDP
packets arrive in the wrong order when sent over the Internet. However,
I have never seen UDP packets arrive in the wrong order when sent
between two hosts on our LAN, and it made me wonder where they actually
happen. I obviously understand that it could simply be that I haven't
observed it, not that it doesn't happen -- but it seems pretty easy to
trigger them over the Internet.
I understand that it's conceptually wrong to assume packet ordering
regardless -- but I'm curious if it can happen anywhere in the NetBSD
network stack.

Generally the notion in a single network device is that it's best that
packets be kept in order. While it's not an internet standards
violation to reorder them, I think most view it happening more than once
in a great while to be a defective implementation.

So two hosts with one Ethernet switch probably have packets remain in
order.

Packets can get reordered from:

queueing disciplines that deviate from FIFO to do priority or
something like that (although typically they try to not reorder within
a flow)

equal-cost multipath in routing, or link trunking, or things like that

routing changes, where a high-latency path is switched to a lower one
and the later packets arrive first


I would expect quite a lot more of that sort of thing in the
ISP/big-router portion of the world than in a house LAN.



--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Michael Richardson]

Post by Jan Danielsson
Where, technically, do packet orders get mixed up?

The place you'd see it is with poorly implemented Equal Cost Multi-Path (ECMP)
As Greg said, good design tries never to do this by dividing traffic
deterministically based upon the layer-3 5-tuple.
A good example of where this might not happen is if there was a layer-2
ECMP solution that did not look at the layer-3 traffic at all. Many early
optical transports did this.

A host with two (or more) "bonded" ethernets that had a *lot* of traffic on
the same 5-tuple, might want to turn off the hash mechanism so that it can
fill both pipes. If the wires were plugged into two different switch chassis
(not at all uncommon), then there might well be very different paths through
the l2 fabric. One path can be more congested than the other.

So in general, it doesn't happen that often... anymore. But it sure can happen.

--
] Never tell me the odds! | ipv6 mesh networks [
] Michael Richardson, Sandelman Software Works | IoT architect [
] ***@sandelman.ca http://www.sandelman.ca/ | ruby on rails [