FAST_IPSEC(?) drops packets?

[Dave Huang]

I'm seeing a strange problem, but I don't really know where it's coming
from :( I have a router running NetBSD-current/i386 as of Feb 11, 2011
with FAST_IPSEC, but it seems like not every packet that comes in gets
sent out through the IPsec tunnel.

I ran tcpdump to capture packets on both the internal and external
interfaces of the router, then had Wireshark decrypt the ESP packets on
the external side and compared the two packet dumps. They match up fine
at first, but then:

On the internal IF (10.1.1.66 is a machine on my side of the tunnel,
10.2.1.3 is on the other side):

[...]
10.2.1.3 10.1.1.66 TCP rsync > 62564 [PSH, ACK] Seq=4128
Ack=79 Win=65471 Len=1398
10.2.1.3 10.1.1.66 TCP rsync > 62564 [ACK] Seq=5526 Ack=79
Win=65471 Len=1398
10.1.1.66 10.2.1.3 TCP 62564 > rsync [ACK] Seq=84 Ack=6924
Win=65532 Len=0
*10.1.1.66 10.2.1.3 TCP 62564 > rsync [ACK] Seq=84 Ack=6924
Win=65532 Len=1460
*10.1.1.66 10.2.1.3 TCP 62564 > rsync [PSH, ACK] Seq=1544
Ack=6924 Win=65532 Len=1460
10.2.1.3 10.1.1.66 TCP rsync > 62564 [PSH, ACK] Seq=6924
Ack=79 Win=65471 Len=1300
10.2.1.3 10.1.1.66 TCP rsync > 62564 [PSH, ACK] Seq=8224
Ack=84 Win=65466 Len=1398
10.1.1.66 10.2.1.3 TCP 62564 > rsync [ACK] Seq=3004 Ack=9622
Win=65532 Len=0

Decrypted ESP packets on the external IF:

10.2.1.3 10.1.1.66 TCP rsync > 62564 [PSH, ACK] Seq=4128
Ack=79 Win=65471 Len=1398
10.2.1.3 10.1.1.66 TCP rsync > 62564 [ACK] Seq=5526 Ack=79
Win=65471 Len=1398
10.1.1.66 10.2.1.3 TCP 62564 > rsync [ACK] Seq=84 Ack=6924
Win=65532 Len=0
10.2.1.3 10.1.1.66 TCP rsync > 62564 [PSH, ACK] Seq=6924
Ack=79 Win=65471 Len=1300
10.2.1.3 10.1.1.66 TCP rsync > 62564 [PSH, ACK] Seq=8224
Ack=84 Win=65466 Len=1398
10.1.1.66 10.2.1.3 TCP [TCP Previous segment lost] 62564 >
rsync [ACK] Seq=3004 Ack=9622 Win=65532 Len=0

So the router saw those two packets marked with the *, but why didn't it
encrypt them and send them out the tunnel?

I had been running 5.1_STABLE with "regular" IPSEC up until Feb 11, and
didn't have any problems. I'm not exactly sure when this started
happening though, so I'm not certain that the upgrade had anything to do
with it... the tunnel mostly works fine, but occasionally TCP
connections will stall and close.

If there's any more info that would be helpful for figuring out what's
going on, let me know :)


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Dave Huang]

Post by Dave Huang
*10.1.1.66 10.2.1.3 TCP 62564 > rsync [ACK] Seq=84 Ack=6924
Win=65532 Len=1460
*10.1.1.66 10.2.1.3 TCP 62564 > rsync [PSH, ACK] Seq=1544
Ack=6924 Win=65532 Len=1460

I just noticed "Len=1460"... MTU issues, perhaps? I have
"net.inet.ipsec.dfbit=2" in my /etc/sysctl.conf from when I was using
IPSEC; should that be changed for FAST_IPSEC? The inner packets in
question do have DF set.

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Matthias Drochner]

Post by Dave Huang
I just noticed "Len=1460"... MTU issues, perhaps? I have
"net.inet.ipsec.dfbit=2"

So is the CANTFRAG number counting in "netstat -pip"?
Shouldn't the encapsulation code send NEEDFRAG ICMPs
then? Do you observe any? Perhaps this part doesn't
work correctly in FAST_IPSEC...

best regards
Matthias



------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDirig Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender),
Dr. Ulrich Krafft (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Dave Huang]

Post by Matthias Drochner
So is the CANTFRAG number counting in "netstat -pip"?
Shouldn't the encapsulation code send NEEDFRAG ICMPs
then? Do you observe any? Perhaps this part doesn't
work correctly in FAST_IPSEC...

Yes, I do see "datagrams that can't be fragmented" increase in "netstat
-pip". However, I don't see the router sending any ICMP fragmentation
needed packets back.

BTW, changing net.inet.ipsec.dfbit to 0 does work around the problem
(but causes fragmentation).

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de