wm* hardware assist breaking gif tunnels

[John Klos]

Hi, all,

I just wrote this LONG email explaining that I couldn't figure out why my
gif tunnels weren't working, only to figure it out as I got near the end
of the email.

When I use the ip4csum option on the interface over which the gif tunnel
communicates, IPv6 over the tunnel stops working. tcpdump shows the
packets coming in, then shows the packets going out, but they never make
it onto the actual wire. Local IPv6 works fine, as does all IPv4.

This is happening on macppc, i386, and amd64, all running netbsd-4, with
several versions of wm* cards.

None of the tcp4csum udp4csum tcp6csum-tx udp6csum-tx tso4 options have
any effect on whether ip4csum kills IPv6.

I don't see any specific PRs referencing this. Does anyone have a good
idea on where to start, or does anyone have other systems on which to test
this?

One wm*:

wm0 at pci1 dev 6 function 0: Intel i82541PI 1000BASE-T Ethernet, rev. 5APC1: Picked IRQ 16 with weight 0
wm0: interrupting at ioapic0 pin 16 (irq 3)
wm0: 32-bit 33MHz PCI bus
wm0: 64 word (6 address bits) MicroWire EEPROM
wm0: Ethernet address 00:1b:21:06:b9:5f
igphy0 at wm0 phy 1: Intel IGP01E1000 Gigabit PHY, rev. 0
igphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto

Another:

wm0 at pci2 dev 0 function 0: Intel i82573L Gigabit Ethernet, rev. 0
wm0: interrupting at ioapic0 pin 17 (irq 11)
wm0: PCI-Express bus
wm0: 256 word (8 address bits) SPI EEPROM
wm0: Ethernet address 00:50:8d:93:2a:1c
makphy0 at wm0 phy 1: Marvell 88E1111 Gigabit PHY, rev. 2
makphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto

There are others, too, which exhibit the same problem. I'll be happy to
plug them in if the information is useful.

Thanks,
John Klos

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Michael van Elst]

Post by John Klos
When I use the ip4csum option on the interface over which the gif tunnel
communicates, IPv6 over the tunnel stops working.

Is this just IPv6 via tunnel or any IPv6 traffic ?


[...]

Post by John Klos
I don't see any specific PRs referencing this. Does anyone have a good
idea on where to start, or does anyone have other systems on which to test
this?

I can try to reproduce the problem.

wm0 at pci3 dev 0 function 0: Intel i82573L Gigabit Ethernet, rev. 0

[David Young]

Post by John Klos
Hi, all,
I just wrote this LONG email explaining that I couldn't figure out why my
gif tunnels weren't working, only to figure it out as I got near the end
of the email.
When I use the ip4csum option on the interface over which the gif tunnel
communicates, IPv6 over the tunnel stops working. tcpdump shows the
packets coming in, then shows the packets going out, but they never make
it onto the actual wire. Local IPv6 works fine, as does all IPv4.

Can we have a look at ifconfig -a?

Dave

[John Klos]

Followup:

While the tunnel starts working if I remove ip4csum, the system won't
route traffic from the tunnel to any local networks (on the same card as
the tunnel or on other interface) or the other direction unless I also
remove tcp4csum and udp4csum.

Hmmm...

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[John Klos]

Post by Michael van Elst

Post by John Klos
When I use the ip4csum option on the interface over which the gif
tunnel communicates, IPv6 over the tunnel stops working.

Is this just IPv6 via tunnel or any IPv6 traffic ?

Just via the tunnel.

Post by Michael van Elst
Can we have a look at ifconfig -a?

Sure. This is with all of the hw assist off (see my other followup):


wm0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
capabilities=2bf80<TSO4,IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Tx,UDP6CSUM_Tx>
enabled=0
address: 00:1b:21:06:b9:5f
media: Ethernet autoselect (1000baseT full-duplex)
status: active
inet 192.80.49.1 netmask 0xffffff00 broadcast 192.80.49.255
inet6 fe80::21b:21ff:fe06:b95f%wm0 prefixlen 64 scopeid 0x1
inet6 2610:1f8:d8:1::1 prefixlen 64
wm1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
capabilities=2bf80<TSO4,IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Tx,UDP6CSUM_Tx>
enabled=0
address: 00:1b:21:06:f8:ce
media: Ethernet autoselect (1000baseT full-duplex)
status: active
inet 38.119.55.170 netmask 0xffffffc0 broadcast 38.119.55.191
inet6 fe80::21b:21ff:fe06:f8ce%wm1 prefixlen 64 scopeid 0x2
inet6 2610:1f8:d8:2::1 prefixlen 64
wm2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
capabilities=2bf80<TSO4,IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Tx,UDP6CSUM_Tx>
enabled=0
address: 00:1b:21:06:b8:ac
media: Ethernet autoselect (1000baseT full-duplex)
status: active
inet 192.168.70.1 netmask 0xffffff00 broadcast 192.168.70.255
inet6 fe80::21b:21ff:fe06:b8ac%wm2 prefixlen 64 scopeid 0x3
inet6 2610:1f8:d8:3::1 prefixlen 64
nfe0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> mtu 1500
capabilities=3f00<IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx>
enabled=0
address: 00:19:21:ef:f4:5f
media: Ethernet autoselect (none)
status: no carrier
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 33648
inet 127.0.0.1 netmask 0xff000000
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
gif0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1280
tunnel inet 192.80.49.1 --> 209.212.68.38
inet6 2610:1f8::1:10:d86 -> 2610:1f8::1:10:d85 prefixlen 128
inet6 fe80::21b:21ff:fe06:b95f%gif0 -> prefixlen 64 scopeid 0x6
gif1: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1280
tunnel inet 192.80.49.1 --> 72.37.180.250
inet6 2610:1f8:d8::10 -> 2610:1f8:d8::11 prefixlen 128
inet6 fe80::21b:21ff:fe06:b95f%gif1 -> prefixlen 64 scopeid 0x7
gif2: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1280
tunnel inet 192.80.49.1 --> 76.168.62.28
inet6 2610:1f8:d8::12 -> 2610:1f8:d8::13 prefixlen 128
inet6 fe80::21b:21ff:fe06:b95f%gif2 -> prefixlen 64 scopeid 0x8


Any suggestions?

John Klos

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[David Young]

Post by John Klos
Any suggestions?

Taking a guess, maybe wm h/w sets received packets' embedded checksums
to 0 (or anything not equal to the actual checksum) when h/w assist
is active? Maybe wm h/w expects for an embedded checksum to be 0 in
transmitted packets, also? I have seen wm(4) lose in a bridge until I
turned off h/w checksum; it may have been for that very reason.
And I see this comment in ip_output(),

/*
* Always initialize the sum to 0! Some HW assisted
* checksumming requires this.
*/
ip->ip_sum = 0;

Dave

[Darren Reed]

Post by David Young

Post by John Klos
Any suggestions?

Taking a guess, maybe wm h/w sets received packets' embedded checksums
to 0 (or anything not equal to the actual checksum) when h/w assist
is active? Maybe wm h/w expects for an embedded checksum to be 0 in
transmitted packets, also? I have seen wm(4) lose in a bridge until I
turned off h/w checksum; it may have been for that very reason.
And I see this comment in ip_output(),
/*
* Always initialize the sum to 0! Some HW assisted
* checksumming requires this.
*/
ip->ip_sum = 0;

Correct. For some NICs, the hardware checksum is computed by specifying
a start and end offset: if ip_sum isn't 0, the start is the beginning of
the IP header
and the end is, well, at the end, the checksum placed in ip_sum will be
wrong.

Darren


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de