IP Filter does not seem to work correctly with bge(4) when hardware checksums are enabled

[Greg A. Woods]

I think there's a similar problem to PR#34799 still happening with
bge(4) in netbsd-4 on an HP Proliant box I'm setting up as a NAT and
firewall.

I.e. ipmon is reporting "bad" packets blocked even though the "pass"
rule they match is triggered. (too bad "bad" isn't well documented!)


ifconifg:

bge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
capabilities=3f00<IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx>
caps_enabled=3f00<IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx>
address: 00:12:79:90:6a:0a
media: Ethernet autoselect (1000baseT full-duplex)
status: active
inet 192.168.0.1 netmask 0xffffff00 broadcast 192.168.0.255

dmesg:

bge0 at pci6 dev 2 function 0: Broadcom BCM5704C Dual Gigabit Ethernet
bge0: interrupting at ioapic1 pin 1 (irq 5)
bge0: ASIC unknown BCM5704 (0x2100), Ethernet address 00:12:79:90:6a:0a
brgphy0 at bge0 phy 1: BCM5704 1000BASE-T media interface, rev. 0

some relevant output from "ipfstat -viohn":

357 27201 @11 pass in quick proto udp from 192.168.0.0/24 to any port = domain keep state group 350 # count 0
362 27201 @31 block in log quick all group 350

some example log records from syslog:

Jan 28 00:01:02 fw ipmon[340]: 00:01:01.980307 bge0 @350:31 b 192.168.0.252,54202 -> dns.ym-nk-wlfdle.rnc.net.cable.rogers.com[24.153.23.114],domain PR udp len 20 87 IN bad
Jan 28 00:01:05 fw ipmon[340]: 00:01:04.979956 bge0 @350:31 b 192.168.0.252,57393 -> dns.nk.rnc.net.cable.rogers.com[24.153.23.66],domain PR udp len 20 87 IN bad

then as soon as I made the client DNS queries go to an inside caching
server I saw lots like this:

Jan 28 00:19:30 fw ipmon[340]: 00:19:29.944216 bge0 @350:31 b 192.168.0.252,50031 -> centos.mirror.iweb.ca[70.38.0.136],http PR tcp len 20 60 -S IN bad
Jan 28 00:19:36 fw ipmon[340]: 00:19:35.943206 bge0 @350:31 b 192.168.0.252,50031 -> centos.mirror.iweb.ca[70.38.0.136],http PR tcp len 20 60 -S IN bad


However once I turned off tcp4csum and udp4csum on bge0 then everything
seemed to begin working OK.

(this machine also has a dual wm(4) board, and I've got all the *4csum
features enabled on it and there are no apparent complaints)

Is this a known problem with bge(4) (vs. IPFilter)? Is this specific to
just the one BCM5704C chip, or more generic?

[Ignatios Souvatzis]

Post by Greg A. Woods
I think there's a similar problem to PR#34799 still happening with
bge(4) in netbsd-4 on an HP Proliant box I'm setting up as a NAT and
firewall.
I.e. ipmon is reporting "bad" packets blocked even though the "pass"
rule they match is triggered. (too bad "bad" isn't well documented!)
bge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
capabilities=3f00<IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx>
caps_enabled=3f00<IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx>

Last time this was reported, I think the culprit was the hardware
checksumming, which ensures that the packets are correct *on the
wire*, but not yet in the machine's main memory. As I understand,
this would apply to transmitted packets only, but on a NAT box or
a router, most packets are transmitted at some point...

Switch .*SUM.* off for a quick test.

-is

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Greg A. Woods; Planix, Inc.]

Post by Ignatios Souvatzis
Last time this was reported, I think the culprit was the hardware
checksumming, which ensures that the packets are correct *on the
wire*, but not yet in the machine's main memory. As I understand,
this would apply to transmitted packets only, but on a NAT box or
a router, most packets are transmitted at some point...
Switch .*SUM.* off for a quick test.

Hmmm... yes, as I said in the rest of the message [:-)], that's what I
did, and it seems to work better now (at least no more packets are
logged as "bad" and my client hasn't called to complain that
everything is blocked).

I'm naively guessing the NAT is changing the UDP header but not
recalculating the checksum, perhaps because (part of) the NAT code
assumes the hardware will fix the checksum on the way out?

Anyway, in PR# 34799 it seemed to me that the problem was buggy
assumptions in the gem(4) driver about what capabilities the chip
actually had.

I'm wondering if the problem is really in the driver/chip or rather
instead in a fundamental conflict between the concept of hardware
checksumming and the IPF/NAT code which assumes it can both manipulate
the packets and then verify the checksums in the packets. I've
forgotten too much about the IPF code to remember where to dig into it
quickly to see if it has any logic flaws in its understanding of
what's possible when the interface does hardware checksumming.