problems with PCIX-bus NICs

[6***@6bone.informatik.uni-leipzig.de]

hello,

I am using a Dell 1850. The onboard Intel NICs works fine.
Additional PCIX cards are not working correct.

ifconfig wm0
wm0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
capabilities=2bf80<TSO4,IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Tx,UDP6CSUM_Tx>
enabled=0
address: 00:04:23:c3:96:78
media: Ethernet autoselect (1000baseT full-duplex)
status: active
inet6 2001:638:c:a00a::3 prefixlen 64
inet6 fe80::204:23ff:fec3:9678%wm0 prefixlen 64 scopeid 0x1

dmesg | grep wm0
wm0 at pci2 dev 12 function 0: Intel i82546GB 1000BASE-T Ethernet, rev. 3
wm0: interrupting at ioapic0 pin 16
wm0: 64-bit 100MHz PCIX bus
wm0: 256 word (8 address bits) MicroWire EEPROM
wm0: Ethernet address 00:04:23:c3:96:78
makphy0 at wm0 phy 1: Marvell 88E1011 Gigabit PHY, rev. 5


bash-4.0# ping6 2001:638:c:a00a::2
PING6(56=40+8+8 bytes) 2001:638:c:a00a::3 --> 2001:638:c:a00a::2
16 bytes from 2001:638:c:a00a::2, icmp_seq=0 hlim=64 time=15239.578 ms
16 bytes from 2001:638:c:a00a::2, icmp_seq=1 hlim=64 time=14164.122 ms
16 bytes from 2001:638:c:a00a::2, icmp_seq=2 hlim=64 time=13160.750 ms
16 bytes from 2001:638:c:a00a::2, icmp_seq=3 hlim=64 time=12168.065 ms
16 bytes from 2001:638:c:a00a::2, icmp_seq=4 hlim=64 time=11164.693 ms
16 bytes from 2001:638:c:a00a::2, icmp_seq=5 hlim=64 time=10161.225 ms
16 bytes from 2001:638:c:a00a::2, icmp_seq=6 hlim=64 time=9168.556 ms
16 bytes from 2001:638:c:a00a::2, icmp_seq=7 hlim=64 time=8165.168 ms
16 bytes from 2001:638:c:a00a::2, icmp_seq=8 hlim=64 time=7161.975 ms


I have changed the Intel NIC and also tested a Broadcom based NIC on
different PCIX Slots. Always the same behavior.

The same test with a onboard NIC reports a ping time less than 1 ms.


Any Ideas what could be the problem?


Thank you for your efforts
Uwe

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Robert Elz]

Date: Wed, 15 Jul 2009 14:26:37 +0200 (CEST)
From: ***@6bone.informatik.uni-leipzig.de
Message-ID: <***@6bone.informatik.uni-leipzig.de>

| Any Ideas what could be the problem?

If you haven't done it already, rre-run your test, and give ping6 the -n
flag - the interaction of ping's packet receivin/timing code and the
DNS resolver library routines generates a huge mess, and you can essentially
ignore the times whenever -n is not used (if you're lucky, and all the
nameservers are correctly configured, and ideally close, then you can
get reasonable results - but if there are misconfigured or non-responding
nameservers, it is all just a mess).

Ideally we'd have a way to have the kernel timestamp arriving packets, and
pass that info back to the application along with the rest of the control
info that's available, but either NetBSD has no way to do that, or
ping6 just doesn't use it, so the time it presents is the time from
when the packet was sent, until when the ping6 application got around to
processing the reply - if it gets itself stuck inside the resolver, the
returned packet can have sat in the kernel waiting to be fetched for a
long time.

I'd tend to suspect that the difference between the ping6 results you're
seeing on the two different interfaces is more a reflection of the
relevant .IP6.ARPA zone configurations for the different addresses
that anything else.

If you still see the same kind of results when you use -n, ask again...

kre


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[6***@6bone.informatik.uni-leipzig.de]

Post by Robert Elz
If you still see the same kind of results when you use -n, ask again...
kre

the problem also occurs with ping6 -n.

the following ping6/tcpdump output shows the problem.


regards
Uwe


bash-4.0# ping6 -n 2001:638:c:a00a::2
PING6(56=40+8+8 bytes) 2001:638:c:a00a::3 --> 2001:638:c:a00a::2
16 bytes from 2001:638:c:a00a::2, icmp_seq=0 hlim=64 time=12271.127 ms
16 bytes from 2001:638:c:a00a::2, icmp_seq=1 hlim=64 time=11200.353 ms
16 bytes from 2001:638:c:a00a::2, icmp_seq=2 hlim=64 time=10196.919 ms
16 bytes from 2001:638:c:a00a::2, icmp_seq=3 hlim=64 time=9204.009 ms
16 bytes from 2001:638:c:a00a::2, icmp_seq=4 hlim=64 time=8200.486 ms
16 bytes from 2001:638:c:a00a::2, icmp_seq=5 hlim=64 time=7196.852 ms
16 bytes from 2001:638:c:a00a::2, icmp_seq=6 hlim=64 time=6204.068 ms
16 bytes from 2001:638:c:a00a::2, icmp_seq=7 hlim=64 time=5200.651 ms
16 bytes from 2001:638:c:a00a::2, icmp_seq=8 hlim=64 time=4197.179 ms


tcpdump -n at 2001:638:c:a00a::3
20:32:54.373521 00:04:23:c3:96:78 > 00:04:23:c2:ab:4e, ethertype IPv6 (0x86dd),length 70: 2001:638:c:a00a::3 > 2001:638:c:a00a::2: ICMP6, echo request, seq 0,length 16
20:32:55.444351 00:04:23:c3:96:78 > 00:04:23:c2:ab:4e, ethertype IPv6 (0x86dd),length 70: 2001:638:c:a00a::3 > 2001:638:c:a00a::2: ICMP6, echo request, seq 1,length 16
20:32:56.447798 00:04:23:c3:96:78 > 00:04:23:c2:ab:4e, ethertype IPv6 (0x86dd),length 70: 2001:638:c:a00a::3 > 2001:638:c:a00a::2: ICMP6, echo request, seq 2,length 16
20:32:57.440749 00:04:23:c3:96:78 > 00:04:23:c2:ab:4e, ethertype IPv6 (0x86dd),length 70: 2001:638:c:a00a::3 > 2001:638:c:a00a::2: ICMP6, echo request, seq 3,length 16
20:32:58.444336 00:04:23:c3:96:78 > 00:04:23:c2:ab:4e, ethertype IPv6 (0x86dd),length 70: 2001:638:c:a00a::3 > 2001:638:c:a00a::2: ICMP6, echo request, seq 4,length 16
20:32:59.448085 00:04:23:c3:96:78 > 00:04:23:c2:ab:4e, ethertype IPv6 (0x86dd),length 70: 2001:638:c:a00a::3 > 2001:638:c:a00a::2: ICMP6, echo request, seq 5,length 16
20:32:59.704285 00:04:23:c3:96:78 > 00:04:23:c2:ab:4e, ethertype IPv6 (0x86dd),length 86: 2001:638:c:a00a::3 > 2001:638:c:a00a::2: ICMP6, neighbor solicitation, who has 2001:638:c:a00a::2, length 32
20:33:00.440954 00:04:23:c3:96:78 > 00:04:23:c2:ab:4e, ethertype IPv6 (0x86dd),length 70: 2001:638:c:a00a::3 > 2001:638:c:a00a::2: ICMP6, echo request, seq 6,length 16
20:33:00.771869 00:04:23:c3:96:78 > 00:04:23:c2:ab:4e, ethertype IPv6 (0x86dd),length 86: 2001:638:c:a00a::3 > 2001:638:c:a00a::2: ICMP6, neighbor solicitation, who has 2001:638:c:a00a::2, length 32
20:33:01.444461 00:04:23:c3:96:78 > 00:04:23:c2:ab:4e, ethertype IPv6 (0x86dd),length 70: 2001:638:c:a00a::3 > 2001:638:c:a00a::2: ICMP6, echo request, seq 7,length 16
20:33:01.839482 00:04:23:c3:96:78 > 00:04:23:c2:ab:4e, ethertype IPv6 (0x86dd),length 86: 2001:638:c:a00a::3 > 2001:638:c:a00a::2: ICMP6, neighbor solicitation, who has 2001:638:c:a00a::2, length 32
20:33:02.448015 00:04:23:c3:96:78 > 00:04:23:c2:ab:4e, ethertype IPv6 (0x86dd),length 70: 2001:638:c:a00a::3 > 2001:638:c:a00a::2: ICMP6, echo request, seq 8,length 16
20:33:03.441568 00:04:23:c3:96:78 > 33:33:ff:00:00:02, ethertype IPv6 (0x86dd),length 86: 2001:638:c:a00a::3 > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2001:638:c:a00a::2, length 32
20:33:04.509114 00:04:23:c3:96:78 > 33:33:ff:00:00:02, ethertype IPv6 (0x86dd),length 86: 2001:638:c:a00a::3 > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2001:638:c:a00a::2, length 32
20:33:05.576510 00:04:23:c3:96:78 > 33:33:ff:00:00:02, ethertype IPv6 (0x86dd),length 86: 2001:638:c:a00a::3 > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has 2001:638:c:a00a::2, length 32
20:33:06.644243 00:04:23:c2:ab:4e > 00:04:23:c3:96:78, ethertype IPv6 (0x86dd),length 70: 2001:638:c:a00a::2 > 2001:638:c:a00a::3: ICMP6, echo reply, seq 0, length 16
20:33:06.644245 00:04:23:c2:ab:4e > 00:04:23:c3:96:78, ethertype IPv6 (0x86dd),length 70: 2001:638:c:a00a::2 > 2001:638:c:a00a::3: ICMP6, echo reply, seq 1, length 16
20:33:06.644246 00:04:23:c2:ab:4e > 00:04:23:c3:96:78, ethertype IPv6 (0x86dd),length 70: 2001:638:c:a00a::2 > 2001:638:c:a00a::3: ICMP6, echo reply, seq 2, length 16


tcpdump -n at 2001:638:c:a00a::2
20:32:54.370781 00:04:23:c3:96:78 > 00:04:23:c2:ab:4e, ethertype IPv6 (0x86dd),length 70: 2001:638:c:a00a::3 > 2001:638:c:a00a::2: icmp6: echo request seq 0
20:32:54.370804 00:04:23:c2:ab:4e > 00:04:23:c3:96:78, ethertype IPv6 (0x86dd),length 70: 2001:638:c:a00a::2 > 2001:638:c:a00a::3: icmp6: echo reply seq 0
20:32:55.441534 00:04:23:c3:96:78 > 00:04:23:c2:ab:4e, ethertype IPv6 (0x86dd),length 70: 2001:638:c:a00a::3 > 2001:638:c:a00a::2: icmp6: echo request seq 1
20:32:55.441557 00:04:23:c2:ab:4e > 00:04:23:c3:96:78, ethertype IPv6 (0x86dd),length 70: 2001:638:c:a00a::2 > 2001:638:c:a00a::3: icmp6: echo reply seq 1
20:32:56.445008 00:04:23:c3:96:78 > 00:04:23:c2:ab:4e, ethertype IPv6 (0x86dd),length 70: 2001:638:c:a00a::3 > 2001:638:c:a00a::2: icmp6: echo request seq 2
20:32:56.445031 00:04:23:c2:ab:4e > 00:04:23:c3:96:78, ethertype IPv6 (0x86dd),length 70: 2001:638:c:a00a::2 > 2001:638:c:a00a::3: icmp6: echo reply seq 2
20:32:57.438055 00:04:23:c3:96:78 > 00:04:23:c2:ab:4e, ethertype IPv6 (0x86dd),length 70: 2001:638:c:a00a::3 > 2001:638:c:a00a::2: icmp6: echo request seq 3
20:32:57.438077 00:04:23:c2:ab:4e > 00:04:23:c3:96:78, ethertype IPv6 (0x86dd),length 70: 2001:638:c:a00a::2 > 2001:638:c:a00a::3: icmp6: echo reply seq 3
20:32:58.441529 00:04:23:c3:96:78 > 00:04:23:c2:ab:4e, ethertype IPv6 (0x86dd),length 70: 2001:638:c:a00a::3 > 2001:638:c:a00a::2: icmp6: echo request seq 4
20:32:58.441550 00:04:23:c2:ab:4e > 00:04:23:c3:96:78, ethertype IPv6 (0x86dd),length 70: 2001:638:c:a00a::2 > 2001:638:c:a00a::3: icmp6: echo reply seq 4
20:32:59.367858 00:04:23:c2:ab:4e > 00:04:23:c3:96:78, ethertype IPv6 (0x86dd),length 86: 2001:638:c:a00a::2 > 2001:638:c:a00a::3: icmp6: neighbor sol: who has 2001:638:c:a00a::3
20:32:59.445288 00:04:23:c3:96:78 > 00:04:23:c2:ab:4e, ethertype IPv6 (0x86dd),length 70: 2001:638:c:a00a::3 > 2001:638:c:a00a::2: icmp6: echo request seq 5
20:32:59.445309 00:04:23:c2:ab:4e > 00:04:23:c3:96:78, ethertype IPv6 (0x86dd),length 70: 2001:638:c:a00a::2 > 2001:638:c:a00a::3: icmp6: echo reply seq 5
20:32:59.701549 00:04:23:c3:96:78 > 00:04:23:c2:ab:4e, ethertype IPv6 (0x86dd),length 86: 2001:638:c:a00a::3 > 2001:638:c:a00a::2: icmp6: neighbor sol: who has 2001:638:c:a00a::2
20:32:59.701588 00:04:23:c2:ab:4e > 00:04:23:c3:96:78, ethertype IPv6 (0x86dd),length 78: 2001:638:c:a00a::2 > 2001:638:c:a00a::3: icmp6: neighbor adv: tgt is2001:638:c:a00a::2
20:33:00.367861 00:04:23:c2:ab:4e > 00:04:23:c3:96:78, ethertype IPv6 (0x86dd),length 86: 2001:638:c:a00a::2 > 2001:638:c:a00a::3: icmp6: neighbor sol: who has 2001:638:c:a00a::3
20:33:00.438191 00:04:23:c3:96:78 > 00:04:23:c2:ab:4e, ethertype IPv6 (0x86dd),length 70: 2001:638:c:a00a::3 > 2001:638:c:a00a::2: icmp6: echo request seq 6
20:33:00.438209 00:04:23:c2:ab:4e > 00:04:23:c3:96:78, ethertype IPv6 (0x86dd),length 70: 2001:638:c:a00a::2 > 2001:638:c:a00a::3: icmp6: echo reply seq 6
20:33:00.769371 00:04:23:c3:96:78 > 00:04:23:c2:ab:4e, ethertype IPv6 (0x86dd),length 86: 2001:638:c:a00a::3 > 2001:638:c:a00a::2: icmp6: neighbor sol: who has 2001:638:c:a00a::2
20:33:00.769407 00:04:23:c2:ab:4e > 00:04:23:c3:96:78, ethertype IPv6 (0x86dd),length 78: 2001:638:c:a00a::2 > 2001:638:c:a00a::3: icmp6: neighbor adv: tgt is2001:638:c:a00a::2

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Manuel Bouyer]

Post by 6***@6bone.informatik.uni-leipzig.de
hello,
I am using a Dell 1850. The onboard Intel NICs works fine.
Additional PCIX cards are not working correct.
ifconfig wm0
wm0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
capabilities=2bf80<TSO4,IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Tx,UDP6CSUM_Tx>
enabled=0
address: 00:04:23:c3:96:78
media: Ethernet autoselect (1000baseT full-duplex)
status: active
inet6 2001:638:c:a00a::3 prefixlen 64
inet6 fe80::204:23ff:fec3:9678%wm0 prefixlen 64 scopeid 0x1
dmesg | grep wm0
wm0 at pci2 dev 12 function 0: Intel i82546GB 1000BASE-T Ethernet, rev. 3
wm0: interrupting at ioapic0 pin 16
wm0: 64-bit 100MHz PCIX bus
wm0: 256 word (8 address bits) MicroWire EEPROM
wm0: Ethernet address 00:04:23:c3:96:78
makphy0 at wm0 phy 1: Marvell 88E1011 Gigabit PHY, rev. 5
bash-4.0# ping6 2001:638:c:a00a::2
PING6(56=40+8+8 bytes) 2001:638:c:a00a::3 --> 2001:638:c:a00a::2
16 bytes from 2001:638:c:a00a::2, icmp_seq=0 hlim=64 time=15239.578 ms
[...]
I have changed the Intel NIC and also tested a Broadcom based NIC on
different PCIX Slots. Always the same behavior.
The same test with a onboard NIC reports a ping time less than 1 ms.
Any Ideas what could be the problem?

I also have Dells 1850s at work. They suffers from interrupt routing
problems with NetBSD (what you see here smeels like interrupt from the
adapter are not making it to the driver's interrupt handler).
On a NetBSD 5.0 system I had to disable ACPI to get a add-on SCSI adapter
working.

[6***@6bone.informatik.uni-leipzig.de]

Hello,

I have tested a kernel without ACPI and the problem doesn't occur.
But can a kernel without ACPI handle only one CPU? This would not be a
solution of the problem.

Regards
Uwe

Post by Manuel Bouyer
I also have Dells 1850s at work. They suffers from interrupt routing
problems with NetBSD (what you see here smeels like interrupt from the
adapter are not making it to the driver's interrupt handler).
On a NetBSD 5.0 system I had to disable ACPI to get a add-on SCSI adapter
working.
--
NetBSD: 26 ans d'experience feront toujours la difference
--

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Manuel Bouyer]

Post by 6***@6bone.informatik.uni-leipzig.de
Hello,
I have tested a kernel without ACPI and the problem doesn't occur.
But can a kernel without ACPI handle only one CPU? This would not be a
solution of the problem.

No, a non-ACPI kernel should be able to use the MPBIOS to use SMP.
It does on my system.

[David Young]

Post by Manuel Bouyer
I also have Dells 1850s at work. They suffers from interrupt routing
problems with NetBSD (what you see here smeels like interrupt from the
adapter are not making it to the driver's interrupt handler).
On a NetBSD 5.0 system I had to disable ACPI to get a add-on SCSI adapter
working.

Is this a bug in the 1850's ACPI BIOS, or in NetBSD's ACPI code? If
it's a BIOS bug, maybe Dell has a BIOS update that fixes it?

Dave

[6***@6bone.informatik.uni-leipzig.de]

Post by David Young
Is this a bug in the 1850's ACPI BIOS, or in NetBSD's ACPI code? If
it's a BIOS bug, maybe Dell has a BIOS update that fixes it?

I updated the complete firmware of the 1850. No result. I cannot make a
service call at Dell because NetBSD is not an OS which is supported by
Dell.
If I test with Linux (knoppix) everything works fine, so I think it is a
problem of the netbsd code.


Uwe


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Joerg Sonnenberger]

Post by 6***@6bone.informatik.uni-leipzig.de
If I test with Linux (knoppix) everything works fine, so I think it is a
problem of the netbsd code.

Please provide me (off-list) the full dmesg from Linux, the dmesg from
NetBSD and the result of "acpidump -o dell1850.dsdt".

Another thing to try is to compile a custom kernel with "no ioapic*" to
force PIC mode.

Joerg

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de