Slow transmission with Marvell SKnet

[Frank Wille]

Hi,

I think we have a problem in the sk(4) driver for Marvell Yukon SK-Net.
Probably since ever. But I didn't notice it before I made one of my sk(4)
based Synology NAS into a media server.

I have three NetBSD/sandpoint based NAS with the same sk(4) chip, which
all show transmit rates between 20 Kib/s and 1800 KiB/s during a simple FTP
test, while receiving is always at more than 5000 KiB/s: DS-101g+, DS-209j
and CS-406e:

skc0 at pci0 dev 15 function 0: irq 18
skc0: interrupt moderation is 0 us
skc0: Marvell Yukon Lite Gigabit Ethernet rev. (0x9)
sk0 at skc0 port A: Ethernet address 00:11:32:00:xx:xx
makphy0 at sk0 phy 0: Marvell 88E1011 Gigabit PHY, rev. 5

While the receive-speed is quite constant, the transmit-speed varies a lot,
depending on the machine and NIC on the other end of the FTP connection.
For example I have seen less than 50 KiB/s on a iBook G4 (NetBSD/macppc)
with gem(4), where the rate quickly oscillates between 20 and 200 KiB/s
during the transmission.

There are no errors or collisions, though:

Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Colls
sk0 1500 <Link> 00:11:32:00:xx:xx 581303 0 426085 0 0

The interface is configured like this (100MBit, because I have no GBit
ethernet or switches):

sk0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
ec_capabilities=5<VLAN_MTU,JUMBO_MTU>
ec_enabled=0
address: 00:11:32:00:xx:xx
media: Ethernet 100baseTX full-duplex
status: active
inet 192.168.41.101 netmask 0xffffff00 broadcast 192.168.41.255

I have a comparable Qnap TS-101 NAS, with the same CPU, RAM and clock, but
with a Realtek NIC. It has no problem transmitting more than 4 MiB/s, to
any client I connect!

Can anybody confirm the problem? Or am I missing something?

[Robert Swindells]

Post by Frank Wille
I think we have a problem in the sk(4) driver for Marvell Yukon SK-Net.
Probably since ever. But I didn't notice it before I made one of my sk(4)
based Synology NAS into a media server.

[snip]

Post by Frank Wille
While the receive-speed is quite constant, the transmit-speed varies a lot,
depending on the machine and NIC on the other end of the FTP connection.
For example I have seen less than 50 KiB/s on a iBook G4 (NetBSD/macppc)
with gem(4), where the rate quickly oscillates between 20 and 200 KiB/s
during the transmission.

Sounds to me like a PHY negotiation problem on the other end of the
link.

Have you tried testing with a crossover cable between the test machines
and no switch ?

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Frank Wille]

Post by Robert Swindells

Post by Frank Wille
While the receive-speed is quite constant, the transmit-speed varies a
lot, depending on the machine and NIC on the other end of the FTP
connection. For example I have seen less than 50 KiB/s on a iBook G4
(NetBSD/macppc) with gem(4), where the rate quickly oscillates between
20 and 200 KiB/s during the transmission.

Sounds to me like a PHY negotiation problem on the other end of the
link.

I would be surprised when there is a problem on the non-sk(4) end of the
link, as it never happens when no SKnet NICs are involved.

Would it change something when I configure a kernel with ukphy at sk instead
of makphy at sk?

Post by Robert Swindells
Have you tried testing with a crossover cable between the test machines
and no switch ?

No, I didn't. Might be worth a check. I will try it this evening and report
back. Thanks!

[Frank Wille]

Post by Robert Swindells
Sounds to me like a PHY negotiation problem on the other end of the
link.
Have you tried testing with a crossover cable between the test machines
and no switch ?

Did that now with two test machines:

1. iBook G4, Apple GMac, gem(4)
Send to: 10 KiB/s
Receive from: 5500 KiB/s

2. Pentium IV 3GHz, Realtek 8139, rtk(4)
Send to: 10 KiB/s
Receive from: 5800 KiB/s

Does that mean we can exclude a problem with PHY negotiation?

[Robert Swindells]

Post by Frank Wille

Post by Robert Swindells
Sounds to me like a PHY negotiation problem on the other end of the
link.
Have you tried testing with a crossover cable between the test machines
and no switch ?

1. iBook G4, Apple GMac, gem(4)
Send to: 10 KiB/s
Receive from: 5500 KiB/s
2. Pentium IV 3GHz, Realtek 8139, rtk(4)
Send to: 10 KiB/s
Receive from: 5800 KiB/s

Ok.

Post by Frank Wille
Does that mean we can exclude a problem with PHY negotiation?

No.

Next, look at ifconfig(8) and netstat -i status for the two test
machines.

Then try explicitly setting 100baseTX and full duplex for the test
machines.

What is the output of:

# ifconfig -m sk0

You could also try running the sk0 as autoselect, both with the test
machines and the switch.

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Frank Wille]

Post by Robert Swindells
# ifconfig -m sk0
You could also try running the sk0 as autoselect, both with the test
machines and the switch.

Thank you very much! I fixed the problem! It was probably my fault.

Explanation: I know that the GBit-mode requires much more energy than 100
MBit, so I always tried to limit the interfaces to 100 MBit. The 8241 CPU
is not capable to exceed the full 100 MBit bandwidth anyway.

For some NICs (for example re(4)) the following ifconfig works fine:
media 100baseTX mediaopt full-duplex

But appranently not for sk(4). These media options do not work:

sk0: flags=0x8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
ec_capabilities=5<VLAN_MTU,JUMBO_MTU>
ec_enabled=0
address: 00:11:32:05:xx:xx
media: Ethernet 100baseTX full-duplex
status: active
supported Ethernet media:
media none
media 10baseT
media 10baseT mediaopt full-duplex
media 100baseTX
media 100baseTX mediaopt full-duplex
media 1000baseT
media 1000baseT mediaopt full-duplex
media autoselect

Configuring the interface with "autoselect" on a 100 MBit switch shows the
following:

media: Ethernet autoselect (100baseTX
full-duplex,flowcontrol,rxpause,txpause)

So I was probably missing the flowcontrol, rxpause and txpause options for
sk(4), which re(4) didn't need?

[Thor Lancelot Simon]

Post by Frank Wille

Post by Robert Swindells
# ifconfig -m sk0
You could also try running the sk0 as autoselect, both with the test
machines and the switch.

Thank you very much! I fixed the problem! It was probably my fault.
Explanation: I know that the GBit-mode requires much more energy than 100
MBit, so I always tried to limit the interfaces to 100 MBit. The 8241 CPU
is not capable to exceed the full 100 MBit bandwidth anyway.

I think -- I'm not certain -- that when we select a particular link
configuration we then disable autonegotiation.

In the modern era, this is a bug! There is no way for our link partner
to know what we did unless it too is explicitly configured to disable
autoneg (some modern link partners can't even be configured that way).

What we should do is what everyone else does: leave autoneg enabled, but
advertise, and accept, only the single mode the user asked for. Then it
would work how you expect and there would be, in particular, no duplex
mismatch between the two sides of the link, which is what caused your
problem.

Thor

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Michael van Elst]

Post by Frank Wille
media 100baseTX mediaopt full-duplex

Did you also fix the switch to 100baseTX or was it left on autoselect?

Post by Frank Wille
media: Ethernet autoselect (100baseTX
full-duplex,flowcontrol,rxpause,txpause)
So I was probably missing the flowcontrol, rxpause and txpause options for
sk(4), which re(4) didn't need?

Unlikely. These options are significant if you connect network segments
with different speeds to throttle the faster segment to not overrun
switch buffers.

[Michael van Elst]

Post by Thor Lancelot Simon
What we should do is what everyone else does: leave autoneg enabled, but
advertise, and accept, only the single mode the user asked for. Then it
would work how you expect and there would be, in particular, no duplex
mismatch between the two sides of the link, which is what caused your
problem.

You cannot mix fixed speed and autoneg. That's why you need to be able
to configure fixed speeds to be able to talk to old gear that doesn't
support autoneg (or not correctly).

Disabling certain speeds from autoneg is a fringe case compared to that,
but we should support it too. But lets not trade one for the other.

[Thor Lancelot Simon]

Post by Michael van Elst
Disabling certain speeds from autoneg is a fringe case compared to that,
but we should support it too. But lets not trade one for the other.

That "fringe case" is the default behavior of Windows, OS X, Linux,
most FreeBSD drivers, and current versions of most router operating
systems. If you want our broken behavior of *implicitly* disabling
autoneg when the user asks for a specific speed/duplex (disabling
the others) you have to ask for it -- if it's supported at all.

Our default behavior of disabling autoneg when the user requests
a particular mode differs from nearly all others and interoperates
poorly (to say the least).

Thor

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Michael van Elst]

Post by Thor Lancelot Simon

Post by Michael van Elst
Disabling certain speeds from autoneg is a fringe case compared to that,
but we should support it too. But lets not trade one for the other.

That "fringe case" is the default behavior of Windows, OS X, Linux,
most FreeBSD drivers, and current versions of most router operating
systems.

In Linux you specify either a fixed speed (with autoneg off) or
the set of speeds that autoneg may advertise. There is no
default behaviour (for what?).

Lots of hardware doesn't even allow you to configure what speeds it
advertises when autoneg is enabled. That's a rather "modern" feature.

The fringe case is to have working autoconfig but advertising a lower
speed than possible. I can imagine a few case where people are willing
to do that, in particular when they can only control one side of the link.
But then it's usually a switch or router on both ends.