managed switches on mii(4)

[Jonathan A. Kollasch]

Hi,

I'm interested in being able to control a 5/6-port managed switch
attached to a bce(4)'s mii(4) interface.

I saw the idea of using bridge(4) to take advantage of the hard-
ware switch, but somehow this seems like an unusual idea. The
switch hardware has more configuration options than just bridg-
ing. (Although those features may not be fully documented pub-
licly.)

A CatOS-like method of interface configuration seems to not match
traditional unix methods, but other methods of configuration such
as those used by OpenWrt sometimes seem clumsy, or at least con-
fusing.

So, what options have people thought about? What are their pros
and cons?

Jonathan Kollasch

[Herb Peyerl]

Post by Jonathan A. Kollasch
I saw the idea of using bridge(4) to take advantage of the hard-
ware switch, but somehow this seems like an unusual idea. The
switch hardware has more configuration options than just bridg-
ing. (Although those features may not be fully documented pub-
licly.)
So, what options have people thought about? What are their pros
and cons?

In my day-life, on another NetBSD platform, the hardware has a
Marvell switch controlled by ioctl()s sent by a userland
application. It's terribly proprietary and won't see human use, so
it's not intuitive and requires a block diagram of the system in
order to use. Previous attempts to configure the switch were made by
heavily modifying our ifconfig(8) but it was too limited, unwieldy,
and (inconclusively) caused at least one brain aneurism. I provide
the below not as an attempt to influence how this is done in the
future, but just in case someone does implement something, it might
illustrate what functions might be needed by other switches.
(wherever you see an option containing 11 digits or letters, then
those args reference all 11 ports on the switch. ie: DDDFCCDFSCS


usage: switchctl ifname reset switch
e.g. switchctl switch0 reset switch
switchctl ifname reset port port_num
e.g. switchctl switch0 reset port 4

switchctl ifname port read
e.g. switchctl switch0 port read
switchctl ifname port mdix 11_mdix_specifiers (A=auto,
X=MDIX, I=MDI)
e.g. switchctl switch0 mdix XXAAAIIIIIX
switchctl ifname port autoneg 11_autoneg_specifiers (0=AN
off, 1=AN on)
e.g. switchctl switch0 autoneg 00011010101

switchctl ifname port force speed port_num speed_val (0=auto|
10|100|1000)
e.g. switchctl switch0 port force speed 9 100
switchctl ifname port force duplex 11_duplex_states (0=no
force, H=half, F=full)
e.g. switchctl switch0 port force duplex 0FFFHHH0FH00
switchctl ifname port force link 11_link_specifiers (0=no
force, 1=force)
e.g. switchctl switch0 port force link 01111010000

switchctl ifname port_power [on|off] all
e.g. switchctl switch0 port_power on all
switchctl ifname port_power [on|off] port port_NUM
e.g. switchctl switch0 port_power off port 7
switchctl ifname port_power 11_power_states ([ofF|oN])
e.g. switchctl switch0 port_power FFFFFFNNNNN
switchctl ifname port_power read
e.g. switchctl switch0 port_power read

switchctl ifname activeport port_num
e.g. switchctl switch0 activeport 4
switchctl ifname activeport read
e.g. switchctl switch0 activeport read

switchctl ifname id productnum
e.g. switchctl switch0 id productnum
switchctl ifname id rev
e.g. switchctl switch0 id rev

switchctl ifname port_state 11_states ([D|B|L|F)
e.g. switchctl switch0 port_state DDDDDDFFFFF
switchctl ifname port_state port_num 1_port state ([D|B|L|F)
e.g. switchctl switch0 port_state 10 L
switchctl ifname port_state read
e.g. switchctl switch0 port_state read

switchctl ifname secureport [on|off] all
e.g. switchctl switch0 secureport on all
switchctl ifname secureport [on|off] port port_NUM
e.g. switchctl switch0 secureport off port 7
switchctl ifname secureport 11_secure_states ([S|U])
e.g. switchctl switch0 secureport UUUUUUSSSSS
switchctl ifname secureport read
e.g. switchctl switch0 secureport read

switchctl ifname 802_1X interrupt [on|off|read]
e.g. switchctl switch0 802_1X interrupt on
switchctl ifname 802_1X [on|off] all
e.g. switchctl switch0 802_1X on all
switchctl ifname 802_1X [on|off] port port_NUM
e.g. switchctl switch0 802_1X off port 7
switchctl ifname 802_1X 11_802_1X_states ([ofF|oN])
e.g. switchctl switch0 802_1X FFFFFFNNNNN
switchctl ifname 802_1X read
e.g. switchctl switch0 802_1X read

switchctl ifname stats flush [all|port_num]
e.g. switchctl switch0 stats flush 10
switchctl ifname stats histogram [rx|tx|both]
e.g. switchctl switch0 stats histogram both
switchctl ifname stats read [port_num]
e.g. switchctl switch0 stats read 8

switchctl ifname atu read
e.g. switchctl switch0 atu read
switchctl ifname atu age time_in_seconds
e.g. switchctl switch0 atu age 3825
switchctl ifname atu age read
e.g. switchctl switch0 atu age read
switchctl ifname atu flush [all|non-static]
e.g. switchctl switch0 atu flush non-static
switchctl ifname atu learn read
e.g. switchctl switch0 atu learn read
switchctl ifname atu learn 11_states (0|1)
e.g. switchctl switch0 atu learn 10000111111
switchctl ifname atu purge ##:##:##:##:##:##
e.g. switchctl switch0 atu purge 00:30:1B:AF:A9:AC
switchctl ifname atu load ##:##:##:##:##:##
allowed_ports_in_binary [normal|static]
e.g. switchctl switch0 atu load 00:30:1B:AF:A9:AC
10111001010 normal

switchctl ifname vlan port port_num allowed_ports_in_binary
e.g. switchctl switch0 vlan port 0 10111001010
switchctl ifname vlan port read
e.g. switchctl switch0 vlan port read
switchctl ifname vlan port reset
e.g. switchctl switch0 vlan port reset

switchctl ifname vlan 802_1Q interrupt [on|off|read]
e.g. switchctl ifname vlan 802_1Q interrupt read
switchctl ifname vlan 802_1Q mode 11_mode_specifiers ([D|F|C|S])
e.g. switchctl switch0 vlan 802_1Q mode DDDFCCDFSCS
switchctl ifname vlan 802_1Q mode read
e.g. switchctl switch0 vlan 802_1Q mode read
switchctl ifname vlan 802_1Q default_vid port_num vid#
e.g. switchctl switch0 vlan 802_1Q default_vid 2 0xB4A
switchctl ifname vlan 802_1Q read
e.g. switchctl switch0 vlan 802_1Q read
switchctl ifname vlan 802_1Q flush
e.g. switchctl switch0 vlan 802_1Q flush
switchctl ifname vlan 802_1Q purge vid#
e.g. switchctl switch0 vlan 802_1Q purge 0xB4A
switchctl ifname vlan 802_1Q load vid# 11_member_tags ([U|N|T|
D]) 11_port_states ([D|B|L|F)
e.g. switchctl switch0 vlan 802_1Q load 0xB4A
UUUNTTTTDDN LLLFFFDBDBB

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Thor Lancelot Simon]

Post by Herb Peyerl
In my day-life, on another NetBSD platform, the hardware has a
Marvell switch controlled by ioctl()s sent by a userland
application. It's terribly proprietary and won't see human use, so
it's not intuitive and requires a block diagram of the system in
order to use. [...]

I have a similar platform, though the switch is a Vitesse chip. We
control this switch over USB -- I like your configuration tool somewhat
better than ours. I can't really see how one could use such a tool
without knowing the block diagram of the system, as you note.

A common way to do this on low-cost/low-part-count systems is to use
an Ethernet MAC that does not have an integrated PHY (or to not use
its integrated PHY) and to use one port on the switch in "reverse MII"
mode as a Special PHY. Most decent switch chipsets have the rMII
feature now, but AFAICT (I have a bunch of databooks here) they do
_not_ allow you to configure the switching engine itself via
manipulating the "PHY" registers. I can see some good reasons why not
to do that, but without it, you need an out-of-band switch configuration
interface, like the USB we use -- and that's going to be totally switch
specific. :-/

Thor

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[David Young]

Post by Herb Peyerl

Post by Jonathan A. Kollasch
I saw the idea of using bridge(4) to take advantage of the hard-
ware switch, but somehow this seems like an unusual idea. The
switch hardware has more configuration options than just bridg-
ing. (Although those features may not be fully documented pub-
licly.)
So, what options have people thought about? What are their pros
and cons?

In my day-life, on another NetBSD platform, the hardware has a
Marvell switch controlled by ioctl()s sent by a userland
application. It's terribly proprietary and won't see human use, so
it's not intuitive and requires a block diagram of the system in
order to use. Previous attempts to configure the switch were made by
heavily modifying our ifconfig(8) but it was too limited, unwieldy,
and (inconclusively) caused at least one brain aneurism. I provide
the below not as an attempt to influence how this is done in the
future, but just in case someone does implement something, it might
illustrate what functions might be needed by other switches.
(wherever you see an option containing 11 digits or letters, then
those args reference all 11 ports on the switch. ie: DDDFCCDFSCS

Let me suggest a different approach to supporting the Marvell switch than
switchctl. We already have a familiar abstraction for an ethernet port in
our existing drivers (tlp(4), wm(4), et cetera), for a switch/hub/bridge
in bridge(4), and for a VLAN in vlan(4). Let us reuse those. Represent
the ports in the Marvell switch using 11 interfaces, mrvl0..mrvl10.
Change each port's media settings and such in the usual way with ifconfig.
Connect/partition ports on the switch with bridge(4). Add and remove
VLAN tags on some ports with vlan(4). Let the underlying hardware driver
set the switching matrix and other parameters in order to optimize the
configuration of mrvl0..10, vlan0..N, and bridge0..M.

Dave

[Herb Peyerl]

Post by David Young
Let me suggest a different approach to supporting the Marvell
switch than
switchctl. We already have a familiar abstraction for an ethernet port in
our existing drivers (tlp(4), wm(4), et cetera), for a switch/hub/
bridge
in bridge(4), and for a VLAN in vlan(4). Let us reuse those.
Represent
the ports in the Marvell switch using 11 interfaces, mrvl0..mrvl10.
Change each port's media settings and such in the usual way with ifconfig.
Connect/partition ports on the switch with bridge(4). Add and remove
VLAN tags on some ports with vlan(4). Let the underlying hardware driver
set the switching matrix and other parameters in order to optimize the
configuration of mrvl0..10, vlan0..N, and bridge0..M.

I'm not immediately convinced that's a good idea. Suddenly you're
moving the configuration of the switch to a bunch of extra utilities
like brconfig, ifconfig... There are other functions like clearing
mac addresses out of the cache on certain ports that don't
necessarily fit into any specific utility, and other functions like
querying the various status registers and event counters in the
switch hardware, that would creep into everywhere...

The project I'm currently on considers this problem 'solved' so I
won't be able to put any time towards it while I have access to
hardware...


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[David Young]

Post by Herb Peyerl

Post by David Young
Let me suggest a different approach to supporting the Marvell
switch than
switchctl. We already have a familiar abstraction for an ethernet port in
our existing drivers (tlp(4), wm(4), et cetera), for a switch/hub/
bridge
in bridge(4), and for a VLAN in vlan(4). Let us reuse those.
Represent
the ports in the Marvell switch using 11 interfaces, mrvl0..mrvl10.
Change each port's media settings and such in the usual way with ifconfig.
Connect/partition ports on the switch with bridge(4). Add and remove
VLAN tags on some ports with vlan(4). Let the underlying hardware driver
set the switching matrix and other parameters in order to optimize the
configuration of mrvl0..10, vlan0..N, and bridge0..M.

I'm not immediately convinced that's a good idea. Suddenly you're
moving the configuration of the switch to a bunch of extra utilities
like brconfig, ifconfig...

I don't understand what you mean by "extra utilities." ifconfig and
brconfig already exist. I reckon that they can cover the most common
switch use cases, too.

Post by Herb Peyerl
There are other functions like clearing
mac addresses out of the cache on certain ports that don't
necessarily fit into any specific utility,

That's a good fit for brconfig(8). The manual page already mentions

deladdr address
Delete address from the address cache.

Post by Herb Peyerl
and other functions like
querying the various status registers and event counters in the
switch hardware, that would creep into everywhere...

Why do you think so?

Dave

[Herb Peyerl]

Post by David Young

Post by Herb Peyerl
I'm not immediately convinced that's a good idea. Suddenly you're
moving the configuration of the switch to a bunch of extra utilities
like brconfig, ifconfig...

I don't understand what you mean by "extra utilities." ifconfig and
brconfig already exist. I reckon that they can cover the most common
switch use cases, too.

Yes, I wasn't clear. I've re-read that and can't think of what I
meant by the word 'extra'. My general sentiment was that
decentralizing the configuration of the switch may not be immediately
desirable.

Post by David Young

Post by Herb Peyerl
and other functions like
querying the various status registers and event counters in the
switch hardware, that would creep into everywhere...

Why do you think so?

Utilities like netstat simply report data kept in struct ifnet, for
example. Data which is updated by the driver handling the packets.
But the switch will be handling packets while the driver is really
just a control-plane interface to the switch; and so stats don't get
collected there, except by a timer-based poll of the switch
hardware... As a result, I don't think you'd get a completely
accurate representation of this sort of data:

Stats for port 9:
inGoodOctetsLo 9122
inGoodOctetsHi 0
inBadOctets 0
inUnicast 62
inBroadcast 0
inMulticast 0
inPause 0
inUndersize 0
inFragments 0
inOversize 0
inJabber 0
inRxErr 0
inFcsErr 0
outOctetsLo 8743
outOctetsHi 0
outUnicast 62
outBroadcast 2
outMulticast 0
outPause 0
outFcsErr 0
excessive 0
collisions 0
deferred 0
single 0
multiple 0
late 0
octets64 7
octets65to127 88
octets128to255 27
octets256to511 1
octets512to1023 3
octets1024toMax 0
inDiscardsLo 0
inDiscardsHi 0
inFiltered 0
outFiltered 0
Stats operation completed successfully.

Going through the rest of the list, I guess I can see a home for the
rest of the functionality... Maybe I'm coming around to this after
all....






--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Thor Lancelot Simon]

Post by David Young
Let me suggest a different approach to supporting the Marvell switch than
switchctl. We already have a familiar abstraction for an ethernet port in
our existing drivers (tlp(4), wm(4), et cetera), for a switch/hub/bridge
in bridge(4), and for a VLAN in vlan(4). Let us reuse those. Represent
the ports in the Marvell switch using 11 interfaces, mrvl0..mrvl10.
Change each port's media settings and such in the usual way with ifconfig.
Connect/partition ports on the switch with bridge(4). Add and remove
VLAN tags on some ports with vlan(4). Let the underlying hardware driver
set the switching matrix and other parameters in order to optimize the
configuration of mrvl0..10, vlan0..N, and bridge0..M.

This might work with the Marvell switch Herb's platform uses. I think it
would be fairly hard to do for the USB-controlled ones I've got, though.

[David Young]

Post by Jonathan A. Kollasch
Hi,
I'm interested in being able to control a 5/6-port managed switch
attached to a bce(4)'s mii(4) interface.
I saw the idea of using bridge(4) to take advantage of the hard-
ware switch, but somehow this seems like an unusual idea. The
switch hardware has more configuration options than just bridg-
ing. (Although those features may not be fully documented pub-
licly.)

What are those configuration options?

Dave

[Jonathan A. Kollasch]

Post by David Young

Post by Jonathan A. Kollasch
Hi,
I'm interested in being able to control a 5/6-port managed switch
attached to a bce(4)'s mii(4) interface.
I saw the idea of using bridge(4) to take advantage of the hard-
ware switch, but somehow this seems like an unusual idea. The
switch hardware has more configuration options than just bridg-
ing. (Although those features may not be fully documented pub-
licly.)

What are those configuration options?

In the meantime I discovered a programmers guide for the hardware.
Google for "bcm5365p.pdf". (I have no idea if this is actually
supposed to be public.) Anyway, this seems to have a complete
description of the hardware. (This includes information on
ubsec(4) and bce(4) btw.) The switch appears to have all the
features of a decent managed switch. VLANs, mirroring, QoS,
MAC address table stuff, statistics collection, etc.

Jonathan Kollasch