Transitioning to 802.11q VLANs on a network using a NetBSD box with multiple physical interfaces

[Douglas Wade Needham]

Greetings,

I am in the process of doing some major reworking of my network to
include 802.11q VLANs, and have some questions for folks who are a bit
more familiar with VLANs and how they are implemented on NetBSD. But
to provide an answer, you first will need to know what my network
looks like at present.

Presently, I have a box called "alpha", running NetBSD with 3 Pro/100
(fxp) interfaces, doing NAPT thanks to ipf. The first interface on
alpha is configured with my 5 public IP addresses, the second has a
secure (internal) subnet attached to it, and the third has a DMZ
subnet on which resides my bastion hosts handling mail, web and other
gateway services. The use of ipfilter/ipnat on this box handles the
NAPT which takes place to direct inbound connections to various boxes,
as well as isolating the three networks. For the sake of the
discussion, we will say that these three subnets are:

fxp0 68.164.221.208/29 Public
fxp1 192.168.0.0/24 Private/Secure
fxp2 192.168.1.0/24 DMZ

All these networks are connected via a DLink DES-3226 switch which can
do 802.11q VLANs, as well as port based VLANs (currently in use to put
all three subnets on the same switch, so that traffic usage can be
monitored).

Now, in recent months, I have started adding boxes for a wireless
network which for various reasons really should be on a fourth subnet,
and other development (working with CPCI SBCs) I am doing may warrant
a fifth. But, given the available wiring in the house, while some of
these devices could easily be connected to a fourth subnet on my
switch, other devices would have to share a connection with my
workstations on my secure subnet. Fortunately, these devices can
handle 802.11q VLANs, and after months of not having the time to look
at this problem due to real-life time drains, I am starting to
consider how I will transtion my network.

Now, in playing around with NetBSD, I found that I could not do the
following:

ifconfig vlan4 create
ifconfig vlan4 vlan 4 vlanif fxp1
ifconfig vlan4 inet 192.168.4.1 netmask 0xffffff00
ifconfig vlan4 vlan 4 vlanif fxp2

The attempt at doing a second vlanif for the vlan fails because unlike
every other switch/router I have dealt with, NetBSD apparently can
only bind a vlan to a single physical interface. And so, I am now
looking at the proper direction to proceed. My thoughts (which I
would like feedback on) are as follows:

1) I could possibly move to having just a single physical interface on
alpha, and switch my ipf/ipnat rules to use vlan pseudo-interfaces
instead. This would seem to be in keeping with what seems to be
the common practice of others on this and other NetBSD lists, but
for me, presents the problem in that once I move to using a Sun QFE
(hme) card, I will have the potential for two physical subnets to
be talking at rates which would saturate a single physical
interface, and starvation of another subnet's connectivity to the
Internet would occur.

BTW... in another site I manange which may soon have to go through
a similar transition, this definitely occurs, as we broadcast data
from a radio telescope receiver to computers on one physical vlan
at the full 100Mbps data rate. On occasion, telecommands will go
through the a firewall just like mine onto this network to the
devices broadcasting the data. Thankfully packets in both
directions are one way broadcasts instead of the normal 2-way like
in TCP.

2) I could configure additional vlan devices, so that say vlan10
vlan11 and vlan12 could handle one vlan on fxp0, fxp1 and fxp2,
vlan20, vlan21 and vlan22 would handle a second vlan on those
interfaces, and so on. It would then mean I would probably have to
do some additional configuration in my ipnat/ipfilter rules.

3) There is some other way which takes care of the whole forest, but
which I am not seeing because of my nose being in the bark of a
single tree.

Anyone have any suggestions/comments? Also, has anyone had experience
using dhcp to serve up the IP addresses on 802.11q VLAN subnets? My
dhcp server is on the private subnet, and alpha is running dhrelay to
pass along requests from the DMZ.

Oh... and alpha will likely be upgraded from 1.6 to 3.1 or current
when I do the transition (since I do not believe that the QFE was
supported way back then). Given it does not accept any connections
externally and only certain protocols from key hosts on the private
subnet, it was not broke, so I did not fix it. ;)

Thanks!

- Doug

[Manuel Bouyer]

Post by Douglas Wade Needham
[...]
Now, in playing around with NetBSD, I found that I could not do the
ifconfig vlan4 create
ifconfig vlan4 vlan 4 vlanif fxp1
ifconfig vlan4 inet 192.168.4.1 netmask 0xffffff00
ifconfig vlan4 vlan 4 vlanif fxp2
The attempt at doing a second vlanif for the vlan fails because unlike
every other switch/router I have dealt with, NetBSD apparently can
only bind a vlan to a single physical interface. And so, I am now

The same vlan interface, yes. Why do you want to have vlan4 on 2 physical
interfaces ? briding ? link aggregation to use more bandwith ?

[Douglas Wade Needham]

At least one of the VLANs (vlan 4 in my example) will have to exist on
multiple physical subnets/segments, given where I need to put the
nodes for that one VLAN. And given some of the traffic I push around
here, putting everything for my firewall on a single interface is not
really a good idea. But it may be that my switch could have one port
configured in a way that packets coming in on that port on that one
vlan will get where they should go without passing through
alpha.... but the docs are a bit crappy in that area.

Well, off to bed for me.

- Doug

Post by Manuel Bouyer

Post by Douglas Wade Needham
[...]
Now, in playing around with NetBSD, I found that I could not do the
ifconfig vlan4 create
ifconfig vlan4 vlan 4 vlanif fxp1
ifconfig vlan4 inet 192.168.4.1 netmask 0xffffff00
ifconfig vlan4 vlan 4 vlanif fxp2
The attempt at doing a second vlanif for the vlan fails because unlike
every other switch/router I have dealt with, NetBSD apparently can
only bind a vlan to a single physical interface. And so, I am now

The same vlan interface, yes. Why do you want to have vlan4 on 2 physical
interfaces ? briding ? link aggregation to use more bandwith ?
--
NetBSD: 26 ans d'experience feront toujours la difference
--

[David Young]

Post by Douglas Wade Needham
At least one of the VLANs (vlan 4 in my example) will have to exist on
multiple physical subnets/segments, given where I need to put the
nodes for that one VLAN. And given some of the traffic I push around
here, putting everything for my firewall on a single interface is not
really a good idea. But it may be that my switch could have one port
configured in a way that packets coming in on that port on that one
vlan will get where they should go without passing through
alpha.... but the docs are a bit crappy in that area.

It sounds to me like you need to create two VLAN pseudo-interfaces,
one for fxp1, and a second for fxp2, and then connect the two
pseudo-interfaces with a bridge. That is,

ifconfig vlan4 create
ifconfig vlan5 create
ifconfig bridge0 create

ifconfig vlan4 vlan 4 vlanif fxp1
ifconfig vlan5 vlan 4 vlanif fxp2

ifconfig vlan4 inet 192.168.4.1 netmask 0xffffff00

brconfig bridge0 add vlan4 add vlan5
ifconfig bridge0 up

Dave

[Manuel Bouyer]

Post by Douglas Wade Needham
At least one of the VLANs (vlan 4 in my example) will have to exist on
multiple physical subnets/segments, given where I need to put the
nodes for that one VLAN. And given some of the traffic I push around
here, putting everything for my firewall on a single interface is not
really a good idea. But it may be that my switch could have one port
configured in a way that packets coming in on that port on that one
vlan will get where they should go without passing through
alpha.... but the docs are a bit crappy in that area.

Unless your switch has some kind of partitioning which would make
it behaves as 2 physically independant switches, a vlan with the same
vlan ID will be the same ethernet broadcast domain on all ports.
If your switch is really partitionned what you need is then a bridge
to make these 2 ethernet broadcast domain a single one:

ifconfig vlan4 vlan 4 vlanif fxp0
ifconfig vlan14 vlan 4 vlanif fxp1
brconfig add vlan4 add vlan14 up

you configure the IP address for vlan 4 on either vlan4 or vlan14 (but
not both !).

[Charlie Allom]

Post by Douglas Wade Needham
ifconfig vlan4 create
ifconfig vlan5 create
ifconfig bridge0 create
ifconfig vlan4 vlan 4 vlanif fxp1
ifconfig vlan5 vlan 4 vlanif fxp2
ifconfig vlan4 inet 192.168.4.1 netmask 0xffffff00

just to keep the vlan thread going,but skew it in my direction, now
that i see some experts :)

how should I configure carp interfaces that belong on vlans? ie. where
should i add the inet address?

% ifconfig wm3 up
% ifconfig wm3 x.x.x.x netmask 0xffffffe0
% ifconfig carp2 create
% ifconfig carp2 vhid 201 carpdev wm3 x.x.x.y netmask 0xffffffe0
% ifconfig vlan200 create
% ifconfig vlan200 vlan 200 vlanif carp200
% ifconfig vlan200 inet x.x.x.z netmask 0xffffffe0

so far - wasting x.x.x.x on wm3, x.x.x.y on carp2, and y.y.y.* on the
n+1 failover carp hosts...

am i wrong? Can I leave the carpN IPs out?

[Manuel Bouyer]

Post by Charlie Allom

Post by Douglas Wade Needham
ifconfig vlan4 create
ifconfig vlan5 create
ifconfig bridge0 create
ifconfig vlan4 vlan 4 vlanif fxp1
ifconfig vlan5 vlan 4 vlanif fxp2
ifconfig vlan4 inet 192.168.4.1 netmask 0xffffff00

just to keep the vlan thread going,but skew it in my direction, now
that i see some experts :)
how should I configure carp interfaces that belong on vlans? ie. where
should i add the inet address?
% ifconfig wm3 up
% ifconfig wm3 x.x.x.x netmask 0xffffffe0
% ifconfig carp2 create
% ifconfig carp2 vhid 201 carpdev wm3 x.x.x.y netmask 0xffffffe0
% ifconfig vlan200 create
% ifconfig vlan200 vlan 200 vlanif carp200
% ifconfig vlan200 inet x.x.x.z netmask 0xffffffe0

I'm not sure you can put vlans on carps - carps are not normal ethernet
interfaces.
But you can have carp on vlan; this is what I use and it works

Post by Charlie Allom
so far - wasting x.x.x.x on wm3, x.x.x.y on carp2, and y.y.y.* on the
n+1 failover carp hosts...
am i wrong? Can I leave the carpN IPs out?

No, because it's the IP which is used in real. But you can use a private IP
for the underlying ethernet interfaces and vlans (this is also what I do
to not waste public IP addresses)

[Charlie Allom]

Post by Manuel Bouyer
I'm not sure you can put vlans on carps - carps are not normal ethernet
interfaces.
But you can have carp on vlan; this is what I use and it works

ok, cheers. i'll go ahead and set this up.

Post by Manuel Bouyer

Post by Charlie Allom
so far - wasting x.x.x.x on wm3, x.x.x.y on carp2, and y.y.y.* on the
n+1 failover carp hosts...
am i wrong? Can I leave the carpN IPs out?

No, because it's the IP which is used in real. But you can use a private IP
for the underlying ethernet interfaces and vlans (this is also what I do
to not waste public IP addresses)

oh - great idea! Thanks Manuel.