bridge(4) - feedback local packets into ether_input()

[Roy Marples]

Hi List!

So my goal is to run a DHCP server and client on two virtual interfaces
and get them talking to each other. After the initial idea of
vether/pair was shot down I looked into what bridge(4) could do after
much prompting by our beloved Taylor.

Here is what I am trying to achieve WITHOUT BPF* (it works with BPF):
DHCP Server --- > tap0 --- > bridge0 <---- tap1 <----- DHCP client

Because tap(4) will drop packets if there is nothing using it as a TAP
device (in this scenario we're just treating it as an endpoint) we have
to solve the problem in bridge(4).

Because DHCP is essentially a broadcast operation, I copied the logic
from bridge_broadcast() into bridge_output() (used for originating local
packets) where we send the packet to ether_input() if it's broadcast or
multicast. Because bridge_output has coding for if src_if == dst_if I
reversed the logic on sending it to ether_input (src_if != dst_if) so
that the receiving interface doesn't double up.

The attached patch allows this to work fine.
Comments are welcome, especially if you think it breaks something.

Roy

* OK the DHCP client still requires BPF but that can be fixed once PR
kern/48280 is.

[Ryota Ozaki]

Post by Roy Marples
Hi List!
So my goal is to run a DHCP server and client on two virtual interfaces
and get them talking to each other. After the initial idea of
vether/pair was shot down I looked into what bridge(4) could do after
much prompting by our beloved Taylor.
DHCP Server --- > tap0 --- > bridge0 <---- tap1 <----- DHCP client
Because tap(4) will drop packets if there is nothing using it as a TAP
device (in this scenario we're just treating it as an endpoint) we have
to solve the problem in bridge(4).
Because DHCP is essentially a broadcast operation, I copied the logic
from bridge_broadcast() into bridge_output() (used for originating local
packets) where we send the packet to ether_input() if it's broadcast or
multicast. Because bridge_output has coding for if src_if == dst_if I
reversed the logic on sending it to ether_input (src_if != dst_if) so
that the receiving interface doesn't double up.
The attached patch allows this to work fine.
Comments are welcome, especially if you think it breaks something.

It at least doesn't break ATF tests under tests/net :)

I don't notice any issues by the change for now. So no objection
from me.

BTW with the change can we write ATF tests of dhcpcd without interfering
the host (or with minimum affects)?

ozaki-r

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Roy Marples]

Post by Ryota Ozaki
BTW with the change can we write ATF tests of dhcpcd without interfering
the host (or with minimum affects)?

You kind of could anyway with ISC dhcpd because it listens and sends via
BPF which worked for this anyway. This patch feeds it back into our
stack so any program can open a PF_INET/SOCK_DGRAM and listen to it
normally, like dnsmasq does.

You would need to setup an ip range on tap0 to serve DHCP from.
Then dhcpcd can run happily on tap1 and test everything with the
following notes

* dhcpcd should be instructed not to install any default routes (-G)
* dhcpcd should run a custom dhcpcd-run-hooks(8) script

There are commandline options for both of these, and also in dhcpcd.conf
You can then write ATF tests to observe addreses and routes installed
and for the variables set for dhcpcd-run-hooks.

Roy

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Ryota Ozaki]

Post by Roy Marples

Post by Ryota Ozaki
BTW with the change can we write ATF tests of dhcpcd without interfering
the host (or with minimum affects)?

You kind of could anyway with ISC dhcpd because it listens and sends via
BPF which worked for this anyway. This patch feeds it back into our
stack so any program can open a PF_INET/SOCK_DGRAM and listen to it
normally, like dnsmasq does.
You would need to setup an ip range on tap0 to serve DHCP from.
Then dhcpcd can run happily on tap1 and test everything with the
following notes
* dhcpcd should be instructed not to install any default routes (-G)
* dhcpcd should run a custom dhcpcd-run-hooks(8) script
There are commandline options for both of these, and also in dhcpcd.conf
You can then write ATF tests to observe addreses and routes installed
and for the variables set for dhcpcd-run-hooks.

Thanks! I'll try to write some tests.

ozaki-r

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Roy Marples]

Post by Roy Marples
Because DHCP is essentially a broadcast operation, I copied the logic
from bridge_broadcast() into bridge_output() (used for originating local
packets) where we send the packet to ether_input() if it's broadcast or
multicast. Because bridge_output has coding for if src_if == dst_if I
reversed the logic on sending it to ether_input (src_if != dst_if) so
that the receiving interface doesn't double up.

Nick suggested just to use bridge_broadcast() and came up with this
patch: http://www.netbsd.org/~skrll/bridge.diff

(note doesn't compile, src_if needs replacing with ifp)

However, this makes the kernel panic with locking against myself on
softnet_lock which we cannot figure out.

Is this an easy fix or should we forget it and go back to the initial patch?

Roy

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Nick Hudson]

Post by Roy Marples

Post by Roy Marples
Because DHCP is essentially a broadcast operation, I copied the logic
from bridge_broadcast() into bridge_output() (used for originating local
packets) where we send the packet to ether_input() if it's broadcast or
multicast. Because bridge_output has coding for if src_if == dst_if I
reversed the logic on sending it to ether_input (src_if != dst_if) so
that the receiving interface doesn't double up.

Nick suggested just to use bridge_broadcast() and came up with this
patch: http://www.netbsd.org/~skrll/bridge.diff
(note doesn't compile, src_if needs replacing with ifp)

It compiles now :)

Post by Roy Marples
However, this makes the kernel panic with locking against myself on
softnet_lock which we cannot figure out.

The backtrace for reference is

Apr 28 13:41:09 uberlaptop2 /netbsd: panic: LOCKDEBUG: Mutex error:
lockdebug_wantlock: locking against myself
Apr 28 13:41:09 uberlaptop2 /netbsd: cpu0: Begin traceback...
Apr 28 13:41:09 uberlaptop2 /netbsd: vpanic() at netbsd:vpanic+0x1be
Apr 28 13:41:09 uberlaptop2 /netbsd: panic() at netbsd:panic+0x43
Apr 28 13:41:09 uberlaptop2 /netbsd: lockdebug_abort1() at
netbsd:lockdebug_abort1+0x1f8
Apr 28 13:41:09 uberlaptop2 /netbsd: mutex_enter() at
netbsd:mutex_enter+0x75
Apr 28 13:41:09 uberlaptop2 /netbsd: bridge_broadcast() at
netbsd:bridge_broadcast+0x11d
Apr 28 13:41:09 uberlaptop2 /netbsd: bridge_output() at
netbsd:bridge_output+0x1db
Apr 28 13:41:09 uberlaptop2 /netbsd: ether_output() at
netbsd:ether_output+0x3e1
Apr 28 13:41:09 uberlaptop2 /netbsd: nd6_output() at netbsd:nd6_output+0x407
Apr 28 13:41:09 uberlaptop2 /netbsd: ip6_output() at
netbsd:ip6_output+0x1b6a
Apr 28 13:41:09 uberlaptop2 /netbsd: mld_sendpkt() at
netbsd:mld_sendpkt+0x2cf
Apr 28 13:41:09 uberlaptop2 /netbsd: in6_delmulti() at
netbsd:in6_delmulti+0xc9
Apr 28 13:41:09 uberlaptop2 /netbsd: in6_leavegroup() at
netbsd:in6_leavegroup+0x16
Apr 28 13:41:09 uberlaptop2 /netbsd: ip6_freemoptions() at
netbsd:ip6_freemoptions+0x2b
Apr 28 13:41:09 uberlaptop2 /netbsd: in6_pcbdetach() at
netbsd:in6_pcbdetach+0x104
Apr 28 13:41:09 uberlaptop2 /netbsd: udp6_detach_wrapper() at
netbsd:udp6_detach_wrapper+0x3d
Apr 28 13:41:09 uberlaptop2 /netbsd: soclose() at netbsd:soclose+0x1cd
Apr 28 13:41:09 uberlaptop2 /netbsd: soo_close() at netbsd:soo_close+0x19
Apr 28 13:41:09 uberlaptop2 /netbsd: closef() at netbsd:closef+0x99
Apr 28 13:41:09 uberlaptop2 /netbsd: fd_close() at netbsd:fd_close+0xf6
Apr 28 13:41:09 uberlaptop2 /netbsd: sys_close() at netbsd:sys_close+0x28
Apr 28 13:41:09 uberlaptop2 /netbsd: syscall() at netbsd:syscall+0xe7
Apr 28 13:41:09 uberlaptop2 /netbsd: --- syscall (number 6) ---


softnet_lock is taken by solock in soclose...

Nick

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Ryota Ozaki]

(I'm reposting the below my reply because it was sent accidentally
with HTML and filtered at the server.)

Post by Roy Marples

Post by Roy Marples
Because DHCP is essentially a broadcast operation, I copied the logic
from bridge_broadcast() into bridge_output() (used for originating local
packets) where we send the packet to ether_input() if it's broadcast or
multicast. Because bridge_output has coding for if src_if == dst_if I
reversed the logic on sending it to ether_input (src_if != dst_if) so
that the receiving interface doesn't double up.

Nick suggested just to use bridge_broadcast() and came up with this
patch: http://www.netbsd.org/~skrll/bridge.diff
(note doesn't compile, src_if needs replacing with ifp)
However, this makes the kernel panic with locking against myself on
softnet_lock which we cannot figure out.
Is this an easy fix or should we forget it and go back to the initial patch?

Not easy because softnet_lock may or may not be held on sending path as
nick's back trace indicates.
If the first patch works for you, we should use it for now.
(softnet_lock should be fixed in the future though.)
ozaki-r

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de