Discussion:
kernel networking stack in userspace
(too old to reply)
Antti Kantee
2008-10-18 14:20:23 UTC
Permalink
Hi,

Recently I added support for the NFS client as a rump file system. Since
NFS uses in-kernel networking, as a side effect kernel TCP/IP networking
is now provided in a userspace library. Apart from being able to use
multiple TCP/IP stacks within a single system (a.k.a. virtualization,
one being the kernel networking stack and the rest running in userspace),
it does not have any real applications I can immediately see. However,
since it is useful for playing, development, writing regression tests, and
generally learning about BSD networking, I'm writing this short tutorial.

There are two choices for operation mode:
1) virtual networking using host sockets (similar to e.g. MMlite)
(librumpnet, librumpnet_sockin)
2) full TCP/IP networking stack
(librumpnet, librumpnet_net, librumpnet_netinet, librumpnet_virtif)

Switching between these two is currently done when the application
is linked. I hope to eventually make this run-time configurable.

Since the caller accesses the networking stack through the sockets
layer, it cannot tell the difference between the two. It is therefore
up to the user/operator to decide which mode is desired.

Option 1 defines a networking domain with the SOCK_DGRAM and
SOCK_STREAM protocols and interfaces with the host kernel via
the kernel's socket syscall interface. This has three implications:

* only the socket code is run in userspace
* all communication happens using the host IP address
* no system administration priviledges are required

On the contrary, option 2 runs the entire kernel networking stack in
userspace. This means that it must have raw ethernet access and
accomplishes this through the use of tap(4) (similar to e.g. qemu).
Contrasting the other option, this brings:

* full networking stack in userspace
* the interface and routing table must be configured before use
* /dev/tap and interface configuration priviledges are required

To configure the interface from your code, you must emulate what
ifconfig(8) and route(8) do on a normal system. There is an
example in src/sys/rump/net/rumptest/rumptest_net.c on how to do
this. I'll probably move that stuff to a library lateron.

Currently, tap must be configured manually as tap0. To do this, execute
(with appropriate priviledges):
golem> ifconfig tap0 create
golem> ifconfig tap0 up

Optionally, if wanting to bridge to a real network, execute:
golem> ifconfig bridge0 create
golem> brconfig bridge0 add tap0 add yourrealif0
golem> brconfig bridge0 up
(yes, the first command really is ifconfig, not brconfig)

Apart from that, both options require threading, which can be enabled by
setting the RUMP_THREADS environment variable to 1. Then it's just a
matter of running the application. If all goes well, running
rumptest_net should dump out the html on the main page www.NetBSD.org.

Enjoy (if you're into this sort of stuff ;)
- antti

pie-in-the-sky: barring interfacing with tap(4), the userspace kernel
networking stack is likely to be highly portable. So if you don't like
networking on $OS, by all means use the NetBSD TCP/IP stack linked
against your application ;)

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Paul Goyette
2008-10-18 16:08:38 UTC
Permalink
On Sat, 18 Oct 2008, Antti Kantee wrote:

8< snip >8
Post by Antti Kantee
On the contrary, option 2 runs the entire kernel networking stack in
userspace. This means that it must have raw ethernet access and
accomplishes this through the use of tap(4) (similar to e.g. qemu).
* full networking stack in userspace
* the interface and routing table must be configured before use
* /dev/tap and interface configuration priviledges are required
To configure the interface from your code, you must emulate what
ifconfig(8) and route(8) do on a normal system. There is an
example in src/sys/rump/net/rumptest/rumptest_net.c on how to do
this. I'll probably move that stuff to a library lateron.
Currently, tap must be configured manually as tap0. To do this, execute
golem> ifconfig tap0 create
golem> ifconfig tap0 up
Can we assume that if you want yet another virtual networking domain,
you create a tap1, tap2, etc.?
Post by Antti Kantee
golem> ifconfig bridge0 create
golem> brconfig bridge0 add tap0 add yourrealif0
golem> brconfig bridge0 up
(yes, the first command really is ifconfig, not brconfig)
What about _routing_ between domains? :)
Post by Antti Kantee
Apart from that, both options require threading, which can be enabled by
setting the RUMP_THREADS environment variable to 1. Then it's just a
matter of running the application. If all goes well, running
rumptest_net should dump out the html on the main page www.NetBSD.org.
Enjoy (if you're into this sort of stuff ;)
- antti
pie-in-the-sky: barring interfacing with tap(4), the userspace kernel
networking stack is likely to be highly portable. So if you don't like
networking on $OS, by all means use the NetBSD TCP/IP stack linked
against your application ;)
----------------------------------------------------------------------
| Paul Goyette | PGP DSS Key fingerprint: | E-mail addresses: |
| Customer Service | FA29 0E3B 35AF E8AE 6651 | ***@whooppee.com |
| Network Engineer | 0786 F758 55DE 53BA 7731 | ***@juniper.net |
----------------------------------------------------------------------

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Antti Kantee
2008-10-19 10:25:58 UTC
Permalink
Post by Paul Goyette
Post by Antti Kantee
Currently, tap must be configured manually as tap0. To do this, execute
golem> ifconfig tap0 create
golem> ifconfig tap0 up
Can we assume that if you want yet another virtual networking domain,
you create a tap1, tap2, etc.?
Yes, but that requires non-hardcoding /dev/tap0 in if_virt.c. I'm
thinking that maybe calling code should open /dev/tap and just pass
in the ifname + file descriptor to rump_virtif_create().

Anyway, if you want to play with it, should be easy to workaround
by just using a counter and snprintf or somelike think that.
Post by Paul Goyette
Post by Antti Kantee
golem> ifconfig bridge0 create
golem> brconfig bridge0 add tap0 add yourrealif0
golem> brconfig bridge0 up
(yes, the first command really is ifconfig, not brconfig)
What about _routing_ between domains? :)
All processes using rumpnet have their own routing tables. Routing
between these should not be a problem. E.g., if you have two processes
using two tap interfaces, and if they are in the same subnet, you can make
them talk to each other simply by briding the tap interfaces. If you
want actual IP routing to happen, you need to configure routing tables.
I have not yet tested with options GATEWAY, but I don't see why that
would be a problem. So, in theory, it should be possible to build a
quite complex network on one host using a set of processes, virtual
interfaces and bridges.

Do you have an actual use case of what you want to do? That would
help with trying to figure out what configuration interfaces to export
to users.

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Antti Kantee
2009-02-08 14:14:47 UTC
Permalink
Post by Antti Kantee
Post by Paul Goyette
Post by Antti Kantee
golem> ifconfig bridge0 create
golem> brconfig bridge0 add tap0 add yourrealif0
golem> brconfig bridge0 up
(yes, the first command really is ifconfig, not brconfig)
What about _routing_ between domains? :)
All processes using rumpnet have their own routing tables. Routing
between these should not be a problem. E.g., if you have two processes
using two tap interfaces, and if they are in the same subnet, you can make
them talk to each other simply by briding the tap interfaces. If you
want actual IP routing to happen, you need to configure routing tables.
I have not yet tested with options GATEWAY, but I don't see why that
would be a problem. So, in theory, it should be possible to build a
quite complex network on one host using a set of processes, virtual
interfaces and bridges.
I did a minitest for that recently. I started two endpoint processes
and a third which acted as a router in between. The router was pretty
simple to write:

rump_init();
configure_interfaces();
pause();

;)

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Loading...