IP_PKTINFO, sendmsg(2) and routing.

[Roy Marples]

Hi List

Whilst traking down a dhcpcd issue for Jared it occured to me that my
use of IP_PKTINFO might not be ideal for NetBSD. Here's the scenario:

interface a: 192.168.0.101/24
interface b: 192.168.0.102/24

The route to 192.168.0.255 belongs on interface a.

If I send a UDP packet to 192.168.0.1 it will by default leave by
interface a.
My question is this - should IP_PKTINFO allow the same message to leave
by interface b - ie effectively bypassing the routing table?

Roy

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[David Young]

Post by Roy Marples
Hi List
Whilst traking down a dhcpcd issue for Jared it occured to me that
my use of IP_PKTINFO might not be ideal for NetBSD. Here's the
interface a: 192.168.0.101/24
interface b: 192.168.0.102/24
The route to 192.168.0.255 belongs on interface a.
If I send a UDP packet to 192.168.0.1 it will by default leave by
interface a.
My question is this - should IP_PKTINFO allow the same message to
leave by interface b - ie effectively bypassing the routing table?

I think so, but I think that if you want to send a packet that both has
a particular source address and transmits on a particular interface,
IP_PKTINFO may not be adequate?

Many years ago I wanted to transmit (multicast) packets on multiple
interfaces. Each interface had a unique address 169.254.x.y/16
assigned. The source address of every packet had to match the
169.254.x.y address of the interface it was transmitted on. The
169.254.x.y address happened to be secondary on each interface. For one
reason or another, I used a different socket for each interface.

Dave

[Roy Marples]

Post by David Young

Post by Roy Marples
Hi List
Whilst traking down a dhcpcd issue for Jared it occured to me that
my use of IP_PKTINFO might not be ideal for NetBSD. Here's the
interface a: 192.168.0.101/24
interface b: 192.168.0.102/24
The route to 192.168.0.255 belongs on interface a.
If I send a UDP packet to 192.168.0.1 it will by default leave by
interface a.
My question is this - should IP_PKTINFO allow the same message to
leave by interface b - ie effectively bypassing the routing table?

I think so, but I think that if you want to send a packet that both has
a particular source address and transmits on a particular interface,
IP_PKTINFO may not be adequate?

Well, our implementation of it is not adequate as it stands, especially
if you share the same address across interfaces.

I suppose it's also a security question because a non privileged process
can set IP_PKTINFO but only root can create routes.
It's works on Linux (my primary use case) because each interface address
has it's own prefix route and the lowest metric wins. Root has already
created the routes, so it's an expression of choice.

Roy

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Roy Marples]

Post by David Young

Post by Roy Marples

Post by Dennis Ferguson
It should send the packet out the IP_PKTINFO interface if packet is sent
with MSG_DONTROUTE/SO_DONTROUTE. If this isn’t working it should probably
be made to, routing protocols in general depend on being able to do that.

I don't see how the two are related.
Routing can still be obeyed to an extent.
Consider the following
Interface A has 192.168.0.1/24
Interface B has 192.168.0.2/24
The subnet 192.168/24 belongs to interface A
There is a default route to 192.168.0.10, again on interface A
If I sent a packet to 1.2.3.4 and IP_PKTINFO says use interface B I
would expect the packet to be sent to the default router of
192.168.0.10 but exit via interface B and not interface A.

If you don't use _DONTROUTE, then I would *hope* for the transmission to
fail with an error, probably EHOSTUNREACH, if 1.2.3.4 has no route on
interface B. However, today I sort of *expect* for the transmission to
go out interface A, since the nexthop and its link-layer information are
on interface A.

So what benefit does IP_PKTINFO bring us for sending exactly?
Because you've just ruled out the use case I tried to illustrate here:
https://mail-index.netbsd.org/tech-net/2019/02/11/msg007254.html

Roy

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[David Young]

Post by Roy Marples

Post by David Young

Post by Roy Marples

Post by Dennis Ferguson
It should send the packet out the IP_PKTINFO interface if packet is sent
with MSG_DONTROUTE/SO_DONTROUTE. If this isn’t working it should probably
be made to, routing protocols in general depend on being able to do that.

I don't see how the two are related.
Routing can still be obeyed to an extent.
Consider the following
Interface A has 192.168.0.1/24
Interface B has 192.168.0.2/24
The subnet 192.168/24 belongs to interface A
There is a default route to 192.168.0.10, again on interface A
If I sent a packet to 1.2.3.4 and IP_PKTINFO says use interface B I
would expect the packet to be sent to the default router of
192.168.0.10 but exit via interface B and not interface A.

If you don't use _DONTROUTE, then I would *hope* for the transmission to
fail with an error, probably EHOSTUNREACH, if 1.2.3.4 has no route on
interface B. However, today I sort of *expect* for the transmission to
go out interface A, since the nexthop and its link-layer information are
on interface A.

So what benefit does IP_PKTINFO bring us for sending exactly?
https://mail-index.netbsd.org/tech-net/2019/02/11/msg007254.html

If you want to send a packet to the directly-connected host
192.168.0.10, and subnets containing 192.168.0.10 are configured on
multiple interfaces, then IP_PKTINFO lets you choose a different
interface than the system default.

I don't know why you would ever use IP_PKTINFO for a destination
that is *not* on a directly-connected subnet (1.2.3.4 in your example).
Is that something you really need to do?

If 192.168.0.0/24 is assigned to interfaces A and B but not to interface
C, the nexthop for 1.2.3.4 is 192.168.0.10, and a program uses
IP_PKTINFO to try to send to 1.2.3.4 over interface C, then what do you
think should happen? Consider a couple of cases: 1) interface C is a
point-to-point link, 2) it is a broadcast link.

Dave

[Dennis Ferguson]

Post by David Young

Post by Roy Marples
Hi List
Whilst traking down a dhcpcd issue for Jared it occured to me that
my use of IP_PKTINFO might not be ideal for NetBSD. Here's the
interface a: 192.168.0.101/24
interface b: 192.168.0.102/24
The route to 192.168.0.255 belongs on interface a.
If I send a UDP packet to 192.168.0.1 it will by default leave by
interface a.
My question is this - should IP_PKTINFO allow the same message to
leave by interface b - ie effectively bypassing the routing table?

I think so, but I think that if you want to send a packet that both has
a particular source address and transmits on a particular interface,
IP_PKTINFO may not be adequate?

Well, our implementation of it is not adequate as it stands, especially if you share the same address across interfaces.
I suppose it's also a security question because a non privileged process can set IP_PKTINFO but only root can create routes.
It's works on Linux (my primary use case) because each interface address has it's own prefix route and the lowest metric wins. Root has already created the routes, so it's an expression of choice.

It should send the packet out the IP_PKTINFO interface if packet is sent
with MSG_DONTROUTE/SO_DONTROUTE. If this isn’t working it should probably
be made to, routing protocols in general depend on being able to do that.

Dennis Ferguson


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Tom Ivar Helbekkmo]

Post by Dennis Ferguson
It should send the packet out the IP_PKTINFO interface if packet is sent
with MSG_DONTROUTE/SO_DONTROUTE. If this isn’t working it should probably
be made to, routing protocols in general depend on being able to do that.

I believe I was the last person to make significant changes to this
code, about a year ago. The plan for my changes back then is here:
https://mail-index.netbsd.org/tech-net/2017/12/28/msg006579.html
...and the completed result was presented here:
https://mail-index.netbsd.org/tech-net/2017/12/31/msg006591.html

The bit that does the actual work on an outgoing packet is the function
ip_pktinfo_prepare() in sys/netinet/ip_output.c. The way it is now
works for all the software I've seen that uses it, but I've never seen
those *_DONTROUTE options used. Note, though, that we set the
IP_ROUTETOIFINDEX flag if an interface was specified, and I believe this
ends up doing what you say it should.

If you can point me to some clear information on how IP_PKTINFO should
behave in various configurations of multi-homed machines, to satisfy
routing protocol implementations, I'd love to play with it, and get our
implementation to do the right thing.

-tih

[Roy Marples]

Post by Tom Ivar Helbekkmo

Post by Dennis Ferguson
It should send the packet out the IP_PKTINFO interface if packet is sent
with MSG_DONTROUTE/SO_DONTROUTE. If this isn’t working it should probably
be made to, routing protocols in general depend on being able to do that.

I believe I was the last person to make significant changes to this
https://mail-index.netbsd.org/tech-net/2017/12/28/msg006579.html
https://mail-index.netbsd.org/tech-net/2017/12/31/msg006591.html
The bit that does the actual work on an outgoing packet is the function
ip_pktinfo_prepare() in sys/netinet/ip_output.c. The way it is now
works for all the software I've seen that uses it, but I've never seen
those *_DONTROUTE options used. Note, though, that we set the
IP_ROUTETOIFINDEX flag if an interface was specified, and I believe this
ends up doing what you say it should.
If you can point me to some clear information on how IP_PKTINFO should
behave in various configurations of multi-homed machines, to satisfy
routing protocol implementations, I'd love to play with it, and get our
implementation to do the right thing.

https://nxr.netbsd.org/xref/src/sys/netinet/ip_output.c#1167
Well the most obvious thing that I would say is "wrong" is that it boils
down to an address. If I don't give an address, the implementation
shouldn't use one. I just want it to go out of the specified interface.

As a side effect of this implementation, it won't work if the same
address is present on more than one interface which is not only
possible, but last I checked, supported by our stack.

Where it starts to get interesting is how we handle ARP, which is also
tied into our routing.
In theory, the interface we want the packet to go out on should also be
used for ARP resolution, but there is no guarantee that the prefix route
for it is on the same interface.
It might well be that we need route metrics to get it to work well.

But, in a nutshell, the most basic multi-homed implementation is the
DHCP client. IP_PKTINFO allows us NOT to need BPF. We should be able to
send from the unspecified address to the broadcast address on a specific
interface using IP_PKTINFO for the DHCP_DISCOVER message.

Now, if we could send and receive ARP without BPF then dhcpcd woudln't
need BPF at all, but that's starting to get out of scope of this
conversation, so I'll shutup now.

Roy

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Tom Ivar Helbekkmo]

Post by Roy Marples
https://nxr.netbsd.org/xref/src/sys/netinet/ip_output.c#1167

Ah, yes, that bit is somewhat simplistic, too. Back when I worked on
this code, the "what if an interface is specified, but no source
address?" question got rather short shrift. It'll be good to get back
to it, and have a more complete implementation.

Post by Roy Marples
But, in a nutshell, the most basic multi-homed implementation is the
DHCP client. IP_PKTINFO allows us NOT to need BPF. We should be able
to send from the unspecified address to the broadcast address on a
specific interface using IP_PKTINFO for the DHCP_DISCOVER message.

I'll try to implement this in a way that as far as possible maintains
compatibility with our existing implementation, Linux, and Solaris, and
orthogonality with the standard for IPv6, as described in RFC3542. :)

I plan to have running code some time during the coming weekend; I'm
going to the cabin for a couple of days' relaxation, so this is perfect.

-tih

[Tom Ivar Helbekkmo]

Post by Tom Ivar Helbekkmo
I plan to have running code some time during the coming weekend; I'm
going to the cabin for a couple of days' relaxation, so this is perfect.

Well, that didn't work out the way I expected - the first thing that
happened after getting up there was that my home system crashed and
didn't reboot, so I was without my development environment, my email,
and my IRC client. NetBSD-current with local modifications... :)

Anyway, I no longer think there's anything to really change in this
code. You're just expecting a bit more from it than it offers.

Here's how it works:

Source address selection: if you supply a source address, it is used,
but it must be bindable, i.e. it must be either a unicast address
currently configured on some interface on the system, or a multicast
address. If you do not supply a source address (that is, you supply
INADDR_ANY in the ipi_addr field), then the ipi_ifindex field is
checked, and the primary address configured on that interface is used.

Interface selection: if you supply an interface index, it will be used
for selecting the outgoing interface, but only if the packet being sent
doesn't require routing. Specifically, you can send to multicast
addresses, or to INADDR_BROADCAST, and direct those packets to the
interface of your choice. Unicast destinations will work if you happen
to point them at the interface they would've been routed out anyway, but
will otherwise fail.

Post by Tom Ivar Helbekkmo
But, in a nutshell, the most basic multi-homed implementation is the
DHCP client. IP_PKTINFO allows us NOT to need BPF. We should be
able to send from the unspecified address to the broadcast address
on a specific interface using IP_PKTINFO for the DHCP_DISCOVER
message.

So no, you can't do that. If you want to send from INADDR_ANY, you have
to use BPF. IP_PKTINFO will only work for sending from bindable
addresses on the sending host, as described above. Raw sockets let you
do more fiddling, but even they won't allow you to send from INADDR_ANY.

Post by Tom Ivar Helbekkmo
So what benefit does IP_PKTINFO bring us for sending exactly?

Its primary purpose is making it possible for connectionless services on
multi-homed systems to correctly respond to incoming packets from the
same address the incoming packet was sent to.

Additionally, IP_PKTINFO will let you transmit broadcast and multicast
packets over specifically chosen interfaces.

In FreeBSD, they solved the multi-homed system problem by adding the
IP_SENDSRCADDR option to complement the existing IP_RECVDSTADDR one,
whereas the Linux folks decided to create IP_PKTINFO modeled on the
standardized IPV6_PKTINFO, and botched it by gratuitously making the API
slightly different. We now have both of these mechanisms, and are
source code compatible with both FreeBSD and Linux - while we also
maintain better orthogonality with IPv6.

-tih

[Roy Marples]

Post by Dennis Ferguson
It should send the packet out the IP_PKTINFO interface if packet is sent
with MSG_DONTROUTE/SO_DONTROUTE. If this isn’t working it should probably
be made to, routing protocols in general depend on being able to do that.

I don't see how the two are related.
Routing can still be obeyed to an extent.
Consider the following

Interface A has 192.168.0.1/24
Interface B has 192.168.0.2/24
The subnet 192.168/24 belongs to interface A
There is a default route to 192.168.0.10, again on interface A

If I sent a packet to 1.2.3.4 and IP_PKTINFO says use interface B I
would expect the packet to be sent to the default router of 192.168.0.10
but exit via interface B and not interface A.

Roy

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[David Young]

Post by Roy Marples

Post by Dennis Ferguson
It should send the packet out the IP_PKTINFO interface if packet is sent
with MSG_DONTROUTE/SO_DONTROUTE. If this isn’t working it should probably
be made to, routing protocols in general depend on being able to do that.

I don't see how the two are related.
Routing can still be obeyed to an extent.
Consider the following
Interface A has 192.168.0.1/24
Interface B has 192.168.0.2/24
The subnet 192.168/24 belongs to interface A
There is a default route to 192.168.0.10, again on interface A
If I sent a packet to 1.2.3.4 and IP_PKTINFO says use interface B I
would expect the packet to be sent to the default router of
192.168.0.10 but exit via interface B and not interface A.

If you don't use _DONTROUTE, then I would *hope* for the transmission to
fail with an error, probably EHOSTUNREACH, if 1.2.3.4 has no route on
interface B. However, today I sort of *expect* for the transmission to
go out interface A, since the nexthop and its link-layer information are
on interface A.

Dave