nd6_free assumes all routers are processed by kernel RA

[Roy Marples]

I haven't looked at the patch yet (haven't had time recently to
even understand the issue properly) - but while there is a discussion
of IPv6 default routers there's another issue I'd like to get fixed.
My home net has a (commercial) IPv6 capable router, which sends RAs
(etc) announcing itself as a router (whether it should or not is
another issue - but one over which I have no control). My home ISP
however has no IPv6 service (available to me anyway) and for various
reasons right now, no other ISP is an option (I could set up tunnels,
but as the purpose here is not specific v6 research/work, all that would
do is increase delays and overall network traffic, over simply using v4).
Anyway, the point is that the RA the router sends has no v6 prefixes
to advertise, the router (for v6 purposes) is a router in name only,
it cannot actually forward anything anywhere, as all it sever sees is
link local v6 addresses (which is why it perhaps should not be advertising
itself as a router by sending RA's in the first place, but again, that
is a topic outside the scope of this list.)
The relevance to NetBSD is what a NetBSD client host does with a network
in this state. Clearly it can simply disable v6, and for "fixed" hosts
that is what I do (if/when v6 becomes available, I'll just enable it again).
But for a movable host (ie: laptop) that's not convenient, as it keeps
moving between places where v6 is available (and should be used) and
where it is not.
As things are now (well, a few months ago's kernel, and perhaps just
slightly older dhcpcd) a default v6 route gets installed (caused by the
RA from the router, I assume), but no v6 addresses (except link local).
In that state the kernel attempts to route packets via the router to
any destination that the DNS returns a v6 address for (like *.netbsd.org).
The effect of this is that the alternate v4 addresses (which work fine)
are not attempted until the attempt at a v6 connection times out (several
SYN retransmits with several seconds between each ... long enough to be
highly irritating anyway!) (And no, I do not want to priotorise v4 over
v6 - when v6 is available, and working, I prefer to use it.)
If I simply "route delete -inet6 default" manually, then everything works
normally again for a while (often days) until something happens, and the
route is reinstalled (after which I can delete it again, over and over,
until I happen to be connected to a net where v6 is working.)

I've not tested it, but is the same true for v4?
private v4 address, default route to a existing router that doesn't forward.

There are several ways that we could do better here.
One (just to avoid someone suggesting this) that is not an option is
to have the kernel refuse to attempt to send to a IPv6 global address
(or even ULA) when all it has is a link local. The kernel (by definition
in this state) does not know where the host with the global v6 address
is located (it has no prefix info at all) and hence does not know that
the host is not on the local link - communications between a LL addr, and
a global addr that is on the same link should always work (assuming the
link and interfaces are working properly of course).
What we could do is have the kernel refuse to use a router to forward
packets for us, if we have nothing except a LL address - packets forwarded
by a router are useless (and forbidden) when the source addr is LL, as
that addr would be meaningless (in the wrong scope) for the recipient.
If the dest addr is on the local link, no router is needed, if it is not
communications don't work - either way the router is no benefit. If we
were to do that, the installed default v6 route (in my case) would never
be used, and hence, things would be just the same as if it were not there.
Or, we could refuse to automatically add routes when there is no non LL
address available to use them (a faster and less weird alternate to my
current situation where the route gets added, only to soon get deleted again).
I believe it is dhcpcd that is adding the default v6 route, as by
"disable v6" for "fixed" hosts above, what I do is simply config dhcpcd
to only use v4, and in that state, everything is fine (except of course it
never uses v6, even if v6 is available and would work). When it is allowed
to use both v4 and v6 (which is what I want for a laptop) is when the issue
arises.

FWIW, kernel RA handling does the same as dhcpcd in regards to this.

Of those, I think I prefer the second version - don't install useless
routes in the first place, rather than having them installed, but ignored
by the kernel (the first version) - as that then leaves things possible for
someone to manually install a route, and have the kernel use it, if somehow
they have a network configuration where that actually works somehow (I would
perfer not to assume that can never happen).
This behaviour (modification to dhcpcd I am presuming) could be controlled by
yet another config option if desired (and I do not care which way the default
would be set), and would result in dhcpcd never installing a route unless the
system has at least one non-LL address installed (however installed - via
dhcpcd or some other way). Instead dhcpcd would remember the routes it would
have installed (along with all the appropriate bookwork to forget any that
have expired, etc - so a one time RA from some router a week ago is not
remembered forever) and then should a non LL address be configured (by dhcpcd
or otherwise) would at that time install all of the routes that it failed
to install earlier (as in, immediately install, not waiting for the next
RA from the router to cause that to happen.)
I'd need to ponder all the variables more carefully, but it might even be
that the correct behaviour would be not to install a route that is via an
interface that has no non-LL addresses installed, regardless of whether some
other interface might have a non-LL address. (That would mean, for example,
no local routes, to local routers, if none of my local interfaces have
non-LL addresses, even if I have a v6 over v4 tunnel set up, and the gif
interface (or whatever tunnel method I am using) does have a global v6
address (any v6 routes in such a situation would beed to be via tunnels).
For me, these two scenarios result in the same result - either I have
v6 addresses on my useful interface, through which the RA was received
(whether that is wireless or wired can vary day by day, or hour by hour)
or I have no (non-LL) v6 addresses at all - but other hosts might not be
quite so clear cut.
I'm also deliberately omitting any discussion of v4 in all of this .. a true
pure (original) v4 net would be just the same (except it had no RAs nor
LL addresses to confuse things) but these days there are very few such nets
in existance - and it is entirely possible that some router somewhere is
willing to accept v4 LL sourced packets and NAT them into a "global" source
addr - so nothing along the lines suggested above for v6 is likely to be
workable for v4.

I disagree.
IPv4LL actually works quite well behind a NAT so the situation I believe
is the same. What works for one can work for the other.

Opinions?

We should do something with ICMP unreachable maybe? I assume the router
did send something along those lines back?

Roy

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Roy Marples]

This hasn't been the case for a long time if you're a dhcpcd user with a
default config.
As such, it's possible for the default IPv6 router as set by dhcpcd
could be erroneously gc'ed by nd6_free.
Attached is a patch which addresses this by no longer considering any
kernel selected default router and just considering if it is a router or
not.
This reduces the scope of the ND6_WLOCK taken as well as fixing an issue
where we write to ln->ln_state without a lock being held.
I would like this commited fairly promptly as I want it pulled up to -9
and -8 (probablly need some adjustments, I've not checked).

While here, does anyone know why this is guarded by !ip6_forwarding ?
I'm pretty sure I wouldn't want a default route gc'ed on a router either.

Roy

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Robert Elz]

Date: Fri, 23 Aug 2019 20:43:53 +0100
From: Roy Marples <***@marples.name>
Message-ID: <31fc6489-907d-dbef-af0e-***@marples.name>

| I've not tested it, but is the same true for v4?
| private v4 address, default route to a existing router that
| doesn't forward.

I think I answered that later in the message, but yes, mostly, though it
is far less likely that (anything operating today, rather than in, say, 1980)
would not forward v4 in this scenario. Further, a "private" v4 address
is not a LL (that would be more a ULA), LL's in v4 are the 169.whatever.it.is
set (the thing dhcpcd assigns when there is no DHCP server reachable), not 10,
192,168., or whatever the class B version is. (I know you know this.)

| FWIW, kernel RA handling does the same as dhcpcd in regards to this.

I can certainly believe that, at first sight, simply processing an RA
and assigning it as a default router is the sane thing to do - one would
not expect a router to send an RA if it had no actual ability to act as
a router (the perils of cheap implementations....). Whatever gets fixed
should get fixed both ways.

Having the kernel simply not use a router to forward packets which have
a LL source addr would handle all of this in one change, but it just seems
like perhaps a little too servere to me.

| I disagree.
| IPv4LL actually works quite well behind a NAT so the situation I believe
| is the same.

Only if you presume the existence of v6 NAT, which is something I would
hope would never actually catch on, despite some people's apparent desire
for it. There is no need for v6 NAT there are plenty of global addresses
so every host can have one - NAT for no purpose is pointless. Further, if
one really wanted NAT, one would be better to use ULA's as the "stable"
addresses (just as net 10, 192.168, ... are used in v4, as they can be
locally routed around multiple local nets) rather than LL addresses, which
generally are designed to work *only* on one link - that is, some of their
uses are designed with that property in the forefront.

| What works for one can work for the other.

But as I said, I would be happy for some change to not install default
routes, or to have the kernel not use them with LL sources, to be
conditional, so if we have someone who really believes that NAT is
worth using, and LL is suitable as a source for it (neither of which is
a sane opinion, IMO, but never mind) then the system can be configured
to allow it.


| We should do something with ICMP unreachable maybe? I assume the router
| did send something along those lines back?

I never bothered to look, but I shall. Sending the packets in the first
place seems wrong to me, what should happen after we do the wrong thing is
or less consequence than not doing it in the first pace.

I'll wait until I see the default v6 route come back (might be a few
days - I don't know what triggers that, it certainly isn't simply every
RA), and then monitor the network while I try a ssh to netbsd.org or a
cvs update, or similar.

kre


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Roy Marples]

Post by Robert Elz
I never bothered to look, but I shall. Sending the packets in the first
place seems wrong to me, what should happen after we do the wrong thing is
or less consequence than not doing it in the first pace.

I think you're over compliating it.

We have an address on a prefix.
We want to send to an address not on the prefix.
We have a default route.

Given these conditions I don't see any alternative but to send the
packet. Otherwise it's not a default route - it's a route that might or
might not send your packet.

Post by Robert Elz
I'll wait until I see the default v6 route come back (might be a few
days - I don't know what triggers that, it certainly isn't simply every
RA),

It *should* be each RA that advertises itself as a default router.
If you use dhcpcd, then `dhcpcd -n` should be enough to force a
soliciation and get the default route back. If not, try restarting
dhcpcd and file a bug with me.

Post by Robert Elz
and then monitor the network while I try a ssh to netbsd.org or a
cvs update, or similar.

OR just ping6 www.google.com.
I don't think NetBSD.org IPv6 is entirely setup correctly - it's like a
blackhole when I ssh.

Roy

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Robert Elz]

Date: Fri, 23 Aug 2019 20:43:53 +0100
From: Roy Marples <***@marples.name>
Message-ID: <31fc6489-907d-dbef-af0e-***@marples.name>

| We should do something with ICMP unreachable maybe? I assume the router
| did send something along those lines back?

It turns out to not have taken very long at all ... when I next went to
look to see if the default route was back, it was, so I tcpdump'd the
interface, and started a ssh to mail.netbsd.org

The short answer is yes, there is an ICMP unreachable (after each SYN
attempt).

But before the rest of that, replies to your subsequent reply...

***@marples.name said:
| I think you're over compliating it.

That's always possible.

| We have an address on a prefix.

No, we don't, not really - we have an LL address, which has no prefix,
(in the usual sense) just a link that owns it.

| We want to send to an address not on the prefix.

We want to send to an address, we have no idea where that is (as
in the kernel does not). It might be on the local link, it might
be anywhere. It certainly has a prefix that is not LL though.

| We have a default route.

Yes.

| Given these conditions I don't see any alternative but to send the packet.

To attempt to send the packet, yes - to use the default route, no.

All we have is the LL address to use as the source addr. The rules
for v6 say that routers *must not* forward packets using LL addresses.
Hence, sending such a packet to a router is guaranteed to fail (unless
the router is perverse, and doing v6 NAT perhaps - and I assure you,
nothing I have will ever do that.)

Sending the packet by first sending a NS for the target addr, to see if it
happens to be on the local link (about which we have no idea what prefix
may have been allocated) would be a reasonable thing to do - if we get
a reply (including from the router, acting as a proxy-NDP-server if that
is what it wants to do) then we can go ahead and send the packet. But
sending it to a router, and just hoping that the router breaks the v6
forwarding rules seems wrong.

| Otherwise it's not a default route - it's a route that might or might not
| send your packet.

Well, that's always possible, regardless.... It is a default route but
only for routable packets. Packets containing LL addresses are not.

[On when the default v6 route returns:]

***@marples.name said:
| It *should* be each RA that advertises itself as a default router.

That I think it is not. As you will see below, RAs appear to be
being sent approx every 4 seconds (another parameter not able to be
varied, as best I can see ... as I said, cheap router) and the default
route certainly doesn't come back that quickly, if it did, I'd never be
able to do almost anything within reasonable time limits.

When I manually delete the route, it stays gone for (at least) hours,
sometimes days. After the experiment for which the packet trace is below,
I deleted the default route (almost immediately) and it is still gone,
and while that's only a bit more than 1 hour so far, that's about 900
RA packets that have probably been received in the interim. I have
no reason to believe that any one RA is different from any other, though
I suppose one that results from an explicit RS could be different (no RS's
were sent during the few minutes of my experiment - aside from running
dhcpcd with -n (which I certainly do not often do, if ever) or restarting
it (which I also don't often do, though I have once or twice when it
seemed slow detecting a new SSID connection and restarting (v4) DHCP to
get a new addr ... but not while I am just at home), is there anything
else which would cause dhcpcd to send a new RS (given RA's keep arriving
sent to the all hosts multicast addr) of its own volition?

While iwm is not the most stable of wireless interfaces around (our driver
seems a bit lacking) and from time to time needs much love and attention,
there have been no problems beween when I last deleted the v6 default route,
and where I did a "route get -inet6 default" to observe it had returned, in
fact, in my window where I have a root sh, there were no intervening
commands at all, and the interface suffered no interruptions (nothing more
serious that my ISP randomly changing the v4 addr it uses - which is not
the one my router gets told I believe, I think there are 2 levels of NAT,
one in my router, from my local net 10, to the addr the ISP issues it, and
another somewhere in the ISP net before it lets packets into the great
unknown .. the latter is the one which changes, killing all existing v4
connections, far too often! None of that is relevant here though.

***@marples.name said:
| If you use dhcpcd, then `dhcpcd -n` should be enough to force a
| soliciation and get the default route back.

That is not generally something I want to accomplish, and as above,
the default was back all by itself when I looked, so I did not
need to try it this time.

| If not, try restarting dhcpcd and file a bug with me.

OK, though I have no reason to suspect that it would not work ... if
for some reason I need to force the default to be added back (by dhcpcd)
I'll try the -n version, and if it doesn't work, do that and let you
know. My hope would be that the circumstances requiring that will
never occur...

***@marples.name said:
| OR just ping6 www.google.com.

I could have done that, and that would have shown the ICMPv6 exists I
presume, but would have lessened the overall effect of demonstrating the
annoyance caused by the current behaviour... (that is, I wanted to also
show the eventual fallback to v4). Of course, I could have used any
other site than netbsd.org (the connection doesn't actually need to
succeed, just the TCP SYN to get through, even a RST reply would be fine)
and I could have used a dest port other than 22 ... this just happens to
be what I do all the time (I get the same kind of effect from my browser
aimed at www.netbsd.org .. though I suspect it might remember what addr
worked last time, and use it again, for a while at least, as once one
web page gets fetched, others tend to not suffer the delays - if the default
v6 route happens to exist, if it doesn't v4 gets used very quickly, and
all is OK.)


***@marples.name said:
| I don't think NetBSD.org IPv6 is entirely setup correctly - it's like a
| blackhole when I ssh.

I have no issues with it at all from hosts (including my laptop when
applicable) which do have proper v6 addresses. I use v6 to netbsd hosts
all the time. The copy of this mail to the list will most likely use v6
from my MTA (which has real v6) to the netbsd mail server.

But even if there was some issue (I have seen no signs of anything wrong)
it would be irrelevant here - the nature of the issue I have is that no
v6 packet can possibly leave my local net (unless I was to set up a tunnel,
and I have no actual requirement to justify that right now) - so the state
of v6 config at netbsd.org (or anywhere else) is completely irrelevant - no
v6 packets from me can ever reach them (and no v6, or for that matter, any
other, address for my laptop is advertised in the DNS, so nothing is going
to be attempting to make incoming connections ... unless I take pains to
find out, I have no idea what my (real) v4 addr is - except that I know
that by tomorrow, the one in use today will no longer be it).

All of the issues are completely local - what the router does (of which only
sending the RA at all is an issue, everything else it is doing seems correct,
if a little over enthusiastic with its RA packets) and what my NetBSD laptop
does. The only one of those I can alter (short term) is what my laptop does.


So, back to the packet dump:

The relevant part (I hope I didn't delete anything important, I have the
binary packet dump and can dump it again as often as needed, with -v
added if that would be of any use ... but I suspect not) of the packet
trace is below.

It starts with a couple of RA packets (I have left all of those included
here, if for no other purpose than to show time passing...) then the DNS
lookups (AAAA and A) for mail.netbsd.org, then a SYN attempt, ICMPv6 reply
(after a NDP sequence so the router can find my laptop's MAC address ...
you'd think that would not really be needed here, but never mind), followed
by a delay, a new v6 SYN attempt, immediate ICMPv6, delay, 3rd v6 SYN
attempt, ICMPv6, more delay (including a refresh of the router's NDP cache
for my laptop) and eventually a v4 SYN attempt (which succeeds) 75 seconds
(approx) after the sequence started. I left in the 3-way v4 TCP handshake,
but didn't keep any packets after that (nothing interesting there). Other
irrelevant traffic passing through the interface has also been deleted from
the trace included below.

In the following fe80::4432:4eff:fe51:1113 (aka 10.100.10.100) is the router,
(which is also the DNS/NAT/DHCP/... server) and fe80::3613:e8ff:fe2f:bbc4
(aka 10.100.10.151) is my laptop. 2001:470:a085:999::25 (aka 199.233.217.200)
is mail.netbsd.org of course.

Times are all UTC+07 (+/- 20 seconds... I don't bother running an NTP
client on my laptop, just ntpdate it every month or so, and it stays
"close enough")

Another 20 mins have passed since the "only 1 hour" above, and the default
v6 route is still absent...

jinx# route get -inet6 default
route: writing to routing socket: not in table

When I looked earlier:

jinx# route delete -inet6 default
delete net default
jinx# route get -inet6 default
route to: ::
destination: ::
mask: default
gateway: fe80::4432:4eff:fe51:1113%iwm0
local addr: fe80::3613:e8ff:fe2f:bbc4%iwm0
interface: iwm0
flags: 0x843<UP,GATEWAY,DONE,STATIC>
recvpipe sendpipe ssthresh rtt,msec rttvar hopcount mtu expire
0 0 0 0 0 0 0 0
jinx# tcpdump -i iwm0 -s 1600 -w /var/log/IWM0-DUMP

The initial "route delete" was yesterday, perhaps even the day
before, the "route get" was this morning, the tcpdump is what captued
the packets shown just below (the ssh happened from a non-root xterm).

kre

08:01:38.000997 IP6 fe80::4432:4eff:fe51:1113 > ff02::1: ICMP6, router advertisement, length 24
08:01:44.861871 IP6 fe80::4432:4eff:fe51:1113 > ff02::1: ICMP6, router advertisement, length 24
08:01:45.920148 IP 10.100.10.151.54189 > 10.100.10.100.53: 13156+ AAAA? mail.netbsd.org. (33)
08:01:46.145309 IP 10.100.10.100.53 > 10.100.10.151.54189: 13156 1/0/0 AAAA 2001:470:a085:999::25 (61)
08:01:46.145427 IP 10.100.10.151.54188 > 10.100.10.100.53: 28632+ A? mail.netbsd.org. (33)
08:01:46.376236 IP 10.100.10.100.53 > 10.100.10.151.54188: 28632 1/0/0 A 199.233.217.200 (49)
08:01:46.376367 IP6 fe80::3613:e8ff:fe2f:bbc4.64233 > 2001:470:a085:999::25.22: Flags [S], seq 950175903, win 32768, options [mss 1440,nop,wscale 3,sackOK,TS val 1 ecr 0], length 0
08:01:46.492703 IP6 fe80::4432:4eff:fe51:1113 > ff02::1:ff2f:bbc4: ICMP6, neighbor solicitation, who has fe80::3613:e8ff:fe2f:bbc4, length 32
08:01:46.492723 IP6 fe80::3613:e8ff:fe2f:bbc4 > fe80::4432:4eff:fe51:1113: ICMP6, neighbor advertisement, tgt is fe80::3613:e8ff:fe2f:bbc4, length 32
08:01:46.494232 IP6 fe80::4432:4eff:fe51:1113 > fe80::3613:e8ff:fe2f:bbc4: ICMP6, destination unreachable, unreachable route 2001:470:a085:999::25, length 88
08:01:52.384914 IP6 fe80::3613:e8ff:fe2f:bbc4.64233 > 2001:470:a085:999::25.22: Flags [S], seq 950175903, win 32768, options [mss 1440,nop,wscale 3,sackOK,TS val 13 ecr 0], length 0
08:01:52.386529 IP6 fe80::4432:4eff:fe51:1113 > fe80::3613:e8ff:fe2f:bbc4: ICMP6, destination unreachable, unreachable route 2001:470:a085:999::25, length 88
08:01:52.849432 IP6 fe80::4432:4eff:fe51:1113 > ff02::1: ICMP6, router advertisement, length 24
08:02:02.679810 IP6 fe80::4432:4eff:fe51:1113 > ff02::1: ICMP6, router advertisement, length 24
08:02:04.405197 IP6 fe80::3613:e8ff:fe2f:bbc4.64233 > 2001:470:a085:999::25.22: Flags [S], seq 950175903, win 32768, options [mss 1440,nop,wscale 3,sackOK,TS val 37 ecr 0], length 0
08:02:04.407008 IP6 fe80::4432:4eff:fe51:1113 > fe80::3613:e8ff:fe2f:bbc4: ICMP6, destination unreachable, unreachable route 2001:470:a085:999::25, length 88
08:02:20.600146 IP6 fe80::4432:4eff:fe51:1113 > ff02::1: ICMP6, router advertisement, length 24
08:02:28.445815 IP6 fe80::3613:e8ff:fe2f:bbc4.64233 > 2001:470:a085:999::25.22: Flags [S], seq 950175903, win 32768, options [mss 1440,nop,wscale 3,sackOK,TS val 85 ecr 0], length 0
08:02:28.447575 IP6 fe80::4432:4eff:fe51:1113 > fe80::3613:e8ff:fe2f:bbc4: ICMP6, destination unreachable, unreachable route 2001:470:a085:999::25, length 88
08:02:29.611501 IP6 fe80::4432:4eff:fe51:1113 > ff02::1: ICMP6, router advertisement, length 24
08:02:33.445243 IP6 fe80::4432:4eff:fe51:1113 > fe80::3613:e8ff:fe2f:bbc4: ICMP6, neighbor solicitation, who has fe80::3613:e8ff:fe2f:bbc4, length 32
08:02:33.445292 IP6 fe80::3613:e8ff:fe2f:bbc4 > fe80::4432:4eff:fe51:1113: ICMP6, neighbor advertisement, tgt is fe80::3613:e8ff:fe2f:bbc4, length 24
08:02:33.605204 IP6 fe80::4432:4eff:fe51:1113 > ff02::1: ICMP6, router advertisement, length 24
08:02:41.490118 IP6 fe80::4432:4eff:fe51:1113 > ff02::1: ICMP6, router advertisement, length 24
08:02:48.775761 IP6 fe80::4432:4eff:fe51:1113 > ff02::1: ICMP6, router advertisement, length 24
08:02:53.565935 IP6 fe80::4432:4eff:fe51:1113 > ff02::1: ICMP6, router advertisement, length 24
08:02:57.669666 IP6 fe80::4432:4eff:fe51:1113 > ff02::1: ICMP6, router advertisement, length 24
08:03:01.501772 IP 10.100.10.151.56761 > 199.233.217.200.22: Flags [S], seq 3492651494, win 32768, options [mss 1460,nop,wscale 3,sackOK,TS val 1 ecr 0], length 0
08:03:01.775397 IP 199.233.217.200.22 > 10.100.10.151.56761: Flags [S.], seq 422082190, ack 3492651495, win 32768, options [mss 1440,nop,wscale 3,sackOK,TS val 1 ecr 1], length 0
08:03:01.775419 IP 10.100.10.151.56761 > 199.233.217.200.22: Flags [.], ack 1, win 4197, options [nop,nop,TS val 1 ecr 1], length 0



--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Robert Elz]

I haven't looked at the patch yet (haven't had time recently to
even understand the issue properly) - but while there is a discussion
of IPv6 default routers there's another issue I'd like to get fixed.

My home net has a (commercial) IPv6 capable router, which sends RAs
(etc) announcing itself as a router (whether it should or not is
another issue - but one over which I have no control). My home ISP
however has no IPv6 service (available to me anyway) and for various
reasons right now, no other ISP is an option (I could set up tunnels,
but as the purpose here is not specific v6 research/work, all that would
do is increase delays and overall network traffic, over simply using v4).

Anyway, the point is that the RA the router sends has no v6 prefixes
to advertise, the router (for v6 purposes) is a router in name only,
it cannot actually forward anything anywhere, as all it sever sees is
link local v6 addresses (which is why it perhaps should not be advertising
itself as a router by sending RA's in the first place, but again, that
is a topic outside the scope of this list.)

The relevance to NetBSD is what a NetBSD client host does with a network
in this state. Clearly it can simply disable v6, and for "fixed" hosts
that is what I do (if/when v6 becomes available, I'll just enable it again).
But for a movable host (ie: laptop) that's not convenient, as it keeps
moving between places where v6 is available (and should be used) and
where it is not.

As things are now (well, a few months ago's kernel, and perhaps just
slightly older dhcpcd) a default v6 route gets installed (caused by the
RA from the router, I assume), but no v6 addresses (except link local).

In that state the kernel attempts to route packets via the router to
any destination that the DNS returns a v6 address for (like *.netbsd.org).
The effect of this is that the alternate v4 addresses (which work fine)
are not attempted until the attempt at a v6 connection times out (several
SYN retransmits with several seconds between each ... long enough to be
highly irritating anyway!) (And no, I do not want to priotorise v4 over
v6 - when v6 is available, and working, I prefer to use it.)

If I simply "route delete -inet6 default" manually, then everything works
normally again for a while (often days) until something happens, and the
route is reinstalled (after which I can delete it again, over and over,
until I happen to be connected to a net where v6 is working.)

There are several ways that we could do better here.

One (just to avoid someone suggesting this) that is not an option is
to have the kernel refuse to attempt to send to a IPv6 global address
(or even ULA) when all it has is a link local. The kernel (by definition
in this state) does not know where the host with the global v6 address
is located (it has no prefix info at all) and hence does not know that
the host is not on the local link - communications between a LL addr, and
a global addr that is on the same link should always work (assuming the
link and interfaces are working properly of course).

What we could do is have the kernel refuse to use a router to forward
packets for us, if we have nothing except a LL address - packets forwarded
by a router are useless (and forbidden) when the source addr is LL, as
that addr would be meaningless (in the wrong scope) for the recipient.
If the dest addr is on the local link, no router is needed, if it is not
communications don't work - either way the router is no benefit. If we
were to do that, the installed default v6 route (in my case) would never
be used, and hence, things would be just the same as if it were not there.

Or, we could refuse to automatically add routes when there is no non LL
address available to use them (a faster and less weird alternate to my
current situation where the route gets added, only to soon get deleted again).

I believe it is dhcpcd that is adding the default v6 route, as by
"disable v6" for "fixed" hosts above, what I do is simply config dhcpcd
to only use v4, and in that state, everything is fine (except of course it
never uses v6, even if v6 is available and would work). When it is allowed
to use both v4 and v6 (which is what I want for a laptop) is when the issue
arises.

Of those, I think I prefer the second version - don't install useless
routes in the first place, rather than having them installed, but ignored
by the kernel (the first version) - as that then leaves things possible for
someone to manually install a route, and have the kernel use it, if somehow
they have a network configuration where that actually works somehow (I would
perfer not to assume that can never happen).

This behaviour (modification to dhcpcd I am presuming) could be controlled by
yet another config option if desired (and I do not care which way the default
would be set), and would result in dhcpcd never installing a route unless the
system has at least one non-LL address installed (however installed - via
dhcpcd or some other way). Instead dhcpcd would remember the routes it would
have installed (along with all the appropriate bookwork to forget any that
have expired, etc - so a one time RA from some router a week ago is not
remembered forever) and then should a non LL address be configured (by dhcpcd
or otherwise) would at that time install all of the routes that it failed
to install earlier (as in, immediately install, not waiting for the next
RA from the router to cause that to happen.)

I'd need to ponder all the variables more carefully, but it might even be
that the correct behaviour would be not to install a route that is via an
interface that has no non-LL addresses installed, regardless of whether some
other interface might have a non-LL address. (That would mean, for example,
no local routes, to local routers, if none of my local interfaces have
non-LL addresses, even if I have a v6 over v4 tunnel set up, and the gif
interface (or whatever tunnel method I am using) does have a global v6
address (any v6 routes in such a situation would beed to be via tunnels).

For me, these two scenarios result in the same result - either I have
v6 addresses on my useful interface, through which the RA was received
(whether that is wireless or wired can vary day by day, or hour by hour)
or I have no (non-LL) v6 addresses at all - but other hosts might not be
quite so clear cut.

I'm also deliberately omitting any discussion of v4 in all of this .. a true
pure (original) v4 net would be just the same (except it had no RAs nor
LL addresses to confuse things) but these days there are very few such nets
in existance - and it is entirely possible that some router somewhere is
willing to accept v4 LL sourced packets and NAT them into a "global" source
addr - so nothing along the lines suggested above for v6 is likely to be
workable for v4.

Opinions?

kre


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Gert Doering]

Hi,

Post by Robert Elz
Sending the packet by first sending a NS for the target addr, to see if it
happens to be on the local link (about which we have no idea what prefix
may have been allocated) would be a reasonable thing to do - if we get
a reply (including from the router, acting as a proxy-NDP-server if that
is what it wants to do) then we can go ahead and send the packet.

This is not actually a reasonable thing to do - in your scenario, this would
mean "send a NS for every single v6 address out there that you want to talk
to, and since no NA ever comes back, the application would run into the same
timeout behaviour as before".

Even if you have something that does proxy-NDP, it would still be a bad
idea to fill the local neighbour table with an entry for every single
communication partner out there (people did - and do - this in v4,
point a default route "to ethernet" and rely on proxy ARP for packets
to go to the router, and all good it does is "hide configuration errors").

gert
--
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never doubted
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany ***@greenie.muc.de

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Roy Marples]

Anyway, the point is that the RA the router sends has no v6 prefixes
to advertise, the router (for v6 purposes) is a router in name only,
it cannot actually forward anything anywhere, as all it sever sees is
link local v6 addresses (which is why it perhaps should not be advertising
itself as a router by sending RA's in the first place, but again, that
is a topic outside the scope of this list.)
The relevance to NetBSD is what a NetBSD client host does with a network
in this state. Clearly it can simply disable v6, and for "fixed" hosts
that is what I do (if/when v6 becomes available, I'll just enable it again).
But for a movable host (ie: laptop) that's not convenient, as it keeps
moving between places where v6 is available (and should be used) and
where it is not.
As things are now (well, a few months ago's kernel, and perhaps just
slightly older dhcpcd) a default v6 route gets installed (caused by the
RA from the router, I assume), but no v6 addresses (except link local).
In that state the kernel attempts to route packets via the router to
any destination that the DNS returns a v6 address for (like *.netbsd.org).
The effect of this is that the alternate v4 addresses (which work fine)
are not attempted until the attempt at a v6 connection times out (several
SYN retransmits with several seconds between each ... long enough to be
highly irritating anyway!) (And no, I do not want to priotorise v4 over
v6 - when v6 is available, and working, I prefer to use it.)
If I simply "route delete -inet6 default" manually, then everything works
normally again for a while (often days) until something happens, and the
route is reinstalled (after which I can delete it again, over and over,
until I happen to be connected to a net where v6 is working.)
There are several ways that we could do better here.

...

Or, we could refuse to automatically add routes when there is no non LL
address available to use them (a faster and less weird alternate to my
current situation where the route gets added, only to soon get deleted again).

https://roy.marples.name/cgit/dhcpcd.git/commit/?id=a3830c3f3d7af8db0353f11a3625979e122819a7

inet6: Don't install a default route if no global address

This works around an issue where some routers advertise themselves
as a default router, but offer no address to actually use.
Thus, it is actually useless as a router.
If an address is added by any other means (DHCPv6, static, etc)
then the default route is installed once more.

This is also dynamic - dhcpcd reacts to addresses being added,
removed, duplicated, detached, etc.

Not sure if the linked patch applies to current as I made some other
changes in this area about reachability as noted on this list earlier.

Robert, could you try my dhcpcd git master and see if it solves your
problem please? Or download a tarball.

git://roy.marples.name/dhcpcd.git
https://roy.marples.name/cgit/dhcpcd.git/snapshot/master.tar.xz

Thanks

Roy

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Robert Elz]

Date: Sun, 25 Aug 2019 15:01:23 +0100
From: Roy Marples <***@marples.name>
Message-ID: <5501a485-6982-a2a6-43bb-***@marples.name>

| Robert, could you try my dhcpcd git master and see if it solves your
| problem please? Or download a tarball.

I have fetched and built the tarball version (no git client...), but
right now I'm in the middle of a couple of huge downloads which will
continue for another couple of hours, and I don't want to take any
risks with the network config while they happen (only restartable, not
resumable). As soon as they're done I'll install this new version and
test it.

Thanks for the mods, it sounds like it should solve my problem perfectly.

One question though:

| inet6: Don't install a default route if no global addresss

Is that literally what it is, or is "no global address" a shorthand
for "no address other than a LL address" ? It makes no difference
to me (to my system) as global addr or only LL address are all my
syste ever sees (that I can think of anyway) - but a system with ULA
addresses should get routes installed too.

No need to reply, I will find out when I test it (I will test the ULA
case, just to find out) but as that is a few hours away yet, and I am
curious....

kre



--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Roy Marples]

Post by Robert Elz
detached addrs are no
different really than addrs that don't exist (at least for a time).

So address persistence is a thing. for dhcpcd, when the carrier drops
all addresses and config are removed until it can validate an address on
a new link - to ensure we don't use an address we're not supposed to.

On NetBSD it's different - because the kernel handles DaD for IPv4 and
IPv6 addresses AND denies sending from them until validated dhcpcd
allows them to persist. This has the added benefit of persisting things
such as ssh if the subnet changes from wired to wireless and back again.

Now, if the link drops, the address will become detached so the new
dhcpcd will then drop the default route. This may kill traffic which
would be bad if the address comes back in a reasonable time. Hence my
thought to just restrict lack of a default route to duplicated addresses.

But I've not entirely thought this through yet, hence the current state
of just using useable addresses.

Roy

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Robert Elz]

Date: Mon, 26 Aug 2019 00:30:27 +0100
From: Roy Marples <***@marples.name>
Message-ID: <236701f8-cd06-7eb0-0004-***@marples.name>

| Now, if the link drops, the address will become detached so the new
| dhcpcd will then drop the default route. This may kill traffic which
| would be bad if the address comes back in a reasonable time. Hence my
| thought to just restrict lack of a default route to duplicated addresses.

OK, that makes sense.

My downloads finished a short while ago, so I have upgraded now.

First, with the old dhcpcd (which was 7.1.1) running dhcpcd -n did not
reinstate the default route (though I did not wait all that long ... more
than 4 seconds though, I think). Restarting that version (via rc.d/..
stop / start) did bring the default route back. I removed the default
route again, updated dhcpcd to the new version (8.0.3), and restarted that
version (same way) and no default route appeared.

I added a global (purely invented) v6 addr, and the default route was
added, then deleted that address again, and the defaut route went away.

So that looks like success. Thanks.

Since you answered the earlier query I don't think I need to test ULA
addresses to verify that the same things work there, so I didn't...

kre




--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Roy Marples]

Post by Robert Elz
| inet6: Don't install a default route if no global addresss
Is that literally what it is, or is "no global address" a shorthand
for "no address other than a LL address" ? It makes no difference
to me (to my system) as global addr or only LL address are all my
syste ever sees (that I can think of anyway) - but a system with ULA
addresses should get routes installed too.
No need to reply, I will find out when I test it (I will test the ULA
case, just to find out) but as that is a few hours away yet, and I am
curious....

I'll reply anyway.
The test is for any valid non LL address.
By valid, that currently means not tentative, duplicated nor detached.
Although I am considering relaxing that to not duplicated.

Roy

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Robert Elz]

Date: Sun, 25 Aug 2019 22:53:40 +0100
From: Roy Marples <***@marples.name>
Message-ID: <e043821a-3a85-a737-e48d-***@marples.name>

| I'll reply anyway.

Thanks, still another 90 mins or so before I can test...

| The test is for any valid non LL address.

Great.

| By valid, that currently means not tentative, duplicated nor detached.
| Although I am considering relaxing that to not duplicated.

I think I'd just drop tentative - duplicated addresses are clearly bad
(I'm not exactly sure why such a thing is even allowed to remain, rather
than just being dropped, but that's unimportant) and detached addrs are no
different really than addrs that don't exist (at least for a time).

tentative though I believe are real addresses - slightly dangerous to use
as they may turn out to be duplicated, but the chances of that are small
(I don't think I have ever seen a non-specifically-manufactured duplicated
v6 address) so it normally is perfectly OK to treat a tentative addr as if
it were not one (which is also why some people believe that not running DaD
at all is acceptable - I am not one of those).

kre

ps: the default route that I deleted after running the test the other day
still has not returned, so before I shut down the current (older) dhcpcd
I'll try running it with -n to see if that does bring it back. Then
assuming it does, I will delete it again before starting the new one.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Gert Doering]

Hi,

Post by Roy Marples

Opinions?

We should do something with ICMP unreachable maybe? I assume the router
did send something along those lines back?

If we do not have a global source address, sending packets via a router
is pretty much guaranteed to violate a number of RFCs.

So hoping that a router will tell us by means of ICMPv6 what we already
know (and hoping that it is even able to do so, which implies special
handling over there to get the ICMPv6 to a LLA out the correct interface)
does not sound like a good plan forward.

gert
--
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never doubted
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany ***@greenie.muc.de

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Robert Elz]

Date: Sat, 24 Aug 2019 10:26:27 +0200
From: Gert Doering <***@greenie.muc.de>
Message-ID: <***@greenie.muc.de>

| If we do not have a global source address, sending packets via a router
| is pretty much guaranteed to violate a number of RFCs.

That's what I expect too, though ULAs, which are not global - that is
the FC00::/7 prefix, can be routed. It is just LLs that cannot.

| So hoping that a router will tell us by means of ICMPv6 what we already
| know

If we do send, there should be less "hope" and more "expectation" there.
The router should (perhaps rate limited, as usual) send an ICMPv6 about
packets it is discarding.

And does in my case.

| (and hoping that it is even able to do so, which implies special
| handling over there to get the ICMPv6 to a LLA out the correct interface)
| does not sound like a good plan forward.

The router knows over which interface it received the packet - it needs
to retain that info in order to know when it might be appropriate to
send a redirect in the cases where it is forwarding the packet, so sending
the ICMPv6 back to the right place is not going to be a problem.

Certainly it would be useful to do something intelligent with such an
ICMP when received, though knowing what that is isn't quite as easy as
it sounds. Blindly treating it as a "cannot get there, don't try again"
opens the host up to simple DoS attacks. Causing the host to try a
different address if one is available, before retrying the one that
reported the ICMP would be good (no DoS, just different routing) but
we don't really have the API model to allow that - the dest addr
choice is in the application, whereas retries to non-responsive SYN
packets is in TCP in the kernel. Getting that to behave in a sane
fashion utilising ICMP mesages as hints, but not necessarily fatal
blocks would be an "interesting" code set to write (and then fit into
every network application that exists). If the API were "connect to one
of these addresses please" things would be easier.

Gert Doering <***@greenie.muc.de> (in a different reply) also said:

| This is not actually a reasonable thing to do - in your scenario,
| this would mean "send a NS for every single v6 address out there
| that you want to talk to,

Yes. Of course only hosts that have only LL addresses, and which have
v6 enabled, act that way, which means not very many of them.

| and since no NA ever comes back, the application would run into the
| same timeout behaviour as before".

Not really, the timeout waiting for a non-replied NS can be much much
shorter (since we know the responder, if it exists, must be on our link)
and we also don't need to retransmit as often (one retransmit is probably
enough, as the only common cause of a missing reply should be recipient
overload, or non-existence. If it is so overloaded that it fails to respond
to 2 successive NS requests it is probably also so overloaded that it isn't
worth attempting to communicate with, so simply treating that case the same
as non-existence seems reasonable.) A delay of 0.5 of a second or so would
be tolerable (barely noticeable), and nothing at all like the 75 second
delay I am seeing (and which is much worse when the destination host has
multiple v6 addresses, that get tried in turn, before a fallback to v4
happens.)

This issue is orthogonal to my actual problem though - this is something
else that perhaps should be fixed (and could also be configurable, so hosts
that don't ever actually want to communicate with a global neighbour addr
when they have no global addr of their own, could avoid doing this). I
believe it is (or was at least) part of the original v6 design however.

| Even if you have something that does proxy-NDP, it would still be a bad
| idea to fill the local neighbour table with an entry for every single
| communication partner out there (people did - and do - this in v4, point
| a default route "to ethernet" and rely on proxy ARP for packets to go
| to the router, and all good it does is "hide configuration errors").

I agree this is not a good way to configure a net - proxy NDP (if anyone
has implemented that anywhere) does not really sound like a reasonable and
certainly not necessary model ... but I would be hesitant to prohibit it
either.

I am certainly not concerned with the size of the neighbour cache, if we
were communicating with all that many hosts, the TCP state tables would
be much larger, and no-one complains about those (further, unlike TCP state,
NDP caches are caches - if one gets too big, we can always just discard the
oldest (counting usage, not creation) and if it turns out to be have still
been needed, simply NS for it again.

Once there are no more packets being sent/received (ie: TCP has shut down)
the neighbour cache will quickly age out and be deleted. TCP is what
matters here, as that is what is used to communicate with all those random
communication partners in practice (web, e-mail, and similar). UCP (and ICMP)
are used typically for a much more restricted set, and/or for very limited
transactions (one DNS query/reply) that would allow the NDP cache to be
cleared quite quickly. In practice, this would not be a problem worth
worrying about. A system with very limited memory resources wouldn't be
making many "random" connections to hosts in the first place -- and recall,
all of this is material only for hosts with only LL addresses - if a host
has an addr of sufficient scope, it simply uses that to communicate, and
that one gets sent via a router, if not to a local destination, and none
of this is relevant.

kre



--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Gert Doering]

Hi,

Post by Robert Elz
Date: Sat, 24 Aug 2019 10:26:27 +0200
| If we do not have a global source address, sending packets via a router
| is pretty much guaranteed to violate a number of RFCs.
That's what I expect too, though ULAs, which are not global - that is
the FC00::/7 prefix, can be routed. It is just LLs that cannot.

Apologies for not writing down precisely what I was thinking. Yes, indeed,
that paragraph should have read "... do not have any source address
but link-local (fe80::), sending packets via a router..."

ULAs are fully routable, though by convention not routed across the
public Internet. So with ULAs, you might end up hitting a NAT66 box
on the way to the Internet, and it would magically work (which I expect
to see in enterprise deployments where renumbering on ISP change is
more expensive than a NAT box...)

[..]

Post by Robert Elz
| This is not actually a reasonable thing to do - in your scenario,
| this would mean "send a NS for every single v6 address out there
| that you want to talk to,
Yes. Of course only hosts that have only LL addresses, and which have
v6 enabled, act that way, which means not very many of them.

"Only LL addresses and have v6 enabled" would be "any recent operating
system in an IPv4 only environment", of which there are many.

Post by Robert Elz
| and since no NA ever comes back, the application would run into the
| same timeout behaviour as before".
Not really, the timeout waiting for a non-replied NS can be much much
shorter (since we know the responder, if it exists, must be on our link)
and we also don't need to retransmit as often (one retransmit is probably
enough, as the only common cause of a missing reply should be recipient
overload, or non-existence. If it is so overloaded that it fails to respond
to 2 successive NS requests it is probably also so overloaded that it isn't
worth attempting to communicate with, so simply treating that case the same
as non-existence seems reasonable.) A delay of 0.5 of a second or so would
be tolerable (barely noticeable), and nothing at all like the 75 second
delay I am seeing (and which is much worse when the destination host has
multiple v6 addresses, that get tried in turn, before a fallback to v4
happens.)

To make that work, the neighbour discovery code could would need to
communicate the timeout back to the TCP SYN sender (or more generic,
"the entity that sent the original IPv6 packet") somehow - "we tried
to send your packet but had to give up". Sort of "TTL exceeded"...

gert
--
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never doubted
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany ***@greenie.muc.de

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Robert Elz]

Date: Sat, 24 Aug 2019 21:33:30 +0200
From: Gert Doering <***@greenie.muc.de>
Message-ID: <***@greenie.muc.de>

| "Only LL addresses and have v6 enabled" would be "any recent operating
| system in an IPv4 only environment", of which there are many.

Unfortunately still, yes... But it also requires that the host do
lookups for AAAA addresses which raises another potential solution to
my real problem. That is, another solution would be to have getaddrinfo()
not request v6 addresses unless the host has a non-LL v6 address of its
own. I don't mych like this. first because it makes getaddrinfo() (slightly)
more expensive for everyone, and second because it prevents obtaining a
v6 address for a target that could have been reached, even with a LL source
addr (ie: a target on the same link as the host wanting to contact it).


| To make that work, the neighbour discovery code could would need to
| communicate the timeout back to the TCP SYN sender (or more generic,

No, not really. Note that the initial problem needs to be fixed first.

We need to avoid, one way or another, sending packets from LL source
addresses to a router to forward to a global (or ULA) address.

That can be done by not installing routes (by default) when there are no
non-LL addresses, so there is nothing to forward via, or by allowing the
routes to be installed but refusing to use them. I prefer the former as
it would be "not automatically install" rather than "cannot install", so
if there is a need, the admin can always add a route, and it would be used
(even if I think that's a dumb thing to actually want to do.)

Once the system looks like that, then sending a SYN from a LL address
can send a local NS, if that fails (no response after a short wait, repeat
NS< and another short wait), the TCP connect simply fails (EHOSTUNREACH)
as there is no other way to send the SYN packet. If the NS succeeds, then
the SYN is sent and whatever happens after happens. No magic required.

This is the way v6 autoconfig should work - the intent was always that
hosts could simply connect to a network (including ad-hoc routerless nets)
and communicate with other connected hosts, with no manual config required.

kre


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de