Google Compute Engine networking stopped working

[Benny Siegert]

Hi!

I have a NetBSD VM with working networking in Google Compute Engine,
and a new one where it does not work.

GCE Networking has DHCP return a host address with a /32 netmask, plus
a gateway with a different address. The trick is that you also need a
host route to the gateway, directly over the interface.

The working setup is on this:

NetBSD 7.99.65 (DIAGNOSTIC) #0: Tue Mar 7 21:47:12 CET 2017

# /usr/pkg/sbin/dhcpcd --version
dhcpcd 6.11.5
Copyright (c) 2006-2016 Roy Marples
Compiled in features: INET IPv4LL INET6 DHCPv6 AUTH

# cat /etc/ifconfig.vioif0
!/usr/pkg/sbin/dhcpcd vioif0
!route add default `ifconfig vioif0 | awk '/inet / { print $2 }' | sed
's/[0-9]*$/1/'` -ifp vioif0

dhcpcd fails to set a default route, then it runs something like
# route add default 10.240.0.1 -ifp vioif0
# netstat -rnf inet
Routing tables
Internet:
Destination Gateway Flags Refs Use Mtu Interface
default 10.240.0.1 UGS - - - vioif0
10.240.0.1/32 link#2 UC - - 1460 vioif0
10.240.0.3 127.0.0.1 UGHS - - 1460 lo0
10.240.0.3/32 link#2 UC - - - vioif0
127/8 127.0.0.1 UGRS - - 33624 lo0
127.0.0.1 lo0 UH - - 33624 lo0
# ifconfig vioif0
vioif0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
ec_capabilities=1<VLAN_MTU>
ec_enabled=0
address: 42:01:0a:f0:00:03
inet 10.240.0.3 netmask 0xffffffff broadcast 10.240.0.3
inet6 fe80::1811:269b:ecb0:bcdf%vioif0 prefixlen 64 scopeid 0x2

The machine where this is not working is a recent -8:

NetBSD 8.0_BETA (GENERIC.201708192310Z)

# dhcpcd --version
dhcpcd 7.0.0-rc1
Copyright (c) 2006-2017 Roy Marples
Compiled in features: INET ARP IPv4LL INET6 DHCPv6 AUTH


From a non-configured network, I run dhcpcd:
# dhcpcd vioif0
DUID 00:01:00:01:21:2d:7a:15:52:54:00:12:34:56
vioif0: IAID 0a:f0:00:02
vioif0: rebinding lease of 10.240.0.2
vioif0: soliciting an IPv6 router
vioif0: leased 10.240.0.2 for 86400 seconds
vioif0: adding route to 10.240.0.2/32
vioif0: adding host route to 255.255.255.255
vioif0: adding default route via 10.240.0.1
forked to background, child pid 336

# netstat -rnf inet
Routing tables
Internet:
Destination Gateway Flags Refs Use Mtu Interface
default 10.240.0.1 UGS - - 1460 vioif0
10.240.0.2 link#2 UHC - - 1460 vioif0
10.240.0.2/32 link#2 UC - - - vioif0
127.0.0.1 lo0 UHl - - 33624 lo0
255.255.255.255 link#2 UHCS - - 1460 vioif0
# ping 10.240.0.1
PING 10.240.0.1 (10.240.0.1): 56 data bytes
ping: sendto: Invalid argument
# ifconfig vioif0
vioif0: flags=0x8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
ec_capabilities=1<VLAN_MTU>
ec_enabled=0
address: 42:01:0a:f0:00:02
inet 10.240.0.2/32 broadcast 10.240.0.2 flags 0x0
inet6 fe80::7a06:ce6a:2f4a:8e44%vioif0/64 flags 0x0 scopeid 0x2

This works:

# route delete default
delete net default
# route add -interface -host 10.240.0.1 10.240.0.2
add host 10.240.0.1: gateway 10.240.0.2
# route add default 10.240.0.1
add net default: gateway 10.240.0.1

How can I get dhcpcd to do the right thing?

--Benny.

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Martin Husemann]

Post by Benny Siegert
How can I get dhcpcd to do the right thing?

Add a script and hook for BOUND ?
See dhcpcd-run-hooks(8).

Martin

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Roy Marples]

Hi Benny

Post by Benny Siegert
I have a NetBSD VM with working networking in Google Compute Engine,
and a new one where it does not work.
GCE Networking has DHCP return a host address with a /32 netmask, plus
a gateway with a different address. The trick is that you also need a
host route to the gateway, directly over the interface.

Then GCE Networking should also set a static route to the gateway like
so (where the gateway IP is 1.2.3.4):
1.2.3.4 1.2.3.4

Or via a CSR option:
32/1.2.3.4 1.2.3.4

Then dhcpcd works fine, and a lot of ISPs already do this via DHCP.

Or as Martin said, script it via a custom exit hook like so:

if $if_up; then
add_host_route
elif $if_down; then
remove_host_route
fi

Roy

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Benny Siegert]

Post by Roy Marples

Post by Benny Siegert
GCE Networking has DHCP return a host address with a /32 netmask, plus
a gateway with a different address. The trick is that you also need a
host route to the gateway, directly over the interface.

Then GCE Networking should also set a static route to the gateway like
1.2.3.4 1.2.3.4
32/1.2.3.4 1.2.3.4

It comes via a classless static route:

# dhcpcd -U vioif0
broadcast_address=10.240.0.2
classless_static_routes='10.240.0.1/32 0.0.0.0 0.0.0.0/0 10.240.0.1'
dhcp_lease_time=86400
dhcp_message_type=5
dhcp_server_identifier=169.254.169.254
domain_name=c.gobuilder-bsiegert.internal
domain_name_servers=169.254.169.254
domain_search='c.gobuilder-bsiegert.internal google.internal'
host_name=netbsd8-dhcptest.c.gobuilder-bsiegert.internal
interface_mtu=1460
ip_address=10.240.0.2
network_number=10.240.0.2
ntp_servers=169.254.169.254
routers=10.240.0.1
subnet_cidr=32
subnet_mask=255.255.255.255


Is it only taking the "routers" field into account and disregarding
the classless_static_routes?

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Roy Marples]

Post by Benny Siegert

Post by Roy Marples

Post by Benny Siegert
GCE Networking has DHCP return a host address with a /32 netmask, plus
a gateway with a different address. The trick is that you also need a
host route to the gateway, directly over the interface.

Then GCE Networking should also set a static route to the gateway like
1.2.3.4 1.2.3.4
32/1.2.3.4 1.2.3.4

# dhcpcd -U vioif0
broadcast_address=10.240.0.2
classless_static_routes='10.240.0.1/32 0.0.0.0 0.0.0.0/0 10.240.0.1'
dhcp_lease_time=86400
dhcp_message_type=5
dhcp_server_identifier=169.254.169.254
domain_name=c.gobuilder-bsiegert.internal
domain_name_servers=169.254.169.254
domain_search='c.gobuilder-bsiegert.internal google.internal'
host_name=netbsd8-dhcptest.c.gobuilder-bsiegert.internal
interface_mtu=1460
ip_address=10.240.0.2
network_number=10.240.0.2
ntp_servers=169.254.169.254
routers=10.240.0.1
subnet_cidr=32
subnet_mask=255.255.255.255
Is it only taking the "routers" field into account and disregarding
the classless_static_routes?

Over way around :)
If classless_static_routes is present then dhcpcd ignores both routers
and static_routes.

From the looks of it, this should already be fixed in -current where we
have dhcpcd-7.0.0-rc1

Can you test that please, or if that fails my latest git trunk?

Roy

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Roy Marples]

Post by Roy Marples
From the looks of it, this should already be fixed in -current where we
have dhcpcd-7.0.0-rc1
Can you test that please, or if that fails my latest git trunk?

I just tested your exact config (based on the output) and does appear to
be working fine for me on -current. Let me know otherwise!

Roy

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Benny Siegert]

I just tested your exact config (based on the output) and does appear to be
working fine for me on -current. Let me know otherwise!

Hm. Both dhcpd 6.11.5 and 7.0.0-rc1 fail at setting the two required
routes (host route to the gateway + default route via the gateway).

dhcpcd 6.11.5 (on -current from March) gets the following routing table:

Internet:
Destination Gateway Flags Refs Use Mtu Interface
10.240.0.1/32 link#2 UC - - 1460 vioif0
10.240.0.3 127.0.0.1 UGHS - - 1460 lo0
10.240.0.3/32 link#2 UC - - - vioif0
127/8 127.0.0.1 UGRS - - 33624 lo0
127.0.0.1 lo0 UH - - 33624 lo0

(Missing the default route. Adding "route add default 10.240.0.1 -ifp
vioif0" gets me working network.)

dhcpcd 7.0.0-rc1 on -8:

Internet:
Destination Gateway Flags Refs Use Mtu Interface
default 10.240.0.1 UGS - - 1460 vioif0
10.240.0.2 link#2 UHC - - 1460 vioif0
10.240.0.2/32 link#2 UC - - - vioif0
127/8 127.0.0.1 UGRS - - 33624 lo0
127.0.0.1 lo0 UHl - - 33624 lo0
255.255.255.255 link#2 UHCS - - 1460 vioif0

(Missing the host route. However, it looks like I can recreate it with
"route add 10.240.0.1/32 10.240.0.2 -ifp vioif0".)


--Benny.

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Benny Siegert]

Can you attach the dhcpcd log output with debug enabled please? You can use
the logfile option if capturing terminal output isn't viable.

Sure!

Sep 03 13:25:26 [116]: dhcpcd-7.0.0-rc1 starting
Sep 03 13:25:26 [116]: vioif0: disabling Kernel IPv6 auto link-local support
Sep 03 13:25:26 [116]: vioif0: disabling Kernel IPv6 RA support
Sep 03 13:25:26 [116]: vioif0: executing `/libexec/dhcpcd-run-hooks' PREINIT
Sep 03 13:25:26 [116]: vioif0: carrier acquired
Sep 03 13:25:26 [116]: vioif0: executing `/libexec/dhcpcd-run-hooks' CARRIER
Sep 03 13:25:26 [116]: DUID 00:01:00:01:21:2d:7a:15:52:54:00:12:34:56
Sep 03 13:25:26 [116]: vioif0: IAID 0a:f0:00:02
Sep 03 13:25:26 [116]: vioif0: adding address fe80::7a06:ce6a:2f4a:8e44
Sep 03 13:25:26 [116]: vioif0: pltime infinity, vltime infinity
Sep 03 13:25:26 [116]: vioif0: delaying IPv6 router solicitation for 1.0 seconds
Sep 03 13:25:26 [116]: vioif0: delaying IPv4 for 0.1 seconds
Sep 03 13:25:26 [116]: vioif0: reading lease `/var/db/dhcpcd/vioif0.lease'
Sep 03 13:25:26 [116]: vioif0: rebinding lease of 10.240.0.2
Sep 03 13:25:26 [116]: vioif0: sending REQUEST (xid 0x88e1ecb7), next
in 3.1 seconds
Sep 03 13:25:26 [116]: vioif0: acknowledged 10.240.0.2 from 169.254.169.254
Sep 03 13:25:26 [116]: vioif0: adding IP address 10.240.0.2/32
broadcast 10.240.0.2
Sep 03 13:25:27 [116]: vioif0: soliciting an IPv6 router
Sep 03 13:25:27 [116]: vioif0: delaying Router Solicitation for LL address
Sep 03 13:25:27 [116]: vioif0: sending Router Solicitation
Sep 03 13:25:31 [116]: vioif0: DAD completed for 10.240.0.2
Sep 03 13:25:31 [116]: vioif0: leased 10.240.0.2 for 86400 seconds
Sep 03 13:25:31 [116]: vioif0: renew in 43200 seconds, rebind in 75600 seconds
Sep 03 13:25:31 [116]: vioif0: writing lease `/var/db/dhcpcd/vioif0.lease'
Sep 03 13:25:31 [116]: vioif0: IP address 10.240.0.2/32 already exists
Sep 03 13:25:31 [116]: vioif0: using Classless Static Routes
Sep 03 13:25:31 [116]: vioif0: adding route to 10.240.0.2/32
Sep 03 13:25:31 [116]: vioif0: adding host route to 255.255.255.255
Sep 03 13:25:31 [116]: vioif0: adding default route via 10.240.0.1
Sep 03 13:25:31 [116]: vioif0: executing `/libexec/dhcpcd-run-hooks' BOUND
Sep 03 13:25:31 [116]: forking to background
Sep 03 13:25:31 [116]: forked to background, child pid 222
Sep 03 13:25:31 [222]: vioif0: sending Router Solicitation
Sep 03 13:25:35 [222]: vioif0: sending Router Solicitation
Sep 03 13:25:39 [222]: vioif0: sending Router Solicitation
Sep 03 13:25:39 [222]: vioif0: no IPv6 Routers available

# dhcpcd --dumplease vioif0
broadcast_address=10.240.0.2
classless_static_routes='10.240.0.1/32 0.0.0.0 0.0.0.0/0 10.240.0.1'
dhcp_lease_time=86400
dhcp_message_type=5
dhcp_server_identifier=169.254.169.254
domain_name=c.gobuilder-bsiegert.internal
domain_name_servers=169.254.169.254
domain_search='c.gobuilder-bsiegert.internal google.internal'
host_name=netbsd8-dhcptest.c.gobuilder-bsiegert.internal
interface_mtu=1460
ip_address=10.240.0.2
network_number=10.240.0.2
ntp_servers=169.254.169.254
routers=10.240.0.1
subnet_cidr=32
subnet_mask=255.255.255.255

I don't get why it creates a route to its own address, and to 255.255.255.255.

--Benny.

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Benny Siegert]

And this is the relevant bit from dhcpcd-6.11.5:

Sep 03 17:33:43 vioif0: using Classless Static Routes
Sep 03 17:33:43 vioif0: adding host route to 10.240.0.3 via 127.0.0.1
Sep 03 17:33:43 vioif0: adding host route to 10.240.0.1
Sep 03 17:33:43 vioif0: adding default route via 10.240.0.1
Sep 03 17:33:43 if_route (ADD): Network is unreachable

Note the local IP is different, otherwise the lease is the same.

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Roy Marples]

Post by Benny Siegert
Sep 03 13:25:31 [116]: vioif0: using Classless Static Routes
Sep 03 13:25:31 [116]: vioif0: adding route to 10.240.0.2/32
Sep 03 13:25:31 [116]: vioif0: adding host route to 255.255.255.255
Sep 03 13:25:31 [116]: vioif0: adding default route via 10.240.0.1
Sep 03 13:25:31 [116]: vioif0: executing `/libexec/dhcpcd-run-hooks' BOUND

This should already be fixed by this upstream commit:
https://dev.marples.name/rDHC510c760cb24ef4b1b16cc14732c5ffe0fbb0d96d

Could you apply that locally and re-test please?

Roy

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Benny Siegert]

Post by Roy Marples
https://dev.marples.name/rDHC510c760cb24ef4b1b16cc14732c5ffe0fbb0d96d
Could you apply that locally and re-test please?

This fixes it, thanks!

Is this pulled up to NetBSD-8 already? It does not look to be the case.

Alternatively: would you mind me just applying the tiny fix and
submitting a pullup request?

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Benny Siegert]

Post by Benny Siegert
Alternatively: would you mind me just applying the tiny fix and
submitting a pullup request?

I went ahead and did just that.

--Benny.

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Roy Marples]

Post by Benny Siegert

Post by Benny Siegert
Alternatively: would you mind me just applying the tiny fix and
submitting a pullup request?

I went ahead and did just that.

Thanks.
Hopefully I'll find the the time to solve a few last minute niggles
before merging a new dhcpcd version.

Roy


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de