PROPSAL: import Apple's mDNSResponder

[Ty Sarna]

I'd like to import Apple's mDNSResponder to the system. I think
zeroconf is becoming more basic functionality, and is desirable across
NetBSD's range of markets these days. Embedded devices and servers
want to advertise services this way, and desktops want to access them.

There seem to be three potential choices out there for mDNS these
days: Avahi, Apple's mDNSResponder, and the NetBSD SoC zeroconf.

Avahi is LGPL. mDNSResponder used to be APSL. Neither of those being
acceptable, I believe that led to the SoC project. That project is
still unfinished however, and in the meantime mDNSResponder has been
relicensed in a combination of Apache2 (server) and BSD 3-clause
(client library). As it is effectively the reference implementation,
is externally maintained, and used in at least two other OSes (OS X of
course, and it looks like OpenSolaris as well), it seems like the
clear choice to me.


So, I would like to proceed as follows. There are a number of open
questions or options.

1) Import mDNSResponder (currently 212.1, vs ancient 108 in pkgsrc) in
src/external/apache2/dist. It will need a prepare-import.sh script to
nuke the $Log$ lines in the sources (ugh!) and we can also prune out
the Windows, OS X, MacOS 9, vxWorks, etc. sources.

2) Add reachover makefile for mdnsd. (QUESTION: do we want to call it
mdnsd, which is how the supplied makefile builds it for "posix", or
mDNSResponder, which is what OS/X calls it? OpenSolaris went with mdnsd)

3) dnsetxd has yacc source that requires a feature not present in our
current yacc (YYPARSE_PARAM), so it will not be built for now.

4) OPTION: the pkg currently installs some debugging tools like
mDNSIdentify, mDNSClientPosix, etc. These don't ship in OS X. Do we
reachover these as well? My feeling is no -- get them from pkgsrc if
you need them.

5) Create a _mdnsd (_mdnsresponder?) user/group and get mdnsd to run
under it. The issue is writing the socket and pid file in /var/run,
which it doesn't have permissions to. Other than that it works fine.
My idea is to create /var/run/mdnsd, owned by _mdnsd, and change the
code to create the socket & pid file under there instead of in /var/run.

6) Create a rc.d script for it (QUESTION: does it default on or off?)

7) Supply client library functionality (QUESTION: separate libdns_sd,
or include in libc a'la OS X? OpenSolaris went separate.)

8) I would like to reachover for dns-sd(1) as well, but that one file
seems to still bear a APSL license. I will contact Apple about getting
that relicensed.

9) Write a new nss_mdns plugin. The current pkgsrc one is a wrapper
around one written for Linux, and it has issues. (OPTION: separate
nss_mdns.so plugin, or if we build the client functionality into libc,
do we build in the nss plugin too?) I have open questions about how
this integrates with regular DNS-SD (a'la PR 32373 -- see David
Young's message on tech-net from 20090303)


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Bernd Ernesti]

Post by Ty Sarna
I'd like to import Apple's mDNSResponder to the system. I think
zeroconf is becoming more basic functionality, and is desirable across
NetBSD's range of markets these days. Embedded devices and servers
want to advertise services this way, and desktops want to access them.

What does that do and how does that affect the NetBSD base programs?

Why is it not enough to use it from pkgsrc?

Bernd


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Ty Sarna]

Post by Bernd Ernesti
What does that do and how does that affect the NetBSD base programs?

I will not try to explain mDNS here, better to read up on it separately:

http://www.multicastdns.org/
http://www.dns-sd.org/
http://developer.apple.com/bonjour/

It does not affect base programs unless mdns is enabled as a
resolution method for hosts in /etc/nsswitch.conf, in which case
lookups in the .local domain will use mdns.

Post by Bernd Ernesti
Why is it not enough to use it from pkgsrc?

1) You could ask that about many things we ship. At some point, one
considers things basic functionality that should be included. For me,
this line has been crossed with mDNS. Especially w/ IPv6, I see it
being very useful -- very nice to be able to get to machines w/o
knowing IPs, especially when other services are down/wrong/etc.

2) "positive network externalities" Some things become useful in
exponential relationship with how widely they're deployed. Making it
trivial to deploy helps make it the service more useful.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Soren Jacobsen]

Post by Ty Sarna
I'd like to import Apple's mDNSResponder to the system. I think
zeroconf is becoming more basic functionality, and is desirable
across NetBSD's range of markets these days.

Violent agreement. I'd love to see mDNSResponder in NetBSD. Thanks
for looking into this!

Post by Ty Sarna
2) Add reachover makefile for mdnsd. (QUESTION: do we want to call
it mdnsd, which is how the supplied makefile builds it for "posix",
or mDNSResponder, which is what OS/X calls it? OpenSolaris went with
mdnsd)

My vote is for mdnsd.

Post by Ty Sarna
4) OPTION: the pkg currently installs some debugging tools like
mDNSIdentify, mDNSClientPosix, etc. These don't ship in OS X. Do we
reachover these as well? My feeling is no -- get them from pkgsrc if
you need them.

Agreed.

Post by Ty Sarna
6) Create a rc.d script for it (QUESTION: does it default on or off?)

Off.

Post by Ty Sarna
7) Supply client library functionality (QUESTION: separate
libdns_sd, or include in libc a'la OS X? OpenSolaris went separate.)

Probably separate.

Soren

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Tonnerre LOMBARD]

Salut,

Post by Ty Sarna
2) Add reachover makefile for mdnsd. (QUESTION: do we want to call
it mdnsd, which is how the supplied makefile builds it for "posix",
or mDNSResponder, which is what OS/X calls it? OpenSolaris went with
mdnsd)

+1 mdnsd

Tonnerre

[David Young]

Post by Ty Sarna
I'd like to import Apple's mDNSResponder to the system.

Yes, please.

Dave

[Steven M. Bellovin]

On Sun, 20 Sep 2009 15:53:49 -0400

Post by Ty Sarna
6) Create a rc.d script for it (QUESTION: does it default on or off?)

Almost certainly off. And if it listens on any INET-domain sockets,
*definitely* off -- that's policy for NetBSD default installations, I
believe.

--Steve Bellovin, http://www.cs.columbia.edu/~smb

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Daniel Carosone]

Embedded devices and servers want to
advertise services this way, and desktops want to access them.

Key questions not addressed/apparent in your proposal:

* What server and client applications in base would want to do this?
* How does base directly benefit?

I didn't see any steps in your list where this integration
was enabled for other things in base - just a client library,
presumably waiting to be used by other apps potentially to be added.

If that's it, I don't really see the integration benefit - those other
apps can just as well pull this in from pkgsrc at the same time as the
rest of whatever they need.

If there's more than that, then please show it to fully make a case
that shows how it integrates with (rather that just adds to) base.

--
Dan.

[David Arnold]

Post by Daniel Carosone

Embedded devices and servers want to
advertise services this way, and desktops want to access them.

* What server and client applications in base would want to do this?
* How does base directly benefit?
I didn't see any steps in your list where this integration
was enabled for other things in base - just a client library,
presumably waiting to be used by other apps potentially to be added.

mDNS is a means of resolving DNS queries in the .local domain, which
is implemented as a cooperatively managed name space by devices on a
LAN. If so configured (via NSS) all applications that use DNS to
resolve host names can benefit. So it's not the kind of thing that
packages would declare an explicit dependency upon, but more something
that the local LAN/host administrator might choose to enable.




d


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[der Mouse]

mDNS is a means of resolving DNS queries in the .local domain, [...]

whois.iana.org says no .local TLD exists. Given that, I think it is a
very bad idea to support anything that by default acts otherwise.

Mind you, I would (as if anyone cares) certainly support efforts to get
.local reserved for site-local uses like this. But until that happens,
I think we shouldn't do things like this.

/~\ The ASCII Mouse
\ / Ribbon Campaign
X Against HTML ***@rodents-montreal.org
/ \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Ty Sarna]

Post by Daniel Carosone
* What server and client applications in base would want to do this?

Potentially all of them. Any client benefits, if we have nss support.
True, nss support can be provided by pkgsrc. But, that's also true of
NIS and Hesiod, which we build in, and mDNS is of much greater general
interest than Hesiod, I think.

Likewise any server potentially benefits by being easier for clients
to find.

Zeroconf is also the kind of thing where once you find yourself in a
situation where you desperately need it, it may be too late to go
download the add-on (imagine if ping, traceroute, etc were moved to
pkgsrc)

Down the road we gain other potential benefits, such as sleep proxy
support (the core support is in the current release, but it will need
some fleshing out on NetBSD).

Post by Daniel Carosone
* How does base directly benefit?

See above, and also consider, how does base benefit from libossaudio?
OpenLDAP? httpd?

Post by Daniel Carosone
I didn't see any steps in your list where this integration
was enabled for other things in base - just a client library,

That comes later. There are a number of possible approaches to this,
and I'm not yet sure how we'd want to proceed there. I think we may
want to expirament and see what works for us there. But first we need
the foundation.

But, really, I think there's a great deal of value before we even get
to that point. Heck, just the server can be very useful, even with no
services registered and no nss or client support at all, just to be
able to find the machine from elsewhere.

Post by Daniel Carosone
If that's it, I don't really see the integration benefit - those other
apps can just as well pull this in from pkgsrc at the same time as the
rest of whatever they need.
If there's more than that, then please show it to fully make a case
that shows how it integrates with (rather that just adds to) base.

Pretty much anything that we offer that is infrastructure as opposed
to applications fails that test -- everything from /sbin/ping to X11.

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[der Mouse]

Post by Ty Sarna

Post by Daniel Carosone
* How does base directly benefit?

See above, and also consider, how does base benefit from libossaudio?
OpenLDAP? httpd?

Just because a mistake has been made once - or even three times - does
not justify, nor even uexcuse, making it again. (Whether this actually
_is_ such a case is a separate question. Just pointing out that this
argument is at least somewhat specious.)

As for my opinion on the matter at hand? I think it does not belong in
base. But I also think that about a lot of other stuff in base.

/~\ The ASCII Mouse
\ / Ribbon Campaign
X Against HTML ***@rodents-montreal.org
/ \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Miles Nordin]

dm> whois.iana.org says no .local TLD exists. Given that, I think
dm> it is a very bad idea

relax. there's an rfc for it.

dm> certainly support efforts to get .local reserved for
dm> site-local uses like this. But until that happens,

It's already happened, but it's reserved for mDNS, not for
``site-local'' use. Some sites are using .local as a bogus TLD in
normal bind-style DNS, and people all over the place are now advising
each other to pick another bogus TLD for this purpose because the
``rough consensus'' in the rfc and the ``running code'' in Mac OS X,
Ubuntu, Solaris have claimed .local for mDNS.

Post by Daniel Carosone
* What server and client applications in base would want to do this?

ts> Potentially all of them. Any client benefits, if we have nss
ts> support.

yeah, IMHO it'd be really great if, whatever it defaults to, while it
is ``on'' it works everywhere without modifying or relinking anything,
the way it does on Mac OS X.

AIUI both Mac OS X and Solaris have a caching non-recursive resolver,
so for example when you change DNS records, even if you are pointed
with resolv.conf to another server your client will not pick up the
changes until you type 'svcadm restart name-service-cache' or
'dnscacheutil -flushcache' to clear the old data. NetBSD obviously
hasn't got a caching resolver---I wonder if growing such a thing is
needed for good-neighbor transparent mDNS, or is everything you need
in the mDNSResponder server part.

[der Mouse]

Post by Miles Nordin

Post by der Mouse
whois.iana.org says no .local TLD exists. Given that, I think it is
a very bad idea

relax. there's an rfc for it.

Which one? I did a quick grep for .local and the only one that looked
promising was 2965, but (a) this is not HTTP state management we're
talking about and (b) 2965 even includes an IESG note warning about the
possibility of its breaking if a .local TLD is ever created. It's also
relatively old.

What did I miss?

Post by Miles Nordin
It's already happened, but it's reserved for mDNS, not for
``site-local'' use.

That strikes me as an almost worse idea, reserving .local for anything
other than local use. The implication of the meaning of the word is
just too strong.

Especially...

Post by Miles Nordin
Some sites are using .local as a bogus TLD in normal bind-style DNS,
and people all over the place are now advising each other to pick
another bogus TLD for this purpose because the ``rough consensus'' in
the rfc and the ``running code'' in Mac OS X, Ubuntu, Solaris have
claimed .local for mDNS.

...using .local by default when turning mDNS on is reasonable.
Reserving .local for that use is not, especially since - like anything
else site-internal - it's unenforceable and undetectable from the
outside.

Not that anyone cares about sanity of domain name use any longer, ever
since .com stopped meaning "commercial", .net "network provider", etc.
Nobody except a few bitter curmudgeonly idealists like me.

/~\ The ASCII Mouse
\ / Ribbon Campaign
X Against HTML ***@rodents-montreal.org
/ \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Steven M. Bellovin]

On Sun, 20 Sep 2009 23:28:04 -0400 (EDT)

Post by der Mouse

Post by Miles Nordin

Post by der Mouse
whois.iana.org says no .local TLD exists. Given that, I think it
is a very bad idea

relax. there's an rfc for it.

Which one? I did a quick grep for .local and the only one that looked
promising was 2965, but (a) this is not HTTP state management we're
talking about and (b) 2965 even includes an IESG note warning about
the possibility of its breaking if a .local TLD is ever created.
It's also relatively old.

I thought he was talking about 4795, but even that doesn't talk
about .local. If memory serves, the issue of .local came up when I was
on the IESG. We didn't like it...


--Steve Bellovin, http://www.cs.columbia.edu/~smb

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Alan Barrett]

Post by Ty Sarna
9) Write a new nss_mdns plugin. The current pkgsrc one is a wrapper
around one written for Linux, and it has issues. (OPTION: separate
nss_mdns.so plugin, or if we build the client functionality into
libc, do we build in the nss plugin too?) I have open questions
about how this integrates with regular DNS-SD (a'la PR 32373 -- see
David Young's message on tech-net from 20090303)

Is this step intended to imply "make mdns usable as a backend for
gethostbyname(3), via nsdispatch(3) and nsswitch.conf(5)"?

--apb (Alan Barrett)

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Ty Sarna]

Post by Alan Barrett
Is this step intended to imply "make mdns usable as a backend for
gethostbyname(3), via nsdispatch(3) and nsswitch.conf(5)"?

Yes, exactly.

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Ty Sarna]

I'm planning to do the first round of commits next week. I should note
that mDNS was already on the 6.0 roadmap, so I consider the question
of wether we want the functionality settled. Nobody objected to the
plan for getting there, so I'm moving forward.

Updates:

- It should be noted that dhcpcd implements RFC3927 for IPv4 Link-
Local addresses, so we already have the other half of zeroconf covered.

- the daemon will be called mdnsd, and will be off by default. It will
run under user _mdnsd.

- there will be a MKMDNS=no option for the luddites ;-)

- client library will be separate (libdns_sd). Although a libc-built-
in client and nss implementation would make it possible to share some
code and avoid duplication of functionality, I didn't see a strong
enough justification for including it in libc right now (and it's a
lot easier to move code into libc later than out of it)

- On closer inspection the dns-sd(1) license is not APSL but a
proprietary license that seems equivalent to BSD 3-clause, while
managing to avoid being OSI-recognized by being mostly different text.
I don't think this is a barrier for NetBSD inclusion so I now plan to
ship dns-sd(1), but I have enquired about relicensing this to standard
BSD 3-clause to avoid confusion and to make life easier on our adopters.

- I won't be shipping the other debugging/demo apps.


I have been writing a new nss_mdns nameswitch plugin (eg "hosts: files
mdns dns" in /etc/nsswitch.conf). It's now fully functional and
already works better than the pkgsrc one in one important way (handles
scoped v6 link-local addresses correctly).

However, there are a few tweaks to do and I want to add a new function
to libc to make writing getaddrinfo plugins a little more sane, so I
expect to do that in a later commit. There will be no DNS-SD
functionality in nss_mdns, any more than there is in the current dns
module. That needs to be addressed as a separate project (likely
involves changing the ABI of getaddrinfo plugins). I may look at that
later, but it's not directly part of this project.

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Jason Thorpe]

Post by Miles Nordin
AIUI both Mac OS X and Solaris have a caching non-recursive resolver,
so for example when you change DNS records, even if you are pointed
with resolv.conf to another server your client will not pick up the
changes until you type 'svcadm restart name-service-cache' or
'dnscacheutil -flushcache' to clear the old data. NetBSD obviously
hasn't got a caching resolver---I wonder if growing such a thing is
needed for good-neighbor transparent mDNS, or is everything you need
in the mDNSResponder server part.

In Snow Leopard, mDNSResponder does all of the DNS lookups for the
entire system (it is no longer done in DirectoryService, as in prior
releases).

I personally would love to see it behave the same in NetBSD ... system-
wide caching for DNS lookups is a good thing.

-- thorpej


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Jason Thorpe]

Post by Ty Sarna
- It should be noted that dhcpcd implements RFC3927 for IPv4 Link-
Local addresses, so we already have the other half of zeroconf
covered.

Cool, that part I meant to ask about (I know it was part of the GSoC
zeroconf project). I'm pretty sure that configd does this bit in Mac
OS X (and thus it is not in mDNSResponder).

-- thorpej


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de