Userland API for threaded programs to reload /etc/resolv.conf

[Martin Husemann]

Hey folks,

assume you have a multi threaded, long running application. The app
does regular DNS queries using the high level APIs (getaddrinfo(3) and
friends) and uses a pool of threads specifically for those lookups.

Every now and then it wants to make sure that /etc/resolv.conf changes get
noticed and used by the resolver.

On FreeBSD and Linux each thread has a local _res variable, so the app
simply does:

res_ninit(&_res);

Scream, tear, feather! This is mixing the ancient (_res) API with the new
thread safe low level resolver API. But it is simple and works. Especially
given that there are NO other calls to the low level API.

Now on NetBSD this gets tricky. We cause an assertion failure when _res is
accessed from threaded programs.

The way to fix this with existing means seems to be: make the timeout
variable global and whenever one resolve threads hits the timeout, make
it reset the global variable atomically and then simply do:

res_init();

Now looking back at the whole picture, it seems that this all should
not be necessary at all. IMHO the resolver library should deal with it
internally, have a (application settable) timeout and after the timeout
expired stat /etc/resolv.conf on next access - and internally deal.

An alternative would be to have a full set of getaddrinfo_r(3),
getnameinfo_r(3), gethostbyaddr_r(3), gethostbyname_r(3), which get
passed a res_state*, so the resolver threads could just use a local
res_state and do res_init() on that whenever they like.

Did I overlook other options? Is there any precedence for the internal
solution (automatic res_ninit on changes of /etc/resolv.conf) in other
systems? Comments?


Thanks,

Martin

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Christos Zoulas]

Post by Martin Husemann
Hey folks,
assume you have a multi threaded, long running application. The app
does regular DNS queries using the high level APIs (getaddrinfo(3) and
friends) and uses a pool of threads specifically for those lookups.
Every now and then it wants to make sure that /etc/resolv.conf changes get
noticed and used by the resolver.

This is not the job of the application. Our resolver handles this just fine.
It uses a kqueue to register for file update events to /etc/resolv.conf,
and updates itself internally when needed.

Post by Martin Husemann
On FreeBSD and Linux each thread has a local _res variable, so the app
res_ninit(&_res);
Scream, tear, feather! This is mixing the ancient (_res) API with the new
thread safe low level resolver API. But it is simple and works. Especially
given that there are NO other calls to the low level API.

Yes, this is what we need; more magic.

Post by Martin Husemann
Now on NetBSD this gets tricky. We cause an assertion failure when _res is
accessed from threaded programs.
The way to fix this with existing means seems to be: make the timeout
variable global and whenever one resolve threads hits the timeout, make
res_init();

This will not work in a threaded environment and is a step backwards.

Post by Martin Husemann
Now looking back at the whole picture, it seems that this all should
not be necessary at all. IMHO the resolver library should deal with it
internally, have a (application settable) timeout and after the timeout
expired stat /etc/resolv.conf on next access - and internally deal.
An alternative would be to have a full set of getaddrinfo_r(3),
getnameinfo_r(3), gethostbyaddr_r(3), gethostbyname_r(3), which get
passed a res_state*, so the resolver threads could just use a local
res_state and do res_init() on that whenever they like.
Did I overlook other options? Is there any precedence for the internal
solution (automatic res_ninit on changes of /etc/resolv.conf) in other
systems? Comments?

You don't neeed getaddrinfo_r and getnameinfo_r. The getaddrinfo and
getnameinfo functions are thread-safe since they create and destroy
their own res internally. Avoid using gethostbyname_r and gethostbyaddr_r
since those are non portable.

If you just use getaddrinfo() and getnameinfo() on NetBSD everything
will work just fine.

christos


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Martin Husemann]

Post by Christos Zoulas
This is not the job of the application. Our resolver handles this just fine.
It uses a kqueue to register for file update events to /etc/resolv.conf,
and updates itself internally when needed.

Cool, didn't know that - this is exactly what I described later, even more
cool that it already is in place.

Post by Christos Zoulas
This will not work in a threaded environment and is a step backwards.

If it does not work, maybe we should add similar traps as for _res?

Post by Christos Zoulas
If you just use getaddrinfo() and getnameinfo() on NetBSD everything
will work just fine.

This is realy great - as the application in question now is forced (via
stupid autoconfig tricks) to believe that we have no res_ninit(), and all
the calls are #ifdef'd out.

So I'll just try to push some comments upstream explaining this and leave
everything as-is.

Is this documented in some man pages?

Martin

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Michael van Elst]

Post by Martin Husemann

Post by Christos Zoulas
This is not the job of the application. Our resolver handles this just fine.
It uses a kqueue to register for file update events to /etc/resolv.conf,
and updates itself internally when needed.

Cool, didn't know that - this is exactly what I described later, even more
cool that it already is in place.

See

http://mail-index.netbsd.org/tech-userlevel/2003/12/28/0009.html

for details and all possible arguments.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Christos Zoulas]

Post by Martin Husemann

Post by Christos Zoulas
This is not the job of the application. Our resolver handles this just fine.
It uses a kqueue to register for file update events to /etc/resolv.conf,
and updates itself internally when needed.

Cool, didn't know that - this is exactly what I described later, even more
cool that it already is in place.

Post by Christos Zoulas
This will not work in a threaded environment and is a step backwards.

If it does not work, maybe we should add similar traps as for _res?

It is easy to do so; I am just afraid on how many broken applications we'll have to fix!

Post by Martin Husemann

Post by Christos Zoulas
If you just use getaddrinfo() and getnameinfo() on NetBSD everything
will work just fine.

This is realy great - as the application in question now is forced (via
stupid autoconfig tricks) to believe that we have no res_ninit(), and all
the calls are #ifdef'd out.
So I'll just try to push some comments upstream explaining this and leave
everything as-is.

Well what should be done probably is to use just 2 entry points in Firefox (call them ff_getaddrinfo() and ff_getnameinfo() for example) and put all the ugliness in there for the ones who need it.

Post by Martin Husemann
Is this documented in some man pages?

I will add something later today if it is not there already.

christos
--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Martin Husemann]

Post by Christos Zoulas
Well what should be done probably is to use just 2 entry points in
Firefox (call them ff_getaddrinfo() and ff_getnameinfo() for example)
and put all the ugliness in there for the ones who need it.

Nothing needed, it is all fine!

As far as pkgsrc goes, firefox does the right thing already, and the
needed changes are about to land upstream in a slightly modified
version (which is why I ran into this right now).

Post by Christos Zoulas
I will add something later today if it is not there already.

Thanks!

Martin

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de