Discussion:
routing socket rouding?
(too old to reply)
Mouse
2012-02-18 05:20:36 UTC
Permalink
I recently had occasion to figure out the routing socket interface.

There is a macro, historically ROUNDUP defined in various .c files,
RT_ROUNDUP in <net/route.h> in 5.1 and presumably later, which appears
to be designed around the undocumented (even in comments) assumption
that sizeof(long) is a power of two.

So, I'm curious: does anyone know what this macro turned into on a
system with sizeof(long) not a power of two but with anything like the
BSD routing socket, if such a system exists?

/~\ The ASCII Mouse
\ / Ribbon Campaign
X Against HTML ***@rodents-montreal.org
/ \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Mouse
2012-02-18 08:35:18 UTC
Permalink
Post by Mouse
There is a macro, historically ROUNDUP defined in various .c files,
RT_ROUNDUP in <net/route.h> in 5.1 and presumably later, [...]
So, I'm curious: does anyone know what this macro turned into on a
system with sizeof(long) not a power of two but with anything like
the BSD routing socket, if such a system exists?
ROUNDUP (or ROUND_UP) macro was replaced by ALIGN on some platforms,
http://lkml.org/lkml/2007/4/1/50
(a) That appears to be dealing with PCI support, not routing socket
support. It's the routing socket macro I'm curious about. (The
ROUND_UP macro being replaced does appear to assume the grain size
being rounded up to is a pwoer of two, so it's related, but since it
doesn't show the definition of the ALIGN it's being replaced with, the
message is pretty much content-free for my purposes.)

(b) The ROUND_UP macro affected by that patch is not the one used by
the routing socket code; it returns zero when passed zero, whereas the
routing socket's ROUNDUP / RT_ROUNDUP returns sizeof(long) when passed
zero.

(c) Your citing this in response to my question implies that Linux runs
on something with sizeof(long) not a power of two. Now I'm also
curious what CPU (and/or machine) that is.
and it appears to assume 2's compliment arithmetic....
Well, two's complement (not "compliment") versus alternatives is
relevant only when dealing with negative values. I don't think
anything in the routing socket code can ever pass a negative value to
ROUNDUP/RT_ROUNDUP; they mostly are sa_len values, and sa_len is
unsigned.

/~\ The ASCII Mouse
\ / Ribbon Campaign
X Against HTML ***@rodents-montreal.org
/ \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Chuck Swiger
2012-02-18 17:27:41 UTC
Permalink
Post by Mouse
Post by Mouse
There is a macro, historically ROUNDUP defined in various .c files,
RT_ROUNDUP in <net/route.h> in 5.1 and presumably later, [...]
So, I'm curious: does anyone know what this macro turned into on a
system with sizeof(long) not a power of two but with anything like
the BSD routing socket, if such a system exists?
http://lkml.org/lkml/2007/4/1/50
(a) That appears to be dealing with PCI support, not routing socket
support. It's the routing socket macro I'm curious about. (The
ROUND_UP macro being replaced does appear to assume the grain size
being rounded up to is a pwoer of two, so it's related, but since it
doesn't show the definition of the ALIGN it's being replaced with, the
message is pretty much content-free for my purposes.)
If you search for both macro names, you'll discover that this Milind Choudhary made a sweep through the Linux kernel, including PCI support, network drivers, and so forth. If the "not invented here" syndrome or whatever other reason means that you don't want to consider a change made in Linux, you'll even discover that it's used in APR-- see APR_ALIGN in include/apr_general.h:

#define APR_ALIGN(size, boundary) (((size) + ((boundary) - 1)) & ~((boundary) - 1))
Post by Mouse
(b) The ROUND_UP macro affected by that patch is not the one used by
the routing socket code; it returns zero when passed zero, whereas the
routing socket's ROUNDUP / RT_ROUNDUP returns sizeof(long) when passed
zero.
I have reason to believe the NetBSD macro resembles:

#define RT_ROUNDUP2(a, n) ((a) > 0 ? (1 + (((a) - 1) | ((n) - 1))) : (n))
#define RT_ROUNDUP(a) RT_ROUNDUP2((a), sizeof(uint64_t))

Passing a=0 to RT_ROUNDUP2 is already aligned, but the ternary ?: causes it to return the "n" or alignment boundary, you get sizeof(uint64_t) back. Whether that's the same as sizeof(long) depends on whether you are running an (I)LP64 platform or not, because RT_ROUNDUP always passes n=8 to RT_ROUNDUP2:

% cat m.c
#define APR_ALIGN(size, boundary) (((size) + ((boundary) - 1)) & ~((boundary) - 1))

#define RT_ROUNDUP2(a, n) ((a) > 0 ? (1 + (((a) - 1) | ((n) - 1))) : (n))
#define RT_ROUNDUP(a) RT_ROUNDUP2((a), sizeof(uint64_t))

#include <stdio.h>

int main() {
printf("align 0, sizeof(int): %ld\n", APR_ALIGN(0, sizeof(int)));
printf("align 0, sizeof(long): %ld\n", APR_ALIGN(0, sizeof(long)));

printf("align 2, sizeof(int): %ld\n", APR_ALIGN(2, sizeof(int)));
printf("align 2, sizeof(long): %ld\n", APR_ALIGN(2, sizeof(long)));

printf("align 4, sizeof(int): %ld\n", APR_ALIGN(4, sizeof(int)));
printf("align 4, sizeof(long): %ld\n", APR_ALIGN(4, sizeof(long)));

printf("align 8, sizeof(int): %ld\n", APR_ALIGN(8, sizeof(int)));
printf("align 8, sizeof(long): %ld\n", APR_ALIGN(8, sizeof(long)));

printf("rt_roundup2 0, sizeof(int): %ld\n", RT_ROUNDUP2(0, sizeof(int)));
printf("rt_roundup2 0, sizeof(long): %ld\n", RT_ROUNDUP2(0, sizeof(long)));

printf("rt_roundup2 2, sizeof(int): %ld\n", RT_ROUNDUP2(2, sizeof(int)));
printf("rt_roundup2 2, sizeof(long): %ld\n", RT_ROUNDUP2(2, sizeof(long)));

printf("rt_roundup2 4, sizeof(int): %ld\n", RT_ROUNDUP2(4, sizeof(int)));
printf("rt_roundup2 4, sizeof(long): %ld\n", RT_ROUNDUP2(4, sizeof(long)));

printf("rt_roundup2 8, sizeof(int): %ld\n", RT_ROUNDUP2(8, sizeof(int)));
printf("rt_roundup2 8, sizeof(long): %ld\n", RT_ROUNDUP2(8, sizeof(long)));
}

% cc -o m m.c && ./m
align 0, sizeof(int): 0
align 0, sizeof(long): 0
align 2, sizeof(int): 4
align 2, sizeof(long): 8
align 4, sizeof(int): 4
align 4, sizeof(long): 8
align 8, sizeof(int): 8
align 8, sizeof(long): 8
rt_roundup2 0, sizeof(int): 4
rt_roundup2 0, sizeof(long): 8
rt_roundup2 2, sizeof(int): 4
rt_roundup2 2, sizeof(long): 8
rt_roundup2 4, sizeof(int): 4
rt_roundup2 4, sizeof(long): 8
rt_roundup2 8, sizeof(int): 8
rt_roundup2 8, sizeof(long): 8

% cc -m32 -o m m.c && ./m
align 0, sizeof(int): 0
align 0, sizeof(long): 0
align 2, sizeof(int): 4
align 2, sizeof(long): 4
align 4, sizeof(int): 4
align 4, sizeof(long): 4
align 8, sizeof(int): 8
align 8, sizeof(long): 8
rt_roundup2 0, sizeof(int): 4
rt_roundup2 0, sizeof(long): 4
rt_roundup2 2, sizeof(int): 4
rt_roundup2 2, sizeof(long): 4
rt_roundup2 4, sizeof(int): 4
rt_roundup2 4, sizeof(long): 4
rt_roundup2 8, sizeof(int): 8
rt_roundup2 8, sizeof(long): 8
Post by Mouse
(c) Your citing this in response to my question implies that Linux runs
on something with sizeof(long) not a power of two. Now I'm also
curious what CPU (and/or machine) that is.
A DEC PDP-7 had an 18-bit word size, and the PDP-10 had a 36-bit word size; I'm not sure you'll find anything much newer than that.

However, my reason for citing the above is that many variants of ROUNDUP / ROUND_UP are buggy, and reasoning about an address or size that might be rounded up or not, depending on whether it is already aligned (or not), or zero, etc leads to problems. (You also get problems when code using it moves from ILP32 to LP64, which is why I'm familiar with this macro and variants.)
Post by Mouse
and it appears to assume 2's compliment arithmetic....
Well, two's complement (not "compliment") versus alternatives is
relevant only when dealing with negative values.
True, and "routing socket rouding?" should be "routing socket rounding?"

Many folks are more apt to fault others than they are to fault themselves for the same error.

Regards,
--
-Chuck


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Mouse
2012-02-18 18:49:21 UTC
Permalink
Post by Chuck Swiger
Post by Mouse
http://lkml.org/lkml/2007/4/1/50
(a) That appears to be dealing with PCI support, not routing socket
support. It's the routing socket macro I'm curious about.
If you search for both macro names, you'll discover that this Milind
Choudhary made a sweep through the Linux kernel, including PCI
support, network drivers, and so forth. If the "not invented here"
syndrome or whatever other reason means that you don't want to
consider a change made in Linux, you'll even discover that it's used
No, it's not that I'm uninterested in things Linux did; if Linux has
had to address this issue, I'm interested in what they did. But you
gave no reason to think that the message you pointed at had anything to
do with routing sockets. Also, the macros you cite still depend on the
relevant sizeof() to return a power of two.

As for "search[ing] for both macro names", I have no convenient way to
do that; I do not have Linux source on hand.
Post by Chuck Swiger
Post by Mouse
(c) Your citing this in response to my question implies that Linux
runs on something with sizeof(long) not a power of two. Now I'm
also curious what CPU (and/or machine) that is.
A DEC PDP-7 had an 18-bit word size, and the PDP-10 had a 36-bit word
size; I'm not sure you'll find anything much newer than that.
What do those have to do with sizeof(long)? I didn't say, number of
bits in a long not a power of two,, nor even size of a long etc; I said
sizeof(long), because that's what was relevant - the macro uses (in all
the variants I've personally seen) sizeof(long) as the basis for its
operations.

You write that the NetBSD macro resembles

#define RT_ROUNDUP2(a, n) ((a) > 0 ? (1 + (((a) - 1) | ((n) - 1))) : (n))
#define RT_ROUNDUP(a) RT_ROUNDUP2((a), sizeof(uint64_t))

which is basically the same thing, except it uses uint64_t; presumably
this refers to NetBSD newer than anything I have at hand (I haven't
tracked NetBSD source at all since the GPLv3 was accepted). This is
technically still at risk, since, even though all divisors of 64 are
powers of two, uint64_t may include padding bits - for example, on a
22-bit machine, uint64_t could have 64 significant bits and 2 padding
bits and thus (depending on the implementors' choice for char) have
sizeof(uint64_t) equal something like 6. (Or not provide uint64_t at
all, since 64 bits is not a native size; that at least would fail to
compile rather than just silently producing weird rounding.)
Post by Chuck Swiger
However, my reason for citing the above is that many variants of
ROUNDUP / ROUND_UP are buggy,
Including all the ones cited so far, on any architecture where the
relevant sizeof() is not a power of two. (Well, possibly-buggy. The
RT_ROUNDUP2() above does round up, even when n is not a power of two;
it just does so rather peculiarly and not very usefully.) That's why I
was wondering whether any such system with anything like the BSD
routing socket existed, and, if so, what choice its designers had made
for dealing with the issue.
Post by Chuck Swiger
Post by Mouse
and it appears to assume 2's compliment arithmetic....
Well, two's complement (not "compliment")
True, and "routing socket rouding?" should be "routing socket rounding?"
Hoist by my own petard! Well spotted.

/~\ The ASCII Mouse
\ / Ribbon Campaign
X Against HTML ***@rodents-montreal.org
/ \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Mouse
2012-02-18 23:57:00 UTC
Permalink
Post by Chuck Swiger
#define RT_ROUNDUP2(a, n) ((a) > 0 ? (1 + (((a) - 1) | ((n) - 1))) : (n))
#define RT_ROUNDUP(a) RT_ROUNDUP2((a), sizeof(uint64_t))
Those macros [were] changed for NetBSD-6 so that the routing messages
are identical for 32-bit and 64-bit kernels.
...at the price of breaking (formerly) correct code dating from the
days before the macros made it into the include files.

This is ugly, on a par with the SCM_RIGHTS flag day. A code author's
choices are to forego backward compatability, forego forward
compatability, or write two versions, one for old kernels and one for
new. ABI compatability is all very well, but there are at least a few
of us out here who care about API compatability too.

/~\ The ASCII Mouse
\ / Ribbon Campaign
X Against HTML ***@rodents-montreal.org
/ \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Matt Thomas
2012-02-19 00:15:06 UTC
Permalink
Post by Mouse
Post by Chuck Swiger
#define RT_ROUNDUP2(a, n) ((a) > 0 ? (1 + (((a) - 1) | ((n) - 1))) : (n))
#define RT_ROUNDUP(a) RT_ROUNDUP2((a), sizeof(uint64_t))
Those macros [were] changed for NetBSD-6 so that the routing messages
are identical for 32-bit and 64-bit kernels.
...at the price of breaking (formerly) correct code dating from the
days before the macros made it into the include files.
This is ugly, on a par with the SCM_RIGHTS flag day. A code author's
choices are to forego backward compatability, forego forward
compatability, or write two versions, one for old kernels and one for
new. ABI compatability is all very well, but there are at least a few
of us out here who care about API compatability too.
And why do you think old binaries don't continue to work? If you bothered
to look at the current sources, you'd see that there compat support for the
old routing socket. Only newly compiled sources will use the new code.



--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Mouse
2012-02-19 00:38:55 UTC
Permalink
Post by Matt Thomas
Post by Mouse
This is ugly, on a par with the SCM_RIGHTS flag day. A code
author's choices are to forego backward compatability, forego
forward compatability, or write two versions, one for old kernels
and one for new. ABI compatability is all very well, but there are
at least a few of us out here who care about API compatability too.
And why do you think old binaries don't continue to work?
My remarks have nothing to do with old binaries; I was talking about
old source code. Old source ("dating from the days before the macros
made it into the include files", as I put it), compiled on a new
system, will explode (if it uses IPv6 or possibly others, since IPv4
sockaddrs have zero bytes of round-up space under either scheme).
Hence my remarks about API compatability.

Perhaps fortunately, in this case it's easier to patch the source so it
works on both systems than it was in the SCM_RIGHTS case.
Post by Matt Thomas
If you bothered to look at the current sources, you'd see [...]
If you bothered to read what I wrote, instead of...I don't know what,
reacting to what you think I might be saying?...
Post by Matt Thomas
Only newly compiled sources will use the new code.
And that only if they are written to the "use RT_ROUNDUP" paradigm
(whence the "before the macros...").

/~\ The ASCII Mouse
\ / Ribbon Campaign
X Against HTML ***@rodents-montreal.org
/ \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Matt Thomas
2012-02-18 21:50:27 UTC
Permalink
Post by Mouse
Post by Chuck Swiger
Post by Mouse
http://lkml.org/lkml/2007/4/1/50
(a) That appears to be dealing with PCI support, not routing socket
support. It's the routing socket macro I'm curious about.
If you search for both macro names, you'll discover that this Milind
Choudhary made a sweep through the Linux kernel, including PCI
support, network drivers, and so forth. If the "not invented here"
syndrome or whatever other reason means that you don't want to
consider a change made in Linux, you'll even discover that it's used
No, it's not that I'm uninterested in things Linux did; if Linux has
had to address this issue, I'm interested in what they did. But you
gave no reason to think that the message you pointed at had anything to
do with routing sockets. Also, the macros you cite still depend on the
relevant sizeof() to return a power of two.
As for "search[ing] for both macro names", I have no convenient way to
do that; I do not have Linux source on hand.
Post by Chuck Swiger
Post by Mouse
(c) Your citing this in response to my question implies that Linux
runs on something with sizeof(long) not a power of two. Now I'm
also curious what CPU (and/or machine) that is.
A DEC PDP-7 had an 18-bit word size, and the PDP-10 had a 36-bit word
size; I'm not sure you'll find anything much newer than that.
What do those have to do with sizeof(long)? I didn't say, number of
bits in a long not a power of two,, nor even size of a long etc; I said
sizeof(long), because that's what was relevant - the macro uses (in all
the variants I've personally seen) sizeof(long) as the basis for its
operations.
You write that the NetBSD macro resembles
#define RT_ROUNDUP2(a, n) ((a) > 0 ? (1 + (((a) - 1) | ((n) - 1))) : (n))
#define RT_ROUNDUP(a) RT_ROUNDUP2((a), sizeof(uint64_t))
which is basically the same thing, except it uses uint64_t; presumably
this refers to NetBSD newer than anything I have at hand (I haven't
tracked NetBSD source at all since the GPLv3 was accepted). This is
technically still at risk, since, even though all divisors of 64 are
powers of two, uint64_t may include padding bits - for example, on a
22-bit machine, uint64_t could have 64 significant bits and 2 padding
bits and thus (depending on the implementors' choice for char) have
sizeof(uint64_t) equal something like 6. (Or not provide uint64_t at
all, since 64 bits is not a native size; that at least would fail to
compile rather than just silently producing weird rounding.)
Those macros where changed for NetBSD-6 so that the routing messages
are identical for 32-bit and 64-bit kernels. I think targeting NetBSD with
non-power-of-2 entities is a waste of time. If and when such a machine
comes along, we can adapt to it. But I don't think it's going to be any time soon
--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Chuck Swiger
2012-02-18 20:14:58 UTC
Permalink
Post by Mouse
As for "search[ing] for both macro names", I have no convenient way to
do that; I do not have Linux source on hand.
Does:

http://fxr.watson.org/fxr/ident?i=ROUNDUP,ALIGN
http://fxr.watson.org/fxr/trackident?i=ROUNDUP
http://fxr.watson.org/fxr/trackident?i=ALIGN

...help?
Post by Mouse
Post by Chuck Swiger
Post by Mouse
(c) Your citing this in response to my question implies that Linux
runs on something with sizeof(long) not a power of two. Now I'm
also curious what CPU (and/or machine) that is.
A DEC PDP-7 had an 18-bit word size, and the PDP-10 had a 36-bit word
size; I'm not sure you'll find anything much newer than that.
What do those have to do with sizeof(long)?
I don't have a PDP-7 or -10 to test on, and I seem to recall that they had 9-bit chars, so maybe sizeof(long) on them would be 2 and 4, rather than 3 and 5...but it was the closest relevant thing which came to mind.
Post by Mouse
You write that the NetBSD macro resembles
#define RT_ROUNDUP2(a, n) ((a) > 0 ? (1 + (((a) - 1) | ((n) - 1))) : (n))
#define RT_ROUNDUP(a) RT_ROUNDUP2((a), sizeof(uint64_t))
which is basically the same thing, except it uses uint64_t; presumably
this refers to NetBSD newer than anything I have at hand (I haven't
tracked NetBSD source at all since the GPLv3 was accepted).
I took it from:

http://fxr.watson.org/fxr/source/net/route.h?v=NETBSD#L270
Post by Mouse
This is technically still at risk, since, even though all divisors of 64 are
powers of two, uint64_t may include padding bits - for example, on a
22-bit machine, uint64_t could have 64 significant bits and 2 padding
bits and thus (depending on the implementors' choice for char) have
sizeof(uint64_t) equal something like 6. (Or not provide uint64_t at
all, since 64 bits is not a native size; that at least would fail to
compile rather than just silently producing weird rounding.)
C99 says that something which provides a typedef for int64_t / uint64_t needs to guarantee exactly that 64 bits are available, with no padding, and that the signed type uses 2's complement representation. This implies that a PDP-7 or -10 would not have uint64_t available, so I agree with your point that it ought to fail to compile on such a platform.
Post by Mouse
Post by Chuck Swiger
However, my reason for citing the above is that many variants of
ROUNDUP / ROUND_UP are buggy,
Including all the ones cited so far, on any architecture where the
relevant sizeof() is not a power of two. (Well, possibly-buggy. The
RT_ROUNDUP2() above does round up, even when n is not a power of two;
it just does so rather peculiarly and not very usefully.) That's why I
was wondering whether any such system with anything like the BSD
routing socket existed, and, if so, what choice its designers had made
for dealing with the issue.
FreeBSD 4 used very nearly the same macros in rt_xaddrs(), rt_msg1(), and rt_msg2(); and that was rewritten in FreeBSD 5:

http://fxr.watson.org/fxr/source/net/rtsock.c?v=FREEBSD4#L520
http://fxr.watson.org/fxr/source/net/rtsock.c?v=FREEBSD5#L617

Regards,
--
-Chuck


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Chuck Swiger
2012-02-18 06:04:29 UTC
Permalink
Post by Mouse
There is a macro, historically ROUNDUP defined in various .c files,
RT_ROUNDUP in <net/route.h> in 5.1 and presumably later, which appears
to be designed around the undocumented (even in comments) assumption
that sizeof(long) is a power of two.
So, I'm curious: does anyone know what this macro turned into on a
system with sizeof(long) not a power of two but with anything like the
BSD routing socket, if such a system exists?
ROUNDUP (or ROUND_UP) macro was replaced by ALIGN on some platforms, ie:

http://lkml.org/lkml/2007/4/1/50

There are buggy variants of it which don't work right even on LP64 platforms,
and it appears to assume 2's compliment arithmetic....

Regards,
--
-Chuck


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Loading...