packet filters for NetBSD in the future

[Thor Simon]

I am curious about the future direction of NetBSD with regard to packet
filtering. Here is why:

1) An intention to replace ipfilter with pf has been announced, but:
* This appears to be largely on the ground that ipfilter is
unmaintained in our tree, and

* pf is also currently unmaintained in our tree.

2) pf is considerably simpler than ipfilter (mostly, it seems, because it
is much newer and has less historical portability baggage).

3) We have third and fourth filter engines in our tree, the ipsec security
policy engine and the altq classification engine. This is clearly
redundant both in the source tree and at runtime (at runtime, doing
PFIL_HOOKS filters and then two other kinds of filtering can slow
things down considerably for little net gain).

* Itojun's intent was to unify the ipsec and altq filtering
using pf as the engine. This is a large part of why pf is
currently in our tree, as I understand it. Nobody has ever
done the analogous work to let ipfilter serve this role.

4) pf is essentially subsystem-locked by the softnet lock. On the
other hand, ipfilter has much finer-grained locking because it can
run in other kernels which have pushed locks down into the network
stack.

Given this set of facts, it's not clear to me what the right path
forward is. #4 seems likely to be a considerable inconvenience in the
future if we go with pf. On the other hand, ipfilter's author is a
NetBSD developer, but even he doesn't have time to maintain the version
in the NetBSD source tree (as far as I can tell) and it seems unlikely
that he or anyone else will step forward to address #3.

Still nobody has stepped forward for the comparatively simple task of
updating and maintaining pf, either.

What are others' opinions on this?

[Quentin Garnier]

Post by Thor Simon
I am curious about the future direction of NetBSD with regard to packet
* This appears to be largely on the ground that ipfilter is
unmaintained in our tree, and
* pf is also currently unmaintained in our tree.
2) pf is considerably simpler than ipfilter (mostly, it seems, because it
is much newer and has less historical portability baggage).
3) We have third and fourth filter engines in our tree, the ipsec security
policy engine and the altq classification engine. This is clearly
redundant both in the source tree and at runtime (at runtime, doing
PFIL_HOOKS filters and then two other kinds of filtering can slow
things down considerably for little net gain).
* Itojun's intent was to unify the ipsec and altq filtering
using pf as the engine. This is a large part of why pf is
currently in our tree, as I understand it. Nobody has ever
done the analogous work to let ipfilter serve this role.
4) pf is essentially subsystem-locked by the softnet lock. On the
other hand, ipfilter has much finer-grained locking because it can
run in other kernels which have pushed locks down into the network
stack.
Given this set of facts, it's not clear to me what the right path
forward is. #4 seems likely to be a considerable inconvenience in the
future if we go with pf. On the other hand, ipfilter's author is a
NetBSD developer, but even he doesn't have time to maintain the version
in the NetBSD source tree (as far as I can tell) and it seems unlikely
that he or anyone else will step forward to address #3.
Still nobody has stepped forward for the comparatively simple task of
updating and maintaining pf, either.
What are others' opinions on this?

Another advantage pf had over ipfilter was binary compatibility with
previous releases. However, it is no longer true.

At this point I think the inconvenience is that we don't have any
packet filter maintained in our tree. Having one of them maintained
would be an improvement regardless of the qualities each of them has
over the other.

So if someone would step up and offer to maintain either one, that
would be a very strong reason to keep the relevant filter instead of
the other.

The thing that worries me is that people might be afraid to step up
because they know they'll have to deal with a bunch a vocal people
that will go at length explaining that the other filter is a lot more
important to have in NetBSD, and that it's scandal that NetBSD is
taking such a direction, etc. Such is the reality of the NetBSD
community.

So let me assure that if someone was to step up and show that they have
the time and motivation to maintain either one, they'd receive my
strongest support.

[matthew sporleder]

Post by Quentin Garnier

Post by Thor Simon
I am curious about the future direction of NetBSD with regard to packet
* This appears to be largely on the ground that ipfilter is
unmaintained in our tree, and
* pf is also currently unmaintained in our tree.
2) pf is considerably simpler than ipfilter (mostly, it seems, because it
is much newer and has less historical portability baggage).
3) We have third and fourth filter engines in our tree, the ipsec security
policy engine and the altq classification engine. This is clearly
redundant both in the source tree and at runtime (at runtime, doing
PFIL_HOOKS filters and then two other kinds of filtering can slow
things down considerably for little net gain).
* Itojun's intent was to unify the ipsec and altq filtering
using pf as the engine. This is a large part of why pf is
currently in our tree, as I understand it. Nobody has ever
done the analogous work to let ipfilter serve this role.
4) pf is essentially subsystem-locked by the softnet lock. On the
other hand, ipfilter has much finer-grained locking because it can
run in other kernels which have pushed locks down into the network
stack.
Given this set of facts, it's not clear to me what the right path
forward is. #4 seems likely to be a considerable inconvenience in the
future if we go with pf. On the other hand, ipfilter's author is a
NetBSD developer, but even he doesn't have time to maintain the version
in the NetBSD source tree (as far as I can tell) and it seems unlikely
that he or anyone else will step forward to address #3.
Still nobody has stepped forward for the comparatively simple task of
updating and maintaining pf, either.
What are others' opinions on this?

Another advantage pf had over ipfilter was binary compatibility with
previous releases. However, it is no longer true.
At this point I think the inconvenience is that we don't have any
packet filter maintained in our tree. Having one of them maintained
would be an improvement regardless of the qualities each of them has
over the other.
So if someone would step up and offer to maintain either one, that
would be a very strong reason to keep the relevant filter instead of
the other.
The thing that worries me is that people might be afraid to step up
because they know they'll have to deal with a bunch a vocal people
that will go at length explaining that the other filter is a lot more
important to have in NetBSD, and that it's scandal that NetBSD is
taking such a direction, etc. Such is the reality of the NetBSD
community.
So let me assure that if someone was to step up and show that they have
the time and motivation to maintain either one, they'd receive my
strongest support.

Doesn't a lot of this simply rehash the old arguments about generic
interfaces for competing packet filters, classifiers, queuers,
slowdowninators, zimbapiladors, etc?

http://mail-index.netbsd.org/tech-net/2003/06/27/0013.html
http://mail-index.netbsd.org/tech-kern/2005/07/01/0011.html
http://mail-index.netbsd.org/current-users/2005/09/22/0021.html

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Ivo Vachkov]

Please, excuse my ignorance,

I just want to ask where to get the latest PF code from ? OpenBSD
source repositories ? FreeBSD source repositories ?

Thank you in advance.

Post by matthew sporleder

Post by Quentin Garnier

Post by Thor Simon
I am curious about the future direction of NetBSD with regard to packet
* This appears to be largely on the ground that ipfilter is
unmaintained in our tree, and
* pf is also currently unmaintained in our tree.
2) pf is considerably simpler than ipfilter (mostly, it seems, because it
is much newer and has less historical portability baggage).
3) We have third and fourth filter engines in our tree, the ipsec security
policy engine and the altq classification engine. This is clearly
redundant both in the source tree and at runtime (at runtime, doing
PFIL_HOOKS filters and then two other kinds of filtering can slow
things down considerably for little net gain).
* Itojun's intent was to unify the ipsec and altq filtering
using pf as the engine. This is a large part of why pf is
currently in our tree, as I understand it. Nobody has ever
done the analogous work to let ipfilter serve this role.
4) pf is essentially subsystem-locked by the softnet lock. On the
other hand, ipfilter has much finer-grained locking because it can
run in other kernels which have pushed locks down into the network
stack.
Given this set of facts, it's not clear to me what the right path
forward is. #4 seems likely to be a considerable inconvenience in the
future if we go with pf. On the other hand, ipfilter's author is a
NetBSD developer, but even he doesn't have time to maintain the version
in the NetBSD source tree (as far as I can tell) and it seems unlikely
that he or anyone else will step forward to address #3.
Still nobody has stepped forward for the comparatively simple task of
updating and maintaining pf, either.
What are others' opinions on this?

Another advantage pf had over ipfilter was binary compatibility with
previous releases. However, it is no longer true.
At this point I think the inconvenience is that we don't have any
packet filter maintained in our tree. Having one of them maintained
would be an improvement regardless of the qualities each of them has
over the other.
So if someone would step up and offer to maintain either one, that
would be a very strong reason to keep the relevant filter instead of
the other.
The thing that worries me is that people might be afraid to step up
because they know they'll have to deal with a bunch a vocal people
that will go at length explaining that the other filter is a lot more
important to have in NetBSD, and that it's scandal that NetBSD is
taking such a direction, etc. Such is the reality of the NetBSD
community.
So let me assure that if someone was to step up and show that they have
the time and motivation to maintain either one, they'd receive my
strongest support.

Doesn't a lot of this simply rehash the old arguments about generic
interfaces for competing packet filters, classifiers, queuers,
slowdowninators, zimbapiladors, etc?
http://mail-index.netbsd.org/tech-net/2003/06/27/0013.html
http://mail-index.netbsd.org/tech-kern/2005/07/01/0011.html
http://mail-index.netbsd.org/current-users/2005/09/22/0021.html

[y***@sdf.lonestar.org]

Post by Thor Simon
I am curious about the future direction of NetBSD with regard to packet
* This appears to be largely on the ground that ipfilter is
unmaintained in our tree, and
* pf is also currently unmaintained in our tree.
2) pf is considerably simpler than ipfilter (mostly, it seems, because

it

Post by Thor Simon
is much newer and has less historical portability baggage).
3) We have third and fourth filter engines in our tree, the ipsec

security

Post by Thor Simon
policy engine and the altq classification engine. This is clearly

redundant both in the source tree and at runtime (at runtime, doing
PFIL_HOOKS filters and then two other kinds of filtering can slow
things down considerably for little net gain).

Post by Thor Simon
* Itojun's intent was to unify the ipsec and altq filtering
using pf as the engine. This is a large part of why pf is
currently in our tree, as I understand it. Nobody has ever
done the analogous work to let ipfilter serve this role.
4) pf is essentially subsystem-locked by the softnet lock. On the
other hand, ipfilter has much finer-grained locking because it can

run in other kernels which have pushed locks down into the network
stack.

Post by Thor Simon
Given this set of facts, it's not clear to me what the right path

forward is. #4 seems likely to be a considerable inconvenience in the
future if we go with pf. On the other hand, ipfilter's author is a
NetBSD developer, but even he doesn't have time to maintain the version
in the NetBSD source tree (as far as I can tell) and it seems unlikely
that he or anyone else will step forward to address #3.

Post by Thor Simon
Still nobody has stepped forward for the comparatively simple task of

updating and maintaining pf, either.

Post by Thor Simon
What are others' opinions on this?

I'm not sure you wanted a mere user response to this, but...

I've been using NetBSD for 12 years and ipf for 10 years or so, I've
even helped track down an ipf bug or two in the past.

ipf is maintained as an independent package with it's own web sites and
mailing lists and is usable in several *nix systems as you say. Darren
has been working on release candidates for v4.1.32. I've also
seen some talk of v5.0.x versions.

Darren occasionally updates the NetBSD version with his base code
releases. My observation is that as he has gotten busier, this is less
frequent than in the past.

I've taken some stabs at updating my local source tree to his
updated base releases, but it's a bit too involved for my skills
and time constraints. It might be easier if we moved from a kernel
integrated code base to an LKM version. I tried to make that happen at one
time and again my limitations stymied me.

The ipf version in NetBSD_4_Stable (ipf v4.1.23) is outdated and
contains some regressions that are apparently addressed in NetBSD_5_Beta
(ipf v4.1.29). The particular issue is rules that include quick and
keep state don't work properly in 4.1.23... There are some workarounds
that only slightly reduce rule processing efficiency.

From what I can tell pf syntax and ipf syntax are pretty similar...
(I've wondered about the cross lineage between the two codes are, but
basic googling did not yield anything for me...)

Because of the above issue I considered switching to pf and picking up
ALTQ support, but translating and debugging the tools and scripts I have
also takes time I do not have, so instead I've just decided to upgrade to
the NetBSD 5 branch sooner than I otherwise might have.

In the long run, I suspect only Darren can maintain the ipfilter code. I
suspect it would be possible to run an SOC project to automate
integration of Darren's base releases into the kernel and userland,
possibly via conversion to LKM for the kernel side.
[This would be my vote at the moment]

Integration of the other aspects (ALTQ, etc.) you mention seems
unlikely given the complexity of the code and Darren's limited time.

--gene









--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Jeremy C. Reed]

Post by y***@sdf.lonestar.org

Post by y***@sdf.lonestar.org
From what I can tell pf syntax and ipf syntax are pretty similar...

(I've wondered about the cross lineage between the two codes are, but
basic googling did not yield anything for me...)

No cross lineage.
http://www.benzedrine.cx/pf-beginning.html

It would be good to have table or chart listing the features available and
not available in IPF and PF. At one time I began a chart (for other packet
filters too), but never got far.
http://reedmedia.net/misc/networking/packet-filter.html
Please suggest features or letting me know what is supported for my chart.

I need to update it to add some features supported by IPF: variable
substitution; tuning during run-time; save state over reboots; active and
testing filter which can be swapped; can generate C code for filter rules
hard-coded in custom kernel; flush specific TCP states (at run-time);
flush idle states that are a certain age (at run-time); provides tools to
generate simple ruleset and testing of rulesets without enabling on real
firewall (and using various packet input formats); able to call kernel
functions per a rule; authentication (such as password) for rules; lookup
tables; packet per second matching; few built in proxies; some load
balancing; checksum verifications. Which of these are supported by PF?
What else to add for IPF and/or PF?


Jeremy C. Reed

uggc://jjj.errqzrqvn.arg/obbxf/cs-obbx/


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Thor Lancelot Simon]

Post by y***@sdf.lonestar.org
Darren occasionally updates the NetBSD version with his base code
releases.

Unfortunately, as far as I can tell this is not so. That is not to say
that it would not be a good thing if it were so, nor that Darren has any
obligation to make it so -- but it's not so.

Other NetBSD developers have in fact updated the in-tree ipf from
Darren's published releases, every time it has been updated in many
years. I think it would be much simpler and easier for all involved if
Darren did it directly but there is no reason why he has to agree with
me about that!

Thor

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Greg A. Woods]

At Thu, 19 Feb 2009 10:35:04 -0500, Thor Lancelot Simon wrote:
Subject: Re: packet filters for NetBSD in the future

Post by Thor Lancelot Simon

Post by y***@sdf.lonestar.org
Darren occasionally updates the NetBSD version with his base code
releases.

Unfortunately, as far as I can tell this is not so. That is not to say
that it would not be a good thing if it were so, nor that Darren has any
obligation to make it so -- but it's not so.

Well, Darren did do the updates to at least two revisions since IPF was
relegated off to the /usr/src{/sys}/dist area, the most important from
this perspective perhaps also being the most recent:

date: 2008/05/20 07:08:06; author: darrenr; state: Exp; lines: +172 -2
Pullup IPFilter 4.1.29 from the vendor branch to HEAD.
See src/dist/ipf/HISTORY for a list of bug fixes since 4.1.23 (although
a few are already in NetBSD)

date: 1999/12/28 07:40:12; author: darrenr; state: Exp; lines: +11 -0
update ipfilter code to 3.3.6

Darren also did do lots more direct work on IPF directly in the NetBSD
repository prior to IPF being moved into the dist areas thus I think
fully justifying Gene's remarks.


I too vote for IPF being my ongoing favourite firewall component, though
I've some interest in learning at least enough of PF to see if it can do
the kinds of things I need without adding new headaches. I personally
haven't had any problem using IPF with Altq and vice versa, though
perhaps it could work more efficiently if it didn't have to peek at
packet headers again. Altq's functionality could fairly easily be
merged into IPF I think.


As for why IPF hasn't been pulled up to the netbsd-4 branch yet is
certainly not something I can figure out. Perhaps it's as simple as
nobody having made a proper request to the release engineering team yet?

Personally I think _everything_ identified as a "fix", including
upgrades of 3rd party code that don't rely on other new features, should
ALWAYS be merged from -current to _all_ relevant releases in a timely
fashion (eg. N weeks after commit and no new bug reports where N
represents some time representative of the complexity of the change).

[Darren Reed]

I should have replied to this long ago...

Post by Greg A. Woods
Subject: Re: packet filters for NetBSD in the future

Post by Thor Lancelot Simon

Post by y***@sdf.lonestar.org
Darren occasionally updates the NetBSD version with his base code
releases.

Unfortunately, as far as I can tell this is not so. That is not to say
that it would not be a good thing if it were so, nor that Darren has any
obligation to make it so -- but it's not so.

Well, Darren did do the updates to at least two revisions since IPF was
relegated off to the /usr/src{/sys}/dist area, the most important from

Correct.

Whilst it may be that what's in NetBSD doesn't match
the latest that can be obtained from elsewhere, that
isn't necessarily a bad thing. I am, however, trying
to move to a release model for ipfilter that means
choosing a particular version to integrate isn't such
a problem that it is now (and means there is less
boring work required to stay current.)

Furthermore, I try to be proactive (from time to time)
on looking at which bugs are open against ipfilter and
looking to see what can be done about them.

When there are "urgent" bugs, they get attended to.

Post by Greg A. Woods
Darren also did do lots more direct work on IPF directly in the NetBSD
repository prior to IPF being moved into the dist areas thus I think
fully justifying Gene's remarks.

Where it is makes no difference... although having it
in dist does make some things easier.

Additionally, I try to involve others, where possible
and where there are willing victims...>;-)

Darren


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de