Discussion:
NPF tuning
(too old to reply)
Egerváry Gergely
2017-01-15 13:26:49 UTC
Permalink
Hi,

It seems I cannot load a ruleset with more than 120 (128?) rules:
npfctl: npfctl_config_send: Invalid argument

Is there a limit on rule count? How to raise?

In IPFilter and PF I also have to tune some other limits, eg. state
table size, max frag size, etc.

PF:

set limit states 120000
set limit frags 20000

IPFilter:

ipfilter_flags="-T
state_max=30011,state_size=40009,nat_table_size=40009,nat_table_max=120000"

Isn't this necessary on NPF? Can it play well with ~80.000 states by
default?

Thanks,
--
Gergely EGERVARY


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Egerváry Gergely
2017-01-15 13:49:08 UTC
Permalink
Post by Egerváry Gergely
npfctl: npfctl_config_send: Invalid argument
Is there a limit on rule count? How to raise?
Hmm,

/*
* Some artificial limits.
* Note: very unlikely to have many ALGs.
*/
#define NPF_MAX_RULES (1024 * 1024)
#define NPF_MAX_ALGS 4
#define NPF_MAX_TABLES 128
#define NPF_MAX_RPROCS 128
#define NPF_MAX_IFMAP 64

Strange. If I remove some (random) rules, it loads.
Trying to figure out...
--
Gergely EGERVARY

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Mindaugas Rasiukevicius
2017-01-15 14:01:16 UTC
Permalink
Post by Egerváry Gergely
Hi,
npfctl: npfctl_config_send: Invalid argument
Is there a limit on rule count? How to raise?
The limit should not be a problem. Can you send me your config off-list?

Thanks.
--
Mindaugas

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Egerváry Gergely
2017-01-15 21:18:37 UTC
Permalink
Post by Mindaugas Rasiukevicius
https://nxr.netbsd.org/xref/src/common/lib/libprop/prop_kern.c#410
Christos bumped it in -current.
128kB instead of 64kB... so... it will support ~240 rules instead of
~120 or will my ruleset with ~500 rules fit in?

64kB is a lot of space for only 120 rules. About 546 bytes for each
rule.
--
Gergely EGERVARY


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Mindaugas Rasiukevicius
2017-01-15 21:27:44 UTC
Permalink
Post by Egerváry Gergely
Post by Mindaugas Rasiukevicius
https://nxr.netbsd.org/xref/src/common/lib/libprop/prop_kern.c#410
Christos bumped it in -current.
128kB instead of 64kB... so... it will support ~240 rules instead of
~120 or will my ruleset with ~500 rules fit in?
64kB is a lot of space for only 120 rules. About 546 bytes for each
rule.
Yes, proplib is a awful [1].. I think there are quite a few developers
who would like to replace it with something more sensible.

[1] https://mail-index.netbsd.org/tech-kern/2014/01/28/msg016489.html
--
Mindaugas

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
John Nemeth
2017-01-18 17:25:09 UTC
Permalink
On Jan 15, 9:27pm, Mindaugas Rasiukevicius wrote:
} Egerváry Gergely <***@egervary.hu> wrote:
} > > Turns out to be a proplib limitation:
} > > https://nxr.netbsd.org/xref/src/common/lib/libprop/prop_kern.c#410
} > > Christos bumped it in -current.
} >
} > 128kB instead of 64kB... so... it will support ~240 rules instead of
} > ~120 or will my ruleset with ~500 rules fit in?
} >
} > 64kB is a lot of space for only 120 rules. About 546 bytes for each
} > rule.
}
} Yes, proplib is a awful [1].. I think there are quite a few developers
} who would like to replace it with something more sensible.
}
} [1] https://mail-index.netbsd.org/tech-kern/2014/01/28/msg016489.html

I just reread this thread. Considering that only three or
four people expressed a negative opinion in the thread, my conclusion
is that the vast majority of developers are agnostic and only a
small handful of very out-spoken developers think it is awful (you
obviously being one of them).

So, looking at your list and suggested replacement with libnv:

Why replace proplib with libnv? Because contrary to proplib:
- It provides the equivalent functionality, but with twice less code.

No it doesn't. It doesn't offer the storage component. Having
to use a seperate library for storage (which can be thought of as
slow transport) is a non-starter. Thus libnv doesn't cut it. I
would also like to see a simple example of the kernel code for
transporting nvlists. proplib has simple to use functions for this
purpose. It doesn't appear that libnv provides anything.

- It does not have insane reference counting like proplib (particularly,

Gotta love the emotionally loaded terminology.

asymmetric prop_dictionary_get() and prop_dictionary_set() routines).
- It does not use inefficient XML-like format. It is binary.

Upstream libprop (Apple) has a binary serialisation format.
We could just adapt it. There's even code in the mailing list
archives for adding other serialisation formats.

- It does not add locking overhead which you do not need, see PR/44094.
- It does not have shared structures, e.g. proplib stores the keys in a
global red-black tree (besides the fact that its implementation is
duplicated, see PR/44090). That means for example that the contention

This has been fixed.

on one subsystem can negatively affect another, unrelated, subsystem!

Not sure about this.

- It does not abuse void * and therefore has more type safe code.
- It supports accumulated error handling (relevant for most use cases).
- Last but not least, it does not have awkward API naming, such as
prop_data_create_data_nocopy() or prop_number_unsigned_integer_value().

The API naming makes sense once you get used to it.

For me, I would say that the biggest lack is schemas. Anything
that doesn't address that is pointless. After reading the manpage
for libnv my conclusion is that it doesn't provide any significant
advantage over proplib, certainly not enough to warrant the effort
of changing.

I noticed that a lot of complaints have to do with the
implementation. We have smart people. We can fix that.

I don't know if the proplib is the best, but I wouldn't call
it absolutely awful. Note that I've used proplib in a couple of
projects so I feel reasonably qualified to talk about using it.

}-- End of excerpt from Mindaugas Rasiukevicius

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Thor Lancelot Simon
2017-01-23 13:21:08 UTC
Permalink
Post by John Nemeth
}
} Yes, proplib is a awful [1].. I think there are quite a few developers
} who would like to replace it with something more sensible.
}
} [1] https://mail-index.netbsd.org/tech-kern/2014/01/28/msg016489.html
I just reread this thread. Considering that only three or
four people expressed a negative opinion in the thread, my conclusion
is that the vast majority of developers are agnostic and only a
small handful of very out-spoken developers think it is awful (you
obviously being one of them).
Count me among those who think proplib is awful.

Thor

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Manuel Bouyer
2017-01-15 22:19:34 UTC
Permalink
Post by Egerváry Gergely
Post by Mindaugas Rasiukevicius
https://nxr.netbsd.org/xref/src/common/lib/libprop/prop_kern.c#410
Christos bumped it in -current.
128kB instead of 64kB... so... it will support ~240 rules instead of
~120 or will my ruleset with ~500 rules fit in?
240 is still not that much. I used to have more than 2000 rules with
ipf (before we replaced this box with a cisco).
--
Manuel Bouyer <***@antioche.eu.org>
NetBSD: 26 ans d'experience feront toujours la difference
--

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Hauke Fath
2017-01-16 07:12:33 UTC
Permalink
Post by Manuel Bouyer
240 is still not that much. I used to have more than 2000 rules with
ipf (before we replaced this box with a cisco).
It is even worse because npf has many restrictions compared to {i,}pf
that lead to rule duplication.

hauke
--
Hauke Fath <***@Espresso.Rhein-Neckar.DE>
Ernst-Ludwig-Straße 15
64625 Bensheim
Germany

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Mindaugas Rasiukevicius
2017-01-17 20:03:46 UTC
Permalink
Post by Hauke Fath
Post by Manuel Bouyer
240 is still not that much. I used to have more than 2000 rules with
ipf (before we replaced this box with a cisco).
It is even worse because npf has many restrictions compared to {i,}pf
that lead to rule duplication.
You can always fall back to pcap-filter in NPF, e.g.:

pass in final pcap-filter "tcp and src 10.0.0.1"

This gives you tcpdump syntax. Also, use tables whenever possible.
--
Mindaugas

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Hauke Fath
2017-01-17 20:54:10 UTC
Permalink
Post by Mindaugas Rasiukevicius
Also, use tables whenever possible.
It would help a lot if those could optionally be inlined. Also - does
<https://marc.info/?l=netbsd-users&m=146118529901157&w=2> still apply?
It was the only thing I could find for the

npfctl: npfctl_config_send: Invalid argument

message (later identified as proplib limitation), but disabling the
tables did not help.

Cheerio,
hauke
--
Hauke Fath <***@Espresso.Rhein-Neckar.DE>
Ernst-Ludwig-Straße 15
64625 Bensheim
Germany

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Mindaugas Rasiukevicius
2017-01-19 22:12:24 UTC
Permalink
Post by Hauke Fath
Post by Mindaugas Rasiukevicius
Also, use tables whenever possible.
It would help a lot if those could optionally be inlined. Also - does
<https://marc.info/?l=netbsd-users&m=146118529901157&w=2> still apply?
The use of ptree is replaced in NPF; this was fixed in -current and
netbsd-7-1 line. That EINVAL error in particular (at line 174) could
have been due to a table entry with an invalid netmask. This was fixed
in -current to produce a user-friendly error message.
--
Mindaugas

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Mindaugas Rasiukevicius
2017-01-15 18:19:55 UTC
Permalink
Post by Egerváry Gergely
Hi,
npfctl: npfctl_config_send: Invalid argument
Is there a limit on rule count? How to raise?
Turns out to be a proplib limitation:

https://nxr.netbsd.org/xref/src/common/lib/libprop/prop_kern.c#410

Christos bumped it in -current.
--
Mindaugas

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Loading...