Discussion:
bpf jit
(too old to reply)
Alexander Nasonov
2012-08-19 15:06:40 UTC
Permalink
I brought this up on source-changes-d@ before but I think the subject
should be discussed on a general list as well. I also cc to the author
of sljit Zoltán Herczeg.

Mindaugas recently committed bpf_jit (just-in-time) code generator to
the NetBSD i386 and amd64 kernels from FreeBSD code.
The new code is still broken (see below for details) and I don't know
when Mindaugas plans to fix it.

I'm finishing an alternative bpfjit library [1] which is based on sljit
library [2]. Sljit library provides a limited subset of registers and
instructions but this subset is machine independent (with few minor
exceptions). The subset is big enough to support all bpf instructions.

At the moment, sljit supports i386, amd64, arm, ppc32, ppc64 and mips32.
There is no sparc support because the author of sljit neither have time
nor hardware. I don't know a status of mips64.

I can confirm that sljit works on arm (it passes all bpfjit tests on
linux in userspace) and on amd64 (it passes all tests on -current in
userspace and it works in the kernel). Sljit kernel module is not big,
it's size is around 34K on amd64.

Performace of generated code is on par with hand written C code on both
arm and amd64.

In my opinion, sljit generator is better than hand-coded generators
for several reasons:

1) bpfjit code is machine independent while bpf_jit has to be written
for every architecture. This argument has an opposite side of course.
If sljit doesn't support some particular architecture, it's a lot
more work to implement all sljit API for the new architecture than to
code up bpf generator manually.

2) FreeBSD only supports i386 and amd64 at the moment. Some people
believe that there is a suport for some other arches but I don't
have any references to implementations. The best I could find is
a post by Robert Watson [3].

3) bpf_jit expects an umlimate knowdedge of instructions encoding and
this badly affects both readability and extensibility. For example,
there are jumps by 12 bytes in bpf_jit generator. In order to tell
what is a destication of a jump (and whether it's valid at all), you
need to know a binary encoding format. On the other hand, sljit
supports jumps integrity using jump and label objects.

4) FreeBSD doesn't support (quite likely for the above reason) reading
data from mbuf chain. They instead fallback to interpreter but it
leads to suboptimal performance for bigger packets and this makes
their implementation look a bit like a toy. Why do they need jit
compiler at all if they only implement it for small packets?

Current NetBSD code doesn't implement code generation for mbuf chain
either but it calls jit code nevertheless, it's a bug I mentioned at
the beginning of my post.

5) bpfjit supports mbuf chain. It didn't take long to implement mbuf
chain using sljit but it's harder to test mbuf chain in userspace
and I didn't test this functionality thoroughly yet.

6) sljit can be used for other things like npf_code.


Sljit has downsides, of course. I mentioned one already in bullet 1) but
there are some others:

- No manual page. Most functionality is documented in sljitLir.h, it
should be moved or copied to a manual page.
- API is not stable yet.
- No proper build script and .so versioning. It's less a problem for
the kernel space but I'd like to build sljit in userspace as well
to run unit tests.

I'd like to conclude my lengthy post with a question. Which of these two
implementations should be used by NetBSD?

Thanks,
Alex

[1] https://github.com/alnsn/bpfjit
[2] http://sljit.sourceforge.net
[3] http://lists.freebsd.org/pipermail/svn-src-head/2009-November/011836.html

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Martin Husemann
2012-08-19 18:22:40 UTC
Permalink
Post by Alexander Nasonov
At the moment, sljit supports i386, amd64, arm, ppc32, ppc64 and mips32.
There is no sparc support because the author of sljit neither have time
nor hardware. I don't know a status of mips64.
I was suprised when the jit code got commited without any previous
discussion, especially as you had pointed to your version before, so
choice was obviously available.

I had a look at what it would take to add support for sparc* to bpf_jit
and sljit - and found that the effort (both amount of work and
complexity) is roughly the same. The hardest part is verifying it, we
don't have reasonable tests for either [in bpf context, I don't know
about a potentialy existing sljit test suite that could be used
indepently of the bpf stuff to verify a new codegen] currently. I will
have a look at it once we settle this discussion.

Martin

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Mindaugas Rasiukevicius
2012-08-19 22:26:34 UTC
Permalink
Post by Martin Husemann
I had a look at what it would take to add support for sparc* to bpf_jit
and sljit - and found that the effort (both amount of work and
complexity) is roughly the same. The hardest part is verifying it, we
don't have reasonable tests for either [in bpf context, I don't know
about a potentialy existing sljit test suite that could be used
indepently of the bpf stuff to verify a new codegen] currently. I will
have a look at it once we settle this discussion.
BPF regression test from FreeBSD has more than 80 test cases, which I
think is pretty good (do we have any other component/interface with more
test cases in our test suite?). Nevertheless we can add more.
--
Mindaugas

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Alexander Nasonov
2012-08-20 00:37:20 UTC
Permalink
Post by Mindaugas Rasiukevicius
BPF regression test from FreeBSD has more than 80 test cases, which I
think is pretty good (do we have any other component/interface with more
test cases in our test suite?). Nevertheless we can add more.
I have 80 test cases and 238 CHECKs.

Alex

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Alexander Nasonov
2012-08-20 02:55:53 UTC
Permalink
Post by Martin Husemann
I had a look at what it would take to add support for sparc* to bpf_jit
and sljit - and found that the effort (both amount of work and
complexity) is roughly the same. The hardest part is verifying it, we
don't have reasonable tests for either [in bpf context, I don't know
about a potentialy existing sljit test suite that could be used
indepently of the bpf stuff to verify a new codegen] currently. I will
have a look at it once we settle this discussion.
Both sljit and bpfjit come with tests. Mine tests should cover all bpf
instructions and many corner cases (e.g. overflows, divisions by zero).

Zoltán may start working on sparc at some point. He replied to me
privately that someone offered him a sparc box but he's busy with other
things at the moment.

Alex

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Mouse
2012-08-19 19:04:12 UTC
Permalink
[...] bpf_jit [...] bpfjit [based on] sljit [...]
I'd like to conclude my lengthy post with a question. Which of these
two implementations should be used by NetBSD?
It seems to me a better first question is, should either of them be
used? They sound to me like solutions in search of problems. Are
there measurements supporting the idea that the BPF pcode engine is a
performance bottleneck? I don't recall seeing any (though of course I
could have missed something)...and, if not, why would NetBSD want to
replace a simple MI engine by a complex MD engine, especially in
something as exposed and security-critical as BPF? Particularly since
it seems to me that the CPUs it supports are the ones least in need of
a performance boost? Unless this proposal involves dropping support
for BPF on all other arches, which I also haven't seen mentioned, the
maintenance load of the pcode engine will not go away, so I'm having
trouble seeing a BPF jitter as a good thing.

I'm also concerned about it from a slightly different perspective.
From the sound of it, this is still a research project, albeit one
that's beginning to see a little use outside the lab. Compilers are,
in general, full of subtle corner cases; jitters, even more so. And
BPF is highly privileged, highly exposed, and security critical; it
seems to me that all the indications are against putting a jitter
there. It's raising way too many of my red flags for me to be
comfortable with it.

In NetBSD proper, that is. It's a lovely thing to experiment with; I
just don't think it belongs in the main tree.

/~\ The ASCII Mouse
\ / Ribbon Campaign
X Against HTML ***@rodents-montreal.org
/ \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Alexander Nasonov
2012-08-20 02:45:18 UTC
Permalink
Post by Mouse
It seems to me a better first question is, should either of them be
used? They sound to me like solutions in search of problems.
If you use bpf only to run tcpdump once in a while then you probably
don't need jit but if you run intrusion detection system or use many
bpf rules for monitoring, then you would probably benefit from jit.
Post by Mouse
Are there measurements supporting the idea that the BPF pcode engine is
a performance bottleneck?
I didn't do any measurements myself of the whole system because I don't
have access to any test framework but jitted code is 3-5 times faster in
my benchmark. In absolute values it's IIRC 30ns per packet on my amd64
and 150-200ns on my Tegra-250 arm. This is for a simple rule, difference
is greater for more complicated rules.
Post by Mouse
Particularly since it seems to me that the CPUs it supports are the ones
least in need of a performance boost?
Mine supports several cpu types. Or do they all fall into "least in need
of a performance boost"?
Post by Mouse
Unless this proposal involves dropping support for BPF on all other
arches, which I also haven't seen mentioned, the maintenance load of
the pcode engine will not go away, so I'm having trouble seeing a BPF
jitter as a good thing.
bpf pcode will not go away. bpfjit will be optional.
Post by Mouse
I'm also concerned about it from a slightly different perspective.
From the sound of it, this is still a research project, albeit one
that's beginning to see a little use outside the lab. Compilers are,
in general, full of subtle corner cases; jitters, even more so.
Why do you think that jitters are more full of subtle corner cases?

Perhaps I should clarify a bit. sljit is not really jit which you
typically find in modern interpreted languages, sljit is much simpler.
It's a code generator with an ability to change jumps and constants on
the fly. But I don't use these facilities in bpfjit. I use sljit as a
pure code generator.
Post by Mouse
And BPF is highly privileged, highly exposed, and security critical; it
seems to me that all the indications are against putting a jitter
there. It's raising way too many of my red flags for me to be
comfortable with it.
In NetBSD proper, that is. It's a lovely thing to experiment with; I
just don't think it belongs in the main tree.
Yes, I'm aware it's highly exposed code but I'm not pushing it to
NetBSD-6. We have time to test and improve before the next branching
of head.

Alex

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Alexander Nasonov
2012-08-20 02:59:51 UTC
Permalink
Post by Alexander Nasonov
I didn't do any measurements myself of the whole system because I don't
have access to any test framework ...
s/framework/network/

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Mouse
2012-08-20 04:01:15 UTC
Permalink
Post by Alexander Nasonov
Post by Mouse
Are there measurements supporting the idea that the BPF pcode engine
is a performance bottleneck?
I didn't do any measurements myself of the whole system because I
don't have access to any test [network] but jitted code is 3-5 times
faster in my benchmark. In absolute values it's IIRC 30ns per packet
on my amd64 and 150-200ns on my Tegra-250 arm.
Then it comes down to, is saving 30ns worh the code size and complexity
(and thus exposure)? I don't know how large a fraction of the cost of
packet handling 30ns is; I suspect it's small enough that I don't
consider it worth the costs. (To pick a number out of thin air, is
handling a packet in 970ns instead of 1us worth it? I think it's not,
and I suspect 1us to handle a packet is highly optimistic, though I
haven't measured that either.)
Post by Alexander Nasonov
Post by Mouse
Particularly since it seems to me that the CPUs it supports are the
ones least in need of a performance boost?
Mine supports several cpu types. Or do they all fall into "least in
need of a performance boost"?
Most of them certainly do. The ones that most need a performance boost
are things like acorn26, sun2, hp300, vax, the slower pmax and
sparc....
Post by Alexander Nasonov
Post by Mouse
Compilers are, in general, full of subtle corner cases; jitters,
even more so.
Why do you think that jitters are more full of subtle corner cases?
They've got most of the complexities of all compilers, plus a bunch
more all their own. It's uncomfortably close to "so complex there are
no obvious defects" rather than "so simple there are obviously no
defects".
Post by Alexander Nasonov
Post by Mouse
And BPF is highly privileged, highly exposed, and security critical;
it seems to me that all the indications are against putting a jitter
there.
Yes, I'm aware it's highly exposed code but I'm not pushing it to
NetBSD-6.
I'm not sure how that makes it any more appropriate to put it into the
main tree at this stage, but it doesn't really matter. I won't be
running -6 myself (for plenty of other reasons); I've pointed out my
concerns, and if NetBSD wants to ignore them, it's NetBSD's decision,
and NetBSD's security on the line.

/~\ The ASCII Mouse
\ / Ribbon Campaign
X Against HTML ***@rodents-montreal.org
/ \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Darren Reed
2012-08-20 11:00:14 UTC
Permalink
Post by Mouse
Post by Alexander Nasonov
Post by Mouse
Are there measurements supporting the idea that the BPF pcode engine
is a performance bottleneck?
I didn't do any measurements myself of the whole system because I
don't have access to any test [network] but jitted code is 3-5 times
faster in my benchmark. In absolute values it's IIRC 30ns per packet
on my amd64 and 150-200ns on my Tegra-250 arm.
Then it comes down to, is saving 30ns worh the code size and complexity
(and thus exposure)? I don't know how large a fraction of the cost of
packet handling 30ns is; I suspect it's small enough that I don't
consider it worth the costs. (To pick a number out of thin air, is
handling a packet in 970ns instead of 1us worth it? I think it's not,
and I suspect 1us to handle a packet is highly optimistic, though I
haven't measured that either.)
At gigabit speeds, the maximum number of packets per second is 1.4 million.

That is a packet every 714ns.

Values of 30ns, 150ns and 200ns are very significant alongside that.

Darren


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Mindaugas Rasiukevicius
2012-08-19 22:13:54 UTC
Permalink
Post by Alexander Nasonov
Mindaugas recently committed bpf_jit (just-in-time) code generator to
the NetBSD i386 and amd64 kernels from FreeBSD code.
The new code is still broken (see below for details) and I don't know
when Mindaugas plans to fix it.
No, it is not broken if used in a certain way (which is actually the
common case) - see below.
Post by Alexander Nasonov
At the moment, sljit supports i386, amd64, arm, ppc32, ppc64 and mips32.
There is no sparc support because the author of sljit neither have time
nor hardware. I don't know a status of mips64.
Well, that covers most of the Tier I ports, which I think is good enough.
Post by Alexander Nasonov
Performace of generated code is on par with hand written C code on both
arm and amd64.
Do you have any numbers? With the current BPF JIT, it is more than 2x
performance increase:

http://www.netbsd.org/~rmind/bpf_jit_times.txt
Post by Alexander Nasonov
In my opinion, sljit generator is better than hand-coded generators
1) bpfjit code is machine independent while bpf_jit has to be written
for every architecture. This argument has an opposite side of course.
If sljit doesn't support some particular architecture, it's a lot
more work to implement all sljit API for the new architecture than to
code up bpf generator manually.
2) FreeBSD only supports i386 and amd64 at the moment. Some people
believe that there is a suport for some other arches but I don't
have any references to implementations. The best I could find is
a post by Robert Watson [3].
3) bpf_jit expects an umlimate knowdedge of instructions encoding and
this badly affects both readability and extensibility. For example,
there are jumps by 12 bytes in bpf_jit generator. In order to tell
what is a destication of a jump (and whether it's valid at all), you
need to know a binary encoding format. On the other hand, sljit
supports jumps integrity using jump and label objects.
I guess (3) is a valid point, but as for (1) and (2) - I am not really
convinced that it is any easier to add support for sljit. But perhaps
it is.. see below.
Post by Alexander Nasonov
4) FreeBSD doesn't support (quite likely for the above reason) reading
data from mbuf chain. They instead fallback to interpreter but it
leads to suboptimal performance for bigger packets and this makes
their implementation look a bit like a toy. Why do they need jit
compiler at all if they only implement it for small packets?
Current NetBSD code doesn't implement code generation for mbuf chain
either but it calls jit code nevertheless, it's a bug I mentioned at
the beginning of my post.
It is true that mbuf chains are not supported, but the size of the packet
does not really matter here. Most of the filtering operations are done
only on layer 3 and/or 4 headers, i.e. it is a common case. IP and other
layers (AFAIK in a vast majority of the cases) it is ensured that these
headers will be in a single (and the first) mbuf. So, if one performs
m_copyback() (which would nearly always be a NOP), to guarantee one
contiguous mbuf, it is perfectly safe to use. It also meets NPF/n-code
needs for now.

I fully agree, though, that adding support for mbuf chains would be great
or rather necessary, if we want to support e.g. L7 filtering (once we will
get there). In any case, it is the reason why I kept BPF JIT disabled by
default for now (and, in fact, without a sysctl option to enable it yet).
Post by Alexander Nasonov
Sljit has downsides, of course. I mentioned one already in bullet 1) but
- No manual page. Most functionality is documented in sljitLir.h, it
should be moved or copied to a manual page.
- API is not stable yet.
- No proper build script and .so versioning. It's less a problem for
the kernel space but I'd like to build sljit in userspace as well
to run unit tests.
I'd like to conclude my lengthy post with a question. Which of these two
implementations should be used by NetBSD?
BPF JIT from FreeBSD is something what is here, works and is more tested.
Seems (or seemed?) smaller and simpler to work with as well, since SLJIT
was not ready to be used as an in-kernel JIT at all. As I understand,
you have addressed this by now.

I guess the problem with SLJIT is that nobody else here is familiar with it
besides yourself, therefore it is difficult to decide on its suitability.
Lack of documentation, as you already pointed out, contributes to this as
well. There are various other questions unanswered, e.g. would we diverge
SLJIT code in favour of better integration into NetBSD? Would the changes
be committed to upstream or we would maintain SLJIT ourselves or perhaps
would even take over the maintenance?

Personally, I do not have a strong preference to use the current JIT code,
but not yet enough confidence to support SLJIT either.
--
Mindaugas

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Alexander Nasonov
2012-08-20 02:07:37 UTC
Permalink
Post by Mindaugas Rasiukevicius
No, it is not broken if used in a certain way (which is actually the
common case) - see below.
I still believe it's broken because you pass a pointer to mbuf chain and
non-zero pktlen. I fixed non-jit path and I pass buflen (which is zero
for mbuf chains) instead of pktlen but I didn't change jit path because
it wouldn't fix the code anyway because jit doesn't support mbuf chains.
Perhaps I should have changed jit path as well because it's safe to
reject a packet of zero length than to read mbuf object as it was a raw
packet data.

Try running bpfiltermchain test with jit enabled. It's rump test, you
will need to recompile rump libs. (disclaimer: I didn't do it myself).
Post by Mindaugas Rasiukevicius
Do you have any numbers? With the current BPF JIT, it is more than 2x
http://www.netbsd.org/~rmind/bpf_jit_times.txt
2x is not very impressive but you benchmarked testcases. It'd be better
to benchmark a common filter. I use the second example from bpf(4):
"accept only IP packets between host 128.3.112.15 and 128.3.112.35"
and I get 3-5x speedup.
Post by Mindaugas Rasiukevicius
It is true that mbuf chains are not supported, but the size of the packet
does not really matter here. Most of the filtering operations are done
only on layer 3 and/or 4 headers, i.e. it is a common case. IP and other
layers (AFAIK in a vast majority of the cases) it is ensured that these
headers will be in a single (and the first) mbuf. So, if one performs
m_copyback() (which would nearly always be a NOP), to guarantee one
contiguous mbuf, it is perfectly safe to use. It also meets NPF/n-code
needs for now.
I probably miss something here. Device drivers don't know whether your
bpf program access only L3 headers or any byte in a large packet.
Therefore, you can't just give the first mbuf to bpf, you have to pass
the whole mbuf chain to bpf.

Size of a packet doesn't affect bpf performance directly but the
interpreter does. If large packets are common in your workload (nfs,
ftp, etc) then it's often quite demanding environment and it's probably
more important to optimize any overhead compared to a case where all
packets are small.
Post by Mindaugas Rasiukevicius
BPF JIT from FreeBSD is something what is here, works and is more tested.
Seems (or seemed?) smaller and simpler to work with as well, since SLJIT
was not ready to be used as an in-kernel JIT at all. As I understand,
you have addressed this by now.
Yes, making it work in the kernel was quite easy.
Post by Mindaugas Rasiukevicius
I guess the problem with SLJIT is that nobody else here is familiar with it
besides yourself, therefore it is difficult to decide on its suitability.
How many people are familiar with x86 binary instruction encoding? How
many people are familiar with encodings of all architectures supported
by sljit?

I encourage you to take a look at my code and at sljit. I believe it's
easier to learn sljit than learning instruction formats for multiple
architectures.
Post by Mindaugas Rasiukevicius
Lack of documentation, as you already pointed out, contributes to this as
well. There are various other questions unanswered, e.g. would we diverge
SLJIT code in favour of better integration into NetBSD? Would the changes
be committed to upstream or we would maintain SLJIT ourselves or perhaps
would even take over the maintenance?
The author of sljit is in the discussion. Perhaps he can answer these
questions?

Alex

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Zoltán Herczeg
2012-08-20 05:01:15 UTC
Permalink
Hi NetBSD devs,
Post by Alexander Nasonov
Post by Mindaugas Rasiukevicius
Lack of documentation, as you already pointed out, contributes to this as
well. There are various other questions unanswered, e.g. would we diverge
SLJIT code in favour of better integration into NetBSD? Would the changes
be committed to upstream or we would maintain SLJIT ourselves or perhaps
would even take over the maintenance?
The author of sljit is in the discussion. Perhaps he can answer these
questions?
The primary design points of sljit:
1) be lightweight - lighter than a library. In PCRE-JIT the compiler is simply included with a #include, so it is not possible to use the JIT compiler through PCRE, and the unnecessary functions (i.e. floating point arithmetic) are also removed by the linker.
2) easy to learn - looks like a simplified assembler.
3) cover the general features of widely used CPUs, instead of creating another RISC instruction translator. That is a major difference compared to other lightweight JITs (libjit, GNU lightning).

We can add new features as long as they don't hurt other use cases, i.e. NetBSD kernel specific defines, if they are properly guarded by ifdefs. We can also add autotools based build system, .so versioning, etc. Just we should never forget that all of these features are optional. Of course we need someone who is willing to maintain these features.

Anyway, thanks to the PCRE library, sljit is now widely tested. I got feedback from IBM AIX users on PowerPC, Longsoon based MIPS sytems, etc, and we were able to make sljit work with a few minor changes.

Regards,
Zoltan


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Alexander Nasonov
2012-08-20 12:56:35 UTC
Permalink
Post by Zoltán Herczeg
1) be lightweight - lighter than a library. In PCRE-JIT the compiler is simply included with a #include, so it is not possible to use the JIT compiler through PCRE, and the unnecessary functions (i.e. floating point arithmetic) are also removed by the linker.
The same effect can be achieved with .a library. I think I can make an
internal (which don't get installed into /usr/lib) static library and
link tests with it.

Alex

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Mindaugas Rasiukevicius
2012-08-20 23:10:00 UTC
Permalink
Post by Alexander Nasonov
Post by Mindaugas Rasiukevicius
It is true that mbuf chains are not supported, but the size of the
packet does not really matter here. Most of the filtering operations
are done only on layer 3 and/or 4 headers, i.e. it is a common case.
IP and other layers (AFAIK in a vast majority of the cases) it is
ensured that these headers will be in a single (and the first) mbuf.
So, if one performs m_copyback() (which would nearly always be a NOP),
to guarantee one contiguous mbuf, it is perfectly safe to use. It also
meets NPF/n-code needs for now.
I probably miss something here. Device drivers don't know whether your
bpf program access only L3 headers or any byte in a large packet.
Therefore, you can't just give the first mbuf to bpf, you have to pass
the whole mbuf chain to bpf.
BPF program producers, such as libpcap or say NPF, can know and can pass
that knowledge. If inspecting outside the L3/L4 headers, you just fallback
to non-JIT code. Also, with m_copyback(), m_pullup() or other routine from
this family you guarantee (i.e. it copies, if needed) that these headers
will be in the first mbuf of the chain, but as already pointed out - other
layers in the stack, e.g. IP, do that already so it is normally a NOP.
Post by Alexander Nasonov
Size of a packet doesn't affect bpf performance directly but the
interpreter does. If large packets are common in your workload (nfs,
ftp, etc) then it's often quite demanding environment and it's probably
more important to optimize any overhead compared to a case where all
packets are small.
We talk about L3/L4 headers as a common case (for n-code, which is my
interest, it would be the only case for now). Unless you talk about
something like application layer filtering, I do not see how the size
matters. However, we already agree that handling of mbuf chains is an
important feature to have.
--
Mindaugas

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Matthew Mondor
2012-08-20 02:14:47 UTC
Permalink
On Sun, 19 Aug 2012 23:13:54 +0100
Post by Mindaugas Rasiukevicius
BPF JIT from FreeBSD is something what is here, works and is more tested.
Seems (or seemed?) smaller and simpler to work with as well, since SLJIT
was not ready to be used as an in-kernel JIT at all. As I understand,
you have addressed this by now.
I guess the problem with SLJIT is that nobody else here is familiar with it
besides yourself, therefore it is difficult to decide on its suitability.
Lack of documentation, as you already pointed out, contributes to this as
well. There are various other questions unanswered, e.g. would we diverge
SLJIT code in favour of better integration into NetBSD? Would the changes
be committed to upstream or we would maintain SLJIT ourselves or perhaps
would even take over the maintenance?
Personally, I do not have a strong preference to use the current JIT code,
but not yet enough confidence to support SLJIT either.
If I recall, LLVM also comes with a register-based VM and a JIT for it,
would this be totally unsuitable for future use here? I admit I'm not
very familiar with it or its portability and usability status.

Thanks,
--
Matt

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Alexander Nasonov
2012-08-20 02:47:53 UTC
Permalink
Post by Matthew Mondor
If I recall, LLVM also comes with a register-based VM and a JIT for it,
would this be totally unsuitable for future use here? I admit I'm not
very familiar with it or its portability and usability status.
LLVM is huge compared to sljit. I've not looked at their code but I feel
that porting it to the kernel will be a difficult task.

Alex

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Loading...