Discussion:
GSoC 2008 - Create an in-kernel API for "packet classes"
(too old to reply)
anish babu
2008-03-29 22:01:53 UTC
Permalink
Hi,

My name is Anish and I am a graduate student at Texas A&M University.
I want to work on the project 'Create an in-kernel API for "packet
classes"' . I made a detailed proposal for this project:

Problem Description:
------------------------------

Create an in-kernel API for registering "packet classes" and for
labeling packets with their class, for special treatment by traffic
shapers (ALTQ) and by network interface drivers. With the registration
part of the API, ALTQ or a driver registers the names of packet
classes it recognizes. For example, ALTQ will register the
'class_name' part of a Class Command . An Ethernet NIC with high- and
low-priority transmit rings may register classes called 'hipri' and
'lopri'. An 802.11 NIC may register 802.11e traffic categories, BE,
BK, VI, VO. Each registration yields a token that is suitable for
labeling a packet - i.e., a small amount of data to stick in an mbuf
packet header or in an m_tag.

Make PF use the packet-classes API to convert PF tag names—see
pf.conf(5) for more about tags—to packet-class tokens, and to label
mbufs with the tokens as they exit PF. Make ALTQ extract the
packet-class tokens from mbufs and use them to select the
packet-scheduling class.

Proposed Approach:
-----------------------------
In order to solve this efficiently, I want to maintain a global table
which maps class names identified by different traffic shapers and
network interface drivers to a token name.

The tentative structure of the data structure is:( I understand that
various extra fields need to be added to this table for efficient
implementation)

+--------------------+---------------+
| class_name | token_name |
+--------------------+---------------+
| | |
+------------------------------------+
| | |
+-----------------------------------+

So whenever a traffic shaper or Interface driver encounters a new
class type, it registers that class name with this structure.

Then pf uses pf tag names to get corresponding token name and labels
mbufs with that token name. ALTQ uses these tokens to select the
corresponding scheduling class.

Extending mbuf header is inefficient as for every mbuf of a packet (
which can have multiple mbufs) we are allocating more kernal space
unnecessarily. It suffices if we add an additional m_tag for each
packet.

---->If I am more abstract about any implementation details please let
me know. Even If I am missing something in this proposal, please feel
free to ask.

Thanks,
Anish

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
David Young
2008-03-29 23:18:09 UTC
Permalink
Post by anish babu
-----------------------------
In order to solve this efficiently, I want to maintain a global table
which maps class names identified by different traffic shapers and
network interface drivers to a token name.
The tentative structure of the data structure is:( I understand that
various extra fields need to be added to this table for efficient
implementation)
+--------------------+---------------+
| class_name | token_name |
+--------------------+---------------+
| | |
+------------------------------------+
| | |
+-----------------------------------+
So whenever a traffic shaper or Interface driver encounters a new
class type, it registers that class name with this structure.
Then pf uses pf tag names to get corresponding token name and labels
mbufs with that token name. ALTQ uses these tokens to select the
corresponding scheduling class.
Your proposal is thin on details. :-) What data type will you use for
the tokens? How do PF, ALTQ, and drivers add tokens to the system?
When do you resolve class names to tokens?

What will you do if the kernel starts to label an mbuf with its class,
and that mbuf is already labelled?
Post by anish babu
Extending mbuf header is inefficient as for every mbuf of a packet (
which can have multiple mbufs) we are allocating more kernal space
unnecessarily. It suffices if we add an additional m_tag for each
packet.
How much more kernel space would you need to add to an mbuf? It seems
to me that there are trade-offs whether you enlarge the mbuf structure
or use an m_tag. What do you think?

Dave
--
David Young OJC Technologies
***@ojctech.com Urbana, IL * (217) 278-3933 ext 24

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Joerg Sonnenberger
2008-03-30 01:16:37 UTC
Permalink
Post by anish babu
Extending mbuf header is inefficient as for every mbuf of a packet (
which can have multiple mbufs) we are allocating more kernal space
unnecessarily. It suffices if we add an additional m_tag for each
packet.
Having done exact the embedding of basic FW state and ALTQ options into
the pkthdr on DragonFly -- it is much simpler and faster, the only
difference is that it has to switch to clusters a bit earlier. That is
not a big deal if you ensure that you have at least 80 byte or so in the
first mbuf free (mbuf size - pkthdr size).

Joerg

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
anish babu
2008-03-30 17:10:46 UTC
Permalink
Post by David Young
What data type will you use for
the tokens? How do PF, ALTQ, and drivers add tokens to the system?
When do you resolve class names to tokens?
I think unsigned long int can be used for tokens. Each of them
maintains a local table which stores the classes found so far and if a
new class is encountered , they call an API which registers the class
name and returns the corresponding token name. This token name can be
stored in local table for further access.
Post by David Young
What will you do if the kernel starts to label an mbuf with its class,
and that mbuf is already labelled?
If mbuf is used,a flag can be maintained in header, or for m_tags, we
can search for the corresponding tag type and then confirm. Right?
I am sorry if any of my answers look trivial as I have not completely
started working with code.
Post by David Young
How much more kernel space would you need to add to an mbuf? It seems
to me that there are trade-offs whether you enlarge the mbuf structure
or use an m_tag. What do you think?
On Sat, Mar 29, 2008 at 8:16 PM, Joerg Sonnenberger
Post by David Young
Having done exact the embedding of basic FW state and ALTQ options into
the pkthdr on DragonFly -- it is much simpler and faster, the only
difference is that it has to switch to clusters a bit earlier. That is
not a big deal if you ensure that you have at least 80 byte or so in the
first mbuf free (mbuf size - pkthdr size).
As suggested by Joerg, going for mbuf seems to be a better option to
me now. What you say?

Thanks for your valuable suggestions David and thank you Joerg
Anish

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Darren Reed
2008-04-08 09:26:24 UTC
Permalink
Post by anish babu
Hi,
My name is Anish and I am a graduate student at Texas A&M University.
I want to work on the project 'Create an in-kernel API for "packet
------------------------------
Create an in-kernel API for registering "packet classes" and for
labeling packets with their class, for special treatment by traffic
shapers (ALTQ) and by network interface drivers. With the registration
part of the API, ALTQ or a driver registers the names of packet
classes it recognizes. For example, ALTQ will register the
'class_name' part of a Class Command . An Ethernet NIC with high- and
low-priority transmit rings may register classes called 'hipri' and
'lopri'. An 802.11 NIC may register 802.11e traffic categories, BE,
BK, VI, VO. Each registration yields a token that is suitable for
labeling a packet - i.e., a small amount of data to stick in an mbuf
packet header or in an m_tag.
Make PF use the packet-classes API to convert PF tag names—see
pf.conf(5) for more about tags—to packet-class tokens, and to label
mbufs with the tokens as they exit PF. Make ALTQ extract the
packet-class tokens from mbufs and use them to select the
packet-scheduling class.
Since you are working in this space, I'd like to see the packet-classes
API be completely
independant of pf and pf tags - this may require more work from you to
change pf but if
this isn't achieved, then you haven't really got a "packet class API",
you've just moved
some of pf's code.

Further, to demonstrate that you've got the API correct, modify ipfilter
to filter on these
packet classes and do this without referencing anything that is pf-specific.

Darren


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
anish babu
2008-04-11 02:05:29 UTC
Permalink
Since you are working in this space, I'd like to see the packet-classes API
be completely
independant of pf and pf tags - this may require more work from you to
change pf but if
this isn't achieved, then you haven't really got a "packet class API",
you've just moved
some of pf's code.
Further, to demonstrate that you've got the API correct, modify ipfilter to
filter on these
packet classes and do this without referencing anything that is pf-specific.
Darren
Darren,

Yes You are right.But I have a doubt regarding implementation of this.

The main use of API is in 2 cases:

1) During the registration part of classes by ALTQ or driver : This
has nothing much to do with pf or pf_tags.

2)When PF calls this API to convert its tag name to token : I am not
sure how can I implement this mapping to find a corresponding token
for a tag which is pf specific.

Did I overlook something? Please let me know.

Thanks,
Anish

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Darren Reed
2008-04-11 08:43:27 UTC
Permalink
Post by anish babu
Since you are working in this space, I'd like to see the packet-classes API
be completely
independant of pf and pf tags - this may require more work from you to
change pf but if
this isn't achieved, then you haven't really got a "packet class API",
you've just moved
some of pf's code.
Further, to demonstrate that you've got the API correct, modify ipfilter to
filter on these
packet classes and do this without referencing anything that is pf-specific.
Darren
Darren,
Yes You are right.But I have a doubt regarding implementation of this.
1) During the registration part of classes by ALTQ or driver : This
has nothing much to do with pf or pf_tags.
2)When PF calls this API to convert its tag name to token : I am not
sure how can I implement this mapping to find a corresponding token
for a tag which is pf specific.
Did I overlook something? Please let me know.
Maybe the key to finding the tag isn't just <tagname> but
<tagname,consumer>,

Darren


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
anish babu
2008-04-11 16:14:12 UTC
Permalink
Post by anish babu
Post by Darren Reed
Since you are working in this space, I'd like to see the packet-classes
API
Post by anish babu
Post by Darren Reed
be completely
independant of pf and pf tags - this may require more work from you to
change pf but if
this isn't achieved, then you haven't really got a "packet class API",
you've just moved
some of pf's code.
Further, to demonstrate that you've got the API correct, modify
ipfilter to
Post by anish babu
Post by Darren Reed
filter on these
packet classes and do this without referencing anything that is pf-specific.
Darren
Darren,
Yes You are right.But I have a doubt regarding implementation of this.
1) During the registration part of classes by ALTQ or driver : This
has nothing much to do with pf or pf_tags.
2)When PF calls this API to convert its tag name to token : I am not
sure how can I implement this mapping to find a corresponding token
for a tag which is pf specific.
Did I overlook something? Please let me know.
Maybe the key to finding the tag isn't just <tagname> but
<tagname,consumer>,
Darren
I want to clarify one more thing on the project that I am working on -
'Create an in-kernal API for Packet Classes'. Instead of tagging the
packets by pf and then converting the tags to tokens, Is it not
possible to send the packets matching a particular rule set directly
to an output queue? I think it should work fine for queues created by
ALTQ but are there any issues with interface drivers?

Please let me know .

Thanks,
Anish

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Darren Reed
2008-04-11 18:55:17 UTC
Permalink
Post by anish babu
...
I want to clarify one more thing on the project that I am working on -
'Create an in-kernal API for Packet Classes'. Instead of tagging the
packets by pf and then converting the tags to tokens, Is it not
possible to send the packets matching a particular rule set directly
to an output queue? I think it should work fine for queues created by
ALTQ but are there any issues with interface drivers?
I think what you're referring to here is "policy based routing", where
(for instance),
you do a rule like this with ipfilter:

pass in on bge0 to fxp0:1.2.3.4 proto tcp from any to any port = 80

...and that bypasses the routing tables (yes) and queues the packet
directly on
fxp0 (yes.) Now if there are multiple output queues per interface, I'm
not sure
how you would address those using the above without some other means to
define what those were (is this ALTQ?)

Darren


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
anish babu
2008-04-11 22:06:35 UTC
Permalink
Post by anish babu
...
I want to clarify one more thing on the project that I am working on -
'Create an in-kernal API for Packet Classes'. Instead of tagging the
packets by pf and then converting the tags to tokens, Is it not
possible to send the packets matching a particular rule set directly
to an output queue? I think it should work fine for queues created by
ALTQ but are there any issues with interface drivers?
I think what you're referring to here is "policy based routing", where (for
instance),
pass in on bge0 to fxp0:1.2.3.4 proto tcp from any to any port = 80
...and that bypasses the routing tables (yes) and queues the packet
directly on
fxp0 (yes.) Now if there are multiple output queues per interface, I'm not
sure
how you would address those using the above without some other means to
define what those were (is this ALTQ?)
Darren
Rules in ALTQ:

altq on fxp0 cbq bandwidth 1Mb queue { ssh,ftp } // creates 2 child queues

queue ssh bandwidth 50 % cbq

queue ftp bandwidth 50% cbq

In pf:

pass out on fxp0 from any to any port 22 queue ssh. // send matching
packets to ssh queue on fxp0 interface

pass out on fxp0 from any to any port 21 queue ftp.



I think this rule set should redirect the packets to that particular
queue right?

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Darren Reed
2008-04-13 08:48:31 UTC
Permalink
Post by anish babu
Post by anish babu
...
I want to clarify one more thing on the project that I am working on -
'Create an in-kernal API for Packet Classes'. Instead of tagging the
packets by pf and then converting the tags to tokens, Is it not
possible to send the packets matching a particular rule set directly
to an output queue? I think it should work fine for queues created by
ALTQ but are there any issues with interface drivers?
I think what you're referring to here is "policy based routing", where (for
instance),
pass in on bge0 to fxp0:1.2.3.4 proto tcp from any to any port = 80
...and that bypasses the routing tables (yes) and queues the packet
directly on
fxp0 (yes.) Now if there are multiple output queues per interface, I'm not
sure
how you would address those using the above without some other means to
define what those were (is this ALTQ?)
Darren
altq on fxp0 cbq bandwidth 1Mb queue { ssh,ftp } // creates 2 child queues
queue ssh bandwidth 50 % cbq
queue ftp bandwidth 50% cbq
pass out on fxp0 from any to any port 22 queue ssh. // send matching
packets to ssh queue on fxp0 interface
pass out on fxp0 from any to any port 21 queue ftp.
I think this rule set should redirect the packets to that particular
queue right?
Yes and so long as it verifies that <fxp0,ssh> is a valid queue
definiion, everything is sensible.

Darren


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Loading...