Google Summer of Code Project: Port OpenAFS Client

[Matt Smith]

Hello,

I've been working on NetBSD for a while at Iowa State University and we have
been running Arla whenever we need AFS support on a client machine. Since
Arla does not yet work with the latest release of NetBSD, we have to use
either NetBSD 3 or 4, so I can see how porting the OpenAFS client over could
be immensely useful. Since I have dealt with both of these and their
interactions so much in the past, I have a lot of interest in seeing this
actually happen and there is no better way to get something accomplished
than to do it yourself.

I do have plenty of experience in using both of the systems, but I have
never actually delved very far into the actual programming aspects of either
of them so I do not know enough at the moment to be able to decide whether
using the new LKM interface or PUFFS would be better to implement the port.
I'm curious as to if anyone knows which would be better or even just the
benefits of using one over the other.

Additionally, I am semi-concerned about the scope of this project possibly
being too large to accomplish in a single summer (depending on what all is
involved), so it will definitely need to be broken down into a number of
different parts in order to decide what can actually be accomplished. I do
realize, however, that there is a FreeBSD client that might be useful.

Does anyone have any additional thoughts regarding this project beyond what
is just listed on the GSoC site?

Thanks,
Matt Smith

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Antti Kantee]

Hi,

Post by Matt Smith
Hello,
I've been working on NetBSD for a while at Iowa State University and we have
been running Arla whenever we need AFS support on a client machine. Since
Arla does not yet work with the latest release of NetBSD, we have to use
either NetBSD 3 or 4, so I can see how porting the OpenAFS client over could
be immensely useful. Since I have dealt with both of these and their
interactions so much in the past, I have a lot of interest in seeing this
actually happen and there is no better way to get something accomplished
than to do it yourself.
I do have plenty of experience in using both of the systems, but I have
never actually delved very far into the actual programming aspects of either
of them so I do not know enough at the moment to be able to decide whether
using the new LKM interface or PUFFS would be better to implement the port.
I'm curious as to if anyone knows which would be better or even just the
benefits of using one over the other.

I would never ever implement a network client in the kernel. It is
extremely difficult to do in the first place and hard to get right even
in very mature code (cf. our nfs client).

The performance advantage argument for kernel file systems is quite
laughable in 2010. Anyone who claims otherwise does not know how to
measure performance.

If done as a non-privileged server, your code will have "mostly usable"
value even if it crashes or locks up, say, once a day. In the kernel
it's not as much so. One of the most misunderstood aspects of systems
development these days is the unability to realize that code will never be
perfect; how fast you recover from catastrophic failures is what separates
a bad system from a good system (IMHO, of course. i could rant a lot
more about that one ;). That's not to say we shouldn't strive to write
code as well as we possibly can, just that we should accept the reality
of our limitations.

Post by Matt Smith
Additionally, I am semi-concerned about the scope of this project possibly
being too large to accomplish in a single summer (depending on what all is
involved), so it will definitely need to be broken down into a number of
different parts in order to decide what can actually be accomplished. I do
realize, however, that there is a FreeBSD client that might be useful.
Does anyone have any additional thoughts regarding this project beyond what
is just listed on the GSoC site?

I don't know anything about AFS, but speaking from the experience of
writing several network file system clients, I think the project is
quite doable within the summer (outside of the kernel). There are a
few steps I'd do it in:

0) do not cache anything before you *measure* the bottlenecks. there
are some "obvious" things to do, but do it the easy way first.
I'd say 99% of file system complexity comes from caching.

0) understand how a file system works. the key concepts are creating
nodes in the driver (with lookup/create/mkdir/mknod/symlink) and
removing that state (partially with inactive and ultimately with
reclaim). especially, understand what open/close is and what it
is *not*. you may (or may not) find my paper on this helpful:
http://2008.asiabsdcon.org/papers/P4B-paper.pdf

1) write a standalone program which can read directories and files from
an AFS server. this includes things like handshake and authentication,
which will remain mostly unchanged when you hook up your code with
the host OS fs hooks.

if there are existing programs which do this, you might consider
using those. however, weigh that against the educational experience.
you will have to understand the ins and outs of the code extremely
well for the next step.

2) hook the up the code from the previous step to the lookup/reclaim
life cycle (all the other ops mentioned in "0" are relevant only
for r/w file systems). be warned: apart from caching, this is the
hardest thing in a file system. discouragingly, you must do this
hard part first for anything to work. the good news is that when
it's done, you're almost finished

3) implement readdir and read (and open if it's necessary for AFS), mostly
by using code you wrote in "1". now you should be able to browse the
server contents.

4) implement write support in a similar fashion to "1". if you're feeling
brave, you can skip the standalone program step and implement it
directly in the file system driver. this is feasible especially if
working with a userspace client.

5) polish, document

6) measure bottlenecks and start working on performance improvements.
i think this is an optional goal for the summer.

Two things I'd like to ask (still keeping in mind I don't know anything
about AFS):

* Is there really no FUSE client? If there is one, what is the
relevance of this project?
* can the arla code (which was written against nnpfs) be used?

good luck ;)

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Greg Troxel]

I would like to see a Coda FUSE client, and then it would work on macs
too. The same would go for afs, either openafs or arla support in FUSE
(or puffs, if you need the added richness).

Even if you fix/port the openafs kernel support, you could run it under
rump/puffs to debug and then to actually run.

[Anders Magnusson]

Post by Antti Kantee
Hi,

Post by Matt Smith
Hello,
I've been working on NetBSD for a while at Iowa State University and we have
been running Arla whenever we need AFS support on a client machine. Since
Arla does not yet work with the latest release of NetBSD, we have to use
either NetBSD 3 or 4, so I can see how porting the OpenAFS client over could
be immensely useful. Since I have dealt with both of these and their
interactions so much in the past, I have a lot of interest in seeing this
actually happen and there is no better way to get something accomplished
than to do it yourself.
I do have plenty of experience in using both of the systems, but I have
never actually delved very far into the actual programming aspects of either
of them so I do not know enough at the moment to be able to decide whether
using the new LKM interface or PUFFS would be better to implement the port.
I'm curious as to if anyone knows which would be better or even just the
benefits of using one over the other.

I would never ever implement a network client in the kernel. It is
extremely difficult to do in the first place and hard to get right even
in very mature code (cf. our nfs client).

Hi pooka,

while I agree with your opinion in general, I think that the AFS client
is a special case
because it already exists and is written to fit in kernel. There is not
much glue code
needed to be written to make it run under NetBSD, compared to start from
scratch,
and there is the benefit of support of the common code from all other
platforms.

I myself would also like to see an AFS client work under NetBSD again,
independent
of whether it is Arla, OpenAFS or anything else.

-- Ragge

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Ken Hornstein]

Post by Anders Magnusson
while I agree with your opinion in general, I think that the AFS client
is a special case because it already exists and is written to fit in kernel.
There is not much glue code needed to be written to make it run under NetBSD,
compared to start from scratch, and there is the benefit of support of the
common code from all other platforms.
I myself would also like to see an AFS client work under NetBSD again,
independent of whether it is Arla, OpenAFS or anything else.

+1 to both.

--Ken

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Christos Zoulas]

Post by Antti Kantee

Post by Christos Zoulas
As for the performance of the server I think it depends on the workload
(how many kernel-userland transitions you end up doing, and how agressive
is your "attribute cache" to use NFS lingo).

While attribute caching in the file server does matter (network latency,
disk latency, what have you), I haven't stumbled across a workload where
lack of kernel attribute caching has any significant impact. I'm not
saying an application scenario doesn't exist, just that I haven't run
into one.

Any tree walkers... find, rdist, rsync, sup, tar, pax, cpio that do more
stat() than I/O under certain conditions (filtering etc.). Try to run:

find /amd/remote/root -type f -name slow -print

with /amd/ being an automounted directory and /remote/ a symlink to the nfs
mounted partition, and see amd's cpu going 100%. This is why I had to add
code to set the automounted directory attribute cache time. Even setting it
to 1 second fixes the problem. Having it 0 is a disaster.

christos


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Christos Zoulas]

Post by Anders Magnusson
Hi pooka,
while I agree with your opinion in general, I think that the AFS client
is a special case
because it already exists and is written to fit in kernel. There is not
much glue code
needed to be written to make it run under NetBSD, compared to start from
scratch,
and there is the benefit of support of the common code from all other
platforms.
I myself would also like to see an AFS client work under NetBSD again,
independent
of whether it is Arla, OpenAFS or anything else.

I agree with ragge too. The kernel code should be simple enough to fix,
very useful to have, and as pooka said it is easier to debug in userland.
As for the performance of the server I think it depends on the workload
(how many kernel-userland transitions you end up doing, and how agressive
is your "attribute cache" to use NFS lingo).

christos


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Antti Kantee]

Post by Christos Zoulas
As for the performance of the server I think it depends on the workload
(how many kernel-userland transitions you end up doing, and how agressive
is your "attribute cache" to use NFS lingo).

While attribute caching in the file server does matter (network latency,
disk latency, what have you), I haven't stumbled across a workload where
lack of kernel attribute caching has any significant impact. I'm not
saying an application scenario doesn't exist, just that I haven't run
into one.

ISTR Linux/FUSE implements kernel attribute caching and it's probably
trivial to add, but I haven't bothered doing it in NetBSD -- "when in
doubt, leave it out".

(ubc is different in this regard, mostly because of readahead support
and because it's easier to *not* have to tightrope with caching and
flushing gigabytes of data in several different caches. after all,
there is only one set of main memory)

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de