Discussion:
tftp protocol
(too old to reply)
Patrick Welche
2009-10-12 17:30:38 UTC
Permalink
For some reason a tftp client (not netbsd) keeps emitting an opcode
of 768 (oddly RFC 768 is UDP) - any idea on how an opcode can be > 5?

Cheers,

Patrick

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Ignatios Souvatzis
2009-10-12 19:05:08 UTC
Permalink
Post by Patrick Welche
For some reason a tftp client (not netbsd) keeps emitting an opcode
of 768 (oddly RFC 768 is UDP) - any idea on how an opcode can be > 5?
Yes. 768 is 3*256 - so this is wrong-endian opcode 3, a DATA request.
Somebody should fix the client, unless you're misreading a packet
dump.

-is
--
seal your e-mail: http://www.gnupg.org/

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Patrick Welche
2009-10-13 13:07:09 UTC
Permalink
Post by Ignatios Souvatzis
Post by Patrick Welche
For some reason a tftp client (not netbsd) keeps emitting an opcode
of 768 (oddly RFC 768 is UDP) - any idea on how an opcode can be > 5?
Yes. 768 is 3*256 - so this is wrong-endian opcode 3, a DATA request.
Somebody should fix the client, unless you're misreading a packet
dump.
Thank you! (The client is windows vista pxeboot - I'll have to put a
work around in tftpd - makes me wonder what else is wrong-endian - tftpd
server is NetBSD/amd64, client is 32bit vista)

Patrick

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Patrick Welche
2009-10-13 14:10:00 UTC
Permalink
If it is PXE boot, then it's likely a BIOS thing. Perhaps you can find
a BIOS update for the hardware?
I think it is more likely to be one of the "Windows AIK" files. One of
pxeboot.n12, bootmgr.exe, boot.sdi - the trouble happens while the .wim
file is being transfered - I assume that it is one of them that is making
the read request?

Cheers,

Patrick

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Patrick Welche
2009-10-14 14:34:16 UTC
Permalink
Let us know how you come out.
The story so far:

I did a BIOS update - the release notes didn't say what the update changed
so who knows what that was good for (Toshiba). (No change in behaviour.)

I stopped receiving the strange 768 opcode with the following patch:

@@ -924,7 +939,7 @@
dp = r_init();
goto done;
}
- if (ap->th_block == block)
+ if (ap->th_block == (u_short)block)
goto done;
if (debug)
syslog(LOG_DEBUG, "Resync ACK %u != %u",

(What are you meant to do if the block number no longer fits in 2 bytes?)
(line numbers may be slightly different as I also have the -p flag patch
applied.)

One oddity:

14:01:42.703489 IP 10.111.52.39.6911 > 10.111.52.92.tftp: 32 RRQ "\Boot\winpe.wim" octet tsize 0
0x0000: 4500 003c 0169 0000 1411 27e8 0a6f 3427
0x0010: 0a6f 345c 1aff 0045 0028 2cc0 0001 5c42
0x0020: 6f6f 745c 7769 6e70 652e 7769 6d00 6f63
0x0030: 7465 7400 7473 697a 6500 3000
14:01:42.706136 IP 10.111.52.92.49376 > 10.111.52.39.6911: 18 tftp-#6
0x0000: 4500 002e 0000 0000 4011 0000 0a6f 345c
0x0010: 0a6f 3427 c0e0 1aff 001a 7d8c 0006 7473
0x0020: 697a 6500 3230 3136 3134 3539 3600
14:01:42.706169 IP 10.111.52.39.6911 > 10.111.52.92.49376: 17 ERROR EUNDEF TFTP Aborted"
0x0000: 4500 002d 016a 0000 1411 27f6 0a6f 3427
0x0010: 0a6f 345c 1aff c0e0 0019 a356 0005 0000
0x0020: 5446 5450 2041 626f 7274 6564 0000

Obviously .39 is the client, .92 is the NetBSD tftpd. Why would tftpd emit
a packet with opcode 6?

The next successful transfer also has an opcode 6 initial reply:

14:01:42.709212 IP 10.111.52.39.6913 > 10.111.52.92.tftp: 44 RRQ "\Boot\boot.sdi" octet tsize 0 blksize 1420
0x0000: 4500 0048 016d 0000 1411 27d8 0a6f 3427
0x0010: 0a6f 345c 1b01 0045 0034 4b45 0001 5c42
0x0020: 6f6f 745c 626f 6f74 2e73 6469 006f 6374
0x0030: 6574 0074 7369 7a65 0030 0062 6c6b 7369
0x0040: 7a65 0031 3432 3000
14:01:42.711909 IP 10.111.52.92.49370 > 10.111.52.39.6913: 29 tftp-#6
0x0000: 4500 0039 0000 0000 4011 0000 0a6f 345c
0x0010: 0a6f 3427 c0da 1b01 0025 7d97 0006 626c
0x0020: 6b73 697a 6500 3134 3230 0074 7369 7a65
0x0030: 0033 3137 3033 3034 00
14:01:42.711942 IP 10.111.52.39.6913 > 10.111.52.92.49370: 4 ACK block 0
0x0000: 4500 0020 016e 0000 1411 27ff 0a6f 3427
0x0010: 0a6f 345c 1b01 c0da 000c a695 0004 0000
0x0020: 0000 0000 0000 0000 0000 0000 0000
14:01:42.711977 IP 10.111.52.92.49370 > 10.111.52.39.6913: 1424 DATA block 1
etc.

but this time the client doesn't treat it as an error.

Cheers,

Patrick

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Patrick Welche
2009-10-14 15:01:13 UTC
Permalink
On Wed, Oct 14, 2009 at 03:34:16PM +0100, Patrick Welche wrote:
...
Post by Patrick Welche
Obviously .39 is the client, .92 is the NetBSD tftpd. Why would tftpd emit
a packet with opcode 6?
Looks like tsize negotiation - it looks as though the client sends tsize 0
with the RRQ, then our opcode 6(?) offers a large tsize which client
responds to with an error, then the client sends a RRQ with tsize 0, blksize
1420, and the server replies with opcode 6 blksize 1420 and the same large
tsize as before. Block number is bound to roll over in this particular
large file case...

Cheers,

Patrick

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Patrick Welche
2009-10-15 11:05:10 UTC
Permalink
Post by Patrick Welche
Looks like tsize negotiation - it looks as though the client sends tsize 0
with the RRQ, then our opcode 6(?) offers a large tsize which client
responds to with an error, then the client sends a RRQ with tsize 0, blksize
1420, and the server replies with opcode 6 blksize 1420 and the same large
tsize as before. Block number is bound to roll over in this particular
large file case...
and of course this is all described in the fine manual:

BUGS
Files larger than 33488896 octets (65535 blocks) cannot be transferred
without client and server supporting blocksize negotiation (RFCs 2347 and
2348).

and opcode 6 is OACK as defined in RFC 2347.

The attached patch to tcpdump would have made this more obvious to me.

Cheers,

Patrick
Havard Eidnes
2009-10-16 20:56:55 UTC
Permalink
Post by Patrick Welche
Post by Patrick Welche
Looks like tsize negotiation - it looks as though the client sends tsize 0
with the RRQ, then our opcode 6(?) offers a large tsize which client
responds to with an error, then the client sends a RRQ with tsize 0, blksize
1420, and the server replies with opcode 6 blksize 1420 and the same large
tsize as before. Block number is bound to roll over in this particular
large file case...
BUGS
Files larger than 33488896 octets (65535 blocks) cannot be transferred
without client and server supporting blocksize negotiation (RFCs 2347 and
2348).
Actually... I suspect that the BUGS statement is in actual fact
wrong, if it implies that the protocol spec makes it impossible
to transfer files larger than 32MiB using tftp with the default
block size of 512 bytes. I've experienced that same problem,
when trying to tftp updates for Cisco wireless controllers, which
also have image files larger than 32MiB.

Cisco's "fix" for this problem is to point to a particular
implementation of tftp daemon functionality in the form of a
Windows program (yuk!) (and, yes, the source is available as
well, but it has Windows-specific code littering the code, if I
recall correctly). The tftp client in the Cisco case uses the
default block size of 512 bytes, and with this combination, it
can actually transfer the wireless controller image file.

The "block numbers" in the tftp protocol are actually not
designed to do "random access" in the server-side file, it's
designed to distinguish between duplicate blocks and new blocks
at the client. If the client and server implements a 0.5 * 2^16
"block number window", and they allow the block number to wrap
around the 2^16 mark, it may actually work to transfer files
larger than 32MiB without increasing the block size above the
default 512 bytes.

The RFCs specifying the block size extension mention speed as a
motivation for the extension, not the ability to transfer files
larger than 32MiB...

I've not actually had the time to sit down to see if a patch
could be made to the NetBSD tftp daemon (and client) to allow
such operation.


Regards,

- HÃ¥vard

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
der Mouse
2009-10-17 01:22:15 UTC
Permalink
Post by Havard Eidnes
Post by Patrick Welche
BUGS
Files larger than 33488896 octets (65535 blocks) cannot be
transferred without client and server supporting blocksize
negotiation (RFCs 2347 and 2348).
Actually... I suspect that the BUGS statement is in actual fact
wrong, if it implies that the protocol spec makes it impossible to
transfer files larger than 32MiB using tftp with the default block
size of 512 bytes.
I don't suppose you considered actually checking whether this was true
or not?

My reading is that there is a limit, of approximately 32 megs, but not
quite any of the above figures. DATA and ACK packets have only two
octets of block number, but block numbers begin with 1, not 0 (block
number 0 is used for acking write requests); also, the final data
packet must have <512 octets, so the limit is actually 33553919 octets.

The number in the quoted manpage excerpt is wrong. 33488896 is
512*(65536-128), which seems weird, until you notice it is also
511*65536; I suspect someone just got careless about which number 1
needed to be subtracted from.

In principle there is no reason the block-number field in the protocol
couldn't be the file block number modulo 65536. But that's not how I
read RFC1350; it is silent on the subject, saying only that the block
numbers "begin with one and increase by one for each new block of
data". Since a two-octet field cannot increase past 65535, this means,
logically speaking, that there cannot be a next block after block
65535.
Post by Havard Eidnes
The "block numbers" in the tftp protocol are actually not designed to
do "random access" in the server-side file, it's designed to
distinguish between duplicate blocks and new blocks at the client.
What's your basis for this statement? I don't see anything in 1350
even hinting at the intent behind the block-number field, and I'm
wondering what I've missed.

/~\ The ASCII Mouse
\ / Ribbon Campaign
X Against HTML ***@rodents-montreal.org
/ \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Robert Elz
2009-10-17 02:25:55 UTC
Permalink
Date: Fri, 16 Oct 2009 22:56:55 +0200 (CEST)
From: Havard Eidnes <***@uninett.no>
Message-ID: <***@uninett.no>

| Actually... I suspect that the BUGS statement is in actual fact
| wrong, if it implies that the protocol spec makes it impossible
| to transfer files larger than 32MiB using tftp with the default
| block size of 512 bytes.

Which is correct, it does, it just doesn't say it explicitly - but there
is nothing in rfc738 to suggest that it is possible for block numbers to
wrap around.

| The "block numbers" in the tftp protocol are actually not
| designed to do "random access" in the server-side file, it's
| designed to distinguish between duplicate blocks and new blocks
| at the client.

This is an assumption, nothing in rfc738 says that. But even assuming
what you say were to be correct, then that doesn't allow block numbers
to be reused, if they are, tftp has no long delayed dup packet protection.

That is, consider a data packet N, transmitted, and "lost" somewhere in the
network (queued, for a long time). IP (v4) allows that to be up to 4.25
minutes (if the original TTL were 255, which it might be, TFTP explicitly
says it doesn't constrain any of the values of the IP header). The
intended recipient of that packet fails to get it within a reasonable time
(few seconds) and requests a retransmit, which happens. After that
all the remaining packets flow again, until we're just about to reuse block
number N, when that long delayed packet shows up, containing block number
N, just after the (re-used) block number N-1 was received. How is the
recipient supposed to know it is a duplicate, and wait for the other copy
of the next-time around block N instead?

The probability of this happening might be low, perhaps low enough for cisco
to get away with a gross hack, but it certainly isn't the TFTP protocol.

(Aside: this is certainly possible - with a RTT of 1ms (allows a few typical
ethernet hops) 65535 blocks take just 65.6 seconds to transfer, wrap around,
if permitted, would occur well within the possible lifetime of a delayed
IP packet.)

What's more, wraparound would require another invention - that is, what
number comes next after 65535? The spec says "block numbers on data packets
begin with one and increase by one for each new block" - zero is only
used as an ACK of a WRQ ... if the sequence is 65534 65535 0 1 2 then you've
just used a block number the spec says doesn't exist, if the sequence is
65534 65535 1 2 3 then you've just been very creative with the concept of
wraparound, and you wouldn't expect that to be implementable without being
very carefully spelled out in the spec.

Next, think to what the original intent was - tftp was invented in the late
1970's (IEN-133 is dated January 1980, so it certainly existed before that.)
Back then, it was uncommon for an entire filestore to be much bigger than
32MB ("huge" 300MB disk drives were not all that long available.) To
even imagine that anyone was concerned about the need to transfer a single
file > 32MB using this protocol, and consequently designed it to be able
to do that, is absurd.

Lastly, using TFTP to transfer files this big is not rational, even today.
TFTP is a very slow file transfer protocol (512 bytes/RTT max, regardless of
available bandwidth). If you're going to be transferring very big files
(like cisco IOS images) it would make far more sense to use TFTP to
download a simple file copy application (including a TCP implementation)
and then use that to transfer the big file - even with the extra overhead
of downloading the TTCP and file transfer protocol (which could be anything)
that's going to result in a faster, and more reliable, transfer mechanism
than using TFTP file files this big.

That's why ...

| The RFCs specifying the block size extension mention speed as a
| motivation for the extension, not the ability to transfer files
| larger than 32MiB...

as if you make the blocks twice as big, TFTP runs twice as fast, which
is useful, being able to transfer files bigger than 32MB just wasn't
an objective.

kre

ps: if we need more confirmation of this, we can just ask Noel Chiappa, who
is credited in rfc738 as the inventor of TFTP.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Michael van Elst
2009-10-17 07:29:21 UTC
Permalink
Post by Havard Eidnes
Actually... I suspect that the BUGS statement is in actual fact
wrong, if it implies that the protocol spec makes it impossible
to transfer files larger than 32MiB using tftp with the default
block size of 512 bytes.
From TCP/IP Illustrated:

15.2 What do you think happens when the TFTP block number wraps around from 65535 to 0? Does RFC 1350 say anything about this?

15.2 Unfortunately, the RFC says nothing about this block number wrap. Implementations should be able to transfer files up through 33,553,920 bytes (65535 x 512). Many implementations fail when the size of the file exceeds 16,776,704 (32767 x 512) since they incorrectly maintain the block number as a signed 16-bit integer instead of an unsigned integer.
--
--
Michael van Elst
Internet: ***@serpens.de
"A potential Snark may lurk in every tree."

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Michael van Elst
2009-10-17 07:36:10 UTC
Permalink
Post by Robert Elz
as if you make the blocks twice as big, TFTP runs twice as fast, which
is useful, being able to transfer files bigger than 32MB just wasn't
an objective.
There is nothing that forbids transfers of larger files, but there
is nothing either that guarantees interoperability.

http://www.compuphase.com/tftp.htm

has a short summary.
--
--
Michael van Elst
Internet: ***@serpens.de
"A potential Snark may lurk in every tree."

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Manuel Bouyer
2009-10-17 16:33:03 UTC
Permalink
Post by Havard Eidnes
Actually... I suspect that the BUGS statement is in actual fact
wrong, if it implies that the protocol spec makes it impossible
to transfer files larger than 32MiB using tftp with the default
block size of 512 bytes. I've experienced that same problem,
when trying to tftp updates for Cisco wireless controllers, which
also have image files larger than 32MiB.
Cisco's "fix" for this problem is to point to a particular
implementation of tftp daemon functionality in the form of a
Windows program (yuk!) (and, yes, the source is available as
well, but it has Windows-specific code littering the code, if I
recall correctly). The tftp client in the Cisco case uses the
default block size of 512 bytes, and with this combination, it
can actually transfer the wireless controller image file.
The "block numbers" in the tftp protocol are actually not
designed to do "random access" in the server-side file, it's
designed to distinguish between duplicate blocks and new blocks
at the client. If the client and server implements a 0.5 * 2^16
"block number window", and they allow the block number to wrap
around the 2^16 mark, it may actually work to transfer files
larger than 32MiB without increasing the block size above the
default 512 bytes.
The RFCs specifying the block size extension mention speed as a
motivation for the extension, not the ability to transfer files
larger than 32MiB...
I've not actually had the time to sit down to see if a patch
could be made to the NetBSD tftp daemon (and client) to allow
such operation.
FWIW, we had to fix a similar issue to tftp a large file from
the windows vista installation. Konstantin Kabassavov came up
with the attached patch; I didn't check it in details.
It also handle '\' vs '/'.
--
Manuel Bouyer <***@antioche.eu.org>
NetBSD: 26 ans d'experience feront toujours la difference
--
Miles Nordin
2009-10-18 03:41:52 UTC
Permalink
re> How is the recipient supposed to know it is a duplicate, and
re> wait for the other copy of the next-time around block N
re> instead?

embed a crc in the downloaded file and reboot if it doesn't verify. :)

re> think to what the original intent was - tftp was invented in
re> the late 1970's

re> ps: if we need more confirmation of this, we can just ask Noel
re> Chiappa, who is credited in rfc738 as the inventor of TFTP.

are you designing a functioning museum piece, or a tftp server? It is
a serious question with two right answers.

re> Lastly, using TFTP to transfer files this big is not rational,
re> even today.

agreed! They could just use NFSv2 like a NetBSD stage2---no need
even to resort to TCP. but what they should have done is maybe not a
good reason to stubbornly desupport it, given actual experience with
clients that exist (i guess windows pxe booting and some cisco access
point, so far).

i've been using linux tftpd-hpa because the netbsd tftpd didn't work
with the client inside the alpha XP1000 srm. That's a Digital
proprietary client, and of course among NetBSD, Digital can do no
wrong, correct? IIRC it was some options-negotiation garbage. :/ I
don't really care about fixing it, but the point is, it might be
appropriate to intentionally allow the tftp server to be a repository
of historical kludges for dealing with the various weird client
implementations people have encountered over the years. Often when
one is using a tftp server, the client is proprietary software. Even
as a museum piece, recording these kludges is probably a more worthy
goal than perfecting the elegance of the Great Noel Chiappa's
masterful 1970's Final Solution to the File Transfer Problem.
Robert Elz
2009-10-18 05:53:52 UTC
Permalink
Date: Sat, 17 Oct 2009 23:41:52 -0400
From: Miles Nordin <***@Ivy.NET>
Message-ID: <***@castrovalva.Ivy.NET>

| but what they should have done is maybe not a
| good reason to stubbornly desupport it,

It isn't a question of de-supporting something, but of how can it
possibly be supported when we don't know what it is?

If there was one agreed way that everyone used to transfer large
files (bigger than tftp was designed to support, even with the large
block extension), that would be fine, but there isn't.

We can't support all of the kludges that exist, as some of them
are incompatible with each other - I certainly agree than when faced
with a weird tftp client, the usual (and often only rational) answer
is to obtain, or create, a tftp server that interoperates with that
client. But NetBSD can't do that, we have no idea what weird clients
the various users will need to support, and have no rational reason to
prefer one particular set of weird clients over another. All that
is rational to do (in the base system) is to support the part of tftp
that is more or less standard, and forget all the rest.

On the other hand, it is perfectly reasonable to create pkgsrc packages
for any other tftpd variant that you can find, or create, so that people
who have one of the weird clients can more easily handle it. It might
even be reasonable to create a tftpd multiplexor, that would receive
the RRQ/WRQ initial packets, and based upon (perhaps) the OUI in the MAC
address (assuming there is one, and the client is on the local LAN) of
the client, or perhaps its IP address, or even the file name it is
requesting, then exec a suitable tftpd process to handle the request
in whatever weird way that client needs - that could also go in pkgsrc...

kre


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Miles Nordin
2009-10-18 07:34:06 UTC
Permalink
re> If there was one agreed way that everyone used to transfer
re> large files (bigger than tftp was designed to support, even
re> with the large block extension), that would be fine, but there
re> isn't.

okay.

re> We can't support all of the kludges that exist, as some of
re> them are incompatible with each other

which ones?

re> All that is rational to do (in the base system) is to support
re> the part of tftp that is more or less standard, and forget all
re> the rest.

re> On the other hand, it is perfectly reasonable to create pkgsrc
re> packages [...]

to whatever extent the incompatible kludge problem is real, ``maybe''.
but so far I think the problem is hypothetical, and we've already got
a patch posted for windows pxe that fixes one bogus client and
probably breaks nothing, which your reasoning says shouldn't be
committed---that's the part I disagree with.

I think you secretly want to boot the tftpd with kludges out to pkgsrc
for taxonomy reasons. But maintaining two tftpd's seems really
unlikely to happen. Maintaining one is hard enough, the care factor
is so low with this stuff---I know myself, you just bang on it until
it works then walk away, because the clients are all buggy proprietary
flash-in-the-pan junk---but this means everyone has to reinvent the
wheel and realize the broken piece is tftpd, something you thought was
too simple to break. I predict someone will read what I'm writing in
2014 and find that working windows pxe tftp large file patch posted
yesterday still languishing in these mail archives, and he will add it
manually to his local branch.

To that person I would suggest: use tftpd-hpa instead. It's Linux's
fork of the OpenBSD tftpd, and it's maintained. it works with alpha
srm firmware in at least one case where netbsd's didn't. It has a
rollover option so that not only will it send >32MB files, but you can
specify whether you want the block counter to roll over to 1 or to 0,
so you can try both with your client and see which one works---I guess
they found an incompatbile aspect to the kludgery after all. I've not
tried it, but by 2014 it will probably include other important kludges
the NetBSD tftpd lacks.
Patrick Welche
2010-01-06 18:42:56 UTC
Permalink
The tftpd discussion was a while ago now. In brief, I had a strange
"opcode 768" error when tftp'ing a large file. This was because
the block number of the transfer overflowed its 16bit integer. This
wasn't obvious to me because the error was obscure. I propose in
the attached patch bailing out and printing a sensible error message
on overflow. It also wasn't obvious because our tcpdump doesn't
know about tftp options, and I attached a patch already in this
thread, which will become redundant once the in-tree tcpdump is
upgraded as tcpdump has learnt about tftp options in the meantime.

The other part was how to deal with clients which use a different path
separator to unix, and I proposed a -p option to tftpd (enclosed once
more). (I prefer this to the one Manuel passed on which swapped all \
for / during connection phase rather than just the filename.)

The number of blocks mentioned in the BUGS section was pointed out to
be incorrect, and that is amended in the attached patch (hopefully
correctly this time, but correct me if I'm wrong!)


We could reopen the argument on whether bailing with an error is better
than choosing to overflow to 0 (with the "block 0" is special for option
acknowledgement problem) or to 1 (I haven't actually seen a client do this).
Neither of those options is "right".

BTW that (u_short) patch I had sent in this thread is wrong, as it
effectively enabled overflow to 0 but didn't treat the not-first block 0 in
the same way as other blocks. I think the patch Manuel passed on did this
correctly.

Thoughts?

Cheers,

Patrick
John Nemeth
2010-01-07 14:28:06 UTC
Permalink
On May 29, 1:18pm, Patrick Welche wrote:
}
} The tftpd discussion was a while ago now. In brief, I had a strange
} "opcode 768" error when tftp'ing a large file. This was because
} the block number of the transfer overflowed its 16bit integer. This
} wasn't obvious to me because the error was obscure. I propose in
} the attached patch bailing out and printing a sensible error message
} on overflow. It also wasn't obvious because our tcpdump doesn't
} know about tftp options, and I attached a patch already in this
} thread, which will become redundant once the in-tree tcpdump is
} upgraded as tcpdump has learnt about tftp options in the meantime.
}
} The other part was how to deal with clients which use a different path
} separator to unix, and I proposed a -p option to tftpd (enclosed once
} more). (I prefer this to the one Manuel passed on which swapped all \
} for / during connection phase rather than just the filename.)

Whilst I think the client should be fixed, this option is
non-contentious. The only thing is I would check to see if any of the
other BSDs supports and use the same option letter.

} The number of blocks mentioned in the BUGS section was pointed out to
} be incorrect, and that is amended in the attached patch (hopefully
} correctly this time, but correct me if I'm wrong!)
}
} We could reopen the argument on whether bailing with an error is better
} than choosing to overflow to 0 (with the "block 0" is special for option
} acknowledgement problem) or to 1 (I haven't actually seen a client do this).
} Neither of those options is "right".

The block counter should wrap as is done by both FreeBSD and
OpenBSD. This is most likely necessary to support downloading large
files on certain devices (i.e. Cisco devices running IOS). I need to
further investigate this, but will most likely be implementing this
change.

} BTW that (u_short) patch I had sent in this thread is wrong, as it
} effectively enabled overflow to 0 but didn't treat the not-first block 0 in
} the same way as other blocks. I think the patch Manuel passed on did this
} correctly.
}
} Index: tftpd.c
} ===================================================================
} RCS file: /cvsroot/src/libexec/tftpd/tftpd.c,v
} retrieving revision 1.32
} diff -u -r1.32 tftpd.c
} --- tftpd.c 16 Mar 2009 01:56:21 -0000 1.32
} +++ tftpd.c 5 Jan 2010 16:14:30 -0000
} @@ -107,6 +107,7 @@
} static int suppress_naks;
} static int logging;
} static int secure;
} +static char pathsep = '\0';
} static char *securedir;
}
} struct formats;
} @@ -141,7 +142,7 @@
} {
}
} syslog(LOG_ERR,
} - "Usage: %s [-dln] [-u user] [-g group] [-s directory] [directory ...]",
} + "Usage: %s [-dln] [-u user] [-g group] [-s directory] [-p pathsep] [directory ...]",
} getprogname());
} exit(1);
} }
} @@ -170,7 +171,7 @@
} curuid = getuid();
} curgid = getgid();
}
} - while ((ch = getopt(argc, argv, "dg:lns:u:")) != -1)
} + while ((ch = getopt(argc, argv, "dg:lnp:s:u:w:")) != -1)
} switch (ch) {
} case 'd':
} debug++;
} @@ -188,6 +189,11 @@
} suppress_naks = 1;
} break;
}
} + case 'p':
} + pathsep = optarg[0];
} + if (optarg[1] != '\0' || optarg[0] == '\0') usage();
} + break;
} +
} case 's':
} secure = 1;
} securedir = optarg;
} @@ -662,6 +668,15 @@
} exit(1);
} }
} }
} + /*
} + * Globally replace the path separator given in the -p option
} + * with / to cope with clients expecting a non-unix path separator.
} + */
} + if (pathsep != '\0') {
} + for (cp = filename; *cp != '\0'; ++cp) {
} + if (*cp == pathsep) *cp = '/';
} + }
} + }
} ecode = (*pf->f_validate)(&filename, tp->th_opcode);
} if (logging) {
} syslog(LOG_INFO, "%s: %s request for %s: %s",
} @@ -853,7 +868,7 @@
} case OACK:
} return "OACK";
} default:
} - (void)snprintf(obuf, sizeof(obuf), "*code %d*", code);
} + (void)snprintf(obuf, sizeof(obuf), "*code 0x%x*", code);
} return obuf;
} }
} }
} @@ -882,6 +897,11 @@
} }
}
} do {
} + if (block > UINT16_MAX) {
} + syslog(LOG_ERR, "tftpd: block number wrapped (hint: increase block size)");
} + nak(EBADOP);
} + goto abort;
} + }
} if (block > 0) {
} size = readit(file, &dp, tftp_blksize, pf->f_convert);
} if (size < 0) {
}

Why are you only handling the sendfile case and not the recvfile
case?

}-- End of excerpt from Patrick Welche

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Patrick Welche
2010-01-08 17:29:44 UTC
Permalink
Post by John Nemeth
Whilst I think the client should be fixed, this option is
non-contentious. The only thing is I would check to see if any of the
other BSDs supports and use the same option letter.
Free/Open don't use 'p', and don't offer similar functionality.
Post by John Nemeth
The block counter should wrap as is done by both FreeBSD and
OpenBSD. This is most likely necessary to support downloading large
files on certain devices (i.e. Cisco devices running IOS). I need to
further investigate this, but will most likely be implementing this
change.
The attached patch does wrap to zero (successfully booted a vista image, and
the wrap warning appeared in the logs.

Cheers,

Patrick
John Nemeth
2010-01-08 21:57:44 UTC
Permalink
On May 31, 12:05pm, Patrick Welche wrote:
} On Thu, Jan 07, 2010 at 06:28:06AM -0800, John Nemeth wrote:
} > On May 29, 1:18pm, Patrick Welche wrote:
} > Whilst I think the client should be fixed, this option is
} > non-contentious. The only thing is I would check to see if any of the
} > other BSDs supports and use the same option letter.
}
} Free/Open don't use 'p', and don't offer similar functionality.

Good enough.

} > The block counter should wrap as is done by both FreeBSD and
} > OpenBSD. This is most likely necessary to support downloading large
} > files on certain devices (i.e. Cisco devices running IOS). I need to
} > further investigate this, but will most likely be implementing this
} > change.
}
} The attached patch does wrap to zero (successfully booted a vista image, and
} the wrap warning appeared in the logs.
}
} Index: tftpd.8
} ===================================================================
} RCS file: /cvsroot/src/libexec/tftpd/tftpd.8,v
} retrieving revision 1.21
} diff -u -r1.21 tftpd.8
} --- tftpd.8 7 Aug 2003 09:46:53 -0000 1.21
} +++ tftpd.8 8 Jan 2010 17:16:46 -0000
} @@ -29,7 +29,7 @@
} .\"
} .\" from: @(#)tftpd.8 8.1 (Berkeley) 6/4/93
} .\"
} -.Dd June 11, 2003
} +.Dd November 13, 2009

This will get changed to the actual commit date.

} .Dt TFTPD 8
} .Os
} .Sh NAME
} @@ -43,6 +43,7 @@
} .Op Fl g Ar group
} .Op Fl l
} .Op Fl n
} +.Op Fl p Ar path separator
} .Op Fl s Ar directory
} .Op Fl u Ar user
} .Op Ar directory ...
} @@ -107,6 +108,11 @@
} .It Fl n
} Suppresses negative acknowledgement of requests for nonexistent
} relative filenames.
} +.It Fl p Ar path separator
} +All occurances of the single character
} +.Ar path separator
} +in the requested filename are replaced with
} +.Sq / .
} .It Fl s Ar directory
} .Nm
} will
} @@ -193,11 +199,15 @@
} and first appeared in
} .Nx 2.0 .
} .Sh BUGS
} -Files larger than 33488896 octets (65535 blocks) cannot be transferred
} -without client and server supporting blocksize negotiation (RFCs
} -2347 and 2348).
} -.Pp
} -Many tftp clients will not transfer files over 16744448 octets (32767 blocks).
} +Files larger than 33,553,919 octets (65535 blocks, last one <512
} +octets) cannot be correctly transferred without client and server
} +supporting blocksize negotiation (RFCs 2347 and 2348). As a kludge,
} +.Nm
} +accepts a sequence of block numbers which wrap to zero after 65535.

I'm not sure I'm crazy about the wording here, but I'm not sure
how I would fix it up. I would probably make it a couple of sentences
and say something about legacy clients that don't support options.

} +.Pp
} +Many tftp clients will not transfer files over 16,776,703 octets
} +(32767 blocks), as they incorrectly count the block number using
} +a signed rather than unsigned 16-bit integer.
} .Sh SECURITY CONSIDERATIONS
} You are
} .Em strongly
} Index: tftpd.c
} ===================================================================
} RCS file: /cvsroot/src/libexec/tftpd/tftpd.c,v
} retrieving revision 1.32
} diff -u -r1.32 tftpd.c
} --- tftpd.c 16 Mar 2009 01:56:21 -0000 1.32
} +++ tftpd.c 8 Jan 2010 17:16:47 -0000
} @@ -107,6 +107,7 @@
} static int suppress_naks;
} static int logging;
} static int secure;
} +static char pathsep = '\0';
} static char *securedir;
}
} struct formats;
} @@ -141,7 +142,7 @@
} {
}
} syslog(LOG_ERR,
} - "Usage: %s [-dln] [-u user] [-g group] [-s directory] [directory ...]",
} + "Usage: %s [-dln] [-u user] [-g group] [-s directory] [-p pathsep] [directory ...]",
} getprogname());
} exit(1);
} }
} @@ -170,7 +171,7 @@
} curuid = getuid();
} curgid = getgid();
}
} - while ((ch = getopt(argc, argv, "dg:lns:u:")) != -1)
} + while ((ch = getopt(argc, argv, "dg:lnp:s:u:w:")) != -1)

What's this -w option?

} switch (ch) {
} case 'd':
} debug++;
} @@ -188,6 +189,11 @@
} suppress_naks = 1;
} break;
}
} + case 'p':
} + pathsep = optarg[0];
} + if (optarg[1] != '\0' || optarg[0] == '\0') usage();

usage() should be on the next line to make the code easier to read.

} + break;
} +
} case 's':
} secure = 1;
} securedir = optarg;
} @@ -662,6 +668,15 @@
} exit(1);
} }
} }
} + /*
} + * Globally replace the path separator given in the -p option
} + * with / to cope with clients expecting a non-unix path separator.
} + */
} + if (pathsep != '\0') {
} + for (cp = filename; *cp != '\0'; ++cp) {
} + if (*cp == pathsep) *cp = '/';
} + }
} + }
} ecode = (*pf->f_validate)(&filename, tp->th_opcode);
} if (logging) {
} syslog(LOG_INFO, "%s: %s request for %s: %s",
} @@ -853,7 +868,7 @@
} case OACK:
} return "OACK";
} default:
} - (void)snprintf(obuf, sizeof(obuf), "*code %d*", code);
} + (void)snprintf(obuf, sizeof(obuf), "*code 0x%x*", code);
} return obuf;
} }
} }
} @@ -918,20 +933,20 @@
} goto abort;
}
} case ACK:
} - if (ap->th_block == 0) {
} + if (etftp && ap->th_block == 0) {
} etftp = 0;
} acklength = 0;
} dp = r_init();
} goto done;
} }
} - if (ap->th_block == block)
} + if (ap->th_block == (u_short)block)

I would use uint16_t to make it clear what size is wanted. If you
need a specific size, it's much better to specify it then make
assumptions about what size a type is.

} goto done;
} if (debug)
} syslog(LOG_DEBUG, "Resync ACK %u != %u",
} (unsigned int)ap->th_block, block);
} /* Re-synchronize with the other side */
} (void) synchnet(peer, tftp_blksize);
} - if (ap->th_block == (block -1))
} + if (ap->th_block == (u_short)(block - 1))

Same thing, use uint16_t.

} goto send_data;
} default:
} syslog(LOG_INFO, "Received %s in sendfile\n",
} @@ -942,6 +957,9 @@
} done:
} if (debug)
} syslog(LOG_DEBUG, "Received ACK for block %u", block);
} + if (block == UINT16_MAX && size == tftp_blksize)
} + syslog(LOG_WARNING,
} + "Block number wrapped (hint: increase block size)");

Don't bother with the hint since the client is usually out of the
sysadmin's control. Maybe also use LOG_DEBUG since this is normal
operation for the clients that do it.

} block++;
} } while (size == tftp_blksize || block == 1);
} abort:
} @@ -980,6 +998,9 @@
} }
} if (debug)
} syslog(LOG_DEBUG, "Sending ACK for block %u\n", block);
} + if (block == UINT16_MAX)
} + syslog(LOG_WARNING,
} + "Block number wrapped (hint: increase block size)");

Where's the wrap? block isn't being wrapped back to 0 nor are
comparisons being done against short/uint16_t. It looks like this will
die at line 1011 when the block number from the client is compared
against 65536. Have you tried uploading a large file?

} block++;
} (void) setjmp(timeoutbuf);
} send_ack:

The only other thing I would do is make wrapping an option in
order to satisfy the purists. With the option, maybe also make it
possible to wrap to 1 instead of 0, since apparently there are some
clients that expect that. However, I don't think that's of vital
importance for the first pass.

}-- End of excerpt from Patrick Welche

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Miles Nordin
2010-01-11 16:40:35 UTC
Permalink
jn> The only other thing I would do is make wrapping an
jn> option in order to satisfy the purists.

okay, flag away, but please make the overall system work by default
with the common clients like IOS and EnTee to the extent that it's not
breaking anything, like other OS's tftpd's do. If you add a bunch of
simon-sez flags you will have some very lonely satisfied purists. I
do not have time to go reading and learn that block wrapping even
*exists* just to satisfy someone's OCD, or satisfy ``it's not a bug
it's a feature'' claim earlier in the thread---there is a lot of other
shit to do now, and the amount of pushback over this fix frightened me
a little.

Loading...