AFS behind NAT: A Short History

Having an AFS client behind a NAT is relatively easy, provided your
UDP timeouts are set high enough so that the server callbacks don't
get destroyed.  Having an AFS _server_ behind a NAT -- well, that can
be problematic.

Derek Atkins

> Second, you still have the timeout issue.

Easy enough to fix with any sane NAT box.  My ipchains/ipmasq box does
this with a single command.  If your Linux NAT box is running ipchains,
the following will work for you:

/sbin/ipchains -M -S   

Where the items between the < > are timeouts, in seconds, for TCP
sessions, TCP sessions after a "FIN" packet is received, and UDP
sessions.  I believe something around 10 minutes is recommended for
the UDP timeouts, so as to not break the AFS callbacks.  If you're not
using ipchains, I'm sure iptables has similar functionality.

Ray Link

The UDP timeout needs to be more than 10 minutes. Unfortunately,
iptables does _not_ support altering masq timeouts. This also affects
the ipchains compatibility code since that is not a real ipchains
implementation but uses iptables infrastructure. 

The timeout can be altered by looking for "#define UDP.*TIMEOUT" in the
appropiate kernel source directory and setting the value higher (default
is three minutes IIRC).

Tino Schwarze

In message <20010521194632.C3303@pc.ilinx> you write:
> Has anyone a patch to allow an option on the iptables command line to
> tune the UDP timeout for a specific rule?  I have a protocol which
> takes more than 30 seconds to return from a UDP query and thus the 180
> second timeout for "streaming udp" does not trigger in time.
> 
> Or any other solutions would be welcome.

Hi Brian!

	It looks like the next approach to tweaking UDP should be a
table inside the UDP module which defines behavior and timeouts for
individual ports.  Of course, with a module param to modify/add to the
table.

Rusty Russell

Hey, this sounds cool.  There is only one issue involved: Who guarantees
you that certain services run on particular ports.  that's nothing more
than common practice...

But when we make this timeouts configurable, what prevents us from 
making the global (i.e. future default) timeouts configurable, too?

During the whole discussion, I get more and more convinced that having
sysctl-configurable timeouts for all conntrack time settings might be
better. 

I still remember the old 'bells-and-whistles' button discussion, but 
have a look at /proc/sys/net/ipv4 already offers plenty of settings,
and I think it's a good idea to follow those examples...

Harald Welte

Actually, /proc/sys/net/ipv4 is a really good example.  Almost every
setting in here reflects:

(1) A bandaid for broken stacks, or
(2) A failure of the code to adapt at runtime.

ie. Almost every setting in here is an admission of failure (ours or
someone else's).

The `ftp timeout' problem in 2.0 and 2.2 was never fixed.  Because
there was a sysctl, and programmers are lazy.  Once we head the same
way, the code will never be really fixed: it'll be up to each and
every user to fix it themselves.

I suggest everyone who is deeply interested in this debate go and read
the first few chapters of Donald Norman's Design of Everyday Things
(or Psychology of Everyday Things as it was originally called).  IMHO
it should be required for every engineer.

Rusty Russell

I can't be sure, but I would guess that you're seeing a subtle
problem with connection timeouts.

In an ftp transfer, the control channel (always to port 21) is
opened first to handle all of the login, and high level communication
work.  When actual data has to be transferred (either a directory listing
or a file transfer), a second channel is opened.

While a large file is being transferred, the data channel sits
idle.  If your file transfer takes longer than the tcp timeout on your
masquerading box, that box removes the control channel from its list of
open tcp connections.  When the file transfer is complete, the ftp client
attempts to handle some housekeeping chores over the control channel, but
the masquerading box not longer has any record of how to talk over that
channel.  In short, even though the client and server both believe the
control channel is open, the masquerading box refuses to masquerade that
connection anymore.

The solution is simple - increase the timeout on the masuqerading
box.  To increase the timeout to 8 hours (28800 seconds), use the
following:

ipchains -M -S 28800 0 0

Those still using ipfwadm need to use:

ipfwadm -M -s 28800 0 0

instead.

Unless someone believes there is still a specific kernel bug
involved in this problem, I would suggest that this thread move to the
ip-masq mailing list.

William (Bill) Stearns