Allow TCP null packet keep alive

[Kevin Gut]

Note: This is a proposal for the FileZilla client, but it would also make sense for servers that are behind a "stupid" NAT device.

Most NAT routers forget open TCP connections after 20 seconds, which causes TCP connections to be silently dropped without the OS being notified.

This is most easily notified if an FTP connection is left idle for about 30 seconds. Any FTO operation (file download, entering directory, etc) then is performed on a dead connection, which then does nothing for 20 seconds, until the connection is forcibly terminated and Filezilla reconnects.

At the moment, this can be fixed with the "send FTP keep-alive commands" options. This however is not needed and is intended for another type of problem. Instead a TCP null packet could be sent (a packet without content). At least in Windows, this can be configured by setting the SO_KEEPALIVE flag and then defining the timeout (I propose 15 seconds). This option can be observed in action on PuTTY (see "Connections" category).

The great thing about this is, that the TCP stack handles the keepalive interval. There is no need for filezilla to manually send any packets. The OS only sends packets if the connection has been idle for the specified number of seconds.

Proposals

adding additional options on the "Connection page"

• enable TCP keep alive: Checkbox; This sets the SO_KEEPALIVE option.
• keep-alive interval: Number input; This sets the interval (15 seconds by default)

Adapting the Network settings wizard

Adding a configuration page for the keepalive settings

The network wizard tries to figure out the external IP at some point. If the IP is different from the local listening endpoint (ie. Computer uses a private IP address) then the keepalive configuration page should have the appropriate option already pre selected to allow the user to "continue with the default".

MSDN article: ​https://msdn.microsoft.com/en-us/library/windows/desktop/ee470551(v=vs.85).aspx

Note: SO_KEEPALIVE is also available in linux as it is part of the TCP specification.

FileZilla Client
----------------

Version:          3.20.1

Build information:
  Compiled for:   x86_64-w64-mingw32
  Compiled on:    x86_64-unknown-linux-gnu
  Build date:     2016-08-03
  Compiled with:  x86_64-w64-mingw32-gcc (GCC) 4.9.1
  Compiler flags: -g -O2 -Wall -g -std=gnu++14

Linked against:
  wxWidgets:      3.0.3
  GnuTLS:         3.4.14
  SQLite:         3.11.1

Operating system:
  Name:           Windows 7 (build 7601, Service Pack 1), 64-bit edition
  Version:        6.1
  Platform:       64-bit system
  CPU features:   sse sse2 sse3 ssse3 sse4.1 sse4.2
  Settings dir:   C:\Users\ \AppData\Roaming\FileZilla\

[Tim Kosse]

FileZilla already enables SO_KEEPALIVE with the default keep-alive interval of 2 hours as per RFC 1122.

According to RFC5382:

   REQ-5:  If a NAT cannot determine whether the endpoints of a TCP
      connection are active, it MAY abandon the session if it has been
      idle for some time.  In such cases, the value of the "established
      connection idle-timeout" MUST NOT be less than 2 hours 4 minutes.

Please return faulty NAT routers that disobey RFC5382 to their makers.

This option can be observed in action on PuTTY (see "Connections" category).

Note the null packets PuTTY sends are actually part of SSH. PuTTY does not allow you to configure the TCP keepalive interval.

[Kevin Gut]

Please return faulty NAT routers that disobey RFC5382 to their makers.

This RFC is not a standard and therefore there is no need to follow it.
This is literally the first sentence of the RFC:

   This document specifies an Internet Best Current Practices for the
   Internet Community, and requests discussion and suggestions for
   improvements.

I cannot return hundreds of my customers NAT routers to the manufacturers because they close connections "too early". The Industry standard at the moment seems to be 20 to 30 seconds regardless of what the optional RFC says.

Even the professional grade firewall I have here closes connections way earlier than 2 hours.

Unless the RFC says "INTERNET STANDARD" on the top right you can forget any company to ever implement it.

RFC that is not a standard: ​https://tools.ietf.org/html/rfc5382
RFC that is a standard: ​https://tools.ietf.org/html/rfc793

[Tim Kosse]

Note that TCP as specified in RFC 793 relies on the end-to-end principle, something that NAT routers violate by their very design!

Unless the RFC says "INTERNET STANDARD" on the top right you can forget any company to ever implement it.

Then why do we have things like NAT and firewalls to begin with? I don't think there's an "INTERNET STANDARD" for these things. Yet companies implemented these things. Since they implemented it without a standard, why can't they fix it without a standard?

The Industry standard at the moment seems to be 20 to 30 seconds regardless of what the optional RFC says.

Why did they select that particular value? There's no "INTERNET STANDARD" that says this value should be used.

It's hardly an industry standard either, personally I've yet to experience any device that's so fundamentally broken as to close connections that are idle for a mere 30 seconds or less.

[Daniel Beardsmore]

Sadly the standard Virgin Media cable router (the Super Hub, a rebranded Netgear wireless router with integrated cable modem) in the UK really does appear to close NAT sessions in this manner. I don't know quite how long, but I'd estimate that NAT entries are dropped in under five minutes. With PuTTY, I've been forced to enable keep-alive otherwise SSH becomes intolerable, as my sessions hang continually any time I leave them alone for a minute or two (e.g. when leaving tail -f running and waiting for something to appear). VPN connections likewise drop out unless you jerry rig something to hold them open.

I could return the router, but then what? That's all you get. (Apparently you can't simply replace it with another DOCSIS 3 modem.) I could change ISP, but that's too great a risk considering my distance from the exchange (2–3 miles), since I could be throwing away a good connection and replacing it with some tragically horrid ADSL service. Getting it to operate in bridge mode is possible, but it's a nightmare, and officially this feature is defunct.

Since I connect to my own site using FTPS instead of SFTP, I can (and do) use FileZilla's FTP keepalive, but this would not be an option for SFTP (and I know, I've checked, as I have some weird issues with SFTP too on another network that I wanted to try fixing with SFTP keepalive).

I know that I had trouble with an old Linksys router in the past (BEFSR41) that would drop NAT sessions after something like 30 minutes or less, because at the end of an FTP upload of that approximate duration, the control connection would be discovered to be dead (you'd get a RST trying to write to it) and then the client would panic and upload the file again, and again, and again ... I fixed that with FTP keepalive in FlashFXP. That wasn't as short as 20–30 seconds, but it's well below the expected 2+ hours.

Keepalive is a horrible hack, but sadly a necessary one at times.

[Kevin Gut]

Apparently you can't simply replace it with another DOCSIS 3 modem.

Correct, unless your provider allows it, which I doubt. You can try to configure it into bridge mode and then plug your own router into it, but unless you are able to pinpoint the only router in this universe that is not closing NAT connections too early, this would not help you anyways. Also it is an expensive fix for an application related problem. If you put your modem into bridge mode you must attach your own router or you will not longer be able to connect multiple devices or have Wifi.

I could change ISP, but that's too great a risk considering my distance from the exchange (2–3 miles), since I could be throwing away a good connection and replacing it with some tragically horrid ADSL service

Don't do that. Here is a graph showing the performance of various technologies. The distance (X axis) is in Kilometers though: ​https://upload.wikimedia.org/wikipedia/commons/1/11/Docsis_operating_distance.jpg

Somehow @codesquid owns the only router in existence that seems to leave NAT connections open for two hours (I doubt he has actually tested this). I don't know if this ticket will ever get implemented, as the status has not yet changed back to "open" and is considered "rejected" but I doubt it, because he thinks we can change the whole router manufacturing industry to follow a semi-standard.

In Windows at least we can modify network sockets that are not ours. Maybe I try to come up with something, that changes the keepalive globally for all TCP connections with an outbound Address.

[Daniel Beardsmore]

I seldom use wireless. I still have my BEFSR41 ... that would give me a NAT extension of 25 minutes or so! I think it was starting to die, though -- I ran it for eight years or more.

I chose not to re-open the case, as that's not my decision; I am simply adding some data to the debate.