Faster downloads online with simultaneous download connections of large files divided into chunks

[Michael Shepard]

When download online (not downloading from a local network server) users are downloading with various limiters blocking the speed of the download per connection: the host website/server, ISPs, company router, traffic shapers, shared wifi access points.

Typically we find when downloading backups Filezilla will quickly work through all the small files using up to 10 connections (Settings > Transfers > Maximum Simultaneous Transfers) and then sit there using only one connection per file slowly downloading the (say 2) last large files for hours since the single connection is rate limited externally.

Suggestion: Divide files larger than x (New Setting) into chunks. Divide the file size by the number of Maximum Simultaneous Connections into even chunks and place each chunk in the transfer queue as a separate file. Each chunk processed in the queue as a regular file download. Use the existing preallocate space for download feature to set up the file for writing or stitch the chunks together after.

Multiple connections will then download different parts of the same file. If the other files in the transfer have a higher priority then once a queue connection runs out of other regular files to download and becomes idle, it starts on the next chunk of the large file. Or the user can use the existing priority feature to go after the large file first.

Example: If the file is 100GB (and there are 10 Maximum Simultaneous Connections) then connection #2 downloads from the 10GB mark to the 20GB mark and so on.

External to Filezilla, the user would be required to connect to a remote server that allows multiple simultaneous connections, which is typically already enabled. Most sites we use let us download 10 files at the same time.

Please ask me if you have any questions. Thanks for the great program!

[Tim Kosse]

It has detrimental effects on overall internet performance.

The perceived extra bandwidth when using segmented transfers isn't for free, it is stolen from other users using only a single connection.

[Michael Shepard]

Hi Tim, the FileZilla setting Multiple Simultaneous Connections already requires its use to be the responsibility of the end user to use the connection fairly (depending on their situation) for downloading lots of files, downloading one file per connection. The current setting allows 1 to 10 connections.

The same would apply to multiple simultaneous downloads of chunks of huge files, the same as the existing feature of up to 10 file simultaneous downloading.

Had a look at the source code. If you are interested, I could work the code for this feature idea. We could discuss how you would like it approached and work out the details.

Thanks!
Michael