Download continues past 100% corrupting downloaded file

[ianclarke]

Version 3.2.6.1

When downloading a .zip file of size 322MBytes, the download continues past 100% and the downloaded file grows to be larger than the source file. Even when the cancel button is clicked the transfer continues. I had to close the window to cancel the transfer.
The downloaded zip file can be opened, but it is not possible to extract the files. Get message that archive is in an unknown format or corrupted.

[Tim Kosse]

You are using a horribly broken Microsoft server.

It clearly acknowledges the resume offset, but then ignores it and still sends the complete file.

You have to upgrade to a proper server.

[bigjames]

Version 3.2.6.1.

I met the same scenario - the latest version FileZilla behaves terrible while downloading zipped files from a "horribly broken Microsoft server".

But, I can successfully download files from the same "horribly broken Microsoft server" by using several other ftp tools, including, but not limited to -
1) a very old version cuteftp;
2) a free tool, named "CoffeCup". (I found the name from an internet evulating result. FileZilla got the 2nd place; "CoffeCup" got the 10th place)

As to Microsoft IIS ftp service, it might don't support "resume" in some old version, but it could be improved at least at 6.0.
If FileZilla was designed not to support IIS, you can reject this bug report again.
If not, would you please verify some other ftp tools to understand how they support IIS?

Maybe IIS is really "horribly broken", but if all the ftp tools except FileZilla can support IIS, then FileZilla is also "horribly broken".

[Tim Kosse]

This is not a bug in FileZilla, it is clearly the server's fault.

Simple analogy: A new bridge has been built but they cut corners during construction so that it doesn't fulfill the specs anymore. All day long only small compact car have come and passed the bridge just fine. Now comes along a truck and promptly the bridge collapses. According to your faulty logic the truck is to blame because compact cars could pass the bridge just fine.

[bigjames]

Let me add more comments.

Scenario:
I tried to download the same set of files (3 files in total, their sizes are: 1.3MB/136MB/543MB) by using FileZilla and CoffeCup. During the downloading, I make the connection broken and then resumed several times for both tools.

Result:
FileZilla failed 100%;
CoffeCup succeed 100%.

Question:
Both of them worked for the same tasks, I can not find any big difference.

1) Why did you use "Truck" and "Compact car" to describe them respectively?
2) Do you think FileZilla support the standard RFC implementation only?
3) How do you think about the reason why CoffeCup can support "resume" in IIS, if IIS tells CoffeCup - "Hey, I can support resume", but it actually ignore the "reset" command sent by CoffeCup?

[Tim Kosse]

1) Why did you use "Truck" and "Compact car" to describe them respectively?

To make it clear that not all gas powered vehicles are of same weight.

2) Do you think FileZilla support the standard RFC implementation only?

Yes, FileZilla does follow the standards.

3) How do you think about the reason why CoffeCup can support "resume" in IIS, if IIS tells CoffeCup - "Hey, I can support resume", but it actually ignore the "reset" command sent by CoffeCup?

Compact car vs. truck again, not all clients do things the same way.

[bigjames]

Continue...:)

1) I know NO two thing are exactly same in the world. But I am quite sure FileZilla does not burn the gas :).

Then, what are those differences?
From the end user's point of view, FileZilla can not support the ftp server in use daily, but other ftp tools can.
From your point of view, FileZilla follows the standards very strictly. You're pround of it.
From my point of view, although I know FileZilla is free, but it is a tool for the users to use, not a laboratorial tool specially designed for the ftp protocol verification.

2) I won't question this point any more, but the end users will appreciate if FileZilla can be more prowerful.

3) The answer ("not all clients do things the same way.") is not compellent to me. If CoffeCup violate the standards in order to support IIS, I agree that FileZilla does not choose the same way. Otherwise, FileZilla may be enhanced a little bit.

BTW: Did FileZilla team test the "resume" functionality in IIS 6.0? Is it still the root cause that preventing FileZilla from working properly with IIS?

[Tim Kosse]

Just upgrade that damn server.

End of discussion.

[Don Milne]

After waiting an hour for a download - only to see it continuing past 100%, and not for the first time, I am now thoroughly pissed off. Then I find this ticket explaining what the issue is: and as a developer myself I have to say that I find codesquid's excuses totally unconvincing. All software in the real world which implements standard protocols has to cater for the fact that not everything it talks to is 100% compatible with those standards. A programmer who can't adapt to the real world is basically a waste of space, and so currently is FileZilla.

Whether the Microsoft server is broken or not, that's what's out there, that's the job that FileZilla is expected to do. If you can't make FileZilla work with popular servers then please withdraw it and stop wasting everyone's time. Meantime I'm already hunting for an alternative.

[Tim Kosse]

Users like you are the reason there's still bad servers out there. I bet you didn't even bother telling your server administrator about his broken server.
It's oh so more easy to bitch all day against people like me who stand up against the bad in the world.

[Don Milne]

There are a lot of Microsoft servers out there in the world. You expect me to notify the administrator of every one? I didn't think so.

[Tim Kosse]

Only the few ones you're using.

[scott carl]

I am getting a similar situation against an old MVS (IBM mainframe) FTP server. I don't have any of the philosophical issues noted above, but could it be more of an issue than just one of MS lack of compliance with standards?

[Kraig Babin]

Codequid: I get your point, but I'd like to correct your analogy of the bridge.

A bridge is built with a set weight limit due to a design decision to use cheaper girders. Everyone knows this limit and people with a heavy vehicle don't use the bridge. You can't drive across the bridge in a truck, of course if you buy a car, you expect to be able to use the bridge because you are in a car.

In this case, the car is the client. People use FileZilla because its an FTP client. There are many not-100%-standards-compliant servers, old mainframes, IIS6, etc... but it exists, its common, its flaws are KNOWN, its used, so users logically expect it to work.

Its a question of whether you care more about users or philosophy...

[Tim Kosse]

Let's extend on the bridge analogy. There are two important things to notice:
a) Not all bridges are built to spec
b) You do not even know which bridge you are driving over. In other words, the FTP specifications provide no means over which a server has to identify itself.

As such you only have two choices:
a) Do not cross the bridge at all. A definite loss as your goods will not reach their destination.
b) Cross the bridge. If that fails you will get your money back since you have the legally binding specifications that were violated by the broken bridge.

And for the love of the flying spaghetti monster, fix your damn server already and stop whining. How fscking hard can it be to double-click a few times?

[Kraig Babin]

There is a third choice
C) get a car(client) that is capable of crossing that and all other bridges (hey, others have done it even with the FTP spec not providing for server software identification)

You incorrectly assume that I/we have control over the server software, and are whining because we do not want to bother. In reality, some of us are employees or contractors that are forced to conform to the existing environment and simply want the best tools to accomplish that. FileZilla typically is a great FTP client with a lot of promise, and its open source!!. Yet its no longer the best tool for the job because you can't know if its going to work (as you said, you don't always know what bridge you're driving over, you just expect it to work)

We are not whining because we are lazy or stupid, we are asking the development team to consider making their hammer capable of hitting all nails we logically expect, not the the round headed ones, or possibly letting users know that their hammer is only compatible with ~70% of servers out there (I would be happy with a notification saying "This server does not properly support the FTP standard and your download will possibly be corrupt.")

[Don Milne]

I took codesquid's advice: I stopped whining, I clicked the mouse a few times, and presto: I was now the proud owner of a copy of SmartFTP - which so far has worked completely reliably (not a single corrupted download despite much heavier use to the same sites).

I don't know why this Trac site even exists, since Codesquid quite obviously doesn't care what problems his users have in the real world (as opposed to his fantasy world in which every server implements the standard correctly, and users all have a say in choosing what FTP software a server runs).

[Tim Kosse]

Let's discuss the car analogy further.

Assume you're producing heavy! machinery and need to deliver something you made to a customer of yours. What you transport weighs 12 imperial yardpoundsgallons (using mock-american units to aid understanding) and you will definitely need to use a truck to transport this, as cars can only carry a single yardpoundsgallon. You carfully plan your route, make a note of all bridges along the way. You see that all bridges along the way are rated for 20 yardpoundsgallons. So all fine and dandy and you start driving. But as soon as you cross the first 20 yardpoundsgallon-rated bridge with your 12 yardpoundsgallons, that very bridge collapses.

Now would you still look me directly into the eyes and honestly say that the truck is to blame? I don't think so.

[Kraig Babin]

We have digressed to the point where the analogy no longer suffices. Your bridge story is great for bridges, but were not talking about physical one-time-construction that is expensive and complicated to change after the fact. We are talking about an intangible, virtual product that is modified, updated and improved regularly...

And why you assume we're American or need fake units to understand something is beyond me...

Obviously you are missing our point. Yes these FTP servers that do not support the standard suck. It makes your life difficult, and if we want to use your software, it makes our life difficult. We are stuck arguing over philosophy and whether it's logical to expect a generic client to support the servers we can expect to encounter.

From an efficiency standpoint, the energy we have all put into arguing over this could have been better put to actually solving the problem (and its evident there are solutions out there). If not, then at least warn the users when the server appears to have implemented the FTP spec incorrectly so the user can take reasonable action to complete his task.

[Don Milne]

In any case I think an important point is being missed, which is that there are two parts to this problem.

Part one is the bug itself and a lack of a fix/workaround implemented on the client side. That's bad enough, but there's also part two, which is total failure to detect the condition when it arises. Having a download aborted by corruption would be annoying enough, but waiting an hour for it to reach 100%, only to see it proceed to 101% and on, is very frustrating. Having an undetected error condition - and refusing to consider doing anything about it... well as a software developer myself I regard that attitude as very surprising.

[Tim Kosse]

I fail to see how this bug can be worked around other than not to use resume at all on any server.

[Jonathan]

Hi,

Instead of not using resume, would it be possible for Filezilla to check the byte size of the source file and stop the transfer once the byte sizes of the source file and destination file match?

Btw, I love FileZilla and have no control over changing server software.

[Don Milne]

Waiting until the download reaches 101% is not exactly what I meant by "detect the condition". One possibility I had in mind was to resume from a point slightly earlier in the file (i.e. give yourself some overlap), that way you can compare the overlapped area to check that the resume worked. Not bulletproof, but better than nothing.

There is also the question of how SmartFTP manages to avoid the problem. I've read the FTP spec and don't see how it can be done, but I'm not an FTP expert: there may be unofficial protocol enhancements that I'm not aware of.

[Tim Kosse]

Standards are the law of the internet. If some server violates the FTP standard, said server needs to be fixed, not a standards-abiding client suffering from the broken server.

[loc]

"Standards are the law of the internet." Okay. Yes. Agreed.

Your "law-abiding" FTP client doesn't work with those "broken Microsoft" servers (which from the posts above are clearly not all Microsoft). Other (presumably) "law-abiding" FTP clients work.

You've spent over two years bashing people for "whining" without so much as taking a peek at the code behind cuteftp, CoffeCup or SmartFTP (which clearly work) and learning from it (I'm assuming here because FileZilla still has this bug until now).

YOUR logic is faulty, not ours. FileZilla is not the truck. It's the bridge. It connects point A (the server) and point B (our computers). This bridge collapses (sometimes, because I've tried redownloading those same files that failed and it works the second or third time). The files are the load the bridge carries. If that truck uses another bridge (i.e. another FTP client), it crosses over fine.

We're the users of your software. We "whine" because we care, because we want FileZilla to be the cream of the crop out there. The least you could do is say thank you. If you don't want people to use it and start "whining" when we're actually pointing out a valid problem, then don't publish it.

[Cheney Ma]

I am an user who uses FileZilla, now I am using version 3.5.3. Let's show my appreciation for the person who provide this free application to us.

I am also trapped by this issue: downloading cannot be stopped automatically after 100%.
I am a QA worked on Software testing over 6 years. I have read all comments posted in defect.
Let's show my thoughts from a QA perspective:
It is obviously an VALID defect and should be fixed by developer.

Below is why:

1. Application should meet end user's requirement,the situation we met in this defect obveriouly makes user uncomfortable.
2. Application should be able to handle exceptions and errors properly.
3. Compatibility is a very important benchmark for a good application.
4. If other tools can handle the situation mentioned in the defect, there MUST be a solution to fix it, so it should be fixed.

For product owner, it is impossible to fixed all defects, but you can fix as many as possible! That makes your application stronger and stronger!

[Pedro]

I have exactly the same problem, but I am downloading close to 1GB files, consuming a lot of time and bandwidth. Since Filezilla does not acknowledge this issue, I just changed to a "proper FTP client tool". CoffeCup free does the trick.

[Josh Davis]

USERS,
A workaround for many sites is to use SFTP. I know I've run into this on non-MS servers (IBM AIX), but wasn't paying attention enough. I just switched to SFTP, assuming it was a payload issue, or mid-stream conversion to ASCII.

DEVELOPERS,
Have you verified that this issue is actually caused by a defective server, or is that an assumption?

Regardless of who's fault the problem is,
Is it possible to detect specifically what causes the problem,
or does this require maintaining a list of faulty servers?

If it's the detectible, then what's the work effort to simply flag this, and disable resume, or work around it the same way other clients do?

If it's not detectible, then what's the work effort required to allow the user to specify manually, by server connection and/or profile, whatever the workaround is?

Also, is the workaround simply to disable resume, or is it something else?

[LongSky]

Still have this bug , IIS6 ,alway fail in 100% ,any file if it bigger than 600MB

[odincreel@mail.com]

CODESQUIDS'S BRIDGE

Like all jokes, codesquid's bridge becomes funnier if you make it more true. So to better define the analogy, I had to adjust the roles > played by

The Bridge > The Server
Cars and trucks > programs that "drive" over The Bridge
A clueless engineer for Filezilla cars and trucks > codesquid
The customer > bigjames
The hero and winner > CoffeeCup

A customer tries to drive a Filezilla truck over a bridge. It makes it over the top, but the brakes fail, the truck careens over the other side, rolls three times, and falls into the ditch.

This is a major issue so Filezilla sends out an engineer to inspect the crash.

"The bridge is broken"

"... what?" says the proud owner of the 2009 Filezilla truck.

"The bridge is specified to have a maximum of 7% grade. I just measured it, and it has a 7.5% grade. You should replace the bridge, it's horribly broken" says the engineer.

The customer is caught of guard. Does this engineer have a sense of humor?

"You realize that every other brand of car and truck is safely passing over the bridge right?"

"Yah, engineers for those trucks must not have read the specifications on the bridge. Those truck drivers really ought to buy new bridges. You should really fix that bridge"

No, the engineer really did exist in another world. Not only was he incapable of understanding that a bridge costs more than a car or a truck, he didn't even understand that many people do not own the bridges they drive over. He even thought that customers would prefer his truck because it couldn't drive over this particular bridge.

It has been three years now. CoffeeCup cars and trucks continue to traverse the biggest and steepest bridges. The customer has switched brands, and Filezilla continues to fade away.

The end

[ftp_user]

This is still an issue in 3.7.3 and can impact any file type, not just zips.

I wasted many hours downloading a file over 300Meg over a very slow connection, only for the file to end up corrupted and growing to an unbounded size. This was an exe installer file. Several smaller pdf files on the same site also failed.

I was able to successfully download the files using WinSCP.

[David Jordan]

Yes, the initial fault is Microsoft's. Whoever could have seen them not fully implementing a standard?!

But this not being fixed by Filezilla in years in ridiculous. And the claim that you can't check it is nonsense as well. I'm not going to bother looking at the sourcecode, but here's some psuedocode to show you exactly how to do it:

void resumeDownloadingFile(theFile, newStream){

downloadUpTo(newStream,8192);
if (newStream[0..8191] == theFile [0..8191]) { Check whether the first 8k of the file is equal to the first 8k of the resume point
downloadUpTo(newStream,16384);
if newStream[3456..11647] == theFile[0..8191] Check if it's a coincidence (lots of formats are going to have similar 8k chunks) by picking another chunk
tellUserCantResumeAndRestartDownload();
}
}
else{
behaveNormally();
}
}

Yes, this will have false positives - the most likely would be if the file contains long chunks of the same byte, probably 0, over and over. This is very unlikely if FTP uses compression normally, for large files at least, which I suspect it does but I'm not going to bother reading the specification. But it will not have false negatives, which is the main point.

Fix your client!

[Tim Kosse]

From the NewTicketFormGuide: "Priority: Leave it at normal, even if the problem is important for you. The developers will decide how important this problem really is."

"Tickets that are filled out incorrectly will get rejected."

[syed]

This problem is occuring not just in IIS but also when connecting to Filezilla server too.

I have been using Filezilla server/Client for well over 2 years now. This is an issue I am facing after the upgrade definitely there is something which has broken. Rather than looking whats causing the issue it seems the ticket has been going nowhere.

Can someone fix this.

[syed]

Wrong ticket updated setting it back to previous status.