File content swap

[doodlebee]

It's been suggested I submit this as a bug report. So here goes:

I upgraded my Filezilla a few months ago to 2.2.32 (which is the latest stable release). Ever since then, I've had a very weird issue where some files are getting swapped content with other files.

Man, I know this sounds *insane*, but I'll give you an example: I do frequent installations of WordPress for the Install4Free site. I have a copy that I keep on file to upload to a myriad of different servers. Over the past few months, I've noticed oddities like the index.php file and the styles.css file will upload with the correct filenames, but the *content* has swapped - so that the index.php file contains the styles.css content, and vice versa.

That is only one example, of course. It has happened with many different programs, and usually happens when I upload many files at once. I can easily fix it by tracking down which two files have been content-swapped and uploading them individually. But when I'm doing mass quantities of uploads (especially for something like ZenCart, where everything is includes and it's hard to track down what file was swapped where) it gets tedious and difficult.

It's truly bizarre (and yes, I *do* check that the files I'm uploading are in proper order - and they are - this only happens when I upload with FileZilla - it's the only common denominator to this issue), and I know it sounds crazy. But I guess someone should know about it and see if I've got some really obscure bug.

[Alexander Schuch]

Can you please name the FTPd software name and version? And... can you test/try FileZilla 3 (beta) next time you have to upload a large chunk of files to see if the problem gets triggered there as well?

[Tim Kosse]

This is a known issue with some broken servers and a few routers using the SO_REUSEADDR socket flag in an improper way. Only solution is to use a different server software.

[doodlebee]

You mean the files/software I've uploaded (by software name and version)? There's so many - I don't think I could name them all. I will name all I can remember if that's what you mean, but I don't want to take up a ton of space here.

But yes, I'm getting ready to get the FileZilla 3 beta and give that a whirl to see if it happens with it - I'll probably be giving that a shot very soon. I'll come back and let you know how it goes.

[doodlebee]

Sorry btog - so you're saying it's server dependent? It's true, it doesn't happen *all* the time. But I upload files to *many* different servers on a frequent basis, so maybe it just seems like it happens a lot *to me* because of that. Now that you mention it, it has never happened to me when I upload to my own server. That's interesting. So even if I upgraded to FileZilla 3, I still may have the same issue?

[Alexander Schuch]

If the server uses that SO_REUSEADDR socket flag, swapped file content should happen with most/all FTP clients using that server.

Anyway, out of curiosity, which FTP server is it (name and version) where you observed the issues?

[Tim Kosse]

The bug is definitely in the sever software. FileZilla is just the unlucky client with the command pattern to trigger this with a rather high percentage.

[doodlebee]

Hi ci-dev -

Oh wow, there's been a few. I can't really remember them all. One does stand out - almost *all* of the files I uploaded were swapped, and then I had one the other day that did it as well - let me see if I can dig up who they were...

Here's one (the one that happened the other day) was pair.com. The other was in the Netherlands - this is the one where almost every file had swapped. i-s-e.nl was the host's name. Those are the two that I can recall off the top of my head.

[doodlebee]

Oh, and sorry - I don't know the FTP version of the one in the Netherlands...but let me find the recent one...

Okay, I can't find a whole lot of info, but maybe you can garner something from this:

Response: 220 ftpEDITED.pair.com NcFTPd Server (licensed copy) ready.
Response: 230-FreeBSD 4.8-STABLE (PAIRqm) #0: Fri Oct 3 11:10:15 EDT 2003
Response: 230-Telnet/FTP: EDITED.pair.com
Response: 230- WWW: EDITED.pair.com

Hope that gives you a little bit of info there (I edited some things out for readability/privacy issues - hope you don't mind)

[dlomneck]

Well, i posted about this here:
​http://filezilla.sourceforge.net/forum/viewtopic.php?t=3877&sid=aa5f56c7d485afa083d3175a9d98bc6d

before I found it here. Any way you can get us a list of FTP servers that you know have this "issue" with your client? There's one listed here (ftpEDITED.pair.com NcFTPd Server), and the one I use is the very popular Serv-U FTP Server v6.0 .

If you are going to tell people they have to change hosting companies, you should atleast tell them what to look for in a hosting company.

David

[Tim Kosse]

Download the server sourcecode, look if SO_REUSEADDR gets used for incoming connections. If that's the case, it is likely that file contents will get swapped.

[dlomneck]

You are assuming that the server code is available. Thanks for putting a little efforts to help your community of users. You are quick to blame the server, but whenever your community asks you for even a list of possible suspect servers, you tell them to look at the code? That is absolutely, 100% lazy.

Here are your mistakes:

1. You Assume your entire community is made up of programmers. I assure you, this is NOT the case
2. You Assume everybody is going to be using open source FTP Servers. This may be for some linux servers, but find me an open source FTP server other than filezilla for Windows. IIS isn't open source, serv-U isn't open source. Now, if you can get your hands on the code for those two and look to see if they are using the SO_RESUEADDR, I'll be quite impressed.
3. You need to get down all your high horse and help to fix the problem instead of passing the buck so quickly. Your commments sound almost angry, like you can't be bothered with this.

I tested this 20 times with CuteFTP, and not once did it mess up. I tested it 20 times with Filezilla, and guess what....failed 17 times. Now, I don't know what "magic" CuteFTP is doing to not cause this problem to occur, but it does seem like there is something you can do in your code to fix this. Instead of passing the buck, why don't you try to actually solve the problem. Won't be the first time a programmer had to change something because of an "incompatibility" or their products with another, even if it isn't their products fault. I deal with that every day when trying to code websites for the non-standard IE browser.

My comment will probably be denounced as a flame, but I am just trying to better the project. So what if it's the Servers fault, you can't change all the servers. Instead of alienating your users, just try to fix the damned problem.

Dave

[Tim Kosse]

I understand your frustration, but I can't fix what's not in my control.

I've spent quite some time with audits of my own code and I can assure you that this problem definitely isn't caused by FileZilla. So naturally I blame the software involved I cannot analyze, i.e. the close source software.

As you've noticed, not every client seems to be affected by this bug. But this doesn't mean that the clients suffering from the problem are bugged. It just shows again that different clients have different access patterns.

There is only one way to work around this problem: Not establish two transfer channels at the same time. But such a workaround would cripple the performance on working server not exhibiting this bug, so I cannot use it.

My opinion stands solid: The server needs to be fixed, end of discussion.

[dlomneck]

Just had my host contact Serv-U, and this is what they state here: ​http://forums.crystaltech.com/index.php/topic,32034.0.html

"Craig,

Serv-U does not use the SO_REUSEADDR package for data transfers, and is not the cause of the error message being received.

Please let me know if there is anything else I can do for you.

Thomas J. Parikka - Technical Support Engineer ​http://www.RhinoSoft.com
Voice: +1(262) 560-9627
FAX: +1(262) 560-9628"

So, what not?

David

[Tim Kosse]

There are other things which can explain this problem, SO_REUSEADDR is just one of them. One other possible reason could be the use of a fixed set of listening ports in passive mode, where the listening ports do not get created on-demand and get reused improperly.

And all these flaws could also be in the protocol inspection and sabotage code of the used firewalls/routers. It is impossible to find the culprit without physical access to an affected network.

[dlomneck]

Well, as you can see, I have three administrators that are at my hosting company actively involved in the thread I posted before. So tell me what I can do to figure this out. Please help me get to the bottom of this. Nobody wants to have file transfers swapped, and if this is something that can be fixed by either the hosting company, the ftp server company or by you, then that's what colaboratio is all about. If you aren't interested in removing potential bugs, or having your users have a trouble free experience, then just tell me so, and blow off a golden opportunity to try to make this problem go away.

David

[doodlebee]

I see this has started an interesting conversation here...

I haven't had the opportunity to use FileZilla 3 yet - but again, as soon as I do, I'll be posting back here on how it went.

But I did have a question: I see you keep mentioning "routers". I don't have access to some server information, but when you mention routers - are you talking about what *I* use as a router to connect to the internet?

For example, my network setup has my cable connection coming into a router, which then disperses static IP address to each computer on my network. So everything I do comes and goes through my router. Is *this* possibly what you are referring to - not something on my host's end?

I just wanted to be clear on that - so sorry if it's a dumb question - but if it's *my* router, then it is something I can look into, at least.

[Tim Kosse]

1. Get two fresh, fully updated windows installations with Windows firewall service and Application Layer Gateway service disabled.
2. Install server and client on the same machine, test locally
3. Connect both machines with a crossover cable, install server on one and client on other, test again.
4. Install firewalls and test again.
5. Connect all other active network components, routers, switches and so on, test inside your LAN.
6. Test over the internet

If it's a problem with either the client or server, you should be able to observe swapped filenames in steps 2 or 3. If that's working, you should be able to figure out the culprit in one of the other steps.

[doodlebee]

I just wanted to update something here. I didn't get a chance to do the actions codesquid suggested (about testing stuff) but I have upgraded to Filezilla 3 beta. It's working great. I've been installing left and right on all kinds of servers, and haven't had a single file-swapping issue. Thanks for the tip off! (and I wouldn't call the resolution "rejected" - it's not by me, anyway - it worked just fine, so I'm changing it ;) )

[doodlebee]

I just wanted to update something here. I didn't get a chance to do the actions codesquid suggested (about testing stuff) but I have upgraded to Filezilla 3 beta. It's working great. I've been installing left and right on all kinds of servers, and haven't had a single file-swapping issue. Thanks for the tip off! (and I wouldn't call the resolution "rejected" - it's not by me, anyway - it worked just fine, so I'm changing it ;) )