Slow startup and local path changes due to querying excessive folders

[Bob Triffles]

I have noticed slow cold startup times (e.g. 15-60 seconds) for FileZilla on several computers. During startup, the hard drive is constantly being accessed. Therefore, I ran Sysinternals' Process Monitor to debug the issue.

I found that the majority of the startup time appears to be spent querying the Windows Cookies folder:
C:\Documents and Settings\USERNAME\Cookies
The query takes a long time when you have a lot of cookies in the folder (e.g. hundreds or thousands). I have over 3000.

Obviously, I could clear the Cookies folder to work around the issue. However, users shouldn't have to clear their Internet Explorer/Windows cookies to obtain reasonable FileZilla startup performance.

Furthermore, I'm not sure why FileZilla is querying that folder at all. I took a quick look at the source code and couldn't see any reference to the Cookies folder. Perhaps some API/library used by FileZilla is doing the deed.

If necessary, I suspect the issue can be reproduced by placing a few thousand small files in the Windows Cookies folder and rebooting to ensure the directory is not cached by Windows.

This issue may explain some of the reports in Ticket #3267. I created this new ticket because most people in that discussion seemed to think that network drives were at fault.

Please let me know if you require any additional information. Thanks!

Running:
-FileZilla Client 3.5.0
-Athlon II X4 640

[Bob Triffles]

I did some additional analysis, and the problem is more generic than my initial report indicated. On startup (and any time you change to a new local folder), FileZilla appears to be querying the contents of every folder along the path to the current local folder (including sibling folders). For example, if my initial local folder is:

C:\Documents and Settings\USERNAME\Desktop

FileZilla will query the contents of each of the following folders:

C:\

C:\Documents and Settings
C:\Program Files
C:\Windows
(Plus every other folder in C:\)

C:\Documents and Settings\All Users
C:\Documents and Settings\USERNAME
C:\Documents and Settings\USERNAME2
(Plus every other folder in C:\Documents and Settings)

C:\Documents and Settings\USERNAME\Application Data
C:\Documents and Settings\USERNAME\Cookies
C:\Documents and Settings\USERNAME\Desktop
C:\Documents and Settings\USERNAME\Favorites
(Plus every other folder in C:\Documents and Settings\USERNAME)

Thus, FileZilla queries my Cookies folder even though it's not my current local folder.

Perhaps there is a good reason why FileZilla queries so many folders that are not the current local folder? If not, this I/O seems like it is unnecessary.

Normally, querying a folder should not take so long, but Cookies (and folders like it) are a special case. It can contain thousands of files created at different times over the course of a few months. Consequently, my guess is that the directory listing (in the MFT?) is stored at numerous different locations on the hard disk, requiring lots of seeking to list the folder's contents. The Windows XP defragmenter has no effect on this problem, as no files are actually fragmented.

Note that I verified the problem exists with no virus scanner running and a fully defragmented hard drive (using the Windows XP defragmenter).

Contrary to what I said before, I don't think the problem can be reproduced by merely creating a few thousand files in a folder. Doing so most likely creates a contiguous directory listing, avoiding the suspected cause (lots of seeks). Regardless, the root cause (FileZilla querying lots of seemingly unneeded folders) is clearly visible in Process Monitor, regardless of the time taken.

[Bob Triffles]

Upon further thought, my guess is that FileZilla is querying all those folders to display the "+" symbol next to folders with subfolders in the folder tree. Unfortunately, this causes the interface to stop responding while folders are being queried after changing the local path. Since FileZilla queries folders to display the initial folder tree on startup, this issue also can also cause slow startup of FileZilla.

I suspect the best solution would be to query all the folders in a separate thread to avoid blocking the GUI or delaying startup. This is what Windows Explorer does in its folder tree. You can see the "+" symbols appear dynamically after the path is changed.

Note that this issue can be most apparent when browsing a slow network drive with lots of folders/files. FileZilla will noticeably freeze if slow data transfer causes the folder queries to be delayed.

[Kevin Swanson]

I am seeing the same issue. I was experiencing the same slowness (more than 30 seconds to start, extremely slow directory listing) when the the local directory was set to a network path that was 4 levels deep.

When I mapped a new drive letter directly to the same directory (eliminating the previous directories in the path), the performance improved to what I would expect.

[Alexander Schuch]

Link issue #8334 with a related problem.