Opened 12 years ago
#8380 new Bug report
Massive Memory and CPU utilization with large dataset Uploads
| Reported by: | amarcionek | Owned by: | |
|---|---|---|---|
| Priority: | normal | Component: | FileZilla Client |
| Keywords: | Cc: | ||
| Component version: | Operating system type: | Windows | |
| Operating system version: | Windows Server 2008 R2 running under VMware Workstation 8 |
Description
FileZilla 3.6.0.2
Windows Server 2008 R2 running under VMware Workstation 8
Two Intel CPUs
2 GB memory
Windows 7 Ultimate 64-bit physical machine.
8 Intel CPUs
16 GB memory
I have been trying to use FileZilla to test an FTP server implementation and how it scales using millions of files. My data set contains 10 million 1 KB files in a fairly deep directory structure. What I have found is that after a few minutes, FileZilla becomes non-responsive. A check of the memory sees it climbing very high. Some test runs, I'm able to get it to crash when the memory gets to ~2 GB consumed, probably because its a 32-bit process. Other times, the CPU just spikes to 100% and the process just looks like its hung when looking at the logs on the server side.
This is pretty simple to reproduce on the two machines listed above.
I personally think it has something to do with the queueing mechanism and how much memory is consumed per item. The queue can grow quite faster than files can transfer leading to rather large lists on the queued side.
Memory and CPU consumption