Opened 3 years ago

Last modified 12 months ago

#12524 reopened Bug report

Colon in filenames on server incorrectly translated to underscore resulting in "No such file or directory" error - Windows Devices

Reported by: MorpheusNyix Owned by:
Priority: high Component: FileZilla Client
Keywords: "No such file or directory", colon, underscore, file exists on server Cc:
Component version: 3.63.2.1 Operating system type: Windows
Operating system version: 22H2 (OS Build 19045.2846)

Description

When attempting to download files over FTP using the queue the download fails with the error "No such file or directory".

This appears to only happen to files with colons in their name and only if the files are added to the queue before they are downloaded.

Sample filenames that fail:

1410488582.H473970P287028.slmp-001-001.sub-domain.domain.com,S=28627:2,S
1416539102.H702473P565036.slmp-001-001.sub-domain.domain.com,S=4132:2,S
1429859102.H103707P208334.slmp-001-001.sub-domain.domain.com,S=10776:2,S

Filename: 1610943074.M291083P16808.slmp-001-001.sub-domain.domain.com,S=3489,W=3545:2,

Sample Connection Log (Processed from Queue):

Status:	Resolving address of ftp.domain.com
Status:	Connecting to 255.255.255.255:21...
Status:	Connection established, waiting for welcome message...
Response:	220---------- Welcome to Pure-FTPd [privsep] [TLS] ----------
Response:	220-You are user number 1 of 50 allowed.
Response:	220-Local time is now 14:34. Server port: 21.
Response:	220-This is a private system - No anonymous login
Response:	220-IPv6 connections are also welcome on this server.
Response:	220 You will be disconnected after 15 minutes of inactivity.
Command:	AUTH TLS
Response:	234 AUTH TLS OK.
Status:	Initializing TLS...
Status:	Verifying certificate...
Status:	TLS connection established.
Command:	USER username
Response:	331 User username OK. Password required
Command:	PASS *********
Response:	230 OK. Current restricted directory is /
Command:	OPTS UTF8 ON
Response:	504 Unknown command
Command:	PBSZ 0
Response:	200 PBSZ=0
Command:	PROT P
Response:	200 Data protection level set to "private"
Status:	Logged in
Status:	Starting download of /mail/domain.com/administrator/cur/1610943074.M291083P16808.slmp-001-001.sub-domain.domain.com,S=3489,W=3545_2,
Command:	CWD /mail/domain.com/administrator/cur
Response:	250 OK. Current directory is /mail/domain.com/administrator/cur
Command:	TYPE I
Response:	200 TYPE is now 8-bit binary
Command:	PASV
Response:	227 Entering Passive Mode (255,255,255,255,144,137)
Command:	RETR 1610943074.M291083P16808.slmp-001-001.sub-domain.domain.com,S=3489,W=3545_2,
Response:	550 Can't open 1610943074.M291083P16808.slmp-001-001.sub-domain.domain.com,S=3489,W=3545_2,: No such file or directory
Error:	Critical file transfer error
Status:	Disconnected from server

Sample Connection Log (Downloaded Directly):

Status:	Resolving address of ftp.domain.com
Status:	Connecting to 255.255.255.255:21...
Status:	Connection established, waiting for welcome message...
Response:	220---------- Welcome to Pure-FTPd [privsep] [TLS] ----------
Response:	220-You are user number 1 of 50 allowed.
Response:	220-Local time is now 14:42. Server port: 21.
Response:	220-This is a private system - No anonymous login
Response:	220-IPv6 connections are also welcome on this server.
Response:	220 You will be disconnected after 15 minutes of inactivity.
Command:	AUTH TLS
Response:	234 AUTH TLS OK.
Status:	Initializing TLS...
Status:	Verifying certificate...
Status:	TLS connection established.
Command:	USER username
Response:	331 User username OK. Password required
Command:	PASS *********
Response:	230 OK. Current restricted directory is /
Command:	OPTS UTF8 ON
Response:	504 Unknown command
Command:	PBSZ 0
Response:	200 PBSZ=0
Command:	PROT P
Response:	200 Data protection level set to "private"
Status:	Logged in
Status:	Starting download of /mail/domain.com/administrator/cur/1610943074.M291083P16808.slmp-001-001.sub-domain.domain.com,S=3489,W=3545:2,
Command:	CWD /mail/domain.com/administrator/cur
Response:	250 OK. Current directory is /mail/domain.com/administrator/cur
Command:	TYPE I
Response:	200 TYPE is now 8-bit binary
Command:	PASV
Response:	227 Entering Passive Mode (255,255,255,255,183,64)
Command:	RETR 1610943074.M291083P16808.slmp-001-001.sub-domain.domain.com,S=3489,W=3545:2,
Response:	150 Accepted data connection
Response:	226-File successfully transferred
Response:	226 0.000 seconds (measured here), 10.27 Mbytes per second
Status:	File transfer successful, transferred 3,489 bytes in 1 second
Status:	Disconnected from server

I anticipate this bug has something to do with the way filenames are processed from the queue. From the above logs you can see the same file name is changed when processed from the queue.

1610943074.M291083P16808.slmp-001-001.sub-domain.domain.com,S=3489,W=3545:2,

becomes

1610943074.M291083P16808.slmp-001-001.sub-domain.domain.com,S=3489,W=3545_2,

It seems that the colon (:) in the filename is translated to an underscore (_) resulting in the server appropriately responding with "No such file or directory".

I anticipate this translation is performed so the file can be stored on a local Windows system that prevents the use of colon's in file names. However it appears that the code incorrectly translates the filename on the server as well and this only happens when the file is processed from the queue rather than a direct download.

Client information:

Version:          3.55.1

Build information:
  Compiled for:   x86_64-w64-mingw32
  Compiled on:    x86_64-pc-linux-gnu
  Build date:     2021-08-09
  Compiled with:  x86_64-w64-mingw32-gcc (GCC) 8.3-win32 20190406
  Compiler flags:  -O2 -g -Wall -Wextra -pedantic -ffunction-sections -fdata-sections -Wno-cast-function-type

Linked against:
  wxWidgets:      3.0.6
  SQLite:         3.31.1
  GnuTLS:         3.6.15

Operating system:
  Name:           Windows 10 (build 17134), 64-bit edition
  Version:        10.0
  Platform:       64-bit system
  CPU features:   sse sse2 sse3 ssse3 sse4.1 sse4.2 avx avx2 aes pclmulqdq rdrnd bmi bmi2 adx lm
  Settings dir:   C:\Users\UserName\AppData\Roaming\FileZilla\

Finally, I recently updated to 3.55.1 from a version 3.52.2. This bug was not present in that previous version so I anticipate it has been recently introduced somehow.

I have confirmed downgrading to version 3.52.2 resolves this issue while upgrading to 3.55.1 causes the issue to re-appear.

After a quick review of the source code in trunk (r10397) I discovered in /src/interface/QueueView.cpp on line 2974 in method CQueueView::ReplaceInvalidCharacters a code-site that may be responsible for swapping colons for underscores however as I do not have sufficient experience with the FileZilla codebase I'm uncertain if this is relevant.

std::wstring CQueueView::ReplaceInvalidCharacters(std::wstring const& filename, bool includeQuotesAndBreaks)
{
	auto& options = *COptions::Get();
	if (!options.get_int(OPTION_INVALID_CHAR_REPLACE_ENABLE)) {
		return filename;
	}

	wchar_t const replace = options.get_string(OPTION_INVALID_CHAR_REPLACE)[0];

	std::wstring ret = filename;
	for (auto & c : ret) {
		if (IsInvalidChar(c, includeQuotesAndBreaks)) {
			c = replace;
		}
	}
	return ret;
}

Further review of that source file shows that line 359 may also be related, but again I don't have sufficient experience with the codebase to know for certain.

An ideal patch would preserve the server file-name in the queue while changing it locally to be compatible with Windows filenames.

If any additional information is required to resolve this please let me know.

To the development team, great work on FileZilla I've been using it for years and this is the first time I've encountered a (minor) bug. Thank you for your hard work.

Change History (8)

comment:1 by MorpheusNyix, 21 months ago

Priority: normal → high

This issue is still present in FileZilla 3.60.2.

Escalating ticket priority as this has not been acknowledged in 11 months.

comment:2 by Tim Kosse, 21 months ago

Priority: high → normal

comment:3 by Tim Kosse, 21 months ago

Resolution: → worksforme
Status: new → closed

Not reproducible.

comment:4 by Shawn Wilson, 12 months ago

So how does one push on this ticket to get it re-opened. Because this is definitely reproducible.

All of these files have colons in their filenames and filezilla is asking for an underscore on the server side instead of subbing in the underscore on the Windows client side.

Just updated to version 3.63.2.1 tonight with no change.

---snip of log---
Status: Starting download of /home/witadmin/scanner_archive/2021/05/03/2_1_2021-05-03T23_45_00.mp3
Command: get "2_1_2021-05-03T23_45_00.mp3" "Z:\Projects\Scanner Archive (police scanner)\03\2_1_2021-05-03T23_45_00.mp3"
Error: /home/witadmin/scanner_archive/2021/05/03/2_1_2021-05-03T23_45_00.mp3: open for read: no such file or directory
Error: File transfer failed
Status: Starting download of /home/witadmin/scanner_archive/2021/05/03/2_1_2021-05-03T00_05_00.mp3
Command: get "2_1_2021-05-03T00_05_00.mp3" "Z:\Projects\Scanner Archive (police scanner)\03\2_1_2021-05-03T00_05_00.mp3"
Error: /home/witadmin/scanner_archive/2021/05/03/2_1_2021-05-03T00_05_00.mp3: open for read: no such file or directory
Error: File transfer failed
Status: Starting download of /home/witadmin/scanner_archive/2021/05/03/2_1_2021-05-03T23_45_00.mp3
Command: get "2_1_2021-05-03T23_45_00.mp3" "Z:\Projects\Scanner Archive (police scanner)\03\2_1_2021-05-03T23_45_00.mp3"
Error: /home/witadmin/scanner_archive/2021/05/03/2_1_2021-05-03T23_45_00.mp3: open for read: no such file or directory
Error: File transfer failed
Status: Starting download of /home/witadmin/scanner_archive/2021/05/03/2_1_2021-05-03T23_55_00.mp3
Status: Starting download of /home/witadmin/scanner_archive/2021/05/03/2_1_2021-05-03T00_05_00.mp3
Command: get "2_1_2021-05-03T00_05_00.mp3" "Z:\Projects\Scanner Archive (police scanner)\03\2_1_2021-05-03T00_05_00.mp3"
Error: /home/witadmin/scanner_archive/2021/05/03/2_1_2021-05-03T00_05_00.mp3: open for read: no such file or directory
Error: File transfer failed
---snip of log---

comment:5 by Shawn Wilson, 12 months ago

Set logging to 3, reproduced the bug. Had to get the log in two steps because the log only keeps 999 lines and in my case I get thousand of errors because the file list is huge.

First Part of log showing a successful download of a single file with colons in the name and Filezilla successfully replacing them with underscores.

---log---
Status: Connecting to 10.10.10.90...
Status: Using username "witadmin".
Status: Connected to 10.10.10.90
Status: Retrieving directory listing...
Status: Listing directory /home/witadmin
Status: Directory listing of "/home/witadmin" successful
Status: Retrieving directory listing of "/home/witadmin/scanner_archive"...
Status: Listing directory /home/witadmin/scanner_archive
Status: Directory listing of "/home/witadmin/scanner_archive" successful
Status: Retrieving directory listing of "/home/witadmin/scanner_archive/2021"...
Status: Listing directory /home/witadmin/scanner_archive/2021
Status: Directory listing of "/home/witadmin/scanner_archive/2021" successful
Status: Retrieving directory listing of "/home/witadmin/scanner_archive/2021/05"...
Status: Listing directory /home/witadmin/scanner_archive/2021/05
Status: Directory listing of "/home/witadmin/scanner_archive/2021/05" successful
Status: Retrieving directory listing of "/home/witadmin/scanner_archive/2021/05/02"...
Status: Listing directory /home/witadmin/scanner_archive/2021/05/02
Status: Directory listing of "/home/witadmin/scanner_archive/2021/05/02" successful
Status: Connecting to 10.10.10.90...
Status: Using username "witadmin".
Status: Connected to 10.10.10.90
Status: Starting download of /home/witadmin/scanner_archive/2021/05/02/2_1_2021-05-02T02:55:01.mp3
Status: File transfer successful, transferred 427,407 bytes in 1 second
Status: Retrieving directory listing of "/home/witadmin/scanner_archive/2021/05"...
Status: Directory listing of "/home/witadmin/scanner_archive/2021/05" successful

comment:6 by Shawn Wilson, 12 months ago

Part 2 showing the last chunk of 999 lines of the log with all the failures when I right-click on a folder and choose download, or drag a folder from server to local.

---log snippet---
Status: Starting download of /home/witadmin/scanner_archive/2021/05/02/2_1_2021-05-02T06_30_01.mp3
Command: get "2_1_2021-05-02T06_30_01.mp3" "Z:\Projects\Scanner Archive (police scanner)\02\2_1_2021-05-02T06_30_01.mp3"
Error: /home/witadmin/scanner_archive/2021/05/02/2_1_2021-05-02T06_30_01.mp3: open for read: no such file or directory
Error: File transfer failed
Status: Starting download of /home/witadmin/scanner_archive/2021/05/02/2_1_2021-05-02T06_30_01.mp3
Command: get "2_1_2021-05-02T06_30_01.mp3" "Z:\Projects\Scanner Archive (police scanner)\02\2_1_2021-05-02T06_30_01.mp3"
Error: /home/witadmin/scanner_archive/2021/05/02/2_1_2021-05-02T06_30_01.mp3: open for read: no such file or directory
Error: File transfer failed
Status: Starting download of /home/witadmin/scanner_archive/2021/05/02/2_1_2021-05-02T06_30_01.mp3
Command: get "2_1_2021-05-02T06_30_01.mp3" "Z:\Projects\Scanner Archive (police scanner)\02\2_1_2021-05-02T06_30_01.mp3"
Error: /home/witadmin/scanner_archive/2021/05/02/2_1_2021-05-02T06_30_01.mp3: open for read: no such file or directory
Error: File transfer failed
Status: Starting download of /home/witadmin/scanner_archive/2021/05/02/2_1_2021-05-02T21_10_01.mp3
Command: get "2_1_2021-05-02T21_10_01.mp3" "Z:\Projects\Scanner Archive (police scanner)\02\2_1_2021-05-02T21_10_01.mp3"
Error: /home/witadmin/scanner_archive/2021/05/02/2_1_2021-05-02T21_10_01.mp3: open for read: no such file or directory
Error: File transfer failed
Status: Starting download of /home/witadmin/scanner_archive/2021/05/02/2_1_2021-05-02T21_10_01.mp3
Command: get "2_1_2021-05-02T21_10_01.mp3" "Z:\Projects\Scanner Archive (police scanner)\02\2_1_2021-05-02T21_10_01.mp3"
Error: /home/witadmin/scanner_archive/2021/05/02/2_1_2021-05-02T21_10_01.mp3: open for read: no such file or directory
Error: File transfer failed
Status: Starting download of /home/witadmin/scanner_archive/2021/05/02/2_1_2021-05-02T21_10_01.mp3
Command: get "2_1_2021-05-02T21_10_01.mp3" "Z:\Projects\Scanner Archive (police scanner)\02\2_1_2021-05-02T21_10_01.mp3"
Error: /home/witadmin/scanner_archive/2021/05/02/2_1_2021-05-02T21_10_01.mp3: open for read: no such file or directory
Error: File transfer failed
Status: Starting download of /home/witadmin/scanner_archive/2021/05/02/2_1_2021-05-02T05_50_01.mp3
Command: get "2_1_2021-05-02T05_50_01.mp3" "Z:\Projects\Scanner Archive (police scanner)\02\2_1_2021-05-02T05_50_01.mp3"
Error: /home/witadmin/scanner_archive/2021/05/02/2_1_2021-05-02T05_50_01.mp3: open for read: no such file or directory
Error: File transfer failed
Status: Starting download of /home/witadmin/scanner_archive/2021/05/02/2_1_2021-05-02T05_50_01.mp3
Command: get "2_1_2021-05-02T05_50_01.mp3" "Z:\Projects\Scanner Archive (police scanner)\02\2_1_2021-05-02T05_50_01.mp3"
Error: /home/witadmin/scanner_archive/2021/05/02/2_1_2021-05-02T05_50_01.mp3: open for read: no such file or directory
Error: File transfer failed

comment:7 by Shawn Wilson, 12 months ago

Component version: 3.55.1 → 3.63.2.1
Operating system version: 1803 (OS Build 17134. 1246) → 22H2 (OS Build 19045.2846)
Priority: normal → high
Resolution: worksforme
Status: closed → reopened

Re-opened ticket. If you need anything else, let me know. This is reproducible every time.

In my case, the connection type is SFTP and I am connecting to an Ubuntu 18.04.5 LTS Linux server.

Note: See TracTickets for help on using tickets.