Essential FTP Features

Author: Kostas Tsakiridis, 2BrightSparks Pte. Ltd.

SyncBack’s embedded FTP engines support a wide range of FTP servers and in most cases it will work fine with them by using the default FTP settings. However, if the FTP server doesn’t support some important features, SyncBack may fallback to using alternative methods, which may have an impact on performance.

In this article we will go through the various FTP features/extensions which the user should be looking for when deciding which FTP server to install and use with SyncBack.

Note that we are referring to FTP and not SFTP, which is a different protocol. All editions of SyncBack support FTP (and FTPS), but only SyncBackPro supports SFTP.

Unicode

Unicode (UTF-8) support is one of the most important features an FTP must have in order to properly transfer files with non-ASCII names. SyncBack will check with the server to see if it supports Unicode and will try to send/receive filenames in Unicode if supported (or if the profile settings tell it explicitly to do so anyway). Some FTP servers do not correctly support UTF-8 so users may want to configure SyncBackPro not to use it. Also, some FTP servers do support it but do not tell FTP clients that they do in which case users may force the use of Unicode in the profile settings.

Failing to properly identify the Unicode capabilities of the FTP server may lead to mismatched/erroneous file names. Many modern FTP servers do support Unicode, but there are some popular ones (typically used by web hosts) that do not.

Simultaneous Connections

When SyncBack scans an FTP server, to see what files and folders are there, it does it one folder at a time on a single connection. This reduces the load on the FTP server, and in some cases, FTP servers only allow one connection at a time from a user. However, if you are allowed multiple simultaneous connections to an FTP server, and server load is not an issue, then you can configure SyncBack to use multiple connections to scan the FTP server. This can drastically reduce the scanning time, but results can vary. You will get the best results when you have multiple folders. Multi-threaded FTP scanning will not improve performance if you are only scanning the contents of one folder that only contains files. So, when deciding which FTP server to use, check whether it allows more than one simultaneous connection per user.

Transfer Compression (Mode Z)

If the FTP server supports the MODE Z FTP extension, it may increase performance on slower networks. It will transmit data to and from the server in compressed form to reduce the transfer time. Enabling this option when the FTP server is on a LAN will actually decrease performance (as the time taken to compress and decompress may be longer than the time to simply send it uncompressed). Also, if you are transferring already compressed data (e.g. compressed files, music, images, etc.) then you may also decrease performance by using MODE Z.

Directory Listing

Initial versions of the FTP protocol, which date back to the 1970s, were designed expecting to be operated directly by users via command line terminals. Those implementations were quite challenging for automation systems when it comes to reading the directory listing and the various file properties such as timestamps, file size and file attributes. Modern FTP extensions such as MLSD and MLST provide a standardized directory listing output which can be parsed easier by FTP clients and therefore you must make sure your future FTP server supports these. SyncBack will fallback to using the traditional LIST command if MLSD/MLST is not available or the user may force the use of LIST command, something that is not recommended.

Additionally, SyncBack supports a special feature of CompleteFTP server (V12.1.0 or newer) which enables a single-call directory scanning. SyncBackPro will automatically use this feature if it is available, resulting in considerably faster scanning of the FTP server.

Timestamps

An essential step during file upload to an FTP server is to set the file’s last modification timestamp so it can match with the source’s timestamp. The server needs to support the following extensions to achieve this:

• MDTM
• MFCT
• MFMT
• SITE UTIME

If SyncBack fails to set the appropriate timestamps then the uploaded files’ timestamps will be set to the actual time of the upload causing the files to show as being different during the next profile execution which may lead to re-uploads, especially during unattended profile run.

If the FTP server is unable to set the files’ timestamps then the only solution is to use Fast Backup.

Hashing

To verify that files have been transferred correctly to and from an FTP server, hashing support is required on the FTP server. This is also required by SyncBack when using File Integrity. The server needs to support one or more of the following extensions:

• XCRC
• XMD5
• XSHA1
• XSHA256

If the FTP server does not support one of these extensions then SyncBack cannot verify files are transferred correctly, file integrity cannot be used and change detection based on a files contents cannot be used.

Conclusion

SyncBack has supported FTP for over 15 years (since V3). When using FTP with SyncBack, it is important that a reliable and competent FTP server is used. Although the FTP protocol is very mature it is not always implemented correctly which can lead to numerous issues. Over the years a considerable amount of effort has been put into making SyncBack work seamlessly with as many FTP servers as possible. However, when using FTP, a broken or inadequate FTP server will stop SyncBack working correctly. With FTP, SyncBack relies on the FTP server to perform actions on its behalf. If it cannot then problems will occur that cannot simply be bypassed.

If possible, we recommend using SyncBack Touch instead of FTP. There are numerous advantages to using SyncBack Touch instead of FTP.