Compression

Author: Michael J. Leaver, 2BrightSparks Pte. Ltd.

File compression reduces the size of files at the cost of processing time. There are many reasons and benefits to compressing files, especially for backup or archiving:

• A compressed file takes up less storage space.
• Smaller files can be transferred over a network quicker, e.g. storing on the cloud. They can also be downloaded in less time.
• If storage usage has a cost, e.g. storing on the cloud, then by reducing a files size you are reducing the cost of storing it.
• Archive file formats, like Zip, store file integrity data, e.g. CRC hash values, allowing for data integrity checking.
• Compressed files can also be encrypted.

There are also drawbacks to compressing files:

• Compressing, and decompressing, takes time. Depending on the compression method and level, it can be very slow.
• A corrupted compressed file may be more difficult to restore or repair than an original uncompressed file.
• Some file types cannot be compressed or benefit very little from being compressed.
• There is overhead in getting the meta-data of a file in a compressed archive. For example, if it's stored on remote storage like the cloud, then you may need to download the entire archive file to see the contents.

Multi Zip and Single Zip

SyncBack supports two different methods of using compression.

Single Zip

All the files are stored in a single Zip file. If you want to encrypt the filenames in the Zip file then this is the only choice. The drawback of using Single Zip compression is that versioning is not supported and it can be cumbersome to use if the backup destination is remote. When the Zip file is stored remotely, e.g. cloud or FTP, then the entire Zip file is always replaced. This is because downloading the Zip file, updating it, and re-uploading it would take a long time. Although versioning is not supported with Single Zip compression, you could use variables (e.g. %AUTOINC%) in the Zip filename to keep multiple backups.

Multi Zip

Each file is stored in its own Zip file. Unlike Single Zip, versioning can be used with Multi Zip. Also, there is no issue of updating a remote Zip file as with Multi Zip each file is stored in its own Zip file, so they are always replaced anyway. Multi Zip can also be considerably faster than Single Zip because multiple files can be compressed and decompressed in parallel.

Compression Methods

SyncBack supports several compression methods, each of which have their benefits and drawbacks.

For high performance, ZStandard is a good choice, if the compression level is normal or lower. For maximum compatibility, choose Deflate64 or Deflate. For the best encryption, use BZip2 with 256-bit AES encryption and encrypt the filenames (this produces a Zip file that can be read using PKZip SecureZip). For the best compression, choose LZMA or LZMA2/XZ (if performance is not an issue).

Compression Levels

Every compression method supports 9 levels of compression, with 1 being the lowest (and quickest) and 9 being the highest (and slowest). A key point is that the higher the compression level then the longer it will take to compress (and possibly decompress) a file. Higher compression levels usually require large amounts of memory and more CPU time.

Often there is little benefit to choosing the maximum compression level as it can greatly increase the backup time with very little reduction in the storage space used. The normal compression level (5) offers a balance of compression and performance. We have a knowledge base article that compares the various compression methods and compression levels.

Uncompressible

Some files cannot be compressed as they are already compressed, or at best they can be compressed but by very little. By default SyncBack will not try and compress well-known already compressed file types. This is based on the filename extension, e.g. .JPG. You can modify your profile to add or remove filename extensions (see the Compression -> Compressed settings page for a profile). When SyncBack encounters of file using one of these extensions it will not waste time trying to compress the file but instead store it uncompressed in the Zip file.

SyncBack V11 introduced a new option on the Compression -> Compressed settings page: Auto-detect files that may not compress. When this is enabled, SyncBack will run a quick test to see if part of the file (the middle 1MB of the file) can be compressed. If it cannot, then it is assumed that the entire file cannot be compressed. Keep in mind that it is impossible to know if a file can be compressed unless you actually try and compress the entire file. However, for the majority of files, if part of it cannot be compressed then it is probable that none of it can be compressed. By enabling this option, you may be able to drastically reduce the backup time. If a file is already known to be compressed (based on the filename extension) then no test is made as it will not be compressed anyway.

Conclusion

With compression, it is tempting to automatically use the highest compression level available. After all, the purpose of compression is to reduce storage space. However, the higher levels of compression can be extremely slow and very taxing on the CPU and memory. Choosing the normal compression level provides better performance and often similar levels of compression.