Why is ReFS is so fast with Veeam backup merges and why it saves so much space


Placement of backups and backup copy jobs is an ongoing discussion and a very interesting one if you’d ask me. Driven by the ever-changing technologies and demand for storage capacity this is not going to stop soon. Once we’re used to a feature and how to use it, it’s almost bound to be replaced by an even newer better version (of itself).
One of the later changes we’ve seen is the introduction of Microsoft’s latest proprietary file system ReFS 3.1 , short for Resilient File System in Windows Server 2016. While it was first introduced as, you might guess this one, more resilient replacement of the New Technology File System, or NTFS. It’s latest release includes features which do not only help against data degradation or the need to run CHKDSK, but also helps businesses immensely by providing block cloning.

ReFS merge


But how does this work?

Normally when backup data is written to an NTFS formatted disk, the data is written as a collection of smaller attributes which include the actual data and are defined on disk in what we call a ‘run’. The start of each run is identified using a VCN, or Virtual Cluster Number. Inside those runs, pointer are created to refer to the start of a file.
Whenever that file needs to be copied all data including the pointer needs to be copied to a new portion on disk and writing a new VCN to it indicating that this cluster actually belongs to that copy of File X. Merging of a backup file effectively is nothing more than a copy of a file with another VCN. This the main reason why backup file merging, when you use the Forever Forward Backup method, can take such a long time whenever a lot of changes have been written to disk;
After all the data has been processed by the Veeam backup proxy and written to disk by the Veeam Repository a backup job needs to apply the retention policy set on a backup job. Effectively creating a huge workload on the disks and generating a lot of I/O before the operation is done creating a huge bottleneck for your backups. Each merge operation generates 2 I/O on disk and because we write on the same disk that we read from we have a lot of wait time before the right portion of a spindle passes underneath a disk’s needle.

With all this information it might be nice to know that a NL-SAS has around 75-100 I/O operations per second, or IOPS. 10K-SAS has 125-150 IOPS, 15K-SAS has around 175-210 IOPS. (source: https://en.wikipedia.org/wiki/IOPS)

To overcome a problem

With ReFS 3.1 this problem is overcome using the Block Clone technology. ReFS is able to perform copies at metadata level instead of copying a series of clusters and is able to perform a copy or merge operation by altering the VCN to a LCN (Logical Cluster Number). With ReFS a file can have the same the same logical cluster allowing a file to share physical disk locations. With this, file copies and merges are done much faster generating a lot less I/O on the storage.

A nice example of the use of Fast Cloning, or Block Cloning, is when Veeam creates a Synthetic Full Backup. After a Daily incremental is written to disk (Block C1) it then used to merge the all the existing blocks into 1 Full Backup backup file by simply writing only metadata on disk, create a full backup which essentially used no space on disk and is created in a very short time.


Backup Statistics


In ReFS on Windows Server 2016, clusters are 4KB in size by default, but when using it together with Veeam, should be set to 64KB for optimal performance. The cluster size is a volume-wide parameter set at format time. Also the ReFS volume must have been formatted with Windows Server 2016, and if Windows Failover Clustering is in use, the Clustering Functional Level must have been Windows Server 2016 or later at format time.

Veeam support

Veeam supports ReFS as of version 9.5 and leverages Fast Clone (Block Cloning) for the following synthetic operations:

In backup jobs:

  • Merge of backup files
  • Synthetic full backup
  • Reverse incremental backup transformation
  • Compact of full backup file

In backup copy job:

  • Merge of backup files
  • Creation of GFS backups (synthetic method)
  • Compact of full backup file

ReFS integration is explained in this WP from Rick Vanover

By |2017-09-28T17:49:00+00:00September 28th, 2017|Veeam|0 Comments