A critical vSphere 6.0 CBT issue during backup has been recently discovered that results in inconsistent incremental virtual machine backups.
All incremental backups which utilize CBT are potentially affected and you should disable the CBT function wherever is possible to avoid data loss.
Changed Block Tracking (CBT), a native VMware feature, backs up only the blocks that have changed, rather than backing up every block of every VM in the infrastructure. This backup solution allows to save disk space and requires less time to complete.
This issue occurs due to a problem with CBT in the disklib area, which results in the change tracking information of I/Os that occur during snapshot consolidation to be lost. The main backup payload data is never lost and it is always written to the backend device. However, the corresponding change tracking information entries which occur during the consolidation task are missed. The vSphere API SubsequentQueryDiskChangedAreas() calls do not include these missed blocks and therefore a backup based on this CBT data is inconsistent.
The problem is reported in the KB 2136854.
Fix
No solution is currently available to fix this matter and the only workaround is to disable CBT or downgrade the ESXi hosts to version 5.5.
If Veeam or Trilead backup solutions are used in your environment, the CBT feature can be disabled performing regular backups that remain consistent.
Update 27/11/2015: VMware released the patch to fix the CBT issue.
Veeam Backup & Replication
Veeam users can disable the use of CBT in the job configuration keeping the incremental backups but space and time required are increased. In the screenshot, an example of backup using CBT.
To disable CBT, edit the Backup Job and select the Storage section. Click on Advanced button to access the advanced job settings.
Select vSphere tab and disable the Use changed block tracking data (recommended) option then click OK to confirm.
Click Finish to save the backup job with CBT disabled.
Trilead VM Explorer
To disable CBT in Trilead Explorer, access the web console and select Datacenter tab. Right click the affected ESXi hosts and select the Edit Server option.
Select Advanced Setting section and disable the Use VD Service. Click Finish to save the configuration.
If your environment is still running vSphere 5.5, keep going this way. Due to the current issues with the new release, not sure if time to upgrade to version 6.0 has arrived.
