The retention policy may not work properly for Windows 2008 domain controllers when differential backups are enabled.
Differential backups are not supported for Windows 2008 domain controllers. When Windows 2008 domain controllers are added to a collection where differential backups are enabled, Recovery Manager will create a full backup by default. The issue is, that these full backups are not counted against the retention policy and therefore keep increasing. This is considered a defect (TF00214083) in the product and will be resolved in a future release.
Remove the Windows 2008 domain controllers from collections where differential backups is enabled, as differential backups are not supported with Windows 2008 DC's.