ADMs are used by the GPO editor to show Administrative Template settings. Once you select, and then deselect a setting, the registry.pol file is modified to reflect that setting.
If you delete a .ADM file, it does not change the behavior of the GPO. It is not advisable to remove the ADM template, because you would then not be able to use the GPO editor to remove/disable those settings.
The proper way to stop a GPO from applying a setting is to edit the GPO, and then reverse/remove the setting.