The following error is seen "mount error(12): Cannot allocate memory" (85510)

During troubleshooting (Direct to target) backup issues a mount test is usually recommended. When mounting the vRanger repository directly to your ESX host you may get the following error: Mount error(12): Cannot allocate memory

You may also see the following event on the repository host.
Event ID: 2017
Level: Error
The server was unable to allocate from the system nonpaged pool because the server reached the configured limit for nonpaged pool allocations.

The repository host may not be configured for server-like loads

Please use the following steps to correct your issue:

1.Edit the registry on the repository computer and restart the server service
Using Registry Editor incorrectly can cause serious problems that may require you to reinstall your operating system. Microsoft cannot guarantee that problems resulting from the incorrect use of Registry Editor can be solved. Use Registry Editor at your own risk.
2. Edit HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management\LargeSystemCache
Set the registry key to ‘1'
3. HKLM\SYSTEM\CurrentControlSet\Services\LanmanServer\Parameters\Size
Set the registry key to ‘3'
4.Restart the server

 Additional Information:

Registry settings were explained in an expired technet article. Portion on that article is below.

        Tuning  NT Workstation and NT Server have vastly different performance
   Workstation  characteristics due to the internal tuning that the NT operating
           for  system, which is identical on both, performs. Most tuning
   Server-like  parameters are inaccessible, but a few are located in the Registry.
        Loads   If you are running Server and you double-click on the Server entry
                of the Services tab in the Control Panel's Network applet, you will
                get a dialog that lets you determine what type of application you
                want the machine to be tuned for. Choices let you select between
                "Minimize Memory Used", "Balance", "Maximize Usage for File
                Sharing", and "Maximize Usage for Network Applications". This
                dialog box is not presented on Workstation installations. The
                various selections actually change the values of two Registry
                values:

                HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\Memory
                Management\LargeSystemCache

                and

                HKLM\SYSTEM\CurrentControlSet\Services\LanmanServer\Parameters\Size

                This table (which was derived from sessions with NTRegmon) presents
                the settings you should select on a Workstation to achieve the same
                effect you would get using the dialog box were your system a
                Server.

                    Tuning Target      LargeSystemCache  Size

                 Minimize Memory Used         0            1

                       Balance                0            2

                     File Sharing             1            3

                 Network Applications         0            3

The full text of the expired article can be found - http://www.decuslib.com/decus/vmslt99a/nt/tips.txt.