Please make sure your system meets the requirements on the Release Notes.
Spotlight on SQL Server is designed to monitor a maximum of 100 SQL Servers, Analysis Services or Replication instances and 100 Windows servers. Exceeding this recommended limit on 32 bit environments may result in poor performance or product instability, due to the 1 Gb memory limit. On 64 bit environments theoretically more connections can be monitored as the 1 Gb limit no longer applies, however testing of this has been limited.
If you are exceeding this recommendation we recommend to break the environment into several Diagnostic Servers domains to handle such a large connection base.
Note also that the maximum memory parameter in the SpotlightDiagnosticServer.ini file is set to 2048 Mb. You can set it to a higher value if you really want to use just one Diagnostic Server.
To do this backup the file {DS dir}\Agent\bin\ SpotlightDiagnosticServer.ini and then edit it and change
"-Xmx2048m"
You could try doubling the value e.g.
"-Xmx4096m"
