No. The retry option only affects the alarm SQL Server connection failure.
To achieve this for the SQL Server Secondary failure, it's possible to create an alarm action like below to
avoid receiving false alerts by email:
For all alarms
where the alarm is Monitored Server - SQL Server Secondary Connection Failure
and the alarm has existed for more than n minutes
send email to (recipient)