Multistream backups failing through Checkpoint Firewalls (79045)

Date - OCT 2008
Affected Product & Version - NetVault: Backup ALL
Affected Module & Version - Vmware Plugin 1.X and Oracle PLugin ALL
OS Version - ALL
Application Information - Vmware and Oracle ALL versions


Description:
This article explains why multistreams backups fail through Checkpoint firewalls.
Backing up the same client via the File Sytem Plugin succeeds.


Symptoms:
When backing up a NetVault client via the Oracle, VMware or other "multistreamed" plugins, the jobs would fail with a channel error.
The same client can be successfully backed up with the File System plugin, proving the firewall configurations from both sides are correct.

Some of the errors logged would simply point to a channel error during data transfer:
Error 2008/07/31 10:14:02 62 Media BUP BUP X:\vtl\drives\8: had channel error
NET :2148 403 0 91320 Got list of '1' ports
NET :2148 400 0 91320 NetBindTcpSocketUsingFWPortList(0012F8F4, 0x080)
NET :2148 17 10048 91320 Fatal error during address bind: Address already in use
NET :2148 17 10048 91320 Fatal error during address bind: Address already in use
NET :2148 17 10048 91320 Fatal error during address bind: Address already in use
DEVMGR :2148 776 0 91350 Data Channel Connect: FAILURE

Network Manager Trace would also show:
4 MESSAGE :16024 47 0 130806 MsgChannelReceiveData loop... 0 0 8 (Ipc = FALSE)
0 NET :16024 15 131 130806 Fatal error during TCP receive: Connection reset by peer
3 MESSAGE :16024 53 0 130806 NetTcpReceive returned unknown error
4 MESSAGE :16024 45 0 130806 MsgChannelDown(74840) 11
NET :16024 15 131 130806 Fatal error during TCP receive: Connection reset by peer
3 MESSAGE :16024 53 0 130806 NetTcpReceive returned unknown error


Reasons:
This behaviour is mainly due to the way Checkpoint Firewalls rule and inspect the state of the open connections they manage.<

The Checkpoint Firewall stateful inspection behaviour can be configured on Checkpoint Firewalls.

- In the Checkpoint Global Properties, reduce the 'TCP End Timeout' setting from the default '180' seconds, down to a much lower value.
- This value is not definite and may vary depending on the firewall version.
TCP End Timeout values between 1 to 30 seconds should allow NetVault to function with no problems.

A low TCP End Timeout value will allow for the firewall connection table entries to be dropped early enough to allow the reuse of that same connection without raising suspicions.
Note: Using a smaller value could cause some connections to be prematurely terminated.