I have over 150 Openswan ipsec vpn tunnels from various parts of North America all connecting to a Fortigate 310b firewall/router
The majority of the tunnels come up fine. The problem is that after an hour or so some (not all) of the tunnel drop for some unknown reason.
I have created a script on each client that checks to see if the tunnel is up and if it is not, the tunnel is to be reestablished which works fine.
The problem is that I am trying to run backup scripts through these tunnels and there up and down which is ruining my backups.
I have been fighting this problem for many days and I have spent countless hours searching forum posts to no avail.

FORTIGATE CONFIG
PHASE1
3DES-SHA1
3DES-MD5
AES128-MD5
DIFFE GROUPS 2&5
KEYLIFE 86400s
NAT-TRANSVERSAL ENABLED
KEEP ALIVE FREQ 10
DPD ENABLE
XAUTH SERVER = YES

PHASE2
3DES-SHA1
3DES-MD5
AES128-MD5
REPLAY DETECTION ENABLED
PFS ENABLED
DIFFE GROUPS 5
KEYLIFE 86400s
AUTO KEEP ALIVE ENABLED
DCHPIPSEC DISABLED

OPENSWAN IPSEC.CONF
config setup
nat_traversal=yes
protostack=netkey
conn home
#CLIENT
leftxauthclient=XXXXXX
leftxauthusername=XXXXXX
leftsourceip=VAR2
left=%defaultroute
#REMOTEHOST
rightxauthserver=yes
right=XXX.XXX.XXX.XXX
rightsubnet=192.168.80.0/24
#GENERAL
authby=secret
auto=start
compress=no
type=tunnel
pfs=yes
forceencaps=yes
#PHASE1
ike=3des-sha1,3des-md5
keylife=86400s
#PHASE2
phase2=esp
phase2alg=3des-sha1,3des-md5;modp1536
ikelifetime=86400s
#REKEYING
rekey=yes
rekeymargin=15m


Here is the tunnel being brought up successfully

Mar 20 08:46:43 claimtools pluto[15697]: "home" #1: initiating Main Mode
Mar 20 08:46:43 claimtools pluto[15697]: "home" #1: received Vendor ID payload [RFC 3947] method set to=109
Mar 20 08:46:43 claimtools pluto[15697]: "home" #1: received Vendor ID payload [Dead Peer Detection]
Mar 20 08:46:43 claimtools pluto[15697]: "home" #1: enabling possible NAT-traversal with method 4
Mar 20 08:46:43 claimtools pluto[15697]: "home" #1: transition from state STATE_MAIN_I1 to state STATE_MAIN_I2
Mar 20 08:46:43 claimtools pluto[15697]: "home" #1: STATE_MAIN_I2: sent MI2, expecting MR2
Mar 20 08:46:43 claimtools pluto[15697]: "home" #1: NAT-Traversal: Result using RFC 3947 (NAT-Traversal): both are NATed
Mar 20 08:46:43 claimtools pluto[15697]: "home" #1: transition from state STATE_MAIN_I2 to state STATE_MAIN_I3
Mar 20 08:46:43 claimtools pluto[15697]: "home" #1: STATE_MAIN_I3: sent MI3, expecting MR3
Mar 20 08:46:43 claimtools pluto[15697]: "home" #1: Main mode peer ID is ID_IPV4_ADDR: 'xxx.xxx.xxx.xxxx’
Mar 20 08:46:43 claimtools pluto[15697]: "home" #1: transition from state STATE_MAIN_I3 to state STATE_MAIN_I4
Mar 20 08:46:43 claimtools pluto[15697]: "home" #1: STATE_MAIN_I4: ISAKMP SA established {auth=OAKLEY_PRESHARED_KEY cipher=oakley_3des_cbc_192 prf=oakley_sha group=modp1536}
Mar 20 08:46:43 claimtools pluto[15697]: "home" #1: XAUTH: Answering XAUTH challenge with user='xxxxx’
Mar 20 08:46:43 claimtools pluto[15697]: "home" #1: transition from state STATE_XAUTH_I0 to state STATE_XAUTH_I1
Mar 20 08:46:43 claimtools pluto[15697]: "home" #1: STATE_XAUTH_I1: XAUTH client - awaiting CFG_set
Mar 20 08:46:43 claimtools pluto[15697]: "home" #1: XAUTH: Successfully Authenticated
Mar 20 08:46:43 claimtools pluto[15697]: "home" #1: transition from state STATE_XAUTH_I0 to state STATE_XAUTH_I1
Mar 20 08:46:43 claimtools pluto[15697]: "home" #1: STATE_XAUTH_I1: XAUTH client - awaiting CFG_set
Mar 20 08:46:43 claimtools pluto[15697]: "home" #2: initiating Quick Mode PSK+ENCRYPT+TUNNEL+PFS+UP+IKEv2ALLOW {using isakmp#1 msgid:a0bec906 proposal=3DES(3)_192-SHA1(2)_160, 3DES(3)_192-MD5(1)_128 pfsgroup=OAKLEY_GROUP_MODP1536}
Mar 20 08:46:44 claimtools pluto[15697]: "home" #2: transition from state STATE_QUICK_I1 to state STATE_QUICK_I
Mar 20 08:46:44 claimtools pluto[15697]: "home" #2: STATE_QUICK_I2: sent QI2, IPsec SA established tunnel mode {ESP/NAT=>0x8b5c8037 <0x3bcfa844 xfrm=3DES_0-HMAC_SHA1 NATOA=none NATD=xxx.xxx.xxx.xxx:4500 DPD=none}

Tunnel was fine until now

Here is the failure point

Mar 20 10:35:57 claimtools pluto[15697]: "home" #1: DPD Info: received old or duplicate R_U_THERE
Mar 20 10:36:57 claimtools pluto[15697]: "home" #1: DPD Info: received old or duplicate R_U_THERE
Mar 20 10:37:57 claimtools pluto[15697]: "home" #1: received Delete SA payload: replace IPSEC State #2 in 10 seconds
Mar 20 10:37:57 claimtools pluto[15697]: "home" #1: received and ignored informational message
Mar 20 10:37:57 claimtools pluto[15697]: "home" #1: received Delete SA payload: deleting ISAKMP State #1
Mar 20 10:37:57 claimtools pluto[15697]: packet from XXX.XXX.XXX.XXX:4500: received and ignored informational message
Mar 20 10:38:07 claimtools pluto[15697]: "home" #3: initiating Main Mode
Mar 20 10:38:07 claimtools pluto[15697]: "home" #3: received Vendor ID payload [RFC 3947] method set to=109
Mar 20 10:38:07 claimtools pluto[15697]: "home" #3: received Vendor ID payload [Dead Peer Detection]
Mar 20 10:38:07 claimtools pluto[15697]: "home" #3: enabling possible NAT-traversal with method 4
Mar 20 10:38:07 claimtools pluto[15697]: "home" #3: transition from state STATE_MAIN_I1 to state STATE_MAIN_I2
Mar 20 10:38:07 claimtools pluto[15697]: "home" #3: STATE_MAIN_I2: sent MI2, expecting MR2
Mar 20 10:38:07 claimtools pluto[15697]: "home" #3: NAT-Traversal: Result using RFC 3947 (NAT-Traversal): both are NATed
Mar 20 10:38:07 claimtools pluto[15697]: "home" #3: transition from state STATE_MAIN_I2 to state STATE_MAIN_I3
Mar 20 10:38:07 claimtools pluto[15697]: "home" #3: STATE_MAIN_I3: sent MI3, expecting MR3
Mar 20 10:38:07 claimtools pluto[15697]: "home" #3: Main mode peer ID is ID_IPV4_ADDR: '70.67.129.119'
Mar 20 10:38:07 claimtools pluto[15697]: "home" #3: transition from state STATE_MAIN_I3 to state STATE_MAIN_I4
Mar 20 10:38:07 claimtools pluto[15697]: "home" #3: STATE_MAIN_I4: ISAKMP SA established {auth=OAKLEY_PRESHARED_KEY cipher=oakley_3des_cbc_192 prf=oakley_sha group=modp1536}
Mar 20 10:38:07 claimtools pluto[15697]: "home" #3: XAUTH: Answering XAUTH challenge with user='XXXXX'
Mar 20 10:38:07 claimtools pluto[15697]: "home" #3: transition from state STATE_XAUTH_I0 to state STATE_XAUTH_I1
Mar 20 10:38:07 claimtools pluto[15697]: "home" #3: STATE_XAUTH_I1: XAUTH client - awaiting CFG_set
Mar 20 10:38:07 claimtools pluto[15697]: "home" #3: XAUTH: Successfully Authenticated
Mar 20 10:38:07 claimtools pluto[15697]: "home" #3: transition from state STATE_XAUTH_I0 to state STATE_XAUTH_I1
Mar 20 10:38:07 claimtools pluto[15697]: "home" #3: STATE_XAUTH_I1: XAUTH client - awaiting CFG_set
Mar 20 10:38:07 claimtools pluto[15697]: "home" #4: initiating Quick Mode PSK+ENCRYPT+TUNNEL+PFS+UP+IKEv2ALLOW {using isakmp#3 msgid:2bec3343 proposal=3DES(3)_192-SHA1(2)_160, 3DES(3)_192-MD5(1)_128 pfsgroup=OAKLEY_GROUP_MODP1536}
Mar 20 10:38:07 claimtools pluto[15697]: "home" #4: transition from state STATE_QUICK_I1 to state STATE_QUICK_I2
Mar 20 10:38:07 claimtools pluto[15697]: "home" #4: STATE_QUICK_I2: sent QI2, IPsec SA established tunnel mode {ESP/NAT=>0x8b5c8071 <0xf6c24685 xfrm=3DES_0-HMAC_SHA1 NATOA=none NATD=XXX.XXX.XXX.XXX:4500 DPD=none}


Any ideas?