Jump to content

[RELEASE] Monitor a process & reLaunch it if it is not running - log everything


mgm

Recommended Posts

 Hi guys,

 

We had a "shutdown the server and relaunch it" script (which was found on this forums - thanks to all posted/improved it) which was working perfect for over 2 months now.

However on first such occurence I realized that if the server crashes unexpectedly, our solution (BEC) will not be able to restart itself so we needed some external monitoring & intervention to check & relaunch it when necessary.

FireDaemon and its friends are nice and all but (1) I don't want an additional software license to pay for and (2) I do not wish to maintain another software with future updates etc.  so I wrote this basic Windows BATCH script which seems to be doing the job. I will update the post if I find issues and/or improve it in any way.

 

This batch script monitors:

( 1 ) a configurable Windows executable

( 2 ) by checking every N seconds

( 3 ) if it is found to be NOT RUNNING, it patiently waits for a configurable N seconds amount of time (to ensure it's not double launched)

( 4 ) if monitoredApp is still not running  reLaunches it

( 5 ) and writes everything to a log file for admin review

( 6 ) and if debug=1 is configured also prints this information into the Windows cmd (console) window so that we can see what's going on without checking any log files

 

Hope it helps someone else save some time... Please post any questions/comments/improvements here.

 

Thanks

 

  Reveal hidden contents

 

Link to comment
Share on other sites

The log file (and debug monitor if you've enabled it) looks like this:

 

  Reveal hidden contents

FAILURE ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) was NOT running on 30/08/2014 23:50:20.99
FAILURE ON 2ND CHECK: MonitoredApp (arma2oaserver.exe) was STILL NOT running on 30/08/2014 23:50:33.33. Relaunching now...
Relaunch command issued on 30/08/2014 23:50:33.33. Taking a nap now, before performing the next check in DurationToNapBeforeNextCheck (45) seconds
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 30/08/2014 23:51:18.25
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 30/08/2014 23:52:03.27
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 30/08/2014 23:53:37.86
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 30/08/2014 23:54:22.26
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 30/08/2014 23:55:07.26
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 30/08/2014 23:55:52.30
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 30/08/2014 23:56:37.31
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 30/08/2014 23:57:22.30
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 30/08/2014 23:58:07.30
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 30/08/2014 23:58:52.31
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 30/08/2014 23:59:37.22
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:00:22.22
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:01:07.29
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:01:52.24
FAILURE ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) was NOT running on 31/08/2014  0:02:02.07
FAILURE ON 2ND CHECK: MonitoredApp (arma2oaserver.exe) was STILL NOT running on 31/08/2014  0:02:14.66. Relaunching now...
RELAUNCH command issued on 31/08/2014  0:02:14.66. Taking a nap now, before performing the next check in DurationToNapBeforeNextCheck (45) seconds
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:02:37.22
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:02:59.25
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:03:22.22
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:03:44.26
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:04:07.27
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:04:29.30
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:04:52.31
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:05:14.24
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:05:37.29
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:05:59.27
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:06:22.22
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:06:44.31
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:07:07.23
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:07:29.25
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:07:52.23
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:08:14.30
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:08:37.24
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:08:59.31
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:09:22.25
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:09:44.24
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:10:07.30
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:10:29.24
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:10:52.30
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:11:14.24
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:11:37.29
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:11:59.26
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:12:22.30
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:12:44.31
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:13:07.27
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:13:29.21
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:13:52.21
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:14:14.32
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:14:37.31
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:14:59.31
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:15:22.23
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:15:44.31
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:16:07.30
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:16:29.28
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:16:52.21
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:17:14.30
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:17:37.27
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:17:59.29
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:18:22.27
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:18:44.23
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:19:07.26
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:19:29.29
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:19:52.31
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:20:14.32
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:20:37.25
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:20:59.26
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:21:22.28
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:21:44.30
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:22:07.24
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:22:29.23
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:22:52.25
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:23:14.28
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:23:37.23
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:23:59.30
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:24:22.29
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:24:44.21
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:25:07.31
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:25:29.26
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:25:52.24
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:26:14.28
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:26:37.25
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:26:59.23
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:27:22.27
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:27:44.23
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:28:07.29
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:28:29.23
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:28:52.26
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:29:14.23
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:29:37.28
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:29:59.31
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:30:22.31
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:30:44.30
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:31:07.24
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:31:29.23
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:31:52.27
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:32:14.23
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:32:37.28
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:32:59.26
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:33:22.28
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:33:44.25
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:34:07.25
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:34:29.23
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:34:52.24
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:35:14.22
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:35:37.20
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:35:59.27
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:36:22.29
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:36:44.25
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:37:07.20
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:37:29.22
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:37:52.30
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:38:14.21
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:38:37.22
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:38:59.24
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:39:22.21
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:39:44.20
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:40:07.29
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:40:29.26
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:40:52.26
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:41:14.29
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:41:37.26
FAILURE ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) was NOT running on 31/08/2014  0:41:59.21
FAILURE ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) was NOT running on 31/08/2014  0:42:22.27
FAILURE ON 2ND CHECK: MonitoredApp (arma2oaserver.exe) was STILL NOT running on 31/08/2014  0:42:44.22. Relaunching now...
RELAUNCH command issued on 31/08/2014  0:42:44.22. Taking a nap now, before performing the next check in DurationToNapBeforeNextCheck (45) seconds
SUCCESS ON 2ND CHECK: arma2oaserver.exe is running on 31/08/2014  0:43:07.23
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:43:07.34
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:43:29.25
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:43:52.23
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:44:14.26
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:44:37.20
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:44:59.26
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:45:22.26
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:45:44.25
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:46:07.24
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:46:29.30
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:46:52.31
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:47:14.22
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:47:37.26
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:47:59.28
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:48:22.21
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:48:44.29
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:49:07.28
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:49:29.22
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:49:52.30
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:50:14.30
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:50:37.29
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:51:22.21
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:52:07.28
FAILURE ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) was NOT running on 31/08/2014  0:52:52.24
FAILURE ON 2ND CHECK: MonitoredApp (arma2oaserver.exe) was STILL NOT running on 31/08/2014  0:53:37.30. Relaunching now...
RELAUNCH command issued on 31/08/2014  0:53:37.30. Taking a nap now, before performing the next check in DurationToNapBeforeNextCheck (45) seconds
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:54:22.30
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:55:07.29
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:55:52.32
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:56:37.21
SUCCESS ON 1ST CHECK: MonitoredApp (arma2oaserver.exe) is running on 31/08/2014  0:57:22.24

Link to comment
Share on other sites

Due to overwhelming response from you guys [ :P ] I'm sharing below my latest addition to this.

The script in post #1 monitors & relaunches arma2oaserver.exe if it ever crashes unexpectedly. It's doing its job fine.

 

My next problem was it kept creating a new instance of itself every time game server had a scheduled auto restart - now I'm using the code bit below, in an EXE compiled version of the code below.

This exe is executed as the last line of my "relaunch_game_server.bat" batch script which is executed three hours.

 

You can get free AutoHotkey from www.autohotkey.com

Sleep, 10000  ; 10 seconds

Runwait, taskkill /im cmd.exe
Sleep, 2000  ; 2 seconds

Runwait, taskkill /im cmd.exe
Sleep, 2000  ; 2 seconds

Run, "C:\dayz_mod_server\__tool.script.restart_batch_script\LaunchIfNotRunning.bat", , min
return

 

Link to comment
Share on other sites

Please sign in to comment

You will be able to leave a comment after signing in



Sign In Now
  • Advertisement
  • Discord

×
×
  • Create New...