Database issues?

[BetterDeadThanZed]

Running Epoch 1.0.4.1 with Napf. I spent time building a base before server restart. I had walls, a workbench, shed and an LAV. When the server restarted, I logged in and it was all gone. My character was the same as before the restart - all same gear. I logged out and checked the database and sure enough it's all still there in the database. Is there an issue where the database doesn't load everything correctly?

[vbawol]

Please post the entire server RPT to pastebin when this happens again.

[Aristoi]

It happened again, though there were some new messages I had not see:

2014/02/14,  6:20:59 Ref to nonnetwork object 2e23d0c0# 1056792: kamaz.p3d
2014/02/14,  6:20:59 Ref to nonnetwork object 2e210800# 1056795: safe.p3d
<...etc...>

Some highlights:

2014/02/14,  6:19:12 "DayZ Epoch: PRELOAD Functions\init [[<No group>:0 (FunctionsManager)],any]"
2014/02/14,  6:19:12 "DayZ Epoch: MPframework inited"

2014/02/14,  6:20:20 "HIVE: Starting"
2014/02/14,  6:20:20 "server_monitor.sqf execution count = 1"
2014/02/14,  6:20:20 "HIVE: trying to get objects"
2014/02/14,  6:20:21 "HIVE: found 1571 objects"
2014/02/14,  6:20:21 "HIVE: Commence Object Streaming..."
2014/02/14,  6:20:21 "HIVE: got 521 Epoch Objects and 1050 Vehicles"
2014/02/14,  6:22:03 "HIVE: Vehicle Spawn limit reached!"

2014/02/14,  6:35:09 "HIVE: Starting"
2014/02/14,  6:35:09 "server_monitor.sqf execution count = 1"
2014/02/14,  6:35:09 "HIVE: trying to get objects"
2014/02/14,  6:35:09 "HIVE: connection problem... HiveExt response:["ERROR","Instance already initialized"]"
2014/02/14,  6:35:09 "HIVE: trying to get objects"
2014/02/14,  6:35:09 "HIVE: connection problem... HiveExt response:["ERROR","Instance already initialized"]"
2014/02/14,  6:35:09 "HIVE: trying to get objects"
2014/02/14,  6:35:09 "HIVE: connection problem... HiveExt response:["ERROR","Instance already initialized"]"
2014/02/14,  6:35:09 "HIVE: trying to get objects"
2014/02/14,  6:35:09 "HIVE: connection problem... HiveExt response:["ERROR","Instance already initialized"]"
2014/02/14,  6:35:09 "HIVE: trying to get objects"
2014/02/14,  6:35:09 "HIVE: connection problem... HiveExt response:["ERROR","Instance already initialized"]"
2014/02/14,  6:35:10 "HIVE: Spawning # of Vehicles: 300"

Full file (Too big for pastebin):

https://dl.dropboxusercontent.com/u/405582/ARMA2OASERVER.RPT

 

[flashback]

i got the exact same issue, after a restart all is fine.

[Kroenen]

Happened to my server earlier today. Restart seemed to cure it. Will post RPT to pastebin when I get some time.

 

EDIT - Can't post to pastebin as the RPT is way over 500k when this happens.

[icomrade]

Make sure you guys wait at least 1 minute between server restart and kill A2OA if it's already running via your restart.bat.

:KILL_A2OA_SERVER
rem kill the task
taskkill /im arma2oaserver.exe 2> nul
rem check the exit status
if errorlevel 128 goto DONE_A2OA_SERVER
goto DONE_A2OA_SERVER
:DONE_A2OA_SERVER
cd\
cd C:\Program Files\Bohemia Interactive\ArmA 2
start start_DayzServer.bat
timeout 2

[BetterDeadThanZed]

 

Make sure you guys wait at least 1 minute between server restart and kill A2OA if it's already running via your restart.bat.

:KILL_A2OA_SERVER
rem kill the task
taskkill /im arma2oaserver.exe 2> nul
rem check the exit status
if errorlevel 128 goto DONE_A2OA_SERVER
goto DONE_A2OA_SERVER
:DONE_A2OA_SERVER
cd\
cd C:\Program Files\Bohemia Interactive\ArmA 2
start start_DayzServer.bat
timeout 2

 

For those of us with a server hosted by a GSP, that's not possible for us to ensure that. Besides, my server is restarted by TCAdmin because I host with Vilayer, so it doesn't use a batch file to shut down the server.

[Scaramanga]

Has happened to me both in 1.0.4.0 and 1.0.4.2. It first happened after i upgraded to MySQL to 5.6.16 from 5.6.14, I ran 1.0.4 without problems for over a week before that so that was my first suspect.

 

I'm not sure if it is related, though I suspect it is at least possible, but one of the side effects seems to be tons of vehicles past the limit set.

 

This is correct, when the issue occurs there's a new set of vehicles spawned.

[Kroenen]

Where do i add this to the normal server files ? I've already done my custom changes, don't really fancy redoing them.

Im with Richie on this one. Can we just have the extra code added and to which file so we can give you the debug info you need without having to recompare all the server files again?

[Aristoi]

Im with Richie on this one. Can we just have the extra code added and to which file so we can give you the debug info you need without having to recompare all the server files again?

 

Sure, I did it myself.  It goes in server_monitor.sqf.

[icomrade]

Im with Richie on this one. Can we just have the extra code added and to which file so we can give you the debug info you need without having to recompare all the server files again?

HAHAHA are my posts invisible?

 

I posted this 2 times!

 

 

Try logging how many times server_monitor is run using the below code. Post your .rpt if you get more than one line saying "server_monitor.sqf execution count = ". the count should only ever = 1 and never more.

 

Put this code at the line below waituntil{isNil "sm_done"}

 

if (isNil "server_initCount") then {

server_initCount = 1;

} else {

server_initCount = server_initCount + 1;

};

diag_log format["server_monitor.sqf execution count = %1", server_initCount];

 

[BetterDeadThanZed]

I believe they added the code to 1.0.4.2 as well to help track this issue.

[Kroenen]

 

HAHAHA are my posts invisible?

 

I posted this 2 times!

 

 

 

HAHAHAHAHA, er no. But both times you never mentioned which file to paste the code into. For us poor 'lesser' server admins out there, we're not mind readers!

[icomrade]

HAHAHAHAHA, er no. But both times you never mentioned which file to paste the code into. For us poor 'lesser' server admins out there, we're not mind readers!

it's in the debug code, sorry I didn't explicitly state the file (although it is right there in the code).

[BetterDeadThanZed]

For some reason the issue was closed in Github but it's still not resolved.

 

I just had it happen on my server after updating to 1.0.4.2, however I don't have a RPT log to show the "server_monitor.sqf execution count" because I thought it was added to the 1.0.4.2 code but apparently it wasn't. I just added it back into the 1.0.4.2 code and when it happens again, I'll post my RPT log.

 

So, why was this closed in Github? Did you guys find a solution?

[flashback]

Yes, the server needs after a restart 15-20 seconds to kill all connections.

 

so if you restart your server 30 seconds after you killed it, you dont have the object_data problem.

[BetterDeadThanZed]

Yes, the server needs after a restart 15-20 seconds to kill all connections.

 

so if you restart your server 30 seconds after you killed it, you dont have the object_data problem.

 

Since my server is hosted with Vilayer, I don't have that option. That's just a work around and not a fix to the actual problem.

[icomrade]

For some reason the issue was closed in Github but it's still not resolved.

 

I just had it happen on my server after updating to 1.0.4.2, however I don't have a RPT log to show the "server_monitor.sqf execution count" because I thought it was added to the 1.0.4.2 code but apparently it wasn't. I just added it back into the 1.0.4.2 code and when it happens again, I'll post my RPT log.

 

So, why was this closed in Github? Did you guys find a solution?

It seems the issue is with the old server process not closing in time for the restart. So, the hive is already initialized by the old process and the new one throws the "hive already initialized" error.

 

If you can't lengthen the restart time or use the .BAT code I posted, you can try locking the server for 1 minute on server startup (you can probably use the scheduler for this if BEC doesn't have a config option). However, if you let your host know that killing the old server process before restart prevent this issue then they could fix it for all users on their system - or by having them add an option to set time between restarts.

[Richie]

It seems the issue is with the old server process not closing in time for the restart. So, the hive is already initialized by the old process and the new one throws the "hive already initialized" error.

 

If you can't lengthen the restart time or use the .BAT code I posted, you can try locking the server for 1 minute on server startup (you can probably use the scheduler for this if BEC doesn't have a config option). However, if you let your host know that killing the old server process before restart prevent this issue then they could fix it for all users on their system - or by having them add an option to set time between restarts.

So how come this is a new issue ? I've hosted Epoch servers for many previous versions and only started with 1.0.4 and upwards.

Something in the code changed, GSP and dedicated servers haven't changed.

[BetterDeadThanZed]

It seems the issue is with the old server process not closing in time for the restart. So, the hive is already initialized by the old process and the new one throws the "hive already initialized" error.

 

If you can't lengthen the restart time or use the .BAT code I posted, you can try locking the server for 1 minute on server startup (you can probably use the scheduler for this if BEC doesn't have a config option). However, if you let your host know that killing the old server process before restart prevent this issue then they could fix it for all users on their system - or by having them add an option to set time between restarts.

 

Again, this is just a workaround. The proper thing to do would be for the devs to identify why this happens and why it didn't start happening until 1.0.4.1, and fix it. Anyone that has been hosted with Vilayer can tell you that asking them to do anything takes a long time. They are horrible with their support response times and it's very likely they would say they can't do anything about it. There are thousands of Epoch server and I'd guess that the majority of them are hosted with GSP's. It's unlikely GSP's would address this issue in their control panels. It's better just to fix the cause.

[Aristoi]

It seems the issue is with the old server process not closing in time for the restart. So, the hive is already initialized by the old process and the new one throws the "hive already initialized" error.

 

I'm not convinced this is the issue.  The log shows the hive being initialized both times (or attempting).  How could that be the previous server instance's hive connections?

[icomrade]

I'm not convinced this is the issue.  The log shows the hive being initialized both times (or attempting).  How could that be the previous server instance's hive connections?

True, but having only seen a couple of logs there may be more than one issue causing this symptom (object_data not being read), ensuring the server is closed by starting another may be an entirely separate cause.

 

Can any of you get a hold of the startup and restart scripts?

[BetterDeadThanZed]

True, but having only seen a couple of logs there may be more than one issue causing this symptom (object_data not being read), ensuring the server is closed by starting another may be an entirely separate cause.

 

Can any of you get a hold of the startup and restart scripts?

 

I sent in a support ticket to Vilayer but it's unlikely there's anything they will do. This problem started with 1.0.4.1 (Maybe 1.0.4 - I didn't install that version). I didn't have that extra line that logs the server_monitor being loaded when it happened to me, which has been twice, but if it happens again and I know about it, I'll submit my log. Hopefully it sheds more light.

[Flosstradamus]

I am still having this error unfortunately and the thing that fixes it is to bring the server down and perform another restart but it does it every now and then :\

[BetterDeadThanZed]

I am still having this error unfortunately and the thing that fixes it is to bring the server down and perform another restart but it does it every now and then :\

 

I've still only had this happen 3 times and none of those times did I have the extra code that reports a loading of the server_monitor.sqf.

[Kroenen]

Don't wish to go all off topic on this, but from what I've read, when it happens, you end up with way too many vehicles on your map....well over cap. Any SQL boffins out there know a good query to run to perhaps remove unlocked and empty vehicles to help get back under the vehicle cap?