Poor client FPS.

[Ghostrider-GRG]

Since updating our production servers to 0.3.9.0 client FPS are in the 3-5 FPS range with both vanilla and modded mission files. I was thinking maybe I missed an enableSimulation setting somewhere. Does anyone have a suggestion as to a fix?

[vbawol]

Hey @Ghostrider-DbD- Are you seeing any errors server or client side?

[Ghostrider-GRG]

@vbawol, I will check. It was strange because when I ran everything on a test server it all seemed fine but on our production server FPS was poor. I saw some database errors - more to come soon

[Ghostrider-GRG]

I am worried I missed something silly along the way but: I see a bunch of what look like database errors related to vehicles, and one server script error:

23:07:21 Error in expression <et) exitWith {};

_sound = EPOCH_sounds param [_soundIndex,-1];
if !(_sound isEq>
23:07:21   Error position: <param [_soundIndex,-1];
if !(_sound isEq>
23:07:21   Error Type String, expected Number
23:07:21 File epoch_server\compile\epoch_antagonists\EPOCH_server_handle_say3D.sqf, line 21

The server log is linked below:

https://www.dropbox.com/s/071z5falljk20gn/arma3server_2016-07-07_22-45-56.rpt?dl=0

On the client side (admittedly, a modded mission file), I don't see anything remarkable other than missing group information:

https://www.dropbox.com/s/57znuyyutmzazbv/arma3_2016-07-07_23-40-07.rpt?dl=0

I appreciate the quick reply. I know our players are looking forward to rejoining the server.

[Grahame]

Some of my users are reporting really poor FPS performance as well. Personally I haven't had better performance ever before but two of my admins are stuck in the <5FPS range.

Not seeing any errors in the RPT files

[klinGiii]

Poor fps range from 3-5 ... vanilla and modified files ... can confirm that :/ and 90% of my community.

Strange thing is , no real Errors client or serverside. 

 

Only thing which is spamming my server side rpt is : 

 

Spoiler

10:37:00 Server: Object 2:16526 not found (message Type_92)
10:37:00 Server: Object 2:16525 not found (message Type_92)
10:37:01 Client: Remote object 21:1 not found
10:37:02 Warning: Cleanup player - person 2:16610 not found
10:37:04 Warning: Cleanup player - person 2:16610 not found
10:37:12 Server: Object 2:14894 not found (message Type_120)
10:37:14 Server: Object 2:16430 not found (message Type_92)
10:37:14 Server: Object 2:16428 not found (message Type_92)
10:37:15 Server: Object 2:14814 not found (message Type_92)
10:37:15 Server: Object 2:14813 not found (message Type_92)
10:37:16 Server: Object 2:14894 not found (message Type_92)
10:37:17 Server: Object 2:15624 not found (message Type_120)
10:37:20 Server: Object 2:14834 not found (message Type_92)
10:37:20 Server: Object 2:14832 not found (message Type_92)
10:37:21 Server: Object 2:15626 not found (message Type_120)
10:37:22 "DEBUG: _event PlantSpawner"
10:37:22 "DEBUG: suppressed PLANT PATCH spawn jammer too close"
10:37:22 Server: Object 2:15626 not found (message Type_92)
10:37:22 Server: Object 2:15624 not found (message Type_92)
10:37:26 Server: Object 2:14832 not found (message Type_120)
10:37:27 Server: Object 2:16430 not found (message Type_120)
10:37:28 Server: Object 2:16428 not found (message Type_120)
10:37:28 Server: Object 2:14834 not found (message Type_120)
10:37:32 Server: Object 2:14861 not found (message Type_92)
10:37:32 Server: Object 2:14862 not found (message Type_92)
10:37:34 Server: Object 2:14861 not found (message Type_120)
10:37:35 Server: Object 2:14862 not found (message Type_120)
10:37:40 Server: Object 2:16526 not found (message Type_92)
10:37:40 Server: Object 2:16525 not found (message Type_92)
10:37:53 Server: Object 2:16526 not found (message Type_120)
10:37:53 Server: Object 2:16525 not found (message Type_120)
10:37:55 Server: Object 2:16428 not found (message Type_92)
10:37:55 Server: Object 2:16430 not found (message Type_92)
10:37:55 Server: Object 2:14814 not found (message Type_92)
10:37:55 Server: Object 2:14813 not found (message Type_92)
10:37:56 Server: Object 2:14894 not found (message Type_92)
10:37:58 Client: Remote object 20:0 not found
10:37:59 Server: Object 2:14814 not found (message Type_120)
10:38:00 Server: Object 2:14834 not found (message Type_92)
10:38:00 Server: Object 2:14832 not found (message Type_92)
10:38:00 Server: Object 2:14813 not found (message Type_120)

 

 

[HarryNutz]

Looks to be 100% of the players on the one server I admin on. Most days have been pretty consistent all day and in the early morning hours with at least 10+ and all I can hear right now are crickets.

[vbawol]

8 hours ago, Ghostrider-DbD- said:

I am worried I missed something silly along the way but: I see a bunch of what look like database errors related to vehicles, and one server script error:

23:07:21 Error in expression <et) exitWith {};

_sound = EPOCH_sounds param [_soundIndex,-1];
if !(_sound isEq>
23:07:21   Error position: <param [_soundIndex,-1];
if !(_sound isEq>
23:07:21   Error Type String, expected Number
23:07:21 File epoch_server\compile\epoch_antagonists\EPOCH_server_handle_say3D.sqf, line 21

The server log is linked below:

https://www.dropbox.com/s/071z5falljk20gn/arma3server_2016-07-07_22-45-56.rpt?dl=0

On the client side (admittedly, a modded mission file), I don't see anything remarkable other than missing group information:

https://www.dropbox.com/s/57znuyyutmzazbv/arma3_2016-07-07_23-40-07.rpt?dl=0

I appreciate the quick reply. I know our players are looking forward to rejoining the server.

This error is from old code, please make sure you have updated everything.  Also what other mods/Anti-Hack are you running?

[klinGiii]

Just noticed since our last restart, when someone connects , that this is standin in serverside rpt :

 

Spoiler

14:25:32 NetServer::SendMsg: cannot find channel #988658866, users.card=3
14:25:32 NetServer: users.get failed when sending to 988658866
14:25:32 Message not sent - error 0, message ID = ffffffff, to 988658866 (Kaban)
14:25:32 NetServer::SendMsg: cannot find channel #988658866, users.card=3
14:25:32 NetServer: users.get failed when sending to 988658866
14:25:32 Message not sent - error 0, message ID = ffffffff, to 988658866 (Kaban)
14:25:32 NetServer::SendMsg: cannot find channel #988658866, users.card=3
14:25:32 NetServer: users.get failed when sending to 988658866
14:25:32 Message not sent - error 0, message ID = ffffffff, to 988658866 (Kaban)
14:25:32 NetServer::SendMsg: cannot find channel #988658866, users.card=3
14:25:32 NetServer: users.get failed when sending to 988658866
14:25:32 Message not sent - error 0, message ID = ffffffff, to 988658866 (Kaban)
14:25:32 NetServer::SendMsg: cannot find channel #988658866, users.card=3
14:25:32 NetServer: users.get failed when sending to 988658866
14:25:32 Message not sent - error 0, message ID = ffffffff, to 988658866 (Kaban)
14:25:32 NetServer::SendMsg: cannot find channel #988658866, users.card=3
14:25:32 NetServer: users.get failed when sending to 988658866
14:25:32 Message not sent - error 0, message ID = ffffffff, to 988658866 (Kaban)
14:25:32 NetServer::SendMsg: cannot find channel #988658866, users.card=3
14:25:32 NetServer: users.get failed when sending to 988658866
14:25:32 Message not sent - error 0, message ID = ffffffff, to 988658866 (Kaban)
14:25:32 NetServer::SendMsg: cannot find channel #988658866, users.card=3

 

 

 

[klinGiii]

We had to Wipe the Database. With new Database everything seem to work fine. No more FPS Problems.

[Grahame]

@klinGiii I'm seeing the same thing in my RPT:

Spoiler

13:52:26 NetServer::SendMsg: cannot find channel #801349341, users.card=0
13:52:26 NetServer: users.get failed when sending to 801349341
13:52:26 Message not sent - error 0, message ID = ffffffff, to 801349341 (TheEscapingGamer)
13:52:26 NetServer::SendMsg: cannot find channel #801349341, users.card=0
13:52:26 NetServer: users.get failed when sending to 801349341
13:52:26 Message not sent - error 0, message ID = ffffffff, to 801349341 (TheEscapingGamer)
13:52:26 NetServer::SendMsg: cannot find channel #801349341, users.card=0
13:52:26 NetServer: users.get failed when sending to 801349341
13:52:26 Message not sent - error 0, message ID = ffffffff, to 801349341 (TheEscapingGamer)

I also noticed a lot of these at server start:

Spoiler

12:07:39 Update of nonlocal object 2:6485 called
12:07:39 Update of nonlocal object 2:6485 called
12:07:39 Update of nonlocal object 2:6485 called
12:07:39 Update of nonlocal object 2:6485 called
12:07:39 Update of nonlocal object 2:6485 called
12:07:39 Update of nonlocal object 2:6485 called
12:07:39 Update of nonlocal object 2:6485 called
12:07:39 Update of nonlocal object 2:6485 called
12:07:39 Update of nonlocal object 2:6485 called
12:07:39 Update of nonlocal object 2:6485 called
12:07:39 Update of nonlocal object 2:6485 called
12:07:39 Update of nonlocal object 2:6485 called
12:07:39 Update of nonlocal object 2:6485 called
12:07:39 Update of nonlocal object 2:6485 called
12:07:39 Update of nonlocal object 2:6485 called
12:07:39 Update of nonlocal object 2:6485 called
12:07:39 Update of nonlocal object 2:6485 called
12:07:39 Update of nonlocal object 2:6485 called
12:07:39 Update of nonlocal object 2:6485 called
12:07:39 Update of nonlocal object 2:6485 called
12:07:39 Update of nonlocal object 2:6485 called
12:07:39 Update of nonlocal object 2:6485 called
12:07:39 Update of nonlocal object 2:6485 called
12:07:39 Update of nonlocal object 2:6485 called
12:07:39 Update of nonlocal object 2:6485 called
12:07:39 Update of nonlocal object 2:6485 called
12:07:39 Update of nonlocal object 2:6485 called
12:07:39 Update of nonlocal object 2:6485 called
12:07:39 Update of nonlocal object 2:6485 called
12:07:39 Update of nonlocal object 2:6485 called

Still failing to see anyone connect properly to the server apart from me which is disconcerting

[klinGiii]

As i mentioned before your Post. We had to Wipe ... After the Wipe FPS Problems seem to be gone. But have to wait a little bit longer when more players are connected.

[vbawol]

@He-Man and I have reproduced this issue on one of his servers at first look it does seem related to large vehicle counts and/or the vehicle simulation handler not working correctly.

For now I would say to try to disable the vehicle simulation handler by setting "simulationHandler" to false:

https://github.com/EpochModTeam/Epoch/blob/release/Server_Install_Pack/%40epochhive/epochconfig.hpp#L31

UPDATE: I have made a commit that forces disabling of the simulation handler by default: https://github.com/EpochModTeam/Epoch/commit/13100982641a9dcbe063cfbbc799f0eb1a9f9a58

Server PBO: https://github.com/EpochModTeam/Epoch/blob/13100982641a9dcbe063cfbbc799f0eb1a9f9a58/Server_Install_Pack/@epochhive/addons/epoch_server.pbo?raw=true

 

 

[Grahame]

Thanks @vbawol 

I have implemented fix to hpp and epoch_server.pbo. Testing now on EpochZ

[klinGiii]

Tested with Vehicle Simulation false;

 

And your fixed server.pbo

 

At low Playercount all seems to be fine. Then our Playercount reached 15-20 Players.

Then .... this happened: Click me

(5FPS+Graphical Glitches)

Every new joined player got the same problem again.

 

 

 

 

[HarryNutz]

Disregard

klinGiii

klinGiii

klinGiii

klinGiii

klinGiii

klinGiii

[Grahame]

After implementing fix I'm afraid that we don't see a resolution, in fact now I have the problems (I was the one person without them). I think I'm going with the DB wipe. I tested that and client FPS was all good. Will have to work out with the players what we can do to make them whole...

[klinGiii]

Wipe isnt the solution. We wiped , and after the next restart (3hours) the FPS trouble began again at medium playercount. Even with the new server.pbo from the Hotfix. 

 

 

[Grahame]

1 minute ago, klinGiii said:

Wipe isnt the solution. We wiped , and after the next restart (3hours) the FPS trouble began again at medium playercount. Even with the new server.pbo from the Hotfix. 

 

 

Oh... so I guess I back out the wipe

[HarryNutz]

This is starting to sound like a revert back to 0.3.8.0 might be in order till things get sorted out.

[Grahame]

Problem with that is half the players in the world don't know how to...  That's why I f****** HATE Steam! 

[HarryNutz]

Just now, Grahame said:

Problem with that is half the players in the world don't know how to...  That's why I f****** HATE Steam! 

True.. but what other alternatives are there? Better to have a little hassle now and at least get the servers back up and running then have them sitting empty.

[He-Man]

Have you set simulationhandler to false? This is the most important!

[klinGiii]

Yes ... 

 

Like i said before , even with the Hotfix vbawol merged 1 Hour ago.

 

Proof :)

[He-Man]

Okay, then take the EPOCH_load_vehicles.sqf from 0.3.8 and give feedback, if this fixed it.