macdog
-
Posts
30 -
Joined
-
Last visited
Content Type
Profiles
Forums
Articles
Posts posted by macdog
-
-
-
So I am assuming this only works on dedicated?
I assume you are asking from the perspective of this working with a managed server provider such as dayz.st/hfb/etc...
Technically you don't need a dedicated server to at least try this out, but that would mean running your HC on a different computer to the server which may cause you other issues. Probably good enough for testing code, but not sure how it would pan out as a live server.
-
True that, but it introduces a lot of other problems like dsync. Played on you server 4 months ago or so. You should know what I am talking about. We left cause of the dsync. The settings where fine, looting was fast, you know, nothing of the normal problems you have to deal with, but air battles and so on where pointless. Dsyncing all over the place. Noticed that on a lot of HC servers. I guess hoster like the extra calculation power they have but underestimate the effect on bandwith if they put to much code in that needs to replicate.
Anyway, digging in to it is fun. So go for it!
Yeah I do remember.
Anyway, I revamped the whole system about a month ago (in-line with my post above) and haven't had any desync reported. You are welcome to come and take a look. Warning though, I do have a number of annoyances/bugs on the server now as I just don't have time do even simple fixes right now. But you should be able to ascertain the performance aspects of the HC. Its running Wicked AI 2.1.x and animated chopper crashes on the HC so most of the time there is around 50 AI units running around.
Thinking back, I think I know the time you were talking about. Back when the server was regularly capped we were the subject of a number of DDOS attacks the symptom of which is massive desync (you basically get alomst zero bandwidth to the server). I think you have (understandably) put 2 and 2 together but got 22 :D
-
So how do you define in the HC setup which server functions are supposed to be offloaded to the HC? For instance, if I wanted to run it just to offload AI functions off the main CPU core.
This is a fairly involved subject and there are various approaches. I have tried a few and have come to my own conclusion in terms of what works best.
- Run everything from the HC. You can get most function to work, but a pure HC only implementation will not work properly with a lot of functions from the various mission systems. Especially when it comes to spawning objects into the game, and for those objects to persist. In some cases you can get around this using anti-hack tweaks but that will weaken your protection. This method also requires a lot of battleye exceptions, also not good.
- Run everything from server (ie, normal mission pack install) and use setOwner to assign the AI units to the HC. Played with this option briefly, but has some known game engine limitation, such as waypoints being lost.
- Hybrid. Run most code from HC, and offload certain tasks (such as spawning objects) to the server. I used this method with reasonable success for some time but ultimately is more complicated and error prone than the next option.
- Hybrid. Run the mission system from the server, and execute AI modules from the HC. This for me has been the most stable method by some margin and allows all typical features to be used. It requires very few (if any) anti-hack bypasses, all the mission logic runs on the server (which is usually as per the design of the mission pack) and only specific scripts are executed on the HC.
With the hyrbid options you do need to build a basic framework to enable the server and HC to work together well. This is what I have done at a high level:
On the server
- I took Goobers server side HC detection routine and adapted and simplified for my needs. His code was better in that it could cope with multiple HCs however it did have some bugs. Since multiple HC is a very niche requirement I decided to re-write it slightly but with the ability to detect only a single HC. This is not a problem unless you plan to have over 200 AI (maybe more) and also would suggest you are running HCs from various machines (as you can only really run 1 per windows instance easily).
- I installed and configured a mission pack of choice in the normal way in my server PBO. Currently for me that is Wicked AI 2.1.x.
- I then adjusted various elements of the mission logic so that it would check from the presence of an HC (see first point) and only start a mission if a valid HC existed. Hint, there is more to this than just having an HC connected to the server, you also need to know that the HC has had a chance to precompile the necessary scripts.
- I then modify all of the scripts that define the functions for spawning AI so that rather than actually doing anything, they parse all the parameters received from the original spawn request in the specific mission file and send it to the HC using <ID> publicVariableClient [spawn parameters]
- I also do something similar for certain functions such as those that clean up AI when missions end (this bit is more complex, but not that bad actually).
- You also should setup a routine on the server (mine runs every minute) that counts AI units controlled by the server and immediately deletes them. This is because if the HC crashes whilst managing AI, the AI will remain and their control passes back to the server, instantly killing you server FPS!
On the HC
- First, everything that runs on the HC does so via a specific client side addon (PBO) that I have signed with my own key. They key is enabled on my server, so in theory anyone could connect to my server and run my addon, but I dont release my signed addon to the public so its a non-issue (unless i get hacked of course. and by hacked this isnt an arma2 hacked, it would be a "proper" hack (well correct term is crack) of my dedicated server). This way, the only reference to my HC anywhere in the mission file that all players download is a single statement in the init.sqf that checks for a !hasInterface and runs \z\addons\macdogHC\init.sqf.
- First thing is to get the HC side of Goobers HC detection/management code installed in the custom HC addon described above working. This is to let the server know an HC is attached, and I have adapted it for my purposes somewhat.
- Then I install and configure the mission pack in my custom addon, making sure to set the configuration options that affect AI as I want them in the config file. Then disable the actual mission system, so when launched (ie, via \z\addons\macdogHC\WAI\init.sqf) doesnt actually start launching missions.
- I then setup a bunch of publicEventHandlers so that when the server executes a <ID> publicVariableClient spawn_group for example, the HC will run \z\addons\macdogHC\WAI\compile\spawn_group.sqf with the parameters originally specified in the specific mission definition running on the server.
- I have to do something similar for the mission clean up scripts as I alluded to in the server section above, but this is more selective.
And that's it really. It works very reliably IMO, yes very occasionally Arma2OA.exe can crash, and I don't have a monitoring system in place to detect and relaunch the client. So if I don't spot it (usually the case now as I have alomst zero time for epoch these days) there are no missions until next restart. I used to have the HC running as a service which did address this, but the move to steam only has really made running the client in a server like way much more difficult so I have to run the client interactively which is a shame.
I also have animated heli/c130 crashes running of the HC. They dont suffer from the stutter that you get when you run them server side, and are indistinguishable from player flown aircraft. I used to have jet crashes run this way, but it meant white listing them in my anti-hack. I could have edited infistars code to allow the jet crashes but still catch illegal jet spawns, but would require me tinkering every time he released a new version so was just too much effort.
-
sorry, I have never looked at running zeds from HC, just AI. I'm not convinced running zeds on HC provide an appreciable benefit, especially given the work involved and the fact they are already offloaded to clients, but running AI certainly does.
-
You still can come up with your own solution. It is possible and a lot of people made it work. It's not even that hard, but from what I experienced, its not the golden bullet. It will probably not fix your performance issues or it will trade them in for more dsync. Could also be that I did something wrong.
headless client makes a huge difference when running AI. Its 2-fold.
1. Strain on the server is much less, so you can run AI with lots of players and still have reasonable server FPS. (the AI will have some effect on the server, as they need to be tracked etc.. but its is drastically reduced)
2. The AI are as responsive with 50 players on the server as with 1. None of this "frozen AI" behaviour you get when running server side with more than a handful of players.
In terms of using it for zombie and loot spawns I am more sceptical. In fact, I think the zombie offload could have a net worse effect, depending on how you do it. Loots spawns I'm not sure of, but I was never convinced by the effort:reward proposition to even try.
With AI though, HC is a game changer.
Also, it is actually quite tricky, or everyone would be doing it. Yeah standing something up to get some units controlled by the HC is simple, but getting a complete mission system to work with all features, reliably, covering all/most scenarios requires a fair amount of work and understanding of locality.
-
thats a fairly broad/open question mate! the post you quoted simply states you need to let the HC execute the playermonitor code. Its called in the mission init.sql file. Depending on whose instructions you have followed you may have prevented that code from running on the HC by changing
if (!isServer) then
to
if ((!isServer) && (hasInterface) then
on the statement that wraps around the playermonitor section. The simplest fix is to not prevent anything running on the HC, although if you want to optimise you can be more selective.
Does that help?
-
It seems the HC disconnects and reconnects every few minutes with no explanation in the rpt logs.
After a quick google search I discovered that others were having this issue as well. Does anyone here have any insights on this behavior?
I believe the instructions here tell you to prevent the player monitor from running on the HC. The problem is that this includes some essential setup tasks inclusing generating/assigning a characterID to the character. If you look in the player monitor FSM the serverside section will boot players with a null characterID after a timeout period.
The simplest fix for this is to allow the player monitor to run for the HC.
-
you dont stand a hope in hell in running anything with more than a handful of players on any of those regency packages below VDS4. Once you hit 4 I would expect you are in the price territory of dedicated hardware (in europe you can get decent hardware cheaper than that, not so sure about the US).
-
Unless you are running multiple servers or a headless client anything more than 3 cores is a waste. The problem with a VPS is similar to the problem you have with a managed provider such as dayz.st, in that they may be (heavily) oversubscribing the resources they are giving you (ie, you may be sharing those 3 cores with a bunch of other people). Very few providers from what I have seen are transparent in that regard and the ones that are might seem more attractive than they really are to the unititiated, for example, giving you a dedicated "thread" which is definitely not the same as a core.
Interestingly I have yet to find a VPS provider, that, spec for spec is in any way cheaper than what you can find for dedicated hardware. This quite simply means that once you hit 4 cores then dedicated hardware is a complete no brainer.
-
interesting. I will give this a go myself and measure the differences. I am using the CIV slot and load is definitively being offloaded from the server to the HC. I have both regularly reporting how many units they are controlling to rpt using this:
[] spawn { while {true} do { _units = {local _x}count allUnits; diag_log format["HC: Units under ***Server or HC*** control: %1", (str _units)]; diag_log format["DEBUG FPS : %1", diag_fps]; sleep 60; }; };I also know because in the past when I had HC stability issues, when the HC crashed all units would move to server control and the server FPS would nose-dive. :huh:
One of my next jobs is to delete AI units that get transferred to the server in the even of a problem with the HC.
Its hard to compare FPS numbers between servers directly as Arma/Epoch performance is not linear and depends on multiple factors, not least the number of objects in memory.
-
ave also followed the guides to make the HC invisible, can always see him though. Found that removing all means of killing a player or unit doesn't beat a headshot or zed munching on you, which comes back to clientsideI found this was all fine when the player monitor wasn't being run (but got the disconnect issue). You could look at adjusting the player monitor server side routine to stop it disconnecting clients that haven't properly authenticated and that way exclude any type of player monitor being run on the HC without issue. At least in theory that seems to be an option.
I've already started down the custom player monitor for the HC, so will finish tweaking that, but do prefer the idea above longer term.
HC is still the way to go for offloading, working out the locality is a minefield.Well, its the only way to go. Ideally arma server would be able to offload AI tasks to a discrete thread (like it does with IO handling I believe) and we could avoid all this "fun".
Locality is also fun, find myself constantly having to define server-side variables/functions on the HC, but to be expected really.
-
well, just to get the HC connection stable (ie, no disconnect) you can just run the player monitor (ie, dont condition it out via hasInterface checks).
Doing this will mean your HC's character will spawn as a normal player, on the coast, visible and able to take damage, etc.. (regardless of what you do with the mission.sqm). so redirect the HC to run your own player monitor. I havent got a full config yet, as I havent finished tweaking it to what I want, but basically edit the FSM (you need the BI tools for this) and look for the various checks where enable damage is made.
-
fixed it, was the fact I wasnt running the player_monitor on the HC. Running it causes other problem, but should all be fixable.
-
haha yeah, obviously i rolled back the AH and the traders, but if I'm honest I haven't left the HC connected without the AI packs running - (although have isolated each to see if one was the problem).
My own debugging to date has been logging states of things that I have been messing with. I guess I should start logging other parameters in general. Thanks for the help;advice.
-
unlikely - have just deployed on a vanilla epoch and started slowly adding changes. literally only added infistar, dynamic traders and the AI missions packs themselves and started to get this. I had ruled out infistar before but I guess it could be multiple things, that or the AI packs themselves of course. Its a pain, as I need to leave each config going for an hour or so to verify if its affected. Better debugging options would certainly be handy!
-
thanks mate.
helicrash loot: could i sworn I checked that, but obviously missed it somehow. as its in dayz_code, could have expected the HC to know about that as the mod is loaded on the HC.
HC getting killed: hmmm, doesn't appear like the player is actually getting killed although the behavior is similar. You don't get the xxx is killed message, so its seems like its getting "kicked" back to the lobby for some reason.
-
Hi,
Just wondering if anyone can help with one strange, and one less strange issue I've been having with a headless client implementation.
First, the strange one that has be completely stumped.
The headless client works as expected, doing exactly what I want. Then, after an indeterminate amount of time it kinda disconnects from the server. I say kinda, because it doesn't actually get kicked from the server (battleye disabled for debugging this), but I do get the following message in the server rpt:11:02:43 "DISCONNECT: HeadlessClient (158029510) Object: HeadlessClient, _characterID: 0 at loc [0,7.96885e-038,52.8183]"
You then see the headless client re-run its mission init.sqf. This is from the HC's rpt:
"DEBUG: loadscreen guard started." "Res3tting B!S effects..."
It generally seems to be preceded by the following in the client's rpt:
Group C 1-1-A (0x19c90100) - network ID 2:6412 - no main subgroup
but I sometimes see the above message without a disconnect happening.
From what I can tell, the HC is being kicked back to the lobby and the rejoining the game, with any AI that was under the HC's control being passed back to the server.
I believe I have the HC configured correctly, with localclient[]={"127.0.0.1"}; in the config.cfg (yes the HC is on the same server) and its correctly slotting into the civ slot.
At the moment I have the HC's scripts being run from the mission pbo. I have used a custom signed mod, which works, but again gives the issue above.
Any ideas why I am seeing these disconnects, or if not, any extra debugging I can do to isolate the cause?
My second issue is a lot more straight forward, and I am surprised I haven't been able to find the solution. I have moved animated heli/c-130 crashes successfully to the HC and everything works great apart from the loot spawn. The HC doesnt seem to know about the variable helicrash which is used to define the loot spawn. There were a few other variables it didn't know about and I managed to find them in the server pbo and make sure they got defined on the HC, but I can't for the life of me find where helicrash is defined in either the server pbo or the dayz_code pbos. Any idea?
Thanks!
-
for some weird reason we only started to get this in the last couple of days. was literally about to start log/script diving and saw this at the top of the list. saved me some time. thanks cen, and thanks turtle for posting this.
-
Why remove the ability to lock while inside a vehicle?
I think that's one of the great features of A3 vehicles where you can lock/unlock a vehicle while being inside it.
agree with cen. considering implementing this and setting max distance to 0 (or whatever the min is) to only be able to lock when inside purely to counter safezone trolling (people jumping into your car when you drive in).
-
Unfortunately I require some form of access so I can modify the HC on the fly....
And Apache over at ARC has created a nice launcher for the server that waits until the hive log streams data before connecting the headless. This means it will not join until a player does and loads the mission.
Understood mate.
Nice idea on the launcher. Today I just use a delay which works on pure luck really. My plan was to create a script using rcon command line to check for players but I think I prefer Apache's method.
-
Happy to help test mate. I already run some headless AI, but nothing as fancy as what you describe!
I won't grant RDP access, but could roll logs to a FTP/Dropbox/other location.
TS3: 85.236.100.27:17647 poke a purple badge person if I am not on TS at the time or message me here.
If it passes some basic tests happy to roll this with live players (most evenings 50 players), and have 2 dedicated non-hyperthreaded cores for exclusive HC use.
-
Well... Live test went badly. Globally banned within 30 seconds. Emailed BE about it. Test on hold until BE replies :(
unlucky mate, thought auto global bans only kicked in when things like memory injection were detected. I'll have to be more careful myself (testing stuff that is, not hacking)
-
Hello there, ive been checking into log file analyzer , log expert, how do i connect my server to it plz so i can view real time?
ps. I use dayz.st btw, Is this the same as looking at the rpt??? because I can make any sense out of that file either!!!
Don't think you can mate. ST only gives you access to the log via ftp, and you need direct access to the filesystem or sftp access. I have never found a tail solution over ftp, but when you consider how ftp works isn't surprising really. TCAdmin based providers (such as hfb) provide the same functionality in their web console, so its change provider or go dedicated really :(
Working Dayz Epoch Headless Client for Zed Spawn and FSM
in A2: Epoch Mods (1.0.5.1)
Posted
Thanks. Sorry, but right now I don't even have time to manage my server properly let alone this. What I can do is send you the code and answer questions about how it all works etc.. Happy for you to then maintain the HC branch. I actually don't think it would add much overhead to maintain both, and eventually I think you could get to this being a add-on module configured from the global condig file all under the single master branch.