Jump to content

[WIP] DZHC - Extendable DayZ Headless Client Framework


(GSG) Az

Recommended Posts

Hi all,

 

I just wanted to post to share a project that I am almost finished and get some input/suggestions as to the degree of features and flexibility I will add.

I'll start by providing a current overview of what this can and cant do, and leave the floor open for criticism, suggestions and the like.

 

I started this after becoming frustrated with the lack of decent modifiable headless clients available for ARMA and ARMA mods alike. Each HC was designed with a particular purpose in mind and was not easily configurable. I then decided that the HC's available simply were not going to cut it and set out to write my own - But with a few differences.

 

Key Features

  • Standalone framework
  • Multiple headless client support
  • Server controlled headless clients
  • Start/Stop headless clients in realtime.
  • Assign priorities to headless client tasks
  • Server/Client/Headless handover support
  • Client framework for client specific variables
  • Headless client automation system
  • Minimal network usage
  • Easy set up and configuration
  • Object Orientated design for performance (To come, this one is HARD)

AND my personal favorite;

  • Dynamic response to low client/server FPS.

Ill go over some of these in a little more detail;

 

Standalone framework

 

None of the code is reliant on DayZ, Epoch or otherwise. It should easily be compatible with any ARMA server or modification.

 

Server controlled HC

 

The HC is controlled 100% by the server. The server issues commands, monitors the HC's connection status, issues the override commands and terminates any appropriate threads. I made it this way because there will be circumstances where interaction with the HC might be desired by the admin and automation of tasks required it. This will mean that you can override a particular HC's default function from within the game environment. 

 

Start/Stop HC in realtime

 

As a consequence of the above, the ability to control HC's is paramount. The HC already has an inbuilt automation system (see below) but allows for manual override at any time. This means admins actively monitoring their performance can kick in a another HC and offload a task currently assigned to another HC to it. If your HC is maxing its threads, you can manually hand-off some code to another HC to cope with the load. 

 

Assign priorities to headless client tasks

 

Each hand-off is deemed as a HC task. You can assign priorities to these tasks to ensure peak performance at all times. If a HC is struggling with load, it will dynamically seek to remedy the issue by offloading lower priority tasks to either the server, client or another HC.

 

Client framework for client specific variables

 

A set of event handlers designed to handle client specific variables (player etc). This is so even client scripts could technically be handed off to a HC for processing and only the end result gets processed on the client. This should result in massive FPS improvements for clients and server alike. 

 

Headless client automation system

 

Each HC is controlled by the server and the server is designed to dynamically allocate tasks to available HC's. It also makes sure that they are still connected and responds appropriately if a HC disconnects without warning. It knows exactly what tasks the HC was running and tries to reallocate these tasks to an available HC, and if none are available, it hands them back to the client/server and waits for a HC to connect. Once there is a HC available, its will reallocate these tasks back to the HC. 

 

Dynamic response to low client/server FPS.

 

Perhaps my favorite idea, this means that the server and clients actively monitor their FPS in bursts. If these values fall below their thresholds (configurable), then the server/client will kick in and request a HC hand-off. Both clients and server will have to assign which scripts are capable of an override and assign priorities to them, but the idea is simple. When the client/server suffers - offload everything it can to an available HC.

 

Minimal network usage/optimisation

 

All network messages sent and received are strict - they are only issued to the machine that actually needs them. The HC system is also smart enough to know when a message is being sent by a headless client and will queue other messages until it is ok to send them. This results in far better performance and minimises the risk of duplicate messages/threads being initialised. During periods of network congestion (Perhaps an ill fated basic.cfg) it will respond by reallocating the task back to the client/server (to prevent network lag at the locality the script belongs to).

 

 

 

 

 

Things that I want to implement;

 

Secure script system

 

The ability to encode your current scripts that are mission side (Buildings/AI/Whatever) so that they are harder to be stolen without your server.pbo. This will work based of UID's of the server (unsure if possible yet, its just an idea). 

 

Online Headless Compatibility Wizard

 

A wizard that allows you to post your current code and check for compatibilty issues with the headless framework (variables like player, addMagazine etc) and provide suggestions for how to fix the issues. 

 

 

Github link (Coming soon)

 

So guys and gals.... What do we think?

Link to comment
Share on other sites

I've been switched on to the idea of running headless clients for a while and have watched a few of the efforts in this forum with great interest.

 

I hope you post something more concrete, a demo or some video perhaps.

 

I love the idea of offloading z's to other servers in my datacentre and running ridiculous amounts of them.

Link to comment
Share on other sites

I've been switched on to the idea of running headless clients for a while and have watched a few of the efforts in this forum with great interest.

 

I hope you post something more concrete, a demo or some video perhaps.

 

I love the idea of offloading z's to other servers in my datacentre and running ridiculous amounts of them.

 

All I have left to do is a few handoff functions and actually testing it on a live server. Expect git links, video and examples to come within a few days :)

Link to comment
Share on other sites

I'm really interested in trying headless clients on my server. A few questions in preparation...

  • Does each headless client need a copy of ArmA 2 OA to run?
  • Can multiple clients be run on the same PC?
  • Ideally, I want the HC(s) and my player client to run on the same box.

 

Thanks! I'm looking forward to trying this and see how it affects my server performance.

Edited by Snakeyes
Link to comment
Share on other sites

I'm really interested in trying headless clients on my server. A few questions in preparation...

  • Does each headless client need a copy of ArmA 2 OA to run?
  • Can multiple clients be run on the same PC?
  • Ideally, I want the HC(s) and my player client to run on the same box.

 

Thanks! I'm looking forward to trying this and see how it affects my server performance.

 

1: Yes. There are ways around this, the server doesn't actually need a legit key - BUT the headless client does. 

2: Yes. Ideally you will want to run the headless client on the server, and connect locally (on 127.0.0.1) but you can run multiple clients on a single PC by using a reg file to change your CD Key when you start the HC, wait for it to connect and load (about 1 minute) and then run another reg file to change your key back. You would need a key for each client you run.

3: There are many reasons why this is a bad idea. The HC is designed to literally suck every bit of performance out of your CPU. User beware, you will need DECENT hardware and cooling to run a client remotely. 

Link to comment
Share on other sites

Small Update:

 

Code coming along nicely. Did a complete rewrite after a stupid ARMA quirk stopped the HC from connecting and broadcasting. I currently have it logging, communicating and ATTEMPTING to offload tasks. I am going to do a rather clever hack tomorrow (Think the server passing an array of strings to a client, then think formatting these as code on the client and compiling and running them). But rest easy! Soon this project will be ready for release!

Link to comment
Share on other sites

Hi,

 

Will this allow you to run different factions and define their behaviour on different headless clients ?.

 

i.e.

2x HC for Zombies

1x HC for AI bandits

1x HC for Trader City PMC guards.

 

I can put the last in via the editor but would rather have their management off-loaded.

 

Also, when you say it will use all of your CPU, my understanding is that ARMA is not multi-threaded and will only use a single thread / core.  Is that incorrect / have you got around this limitation or will it max out a single thread / core per HC ?.

 

I run on a dual X5570 server so would be very interested in being abl to use all the cores rather than having most sitting idle.

 

RB

Link to comment
Share on other sites

Hi,

 

Will this allow you to run different factions and define their behaviour on different headless clients ?.

 

i.e.

2x HC for Zombies

1x HC for AI bandits

1x HC for Trader City PMC guards.

 

I can put the last in via the editor but would rather have their management off-loaded.

 

Also, when you say it will use all of your CPU, my understanding is that ARMA is not multi-threaded and will only use a single thread / core.  Is that incorrect / have you got around this limitation or will it max out a single thread / core per HC ?.

 

I run on a dual X5570 server so would be very interested in being abl to use all the cores rather than having most sitting idle.

 

RB

 

Controlling different factions will be easy. Offload each separate AI instance to the HC's you desire. The automation will do this for you as long as they all have the highest priority, and the priority is the same for each. 

 

There is a lot of misunderstanding surrounding ARMA multithreading and generally the advice you find is bad and inaccurate. ARMA does in fact utilise 2 threads, not 2 cores. But the caveat here is intel's hyper threading technology. I have seen in perfect consistency tests showing arma utilising 4 threads, 2 are obvious and native, 2 are subtle and load distributed by the HT tech. The limiting factor here is the compiler and its efficiency to communicate effectively with HT. This is why disabling hyper threading will give you a MASSIVE boost in performance. The engine will not have to rely on load balancing to utilise the ghost threads created by HT. That being said, two threads maxing alone will create enough heat to challenge most desktop CPU's and their stock coolers. Hence the warning. 

 

There is a way to get ARMA to use more than 2 threads natively, and I have semi-succeeded in doing so. It involves exploiting branch prediction. It is buggy, messy and almost always results in simulation diversion IE critical thread violation exceptions. If you know enough, you might succeed, but I highly doubt it. 

Link to comment
Share on other sites

Quick update.

Ill be uploading benchmarks and teaser footage by late tomorrow.

So far I havr had massive success with this project. I am not normally one to gloat but thr DZHC works with things I didnt think possible.

A quick technical overview for those interested. The HC now works via its own PBO. Files are still needed in the mission file and server pbo but these are just compiles and variables. All HC handoffs can run via the HC pbo which means TINY mission files and protected scripts :)

I have currently ported zombie spawning and WAI with absolutely no technical difficukty. Took 5 minutes to modify them to work.

And perhaps the most awesome and unexpected news of all:

Buildings running PURELY on the HC only. Clients seem to recieve it in a stream. The advantages so far are a 25% improvement in client FPS and much quicker load times for players.

Im currently running tests with 100's of AI and zombies statically spawned. I will also run tests using non-static spawns and report back FPS details.

I have also forked the epoch git and I plan to integrate the system into the base code following the coding practices currently employed by the dev team (init variables to enable/disable etc). If your interested VB, ill send a link to that fork once I am confident it will work.

Signing off for the night gents and ladies!

Link to comment
Share on other sites

Not likely I'm afraid. Might be able to be done with WINE? Realistically, the code is fine, it will work with anything. The problem isnt the code, its actually running the client and connecting it, changing the CD key etc.

 

So to answer in a nutshell;

 

Yes

and

No

Link to comment
Share on other sites

Would this ever have Linux DayZ Server compatibility?

Sorry Dean, allow me to rephrase that. It was late and I wasn't thinking clearly.

It will absolutely work with Linux as it only requires PBO's to run but it would be a challenge to run the Headless client on the server. The HC could run remotely and connect but I will have to get a key for the headless PBO (as it runs on the HC only). This carries with it it's own set of challenges, as bandwidth then becomes a real issue. I would be interested in setting up a test server on your box to test the viability of this if you are interested?

Worst case scenario I may need to recode some of the basic functions involving network messages, and optimise the reliance on a response in a timely fashion - but most of these features are already there. It's just a matter of testing whether they actually do what I think they do haha!

Link to comment
Share on other sites

Any indication what  volumes of zombies a specific set of resources may allow (i.e. 2x 3GHz cores and 2GB ram =  XXX HC zombies).

 

I have just been playing with the ARMA II editor and the number of zombies I could spawn without the preview turning in to a slideshow was a bit dissapointing (i5-2400 CPU, 8GB ram, Win7 64).

 

Thanks

RB

Link to comment
Share on other sites

Well... Live test went badly. Globally banned within 30 seconds. Emailed BE about it. Test on hold until BE replies :(

 unlucky mate, thought auto global bans only kicked in when things like memory injection were detected.  I'll have to be more careful myself (testing stuff that is, not hacking)

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Advertisement
×
×
  • Create New...