Jump to content

Server Hardware and OS setups - Everything under the app level.


Recommended Posts

Hi,

 

Not really a correct place to post this but I guess 'Resources" is probably more appropriate than the other sub-forums.

 

There is lost of talk about scripting and the software side of things but not very much on the hardware and OS side of running an Epoch server.  Hopefully this thread can help to bridge that gap.

 

Hopefully we can get a decent discussion going on various 'under the app' topics like;

  • Server hardware
  • OS configuration
  • Storage preferences
  • Backup strategies

 

The majority of people hosting Epoch Servers are most likely to be doing so with a server rented though a hosting company but there are people like myself who run dedicated hardware through collocation in datacenters or at home. 

 

As an example, I am currently running the following.

Dell C6100 node

 - Dual X5570 cpus (2.93GHz).

 - 48GB ram (ECC R-Dimm).

 - LSI 9211-8i SAS controller (For SATA III).

 - 2x 120GB SSD (1 for OS, 1 for Game data - need to get some redundancy in there).

 - 1x 1GB ram drive (MySQL DB - dump to SSD every 15 minutes).

 - 2TB backup drive.

 - OS: Windows 2012 R2 Essentials.

 - Backups of the MySQL DB taken every 15 minutes and storred on the game SSD in a directory available to other machines via a SMB share for off server backup.

 

I am currently looking to run a number of headless clients via VMWare Workstation to better utilize the machines resources.

 

My server is not open to the public at this point so I cannot comment of performance under load.

 

A newer E3-1240 v1/2/3 based server would be a good option.  HP Microservers or entry level SMB servers could be a good starting block.  Pair up with some 3rd party ram and drives and you have a nice game server.  If you are looking to VM your game servers then E5s may be a better alternative allowing more ram.

 

What are other people rocking and how do they find it ?.

 

RB

Link to comment
Share on other sites

Hey,

 

There is probably not that much discussion on this as there isn't a massive amount you can do hardware-wise, so really most optimizations are software based (and by that I mean SQF optimizations rather than system software).  Of course, you need to start with decent hardware, but you very quickly hit a wall.

 

IMO the main things to focus on are CPU and network, and even then once you hit a point (quite early on), there is little you can do.

 

CPU: The arma2 engine is known to be largely single threaded.  Yes, it "can" use multiple CPUs, but in reality the amount it can effectively do in parallel is low, so you really end up with a 2 CPU/Core situation.  And the second thread is really just IO offload (handling network connections etc..).  What that means is that you want to get CPU with the highest clock speed possible, not the most cores.  In practical terms that usually means a quad-core processor and in fact the desktop range of CPU (such as intel i7) can be a good bet here (some more on that later).  If you really want max performance, make sure hyper-threading is off also (only makes sense if you want to run multiple servers on the same hardware at a profit, as opposed to running a serve at max performance).

 

Network.  You don't need much bandwidth, even for a high pop server, but consistently is important (both bandwidth and latency).  If you are renting from a hosting provider then you will likely to be subject to what they provide.  The lowest you usually see a dedicated hardware server rental come with is 100mbps guranteed.  This is more than enough, the issue is the quality, and in reality, you are unlikely to know until you try it, although you can do some research to get an indication (check their peering etc.. but I wont go into that here).

 

Really, not much else is likely to make a difference hardware wise.

 

Memory. Arma2 server is 32bit, so theoretical max usage is 4GB.  From memory I don't think it has been compiled to use large address spaces (wrong term, but cba to look it up know) so in fact I believe actually caps out at 2GB.  Even so, a 50 player epoch server with ~5k DB objects still comes in comfortably below that level.  Memory speed however will make a marginal difference.  ECC ram (the type you get in proper servers) will be slower but includes more error checking.  Its a game server, not a banking system, so you dont need ECC, so best to go for performance.  That again usually means looking at the desktop range of technology as opposed to server tech (ie, the i7 mentioned above).  So, for a windows instance running 1 game server, anything above 4GB (even with mysql running on the same box) is wasted.

 

Storage: Maybe I am missing something, but from all testing/measuring I have done, this makes almost zero difference to performance.  The arma2 server process should be running entirely in RAM and unless you have badly underspecced the server (see above) then nothing should be getting swapped out to disk.  The epoch DB is so small (even with ~5k objects), that I'm pretty sure will be entirely cached by mysql.  Even so, I haven't checked but I assume DB writes are asych (ie, lazy) so shouldn't really impact performance anyway.  Most reads happen when the game server starts up.  In this case the only difference I would expect an SSD to make is faster server restarts (of the order of maybe 30 secs vs 1min).  Again, keen to hear if I have this one wrong, as I see lots of people bragging about SSDs, but honestly dont think it gives them anything.

 

Everything else comes down to how reliable do you want it to be.  I have mentioned desktop class tech a couple of times.  There are providers that do this, but you may not get things like redundant power supplies.  All hardware goes pop at some point, so this means you could be left off-line for a few hours to maybe a day or 2.

 

Personally, this is what I use: http://www.online.net/en/dedicated-server/dedibox-lt2k14  Not quite following all my own advice, but it close enough to ideal, reliable (enough) and cheap!  This currently hosts 1 epoch server that can cope with 50 players, but only has half the resources assigned, so should cope with 2 maxed out servers.  Thats even with 2 AI mission systems running, so if I get around to sorting out headless clients, things should get better.

 

What disappointing me the most is from the little I have read into Arma3 servers, most of the same limitations apply.  Not done much reading, so honestly hope I am wrong here and someone can correct me :)

Link to comment
Share on other sites

Well here may be a little more than appears at first glance. 

 

If you swap the C6100 for a DL380 G6 then the setup I am currently using can be had for around $1,400 (Used equipment apart from the drives).

 

Alternatively someone could build a server level machine using an E3-1245v2 and a Intel S1200KP® board with ECC ram in a mITX case for a near silent unit with up to 16GB ram in a very small form factor.  Should be doable for under $800 with a couple of SSDs and 8GB ram.  Some people may like the option of being able to transport an Epoch server around easily for Lan parties or the like.

 

Going desktop grade but keeping with small form factor, how about a H61 board (mITX) and i5-2400 or eqiv cpu and 8GB ram.  Should be around $600

 

There also seems to be no benchmarks available and it would be nice to be able to answer the following questions.

  • What sort of IOPS does a fully populated server see (OS/App/DB areas if segregated) ?.
  • What sort of growth rate of the DB without housekeeping can be expected ?.
  • What is the optimum drive configuration (speed / redundancy / segregation of OS and game) ?.
  • What is the impact of running the server as a VM, does RDM perform better than VMFS formatted disks (ESXi) ?.
  • How does it scale (i.e. 2 servers processes with a single HDD possible but 6 with a SSD) ?.
  • What is the affect of using SAN storage rather than local and what would be the minimum recommendations for speed / config ?.
  • How best to configure multiple servers for failover ?.
  • How to configure more than one server to use a single hive and what are the pros and cons (ok this is a bit more app level) ?.

I will amend the thread title if possible as the intent was to promote discussion on underlying hardware, OS and MySQL configurations rather than just the bare metal components.  This could cover HDD vs SSD vs Ramdrive, raid configs, MySQL backup or replication strategies etc.  Basically everything below the ARMA II application level.

 

RB

Link to comment
Share on other sites

  • What sort of IOPS does a fully populated server see (OS/App/DB areas if segregated) ?.

 

This is why I think storage is a red herring!

8woycpO.png

 

This is the real-time IO chart (20 sec sample rate) from our Epoch server running at full cap 45/45 with over 5k objects.  Be sure to check the scale on each side.  Thats the storage IO for the whole server which also hosts the mySQL database. EDIT: oh and the "spikes" every 15 mins is just the DB backup running.

 

Of course, we could be experiencing spikes within that 20sec sample window, but on the basis of that chart I have spent literally zero time trying to optimise storage.  By the way, the storage in use is 2 x 1TB SATA @ 7.2k rpm in RAID1 (hw controller, no cache - i think).

 

DB growth rate.  In data terms its tiny, our DB is at around 3MB after a couple of months.  You will likely never need to worry about the DB size in and of itself, rather from the perspective of the number of objects that get loaded in by the arma2 server when it starts.  There are other threads on this forum detailing various maintenance techniques.  The big one is turning on Epoch's modular building maintenance feature, but there is also deleting stale character data and cleaning up old/broken/abandoned vehicles.

 

Multi-hive. Never tried myself.  Epoch's base building element means its probably not well suited to shared hives.  Sure I read somewhere that someone was doing it, but most people with multiple servers run them independently.

 

In terms of software config, that isn't SQF, but still Arma specific, you could take a look at the basic.cfg file.  Mainly network related settings to play around with in there.  Again, other threads on that in this forum.

 

EDIT: One more thing I just remembered seeing the post about 4.2Ghz CPU above.  To make sure you are getting the absolute highest clock rate, only use 2 cores on any given processor.  The CPUs will dynamically overclock if only some cores are being used.  If you stick another game server on the remaining cores, then you will be stuck at the stock rate most/all of the time.

Link to comment
Share on other sites

Thanks good info.  Will look at the chart when home.

 

On the DB growth, that is what I have been seeing but good to get confirmation based on a popular server.  Yep I am aware of the houskeeping threads here and have been offering sql based solutions for some of the areas. 

 

With the hive, my current understanding is that the ARMA II server engine reads fromt he DB on startup and then writes every XX minutes but does not read again until it is restarted.  I would love to be proved wrong here though as a more interactive communication between the DB and the Server app would be great.  I have seen something about using a MySQL mod for DB connectivity but have not dug in to it too much.

 

There seem to be a lot of calls of lag on servers and if the serve requirements are not so high (give or take CPU spec), then what is causing the lag for users.  Sure some may be a bottleneck on the network between the server and client or even due to activity on the client side but I see the broken connection (red links) symbol now and then and he desktop and server are on the same internal GbE network.

 

RB

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Advertisement
  • Discord

×
×
  • Create New...