Jump to content
  • 0
RimBlock

ObjectUID calculation and how to improve it...

Question

So, as many will know, there is an issue in A2Epoch (possibily from DayZ) in the unique identifier generation not always producing unique identifiers.

 

What is the unique identifier and how is it used ?.

 

Epoch uses two object identifiers.

  • ObjectID
  • ObjectUID

ObjectID is generated by the MySQL DB backend.  ObjectUID is generated within the ARMA 2 game world via SQF code.

 

Why two identifiers ?.

 

For some reason... there is only one HiveExt.dll call to get the ObjectID for an object in the DB.  This call is the one used to load all the DB objects in to ARMA.  The offshoot of this is that when a new object is created and saved to the DB, the DB creates a unique objectID but this objectID cannot get back to the ARMA 2 game world until the ARMA 2 server process is restarted and all the objects are then read from the DB again.

 

To get around this, the ObjectUID is created when a new object is created in the game world.  This is used to reference the object in the game world and on the DB if the ARMA 2 engine does not know the DB assigned ObjectID.

 

So, what is the problem ?.

 

Whilst the code to generate a unique objectUID looks fairly good, it does not always create a unique ID.  THis means that manipulation of any new objects that have not yet got an objectID in the game world (ie. created after the last server restart) but having an objectUID that matches the objectUID of an object currently in the DB will cause all sorts of problems.

 

How is the objectUID calculated currently ?.

 

There are three functions used to calculate the objectUID currently (all defined in server_functions.sqf).

  • dayz_objectUID - No longer used it seems.
  • dayz_objectUID2 - Used for objects objectUID generation.
  • dayz_objectUID3 - Used for vehicle objectUID generation.

dayz_objectUID just works out the required input parameters and then calls dayz_objectUID2.

 

dayz_objectUID2 generates the objectUID for the object by applying the following calculation.

  •  x,y,z coords each multiplied by 10.
  • Results made positive if required by multiplying by -10 and rounded to a whole number.
  • 3 numbers are concatenated together (X + Y + Z).
  • The objects direction (in degrees) after rounding is then concatenated to the result.

dayz_objectUID3 does the same but adds the mission time (time in seconds since the ARMA 2 mission was started) to the direction value before adding it to the result for the x/y/x/ * 10 part.

 

Worked example for dayz_objectUID2 

Worldspace = [351.545[11079.8,11834.3,-0.380997]]

=

X = 11079.8 * 10 = 110798 

Y = 11834.3 * 10 = 118343

Z = -0.380997 * 10 = -3.80997 * -10 (make positive) = 38.0997 round = 38

DIR = 351.545 rounded = 352

 

= concat  110798 + 118343 + 38 + 352

Calculated objectUID

11079811834338352

 

Stored ObjectUID

 

11079811834338352

 

So why is it not random ?.

 

A set of coodinates ( [dir,[x,y,z]] ) for an object of [0,[1,1,10]] will produce the same result as a set of coordinates of [0,[1,10,1]].  The uniqueness plays on the probability of combination resulting in the same onjectUID being generated as unlikely.  The more items you have in your DB then the more likely it is to happen.

 

Ok, so what can we do about it ?.

 

So this is the purpose of this thread.  Both to highlight the issues to those who were not aware and to have a discussion on peoples thoughts on a better solution.

 

I am curious as to why the code does not start at 0 and each time a objectUID is needed it just increments (possibly by 2 / 5 / 10).  A server global variable could hold the current value and be populated on server restart by the highest objectUID found in the loaded objects.  The only issue I can see is if multiple requests for a new objectUID resulted in the same objectUIDs being given out to multiple requestors as the global 'current objectUID' variable cannot be updated quickly enough.  Presumabily that ould be managed by a basic schedualing system enforcing a FIFO queue so each request could be processed one at a time.

 

Any other thoughts or observations ?.  I am hoping we can all come up with a better solution that does not require any custom .dlls or DB table changes.  Good news is that any new generation system should be pretty easy to plugin to Epoch.

 

Over to you...

Share this post


Link to post
Share on other sites

9 answers to this question

Recommended Posts

  • 0

Hm, I am not sure if the calculation is the same for me.

Examples:

 

Obj-UID: 11079811834338352 and Worldspace: [351.545,[11079.8,11834.3,-0.380997]]

It takes x*10 and adds y*10 at the end. From z it seems to take only the first 2 values (38) after *10 and then adds the rounded value of dir at the end.

 

Obj-UID: 1577121024771131 and Worldspace: [131,[15771.2,10247.7,0.0817103]]

Here it seems that if the z value after multiplying with 10 is still <0 that just 1 is taken (the 1 before the dir of 131...the brown one).

 

I never had a look into this but as it looks to me I cannot believe that there will be 2 ObjUIDs which are the same.

This could also be different for me using the Linux server...I don't know.

 

Edit (after adding some colors): Hm, if I am right with the rounding of the z value to 1 it might be that 2 objects which are above each other and these have the same direction and the same x,y coords that it can be the case. Then it would be much better to just take a random number between 0 and 9 for this value. In my opinion this number can only be the candidate why these IDs can be identical.

 

I need some sleep and so I maybe understood everything wrongly but I will have another look tomorrow ;)

Share this post


Link to post
Share on other sites
  • 0

My lack of clarity  :) .  I will amend the first post.  Thanks for highlighting.

 

The numbers are converted to strings and then concatenated.

 

11079.8,11834.3,-0.380997

=

X = 11079.8 * 10 = 110798 

Y = 11834.3 * 10 = 118343

Z = -0.380997 * 10 = -3.80997 * -10 (make positive) = 38.0997 round = 38

DIR = 351.545 rounded = 352

 

concat  110798 + 118343 + 38 + 352

Calculated objectUID

11079811834338352

 

Stored ObjectUID

11079811834338352

Share this post


Link to post
Share on other sites
  • 0

Did you see what I've written with the z coordinate. If after doing z*10 and this is still smaller then 0 that in this case always a 1 is used?

If you have a look at the second example it's 1 digit smaller:

 

11079811834338352
1577121024771131

 

And after comparing it with more values it seems that the part I've added in brown color is always 1 for a z value which has an additional 0 after the comma.

.

Is that not the case for you?

 

Edit: hm, I've found the function now:

 

dayz_objectUID2 = {
    private["_position","_dir","_key"];
    _dir = _this select 0;
    _key = "";
    _position = _this select 1;
    {
        _x = _x * 10;
        if ( _x < 0 ) then { _x = _x * -10 };
        _key = _key + str(round(_x));
    } count _position;
    _key = _key + str(round(_dir));
    _key
};

 

Only if the value is negative it's again multiplied with -10.

But if the original z value is 0.0654: it get's multiplied with 10, so it's now 0.654. The check for < 0 fails now as it's not negativ.

So it just gets rounded...and after rounding to ne next integer value it's always 1 (for all z values not negative and <0.1)?

And that's as far as I understand it the way the UID can be identical cause if z is 0.0835 the end value is also 1. Coords and dir might be the same --> same UID.

 

So something after rounding like: if (_x == 1) then _x = round (random 9); could help here to make the number more random. Or even a longer random number instead of z and dir.

Share this post


Link to post
Share on other sites
  • 0

There are many ways the resulting number can be the same but the highest probability of it happening is probably with the situation you describe where the Z coord is below 0.0 and higher than 0 (ie 0.01xxx -> 0.09xxx) with objects.

 

Randomising the Z if it rounds to 1 is ok and may we do the job but why randomise it at all rather than just keeping a running counter for allocation ?. 

Share this post


Link to post
Share on other sites
  • 0

Hm, a good solution mainly depends on understanding the problem I guess. I've never heard before that people have issues with identical object uids or I never had any.

So I am not sure if this is the best way. Maybe also just a random 17 digit number is the best solution and with the 'low' number of objects existing in the DB there will never be 2 identical ones. I do not know if these coords and direction values are somewhere else needed (or extracted again from the UId)...so that it is important to have them. But I think that was just a way to try to guarantee that there are no duplicates.

 

Also deleting objects not needed anymore will help. From what you've described the only thing that came to my mind with the algorithm given is this z-value thing for objects above each other.

If that does not describe the real problem then my guess might not help at all ;)

 

For a running counter it's not clear to me how that works. You need to remember the counter somewhere so that the last number is still known if the server restarts. Or at server restart you need to check what the last number was.

Do you alreay have something for this?

Share this post


Link to post
Share on other sites
  • 0

The issue has been reported on the Epoch GitHub here.

 

The ObjectUID is only used as a unique object identifier until the server is restarted and the actual DB assigned objectID is read in to the ARMA server.

 

For the running counter... it starts at 1 and increases on first run.  After that, as the objects are loaded on a server restart, the highest objectsUID is stored, incremented by one and is used as the starting point until the next server reboot.

 

Very simple.  The only concern is that the ARMA engine cannot keep up with new objectUID requests and hands out a duplicate one.  Saying that though, the requests should be from calls to the function which are in a scheduled env so they will run and probably complete sequentially.  Should not be a problem with duplication but I guess I can test.  Speed is also not so much of a concern as it will be used for building / creating game world objects so does not need to be lightning fast.  Throttling it if need be should not cause any issues.

 

Should be pretty easy to write.

 

For people with objects in their DB already, a sql script to zero the objects objectUID in the DB will get over any historical issues.  If the server has the objectID then the objectUID is no longer required.

 

I personally prefer an engineered solution rather than just picking a number out of a hat.  Admittedly the likelihood of the same number coming out twice is very low but there is still a chance.

Share this post


Link to post
Share on other sites
  • 0

Ok, it appears that there is a call in the hiveext to get the objectID for an object in the DB (Found by JustChil).  Call 388 will get a DB objects objectID but it is currently only used for publishing vehicles and has a 10 second timeout if no response is received.

 

I will have a look at adding the call to my publishFullObject script (part of A Plot for Lifes Take Ownership function).  This script gets hammered when someone take over a base ownership as the base objects all have to be deleted and recreated (no call to update the fields I need updated).  That should be a good stress test.  It will not remove the issue with dup ObjectUIDs sometimes occuring but if it is robust enough, it could remove the need for them.

Share this post


Link to post
Share on other sites
  • 0
On 1/6/2015 at 8:27 AM, RimBlock said:

Ok, it appears that there is a call in the hiveext to get the objectID for an object in the DB (Found by JustChil).  Call 388 will get a DB objects objectID but it is currently only used for publishing vehicles and has a 10 second timeout if no response is received.

 

I will have a look at adding the call to my publishFullObject script (part of A Plot for Lifes Take Ownership function).  This script gets hammered when someone take over a base ownership as the base objects all have to be deleted and recreated (no call to update the fields I need updated).  That should be a good stress test.  It will not remove the issue with dup ObjectUIDs sometimes occuring but if it is robust enough, it could remove the need for them.

Has this veber been fixed? Yesterday My server failed, stuck at waiting to authenticate. After a quick look, I was getting server monitor error. I checked the database and sure enough had 14 duplicated ObjectUid's and 1 was triplicated!.

I changed the numbers and the server loaded.

Again, this morning, I have another duplicted, and the server wont load. This time, I have changed the number and it will wont load, Im still getting the same error message, so not sure what else is stopping it.

This is the last straw after several annoying server destroying things happening, you fix one and then something else happens.

Can I just put all object Uid's to zero for now, just to get the thing to load?

Share this post


Link to post
Share on other sites
  • 0

I have been spending a lot of time to find a robust solution with a calculation algorithm with SQF, but still had issues with it. I have created an Arma2NET AddIn to fix this issue. It requires you to use Arma2NET on the server only. Feel free to try this solution. It does not matter if you are using custom scripts or not like: Plot Management, Vector Building etc.

 

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...

  • Similar Content

    • By lordacus
      so i have been making a server between trying to get my provider to update their mods section. after getting this don i found the server with epoch and infistar were conflicting each other such as: when joining i can use infistar but when i press f9 or i to enter my inventory or f10 for the mobile pad it then conflicts (i think) infistar and i can no longer use it. any ideas on how to stop this
       
    • By vbawol
      Open world survival mod set just two years after the mass extinction of billions of people. Those that remain are left with remnants of a once technological society. Try to survive, build, or explore your way through the harsh dynamic environment.
      Features
      Powerful persistence framework for Arma 3 with native Windows and Linux dedicated server support. Fully configurable script based security checks. Base Building re-envisioned for Arma 3 PhysX. Persistent secure and insecure storage devices. Upgradeable locking doors. Barter based economy and trading systems backed a single currency ¤ called "Krypto". Hostile Environment with several different types of antagonists, Air and Water temperature extremes, and Earthquakes. Hunting, Fishing, Tame Dogs, Explore & Loot: Wrecked ships, Abandoned vehicles and buildings, Perform task based missions and more! Open source APL-SA Epoch Survival MP gamemode. Open source APL-SA Modders resource configs and Class List. Assets can be used as a mod dependency on Steam workshop.
    • By Achef
      Greetings!
      So I recently started up in a new host and haven't gotten to make the server run, found that a ca/modules bis_fnc issue is recurring but doesn't affect players when running coop missions like the default arma 2 oa ones, but when trying with Epoch hive won't connect/start despite being configured.
      Hive error
      Meanwhile at line 23...
       
      I'm still bad with Pastebin, sorry.
       
      This error has been chasing me for 2 days now
    • By WeepiestLem0n
      So I've been wanting to create a private ArmA 2 DayZ Epoch server for a bunch of my mates to play on, however I dont have a dedicated machine and want to run it on my PC. So I looked at the server download files on the GitHub page and followed the ReadMe, however I absolutely cannot figure out how to get the MySQL server and DB set up. I have downloaded MySQL Server on my PC and created the admin role, root passwords and ran it as a windows service. I now have no clue how to create the database and am wondering if anyone can help me set this up or let me know if I've done anything wrong. 

      I can add on Steam or Discord if need be for convenience.
    • By WinterLuka
      Winter Gaming is a gaming community, that was created in early 2018. We're currently running DayZ Epoch 1.0.6.2 server.
      Winter Gaming is a project created with passion – for players from players.
      On our server, the rules are equal for all players – no Pay2Win, no benefits for donating.
       
      Server IP: 164.132.203.240:2302
      Discord: https://discordapp.com/invite/ECBVKGA
      Website: http://www.wintergaming.eu
      Forum: http://www.wintergaming.eu/forums
       
      Server Configuration:
      Building Maintain - 10 Days Abandoned Vehicle Unlock - 7 Days Abandoned Safes/Lockboxes Unlock - 10 Days Building Limit – 250 Objects Max Base Height - 30m  
      Features:
      High FPS XP System (with awesome Perks and Trader) JSRS Compatible Wear Zombie Skin Plot Management Door Management Snap and Vector Building 20k Starting Money Coin System Running Zombies Ai Missions Ai Convoys Custom Loot Custom Locations Custom Traders Hidden Trader Selling Rare Items Toggle Grass/Environmental Sounds Deploy Bike/Motorcycle/Mozzie Walk Amongst The Dead Harvestable Hemp Tow/Lift Repair/Rearm – Service Points Bury Bodies Take Clothes Locate Vehicle Paychecks every 30 minutes Spawn Selection Abandoned Safe Mission Indestructible Metal/Cinder Bases Blow up door and more!  
      I hope to see You on the server!
  • Advertisement
  • Supporters
  • Discord

×
×
  • Create New...