ObjectUID calculation and how to improve it...

[RimBlock]

So, as many will know, there is an issue in A2Epoch (possibily from DayZ) in the unique identifier generation not always producing unique identifiers.

 

What is the unique identifier and how is it used ?.

 

Epoch uses two object identifiers.

• ObjectID
• ObjectUID

ObjectID is generated by the MySQL DB backend.  ObjectUID is generated within the ARMA 2 game world via SQF code.

 

Why two identifiers ?.

 

For some reason... there is only one HiveExt.dll call to get the ObjectID for an object in the DB.  This call is the one used to load all the DB objects in to ARMA.  The offshoot of this is that when a new object is created and saved to the DB, the DB creates a unique objectID but this objectID cannot get back to the ARMA 2 game world until the ARMA 2 server process is restarted and all the objects are then read from the DB again.

 

To get around this, the ObjectUID is created when a new object is created in the game world.  This is used to reference the object in the game world and on the DB if the ARMA 2 engine does not know the DB assigned ObjectID.

 

So, what is the problem ?.

 

Whilst the code to generate a unique objectUID looks fairly good, it does not always create a unique ID.  THis means that manipulation of any new objects that have not yet got an objectID in the game world (ie. created after the last server restart) but having an objectUID that matches the objectUID of an object currently in the DB will cause all sorts of problems.

 

How is the objectUID calculated currently ?.

 

There are three functions used to calculate the objectUID currently (all defined in server_functions.sqf).

• dayz_objectUID - No longer used it seems.
• dayz_objectUID2 - Used for objects objectUID generation.
• dayz_objectUID3 - Used for vehicle objectUID generation.

dayz_objectUID just works out the required input parameters and then calls dayz_objectUID2.

 

dayz_objectUID2 generates the objectUID for the object by applying the following calculation.

•  x,y,z coords each multiplied by 10.
• Results made positive if required by multiplying by -10 and rounded to a whole number.
• 3 numbers are concatenated together (X + Y + Z).
• The objects direction (in degrees) after rounding is then concatenated to the result.

dayz_objectUID3 does the same but adds the mission time (time in seconds since the ARMA 2 mission was started) to the direction value before adding it to the result for the x/y/x/ * 10 part.

 

Worked example for dayz_objectUID2 

Worldspace = [351.545[11079.8,11834.3,-0.380997]]

=

X = 11079.8 * 10 = 110798 

Y = 11834.3 * 10 = 118343

Z = -0.380997 * 10 = -3.80997 * -10 (make positive) = 38.0997 round = 38

DIR = 351.545 rounded = 352

 

= concat  110798 + 118343 + 38 + 352

Calculated objectUID

11079811834338352

 

Stored ObjectUID

 

11079811834338352

 

So why is it not random ?.

 

A set of coodinates ( [dir,[x,y,z]] ) for an object of [0,[1,1,10]] will produce the same result as a set of coordinates of [0,[1,10,1]].  The uniqueness plays on the probability of combination resulting in the same onjectUID being generated as unlikely.  The more items you have in your DB then the more likely it is to happen.

 

Ok, so what can we do about it ?.

 

So this is the purpose of this thread.  Both to highlight the issues to those who were not aware and to have a discussion on peoples thoughts on a better solution.

 

I am curious as to why the code does not start at 0 and each time a objectUID is needed it just increments (possibly by 2 / 5 / 10).  A server global variable could hold the current value and be populated on server restart by the highest objectUID found in the loaded objects.  The only issue I can see is if multiple requests for a new objectUID resulted in the same objectUIDs being given out to multiple requestors as the global 'current objectUID' variable cannot be updated quickly enough.  Presumabily that ould be managed by a basic schedualing system enforcing a FIFO queue so each request could be processed one at a time.

 

Any other thoughts or observations ?.  I am hoping we can all come up with a better solution that does not require any custom .dlls or DB table changes.  Good news is that any new generation system should be pretty easy to plugin to Epoch.

 

Over to you...

[jahangir13]

Hm, I am not sure if the calculation is the same for me.

Examples:

 

Obj-UID: 11079811834338352 and Worldspace: [351.545,[11079.8,11834.3,-0.380997]]

It takes x*10 and adds y*10 at the end. From z it seems to take only the first 2 values (38) after *10 and then adds the rounded value of dir at the end.

 

Obj-UID: 1577121024771131 and Worldspace: [131,[15771.2,10247.7,0.0817103]]

Here it seems that if the z value after multiplying with 10 is still <0 that just 1 is taken (the 1 before the dir of 131...the brown one).

 

I never had a look into this but as it looks to me I cannot believe that there will be 2 ObjUIDs which are the same.

This could also be different for me using the Linux server...I don't know.

 

Edit (after adding some colors): Hm, if I am right with the rounding of the z value to 1 it might be that 2 objects which are above each other and these have the same direction and the same x,y coords that it can be the case. Then it would be much better to just take a random number between 0 and 9 for this value. In my opinion this number can only be the candidate why these IDs can be identical.

 

I need some sleep and so I maybe understood everything wrongly but I will have another look tomorrow ;)

[RimBlock]

My lack of clarity  :) .  I will amend the first post.  Thanks for highlighting.

 

The numbers are converted to strings and then concatenated.

 

11079.8,11834.3,-0.380997

=

X = 11079.8 * 10 = 110798 

Y = 11834.3 * 10 = 118343

Z = -0.380997 * 10 = -3.80997 * -10 (make positive) = 38.0997 round = 38

DIR = 351.545 rounded = 352

 

= concat  110798 + 118343 + 38 + 352

Calculated objectUID

11079811834338352

 

Stored ObjectUID

11079811834338352

[jahangir13]

Did you see what I've written with the z coordinate. If after doing z*10 and this is still smaller then 0 that in this case always a 1 is used?

If you have a look at the second example it's 1 digit smaller:

 

11079811834338352
1577121024771131

 

And after comparing it with more values it seems that the part I've added in brown color is always 1 for a z value which has an additional 0 after the comma.

.

Is that not the case for you?

 

Edit: hm, I've found the function now:

 

dayz_objectUID2 = {
    private["_position","_dir","_key"];
    _dir = _this select 0;
    _key = "";
    _position = _this select 1;
    {
        _x = _x * 10;
        if ( _x < 0 ) then { _x = _x * -10 };
        _key = _key + str(round(_x));
    } count _position;
    _key = _key + str(round(_dir));
    _key
};

 

Only if the value is negative it's again multiplied with -10.

But if the original z value is 0.0654: it get's multiplied with 10, so it's now 0.654. The check for < 0 fails now as it's not negativ.

So it just gets rounded...and after rounding to ne next integer value it's always 1 (for all z values not negative and <0.1)?

And that's as far as I understand it the way the UID can be identical cause if z is 0.0835 the end value is also 1. Coords and dir might be the same --> same UID.

 

So something after rounding like: if (_x == 1) then _x = round (random 9); could help here to make the number more random. Or even a longer random number instead of z and dir.

[RimBlock]

There are many ways the resulting number can be the same but the highest probability of it happening is probably with the situation you describe where the Z coord is below 0.0 and higher than 0 (ie 0.01xxx -> 0.09xxx) with objects.

 

Randomising the Z if it rounds to 1 is ok and may we do the job but why randomise it at all rather than just keeping a running counter for allocation ?. 

[jahangir13]

Hm, a good solution mainly depends on understanding the problem I guess. I've never heard before that people have issues with identical object uids or I never had any.

So I am not sure if this is the best way. Maybe also just a random 17 digit number is the best solution and with the 'low' number of objects existing in the DB there will never be 2 identical ones. I do not know if these coords and direction values are somewhere else needed (or extracted again from the UId)...so that it is important to have them. But I think that was just a way to try to guarantee that there are no duplicates.

 

Also deleting objects not needed anymore will help. From what you've described the only thing that came to my mind with the algorithm given is this z-value thing for objects above each other.

If that does not describe the real problem then my guess might not help at all ;)

 

For a running counter it's not clear to me how that works. You need to remember the counter somewhere so that the last number is still known if the server restarts. Or at server restart you need to check what the last number was.

Do you alreay have something for this?

[RimBlock]

The issue has been reported on the Epoch GitHub here.

 

The ObjectUID is only used as a unique object identifier until the server is restarted and the actual DB assigned objectID is read in to the ARMA server.

 

For the running counter... it starts at 1 and increases on first run.  After that, as the objects are loaded on a server restart, the highest objectsUID is stored, incremented by one and is used as the starting point until the next server reboot.

 

Very simple.  The only concern is that the ARMA engine cannot keep up with new objectUID requests and hands out a duplicate one.  Saying that though, the requests should be from calls to the function which are in a scheduled env so they will run and probably complete sequentially.  Should not be a problem with duplication but I guess I can test.  Speed is also not so much of a concern as it will be used for building / creating game world objects so does not need to be lightning fast.  Throttling it if need be should not cause any issues.

 

Should be pretty easy to write.

 

For people with objects in their DB already, a sql script to zero the objects objectUID in the DB will get over any historical issues.  If the server has the objectID then the objectUID is no longer required.

 

I personally prefer an engineered solution rather than just picking a number out of a hat.  Admittedly the likelihood of the same number coming out twice is very low but there is still a chance.

[RimBlock]

Ok, it appears that there is a call in the hiveext to get the objectID for an object in the DB (Found by JustChil).  Call 388 will get a DB objects objectID but it is currently only used for publishing vehicles and has a 10 second timeout if no response is received.

 

I will have a look at adding the call to my publishFullObject script (part of A Plot for Lifes Take Ownership function).  This script gets hammered when someone take over a base ownership as the base objects all have to be deleted and recreated (no call to update the fields I need updated).  That should be a good stress test.  It will not remove the issue with dup ObjectUIDs sometimes occuring but if it is robust enough, it could remove the need for them.

[lwbuk]

On 1/6/2015 at 8:27 AM, RimBlock said:

Ok, it appears that there is a call in the hiveext to get the objectID for an object in the DB (Found by JustChil).  Call 388 will get a DB objects objectID but it is currently only used for publishing vehicles and has a 10 second timeout if no response is received.

 

I will have a look at adding the call to my publishFullObject script (part of A Plot for Lifes Take Ownership function).  This script gets hammered when someone take over a base ownership as the base objects all have to be deleted and recreated (no call to update the fields I need updated).  That should be a good stress test.  It will not remove the issue with dup ObjectUIDs sometimes occuring but if it is robust enough, it could remove the need for them.

Has this veber been fixed? Yesterday My server failed, stuck at waiting to authenticate. After a quick look, I was getting server monitor error. I checked the database and sure enough had 14 duplicated ObjectUid's and 1 was triplicated!.

I changed the numbers and the server loaded.

Again, this morning, I have another duplicted, and the server wont load. This time, I have changed the number and it will wont load, Im still getting the same error message, so not sure what else is stopping it.

This is the last straw after several annoying server destroying things happening, you fix one and then something else happens.

Can I just put all object Uid's to zero for now, just to get the thing to load?

[SKO85]

I have been spending a lot of time to find a robust solution with a calculation algorithm with SQF, but still had issues with it. I have created an Arma2NET AddIn to fix this issue. It requires you to use Arma2NET on the server only. Feel free to try this solution. It does not matter if you are using custom scripts or not like: Plot Management, Vector Building etc.