BE Keyword Picker (shell script)

[mgm]

BE KEYWORD PICKER ALPHA RELEASE





One-liner Intro
This is a chain of UNIX console commands that will help you compile a list of ALL BE filters you will need.





Detailed Intro
This tool solve a specific problem I have encountered with BattlEye filters, namely there are too many of them and for my taste this is a VERY time consuming, error-prone process to generate a DEFINITIVE list of all BattlEye Filter Exceptions your script.
I just started scripting and my only WIP mod has about 7.2K lines of code in version 0.3.0 (STATS produced by my counter script below). I can see bigger projects has well over 12K (STATS for UPSMON also copied below and yes I do realize the example is actually a server-side script)



example: Source Code STATS for:     Transport for Arma v.0.3.0

CURRENT_EPOCH_TIMESTAMP        : 1428877822
LINES_ALL_TOTAL            : 11029
LINES_ALL_EXCEPT_COMMENTS    : 7241
LINES_COMMENTS            : 3789
LINES_ALL_EXCEPT_COMMENTS_AND_BLANKS___THIS_IS_ACTUAL_CODE_DOES_NOT_EXCLUDE_TOBEEXCLUDED_FILES: 5980
LINES_IN_TOBEEXCLUDED_FILES    : 0
LINES_FINAL_ACTUAL_CODE        : 5980
CODE_COMMENT_RATIO        : 63

example: Source Code STATS for:     UPSMON_6.0.9.5    [ http://dev.withsix.com/projects/upsmon/wiki ]

CURRENT_EPOCH_TIMESTAMP         : 1428969486
LINES_ALL_TOTAL         : 15976
LINES_ALL_EXCEPT_COMMENTS               : 15359
LINES_COMMENTS          : 618
LINES_ALL_EXCEPT_COMMENTS_AND_BLANKS___THIS_IS_ACTUAL_CODE_DOES_NOT_EXCLUDE_TOBEEXCLUDED_FILES          : 12831
LINES_IN_TOBEEXCLUDED_FILES             : 0
LINES_FINAL_ACTUAL_CODE         : 12831
CODE_COMMENT_RATIO              : 4

I can't imagine what kind of trouble it would be when the codebase is over 10-15K overall for complex projects. Therefore on the above detailed problem, I started taking basic steps towards creating a solution.
The solution is based on standard UNIX utilities so to use this you will need either UNIX clone, Linux, MAC OS X or Cygwin for Microsoft Windows.

I only spent about an hour on this so far however I have already created a git repo on this and it might receive more attention in time. Here is the git repo:    https://gitlab.com/themgm/Arma_3_Misc/tree/master/BE_keyword_picker''>https://gitlab.com/themgm/Arma_3_Misc/tree/master/BE_keyword_picker'>https://gitlab.com/themgm/Arma_3_Misc/tree/master/BE_keyword_picker





How to use this?
Basically this is one block of command you copy & paste to shell & execute AFTER you modify the paths to reflect your own setup. Path names are quite self-explanatory (I hope?) but do let me know if you have any questions.





Step-by-step Guide

• Copy the command chain, beginning with this line ==>     bind '\C-i:self-insert'[*/]
• Copy till the last line, the last line you should copy is ==>        > $THIS_SCRIPT_HOMEDIR/BE_filter_generator.new_BE_filters_you_might_want_to_add.txt[*/]
• Paste the copied command to vi or notepad or similar[*/]
• Modify the paths to reflect your setup[*/]
• Copy the UPDATED command chain[*/]
• Paste the command chain to the shell and DO NOT press ENTER to execute [or launch as a shell script from a file if you prefer so].[*/]

To make the output meaningful, we should make this script 'aware' of BattlEye keywords, so that it will know what to "look for". I will work on this in the coming days.
I have added only about SIX keywords so far to test and it can process 3486 lines in less than a second (real    0m0.640s / user    0m0.136s/sys     0m0.198s).
Considering the fact that for about 3.5K files it has checked EVERY single BattlEye filter with zero errors, it's a tad bit than my manual processing speed (I can do about 2K lines per second, on a good day).

The output I see here is things like the below:

/cygdrive/h/git_repos.private/mgmTfA__DEVELOPMENT_DIR/client-side/mgmTfA.Altis/custom/mgmTfA/mgmTfA_fnc_client_sendBookingRequestFixedDestinationTaxi.sqf:  publicVariableServer  mgmTfA_gv_pvs_req_fixedDestinationTaxiToMyPositionPleaseConfirmPacket
/cygdrive/h/git_repos.private/mgmTfA__DEVELOPMENT_DIR/client-side/mgmTfA.Altis/custom/mgmTfA/mgmTfA_scr_client_init.sqf:  publicVariable  mgmTfA_pvdb_PUIDsAndPlayernamesTextStringArray
/cygdrive/h/git_repos.private/mgmTfA__DEVELOPMENT_DIR/client-side/mgmTfA.Altis/custom/mgmTfA/mgmTfA_scr_client_presentCatpActionMenu.sqf:  _CatpNearByObject  setVariable  menuAttachedStatusyesObjectDoesHaveToggleMenutrue
/cygdrive/h/git_repos.private/mgmTfA__DEVELOPMENT_DIR/client-side/mgmTfA.Altis/custom/mgmTfA/mgmTfA_scr_client_presentCatpActionMenu.sqf:  _CatpNearByObject  setVariablemenuAttachedStatus  noObjectDoesNotHaveToggleMenu  true
/cygdrive/h/git_repos.private/mgmTfA__DEVELOPMENT_DIR/client-side/mgmTfA.Altis/custom/mgmTfA/mgmTfA_scr_client_presentCatpActionMenu.sqf:  publicVariable  menuAttachedStatus

As the NEXT ACTION, I will add these to my BattlEye filters, test them, and once I'm fine with it will make its way into project documentation. At that point it will also be added to 'known_filters' document thus any future executions of the script won't warn me any more.





This solution can obviously be used (maybe better) by people who are NOT friendly with the code; so you download some code, you have NO IDEA at all what publicVariables/setPos commands used, what do you do?
You just run this and it gives you an overview quickly.

I have been using grepWin and other utilities to do multi-directory search and it is still a valid  method, I just wanted something I could eventually integrate into my build process.
What I have on mind is: after making changes to code, I click on a SINGLE link, which delete PBO + parse code as per above utility and BREAK the build process if there is an unacknowledged BE filter keyword is detected.
This will efffectively take my ability to make mistakes on publishing undocumented BattlEye keywords.
Of course, if projects-known-list-of-BE-filters-document and the source code match each other then like any support utility would do this script will silently end its run, allowing build process to continue with the next stage (namely PBO creation).

Hope this is useful to others too. Feel free to ask any questions or post positive comments lol





.