Page 1 of 30 123456789101126 ... LastLast
Results 1 to 20 of 582

Thread: Bovine M2TW Checker

  1. #1

    Default Bovine M2TW Checker

    Download the checker

    Note that the date stated on the download page says it is from August 2018, but this is not true. I have been updating it several times after feedback from users, but this does not get reflected in that date.

    ----------------

    Over the years in EB, I have worked with various pieces of the mod. I have also been tinkering with making automated tests to verify the integrity of references around the place and report any mismatches. These have picked up on a lot of errors and made fixing them easier, and I figure they could be of use to others as well. Running the tool makes no changes to game files, unless you specify in the configuration that you want to do something to unused (excess) files.

    The tool is generally tested on a handful of mods, where EB2 gets the most runtime. If the tool fails on your mod, let me know and I'll try to figure out what's wrong and how to make it more flexible.

    Kotlin source included. Anyone can use it however you like, except copyrighting it.

    -------------

    How does it work

    I read a number of files across the mod:
    Spoiler Alert, click show to read: 
    * text/export_units.txt
    * unit_models/battle_models.modeldb
    * descr_skeleton.txt
    * descr_mount.txt
    * folder structure under unit_models
    * folder structure under animations

    * export_descr_sounds_advice.txt
    * export_descr_sounds_narration.txt
    * export_descr_sounds_soldier_voice.txt
    * export_descr_sounds_stratmap_voice.txt
    * export_descr_sounds_units_voice.txt
    * text/diplomacy_speech
    * folder structure under sounds

    * export_descr_ancillaries.txt
    * export_descr_character_traits.txt
    * export_descr_buildings.txt
    * text/export_ancillaries.txt
    * text/export_vnvs.txt
    * campaign_script.txt
    * descr_strat.txt

    * descr_sm_factions.txt


    Along the way I parse various names and references in order to crossreference them. Any mismatches are written to one of a set of output files.

    -------------

    Benefits

    A number of errors can be detected, for easier fixing than unassisted.

    -------------

    Setup

    1. Ensure you have Java. Any version since Java 8 should work.

    2. Download the zip file linked at the top of this post. Unpack to anywhere you like.

    3. Edit the configuration.txt file. You must alter thistotell the tool:

    * Where your mod data folder is
    * Which map folder to read from (typically imperial_campaign)

    Here are these settings for the mods I'm typically testing the checker on, which hopefully gives you a clue of what you need to set them to yourself.
    Spoiler Alert, click show to read: 

    Quote Originally Posted by Europa Barbarorum 2
    ;Set DATAFOLDER to the data folder for your mod, such as C:/Games/M2TW/mods/my_mod_name/data
    ;C:/Games/M2TW needs to be changed to wherever your M2TW is installed to.
    ;my_mod_name needs to be changed to whatever your mod folder is called.
    DATAFOLDER=../../../mods/EBII/data

    ;set STRAT to the folder containing your campaign script and descr_events.
    ;Probably this is the default imperial_campaign.
    STRAT=world/maps/campaign/imperial_campaign
    Note that the ../../../ part tells it to look three folders upwards from its current location (the tool lies under M2TW/documentation/tools/BovineM2twChecker, to be precise). Then, the mod's data folder is mods/EBII/data.

    -----------

    Quote Originally Posted by Rise of the three Kingdoms
    ;Set DATAFOLDER to the data folder for your mod, such as C:/Games/M2TW/mods/my_mod_name/data
    ;C:/Games/M2TW needs to be changed to wherever your M2TW is installed to.
    ;my_mod_name needs to be changed to whatever your mod folder is called.
    DATAFOLDER=../../../../M2TWOtherMods/mods/rotk/data

    ;set STRAT to the folder containing your campaign script and descr_events.
    ;Probably this is the default imperial_campaign, but possibly base.
    STRAT=world/maps/campaign/imperial_campaign
    ;STRAT=world/maps/base
    Quote Originally Posted by The Great Conflicts
    ;Set DATAFOLDER to the data folder for your mod, such as C:/Games/M2TW/mods/my_mod_name/data
    ;C:/Games/M2TW needs to be changed to wherever your M2TW is installed to.
    ;my_mod_name needs to be changed to whatever your mod folder is called.
    DATAFOLDER=../../../../M2TWOtherMods/mods/TGC/data

    ;set STRAT to the folder containing your campaign script and descr_events.
    ;Probably this is the default imperial_campaign, but possibly base.
    STRAT=world/maps/campaign/imperial_campaign
    ;STRAT=world/maps/base
    Quote Originally Posted by Stainless Steel Historical Improvement Project
    ;Set DATAFOLDER to the data folder for your mod, such as C:/Games/M2TW/mods/my_mod_name/data
    ;C:/Games/M2TW needs to be changed to wherever your M2TW is installed to.
    ;my_mod_name needs to be changed to whatever your mod folder is called.
    DATAFOLDER=../../../../M2TWOtherMods/mods/ss6.3/data

    ;set STRAT to the folder containing your campaign script and descr_events.
    ;Probably this is the default imperial_campaign, but possibly base.
    STRAT=world/maps/campaign/imperial_campaign
    ;STRAT=world/maps/base
    As you can see, I have the other mods than EB2 under a copy of M2TW which I have called "M2TWOtherMods".


    You may also want to alter some of the other options below.

    * Whether the vanilla animation files are required in the mod folder (will report missing vanilla animation files) or used from the vanilla files (suppresses reports for missing vanilla animation files).
    * Whether hidden traits having specific descriptions should be reported or suppressed.
    * Whether the tool should check that your voice mod sound files have expected prefixes for the accent they are being used in
    * Whether the tool should ignore missing sound files (useful if you don't have them in as individual files, but only as a pack)
    * Which stratmap agents should be ignored when checking for missing files. Should match the agents your mod is not using, if any.
    * Any operations to perform when detecting files in your mod that are not in use.

    4. Verify that the checker works for your mod. It may take minutes to run, but may and may not fail really quickly if it's not working.

    5. Configure reports to suppress across these files under the config folder: acceptedMissingFiles, acceptedMissingMounts and the rest of the accepted[something] files, as well as genericSounds and unquotedDiplomacySounds.

    6. Configure the correct accent prefixes for battle sounds by culture in acceptedAccentPrefixes. The supplied file works for EBII's cultures and will not suit your mod. Vanilla accents are included, courtesy of Gigantus.

    7. Comment out the opening of output files you have no interest in - different modders will work on different kinds of issues.

    -------------

    Regular use
    Double click runCheck.bat. It will run for a while and open a set of Notepad instances with the detected mismatches. You can exchange notepad for your favourite editor by changing in the runCheck.bat file. NB! If any of the opened files are completely blank, this means the tool has crashed. Post here to let me know which mod it's failing on.

    -------------

    What does it report

    Spoiler Alert, click show to read: 

    * Animation files not used in descr_skeleton
    * Animation files used in descr_skeleton which do not exist

    * Model / sprite files not used in the model db
    * Model / sprite files used in the model db which do not exist
    * Model db line length markers inconsistent with actual line length
    * Model db lines with none or more than 1 trailing space characters
    * Model db incorrect entry count at the top of the file
    * Model db references to skeletons which do not exist (primary, secondary, mount, main weapon, offhand weapons, secondary weapon)
    * Model db unit texture assignments with no corresponding attachment texture assignments
    * Model db attachment texture assignments with no corresponding unit texture assignments
    * Invalid mount class references
    * Mount models not used by mounts
    * Skeletons not used in the model db
    * Model db entries not used in EDU

    * EDU mount references which do not exist
    * EDU soldier references which do not exist
    * EDU officer references which do not exist
    * EDU armour references which do not exist
    * Units recruitable in EDU by a faction which lack texture assignments (silver surfers)

    * Sound references to files which do not exist
    * Sound files present but not referenced in game files

    * Duplicate diplomatic text descriptions
    * Diplomatic text not used by any diplomat accent
    * Diplomacy sounds without a corresponding entry in diplomacy_speech.txt

    * Battle map voices with a prefix unexpected for the unit culture

    * Duplicate entries with the same ancillary name
    * EDA triggers with too many affects clauses (more than 10 causes crash)
    * Duplicate ancillary trigger names
    * Missing ancillary description text
    * Unused ancillary descriptions

    * Duplicate entries with the same trait name
    * EDCT triggers with too many affects clauses (more than 10 causes crash)
    * Duplicate trait trigger names
    * Missing trait description text
    * Unused trait descriptions
    * Traits which are never awarded
    * Antitraits with effects
    * Visible traits with hidden decription
    * Hidden traits with visible decription

    * Building references across script, EDCT, EDA, EDB, descr_strat which do not exist
    * Trait references (awarding/checking) across script, EDCT, EDA, descr_strat which do not exist

    * Script non-event counters used as event counters
    * Script non-event counters used or checked but not declared
    * Script non-event counters used before being declared
    * Script event counters used as non-event counters
    * Script event counters set but never checked
    * Script event counters checked but never set
    * Script event counters checked but never set

    Plenty more things have been added since I wrote this, and there are probably also some more things I forgot or missed when scanning the code too. In the future, I may have each check output its name so this list will be easier to maintain.

    For most mods, it will find quite a few things to correct. There remains many thousand reports on EB2 to this day.
    Last edited by bovi; May 31, 2021 at 03:47 PM. Reason: Updated instructions

    Having problems getting EB2 to run? Try these solutions.
    ================
    I do NOT answer PM requests for help with EB. Ask in a new help thread in the tech help forum.
    ================
    I think computer viruses should count as life. I think it says something about human nature that the only form of life we have created so far is purely destructive. We've created life in our own image. - Stephen Hawking

  2. #2

    Default Re: Bovine M2TW Checker

    PLEASE NOTE THAT THIS POST IS OBSOLETE.

    You are no longer supposed to put your mod's data folder or strat folder into runCheck.bat! Instead, edit configuration.txt.

    Quote Originally Posted by _Tartaros_, private message
    Hi,
    i downloaded your tool yesterday and tried to implant it to our TGC_09 internal alpha, but i somehow didn´t get it right. i altered the runcheck.bat like this:

    Code:
    set DATAFOLDER=../../../mods/TGC_09/dataset STRAT=world/maps/campaign/imperial_campaign
    REM javac -d ./target src/*.java
    
    
    java -cp target BovineM2twCheck %DATAFOLDER% %STRAT%
    start notepad eda.error.log
    start notepad edct.error.log
    start notepad battle_models.error.log
    start notepad sounds.error.log
    start notepad script.error.log
    start notepad building.error.log
    I think your problem is this:
    set DATAFOLDER=../../../mods/TGC_09/dataset STRAT=world/maps/campaign/imperial_campaign

    This needs to be two lines, not one:
    set DATAFOLDER=../../../mods/TGC_09/data
    set
    STRAT=world/maps/campaign/imperial_campaign

    Also, ../../../mods/TGC_09/data is a relative path. This is correct for EBII because the tool is in the same folder structure, specifically under M2TW/documentation/tools/BovineM2twCheck, so it steps up 3 times to the M2TW folder and then goes down to mods/EBII/data.

    I don't know where you downloaded the checker to, so I cannot know precisely what you should set the data folder to. But it needs to point to your mod's data folder, so possibly this?

    C:/games/M2TW/mods/TGC_09/data

    The C:/games/M2TW part needs to be changed to wherever your Medieval II Total War is installed to.
    Last edited by bovi; May 31, 2021 at 03:49 PM. Reason: Marking as obsolete

    Having problems getting EB2 to run? Try these solutions.
    ================
    I do NOT answer PM requests for help with EB. Ask in a new help thread in the tech help forum.
    ================
    I think computer viruses should count as life. I think it says something about human nature that the only form of life we have created so far is purely destructive. We've created life in our own image. - Stephen Hawking

  3. #3
    _Tartaros_'s Avatar "Harzschütze"
    Join Date
    Aug 2009
    Location
    kvet.lɪnˌbuʁk
    Posts
    4,492

    Default Re: Bovine M2TW Checker

    ahh!! thank you
    will give it a go

  4. #4

    Default Re: Bovine M2TW Checker

    Until you have verified that it works with your mod, I suggest doing this:

    4. Verify that the checker works for your mod. Running the checker, any processing error will result in the command window closing faster than you can see it, so add a pause command at the very end of runCheck.bat. Run. If it succeeds, you can remove this pause command for subsequent runs.

    The console output you can then see will give you a better idea of why it fails, and will assist me in helping you.

    Having problems getting EB2 to run? Try these solutions.
    ================
    I do NOT answer PM requests for help with EB. Ask in a new help thread in the tech help forum.
    ================
    I think computer viruses should count as life. I think it says something about human nature that the only form of life we have created so far is purely destructive. We've created life in our own image. - Stephen Hawking

  5. #5
    _Tartaros_'s Avatar "Harzschütze"
    Join Date
    Aug 2009
    Location
    kvet.lɪnˌbuʁk
    Posts
    4,492

    Default Re: Bovine M2TW Checker

    sadly, still no result
    here´s my setup
    Attached Thumbnails Attached Thumbnails Tool.jpg  

  6. #6

    Default Re: Bovine M2TW Checker

    PLEASE NOTE THAT THIS POST IS OBSOLETE.

    You are no longer supposed to put your mod's data folder or strat folder into runCheck.bat! Instead, edit configuration.txt.

    Thanks for the screenshot, it helps a lot.

    I believe the problem here, is that you have your M2TW folder in a path that includes a space (Eigene Dateien). It didn't occur to me that people would have that, but it really isn't all that uncommon I suppose. Also, in EBII's case the relative path has worked around this problem that I didn't realize was there. I'm going to put out a new version where this fix below is already in place, and two more checks have been added, but this should make progress in the meantime.

    Add quotemarks around the data folder in the java command in runCheck.bat, like this. It should then correctly find the folders.

    java -cp target BovineM2twCheck "%DATAFOLDER%" %STRAT%
    Last edited by bovi; May 31, 2021 at 03:50 PM.

    Having problems getting EB2 to run? Try these solutions.
    ================
    I do NOT answer PM requests for help with EB. Ask in a new help thread in the tech help forum.
    ================
    I think computer viruses should count as life. I think it says something about human nature that the only form of life we have created so far is purely destructive. We've created life in our own image. - Stephen Hawking

  7. #7
    _Tartaros_'s Avatar "Harzschütze"
    Join Date
    Aug 2009
    Location
    kvet.lɪnˌbuʁk
    Posts
    4,492

    Default Re: Bovine M2TW Checker

    Quote Originally Posted by bovi View Post
    java -cp target BovineM2twCheck "%DATAFOLDER%" %STRAT%
    seems like the quotemarks did the job.

  8. #8

    Default Re: Bovine M2TW Checker

    Great! Did it pick up on any errors in your mod?

    Having problems getting EB2 to run? Try these solutions.
    ================
    I do NOT answer PM requests for help with EB. Ask in a new help thread in the tech help forum.
    ================
    I think computer viruses should count as life. I think it says something about human nature that the only form of life we have created so far is purely destructive. We've created life in our own image. - Stephen Hawking

  9. #9

    Default Re: Bovine M2TW Checker

    Quote Originally Posted by bovi View Post
    I'm going to put out a new version where this fix below is already in place, and two more checks have been added.
    Just did that! The two new checks added are:

    * Historic events crossreferenced with triggers of historic events
    * Listing of trait triggers using WhenToTest BecomesFactionLeader

    Having problems getting EB2 to run? Try these solutions.
    ================
    I do NOT answer PM requests for help with EB. Ask in a new help thread in the tech help forum.
    ================
    I think computer viruses should count as life. I think it says something about human nature that the only form of life we have created so far is purely destructive. We've created life in our own image. - Stephen Hawking

  10. #10

    Default Re: Bovine M2TW Checker

    thank you bovi, your tool found some duplicate unit names in the export_unit text file,just removed them

  11. #11
    _Tartaros_'s Avatar "Harzschütze"
    Join Date
    Aug 2009
    Location
    kvet.lɪnˌbuʁk
    Posts
    4,492

    Default Re: Bovine M2TW Checker

    Quote Originally Posted by bovi View Post
    Great! Did it pick up on any errors in your mod?
    only in battle_models.error.log
    Attached Files Attached Files

  12. #12

    Default Re: Bovine M2TW Checker

    Quote Originally Posted by _Tartaros_ View Post
    only in battle_models.error.log
    If there are no errors detected, there should be a single line in each file saying no errors were detected! Yours are completely empty, which means the checker has crashed at some point, I believe. This is also evident by the end of the battle models log file being an incomplete line:

    11847: (Entry mtwm_hr_lance's animation die_refusing) Reference "animations/hrm_lance_pennon/horse/hr_sword_fs_horse_die_refusing_modified.cas" does not exist.
    11847: (Entry mtwm_hr_lance's animation die_refusing) Reference "animations/hr_lance/hr_lance_horse_req/hr_lance_fs_horse_die_refusing.evt" does not exist.
    11848: (Entry mtwm_hr_lance's animation
    Could you post the console output? It likely has some indication of why, whether that is a readable error message I have written, or an unhandled exception. In either case, I should be able to investigate. I may need a set of your mod's files to do so, we'll get back to that in case.

    Also, you seem to be using the vanilla animations as a packed file? On EB2, we have them as expanded files in the folder structure (and then pack them again together with our custom animations), so I wrote the checker to expect whichever vanilla animations we use to be there. Since yours are packed, it isn't finding them there, so it is reporting a metric ton of missing files, but they are still available for use by your mod in the pack.

    After we locate the reason for crashing, I'll fix whatever that is, and make a new version where you can turn on or off dependence on vanilla animations.
    Last edited by bovi; September 07, 2018 at 06:03 AM.

    Having problems getting EB2 to run? Try these solutions.
    ================
    I do NOT answer PM requests for help with EB. Ask in a new help thread in the tech help forum.
    ================
    I think computer viruses should count as life. I think it says something about human nature that the only form of life we have created so far is purely destructive. We've created life in our own image. - Stephen Hawking

  13. #13
    _Tartaros_'s Avatar "Harzschütze"
    Join Date
    Aug 2009
    Location
    kvet.lɪnˌbuʁk
    Posts
    4,492

    Default Re: Bovine M2TW Checker

    ok - here´s the console output
    Attached Thumbnails Attached Thumbnails Tool01.jpg  

  14. #14

    Default Re: Bovine M2TW Checker

    Right. It is trying to turn "mount_barded_horse" into an integer, which doesn't work very well. Apparently your model DB is not formatted the same way EB2's is, at some point. I don't know whether yours is a valid format or not, but I would think it is, or you would see more problems I guess.

    Could you put your model DB file (unit_models/battle_models.modeldb) somewhere I can download from? I'll see if I can make it parse your formatting correctly.
    Last edited by bovi; September 07, 2018 at 06:52 AM.

    Having problems getting EB2 to run? Try these solutions.
    ================
    I do NOT answer PM requests for help with EB. Ask in a new help thread in the tech help forum.
    ================
    I think computer viruses should count as life. I think it says something about human nature that the only form of life we have created so far is purely destructive. We've created life in our own image. - Stephen Hawking

  15. #15
    _Tartaros_'s Avatar "Harzschütze"
    Join Date
    Aug 2009
    Location
    kvet.lɪnˌbuʁk
    Posts
    4,492

    Default Re: Bovine M2TW Checker

    send you a pm

  16. #16

    Default Re: Bovine M2TW Checker

    Thanks. My initial problem is that you have an entry at the start of the file like this:
    5 blank 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
    Parsing the file, I expect each entry's segments to be on own lines. This dummy entry is crunched into one line, which is pretty human readability hostile but valid, and the checker's parser doesn't handle this at all. I think I must change to read bytes sequentially rather than interpret line by line in order to cope with this. This may take a bit of time.

    Even when I delete this one-line entry, it runs into other problems further down the file that I haven't looked properly at yet. At first glance it seems the problem is that a space is included in the referenced file name, but I'll need to spend some more time on this, hopefully tomorrow.
    Last edited by bovi; September 07, 2018 at 12:58 PM.

    Having problems getting EB2 to run? Try these solutions.
    ================
    I do NOT answer PM requests for help with EB. Ask in a new help thread in the tech help forum.
    ================
    I think computer viruses should count as life. I think it says something about human nature that the only form of life we have created so far is purely destructive. We've created life in our own image. - Stephen Hawking

  17. #17
    _Tartaros_'s Avatar "Harzschütze"
    Join Date
    Aug 2009
    Location
    kvet.lɪnˌbuʁk
    Posts
    4,492

    Default Re: Bovine M2TW Checker

    i see
    the mount_barded_horse is right next to the blank one.

  18. #18

    Default Re: Bovine M2TW Checker

    I think I have rewritten it to successfully read sequentially now. I have successfully tested it on what is almost your model DB, but I haven't yet made sure it still succeeds with ours. I'll test some more before uploading a new version.

    The change I needed to do to yours, was to the blank entry. I don't know why it should be so long? I fail to find a way to parse it that doesn't end up in its segments not following the format of everything else. In fact, I don't know why you need to have the blank entry at all, but I guess you have a good reason...?

    The change I found I had to do was to cut it short, making it have the same segments as other entries; significantly shorter than yours.
    5 blank 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0

    Note that I cannot run the game to test whether this change actually works ingame, as I don't have the rest of your mod files that correspond to all the references in the model db. But the shorter line should follow the format, at least.

    Having problems getting EB2 to run? Try these solutions.
    ================
    I do NOT answer PM requests for help with EB. Ask in a new help thread in the tech help forum.
    ================
    I think computer viruses should count as life. I think it says something about human nature that the only form of life we have created so far is purely destructive. We've created life in our own image. - Stephen Hawking

  19. #19

    Default Re: Bovine M2TW Checker

    Quote Originally Posted by bovi View Post
    The change I needed to do to yours, was to the blank entry. I don't know why it should be so long? I fail to find a way to parse it that doesn't end up in its segments not following the format of everything else. In fact, I don't know why you need to have the blank entry at all, but I guess you have a good reason...?
    It's coming back to me now. We also have special additional segments on the first entry, it's just that we have proper values in it as well. I think that when I take this into account, I should be able to read your blank line as is.

    Having problems getting EB2 to run? Try these solutions.
    ================
    I do NOT answer PM requests for help with EB. Ask in a new help thread in the tech help forum.
    ================
    I think computer viruses should count as life. I think it says something about human nature that the only form of life we have created so far is purely destructive. We've created life in our own image. - Stephen Hawking

  20. #20

    Default Re: Bovine M2TW Checker

    I have uploaded a new version which successfully reads both EB2's and Tartaros' model db, at least on my computer. Give it a spin when you find the time! Note that there is a new command line argument.
    Last edited by bovi; September 08, 2018 at 10:50 PM.

    Having problems getting EB2 to run? Try these solutions.
    ================
    I do NOT answer PM requests for help with EB. Ask in a new help thread in the tech help forum.
    ================
    I think computer viruses should count as life. I think it says something about human nature that the only form of life we have created so far is purely destructive. We've created life in our own image. - Stephen Hawking

Page 1 of 30 123456789101126 ... LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •