Page 16 of 30 FirstFirst ... 67891011121314151617181920212223242526 ... LastLast
Results 301 to 320 of 582

Thread: Bovine M2TW Checker

  1. #301

    Default Re: Bovine M2TW Checker

    You were right to report that, and you're wrong about it being a small detail. The report would have disappeared once you removed one duplicate!

    That was a coding error, as the count of each encountered name was set to 0 on the first occurrence rather than 1. That means duplicates were not reported unless there were at least 3 instances. With the fix now in place, more true reports are surfacing.

    Having problems getting EB2 to run? Try these solutions.
    ================
    I do NOT answer PM requests for help with EB. Ask in a new help thread in the tech help forum.
    ================
    I think computer viruses should count as life. I think it says something about human nature that the only form of life we have created so far is purely destructive. We've created life in our own image. - Stephen Hawking

  2. #302

    Default Re: Bovine M2TW Checker

    Ohhh I see, I didn't think of that!

    Sent from my FP2 using Tapatalk
    Belovèse's Toolbox: export text files to spreadsheet, detailed unit stats
    Stainless Steel Historical Improvement Project (SSHIP) team member.
    Mini-mods: diplomacy and relation/reputation - detailled unit stats

  3. #303

    Default Re: Bovine M2TW Checker

    Uploaded a new version.

    Bugfixes:
    Duplicate usage of names for a faction is no longer reported, as that is being used for making a given name more popular than others.
    Duplicate counting now starts count at 1 for the first occurrence, rather than at 0. This caused duplicates to only be reported when there were at least 3 duplicates, now 2 should be enough.
    Reading spawn_character commands in scripts now reads the faction from that line instead of whatever was declared in the last spawn_army command.

    Converted a smattering of classes to Kotlin.
    As part of making FileReferenceRecord into Kotlin, reordered constructor arguments for FileReferenceRecord to get all optional ones at the end. Similar changes to VoiceFileReference.

    Having problems getting EB2 to run? Try these solutions.
    ================
    I do NOT answer PM requests for help with EB. Ask in a new help thread in the tech help forum.
    ================
    I think computer viruses should count as life. I think it says something about human nature that the only form of life we have created so far is purely destructive. We've created life in our own image. - Stephen Hawking

  4. #304

    Default Re: Bovine M2TW Checker

    bovi, I'm working on a tool to automatically move the files flagged as unused by your checker from the mod folder to a backup folder.

    For now it only checks graphics_n_units_error, and just process the lines starting with "File". I tested it on SSHIP, and could remove more than 3 gb of leftover files!

    If I understand well your logs, here is what files I could automatically move to a backup folder:
    - graphics_n_units_error
    -- lines starting with "File"
    -- lines starting with "Animation file"
    - sounds_error
    -- lines starting "Warning - strat map sound file "

    I have two questions:
    1) As I only tested it with SSHIP, is there other files that could be flagged by you tool for removal I could add to mine? If so, could you list them to me?
    2) Would you agree, to save me some time, to change all the lines of you logs pointing to unused files so they look something like "unused_file PATH_TO_FILE"? For now my script makes an array out of every line in your log file, and look for lines starting with "File". But if that's inconvenient to you I could make it react to how your logs are written in the actual version, of course.

    Anyway, I'll start cleaning up my tool so I can share it and keep you posted if you're interested, it may take some time as programming is just a hobby so I'm not really proficient!

  5. #305
    Gigantus's Avatar I am not special - I am a limited edition.
    Patrician Moderator Emeritus Administrator Emeritus

    Join Date
    Aug 2006
    Location
    Goa - India
    Posts
    53,095
    Blog Entries
    35

    Default Re: Bovine M2TW Checker

    That sounds like a great tool. Certainly would have saved me a couple of hours work with the latest RotK release.










  6. #306

    Default Re: Bovine M2TW Checker

    Quote Originally Posted by Gigantus View Post
    That sounds like a great tool. Certainly would have saved me a couple of hours work with the latest RotK release.
    Well, I'll get to cleaning the code and interface and will release it. I'll also add something we talked a few month ago, to clean unused export_units entries.
    Belovèse's Toolbox: export text files to spreadsheet, detailed unit stats
    Stainless Steel Historical Improvement Project (SSHIP) team member.
    Mini-mods: diplomacy and relation/reputation - detailled unit stats

  7. #307

    Default Re: Bovine M2TW Checker

    Hmm... I have been reluctant to automate any "fix", as it's hard to know what the intention was. If a file is not in use, is it superfluous or merely a typo when referencing it? I see a lot of potential damage it could cause, such as deleting files that were really meant to be used - and certainly if the tool has a bug that makes it falsely flag something!

    Still, I can understand the desire to make fixing stuff quicker, whether that is deleting superfluous files or altering game files. Making something on top of the checker's output may not be the best way, as it's liable to change every so often. I'd think it better to have the checker itself output potential remedies. Regardless, people will need to use it with caution.

    As there is now interest from others, I think I'll put the source for the tool on Github rather than EB's SVN. That will enable Belovese or anyone else to contribute to the code. It's even not entirely messy anymore, so others might understand how it works.

    -------

    As to your concrete questions, the UNIT file references are still in a messy state and I intend to change how it works. They are meant to be handled in the same way as other file references, ending up in the file reference log. I'm afraid the current parsing you're doing will be broken as part of that. I believe the file reference log has more uniform output.
    Last edited by bovi; February 24, 2021 at 01:02 AM.

  8. #308

    Default Re: Bovine M2TW Checker

    Thanks for your reply, I think it would be great if the codes becomes collaborative, but for me personally I'm not sure I'll be of much help since I'm not a programmer: I just learned JS half for fun, half to use it in electronic music creation.

    I think the easiest for me would be to simply adapt my tool to your releases, even if it is not the more efficient or elegant way.

    I'm aware of the problems of an automatic fix, that's why my tool make a backup of the files and the folder structure first. I think of this tool as a kind of last resort: when the files have not been cleaned in so much time that there is thousand of log entries to go through! At some point I might be easier and quicker to re-implement missing files than purging all those old ones. But maybe I'm mistaken, being still new to M2TW modding I could underestimate potential problems.
    Belovèse's Toolbox: export text files to spreadsheet, detailed unit stats
    Stainless Steel Historical Improvement Project (SSHIP) team member.
    Mini-mods: diplomacy and relation/reputation - detailled unit stats

  9. #309
    Gigantus's Avatar I am not special - I am a limited edition.
    Patrician Moderator Emeritus Administrator Emeritus

    Join Date
    Aug 2006
    Location
    Goa - India
    Posts
    53,095
    Blog Entries
    35

    Default Re: Bovine M2TW Checker

    The main area this tool would be very handy, provided it backs up the removed files, would be the unit_model and sprites directories**. The findings of Bovi's tool are straight forward here: either the file is surplus or a type - either way it's of no use to the mod. The back up will give a chance to recover files after adjusting whatever was incorrect should it have been a type issue - obviously one should always go through the 'file is missing' section first (that will reveal typos in the 'surplus' messages) before executing a removal of 'surplus'.

    ** I would be weary of removing strat model stuff due to it's use in multiple files including scripts.










  10. #310

    Default Re: Bovine M2TW Checker

    Quote Originally Posted by Belovèse View Post
    Thanks for your reply, I think it would be great if the codes becomes collaborative, but for me personally I'm not sure I'll be of much help since I'm not a programmer: I just learned JS half for fun, half to use it in electronic music creation.

    I think the easiest for me would be to simply adapt my tool to your releases, even if it is not the more efficient or elegant way.
    Okay, if you think that is best. I'd say if you've already learned Javascript, it's not a huge leap at all to understand Java / Kotlin code, but there is no pressure.

    Having problems getting EB2 to run? Try these solutions.
    ================
    I do NOT answer PM requests for help with EB. Ask in a new help thread in the tech help forum.
    ================
    I think computer viruses should count as life. I think it says something about human nature that the only form of life we have created so far is purely destructive. We've created life in our own image. - Stephen Hawking

  11. #311

    Default Re: Bovine M2TW Checker

    Quote Originally Posted by bovi View Post
    Okay, if you think that is best. I'd say if you've already learned Javascript, it's not a huge leap at all to understand Java / Kotlin code, but there is no pressure.
    If guess you're right, and I certainly have the curiosity to try Java and Kotlin, only problem is the time!

    Quote Originally Posted by Gigantus View Post
    The main area this tool would be very handy, provided it backs up the removed files, would be the unit_model and sprites directories**. The findings of Bovi's tool are straight forward here: either the file is surplus or a type - either way it's of no use to the mod. The back up will give a chance to recover files after adjusting whatever was incorrect should it have been a type issue - obviously one should always go through the 'file is missing' section first (that will reveal typos in the 'surplus' messages) before executing a removal of 'surplus'.

    ** I would be weary of removing strat model stuff due to it's use in multiple files including scripts.
    For now i'ts removed things in units_models, units_sprites and siege_engines. Would it be safe you think to tackle the animations also?
    Belovèse's Toolbox: export text files to spreadsheet, detailed unit stats
    Stainless Steel Historical Improvement Project (SSHIP) team member.
    Mini-mods: diplomacy and relation/reputation - detailled unit stats

  12. #312
    Gigantus's Avatar I am not special - I am a limited edition.
    Patrician Moderator Emeritus Administrator Emeritus

    Join Date
    Aug 2006
    Location
    Goa - India
    Posts
    53,095
    Blog Entries
    35

    Default Re: Bovine M2TW Checker

    I am really not an animations guy. I mean there are ways to pack animations that defy regular directory order by directly working with the pack files. That has thrown me a couple of times with RotK. Apart from that it's utterly rare that a mod will actually provide the unpacked animation files.

    Gonna give the tool a good workout with my new mod undertaking, will be nice to use it from the outset. Had the compression tool do a thing already..










  13. #313

    Default Re: Bovine M2TW Checker

    Quote Originally Posted by Belovèse View Post
    If guess you're right, and I certainly have the curiosity to try Java and Kotlin, only problem is the time!
    If you eventually do find the time, the source code for the tool is included, it's in the "src" folder after unpacking. You can let it run step by step through the code, or just throw your eyes at it to see if you can figure heads from tails.

    Its main parts are:
    data - Storage of everything read from game files
    sources - Parsing of game files, putting what it finds into data
    processing - The tests performed to discover faults

    On startup the main function in BovineM2twCheck.kt is where it all starts.

    Having problems getting EB2 to run? Try these solutions.
    ================
    I do NOT answer PM requests for help with EB. Ask in a new help thread in the tech help forum.
    ================
    I think computer viruses should count as life. I think it says something about human nature that the only form of life we have created so far is purely destructive. We've created life in our own image. - Stephen Hawking

  14. #314

    Default Re: Bovine M2TW Checker

    Uploaded a new version.

    Now reading more details from descr_mount.txt.
    NEW configuration item: REMOVE_UNUSED_FILES
    Unused files can now be automatically removed: Animation files and files checked in the FileReferences class.

    New checks:
    Mounts' models exist in the model DB.
    Mounts' models are mount models, not soldier models.
    Mounts' rider count is the same as the number of rider offsets.
    Mounts' rider offsets are not equal.
    Mounts have a water trail effect.
    Mounts' water trail effect is in a list of recognized values.
    Mounts have a recognized mount class.
    Mounts of class elephant have at least two riders.
    I'd like to point out how the new configuration item works. It's described in the configuration.txt, but perhaps this will illuminate it more.

    Quote Originally Posted by configuration.txt
    ;When encountering a file (animation, sound, model etc) where it cannot see
    ;any usage, it gets reported as unused. You will probably want to actually
    ;reference it, but possibly you instead wish to remove it to reduce mod size.
    ;Provide commands for what to do with these below.
    ;Suggested variants below, not all of them will be relevant for you.
    ;Use this with caution, and ONLY after checking for possible typos in
    ;references and files that you really want to use in the mod after all!
    ;You can define more than one - this will cause each of them to be executed,
    ;in sequence. If you want to move to a backup folder, ensure that works
    ;properly before enabling the deletion command!
    ;REMOVE_UNUSED_FILES=COPY_TO unused_backup
    ;REMOVE_UNUSED_FILES=CONSOLE_COMMAND svn delete %DATAFOLDER%/%FILENAME%
    ;REMOVE_UNUSED_FILES=CONSOLE_COMMAND git rm %DATAFOLDER%/%FILENAME%
    ;REMOVE_UNUSED_FILES=CONSOLE_COMMAND del %DATAFOLDER%/%FILENAME%
    Note that no action is taken by default - it is merely reported as unused in the logs as before. Also note that when a removal action has taken place, it will no longer be reported as an unused file in subsequent runs, as it will no longer be there.

    You should be able to use backslash (\) and slash (/) interchangeably in the commands, though I have only tested it on Windows.

    --------------

    ;REMOVE_UNUSED_FILES=COPY_TO unused_backup

    I have tested the command for copying to a backup location (COPY_TO [folder]). It made a tree of 6834 files on EB2. Even so, for your safety you MUST TEST that this part works before enabling the actual deletion.

    That is, unless you're fine with files possibly being deleted in error. If they are in source control, you're probably fine as you can revert the deletions.

    The folder you state will be created in your mod's data folder. If you want it outside of that, you can prefix with as many dots as you want to get it further up the tree. For instance, if you want that folder in the root of your M2TW folder:
    REMOVE_UNUSED_FILES=COPY_TO ../../../unused_backup

    You can not provide an absolute path to this folder. I could make it recognize that, but didn't want to make it any more complicated than necessary. You should be able to make it work like it is?

    --------------

    ;REMOVE_UNUSED_FILES=CONSOLE_COMMAND svn delete %DATAFOLDER%/%FILENAME%
    ;REMOVE_UNUSED_FILES=CONSOLE_COMMAND git rm %DATAFOLDER%/%FILENAME%
    ;REMOVE_UNUSED_FILES=CONSOLE_COMMAND del %DATAFOLDER%/%FILENAME%

    These are options for how to perform the actual removal, once you're satisfied the backup is working for you. In EB2's case, we would use the top one, which would delete the file and let SVN know it should be removed from the repository as well. I have tested this option by stepping through the code for a single file, then restoring it with SVN revert.

    The second one should work for you if you use GIT as your version control system. I have not tested it.

    The third should work for you if you do not use any kind of version control (why...?) - as long as you're doing this on a Windows computer. If you're using some kind of Unix system, I imagine substituting del with rm should do the trick.

    Having problems getting EB2 to run? Try these solutions.
    ================
    I do NOT answer PM requests for help with EB. Ask in a new help thread in the tech help forum.
    ================
    I think computer viruses should count as life. I think it says something about human nature that the only form of life we have created so far is purely destructive. We've created life in our own image. - Stephen Hawking

  15. #315

    Default Re: Bovine M2TW Checker

    Hi, does this tool also check the descr_mission?
    I recently noticed a nasty crash in TATW's missions, where every reward has a random chance so you could end up with no reward and that crashes when you get the mission.

  16. #316

    Default Re: Bovine M2TW Checker

    Thank you so much for implementing this removal option. I had an error using the third del command (I still have to learn svn or git but will look into it, and it is Jurand that keeps the main build for SSHIP), tested the backup command on a previous run and looked fine.
    Code:
    Initializing the tool.
    Starting reading process.
    Starting processing of checks.
    Unsuccessful run. Post about it in the forum and I should be able to help.
    Encountered error: Cannot run program "del": CreateProcess error=2, The system cannot find the file specified
    java.io.IOException: Cannot run program "del": CreateProcess error=2, The system cannot find the file specified
            at java.lang.ProcessBuilder.start(Unknown Source)
            at java.lang.Runtime.exec(Unknown Source)
            at java.lang.Runtime.exec(Unknown Source)
            at java.lang.Runtime.exec(Unknown Source)
            at main.BovineM2twCheck.performRemovalInstructions(BovineM2twCheck.kt:359)
            at processing.Unit.checkAnimationUsage(Unit.java:475)
            at processing.Unit.checkAnimationUsage(Unit.java:467)
            at processing.Unit.performChecks(Unit.java:206)
            at main.BovineM2twCheck.performChecks(BovineM2twCheck.kt:162)
            at main.BovineM2twCheck.run(BovineM2twCheck.kt:129)
            at main.BovineM2twCheckKt.main(BovineM2twCheck.kt:31)
    Caused by: java.io.IOException: CreateProcess error=2, The system cannot find the file specified
            at java.lang.ProcessImpl.create(Native Method)
            at java.lang.ProcessImpl.<init>(Unknown Source)
            at java.lang.ProcessImpl.start(Unknown Source)
            ... 11 more
    
    (Halting the run because of the error, so you can see this message - press ENTER to dismiss it.)
    Do you need more info or files?
    Belovèse's Toolbox: export text files to spreadsheet, detailed unit stats
    Stainless Steel Historical Improvement Project (SSHIP) team member.
    Mini-mods: diplomacy and relation/reputation - detailled unit stats

  17. #317

    Default Re: Bovine M2TW Checker

    EB2 doesn't use any of the randomized missions, rather making own specific missions using other mechanics, so this area has not been any consideration for this tool before it was released for other mods to use. No, I haven't done anything with missions yet, but I can make it do so.

    You're saying that each mission must have at least one guaranteed reward, I suppose? As an example, this mission has two potential rewards and one guaranteed reward:

    Code:
    mission council_blockade_port
    {
        COUNCIL_MISSION
        duration 10
        exclude_duration 40
        score_modifier 0.8
        max_score        100.0    ; max score clamp
        turn_start        20
        paybacks
        {
            random 0.33    payback_id council_mod_income
           random 0.5    payback_id council_mod_unit
            payback_id council_mod_unit_best
        }
    
        num_turns_success       3        ; num turns blockade must stand for success
        min_turns_distance      0        ; min search distance from ships and ports for a valid target
        max_turns_distance      1        ; max search distance from ships and ports for a valid target
    }
    When at it, I would also check that all payback_list entries are in use in at least one mission, and that all rewards in each mission are valid payback_list entries. Anything else spring to mind?

    Having problems getting EB2 to run? Try these solutions.
    ================
    I do NOT answer PM requests for help with EB. Ask in a new help thread in the tech help forum.
    ================
    I think computer viruses should count as life. I think it says something about human nature that the only form of life we have created so far is purely destructive. We've created life in our own image. - Stephen Hawking

  18. #318

    Default Re: Bovine M2TW Checker

    java.io.IOException: Cannot run program "del": CreateProcess error=2, The system cannot find the file specified
    That's weird. You're running Windows, right? I'll test it on my end to see if it's failing the same way here.

    Edit: Yes, it fails. Will fix.

    (I still have to learn svn or git but will look into it, and it is Jurand that keeps the main build for SSHIP)
    It's a bit of work to get to know them. Subversion is somewhat easier to learn than Git, in my experience, and also slightly easier to work with. Using the Tortoise client helps a lot with either of them, and Git has other benefits over Subversion. There are others out there like Mercurial and CVS, which I haven't used and cannot judge. I have used Visual Source Safe though, which I would certainly not recommend over any of the others.

    This may be of use when determining which of the systems to go for: https://medium.com/@deryacortuk17/ve...n-21b2a71226e4
    Last edited by bovi; February 26, 2021 at 08:46 AM.

    Having problems getting EB2 to run? Try these solutions.
    ================
    I do NOT answer PM requests for help with EB. Ask in a new help thread in the tech help forum.
    ================
    I think computer viruses should count as life. I think it says something about human nature that the only form of life we have created so far is purely destructive. We've created life in our own image. - Stephen Hawking

  19. #319

    Default Re: Bovine M2TW Checker

    Well I installed tortoise just before seeing your post, thanks for all the info!

    Just remembered something that might be relevant: when trying my own spin of automatic removal I had some trouble with some files in sship that had non-standard characters in their names, like an "à". In your log the special character was not displayed, but other characters. Might be the same problem, want me to reproduce it and send you the log? And yes I'm on windows.
    Belovèse's Toolbox: export text files to spreadsheet, detailed unit stats
    Stainless Steel Historical Improvement Project (SSHIP) team member.
    Mini-mods: diplomacy and relation/reputation - detailled unit stats

  20. #320

    Default Re: Bovine M2TW Checker

    Special characters may have special problems, but it's not what's happening here. I've found out you cannot run "del" directly, as it is not an executable, but rather an interpreted command for the cmd process. I needed to change the command to this:

    REMOVE_UNUSED_FILES=CONSOLE_COMMAND cmd /c del %DATAFOLDER%/%FILENAME%

    But also, the code flips "/" to "" on Windows (while on Unix systems "" is flipped to "/"), and this makes it fail without an additional code change. So, unfortunately you cannot make the fix yourself, and I can't release a fix right away, need to make dinner.

    Having problems getting EB2 to run? Try these solutions.
    ================
    I do NOT answer PM requests for help with EB. Ask in a new help thread in the tech help forum.
    ================
    I think computer viruses should count as life. I think it says something about human nature that the only form of life we have created so far is purely destructive. We've created life in our own image. - Stephen Hawking

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •