So, how do I get it to work? I can double-click the .exe whatever I want, but nothing happens other than a DOS-window popping up and disappearing. A more elaborate explanation as to how to make it work would be very helpful.
So, how do I get it to work? I can double-click the .exe whatever I want, but nothing happens other than a DOS-window popping up and disappearing. A more elaborate explanation as to how to make it work would be very helpful.
Sengoku: Total War (a Shogun mod for M2TW) - Work In Progress
Late Roman Era Campaign Map for M2TW
Late Roman Units for M2TW
Globalization: making someone else's problem your problem
You need to make .bat files with various commands. All info is in the ReadMe.
Version 1.1 brings:
Bugfixes
Fix memory leak resulting from exit() in the usage info function.
Misc
Change naming conventions for binary blobs extracted from event (EVT) packs. Numbering now starts at zero for every "frame", rather than just at the beginning of the pack. This change would allow other tools to guess the event (bank) that corresponds to a given file name based on the number before the bin suffix.
Emperor is basically right, alternatively you can use the commandline manually. For instance, to extract the EVT blobs from events.dat/events.idx into an "output" directory:
The above code commands first define a handy variable which will refer to the exe to launch. This example assumes you run 64bits Windows, so it picks xidx_amd64.exe.Code:set xidx=D:\xidx\bin\xidx_amd64.exe cd D:\M2TW\mods\my_mod\data\sounds mkdir output cd output "%xidx%" -xbe ../events
Then it changes the working directory to the sounds directory of your mod, creates an output directory in that place, and then changes the working directory to the output directory.
Next it calls the XIDX program "%xidx%" using the variable, passes it the command to extract the event packs and add bin file suffixes to the output files. The pack which will be extracted is the events DAT/IDX pair in your mod's sounds directory (that's what the ../events refers to).
@Emperor of Hell: How could I have missed that! I've got something working now. Copied and modified one of Vercingetorix's .bat files to extract the packs. Thanks for pointing me to the readme. I missed the important bits.
@Tellos Athenaios: I feel like an idiot. This is quite technical nearly above my capabilities to understand for a humble alpha person like me.
This line is the path to the xidx_amd64.exeCode:set xidx=D:\xidx\bin\xidx_amd64.exe
This line created an .idx pack file in the directory which you specified (create directory?).Code:cd D:\M2TW\mods\my_mod\data\sounds
Can't quite translate this.Code:mkdir output cd output
Can't quite understand the "%xidx%" part, even after reading your explanation.Code:"%xidx%" -xbe ../events
Sorry, just trying to understand this.
Last edited by Razor; March 26, 2012 at 02:48 PM.
Sengoku: Total War (a Shogun mod for M2TW) - Work In Progress
Late Roman Era Campaign Map for M2TW
Late Roman Units for M2TW
Globalization: making someone else's problem your problem
More precisely it defines a variable called "xidx". The contents (value) of that variable is the path to the xidx exe which you want to use.
. No, as explained in the previous post what it does is changing the "current working directory". If a program, say xidx, attempts to read a file called "../events.dat" then what it will actually get from the OS depends on the current working directory. The code above first changes the working directory to "D:\M2TW\mods\my_mod\data\sounds\". Then it runs the mkdir program which is a program that creates directories, so what happens is that a directory called "output" is created inside "D:\M2TW\my_mods\data\sounds". Then, the next cd command switches the working directory to this output folder, so that any files created by XIDX will be "inside" that.This line created an .idx pack file in the directory which you specified (create directory?).Code:cd D:\M2TW\mods\my_mod\data\sounds
Can't quite translate this.Code:mkdir output cd output
This line is the one which does all the work, it runs the XIDX program. "%xidx%" bit simply substitutes the value of the xidx variable defined earlier, so you end up running the following from inside that output directory:Can't quite understand the "%xidx%" part, even after reading your explanation.Code:"%xidx%" -xbe ../events
The quotes are necessary in case you have spaces in the path of the XIDX program (and hence, the value of the xidx variable).Code:"D:\xidx\bin\xidx_amd64.exe" -xbe ../events
New version available.
Bugfix
Version 1.2 fixes a memory/file descriptor leak when opening archives with the wrong archive type flag. (For example specifying the -a flag for animations packs when trying to list the contents of a sound IDX/DAT pair.)
Nice one. Thank you.
I am unsure of the difference between the x86 and amd64 versions. For unpacking sound files are there some processors on which the x86 version won't work and amd64 should be used instead?
Great. Thanks.
New version available:
Bugfixes
- Fix non-virtual destructors
- Skip packing of EVT blobs which do not conform to the filename conventions used by XIDX.
- Fix error messages when packing an EVT blob fails
- Improve file name parsing of EVT blobs (no longer limited to files from the current directory).
Feature
- Add makefile flag to skip cross compilation (MinGW targets).
Reminder: beware of the data folder
XIDX rewrites paths so they are relative to the root of the mod (or installation if you do not use a mod folder), and it assumes that the "root" data directory is the last "data/" in the path. This is pretty much necessary to make packing work by default on Windows (unless you wish to try you hand at this sort of thing, but it does mean that your last "data/" must be the right "data/". Otherwise the path saved in the IDX file will be an invalid value. Example: "mods\EBII\data\sounds\data\SFX\effect.wav" would be rewritten to "data\SFX\effect.wav" instead of "data\sounds\data\SFX\effect.wav". So if the game tries to load "data\sounds\data\SFX\effect.wav" because that is what you put in your code, it would fail. Easy workaround: do not have data folders inside your data folder, at least not in the parts that you want to pack with XIDX.
This message does not apply to packing events, since the tool does not store filenames for those in generate IDX/DAT packs anyway. Note that for events you will want to sort filenames alphabetically so the filename convention for EVT blobs will guarantee that they appear in the right order. Example command for packing EVT blobs:
Code:dir /S /A:-D /B /O:N "path/to/EVT/blobs/directory" "*.bin" | xidx -cbef "path/to/mods/my_mod/data/sounds/events"
Version 1.4 of XIDX is now available and it contains the following updates:
Features
- Add support for creating RTW event packs (add support for the RTW file version stamp)
- Extended the Readme with a lot more information on how to correctly generate packs, in particular event packs.
Fixes
- Fix file version stamps for event packs (though the game seems to ignore them anyway)
- Don't forget the Windows users: extend previous bug fix for sniffing out type numbers to paths with backslashes in them as well.
Version 1.5 of XIDX is now available.
Features
- Add -B option for arbitrary suffix manipulation. The -b switch is now equivalent to using `-B bin'
Fixes
- Fix entry size in generated animation packs when using suffix manipulation
am having problems getting re-packed anims/skeletons to work (M2 crashes on start up)
I think problem is due to
I'm trying to re-pack stuff that the anim_list had asOriginally Posted by readme
mods/Dragonborn/data/animations/
mods/akthross/data/animations/
etc...
unpacking the original (game generated?) anims put the extracted files into those folders - XIDX_x86 couldn't find the files when I left them there, so I moved the files into just data/animations/ and it made the pack.idx /.dat files fine...
however the extracted skeleton files have the mods/akthross/..... .cas path names inside them - and that seems to stay like that through re-packing process
so I'm getting my trace log file attempting to find the first .cas listed in skeletons.dat at mods/akthross/data/animations/MTW2_Spear/MTW2_Spear_Stand_A_idle.cas
which presumably isn't there anymore because the mods folder info was stripped out of the pack files
any ideas? or have I missed something else more fundamental?
Visit Total War Center Wiki for:
Total War Mods - Modding Portal - Total War Series
M2TW Modding - Battle Map Buildings - Techtrees - worldpkgdesc
Rome Remastered Modding - New Campaign Map
IWTE functions for RR - Unit models in RR
I thought there was at least one post of mine in which I attempt to clearly outline what is the matter with animations. Turns out, there isn't. So here goes.
XIDX deals with archives in the IDX/DAT file format. These archives are split over two files, one IDX file that contains meta data which describe the contents of the payload DAT file. The payload DAT file is, essentially concatenated contents of a lot of files. XIDX does not really understand any of the various files that go into the payload. It simply reads the data and copies it to the payload DAT file as is. All of the 'cleverness' involves manipulating the IDX meta data.
In the case of animations, however, the game does not use the same file format for animations inside the payload DAT file as whatever is produced by the tools to work with animations. The tools to edit animations focus on working with what I term 'source' files, which is basically the M2TW format of the animation CAS files. The game, on the other hand uses something that looks like RTW CAS animations instead. So before packing them with XIDX you would need to convert your M2TW source animations into the payload format.
Posts of mine which cover the subject, for reference:
- http://www.twcenter.net/forums/showt...1#post10484526
- http://www.twcenter.net/forums/showt...5#post13016055
- http://www.twcenter.net/forums/showt...1#post13455667
- http://www.twcenter.net/forums/showt...1#post13452825
And the file format you need to generate, as far as I understand it:
Spoiler Alert, click show to read:
Thanks for the info - I think you're overestimating what I'm attempting to do though.
As far as I understand the system/possibilities you have;
Modder/CA made .cas files - these can be worked on & can also be used by the game to re-generate it's own packs. To get game to regenerate own packs you need absolutely everything that's listed in descr_skeleton.txt in that correct .cas format and the .evt files.
Once you've got an engine generated pack set you can extract the anims and skeletons with XIDX - although you don't get .evt files back and the XIDX .cas files aren't the same sort of format that modders can work with & are also not the right sort of format for the game to use to re-generate it's own packs...
...however XIDX can re-pack those back into game readable pack format.
You can't mix and match the two types of stuff - so XIDX won't re-pack modders .cas's and the game won't re-pack XIDX .cas's... that's all fine so far.
The stage I've got to is I have existing game generated packs for the mod as it stands which included a whole heap of stuff I'm having difficulty finding the original modders versions of.
I have a smaller, new, game generated pack set which only contains two new skeletons and anims (produced by deleting everything else from descr_skeleton.txt temporarily)
If I unpack both of those I get XIDX friendly versions of the .cas's for everything, can append the two sets of lists and re-pack.
My issue seems to be that the XIDX re-packing process strips out the mods/my_old_mod/ info from the pack.idx file but not from the skeletons.dat file.
I could re-make the smaller pack by running it from M2/data level, but I can't re-make the larger pack without the mods/my_old_mod stuff, because I don't have the necessary bits, see start of problem...
Game always seems to append the mods/where_it_was_built/ info to the paths in the packs/skeletons it creates if the re-build animations process is run from a mod-folder, this doesn't seem to affect the portability of the files as the existing mod anims with that in have been running quite happily without the referred to mods being present - so I assume it's just a cross reference between the skeletons.dat info and the pack.idx stuff, which is now broken due to it getting stripped from the pack.idx only?
EDIT; also the one big existing anim pack already has references to two different mod paths in it, e.g. mods/modA/data and mods/modB/data,
so at some stage in the past someone has managed to stick two game generated packs together, but I can't see a way of achieving that with the current tool, or in game.
Was there ever a previous version that didn't strip the stuff before /data?
Last edited by makanyane; March 23, 2014 at 12:22 PM.
Visit Total War Center Wiki for:
Total War Mods - Modding Portal - Total War Series
M2TW Modding - Battle Map Buildings - Techtrees - worldpkgdesc
Rome Remastered Modding - New Campaign Map
IWTE functions for RR - Unit models in RR
Yep.
As an aside, the .evt files become part of the generated skeletons when the game regenerates the skeletons.idx/skeletons.dat files.Once you've got an engine generated pack set you can extract the anims and skeletons with XIDX - although you don't get .evt files back
Yes. Please note that the game generates packs with absolute file names in some cases. That means repacking them is required if you want to share your pack (and have it work) with others.and the XIDX .cas files aren't the same sort of format that modders can work with & are also not the right sort of format for the game to use to re-generate it's own packs...
...however XIDX can re-pack those back into game readable pack format.
More precisely (in case others read this) ... XIDX will not perform the conversion for you so if you try without converting first you will get a pack which contains invalid payload contents as far as the game is concerned.You can't mix and match the two types of stuff - so XIDX won't re-pack modders .cas's and the game won't re-pack XIDX .cas's... that's all fine so far.
What probably happened is a combination of two things:The stage I've got to is I have existing game generated packs for the mod as it stands which included a whole heap of stuff I'm having difficulty finding the original modders versions of.
I have a smaller, new, game generated pack set which only contains two new skeletons and anims (produced by deleting everything else from descr_skeleton.txt temporarily)
If I unpack both of those I get XIDX friendly versions of the .cas's for everything, can append the two sets of lists and re-pack.
My issue seems to be that the XIDX re-packing process strips out the mods/my_old_mod/ info from the pack.idx file but not from the skeletons.dat file.
I could re-make the smaller pack by running it from M2/data level, but I can't re-make the larger pack without the mods/my_old_mod stuff, because I don't have the necessary bits, see start of problem...
Game always seems to append the mods/where_it_was_built/ info to the paths in the packs/skeletons it creates if the re-build animations process is run from a mod-folder, this doesn't seem to affect the portability of the files as the existing mod anims with that in have been running quite happily without the referred to mods being present - so I assume it's just a cross reference between the skeletons.dat info and the pack.idx stuff, which is now broken due to it getting stripped from the pack.idx only?
- The game expects that the names of the skeletons are just 'names' inside the archive. It is not prepared for paths, actually. You can get that to work with XIDX 1.5 but it's a bit tricky because you need to pack from within the data\animations directory (and of course you cannot have skeletons inside subdirectories).
- Additionally the skeleton files themselves contain references to animations. When the game regenerates them it uses absolute paths for references but I checked with the skeleton pack from a Kingdoms campaign (teutonic) in which all those paths were proper relative paths (same as what goes in the animations archive). That means the references in the skeletons which the game regenerates for you are 'chained' if you will to the exact location which they were generated from as well. You should fix that if you move files around (for instance if you import skeletons from one mod to another) or if you intend to share the resulting skeletons archive with anyone else.
To deal more gracefully with the first issue I have prepared XIDX version 1.6. See next post:
Version 1.6 of XIDX is now available. It contains:
Features
- Add -P and -p options for disabling implicit path mangling.
Bug fixes
- Disable scaling for M2TW animation archives.
- Write skeleton names instead of paths by default. (May be overriden using -P or -p.)
Misc
- Add extra warnings and mention default file paths in --help output.
- Add note on correctly packing skeletons, animation CAS file format(s) conversion.
Thanks for the quick response and the updated version - am trying that now.
Just a quick note - the new version of xidx_x86.exe now requires "libwinpthread-1.dll" to run on my windows system, have downloaded a version of that .dll and put it in folder with the .exe and it now seems to run fine.
Visit Total War Center Wiki for:
Total War Mods - Modding Portal - Total War Series
M2TW Modding - Battle Map Buildings - Techtrees - worldpkgdesc
Rome Remastered Modding - New Campaign Map
IWTE functions for RR - Unit models in RR
I also noticed a couple of other more minor issues, so I've pushed out a new version.
Version 1.6.1 of XIDX is a point release addressing the following issues:
Bug fixes
- Fix Windows builds to not complain about libwinpthread-1.dll (-static).
- Fix -h option.
- Report first of -p or -P as not affecting extracting/listing archives.