Page 1 of 3 123 LastLast
Results 1 to 20 of 48

Thread: Return of XIDX: with bug fixes and updates for M2TW.

  1. #1

    Default Return of XIDX: with bug fixes and updates for M2TW.

    The Europa Barbarorum Team is proud to announce the return of one of the most venerable modding tools dating back to the early days of RTW: XIDX, with bug fixes and updates for M2TW. The “XIDX” project provides modders with a tool to assemble and disassemble IDX/DAT pairs of files as used in RTW and M2TW (including Kingdoms). Originally developed by Vercingetorix for use with Rome Total War versions of these archives on Windows only, the tool now compiles and runs on Linux as well (and should run on any proper POSIX x86 or AMD64 environment).

    The project is based on the source code of the XIDX program as published on the TWC. It has been heavily modified, the comprehensible change log is as follows:

    Features:
    • Add -m option for building Medieval II versions of supported pack types.
    • Add -b option for transparent suffix ('.bin') manipulation (useful with skeletons for avoiding name collisions).
    • Add -e option for assembly/disassembly of event packs.


    Bug fixes:
    • Port/fixes to build on Linux with GCC (tested with various versions >= 4.3.2)
    • Builds with GCC-Mingw32 and GCC-Mingw64 (tested with various versions >= 4.4.4)
    • Fix lurking segfaults on exit and inside the fork function of the XIDX shell.
    • Fix memory allocation, so the tool does not allocate lots of memory it never uses.


    Misc:
    Strip batch files, MS Visual Studio project file etc., added makefile.

    The program is available as both a source and a binary distribution from our FTP server. Both distributions are packaged as 7zip archives. Programs such as 7-zip can be used to extract these contents. (You can get 7-zip from here: http://www.7-zip.org/ or commandline equivalent p7zip from here: http://p7zip.sourceforge.net/ )

    The program is made available under the GPL v.2 or later, copy of which is provided in licence.txt of either distribution. Below follows the summary:

    Terms of Use
    This software is free and there is no warranty what so ever. Use at your own risk.
    See licence.txt for a copy of the program licence.

    Copyright (C) 2004, 2005 Vercingetorix <vercingetorix11 at gmail dot com>
    Copyright (C) 2011 The Europa Barbarorum Team <webmaster at europabarbarorum dot com>

    This program is free software; you can redistribute it and/or modify
    it under the terms of the GNU General Public License as published by
    the Free Software Foundation; either version 2 of the License, or
    (at your option) any later version.

    This program is distributed in the hope that it will be useful,
    but WITHOUT ANY WARRANTY; without even the implied warranty of
    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
    GNU General Public License for more details at
    http://www.fsf.org/licensing/licenses/gpl.txt

    Comments, criticism, improvements and bug reports are welcome. Regards,

    The Europa Barbarorum Team.
    Last edited by Tellos Athenaios; October 19, 2011 at 06:10 PM. Reason: Point links to TWC download db
    -Tellos Athenaios
    CUF tool - XIDX - PACK tool - SD tool - EVT tool

    “ὁ δ᾽ ἠλίθιος ὣσπερ πρόβατον βῆ βῆ λέγων βαδίζει” – Kratinos in Dionysalexandros.

  2. #2

    Default Re: Return of XIDX: with bug fixes and updates for M2TW.

    THANK THE LORD!!!!!


    I can now finally extract animation idxs on Medieval 2! All I needed was the animation files, now I can repack with ease..... I think.

  3. #3

    Default Re: Return of XIDX: with bug fixes and updates for M2TW.

    EDIT, nope, maybe I got this function wrong. But it still doesn't extract them right. It extracts them in name only, if you look at it via hex editor, it is showing them internally as a rome total war cas not a medieval 2 one, You can pack them right, but not extract them.

  4. #4

    Default Re: Return of XIDX: with bug fixes and updates for M2TW.

    Need to check what's happening there then as it was only updated to support packing explicitly (which consists of writing the right version stamp, basically since packing should not touch the actual contents of the CAS files).
    Last edited by Tellos Athenaios; October 17, 2011 at 08:24 PM.
    -Tellos Athenaios
    CUF tool - XIDX - PACK tool - SD tool - EVT tool

    “ὁ δ᾽ ἠλίθιος ὣσπερ πρόβατον βῆ βῆ λέγων βαδίζει” – Kratinos in Dionysalexandros.

  5. #5

    Default Re: Return of XIDX: with bug fixes and updates for M2TW.

    yeah, if you can figure this out I'll rep you so hard, you'll be on the ground.

  6. #6

    Default Re: Return of XIDX: with bug fixes and updates for M2TW.

    Okay it turns out that the old code has this nugget burried in its depths:

    Code:
    bool aidxSubFile::PrepareDataOut(){
        bool result;
        scale = 1.0f / scale;
        result = Scale();
        scale = 1.0f / scale;
        return result;
    }    
         
    bool aidxSubFile::Scale(){
        if( buffer==NULL ) return false;        
        
        if( (scale != 1.0f) && (buffer[4] == 1) ){           
            //if( CheckProblemCas() ) return true; 
                   
            //float* pelvis = (float*) &buffer[ (numFrames*numBones*16)+5 ];        
            //float* dist = (float*) &pelvis[ numFrames*3 ];
            //float* root = (float*) &dist[ 4*((numFrames-1) / 2) ];
            //float* end = &root[ numFrames*3 ];
            //for( int i = 0; i < 3*numFrames; i++ ){            
            //    pelvis[i] *= scale;
            //    root[i] *= scale;
            //}    
            //for( int i = 0; i < 4*((numFrames-1) / 2); i++ ){
            //    dist[ i ] *= scale;
            //}
            float* start = (float*) &buffer[ (numFrames*numBones*16)+5 ];
            int loop = numFrames*3*2 + (4*((numFrames-1) / 2)) + 8;
            
            for( int i = 0; i < loop; i++ ){
                start[i] *= scale;
            }    
        }    
        return true;
    }
    This code interprets a data block as 32bit floating point data items which are apparently simple 3d coordinates and it scales the distance to the origin of each coordinate. I'd need to delve into some of the CAS importers/exporters for M2TW to see if this is still “correct”. (You wouldn't happen to have a few links, would you?) If this is not your problem then that would imply that the packs actually contain plain old RTW CAS files...
    -Tellos Athenaios
    CUF tool - XIDX - PACK tool - SD tool - EVT tool

    “ὁ δ᾽ ἠλίθιος ὣσπερ πρόβατον βῆ βῆ λέγων βαδίζει” – Kratinos in Dionysalexandros.

  7. #7

    Default Re: Return of XIDX: with bug fixes and updates for M2TW.

    here, one is what it should be and the second one(the one with 2 in the name) is what is being extracted.
    http://www.gamefront.com/files/20899323/MTW2_Swordsman_at_mid_c_slashlr

  8. #8

    Default Re: Return of XIDX: with bug fixes and updates for M2TW.

    Thanks for the sample. That will be useful for checking where it goes wrong.
    -Tellos Athenaios
    CUF tool - XIDX - PACK tool - SD tool - EVT tool

    “ὁ δ᾽ ἠλίθιος ὣσπερ πρόβατον βῆ βῆ λέγων βαδίζει” – Kratinos in Dionysalexandros.

  9. #9
    Polycarpe's Avatar Back into action!
    Join Date
    Feb 2010
    Location
    Quebec, Canada
    Posts
    3,338

    Default Re: Return of XIDX: with bug fixes and updates for M2TW.

    If I understand correctly, this tool can import animations of RTW to M2TW? Sorry for the stupid question, I didn't understand properly the features it brought here.

  10. #10

    Default Re: Return of XIDX: with bug fixes and updates for M2TW.

    No, it can do what the old XIDX extractor could do, only better.
    -Tellos Athenaios
    CUF tool - XIDX - PACK tool - SD tool - EVT tool

    “ὁ δ᾽ ἠλίθιος ὣσπερ πρόβατον βῆ βῆ λέγων βαδίζει” – Kratinos in Dionysalexandros.

  11. #11

    Default Re: Return of XIDX: with bug fixes and updates for M2TW.

    Quote Originally Posted by warman222 View Post
    EDIT, nope, maybe I got this function wrong. But it still doesn't extract them right. It extracts them in name only, if you look at it via hex editor, it is showing them internally as a rome total war cas not a medieval 2 one, You can pack them right, but not extract them.
    I think I know what the problem is, now, and it is *not* with the XIDX tool.

    I've tested round tripping of your CAS file (pack it as animation archive, unpack it, verify the file contents do not change with a SHA256 hash). That works fine, so my understanding of the XIDX code is correct and no modifications are made when packing the file (absent any scaling).

    So I checked the Vanilla archives to see if there was an issue there. Lo and behold: the Vanilla animation packs do not contain any such CAS files as the ones you submitted. Instead, they contain (only) RTW CAS data.

    The metadata which is contained in these new M2TW CAS files such as bone names is apparently stripped from the CAS file before packing. Presumably this is what the skeleton pack is for (that DAT file does contain strings which look like bone names such as “bone_Rfoot”), but I know of no tool to generate the skeletons (though there has been some research into that subject).

    I do not have the necessary understanding of either the two different CAS file formats and the binary stuff in the skeletons pack to write a tool for that. Do you happen to know if there are any CAS exporter/importer tools floating around that come with source code?
    -Tellos Athenaios
    CUF tool - XIDX - PACK tool - SD tool - EVT tool

    “ὁ δ᾽ ἠλίθιος ὣσπερ πρόβατον βῆ βῆ λέγων βαδίζει” – Kratinos in Dionysalexandros.

  12. #12

    Default Re: Return of XIDX: with bug fixes and updates for M2TW.

    I am no expect on this ethier, but I would try some any of the current importer/exporters for 3dsmax, they are in maxscript format and can be opened up easily.


    EDIT

    I just notice something, if you use extract skeletons function in xidx and open up one of the new files in a hex editior, it has bone data and the list of animation names, look into that aswell.
    Last edited by warman222; October 20, 2011 at 07:41 PM.

  13. #13

    Default Re: Return of XIDX: with bug fixes and updates for M2TW.

    Update with the following bug fixes:

    Bugs
    • Fully zero out the buffers before using them, instead of just assigning '\0' to the first byte. This makes valgrind a bit happier (quieter), which is a good thing.
    • Fix a bug in sound packing code. A 1 had crept in where a 4 should have been. Ouch.
    • Some code cleanup to have less code in a few macro's.


    Misc
    • Generate statically linked binaries for Windows. This has the benefit of actually working if the user doesn't have the MinGW version of a couple of DLLs installed on his system (which is likely). The system I used for testing of the previous release masked the problem because these DLLs were present...
    -Tellos Athenaios
    CUF tool - XIDX - PACK tool - SD tool - EVT tool

    “ὁ δ᾽ ἠλίθιος ὣσπερ πρόβατον βῆ βῆ λέγων βαδίζει” – Kratinos in Dionysalexandros.

  14. #14

    Default Re: Return of XIDX: with bug fixes and updates for M2TW.

    Care to post another link to the download link the ftp server seems to be defunct.

  15. #15

    Default Re: Return of XIDX: with bug fixes and updates for M2TW.

    Both links work for me; which one appears to be broken for you?
    -Tellos Athenaios
    CUF tool - XIDX - PACK tool - SD tool - EVT tool

    “ὁ δ᾽ ἠλίθιος ὣσπερ πρόβατον βῆ βῆ λέγων βαδίζει” – Kratinos in Dionysalexandros.

  16. #16

    Default Re: Return of XIDX: with bug fixes and updates for M2TW.

    None of the 2 work for me whenever I left click the download button a message comes up saying "Enter password for ftp on ftp://europabarbarorum.org"

  17. #17

    Default Re: Return of XIDX: with bug fixes and updates for M2TW.

    Could you re-upload them here or on fileserve etc. It would be much appreciated.

  18. #18

    Default Re: Return of XIDX: with bug fixes and updates for M2TW.

    Which means whatever your browser is (probably Firefox), it doesn't understand the concept of anonymous FTP.

    I'll edit the links, see if that works?
    -Tellos Athenaios
    CUF tool - XIDX - PACK tool - SD tool - EVT tool

    “ὁ δ᾽ ἠλίθιος ὣσπερ πρόβατον βῆ βῆ λέγων βαδίζει” – Kratinos in Dionysalexandros.

  19. #19

    Default Re: Return of XIDX: with bug fixes and updates for M2TW.

    Thanks man!Much rep

  20. #20

    Default Re: Return of XIDX: with bug fixes and updates for M2TW.

    Minor update, containing a few miscellaneous fixes and decided to call it 1.0:
    • Fix bug in generated file names when extracting event packs caused by not dividing the log(position in the pack) by log(10). This ensures that when you list the files (dir, ls, find) they will appear in the same order as they should be placed in the pack because of how lexicographic sorting works. (Simplifies repacking.)
    • Fix version reporting.
    • Some minor improvements to the readme.
    -Tellos Athenaios
    CUF tool - XIDX - PACK tool - SD tool - EVT tool

    “ὁ δ᾽ ἠλίθιος ὣσπερ πρόβατον βῆ βῆ λέγων βαδίζει” – Kratinos in Dionysalexandros.

Page 1 of 3 123 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •