For a good part of the year or so, I have been working closely with Knight Errant, a programmer who wrote tools for M2TW modding, on cracking the ever elusive skeleton file of RTW.
First of all, let's review a bit about RTW's skeleton and animation system. (Be warned that many terminologies will not make a lot of sense unless you know a little bit about RTW animations)
In data/animations folder, you get these files:
pack.dat
skeleton.dat
pack.idx
skeleton.idx
In these files, if you extract from them, you get the skeleton files such as fs_dagger, fs_archer, etc, and the animations (inside the data/animations folder in this folder) such as LIS 01 Stand Idle.cas. You also get 2 text files: anim_list.txt and list.txt. A brief explanation of what these are:
- Skeletons contains the location of bones, how many there are and entries on which animation to trigger during specific events such as attacking, dying, running, etc (let's call them "animation triggers"). A regular soldier skeleton contains 19 bones while an elephant has 24. An exception is the slinger skeleton which has 20 bones so regular soldier model (19 bones) cannot use slinger skeleton while slinger model (20 bones) cannot use regular soldier skeleton.
- Animation files, while also in the CAS format, is different than the CAS file for models. In this, the file only contains ROTATIONS of the bones. There is no data on the location of the bone or the animation of movement (except Scene Root). What this means is that when animation is applied to a skeleton, the bones will rotate regardless of their x,y,z coordinate. As such, you cannot animate bones moving on a plane by itself, they can only rotate.
- anim_list.txt lists all of the animations to be re-packed into pack.dat. If you made a new animation and it doesn't override old animations, you must list it in here to be re-packed (using XIDX pack_animations.dat) before the game can use it.
- list.txt is like the above but instead it lists skeletons. If you made a new skeleton, say, fs_swordsman_new, you must add it here, then re-pack into skeleton.dat using XIDX pack_skeletons.dat.
We can create a "new" skeleton by copying an existing skeleton (i.e. copy fs_swordsman into fs_swordsman_new) then using Vercingetorix's XIDX tool, we can hack into it and replace certain lines. For instance, when hacking fs_swordsman, we can replace the line:
data/animations/LIS 18_03 Stabbing downwards.cas
with
data/animations/LIJ 07 Javlin throw_FASTER.cas
This would replace the animation of stabbing with the animation of throwing the javelin. HOWEVER, the sound trigger of this animation will not change because we cannot alter that with Verc's XIDX. For instance, you replaced the animation of firing a bow with firing a gun. When the animation is played, the sound played will be of firing a bow, so your option left to using the sound of firing gun is to globally replace all bow firing sound with that of a gun. Replacing animation entries is about the limit of XIDX (other than changing the size of a skeleton).
With that in mind, here are the basic problems with RTW animation and skeleton system:
1. You cannot add new bones, as in you cannot make a new skeleton type. Sure, you can export new bones on your model, or even export new animations with that new skeleton. But the game cannot recognize it because the skeleton file, the fs_whatever, will not accept the new bones. It can only use what they already have. For instance, you want to add wings on a horse. This will require at least 2 new bones on the sides of the horse. You cannot do this at all. You can export the model with the new bones but the skeleton file (fs_horse) still doesn't have those 2 bones to recognize. Case in point: fs_slinger_new, which has one extra bone, a weapon bone on the right hand. You cannot use the slinger's model on any other skeleton without first removing the extra bone, nor can you use the fs_slinger_new skeleton on any other model without first adding the extra bone. If you want to see this problem, try changing the skeleton type in DMB for slingers then see how messed up the model looks in-game.
2. You cannot re-arrange bones. You're stuck with whatever skeletons already existed because in order to create "new" skeletons, you must copy from an existing fs_whatever. The animations also does not contain any positions of the bones, only their rotations (except for Scene Root). So far, with Vercingetorix's tool, you can only hack into the skeleton file and change animation entries. For instance, you want to create a soldier with very broad shoulders. You can only do this by moving the arm bones further apart from the torso bone. This cannot be done. Exporting the model with rearranged bones will not work as the game puts those bones back into the coordinates specified in the fs_whatever skeleton.
3. You cannot add or remove sound triggers. Sound triggers are sound played (when exists) during an animation. Originally this is done via EVT files. But the skeleton does not care for this EVT files which contains the sound file's name and the time it is to be triggered. This is dilemmatic for my mod, where the wargs sound like horses because they use the same skeleton thus having the same sound triggers. You either use the horse sounds, globally change them or none at all. The problem lies, again, in fs_whatever skeleton file.
4. You cannot mount riders on any other skeleton that does not already be designated as mounts. These are elephants, horses and camels (cavalry). You cannot put riders on dogs, pigs and humans even if you identify them as cavalry. For instance, my wargs use the skeleton of a horse but with new animations. My trolls are using human skeleton but listed as elephant, therefore the "crew" on this elephant is "missing", as in the game engine will not render them. This is because non-cavalry skeletons do not have a "mount bone", as I call it. A bone designated to be the mounting point. If you want to see this, just use the wardog entry in DMB and use it as a mount. The rider will be missing until the mount is killed, then the rider will pop into existence and start the death animation.
5. In-completions of animation triggers of some skeletons. For instance, fs_archer cannot engage in melee (thus an archer DMB entry has 2 skeletons: "fs_archer, fs_dagger") because there is no animation trigger for melee fight in fs_archer while fs_slinger_new could (the entry is "fs_slinger_new, fs_slinger_new"). But fs_slinger_new has no "battle reaction" animations. It's missing dodging, feigning and blocking. They just don't trigger and the soldier just stands there. You can't do anything since you can't add triggers, or even remove them.
Now, all of the problems stated above has been resolved, with a new tool written by Knight Errant. The tool is still in experimental stage and it requires a great deal of knowledge of the animations system and manual work to use it though. So I will try to share whatever knowledge and information I managed to extort from it.
NOTE: Keep in mind that this thread will focus on the skeleton editing and NOT on animations, packing or other parts of modding. Please do not clutter the thread with general questions about animating in MAX, modeling and those other stuff. There are already tutorial threads on those aspects of modding.
This the tool: RTWskeletonconverter
It requires you to have Python 3.0 installed.
Put it in data/animation where you have all of the skeletons for easier use. This tool can do 2 things:
1. It will read, for instance, fs_dagger and creates fs_dagger.txt. In other words, it converts the skeleton file into a text file which you can read and alter.
2. It can convert the text file back into skeleton file. For instance, you modified fs_dagger.txt and renamed it fs_twoknives.txt. The tool will convert it into fs_twoknives_modified. The "_modified" addition is done to keep it from overriding an existing skeleton by mistake.
Ok, now let's crack open fs_dagger and get fs_dagger.txt. If you open it up, you will see this:
Somewhat daunting, I know. And there's no manual for it too, unlike many text files you can modify in RTW. There are only numbers, which we must try to interpret and explore what they do.
This is what I have found out about what they mean or might mean:
A. This is the size of the model. Verc's XIDX can resize a model by multiplying this number and E with the new size. You can use that for easiest resizing, or you can edit this number and the coordinates to get a resize.
B. This is the number of bones in the model, excluding Scene Root, which technically isn't a bone.
C. Now this, is still elusize. I have no idea what these numbers do. If they're for some reason does SOMETHING, most of them are unused in fs_slinger_new, which uses 0 instead of a number. So I have no idea what they do. Making them all 0 (which I did try) doesn't appear to change anything on the soldier's behavior, except during the firing of a missile. It's still too hard to notice anything right now.
D. This is pretty useful. 5 0 0 0 means it's the "right hand" so far only used as an attach point for pikes. Naturally it's on the right hand bone. If you change it to head bone, then pikes for pikemen units will be attached to the soldier's head. Making it all 0 will make pikes attached to Scene Root, aka. the pelvis. 4 0 0 0 is used on the left hand bone. As of now, I have no idea what this is used for. 1 0 0 0 is the bone designated for the mounting of horse or camel. Basically it means 1 rider per mount and it follows the triggers of a horse type. 2 0 0 0 is used for elephant skeleton, so this is a mounting bone for a skeleton used as elephant. If you used this attribute on another bone, the engine will attach the crew/rider of the mount on that bone. So adding 1 0 0 0 on a bone will enable that skeleton to attach a horse rider on it. I have no idea if 3 0 0 0 is used at all since I couldn't find a skeleton that uses it.
E. Now this is the main attraction. This is the list of the bone's coordinates in respect to its parent. Changing the values of these will alter the appearance of a skeleton. You can extend arms, or the neck or create an entirely new skeleton.
F. This is the hierarchy of the bones, which bone is which bone's children, etc. I will explain how to make the list easily later in this post.
G. Unknown. Doesn't seem to do anything, really. fs_slinger_new and fs_archer even list them as all 0s.
H. The number of animation entries.
I. The entry for the animation to use. The list of animation triggers appear to be a set list. You cannot move stand_idle to the bottom and expects the engine to activate that for standing idle. It will always read the first entry as stand_idle. There appears to be a pre-determined set of trigger to be read when the engine reads a skeleton file.
J. Most of these is unknown. There's little difference between this entry's numbers and the others. There are some exceptions, most notable is the IMPACT FRAME number, where it indicated the frame an impact is supposed to occur when an attack animation is played so the opponent may react accordingly. This one is interesting because during my testing, if you put the impact frame to be 0, then the attack will not commence. If you put impact frame on all attacks to be 0, you will get a unit/soldier that CANNOT attack! Further research required.
K. This is the number of entries of sound triggers. If there is no sound to be activated, this number is set at 0.
L. I'm semi-certain the number here corresponds to the type of sound data banks. Further research required.
M. I'm semi-certain the first number is the frame the sound should start to trigger and the third number is the last frame the sound could trigger. Having the same numbers mean the sound is always triggered at that exact frame, having a range means they could be triggered at any time within the range. Note that this is still my speculation.
N. The name of the sound file/bank to derive the sound files from. This is what you change if you want this particular skeleton to produce a different sound than the one you copied it from.
O. This is still unknown. It appears to be relevant but I still don't know what they're for.
Then, we see this part:
The "0 0 0" is from fs_dagger, which you cannot use for a missile unit. That "0 0 0" is actually 3 animation triggers that are deactivated, they are missile_aim, missile_hold and missile_fire, which can be seen at the figure below it from fs_archer. I have tested and copied the 3 entries from fs_archer and paste it on fs_dagger to make a new skeleton and it works. I can just use "fs_archerdagger, fs_archerdagger" for entry in DMB instead of "fs_archer, fs_dagger". I am unsure why CA would deactivate some animation triggers. Perhaps there's a limit on the number of triggers? Or perhaps they do it simply to make skeleton files less cluttered, as in less triggers better performance.
If we compare all of these entries then we find that each skeleton is read in a series of triggers. If the entry is simply "0" then the game would say "There's no entry for it, can't play, let's crash instead!" like if you used "fs_dagger, fs_dagger" for a unit with a missile attack since missile animations are missing from fs_dagger.
Ok, so let's get to the meat of the possibilities. What are really possible with this tool? Here's an example:
This is a screenshot of my testing of a completely different skeleton but a very simple one: a snake, which is just a straight line.
This is the model I quickly made up for this testing.
I made 1 animation to test and the result can be seen here:
But what can a new kind of skeleton do for existing models/units? Well, we can now do what M2TW modders already did with adding new bones, like this:
Note that there is a drawback to this method, meaning the bow cannot be "hidden" by the engine when switching weapons. This is because it is animated, thus will not be read as "primary weapon" stand-alone object. We are also bound by the hardcode: there's a maximum of 24 bones per skeleton. I tried adding more and the game just CTD without error message.
Even though it only appears I added very few new animations on the archer, in fact ALL of the animations are new. Basically, I had to export every single animation to be used by the new skeleton (with 24 bones instead of the regular 19 bones) because the way the engine reads animations.
The RTW engine (which I think was rectified in M2TW) reads every bone in the skeleton by a set list. For the regular skeleton RTW use, this is the list:
The list's coordinates can be seen in part E above. If you convert fs_archer and fs_slinger_new you will see the difference. The slinger has an extra bone after bone_Rhand called bone_Weapon. Note that the names of bones are irrelevant. The engine ignores the names and only reads from the list. Meaning if you used a regular skeleton's animation (19 bones) on a slinger (20 bones), the rotation value of bone_Rthigh will be applied to the slinger's bone_Weapon and so on, until the slinger's last bone, bone_Rfoot, receives no rotation. Importing a slinger's animation on a skeleton with less than 20 bones will cause an error on the MAX script. You can easily test this in MAX: open the slinger's model, then import a regular animation, it will show up weird. Now try it again but delete bone_Weapon before importing the same animation. The slinger model should animate normally.
So making a new skeleton for RTW has its drawback: you have to make every single animation it's going to use. Sure, it's not really that hard since in my case, I just imported existing animations, add in the new bones then export them. Only 3 animations are using the new bones: readying the bow, holding the bow and firing the arrow. It's a time-consuming labor mostly.
So, in summary, to create a new skeleton, this is what you must do:
1. Make the model with the new skeleton.
2. Export it as a CAS model, then press F11.
3. Copy the data shown by the scipt, this is important.
4. Make all the new animations you want to use on your new skeleton, export them with all the bones.
5. Extract all the skeletons and animations.
6. Pick the closest skeleton you want to use, copy it and rename.
7. Convert the new skeleton into text file, open it.
8. At the bone coordinate part and hierarchy, use the data from part 3. However, be sure to convert the coordinates rom (x, y, z) in MAX to (-x, z, -y) for the skeleton.
9. Change all the animations into the new animations. Change any sound trigger if you want.
10. Convert the edited skeleton text file into skeleton file. Delete the old one, rename the new one.
11. Add the new skeleton name in list.txt.
12. Add all the new animations in anim_list.txt
13. Repack skeletons and animations.
14. Use the new skeleton in DMB.
So what else can be done with editing the skeleton? Here are some possibilities:
- Animated banner on banner carriers. Those banners can be given bones and animated as if fluttering in the wind.
- Centaurs! Now instead of having riders with invisible bottom and horses with invisible heads, you can have centaurs that act like regular soldiers: carry siege weapons, climb ladders, etc. Although they would loose their charge power since they're no longer cavalry....
- Dragons, serpents, pegasus, angels, demons, 4-armed warriors, all sorts of mythical creatures. As soldiers or mounts.
- Different druids and screechers having different sounds! One druid unit can chant gaelic while another druid unit can chant verses from the Bible, for instance. This is done by giving one unit a different skeleton with edited sound triggers so they don't use the same sounds. The priests in Barbarian Invasion can chant latin instead of going "ooOoOOOOOooOOOOOOooo" all the same with the druids.
- Correct hit or killing blows for custom animation. Let's say you made this sweet sword slashing animation but it's twice as long as the normal sword slash. In your animation, the sword should hit at target on the 24th frame, while in the original it hits on the 8th frame. If you only replaced the animation then the target soldier will keel over and die before the sweet animation even do the killing stroke! Now you can specify when the kill-hit actually lands.
and plenty more!
Now let's get crackin' and figure out what else we can uncover from this!
Many thanks to Knight Errant for writing this wonderful tool and putting up with my endless nagging for revisions and info.
PS. Also from Knigh Errant, a fix for Vencingetorix's CAS Export/Import tool here: CAS_IE_b79_modbyKE_ver3.ms
This updated tool does nothing for the average modder who only deals with models. The fixes are for animations. It fixes the animation glitch where soldiers will not reach their actual destination if the walking animation is edited. They will now reach their intended spots. Also fixed is the glitch where the soldier have to do a 90 degrees turn and the engine cannot turn them partially. They can now do so without glitching. One more glitch still exists though, which is the not-reaching-destination glitch for mounted units.
Last edited by wlesmana; April 20, 2010 at 05:06 AM.
Reason: grammar and more clarifications
I've managed to view the .org tutorial and will add it to their index system when it lets me.... but .org is playing up for me at the moment (not sure if anyone else is having problem viewing)
@ REQUIEM it's for LOTR:TW (check sig). It's a great mod for Alexander: Total War. If you don't have Alexander: Total War, you need to get it. It is a great expansion pack in itself (and it's a lot more challenging than Rome or BI)
I found a working link to the tutorial for anyone who can't access wlesmana's link
So what exactly will this change?? i mean im not down playing this or anything, but havnt you change the animations and skeñeton with the fell beast or am i missing anything?? either way i havent seen the bow animation in any other mod sooo i must be missing something. anyway good job
@ REQUIEM it's for LOTR:TW (check sig). It's a great mod for Alexander: Total War. If you don't have Alexander: Total War, you need to get it. It is a great expansion pack in itself (and it's a lot more challenging than Rome or BI)
I found a working link to the tutorial for anyone who can't access wlesmana's link
Yea, the org is being fickle for me as well. Someone asked a question for me there about being unable to run KE's converter but I am unable to reply. The reply, if he's reading this, is: "The converter requires Python 3.0 which is apparently not backward compatible with earlier versions". I had the same problem when KE first sent me the first version of the converter.
I still don't know why TWC won't load for me unless I go through proxy.
RTW skeletons CRACKED!!
Reply With Quote
Re: RTW skeletons CRACKED!!
Originally Posted by Aradan
