v1.3 - NTW Model Converter Now Released!

[Primergy]

And this is how it looks on my PC after exporting the equipment2 as a new file..you see names are there, the name error has something to do what you are doing:

ORIGINAL equipment NTW placement at LOADING

Read some of the Emails T.C posted (few pages back), he already explained what his issues are but maybe you can bring in a better technical solution which is doable?

Keep in mind that he's doing this in his freetime and for free.

[Ezekion]

Hi everybody. I read most threads but like i'm french i didn't understand everything.

What you do guy is awesome. But i don't have napoléon or empire so i believed understand that we can't modify Shogun 2 files, but is it still right?

[T.C.]

@ Kungfuserge

I asked you for a step-by-step description of what you are doing that crashes the export, AND if you use the latest version... No answer from you - the only thing you answered is "it crashes 50% of the time, make it more stable"

He also makes it clear that people should read previous emails, the same mistakes are being made quite alot

[kungfuserge]

cancelled

[Primergy]

// This is the index of this bone in the reference skeleton (NEEDED for re-export if bone order changed)
refBoneIndex 40
Could you explain this to us ??:

That means that you have to click on a joint, then click on the comment button and then type in which refboneindex the bone has (from 1 -40). This is only necessary when you are using a 3rd Party Program which doesn't preserve the comments.

Also ALL equipment groups are misplaced after Export even if I pick up a skeleton

Sounds like an export error, normaly equipment doesn't ask for a skeleton. Just curios, did you used the import function, too? Or saved the equipment file as ms3d from uu3d and then simply opened the file?

[kungfuserge]

cancelled

[T.C.]

@ kungfuserge:

If you would actually read what the messagebox asking for a skeleton is
telling you, you would understand why it sometimes asks, and sometimes
doesn't. It doesn't ask by chance.

There's different formats in which the data can be stored - one, and only
one of those formats, has precomputed data in it, and we need that
skeleton to display it correctly.

The other types don't have this problem, and the equipment is not wrong,
it's exactly how the game stores it (lying on the side), because the
movement is done by the attachment bone.

ALL of this is not only discussed in this very forum, but in those message
boxes as well. You just have to actually read it.

And - if something is not preserved, you pass it through other tools. Once
again, this is written at least 5 times in this forum. Your step-by-step
description doesn't mention that, either.

Then, if the game needs a hand model that you want to export, your scene
shouldn't have hands, equipment, head, etc. in it - I can't guess how your
model should look at the end.

And finally, once again - I'm not TC, so your PM didn't reach me.

[kungfuserge]

cancelled

[T.C.]

I missed this last time, my apologies Serge.

----------------------------------


And in french, because I'm a nice guy (and sorry for the missing accents,
I'm using a US keyboard right now and I'm lazy to insert them manually /
dsl qu'il y a pas d'accents, je bosse avec un clavier ricain, et suis trop
paresseux de les mettre a la main):

Si tu lisais ce que le MessageBox affiche (celui qui te demande si tu veux
ouvrir un fichier .anim en tant que squelette), tu comprendrais pourquoi
le plugin ne te demande pas pour chaque fichier.

Ce n'est pas au pif. Il y a 3 types de sous-formats pour stocker les
donnees, dont un (et seulement un) contient des donnees precalculees pour
le jeu. Pour les afficher correctement, il faut une squelette. Ce type de
sous-format est utilise pour presque tous, sauf l'equipement et les
plumes.

Ce que t'as a l'ecran pour les autres sous-formats n'est pas incorrect -
c'est *exactement* ce que tu trouves dans ces fichiers. La rotation, etc.
qui a l'air etre mauvaise apres avoir charge l'equipement, tu la trouves
dans l'os qui est utilise pour attacher l'equipement.

TOUT ca, tout ce que je viens d'ecrire, c'est ecrit dans ce forum, faut
juste le lire. Meme plus, le MessageBox mentionne en haut, l'explique
aussi.

En plus, s'il y a des trucs (commentaires/"Comments" dans Milkshape) qui
n'existent plus a l'export, ben, c'est probablement parce que tu passes
par d'autres outils. C'est les autres plugins import/export qui s'en
foutent de ces donnees. Je voudrais bien gicler la necessite de dependre
de ces commentaires, mais je vois pas vraiment ou, ou disons _comment_
stocker ces donnees differemment dans Milkshape. Si t'as des idees, ce
serait pas mal de nous les faire parvenir.
Ta description pour reproduire le crash ne parle pas du tout de passer par
un autre outil - on devine la, vu que t'as ajoute des screenshots.

Et ben, vu que NTW est base sur un systeme de plusieurs composants
(.variant_part_mesh) pour - dans le jeu - bricoler un bonhomme, il faut
les exporter separemment, bien sur. Si t'as des mains, ben, faut exporter
les mains, rien d'autre. C-a-d, exporter une scene de Milkshape qui
contient de l'equipement et une tete, et des mains, etc. ca va clairement
pas marcher. Eske ca se peut que t'essaies d'exporter plusieurs choses en
meme temps? Ben, dsl, je peux pas vraiment deviner dans le code ce que
t'essaies d'exporter a la fin...

Et ENCORE UNE FOIS, je _ne_suis_pas_ TC, il ne fait que poster mes messages.


----------------------------------


If somebody needs it in german as well, let me know...

And some more plugins

Hi,

well... I have some files for you, however, they are not the .variant_part_mesh import/export plugins. I hacked together two little plugins, that import and export all the comments of a ms3d scene. Export runs over the entier scene and exports comments for model, groups, materials and joints. A subsequent import applies all those comments again, to:
- model
- groups (by looking up node names - so they should be unique on export)
- materials (like groups)
- joints (like groups)

If some nodes are not uniquely names, the comment of the first in the scene will be exported, and on import, applied to all nodes with the same name. I could make this better, but well... it's useful already.
The best thing is, this is completely decoupled from the NTW import/export plugins, so it might even be useful for other tasks...

Please post this, and tell the guys in the forum about it - I guess primergy will be happy. I didn't test it extensively, but should work... Let me know!

Cheers

Files in OP.

[kungfuserge]

cancelled

[T.C.]

@kungfuserge:

english:
--------
Well, we tried multiple times to explain it to you in detail. Believe me,
every single one of you questions got answered. I'm sorry that you got
frustrated - key to understanding how this works is to see how the game
loads and assembles the data. The plugins don't have influence over that.
But yes, I'm c sure somebody will write a tutorial at some point.

french:
-------
Ben, on a essaie de t'expliquer comment ca marche, en detail. Tu peux me
croire, t'as des reponses a toutes tes questions dans ce forum. Je suis
desole que t'es frustre - pour vraiment capter comment les plugins
marchent, il faut savoir comment le jeu charge et combine ces fichiers.
Les plugins ne changent pas ce comportement du jeu. Mais bon, je suis sur
qu'il y a quelqu'un qui va ecrire un tutorial bientot.

[B-DizL]

Is this tutorial for ETW still valid for getting new rifles into NTW?
http://www.twcenter.net/forums/showt...custom+weapons

[T.C.]

Is this tutorial for ETW still valid for getting new rifles into NTW?
http://www.twcenter.net/forums/showt...custom+weapons

The general concept is the same as far as I know (haven't had time to properly use this yet), you just need to be careful with your workflow.

[T.C.]

I sent this message:

> Ok, just a few questions (apologies if it's stupid, I've not had time to play about with this system extensively as of yet)
>
>
> Details are stored in the comments for certain model types, but these are potentially lost in a long workflow, correct? If this is the case, couldn't you get the plugin to make a text based file somewhere and store comment data for that particular file. Then, if a file is loaded up and is missing comment data (due to workflow) prompt the user to load a text file, with the comment data?
>
> Source perhaps isn't an issue - what we'd love is if you could update any incorrect file documentation and explain any difficulties you encountered in technical terms - that way, if we need someone to do a re-write in the future, or for newer games (which we presume will only have slight alterations on current file types) we can just send them the info and try and avoid the many weeks of teething issues you had to work around. Basically anything that, if you had known when you started this, would have made the whole process easier and/or quicker

And got this reply. Should be very helpful for future plugin design, if necessary.

No problem... the comments basically store all the important information, that didn't find a home in milkshape.
So, call it "metadata", or whatever.... everything that is important for the files and the game, which can't be stored otherwise, is in the comments. In fact, some of it is redundant (e.g. equipment names shorter than 40 chars don't need that extra name in the comments, in theory, but it doesn't hurt). I need this information at least on export, to recreate what the game uses as data... of course, most of those settings can be changed to other values, e.g. shader parameters or attaching a drum on the back instead of the hips, etc.. Don't know how hardcoded the game is, you'll see when you play around with it.

About your text file - that is *exactly* what the extra-plugins do. It's manual, but I think cleaner for the following reasons:
- you can share "comments" across different files (e.g. all hats probably work the same, etc.)
- it's decoupled and reusable for other stuff you do in milkshape (there's many other plugins that depend on those comments, I noticed that while searching for something in the ms3d forums)
- you can preserve your own notes
- you don't get bugged on every export where that file is, etc..
- don't know what the exact ETW bone, that Primergy mentioned, was on the technical side, but I would bet that this approach might solve this as well (but maybe the author just hardcoded the ETW skeleton in the plugin, back then)

A typical workflow would look like:
1) load a .variant_part_mesh
2) export the comments to some file
3) export the ms3d scene to some other program, e.g. blender, 3dsmax, uu3d..., close milkshape
4) open blender, 3dsmax, uu3d, ... whatever, modify your data, then reexport to ms3d
5) open new unit mesh in ms3d
6) import the comments metadata stored in 2)
7) export back to .variant_part_mesh

About any bugs, etc. you come across - the file .variant_part_mesh format is very simple, and since I support it completely by now... no problem - just contact me if something needs to be changed or whenever something comes up. I'll be able to squeeze that in, somehow.
About the "many weeks" of fiddling around - in fact, like I said, I'm just super-busy at the moment and simply didn't get around working on it, a lot... if I count the evenings I spent on writing this, I might have spent 5 evenings, total, not more. I mean, I do pretty much exactly the same stuff at work every day, so it's not really a big mystery to me... and thanks to 'just', we didn't have to reverse engineer.

Everything is straightforward in those files, except for one detail in the type 1 models. The catch is type 1's "double position" and "double normal" storage, which is uncommon - however, after thinking about the reason for a few minutes, and knowing that there's a max bone influence count of 2 per vertex, it was pretty clear - it's simply precomputed, namely:

pos0 = originalPos * inverseGlobalBindPoseMatrixOfBoneInfluence0
pos1 = originalPos * inverseGlobalBindPoseMatrixOfBoneInfluence1

Same for the normals, IF the scale of the mesh is uniform (which seems to be the case for all the models).

This design saves 2 matrix computations per vertex per frame in the game, and (my guess) makes it probably easier to implement this component based unit system. This is probably a nice optimization, b/c the game deals with a lot of units. It's not commonly used in other games, as the drawbacks are decreased flexibility, in some ways, namely blending animations might be harder to do, inverse kinematics as well, and the file sizes increase. All this is not important for the game, though... There might be more reasons, but well... creative assembly would know best. I have not seen such a system applied somewhere else, so that's probably why other people trying to write loaders got stuck at this point - simply not knowing why this is stored that way.

I don't think there was anything else in the file format itself that kept me busy... The orientation issue turned out to be an issue with milkshape. Milkshape wants to have bone orientations as Euler angles, and there's 24 different possible combinations of specifying those... and well, Milkshape doesn't tell you which one. Furthermore it's not clear in which space they are applied, if it's degrees or radians, etc.. It turns out that Milkshape uses parent space, radians and XYZs order... the latter is a bit weird, as the gimbal lock problem tends to happen in a non-intuitive place, but well... we don't animate in milkshape. That took me a while to figure out, however, writing plugins for more advanced tools shouldn't be a problem, as most of them accept this data in alternative representations.

No really, that said, I guess there is really nothing fancy in the file format. The way data is stored is common, mainly to save space, sacrificing precision. 16 bit floating point numbers are possibly a bigger problem to implement, as most programming languages don't support them natively (and the hardware doesn't either, except for a lot of gfx cards). Especially in scripting languages, they might be a pain in the neck. Personally I had own code for that, already, so it wasn't a problem for me.

Maybe just post this? Might be interesting for others...

[T.C.]

v1.0 now out

Hi,

I added the tangent computation/generation on export, so this is the official first release... I ran a few very quick tests, so sorry if I missed some bugs. Let me know how it goes and/or if there's stuff that should be changed, etd..

This includes all 4 plugins you need for (conveniently) working with .variant_part_meshes!

Hope everything will work, and yes... start writing tutorials

Cheers

PS: I noticed that the original meshes are sometimes unoptimized - my plugin optimizes vertex usage, so importing and exporting equipments saves a few hundred vertices, at the moment - polycount stays the same, of course, so this is lossless. The original meshes simply seem to be unoptimized, sometimes, containing duplicated vertices. There's a chance this might be intended by the game's devs, but I don't think so... just looks unoptimized to me (which is common in games, especially with deadlines).

File in OP

[Dominicvs]

Magnificent, please let the developer you're working with know our huge gratitude, from modders and community alike!

AVE

[Primergy]

v1.0 now out

File in OP

Very nice, a huge thank you by me!

To bad i can test this only in 2-3 weeks :/

[Primergy]

To bad i can test this only in 2-3 weeks :/

Well, i've found a bit of time, and made some tests.

When using a vanilla model and using my usual workflow i get this error after exporting as vpm:



I should add that i didn't changed anything with the bone weigthing, simply export as fbx, import in max, saving as fbx, saving as ms3d (UU3D), open with milkshape loading the comments (works fine btw) and exporting as vpm.

I looked what could have been changed and saw that the bone order in milkshape is a bit different (the weapon bones are at the end instead of the beginning as they should)



I assume that the vertext with now bone is also responsible for the crashed i suffered with the trousers?

[T.C.]

well... I'm not at home right now, so I can't look into this, but AFAIRC,
the weapon bones are not part of the reference skeleton, right? Are they
in MS3D right after loading this one unit-piece?

When you export, the skeleton in the MS3D scene, should be the same as the
one the game actually expects for the unit part.

Anyways, I expect the tool having a few bugs, simply because I don't have
the game to test what I write. Does it work to load the vanilla unit part
into ms3d, then export it directly, unmodified? Or does this yield the
same messagebox?
If it does, then it seems that the vanilla mesh file has unassigned
vertices. This shouldn't crash the game if they handle that. It might even
be intended, but that depends on the application and well... I don't think
it is.

If you get really stuck, please attach all of your intermediate meshes of
your workflow, and I'll have a look.

Oh, I forgot... as long as the comments of the bones still contain the
index, the order in which they are displayed in MS3D shouldn't matter.

[Primergy]

Does it work to load the vanilla unit part
into ms3d, then export it directly, unmodified?

Nope, here everything works fine :/

I've attached the model files, from my workflow + an image wich i get when i import the fbx to max + comments, it would be amazing if you could take a look

Link