Page 1 of 21 1234567891011 ... LastLast
Results 1 to 20 of 402

Thread: v1.3 - NTW Model Converter Now Released!

  1. #1

    Default v1.3 - NTW Model Converter Now Released!

    Hi everyone,
    I've been working with a plugin developer in recent weeks with the aim of getting some progress made on NTW model conversion. I've had quite a bit of success, and today I can share a beta with everyone. One of my requests was that the plugin be designed for cheap software; he kindly obliged me and we now have a plugin for Milkshape 3d.

    Remember to only download the latest Beta, you don't need all of them

    Older beta's (Information still relevant!):

    Spoiler Alert, click show to read: 
    The type 1 models work, but I still have to get around to figure out where to get the bone transformations from, so don't be surprised that they look distorted for now. So yes, consider this a preview... however, I think it's nice to look at all that equipment already

    Oh yes, at the moment, there is no material support either, so it will all be untextured in milkshape.

    Play around with it and see if it loads - you find the plugin in milkshape under "Import -> Napoleon: Total War VPM...". Sorry, no exporter for now, either. Tell me about any crashes, I have not really spend a lot of time on border cases or testing type 1 models extensively, as there's more work to be done on this part of the code anyways.

    Feel free to announce that stuff on the forums, posting screenshots, whatever..., it's probably motivating for the community that there's something coming up. However, if you post the .dll as well, please don't forget to point out that this is not finished, of course, and that the type 1 skeleton stuff is still missing.

    Oh yes, last but not least - I have a pretty good picture on how to preserve data for the export, by now, e.g. LODs, etc. - you guys will just have to stick to a naming convention for those "groups" in milkshape.

    One thing that milkshape doesn't handle at all, though, is tangents (for normal mapping, etc.), which means that the future exporter will probably just compute them from scratch. This might not necessarily match perfectly custom normal maps (depending on how you created them), but yes, for now, you guys are probably more interested in loading anyways.

    Hope you enjoy it! More to come soon...
    Cheers
    I'll be updating this thread with progress very soon; in the coming days and weeks we expect full import/export capacity for all NTW model file types!

    To install just drop in your Milkshape directory, start MS and use the import option.


    Update: Beta 2

    Oh yes, I found the skeletons - as suspected, they are in the .anim files, and the reference/tpose is probably what we need for human-like characters. They even left the motionbuilder (character studio) file in that directory (.cs2), which I probably can pull up in 3dsmax to see what the skeleton looks like.

    However, there's a few things that I would like you to ask:

    1) I don't see reference poses for non-human entities, like horses, etc.. Of course. I could just load an animation, but the reference might be crucial for remapping

    2) does anybody of you have a file format spec for the .anim files? The hierarchy itself is simple, and I already extracted it (it's some header, bone count, then for each bone a 16bit number of chars in UCS2 format, followed by the name, then an index to the parent bone (where -1 means 'no parent'). However, the rest of the file is way to long for just storing a single frame (the reference pose), so I guess there are animations in this file, as well... and since I'm limited in time right now, it'd be cool not to re-reverse-engineer it, in case somebody already did it... maybe for another game of the series? Did you come across anything?

    3) the bone visualization of MS3D pretty much sucks for small-scaled models - do you know of a better way in milkshape to visualize this? Is it true, that you can only import/export bones, but not create bones in MilkShape?

    If not, maybe you could just post these 3 questions, and see if somebody knows...?
    I've answered these questions as best I can, but a little more help would be great!

    Howdy!

    ok, I call it a night... find attached another update. This version loads all the bones out of a skeleton (and bugs you for it, when loading a type 1 model), except for the transformations (see previous mails). But yes, the joints will be displayed with their correct names in ms3d.
    Furthermore, the shader parameters can be looked at after loading, by clicking on "comments" under the "model" tab.
    Then, attachment points for equipment, etc. will be loaded as joints with special names, namely "LOCATOR_" as prefix.
    Update: Beta 3

    Hi all,
    Got two emails during the night.

    Hi, another teaser... multi-import of several meshes sharing one skeleton works - the model on the screenshot is a:
    - french cuirass shoulderpad
    - russion infantry pants
    - austrian bearhat
    - wellington head
    - some weird belt
    - cossack torso
    ... well... something.

    It looks a bit weird, b/c every LOD level is displayed at once, and the normals are still broken. But yes, we are nearly there. This multi-import is something you guys will only need for visualization, I guess, as you have to export those unit parts seperately, but well... it's fun.

    Cheers

    Alright, find attached another beta, and a screenshot. Well, I guess you can just post all of this mail again...

    - textures have to be assisgned by hand in MS3D, as they are not referenced by the vpm files themselves, however, this is just for the looks in MS3D and not needed in the data (and for the future exporter) itself
    - MS3D seems to be buggy about rendering in the viewport - the mesh file itself is correct, but MS3D somehow doesn't use a depth-buffer for the viewport? Maybe a driver issue on my machine...
    - shader settings are in the comments box of the model (will be overwritten for subsequent mesh imports, at the moment)
    - materials are not really supported, right now, they are not stored in the mesh files, and I just set a grey default something
    - MS3D seems to be buggy about texture handling as well, see screenshot
    - multi-part import is possible, except for skeletons (e.g. a soldier and a horse, etc..), to do so, load the first part _with_ selecting a skeleton, the second without, etc.
    - there is no way to load type 1 files without a matching skeleton, if the data should be non-distorted - this is inconvenient, but the file format stores the vertices in bone space, so there is no way to get around it
    - when loading a part mesh, all LOD-levels (or items, in case of equipment) will be shown at once, select and show/hide stuff using the group-tab
    - for the exporter, you'll probably have to stick to single-unit-part-saving by respecting naming conventions for the LOD levels and the shader parameters, but yes, I'm not quite sure when I'll get around getting my hands dirty on it (I'll let you know)

    I guess that's it... please test it and tell me about all the problems you guys will encounter. At the moment, the tool is a bit verbose to show what's going on, e.g. when working on a wrong skeleton, etc.. - if it's annoying, let me know.

    Cheers

    Update: Beta 4

    And now, the moment we've all been waiting for

    Alright, here's a first version of a working exporter. However, this exports ONLY type 2 meshes, at the moment (meaning everything except equipment or plumes). Type 2 is the more complex type, so the rest is rather trivial, and well... will follow, soon. If you try to export loaded equipment or plumes, the output file will be corrupt. For type 2 exports, you'll need a skeleton, for sure... Anyways, this is just another beta, so yes, there's not a whole lot of error-feedback in there, either.

    As a sidenote, this exporter is not optimized, at all - however, since we deal with 64k triangle meshes, max, this shouldn't be noticeable.
    The importer is attached, as well, as it had a bug in the normal loading code, which has been fixed.

    All in all, I have a pretty good picture of what milkshape is capable of doing, and by sticking to a few conventions, we get every single feature of the file format covered, which is good. However, as a little warning - Milkshape has, due to its age, a few hard limitations, which aren't a problem for NTW models, but maybe later games. To be honest, NTW models just barely fit in, so I guess that one of the upcoming TW games will probably have models that will be too complex for Milkshape. So yes... if anybody plans to work on stuff in the future, Milkshape is probably not a good choice, anymore.

    The only 2 limits I'm worried about for NTW is a bone count limit of 127 bones max (please let me know if there are any models with more bones), and node/bone/mesh names of 32 characters, max. For attachables/locators (bones that are used to attach, e.g. plumes to helmets) I use the convention in Milkshape to prefix them with "LOC_" (needed for export), so this will be a limit of 28 characters. Again, let me know if you guys come across stuff that gets truncated.

    More to come, soon... just post the entire mail, again, I guess.

    Cheers
    Oh yes, one more thing... just's format description briefly mention the vmmt flags, and well, he doesn't know what they are used for. Me neither..., but he mentions that 1 out of 3 seems to have an influence in the game. Since I don't have the game, can you ask him what he observed when fiddling around with the first value?

    Thanks!
    Update: Beta 5

    And here is the latest Beta:

    Hi,

    well, another beta - it exports ALL the types, now, I haven't really tested type 0 (plumes), though.

    Anyways, there's a few issues:

    - tangents and binormals are not exported, at the moment, meaning you'll probably get wrong per pixel lighting in the game
    - it's unoptimized, but who cares, it's fast enough
    - bigger issue: the equipment files has meshes with up to 40 characters in their names, which is 8 more than what milkshape supports - at the moment they are truncated, I'll look into solutions, though (I'll probably use the mesh comments string, again)
    - for type2, which is equipment, there is an index into the skeleton pointing at a bone to which the equipment should be attached to. This is absolutely needed, and I preserve it in the comments field per mesh, on import and export. However, if your extra-exports and reimports to and from other tools, don't preserve those comments, you'll have to reenter that stuff by hand. There's a message box warning you about this, it pops up per mesh that misses this index, though... meaning 150 times in the worst case of that equipment file (yes, annoying, I'll change this, but anyways, you better think of this index)

    All in all, we are getting there... with quite a few constraints thanks to Milkshape, though.

    And yes, if somebody manages to make MS3D crash (I'm sure there are ways), please post a precise step-by-step list to reproduce the crash.

    Thanks!
    Update: Beta 6

    Hi,

    find attached the latest plugins... what's new:
    - I made a few tiny changes to the strings in the ms3d comments to have a "param value"-system
    - bugfixed to shader 4 param handling
    - bone order on export is preserved, if bone comments are
    - names for meshes longer than 32 characters are preserved, if ms3d comments on those meshes are

    Well, except for convenience stuff that might come up in the future, the only thing that is missing, now, is a tangent recomputation on export. So don't worry if the lighting in game will be corrupt, at the moment. I'll add that soon... everything else is feature complete, now.

    As you can see, a lot of the data depends on the comments strings in ms3d, namely:
    - joints always, when loaded
    - meshes for type 2, to preserve the full name and bone index to attach to (of the reference skeleton)
    - entire model for the shader params
    - locators/attachment-bones

    Well, let me know how it goes, it's definitely crucial to somehow preserve this data. If there's other/better ways, tell me...

    Cheers and happy modding
    Update: Beta 6.1

    This isn't technically a new Beta, but we now have plugins that automate the commenting procedure - these should be used along with the core Beta files to reduce errors and to simplify the conversion procedure.

    Hi,

    well... I have some files for you, however, they are not the .variant_part_mesh import/export plugins. I hacked together two little plugins, that import and export all the comments of a ms3d scene. Export runs over the entier scene and exports comments for model, groups, materials and joints. A subsequent import applies all those comments again, to:
    - model
    - groups (by looking up node names - so they should be unique on export)
    - materials (like groups)
    - joints (like groups)

    If some nodes are not uniquely names, the comment of the first in the scene will be exported, and on import, applied to all nodes with the same name. I could make this better, but well... it's useful already.
    The best thing is, this is completely decoupled from the NTW import/export plugins, so it might even be useful for other tasks...

    Please post this, and tell the guys in the forum about it - I guess primergy will be happy. I didn't test it extensively, but should work... Let me know!

    Cheers
    Update: v1.0 Full Release

    Hi,

    I added the tangent computation/generation on export, so this is the official first release... I ran a few very quick tests, so sorry if I missed some bugs. Let me know how it goes and/or if there's stuff that should be changed, etd..

    This includes all 4 plugins you need for (conveniently) working with .variant_part_meshes!

    Hope everything will work, and yes... start writing tutorials

    Cheers

    PS: I noticed that the original meshes are sometimes unoptimized - my plugin optimizes vertex usage, so importing and exporting equipments saves a few hundred vertices, at the moment - polycount stays the same, of course, so this is lossless. The original meshes simply seem to be unoptimized, sometimes, containing duplicated vertices. There's a chance this might be intended by the game's devs, but I don't think so... just looks unoptimized to me (which is common in games, especially with deadlines).
    Update: v1.1 Bug Fix Release

    Hi,

    Primergy and I finally got together to flesh out the issues, directly. Find attached version 1.1 - his models crashed b/c the game seems to expect 2 weights per vertex in ANY case, so even rigidly skinned models need the second weight, which will use any bone and a weight of 0 in that case. Makes sense to streamline the code, but well... didn't really expect this. Anyways, this should be fixed, now... once again, I don't have the game, so I'm working a bit caveman-style, meaning no guarantee that there won't be other bugs popping up.

    Cheers


    Update: v1.3 Bug Fix Release

    Hi,

    since Steph found a small glitch in the plugin, here's version 1.3 - can you please put this in your very first post of http://www.twcenter.net/forums/showthread.php?t=459454 ?
    Steph has a version, already, but it would be nice to have it in the initial post, as usual, to not confuse users.

    Thanks!

    File documentation (sort of)
    Everything is straightforward in those files, except for one detail in the type 1 models. The catch is type 1's "double position" and "double normal" storage, which is uncommon - however, after thinking about the reason for a few minutes, and knowing that there's a max bone influence count of 2 per vertex, it was pretty clear - it's simply precomputed, namely:

    pos0 = originalPos * inverseGlobalBindPoseMatrixOfBoneInfluence0
    pos1 = originalPos * inverseGlobalBindPoseMatrixOfBoneInfluence1

    Same for the normals, IF the scale of the mesh is uniform (which seems to be the case for all the models).

    This design saves 2 matrix computations per vertex per frame in the game, and (my guess) makes it probably easier to implement this component based unit system. This is probably a nice optimization, b/c the game deals with a lot of units. It's not commonly used in other games, as the drawbacks are decreased flexibility, in some ways, namely blending animations might be harder to do, inverse kinematics as well, and the file sizes increase. All this is not important for the game, though... There might be more reasons, but well... creative assembly would know best. I have not seen such a system applied somewhere else, so that's probably why other people trying to write loaders got stuck at this point - simply not knowing why this is stored that way.

    I don't think there was anything else in the file format itself that kept me busy... The orientation issue turned out to be an issue with milkshape. Milkshape wants to have bone orientations as Euler angles, and there's 24 different possible combinations of specifying those... and well, Milkshape doesn't tell you which one. Furthermore it's not clear in which space they are applied, if it's degrees or radians, etc.. It turns out that Milkshape uses parent space, radians and XYZs order... the latter is a bit weird, as the gimbal lock problem tends to happen in a non-intuitive place, but well... we don't animate in milkshape. That took me a while to figure out, however, writing plugins for more advanced tools shouldn't be a problem, as most of them accept this data in alternative representations.

    No really, that said, I guess there is really nothing fancy in the file format. The way data is stored is common, mainly to save space, sacrificing precision. 16 bit floating point numbers are possibly a bigger problem to implement, as most programming languages don't support them natively (and the hardware doesn't either, except for a lot of gfx cards). Especially in scripting languages, they might be a pain in the neck. Personally I had own code for that, already, so it wasn't a problem for me.



    Note that I've re-compressed the latest betas (hence they're smaller than v4). My internet is really that bad at uploading

    To install just drop all of the dlls in the attached folder into your Milkshape directory
    Last edited by T.C.; June 19, 2012 at 04:06 PM.
    My Tools, Tutorials and Resources

    Was running out of space, so see the full list here!

    Consider the postage stamp: its usefulness consists in the ability to stick to one thing till it gets there.- Josh Billings
    The creatures outside looked from pig to man, and from man to pig, and from pig to man again; but already it was impossible to say which was which.- George Orwell

  2. #2

    Default Re: NTW Model Converter Beta

    Sweet
    Developer of The Great War | Leader of WW2: Sandstorm | Under the Woolen Patronage of Mitch | King of All

    Quote Originally Posted by Admiral Van Tromp View Post
    History has always been a bit of the State's slut.

  3. #3
    ♔Oggie♔'s Avatar Senator
    Join Date
    Nov 2008
    Location
    The Netherlands
    Posts
    1,471

    Default Re: NTW Model Converter Beta

    Wow I'm no modder, but I can imagine the things that could be done with something like that!
    Awesome!

  4. #4
    Primergy's Avatar Protector of the Union
    Join Date
    Mar 2009
    Location
    Augsburg
    Posts
    2,491

    Default Re: NTW Model Converter Beta

    Very nice, good that there is some time till the exporter is finished, so i have time to prepare some model parts .

  5. #5
    Inevitability won
    Patrician Citizen

    Join Date
    Mar 2010
    Posts
    9,594

    Default Re: NTW Model Converter Beta

    This will mean big things. Big things indeed.

    I hope that some of the old pre-warscape modders catch-on to the possibilties we now have with ntw/warscape.

  6. #6

    Default Re: NTW Model Converter Beta

    What exactly will this let us do? Sorry I don't know much about modding.

  7. #7
    Inevitability won
    Patrician Citizen

    Join Date
    Mar 2010
    Posts
    9,594

    Default Re: NTW Model Converter Beta

    Use new unit models using the variant system.

    So basically we can now create any kind of unit, be that Starwars stromtroopers, Lord of the Rings Orcs, or just brand new Napoleonic units.

    We now have all the modability of MTW/RTW, but without the campaign map modability, plus we have many more things we can mod in the engines, that the older ones can't.

  8. #8
    'Gunny's Avatar Überrock über alles
    Content Emeritus

    Join Date
    Jun 2009
    Location
    Sunny, sunny Florida
    Posts
    8,367

    Default Re: NTW Model Converter Beta

    Wunderbar!

  9. #9

    Default Re: NTW Model Converter Beta

    Quote Originally Posted by .Mitch. View Post
    Use new unit models using the variant system.

    So basically we can now create any kind of unit, be that Starwars stromtroopers, Lord of the Rings Orcs, or just brand new Napoleonic units.

    We now have all the modability of MTW/RTW, but without the campaign map modability, plus we have many more things we can mod in the engines, that the older ones can't.
    that's great, will that be also possible to change houses, make castles, entire new models for naval warfare?

  10. #10
    Inevitability won
    Patrician Citizen

    Join Date
    Mar 2010
    Posts
    9,594

    Default Re: NTW Model Converter Beta

    Houses can already be changed, as can castles as you'v seen Kungfu do. (All are rigid models iirc) which can already be converted. Naval models still are very hard to do.

  11. #11
    wangrin's Avatar Unguibus et Rostro
    Patrician Artifex

    Join Date
    Feb 2005
    Location
    France
    Posts
    4,397

    Default Re: NTW Model Converter Beta

    This is a very great news !
    Thanks for your perseverance

    Is it possible to hope a similar converter for S2TW ?


    « Le courage, c’est de ne pas subir la loi du mensonge triomphant qui passe, et de ne pas faire écho de notre âme, de notre bouche et de nos mains aux applaudissements imbéciles et aux huées fanatiques.. » Jean JAURES

  12. #12
    Inevitability won
    Patrician Citizen

    Join Date
    Mar 2010
    Posts
    9,594

    Default Re: NTW Model Converter Beta

    S2TW updated the model formats to a slightly different format of the ones in NTW, they are infact "Version2" of the format, but I can imagine they are pretty similar, I think the S2TW ones have more bones, nothing that largely different.

    The guy is doing this in his free time for free though, so I don't think we should ask for too much.

  13. #13
    EmperorBatman999's Avatar I say, what, what?
    Join Date
    May 2009
    Location
    Why do you want to know?
    Posts
    11,891

    Default Re: NTW Model Converter Beta

    I foresee custom variants in our immediate future!

  14. #14
    Praepositus
    Join Date
    Mar 2009
    Location
    California
    Posts
    5,616

    Default Re: NTW Model Converter Beta

    What a breakthrough! Hasn't the entire league been waiting for something like this for a year and a half now?

  15. #15
    Primergy's Avatar Protector of the Union
    Join Date
    Mar 2009
    Location
    Augsburg
    Posts
    2,491

    Default Re: NTW Model Converter Beta

    Houses can already be changed, as can castles as you'v seen Kungfu do. (All are rigid models iirc) which can already be converted. Naval models still are very hard to do.
    Partially right, while you can change the look, you still have to follow the shape of a vanilla model if you want proper interaction with the enviroment. So no Castle as long as we can't edit the cs2.parsed files :/

  16. #16

    Default Re: NTW Model Converter Beta

    Is this the project where was also working jack blackshroud?

  17. #17

    Default Re: NTW Model Converter Beta

    Quote Originally Posted by Primergy View Post
    Partially right, while you can change the look, you still have to follow the shape of a vanilla model if you want proper interaction with the enviroment. So no Castle as long as we can't edit the cs2.parsed files :/
    I found this guy by spending a little time using google; I think it's about time we all took a more proactive stance towards file formats still eluding us!
    My Tools, Tutorials and Resources

    Was running out of space, so see the full list here!

    Consider the postage stamp: its usefulness consists in the ability to stick to one thing till it gets there.- Josh Billings
    The creatures outside looked from pig to man, and from man to pig, and from pig to man again; but already it was impossible to say which was which.- George Orwell

  18. #18

    Default Re: NTW Model Converter Beta

    Oh yes, I found the skeletons - as suspected, they are in the .anim files, and the reference/tpose is probably what we need for human-like characters. They even left the motionbuilder (character studio) file in that directory (.cs2), which I probably can pull up in 3dsmax to see what the skeleton looks like.

    However, there's a few things that I would like you to ask:

    1) I don't see reference poses for non-human entities, like horses, etc.. Of course. I could just load an animation, but the reference might be crucial for remapping

    2) does anybody of you have a file format spec for the .anim files? The hierarchy itself is simple, and I already extracted it (it's some header, bone count, then for each bone a 16bit number of chars in UCS2 format, followed by the name, then an index to the parent bone (where -1 means 'no parent'). However, the rest of the file is way to long for just storing a single frame (the reference pose), so I guess there are animations in this file, as well... and since I'm limited in time right now, it'd be cool not to re-reverse-engineer it, in case somebody already did it... maybe for another game of the series? Did you come across anything?

    3) the bone visualization of MS3D pretty much sucks for small-scaled models - do you know of a better way in milkshape to visualize this? Is it true, that you can only import/export bones, but not create bones in MilkShape?

    If not, maybe you could just post these 3 questions, and see if somebody knows...?
    I've answered these questions as best I can, but a little more help would be great!

    Howdy!

    ok, I call it a night... find attached another update. This version loads all the bones out of a skeleton (and bugs you for it, when loading a type 1 model), except for the transformations (see previous mails). But yes, the joints will be displayed with their correct names in ms3d.
    Furthermore, the shader parameters can be looked at after loading, by clicking on "comments" under the "model" tab.
    Then, attachment points for equipment, etc. will be loaded as joints with special names, namely "LOCATOR_" as prefix.
    Update in the OP.
    Last edited by T.C.; June 14, 2011 at 06:31 AM.
    My Tools, Tutorials and Resources

    Was running out of space, so see the full list here!

    Consider the postage stamp: its usefulness consists in the ability to stick to one thing till it gets there.- Josh Billings
    The creatures outside looked from pig to man, and from man to pig, and from pig to man again; but already it was impossible to say which was which.- George Orwell

  19. #19

    Default Re: NTW Model Converter Beta

    if some people make some tries I would be curious to see exactly what is possible to do, thanks!

  20. #20

    Default Re: NTW Model Converter Beta

    Update posted
    My Tools, Tutorials and Resources

    Was running out of space, so see the full list here!

    Consider the postage stamp: its usefulness consists in the ability to stick to one thing till it gets there.- Josh Billings
    The creatures outside looked from pig to man, and from man to pig, and from pig to man again; but already it was impossible to say which was which.- George Orwell

Page 1 of 21 1234567891011 ... LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •