IWTE - World editing(Added tutorial)

[wilddog]

current version is IWTE13_01_A from January 8th 2013.

http://www.twcenter.net/forums/downl...o=file&id=2741

with version 13_01_A
1) changed exe to uncompressed as this was causing some issues on initial load.
2) minor changes on TGA's to check for 24 bit or 32 bit (some were still assuming 32 bit).
3) New option for vegetation which resets the normals when loading from milkshape ie they are all recalculated and assumes everything should be smoothed.
It is recommended to use this for all vegetation loads as it automatically corrects seams and resets the normals on the 2d planes used for the branches/leaves.
4) amended 3D viewer to view normals for some file types.

Bugs and next version new items
1) Bug - Some issues around paths and doors - fix completed for next version. Existing version got assignments mixed up.
1) Full GeographyDB editor - removes the need for generating the GeographyDB via the game and includes ability to change the per season RGB values. - awaiting testing by Mak.
2) Change to VegetationDB editor - season related items (mainly the assignment of models) is handled on a by season basis similar to geography to make it easier to follow - awaiting testing by Mak.
3) Add further vegetationDB items to be editable in screens similar to GeographyDB - need to test
4) world.anim file editing - ability to edit effects used - in progress.
5) world.anim file editing - ability to extract animations to milkshape - in progress. main issue is incorrect rotations.




Please ensure you back up any files first!

3D view examples
NE Fortress









Tutorials
*********
King Kong has released a tutorial to help guide people. refer to URL
http://www.twcenter.net/forums/showt...81#post8500981

Leo.civil.uefs has released a tutorial IWTE - Custom settlements - Step by Step for creating. Most important things you must know. to help guide people. refer to URL
http://www.twcenter.net/forums/showt...now&highlight=

Wilddog Tutorial on editing worldterrain files
http://www.twcenter.net/forums/showt...1#post10188991

Gigantus has converted King Kong's Tutorial to a PDF
KK's manual into PDF


The tool originated based on information and research from Argantonio and then later by Makanyane and myself as we delved deeper into meanings of each field.

Also thanks to KnightErrant who's initial code was used as the basis for this tool particularly to understand the initial file layouts.

A special thanks also for King Kong for his endless testing support and bug finding and of course for the tutorial.

[makanyane]

STARTER KIT

I've put a replacement starter kit here: http://www.twcenter.net/forums/downl...o=file&id=3596
as reported elsewhere the link to KingKong's starter kit is kaput
My version is different so will not necessarily match everything described in his tutorial.

The "starter kit for IWTE" download contains files named to be used in place of the existing north_european castle_a - to install it as that unzip folder and place 'castle' folder at
mod-folder/data/settlements/north_european/castles/castle

Please do not replace any existing files without making back-up copy of them (or re-name existing files to avoid overwriting).

I've included copies of the wall and gate animations most likely to be useful; these are the three heights of stone walls, a portculis gate and large and medium swing gates.

Original post follows;

For IWTE download link see first post. This tool works as a standalone .exe file and does not need a python installation. It covers various aspects of battlemap building editing including their immediately surrounding terrain.

It's developed by wilddog from basis of earlier tools provided by KnightErrant, and now provides enhanced features such as the fundamentally important ability to be able to add or delete objects.
It's released as a beta only at the moment - partly because its still under development, and partly because its not been properly documented yet (which I'm trying to make some effort to address in this post.... that bit's my part of the job)


some basic info about battlemap building and terrain editing can be found here:
http://www.twcenter.net/wiki/Battle_...ildings_-_M2TW
and in the 'world modified' thread here:
http://www.twcenter.net/forums/showthread.php?t=166329


assuming you've understood some bits of that and have worked out at least which files you're meant to be changing.... (if not we're not undertaking to educate anyone from scratch in building editing)


Finding Complexes
launch the IWTE 1v2 beta.exe and select the 'world' tab, select the 'Create World Complex TGA' and navigate to the .world file you would like to change.

The resultant .tga file will be created (eventually) in the same folder as the .world file, if you open that tga in graphics program you'll find plan of existing complexes. Each complex is coloured slightly differently, if you use dropper tool to select colour the 'number' of the green element should tell you which complex it is.

eg:

this example shows complex number 12

you should attempt to keep changed structures in same location as original complexes (or include them in nearest available complex) to avoid having to completely distort the complex bounding box system - its not been proved yet if that causes any major problems but its likely to increase strain on memory / game engine....

Changing Structures
Once you've identified the area you want to change go to 'structures' tab or one of the others with 'read binary world file' option and click that and load the .world file you want to eventually change. Select the structures from the complex you want to change and click 'write ms3d file' to create milkshape file of the structure. (if the structure exceeds milkshape limits you'll get error message in dos window - otherwise it'll write .ms3d to same folder as .world file)

when you open the .ms3d file you'll find the groups are named in a precise way eg:

the c10s1 in the image indicates its complex 10 structure 1. This element is not required for the write back of the file - its just there to help you to identify components if you start merging files.

the Eoxx part shows the existing object number within the complex not the whole .world file.... it's done like that to help track changes when you re-imput something, there's some unfortunate peculiarities about the way CA numbered objects that means they can be arranged over several structures within a complex not necessarily consecutively.... if you want to keep existing attributes of an object, e.g. if it has pathfinding entry, it thinks its a tower... it has little flame effect that comes up when its damaged etc... then you need to keep the object number the same when you put .ms3d back into .world file -- otherwise the editor tool will delete the old object and add it back as new one without any attributes or collision volume (most attributes can be added back via text editing but its not an easy process, so please don't accidentally delete things!)

the g_0 , g_1 section shows the group number within the object. Typical objects have two groups 0 and 1; with 0 being the undamaged version, and 1 the version that shows up if the object is damaged, some objects only have one group (can't be damaged) and some have more, eg, gates and walls which have more transition states.

If you want to add new objects you should add them after the end of the objects you want to retain like:


that's retaining objects 7, 10 and 12 from that complex and adding a new single group object and a new paired group object. The naming convention for new objects is
'new object number' 'any symbol as spacer' 'new group number'
so the
2-0
2-1
groups are going to form one new object with two groups...

EDIT: IWTE reads group numbering like this
fred230gate-x10
will be read as 'something that isn't Eo so isn't an existing object' 'two hundred and thirty' 'some text/symbols to match' 'ten'
if next group name says
blah230gate-x11
then both groups should end up as damage stages within the same object
NOTE: because IWTE needs to be able to read/understand and cross match any numbers within that first section of the group name they need to be readable as 2 byte integers i.e. less than 65535..... Do not use very long unbroken numbers like 100000000 as it will attempt to read that as Xmillion and crash whilst writing your file.

Don't add new groups until the end of the existing objects, editor will treat anything after first new group as also 'new'

You need to decide if you want objects to go back in as new objects or as existing objects and name them appropriately. You can change shape of something but still call it an existing object if you want to retain its attributes (e.g. game recognises it as a gate or arrow tower, or if it has light / flame effect you want to keep). Obviously do the opposite if you want to get rid of attributes and put object back with 'new' name so its accepted as new object, and old one with its attributes is deleted.

(beware of deleting all objects before a gap in object numbers, CA had some inspired object numbering sequences that don't run through structures consecutively, deleting objects that cause structures to try and switch order, without deleting the whole structure still causes problems..)


Deleting Structures
You can delete most structures with this tool - with one condition: you can't delete the last structure in a complex (the complex numbers need to remain the same) - if you want to entirely 'lose' a complex then just replace the last structure with a 1 triangle 1 group structure with the triangle hidden somewhere under ground level.

Don't delete the last gate in a wall perimeter area - at least without further amendments to perimeter / .worldpathfinding.... game will decide siege battle is either already won, or un-winnable

Text Files
The tool automatically generates a number of .txt files into the same folder the .world file is written to when it reads files or makes changes, these are primarily for de-bugging purposes, and can be safely deleted once you know the process has succeeded. For large .world files the text files are also very large - so please do delete un-needed .txt files regularly (or you'll run out of disc space!)


The tool also has the function to deliberately write text versions of the binaries for the .world file and the .worldcollision files - buttons for this are on the 'world' tab. This is similar to the 'textrep' function in KnightErrant's original editors. These files can be amended in notepad or similar and converted back to binary if you need to do any manual editing of areas not yet automated by the tool.


Revised File Names
When you perform an operation with the tool it will ask you to supply a file name for the revised files to be saved with. For operations which can change the number of objects (like replacing .ms3d's for structures) it will save new versions of the .world file, the .worldcollsion file and the .animinstances file,
all with the same prefix name you specified. It is recommended that you DO NOT attempt to overwrite the file that you're amending, but supply a new file name. If you undertake repeated operations in the 'structure' tab, the tool will produce a set of binary files for each operation, with a sequence number added to your selected file name. e.g. if you selected file name 'fred' and perform 3 operations you'll end up with files;
fred.world, fred.worldcollision, fred.animinstances
fred_1.world, fred_1.worldcollision, fred_1.animinstances
fred_2.world, fred_2.worldcollision, fred_2.animinstances

To test files in game rename the last set of .world, .worldcollision and .animinstances files produced (fred_2 in example) to the required name.

Intermediate files generated by this process (fred & fred_1 in example) can be safely deleted when you know that the final file produced works in game.


write up of instructions is very WIP, some more will come later but this version of tool is unlikely to be fully documented till its reached more final version....

[wilddog]

Decided to add the bug fixes and enhancements to the first post for ease of updating.

[Devils_Advocate]

Hi
In order not to cloud too may other threads I thought it may be advisable for people to post questions/issues with the tool here.

Okay, then I am just popping in.
I am slowly loosing any last bit of overview about that whole stuff. Apart of the general fact I am not understanding the major things you are talking about, and I wish I would, I am wondering as trivial as it sounds what "useable" thing can an average modder now really accomplish with that tool; can I now create costum cities (more or less properly); whatever? Or are you just teasing me/us by what looks like a progress into that direction?

I had a look a that tool though. Unfortunately I have not the time nor patient to experiment with it but from KEs tools I understand some of it functions. What really makes me stunned is that we can now remove structures, right. That is great!

To the point, your recent tuts Wilddog and Makayane as great as they look I believe it is mostly just stuff for the "inner-party", i.e. a layman like me is desperatly lost. To see the instruction you give executed on an example where those tools are practically used with a playable result would imo be a better tut.

[SigniferOne]

What?!

Major rep!

[makanyane]

@ DA
we've tried to make the point that this is a beta level release and not suited to people who haven't already got a good understanding of the .world type files.

It's still under development and some functions which you'd currently have to do a lot of in depth text editing to fix, might shortly become automated, so I don't want to waste a lot of time writing them up if they're likely to be out of date by the time I've finished.... For instance the collision tutorial http://www.twcenter.net/forums/showthread.php?t=310750
just went from a lot of very technical info (that's now in spoilers) to basic instruction for adding collision being to press the 'Add Object Collision' button.



If all that was holding people back from editing before was not being able to change number of objects / structures etc because of the sequence number problems in the world file, then this should fix it for them.


A basic thing you could do at the moment is to replace some of the 'decorative' structures, like the blocks of houses in the settlements. That's just a matter of identifying where they are (which existing structures) and replacing the .ms3d's. When you've replaced the .ms3d you'd need to go through and use the 'add object collision' button for each of the objects in your new structure... (so its best if you can do it with fewer objects)

If you then need to change the pathfinding at ground level you just do it by creating and replacing the pathfinding TGA2 (see the terrain tab)


what's more complicated at the moment is if you want to change or add walls / towers or gates in shape or position and get on wall pathfinding, arrow firing and gate opening to work.... or if you want to add / change the flame & smoke effects etc... that's doable but at the moment would involve the sort of tutorial you're not going to like

[Devils_Advocate]

that's doable but at the moment would involve the sort of tutorial you're not going to like

I agree.
I guess it´s better to go back on 'lurker' position again.

[SigniferOne]

If all that was holding people back from editing before was not being able to change number of objects / structures etc because of the sequence number problems in the world file, then this should fix it for them.

Just to confirm: you have modified KE's original scripts to make them update the number of structures etc? If so that was the major thing holding settlement editing back!

[wilddog]

Just to confirm: you have modified KE's original scripts to make them update the number of structures etc? If so that was the major thing holding settlement editing back!

No not quite. The original scripts were the basis and they have been significantly updated. New scripts have been added to handle bounding spheres and animinstances file also.

The updated scripts had :-
- the sequence number automated in .world and .worldcollisions (yes there was a simple one there also).
- it had most of the pads removed (these are extra 2 byte values added on first time usage of something) from the text editing. To avoid people tripping up on those.
- it recalculates all the object numbers and removes them or adds them as blank entries to .world, worldcollision and animinstances.
- it handles assignment of textures and recalculates new sequence numbers for those.
- it handles the recalculation of the header values which go with the sequence number (The issue is the last usage of those in the 6th float tables). Believe me this was a far bigger issue than a resequencing item and there are some cases which may cause it to fallover).
- it then was developed to handle basic adding and deleting full structures (avoids the max milkshape issue).
- adding collision entries to the collision tables.

These are basic simplifications only and aids to text editing. The real work is adding in the pathfinding bit around getting onto walls etc. so there is a lot more to do.

[Argantonio]

Impressive....

I can not open IWTE_1v2_beta .
A dos window appear but It closes immediately, I can not read that indicates the screen.
I opened IWTE_1_0 with no problems with phyton
What happened? What i`m doing wrong?

[wilddog]

Impressive....

I can not open IWTE_1v2_beta .
A dos window appear but It closes immediately, I can not read that indicates the screen.
I opened IWTE_1_0 with no problems with phyton
What happened? What i`m doing wrong?

Hi it should be a straight .exe which should open. We thought that was easiest way whilst in beta form rather than getting into debates on Python loads (I had enough problem with that).
You could open via a bat file and call the .exe and then add cmd /k. That should keep the command window open and allow you to see the error message
On the other hand what are you running on? We built the .exe for windows with XP so it may be compatibility issue.

[Argantonio]

Thank Solved ...

The error was fixed
Error loading Python DLL : c:\...\_MEI39842\python26.dll ( error code 14001)

We must install Python 2.6 , IWTE_1v2_beta.exe works on Xp -vista.

KE tools works on Python 2.5 , its posible some kind of interference ...

[makanyane]

Thank Solved ...

The error was fixed
Error loading Python DLL : c:\...\_MEI39842\python26.dll ( error code 14001)

We must install Python 2.6 , IWTE_1v2_beta.exe works on Xp -vista.

KE tools works on Python 2.5 , its posible some kind of interference ...

aarghh that's annoying, the point of the .exe is it's meant to have included all the .dll's etc inside it and run independently

it works on my XP computer that is still using Python 2.5 - and I tested a slightly earlier version of the .exe with Python completely un-installed - didn't try that on this one though as it's time consuming to take python on and off.. "shoot the tester!"

Wilddog is using 2.6 now, and we noticed that KE had recently released an animation tool that needed 3.0 - that was part of reason for sending as an .exe - it's meant to cope with no / any python!!

thanks for letting us know what answer was anyway....

[Augustus Lucifer]

There's another solution which allows you to package critical python components with the tool. It's called cx_Freeze, and it works for Python 2.5/2.6/3.0. I tested it some time ago when I wrote a really simple EDU file swapper by sending it to a teammate who didn't have the python version(3.x) installed, and they were able to run it just fine. It isn't as 'clean' as just an EXE, because it comes with a library and a DLL, but it does the job and you still click the EXE to start it. And it's not like the distro is for players, I doubt any modder who will take on the tool is going to be squeamish about a few files lying around.

The tool worked for me, but then I have 2.5 and 2.6 and 3.1 installed.

[makanyane]

Thanks for the idea, we've so far tried with py2exe, pyinstaller (which I think was responsible for exe on download), bbfreeze - which came out at 100mb plus, and we also had short attempt with cx_freeze which also looked like it was coming up on the somewhat large side.... but we might have been missing some adjustments that can be done to refine output.

Major problem with all of them at the moment is we can't get any of them to compile to include mayavi2 functions, apparently because its packaged as egg with path under enthought. - (there's some advice on web for that.... our egg's aren't zipped and we've tried the revised __init files, but aren't getting anywhere) , Wilddog would like to re-include mayavi2 viewing functions, so that is probably going to be deciding factor on how next version goes out.

If you know anything about that aspect any advice would be much appreciated.

[Augustus Lucifer]

Hm, I've not worked with Mayavi or the Enthought modules. I can't find much on the distribution of those packages, the only thing I found was this page which makes a small note of 64-bit vs. 32-bit Python. Problem is the nature of programs like cx_Freeze should be that if they work on your computer and the pyc files import properly it should package them into a library properly. Why that wouldn't be the case for enthought packages I'm not sure, unless it has to do with PATH variables on your system which enable you to personally run the files but cx_Freeze doesn't key on them to find other package locations. HMMMMMMM.

[wilddog]

Hm, I've not worked with Mayavi or the Enthought modules. I can't find much on the distribution of those packages, the only thing I found was this page which makes a small note of 64-bit vs. 32-bit Python. Problem is the nature of programs like cx_Freeze should be that if they work on your computer and the pyc files import properly it should package them into a library properly. Why that wouldn't be the case for enthought packages I'm not sure, unless it has to do with PATH variables on your system which enable you to personally run the files but cx_Freeze doesn't key on them to find other package locations. HMMMMMMM.

Hi, I haven't tried the CX Freeze package but after trying now 3 types of packages I got one to work - almost (BBfreeze). That created a 118MB file and still didn't accept the commands to correctly position Mayavi against PYQT4 to allow the windows to shut. Its probably something I can look at on paths but its taking longer to do that than some of the added functionality I'd rather spend time on. Using straight py files its not a problem.

I originally started off thinking the .exe's would be easier but have since changed my mind

[Gorrrrrn]

Have the latest version of this and no Python my hard drive.

seems to work perfectly well.

now all I need is that tutorial on how to convert arrow towers into ballista towers on large castles and fortresses for my mod that uses them as preplaced "forts".

I've looked through the unpacked m2tw files didn't spot anything that told the fortresses where to put ballista (and cannon) towers when they are built.
Yet it does for the towers when they get upgraded either in descr_strat as part of a starting settlement or as a building upgrade in game.

[makanyane]

@Rosanov
the problem isn't the where but just the when....
if you look at the .world files including the fort ones, and find the structures that contain the towers, you'll see that the structure will have sets of groups for each stage 'arrow tower','balista tower', 'cannon tower'. Viewed in ms3d they're all in same place, and fit on same bottom section of tower.

The table that indexes object attributes in the .world file is table 2 (viewed after binary to text conversion) it looks like this:

for a tower with the three levels. Number at end of row is the object number - in illustration object number 20 is the arrow tower version, 21 = balista, 22 = canon. The only differences in that table between them are the 111, 79, 79 numbers in column 4. 79 always seems to be used for a tower which is an upgrade.... and in column 6 the 0 4 8 numbers. Those refer to the animstring it uses (but x4) that example means its using the first second and third string

for the example shown.... if you changed the 0 to 4, you'd get the balista tower animation.... but I think that relates only to what is shown as it collapses, not when it fires.

Those three stage towers have entry like

in the 'Game Objects' section - you can see the 20, 21 & 22 object numbers listed there at top of section.... you'll then get 'FlagBase' and series of 'ArrowSlot's which relate to each object. Unfortunately the entries for each object (ie each upgrade level) look exactly the same to me... ie. they're all ArrowSlots you don't get "BalistaSlot"s! So as far as I can tell nothing in that lot is telling it what projectile to use.


I think the answer has to be outside the .world files.... either .worldpkgdesc as you mentioned in other thread (I assume that didn't work?) or something that would be needed to be specified in descr_strat for the fort entry?

[Gorrrrrn]

yes I looked at the world file and could only find arrow towers.

I found the ballista and cannon tower debugdata files in the unpacked m2tw files.

not found anything else yet. I'm thinking maybe ballista and cannon towers just replace the top level of the tower structure?

Haven't tried the tower level approach yet.

Tx for taking the time to explain.