Guide to the ETW map

[deLancey]

So I'm guessing... you can't do something like in Rome Total War where you could make a campaign map from a smaller part of a larger map (ie. make an Aegean only map from the larger all-Europe map)?

[uanime5]

I've been conducting some more research in the campaign pathfinder. I noticed that in the startpos while the fort_obstacle entries contained coordinates the obstacle_base_grid_node entries did not. I initially though the lack of coordinates meant that they weren't related to anything on the map but I now think they use a different system to tell ETW where everything is located.

Below is an obstacle_base_grid_node entry.

  <rec type='OBSTACLE_BASE_GRID_NODE'>
   <u>1796</u>
   <u>3</u>
   <ary type='MANAGED_OBSTACLE_BOUNDARY'>
    <rec type='MANAGED_OBSTACLE_BOUNDARY'> 
     <u>2359</u>
     <u>1</u>
     <yes/>
     <u4_ary>580745538 1</u4_ary>
    </rec>
    <rec type='MANAGED_OBSTACLE_BOUNDARY'>
     <u>2547</u>
     <u>1</u>
     <yes/>
     <u4_ary>580702690 0 580745538 1</u4_ary>
    </rec>
    <rec type='MANAGED_OBSTACLE_BOUNDARY'>
     <u>58</u>
     <u>0</u>
     <yes/>
     <u4_ary>580702690 0</u4_ary>
    </rec>
   </ary>
   <ary type='MANAGED_OBSTACLE_BOUNDARY'>
    <rec type='MANAGED_OBSTACLE_BOUNDARY'>
     <yes/>
     <u4_ary>580745538 1</u4_ary>
    </rec>
    <rec type='MANAGED_OBSTACLE_BOUNDARY'>
     <yes/>
     <u4_ary>580702690 0</u4_ary>
    </rec>
    <rec type='MANAGED_OBSTACLE_BOUNDARY'>
     <yes/>
     <u4_ary>580702690 0 580745538 1</u4_ary>
    </rec>
   </ary>
  </rec>

I believe that rather than use coordinates the obstacle_base_grid_node entries and managed_obstacle_boundary entries use grids created by the pathfinding.esf to tell ETW where everything is. So 1796, 2359, 2547, and 58 refer to 4 grids in the Indian theatre. Also the 1 or 0 immediately after the managed_obstacle_boundary entry probably indicated whether there is something in this grid. I don't know what the 9 digit numbers represent but they are repeated a lot.

As I couldn't find an entry higher than 7886 this is probably the number of grids in India. As India is 240 x 260 pixels this means each grid is 2.7 x 2.9 pixels and has an area of 7.9 pixels^2.

UPDATE: the vertices in the pathfinfing.esf use large map coordinates that correspond to the theatre they're located in. So the first pathfinding_Areas/vertices contains large map coordinates for the Indonesia theatre, the second contains coordinates for the American theatre, etc. These coordinates don't correspond to any specific point on the map, such as settlements or towns, and are probably used to divide each theatre into various grids. As many of the coordinates are negative I believe that they're converted into another format for the campaign_pathfinding because the campaign_pathfinding cannot cope with negative numbers.

The large blocks of numbers in the startpos obstacle boundary aren't random, they follow a specific pattern. For example this code from the Indonesia threatre:

1 2400256257 4194304 30 0 1 2400256257 4194304 31 0 1 2400256257 4194304 32 0 2 2266038529 6291456 219217928 6291457 33 0 2 2147483905 6291458 2400256264 6291459 34 0 1 2400256264 4194304 35 0 2 2266038536 6291460 2 4290773004 36 0 5 2147483912 6291461 2 4290773016 68157441 6291462 220266504 6291463 16842760 6291464 37 0 2 1 6291465 2400257544 6291466 38 0 1 2400256264 4194304 39 0 1 2400256264 6291467 40 0 2 2383413512 6291468 16842753 6291469 41 0 1 2366701825 6291470 42 0 1 2400256257 4194304 43 0 1 2400256257 4194304 44 0 1 2400256257 4194304 45 0 1 4279304465 4194304 65566 0 1 4279304465 4194304 65567 0 1 4279304465 4194304 65568 0 2 3054502161 6291471 1829830680 6291472 65569 0 1 4279304472 4194304 65570 0 1 4279304472 4194304 65571 0 3 3054502168 4194333 2 4290773029 16842752 50 65572 0 5 2 4292870161 2164392448 63 2 4290773062 536870929 6291474 1627455528 6291475 65573 0 4 4043375128 4194384 2 4290773079 2216689920 96 2 4290773095 65574 0 2 4245750040 4194412 2 4290773106 65575 0 1 4279304472 4194304 65576 0 1 3205562648 4194304 65577 0 2 2517631240 6291476 1829830673 6291477 65578 0 1 4279304465 4194304 65579 0 1 4279304465 4194304 65580 0 1 4279304465 4194304 65581 0 1 4279304465 4194304 131102 0 1 4279304465 4194304 131103 0 1 4279304465 4194304 131104

Means this

1 
2400256257 4194304 
30 0 
1 
2400256257 4194304 
31 0 
1 
2400256257 4194304 
32 0 
2 
2266038529 6291456 
219217928 6291457 
33 0 
2 
2147483905 6291458 
2400256264 6291459 
34 0 
1 
2400256264 4194304 
35 0 
2 
2266038536 6291460 
2 4290773004 
36 0 
5 
2147483912 6291461 
2 4290773016 
68157441 6291462 
220266504 6291463 
16842760 6291464 
37 0 
2 
1 6291465 
2400257544 6291466 
38 0 
1 
2400256264 4194304 
39 0 
1 
2400256264 6291467 
40 0 
2 
2383413512 6291468 
16842753 6291469 
41 0 
1 
2366701825 6291470 
42 0 
1 
2400256257 4194304 
43 0 
1 
2400256257 4194304 
44 0 
1 
2400256257 4194304 
45 0 
1 
4279304465 4194304 
65566 0 
1 
4279304465 4194304 
65567 0 
1 
4279304465 4194304 
65568 0 
2 
3054502161 6291471 
1829830680 6291472 
65569 0 
1 
4279304472 4194304 
65570 0 
1 
4279304472 4194304 
65571 0 
3 
3054502168 4194333 
2 4290773029 
16842752 50 
65572 0 
5 
2 4292870161 
2164392448 63 
2 4290773062 
536870929 6291474 
1627455528 6291475 
65573 0 
4 
4043375128 4194384 
2 4290773079 
2216689920 96 
2 4290773095 
65574 0 
2 
4245750040 4194412 
2 4290773106 
65575 0 
1 
4279304472 4194304 
65576 0 
1 
3205562648 4194304 
65577 0 
2 
2517631240 6291476 
1829830673 6291477 
65578 0 
1 
4279304465 4194304 
65579 0 
1 
4279304465 4194304 
65580 0 
1 
4279304465 4194304 
65581 0 
1 
4279304465 4194304 
131102 0 
1 
4279304465 4194304 
131103 0 
1 
4279304465 4194304 
131104

I'm unsure what these numbers mean but '30 0' seems to be the entry number, '1' is the number of entries, and '2400256257 419430' is an entry. 2400256257 appears several times in the pathfinding.esf/pathfinding_areas/grid_data but 4194304 doesn't. So far I've only found 4194304 as a negative number in the pathfinding_areas/vertices.

[Peteriscoolz]

Messing around with heightmaps.

[ForteS]

There are some interesting files about the maps in testdata.pack does somebody know what they do?

Spoiler Alert, click show to read: 

here's one:



and here is another, this one is hard to see and is a model but i think i can see india and america on it:



more one:

I know I tried looking for the border files in the model.pack and ui.pack but couldn't find them. The ui.pack was quite fascinating and I saw some interesting things, such as a Buddhism pip (which indicates that Buddhism was meant to be a religion in ETW) and some early maps which contain more regions.

Spoiler Alert, click show to read: 

Those maps explain the extra factions and the indian one matches the one that I've already uploaded where we can see a extra region in the south.

Spoiler Alert, click show to read: 

Those guys from CA could have made a much better game than this one that we have

Edit: what does the 'theatre_map.luac' do?

[Pikemen]

and here is another, this one is hard to see and is a model but i think i can see india and america on it:

This looks like resource/City/town placements as far as I can tell... Looks a little bit like RTW/MTW2 way of marking the cities on a 2-D map, but in this case it is only the markers.

Just a crazy though, but has anyone tried placing these maps on top of each other in layers through photoshop?

And a even crazier thought.... What if the developers made the map that way? Simulare to how RTW/M2TW was done, but packed together into these Pack.files and divided up for each different element (Sea Borders, Town Placements, Height)? The game then recognizes them and recombines them with the ESF files giving values to the marked region.

[Robertclive]

Looks like CA half did the map because this was the originally planned map:

[Primergy]

And a even crazier thought.... What if the developers made the map that way? Simulare to how RTW/M2TW was done, but packed together into these Pack.files and divided up for each different element (Sea Borders, Town Placements, Height)? The game then recognizes them and recombines them with the ESF files giving values to the marked region.

Of course CA is using a more intuitive way of placing towns and borders then typing those coordinates in the esf files.
But before creating a tool you have to know where the esf stores the information and how.

But it's obvious that for useful map creating in empire you will need a 3D software mabye also some GIS data, the times were gimp was enough have ended.

WarriorOne talked about the world_.stpd and world_stpi in this thread:
http://www.twcenter.net/forums/showt...56#post4718356

It's possible that you may be able to open or edit these files with Autodesk 3ds Max (they have a free trial). Unsure how useful these files are but I do know that ETW won't load the campaign without the supertexture.pack.

I own 3ds Max and it can't open stpd files.
I also couldn't open it with a model converter which could open stp files, so there is a difference, too.

But the post below, with the vector data etc, is interesting since this also fits with Taws experience the region.esf is full with vector data.
So many things are pointing to the direction that CA is using GIS data for the map.

[ForteS]

Strange... there are two models called of 'world' and 'mountains' in testdata.pack, they are rigid_mesh files however 'ETW Model Converter' can't open them... it opens 'towns' and 'forest_boundary_marker' but why not those other two?...are they hard coded?

Edit: it also doesn't open 'westerneuborders.RIGID_MESH', 'victory_side_material.RIGID_MESH' and 'test.RIGID_MESH', in fact there are much more files in there that we can't open, files that might be important... and I'm not only talking about RIGID_MESH files, that pack seem to have lots of information.

Also I've notified that towns.RIGID_MESH makes reference of all existing slots even the ones that are in cities and I still believe that the reason of the black areas is in supertextures.pack so i think that need to find a way to open those files.

Edit2: About the black areas i still thinking that the black is the texture of those areas that would explain why we can see snow if we change the height map. Those textures are much probably in supertextures.pack thought this is just my opinion.

obs: just like Primergy I've tried to open the two files in supertextures.pack with a stp editor but it didn't work.

[Peteriscoolz]

Some more Pictures on the heightmap.

Spoiler Alert, click show to read: 

Here's what I did to the heightmap (colour picker.Paint.Net)



And the results ingame :










Excuse the size of the pictures.

[Magefsx]

Does that make terrain impassable, or not?

[uanime5]

After learning that each NTW campaign has its own stpd and stpdi files I decided to swap these files round and see what happened. Below are the results.

Italy campaign with the Egypt campaign's stpd and stpdi files

The stpd and stpdi files affect where the sea is on the map. But not the trees, rivers, or settlements. It doesn't seems to control the fog of war.

Spoiler Alert, click show to read: 

Though this is only cosmetic as characters can walk through any area not defined as seas in the regions.esf.

Spoiler Alert, click show to read: 

Also controls where the borders are located.

Spoiler Alert, click show to read: 

Italy campaign with the Europe campaign's stpd and stpdi files

Inside the campaign

Spoiler Alert, click show to read: 

Outside the campaign

Spoiler Alert, click show to read: 

What this means for mapping

If you can't make new stpd and stpdi files you can't make a new map because you won't be able to control where the sea is located. You'll also can't go beyond the vanilla campaign borders.

[Peteriscoolz]

Does that make terrain impassable, or not?

Yes,it's all passable and is no obstacle to the Ai.

[ForteS]

I don't have the word 'impossible' in my dictionary but with taw saying that much probably we'll never be able to edit those files, i think we can say bye bye to China, Brasil, etc

It has to be a way to edit those files, CA says that E:TW is easy to mod so we can not surrender.

[Primergy]

Well, those regions are already implemented (only hidden by the black), mabye they are also fully implemented regarding coastlines, seas, textures etc (what those stpd/stpid files seam to control) so a China mod till Christmas might be still possible.

Maps which might be impossible would be Maps for "Zoom Mods" (similar to the maps of NTW, enlarged versions of an area) or Total Conversions with fictional maps eg middle earth ones.

[ForteS]

Well, those regions are already implemented (only hidden by the black), mabye they are also fully implemented regarding coastlines, seas, textures etc (what those stpd/stpid files seam to control) so a China mod till Christmas might be still possible.

Maps which might be impossible would be Maps for "Zoom Mods" (similar to the maps of NTW, enlarged versions of an area) or Total Conversions with fictional maps eg middle earth ones.

Well, maybe ( and i hope so) you are right, but for that we still need to change the black areas, that might be in those stp* files right? By the way you mean next Christmas right?

Edit: imagining that the black areas aren't in the stp* files so where else can they be? are they so hard coded that we still unable to found them?

[husserlTW]

If by modding CA meant units retexturing , then yes... it is easy to mod ETW.

Btw I was thinking that perhaps the map modding tools we are looking for are the type of creating xml which then can be finally converted in game's file like esf with taw's tool. Xml could be fed with data by a program like QGis or such. That means that maybe we think backwards...

[ForteS]

If by modding CA meant units retexturing , then yes... it is easy to mod ETW.

Btw I was thinking that perhaps the map modding tools we are looking for are the type of creating xml which then can be finally converted in game's file like esf with taw's tool. Xml could be fed with data by a program like QGis or such. That means that maybe we think backwards...

Couldn't agree more.

Edit:Just a idea as somebody tried to ask some help about the campaign map to the CA guys? well i don't believe them to ever help us, but i think that we should give it a try anyway.

[uanime5]

Well, those regions are already implemented (only hidden by the black), mabye they are also fully implemented regarding coastlines, seas, textures etc (what those stpd/stpid files seam to control) so a China mod till Christmas might be still possible.

If these areas are available they'll probably be like Arabia or the Red Sea. Though they'll be separated into land regions and seas they won't have any textures or trees (we'll have to add these).

Maps which might be impossible would be Maps for "Zoom Mods" (similar to the maps of NTW, enlarged versions of an area) or Total Conversions with fictional maps eg middle earth ones.

If we can't edit the stpd and stpi files we can't change where the sea is. So we're limited to Europe shaped maps. Fortunately changing the stpd and stpi files in NTW didn't prevent units from leaving settlements so it should be possible to add, remove, and move settlements without remaking these files.

Well, maybe ( and i hope so) you are right, but for that we still need to change the black areas, that might be in those stp* files right? By the way you mean next Christmas right?

Edit: imagining that the black areas aren't in the stp* files so where else can they be? are they so hard coded that we still unable to found them?

The black area are in the stp* files. When I used the Egypt campaign's stp* files in the Europe campaign most of the map was black.

Couldn't agree more.

Edit:Just a idea as somebody tried to ask some help about the campaign map to the CA guys? well i don't believe them to ever help us, but i think that we should give it a try anyway.

I think you're right Fortes. Lets create a list of everything we know, send it to the CA staff, and ask them if they can give us any help figuring out what we need to do.

I'll start on the list in the next post, feel free to contribute. I don't know what the 'CAI_world_units' does; is this units roaming around the map or does it represent each type of unit?

[uanime5]

What we currently know

Poi.esf

Spoiler Alert, click show to read: 

Contains a list of:

• The distance from various points on the map to nearby settlements.
• Which settlement is closest.

Regions.esf

Spoiler Alert, click show to read: 

Uses a map of the world that is 2560 by 1280, going from -1280 by -640 to 1280 by 640.

Theatre and region keys

Determines:

• The size of each theatre.
• Whether it's trade theatre or not
• The regions in this theatre.

Region data

Determines:

• Where the regions are located.
• Whether it's a land or sea region.
• Whether this region is explorable (land / sea units can't go to unexplorable regions).
• How many areas it contains.
• What's in the region (settlements/towns/ports/resources).
• Where everything in the region is located (settlements/towns/ports/resources).

Sea_grids.esf

Spoiler Alert, click show to read: 

Contains a list of:

• Which regions connect to which sea.
• Which regions can trade with other regions using the sea.
• Location of ports (trade goes from one port to another port).

Startpos.esf

Spoiler Alert, click show to read: 

Uses a map of the world that is 2^20 times larger than the map used in the regions.esf.

World / Region Manager entries

These entries are only for playable land regions and determine the following:

• The population of the region
• What buildings are in this region, where they're located, whether they've emerged, who owns them, whether they're damaged, and their current level.
• Whether anything in this region is garrisoned.
• Hidden resources in this region (used for recruitment).

CAI interface/CAI world

All CAI entries contain BDI entries so different factions can determine how much they want each region. Each of these entries determine the following:

• CAI_world_theatre: theatres.
• CAI_world_factions: factions.
• CAI_world_regions: all regions (including the unexplorable ones), their locations, which faction controls them, and what they contain.
• CAI_region_boundaries: the length of the border neighbouring regions share.
• CAI_world_settlements: locations of settlements (capital of region), which theatre they're in, which region they're in, what area they're in, which faction owns them, whether they're garrisoned, and their building_slots.
• CAI_world_building_slots: number identifying which building in the Region Manager they are, and the CAI_world_settlements or CAI_world_region_slots they're paired with.
• CAI_world_region_slots: locations of region_slots (anything in a region that isn't the settlement), which theatre they're in, which region they're in, what area they're in, which faction owns them, whether they're garrisoned, and their building_slots.
• CAI_world_trading_posts: where the trade resources are located. What they trade is controlled by the campaign_trade_manager/theatres.
• CAI_world_resource_mobiles: locations of ships and armies, which theatre they're in, which region they're in, what area they're in, and which faction owns them.
• CAI_world_units: tell ETW which resource_mobiles corresponds to which character.
• CAI_world_character: which faction control this character.
• CAI_world_forts: locations of forts, which theatre they're in, which region they're in, what area they're in, which faction owns them, and whether they're garrisoned.
• CAI_world_technology_trees: the technology each factions possesses.
• CAI_world_governorship: how many governors each factions possesses (they need 1 per theatre).
• CAI_world_transition_areas: where the transition areas are located.
• CAI_world_HLCIS: locations of areas, which theatre they're in, which region they're in.
• CAI_world_sea_grid_cells: whether a sea grid cell contains a ship.

Trade_routes.esf

Spoiler Alert, click show to read: 

Contains a list of:

• Which regions can trade with which regions.
• Distance between two settlements.

stp* files (supertexture.pack)

Spoiler Alert, click show to read: 

Determines:

• Where all the sea textures are located.
• Where the campaign borders are located.
• Where the black areas outside the campaign borders are located.

Moving a settlement/town/port/resource within a region

Spoiler Alert, click show to read: 

Guide

• In the regions.esf/Region_data and locate the region you want to edit.
• Open the region, such as Cherokee_Territory; and find what you want to move, such as 'corn:cherokee_territory:north'.
• Change the two sets of coordinates for the resource's location, in this example -633.9398 ; 257.4009.
• Change the two sets of coordinates for the resource's selection area, in this example [-634.2933 ; 258.3908 ; -633.5862 ; 258.3908 ; -632.9498 ; 257.7544 ; -632.9498 ; 257.0473 ; -633.5862 ; 256.4109 ; -634.2933 ; 256.4109 ; -634.9297 ; 257.0473 ; -634.9297 ; 257.7544 ].

• In the 'startpos.esf/campaign_env/campaign_model/world' locate the region you want to edit.
• Open the region, such as Cherokee_Territory; and find what you want to move, such as 'corn:cherokee_territory:north'.
• Change the sets of coordinates for the resource's location, in this example -664734016 and 269904384.

• In the 'startpos.esf/campaign_env/campaign_model/cai_interface/cai_world' locate the region_slot you want to edit (in this example 3295).
• Change the sets of coordinates for the resource's location, in this example -664734016 and 269904384.

Start up ETW and the resource is now in a new location.


Current problem

Units cannot leave the moved settlement/town/port/resource. If the settlement/town/port/resource is covered by the fog of war armies will automatically teleport to it, rather than walk to it. I feel that I have not edited something but I am unable to determine where this file is located.

Adding a new town/port/resource to a region

Spoiler Alert, click show to read: 

Guide

• In the regions.esf/Region_data and locate the region you want to edit.
• Open the region, such as Cherokee_Territory, and clone an existing entry (such as 'corn:cherokee_territory:north').
• Give the entry a new name, such as 'corn:cherokee_territory:natives_only'.
• Give it two sets of coordinates for the resource's location, for example -612.2235 ; 244.3816.
• Change the two sets of coordinates for the resource's selection area, in this example [-612.5771 ; 245.3715 ; -611.8699 ; 245.3715 ; -611.2336 ; 244.7351 ; -611.2336 ; 244.028 ; -611.8699 ; 243.3916 ; -612.5771 ; 243.3916 ; -613.2134 ; 244.028 ; -613.2134 ; 244.7351 ].
• Keep everything else the same.

• In the 'startpos.esf/campaign_env/campaign_model/world' locate the region you want to edit.
• Open the region, such as Cherokee_Territory, and clone an existing entry (such as 'corn:cherokee_territory:north').
• Give it a new number, such as 587028431.
• Give the entry a new name, such as 'corn:cherokee_territory:natives_only'.
• Give it new coordinates for the new resource's location, for example -641962880 and 256252672.
• Keep everything else the same.

• In the 'startpos.esf/campaign_env/campaign_model/cai_interface/cai_world' open the CAI_world_region of the region you added the new resource to (in this example 'cherokee_territory').
• Add your new region_slot to the list of region_slots in this region.

• In the 'startpos.esf/campaign_env/campaign_model/cai_interface/cai_world' clone a building_slot entry (3294 is the vanilla building_slot for 'corn:cherokee_territory:north').
• Give it a new ID, such as 93595.
• Give it a new region_slot ID such as 93596.
• Delete all the BDI pool entries.
• Keep everything else the same.

• In the 'startpos.esf/campaign_env/campaign_model/cai_interface/cai_world' clone a region_slot entry (3295 is the region_slot for 'corn:cherokee_territory:north').
• Give it a new ID, such as 93596.
• Change the sets of coordinates for the resource's location, in this example -641962880 and 256252672.
• Give it a new building_slot ID such as 93595.
• Delete all the BDI pool entries.
• Keep everything else the same.

• In the main_pack/db/campaign_map_slots clone an existing entry, such as 'corn:cherokee_territory:north' and rename it 'corn:cherokee_territory:natives_only'.
• Keep everything else the same.

• In the In the local_en.pack/text/localisation.loc clone an existing entry, such as 'campaign_map_slots_onscreen_corn:cherokee_territory:north' and rename it 'campaign_map_slots_onscreen_corn:cherokee_territory:natives_only'.
• Change the onscreen name, in this example Atoka Farmland, to anything you want.
• Keep everything else the same.

Start up ETW and the you now have a new resource.

Current problem

Units cannot leave the moved settlement/town/port/resource. If the settlement/town/port/resource is covered by the fog of war armies will automatically teleport to it, rather than walk to it. I feel that I have not edited something but I am unable to determine where this file is located.

[ForteS]

I'll start on the list in the next post, feel free to contribute.

1. The height map determines the height of the land.

This files are related to the map but i don't know what is their effect in game:

1. coastline.rigid.mesh
2. towns.rigid_mesh -> makes reference to all slots in the map
3. forest_boundary_marker.rigid_mesh
4. land.rigid_mesh

There are more but the ones above are the only ones that I've been able to open.

main.pack

1. campaign_map_playable_areas_tables -> list of all theaters
2. campaign_map_settlements_tables -> list of all settlements
3. campaign_map_towns_and_ports_tables -> list of all ports/towns
4. campaign_map_slots_tables -> list of slots (corn, timber, silver, etc), region where they belong
5. regions_continents_tables -> list of continents
6. regions_tables -> list of regions, and to which continent they belong
   

----
note:

Having a look at models.pack I've found the gold things that surround the theaters transitions areas:
models.pack\db\campaignprops\campaignprops\campaignprops


...updating...