# Thread: Guide to the ETW map

1. ## Guide to the ETW map

I've been trying to figure out how the ETW maps works but I'm finding it difficult to progress further, so I decided to post everything I've found out to see if someone else can figure out the rest of this.

Everything I've found is explained in more detail here:
http://www.twcenter.net/forums/showt...272139&page=19

Below is a summary.

In the regions.esf I noticed that many of the coordinates showing where various regions were located on the ETW's map contained negative coordinates. As ETW cannot have regions that are not on the map I realised that the zero point must be in the centre of the map, and that the negative coordinates represented left or down from the centre, while the positive coordinates represented right or up from the centre. This is demonstrated on the map below (made from the campaign_detail_mask.dds).

While I don't know why CA choose to use this method I believe that it may be related to latitude and longitude, as their respective zero points are near to ETW's zero point.

Thought the exact area of the map is unknown as the Pacific Ocean region covers an area 2560 by 1131, it is highly likely that the map is 2560 by 1280. To test whether this was correct I enlarged the 2048 x 1024 campaign_detail_mask.dds to 2560 by 1280, resized the lookup.tga files to be the size of the theatres as listed in the regions.esf, and put on them on the enlarged campaign_detail_mask.dds at the coordinates listed in the regions.esf. As shown on the map below shows these theatres fit these coordinates perfectly.

Each region is made up of one of more areas, which are used to tell ETW which islands belongs to which region and when two regions overlap they tell ETW which cities belong to which region. After adding several areas to the above map I found that the areas had a large degree of overlap, which would prevent them having the precise borders that can be observed in ETW. So there must be another entry which tells ETW what shape the borders should be.

The shape of the borders may be related the connectivity in the regions.esf or the 'land.rigid_mesh'.

While checking the regions mentioned in the regions.esf I wondered whether CA had made a world map but had hidden some of the unplayable regions. To test this I edited the coordinates of the India theatre so that I could travel beyond the wooden border. I found that to the right of the Indonesia region there was a Chinese Empire region and a South Chinese Sea. This means that ETW's map contains may additional regions that are not visible without modding.

UPDATE: 9th May 2010

I have done more research on the startpos.esf and the Poi.esf and I believe I know what the latter does. The full research is posted here:
http://www.twcenter.net/forums/showt...65#post7296965

The main findings were that some of the numbers in the poi.esf were coordinates for the locations of settlements, ports, towns, and forts for a 2684354560 by 1342177280 map. So the purpose of the Poi.esf is to tell ETW where everything is on the map and how far away it is. This is probably so the AI can make better decisions and won't wander around aimlessly.

UPDATE: 22nd May 2010

alpaca has posted the following:

Originally Posted by alpaca
Anyhow, the borders in the game are defined using rigid splines. Each region defines its own borders as a sequence of points, which are saved as .rigid_spline files and stored in models.pack/rigidmodels/campaignborders. The same is true for roads and rivers. Not sure about bridges, they use a different method.

The height map is in the main pack.
So this means that we'll need to make new .rigid_spline files to add new regions.

UPDATE: 7th November 2010

I've done further testing on how ETW determine where each settlement is located and what needs to be edited. In the regions.esf for each city, town, port, resource there are coordinates that tell ETW where the strat model is located on the map. There are also 8 coordinates that tell ETW the area of the interactive area (not visible using esfeditor 1.45). Below is an explanation of these coordinates.

Code:
Coordinates for Baghdad in Mesopotamia: 314.25 ; 238.9004

8 Coordinates for Baghdad's octagon: [313.8965 ; 239.8903 / 314.6035 ;  239.8903 / 315.24 ; 239.2539 / 315.24 ; 238.5468 / 314.6035 ; 237.9104 /  313.8965 ; 237.9104 / 313.26 ; 238.5468 / 313.26 ; 239.2539 ]

Difference between the original coordinates and each octagon entry:

-0.3535 ; +0.9899

+0.3535 ; +0.9899

+0.99 ; +0.3535

+0.99 ; -0.3536

+0.3535 ; -0.99

-0.3535 ; -0.99

-0.99 ; -0.3536

-0.99 ; +0.3535
There's also the CAI interactive which I believe tells the AI where everything is located. I'm still investigating whether this needs to be updated.

UPDATE: 31st December 2010

I've figured out what each of the CAI interface/CAI world entries are for.

All CAI entries contain BDI entries so different factions can determine how much they want each region. Each of these entries determine the following:

• CAI_world_theatre: theatres.
• CAI_world_factions: factions.
• CAI_world_regions: all regions (including the unexplorable ones), their locations, which faction controls them, and what they contain.
• CAI_region_boundaries: the length of the border neighbouring regions share.
• CAI_world_settlements: locations of settlements (capital of region), which theatre they're in, which region they're in, what area they're in, which faction owns them, whether they're garrisoned, and their building_slots.
• CAI_world_building_slots: number identifying which building in the Region Manager they are, and the CAI_world_settlements or CAI_world_region_slots they're paired with.
• CAI_world_region_slots: locations of region_slots (anything in a region that isn't the settlement), which theatre they're in, which region they're in, what area they're in, which faction owns them, whether they're garrisoned, and their building_slots.
• CAI_world_resource_mobiles: locations of ships and armies, which theatre they're in, which region they're in, what area they're in, and which faction owns them.
• CAI_world_units: tell ETW which resource_mobiles corresponds to which character.
• CAI_world_character: which faction control this character.
• CAI_world_forts: locations of forts, which theatre they're in, which region they're in, what area they're in, which faction owns them, and whether they're garrisoned.
• CAI_world_technology_trees: the technology each factions possesses.
• CAI_world_governorship: how many governors each factions possesses (they need 1 per theatre).
• CAI_world_transition_areas: where the transition areas are located.
• CAI_world_HLCIS: locations of areas, which theatre they're in, which region they're in.
• CAI_world_sea_grid_cells: whether a sea grid cell contains a ship.

http://www.twcenter.net/forums/showt...04#post8700104

If you delete the BDI_pool entries in the CAI_world entries ETW will make new BDI_pool entries. So if you want to add new towns/ports/resources or increase the number of buildings in a settlement you don't need to update the BDI entries, you can simply delete them.

The black barrier and borders are caused by the stpd and stpdi files in the supertexture.pack. This file also where the sea is located (purely cosmetic) but not the rivers, mountains, or trees. Unless these files can be edited it will be impossible to make new maps for ETW, though the existing one can still be edited if someone can figure out how to edit them.

http://www.twcenter.net/forums/showt...355187&page=15

UPDATE: 8th January 2011

Myself and husserlTW have been conducting more research on the following:

Function of the various map files
http://www.twcenter.net/forums/showt...46#post8707346

How the pathfinding.esf and the startpos.esf/campaign_pathfinder are connected
http://www.twcenter.net/forums/showt...64#post8738864

http://www.twcenter.net/forums/showt...00#post8757200

UPDATE: 17th January 2011

I'm a little late posting this but all the coordinates for the regions, region boundaries, and rivers are in the regions.esf. Taw was able to show this by using a converter to turn all the coordinates into an image.

http://www.twcenter.net/forums/showt...25#post8695825

UPDATE: 1st March 2011

I'm posting all the new findings to prevent them being lost.

How the boundaries are calculated in the regions.esf for land regions:
http://www.twcenter.net/forums/showt...74#post8958774
http://www.twcenter.net/forums/showt...87#post8961187

How the boundaries are calculated in the regions.esf for sea regions:
http://www.twcenter.net/forums/showt...81#post9043081

Various maps made by Taw using the region.esf information:
http://www.twcenter.net/forums/showt...49#post8994549

How transition areas work:
http://www.twcenter.net/forums/showt...76#post8998376

Information about the wind maps in the regions.esf:
http://www.twcenter.net/forums/showt...26#post9051126

Where the vertices entries in the pathfinding.esf are located on the campaign map:
http://www.twcenter.net/forums/showt...39#post8964239

http://www.twcenter.net/forums/showt...71#post9015971

Regarding mapping until we figure out how the pathfinding.esf works, especially the grid_data, we won't be able to add any new towns, ports, resources, settlements, or regions to the map. Though we can add more buildings to existing settlements.

UPDATE: 5th March 2011

An explanation of how the trade_routes.esf works:
http://www.twcenter.net/forums/showt...66#post9097166

UPDATE: 21st April 2011

Figured out what the <u2_ary> in the grid_data means:
http://www.twcenter.net/forums/showt...16#post9410016

and what the <u4_ary> entries in the pathfinding_areas mean:
http://www.twcenter.net/forums/showt...71#post9414371

UPDATE: 25th April 2011

Analysis of the grid_data in the pathfinding.esf
http://www.twcenter.net/forums/showt...10#post9438110

UPDATE: 24th June 2011

Ulsterman has found that you can use the pathfinding information of existing buildings for new buildings. This removes the problem of units being unable to leave new buildings but means that all units can only enter and exit the new buildings using the old building.
http://www.twcenter.net/forums/showt...65#post9819365

UPDATE: 1st July 2011
PietroMicca has investigated the pathfinding.esf and shown how it is used to create a map.
http://www.twcenter.net/forums/showt...20#post9853720

UPDATE: 4th October 2011
PietroMicca explains what the grid data means:
http://www.twcenter.net/forums/showt...3#post10366563

http://www.twcenter.net/forums/showt...7#post10406517

UPDATE: 26th December 2011
PietroMicca converts the hex data in the pathfinding.esf into 3 maps. There function is currently unknown.
http://www.twcenter.net/forums/showt...3#post10785573

PietroMicca explains the climate maps:
http://www.twcenter.net/forums/showt...4#post10785894

http://www.twcenter.net/forums/showt...0#post10786570

UPDATE: 31st December 2011
Uanime5's list of the files that effect climate:
http://www.twcenter.net/forums/showt...2#post10806482

UPDATE: 7th January 2012
PietroMicca made a version of the pathfinding.esf that allows land units to walk on land masses in East Asia.
http://www.twcenter.net/forums/showt...1#post10832511

UPDATE: 8th January 2012
PietroMicca explain how to calculate the coordinates for a theatre in the grid_data (pathfinding.esf).
http://www.twcenter.net/forums/showt...1#post10843851

UPDATE 10th January 2012
Uanime5 post a list of regions and the numbers which correspond to the u2 in the grid_data (pathfinding.esf).
http://www.twcenter.net/forums/showt...2#post10852242

UPDATE 13th January 2012
PietroMicca and Uanime5 made maps of the various theatres using the grid_data (pathfinding.esf).
http://www.twcenter.net/forums/showt...2#post10863872
http://www.twcenter.net/forums/showt...8#post10882598
http://www.twcenter.net/forums/showt...9#post10883819
http://www.twcenter.net/forums/showt...0#post10884040
http://www.twcenter.net/forums/showt...2#post10884382

UPDATE 16th January 2012
PietroMicca posts the pathfinding for Madeira.
http://www.twcenter.net/forums/showt...8#post10893398

UPDATE 23rd January 2012
Uanime5 posts a list of colours in the pathfinding.esf and the ground types they correspond to.
http://www.twcenter.net/forums/showt...9#post10925529

UPDATE 8th March 2012
Research on editing the grid_data in the pathfinding.esf.
http://www.twcenter.net/forums/showt...2#post11145062

UPDATE 12th March 2012
Taw has finished his Quad tree exporter and importer. It is now much easier to remove the fog of war around new units.
http://www.twcenter.net/forums/showt...1#post11157841

UPDATE 17th March 2012
Information about the lower <u> entries in the grid_data (pathfinding.esf).
http://www.twcenter.net/forums/showt...7#post11179867

UPDATE 18th March 2012
Taw and PietroMicca explain how the query_info work.
http://www.twcenter.net/forums/showt...3#post11181073

UPDATE 2nd April 2012
The grid_data now is now in an easier to read format.
http://www.twcenter.net/forums/showt...7#post11262117

UPDATE 5th April 2012
PietroMicca explains how to calculate the vertex_index.
http://www.twcenter.net/forums/showt...7#post11278047

UPDATE 9th April 2012
PietroMicca explains how the grid_cells determine the type of path.
http://www.twcenter.net/forums/showt...0#post11298960

UPDATE 22nd May 2012
uanime5 and PietroMicca post information about the 0-3 flags, passable_part, and unknown2.
http://www.twcenter.net/forums/showt...6#post11480236
http://www.twcenter.net/forums/showt...5#post11497065

UPDATE 22nd May 2012
uanime5 provides regular expressions for changing unknown2 values.
http://www.twcenter.net/forums/showt...5#post11519075

UPDATE 22nd May 2012
PietroMicca explains how the unknown2 effects whether you can travel between 2 boundaries
http://www.twcenter.net/forums/showt...5#post11527055

UPDATE 15th June 2012
PietroMicca explains the last 32 bits of information in the empty cells:
http://www.twcenter.net/forums/showt...3#post11590693

UPDATE 19th June 2012
uanime5 has successfully added a new corn resource to the Cherokee Territory:
http://www.twcenter.net/forums/showt...9#post11607739

UPDATE 19th September 2012
A link has been found between the grid_data in the pathfinding.esf and the campaign_pathfinding in the startpos.esf.
http://www.twcenter.net/forums/showt...0#post11978900
http://www.twcenter.net/forums/showt...2#post11994842
http://www.twcenter.net/forums/showt...8#post12018788

UPDATE 6th May 2013
I have used the official tools to move a farm in TWS2. I have posted all the changes between the vanilla version of the pathfinding and my modified version here:
http://www.twcenter.net/forums/showt...1#post12820027

Summary

1) ETW's full map 2684354560 by 1342177280. This map is -1342177280 to 1342177280 along the X axis (left to right) and -671088640 to 671088640 along the Y axis (bottom to top).

2) ETW's has a smaller map that's 2560 by 1280. This map is -1280 to 1280 along the X axis (left to right) and -640 to 640 along the Y axis (bottom to top).

3) All the coordinates on the smaller map are the full map's coordinates divided by 1048576 (2^20). The maximum length of a regions.esf coordinate is 7 digits (392.3914, 40.37837, etc). This means 1 pixel on the smaller map represent over 1 million pixels on the larger map.

4) ETW's map covers the whole world.

5) ETW's map is divided into 7 theatres, the 3 playable theatres are divided into several regions, each region is divided into 1 or more areas.

6) To prevent people seeing some unplayable regions CA has done the following:
6A) Restricted the the player so they can only move the cursor over a few parts of the world (the 7 theatres).
6B) Covered the unused regions with a black barrier to hide them. The black barrier and the theatre borders are created by the stpd and stpdi files in the supertexture.pack

7) Regions.esf determines where all the regions are; the region borders; whether a region is land or sea; which theatre the region belongs to; where all the rivers are located; how many areas each region has; where all the ports, towns, settlements, and resources are; and possibly which battle maps are used.

8) Startpos.esf determines what level all the buildings are at and their maximum level. The BDI_pool entries control how much an AI faction wants to do anything.

9) Pathfinding.esf tells ETW where units can walk, where ships can sail, and where all the settlements/towns/ports/resources are on the campaign map.

10) Poi.esf tells the AI how to get to different regions.

11) Sea_grids.esf tells ETW which regions each port can trade with and which seas it's connected to.

12) The heightmap.tga in the main.pack determines the height of the land on the campaign map. It has no effect on the sea.

13) The coastline_group and coastline_group.cs2 rigid_mesh files tell ETW how the waves should hit the coastlines.

14) The rigid_spline files tell ETW the shape of each region (their borders). However they are purely cosmetic.

15) ETW is constantly updating the BDI_pool entries, so anything added to ETW does not require new BDI_pool entries.

16) The stpd and stpi files form a giant image that is painted onto the ETW world.

2. ## Re: Guide to the ETW map

Nice

Hopefully this will lead to some further advances in the modding of ETW

+Rep

3. ## Re: Guide to the ETW map

Excellent news. Would be great to have Brazil and the East Indies in the game.

4. ## Re: Guide to the ETW map

nice information uanime5, I'd give you rep but I already did years ago for your conquest mod
does this mean that we may be able to add meshes to these expanded regions into the game?

I wish CA would release documentation on how to do this..

5. ## Re: Guide to the ETW map

I'm not sure whether we need to add meshes or whether we just need to edit the regions.esf to make these regions playable.

Though we'll probably need to edit the land.rigid_mesh to be able to see them.

6. ## Re: Guide to the ETW map

Has anyone looked over all of the NTW patch packs and other .pack files since the last patch 3 update?

I have never seen this before (not that I was looking for it) but there is a rigidmesh.pack file in the main NTW data folder that has rigid_splines for all of the regions. I think CA might be slipping things in (unlocking) without telling anyone which was why I was looking. I saw the files for the uniteditor in there as well. Opening up the meshes it appears to not be complete yet.

see: rigidmodels.pack/rigidmodels/campaignborders/

There are also all of the rigid splines for roads, rivers, coastlines, etc. These files can be opened in Cinema or other 3d programs (which I don't currently have).
Maybe this pack has been there all along but thought I'd point it out.

7. ## Re: Guide to the ETW map

I'll have to check that out. I suspect that understanding this mesh file will be an important part of understanding how the map works and how to improve it.

UPDATE: the first post has been updated with additional information about the size of the full map ( 2684354560 by 1342177280) and what the poi.esf does.

8. ## Re: Guide to the ETW map

There is also a fogmask.dds file in the campaign map folders (one for each campaign). Is this the fog that prevents you from seeing the non-playable regions or is it an actual fog "weather" effect on the campaign map?

9. ## Re: Guide to the ETW map

I also have a question and observation:

Did Steam download the Warpath campaign to everyone's ETW installation and then only those that pay for the campaign can have it unlocked?

If so, a good reference source to study would be the changes between the ETW America map and the Warpath America map. It could lead to a gold mine of information how to add a region. If we knew exactly which tables and files were used to incorporate the Warpath changes that will pretty much spell out what we need to be researching to add regions to a map.

I did not buy the warpath campaign so I cannot check for the before and after tables/files. Can someone else pick up on this that has purchased the Warpath campaign?

10. ## Re: Guide to the ETW map

Originally Posted by Hazbones
There is also a fogmask.dds file in the campaign map folders (one for each campaign). Is this the fog that prevents you from seeing the non-playable regions or is it an actual fog "weather" effect on the campaign map?
I'm not sure. I tried editing the map_visibility file but it didn't let me see any more of the map.

Originally Posted by Hazbones
I also have a question and observation:

Did Steam download the Warpath campaign to everyone's ETW installation and then only those that pay for the campaign can have it unlocked?

If so, a good reference source to study would be the changes between the ETW America map and the Warpath America map. It could lead to a gold mine of information how to add a region. If we knew exactly which tables and files were used to incorporate the Warpath changes that will pretty much spell out what we need to be researching to add regions to a map.

I did not buy the warpath campaign so I cannot check for the before and after tables/files. Can someone else pick up on this that has purchased the Warpath campaign?
Warpath was downloaded onto everyone's ETW installation, though only people who purchase it can play it.

Did Warpath add any additional regions? If it did then if we compare the Warpath files to the episode 1 files this will make any changes obvious because both are set only in the US.

11. ## Re: Guide to the ETW map

Yes the Warpath campaign added regions to the US map. I did not purchase Warpath so cannot compare the maps. If you can compile a list of all the files that were used to update for the Warpath addition you'll have all the files identified that need to be modded to add regions.

12. ## Re: Guide to the ETW map

It's pretty cool that the information we need for other areas of the world, like China, is largely done for us.

13. ## Re: Guide to the ETW map

Just checked the warpath files and you were right, it does contain more regions. Below are the maps of America from the vanilla ETW (left) and Warpath (right) which show how the two campaigns differ.

After checking the regions.esf I have learned the following.

These regions were renamed (probably so the regions would have different names on the map). The left name is original ETW name, the right name is the Warpath name.

huron / natives_huron
ruperts_land / natives_ruperts_land
michigan / natives_michigan
algonquin / natives_algonquin
upper_louisiana / natives_upper_louisiana
northwest territories / natives_nw_territory
great_plains / natives_black_hills

For unknown reasons the northwest territories was moved to the bottom of the list of files in the Theatres_and_region_keys. Some of the 'Region_data/regions' entries are in a different place as well.

The Theatres_and_region_keys entries all have the same coordinates but some of the regions' Region_data/regions coordinates are different.

The following are new regions.

natives_chickasaw_territory
natives_wichita_territory
natives_cheyenne_territory
natives_sioux_territory
natives_chihuahua
natives_ohio_territory
natives_indiana_territory
natives_apache_territory

The only difference between the ETW and Warpath versions seems to be that a different 'campaign_maps' and 'campaigns' folders are used. So only the following need to be edited to add new regions.

pathfinding.esf
poi.esf
regions.esf
sea_grids.esf

startpos.esf
scripting.lua

The 'local_en.pack' and other variants of this pack have most likely been altered to contain all the new region names.

The 'america_lookup.tga', 'america_map.tga', and 'stratradar_america.tga' are also different but it may not be essential to alter them.

14. ## Re: Guide to the ETW map

I would guess that if we wanted to create our own regions we will have to mod folder our changes as CA did with the Warpath campaign and map.

This also means that the borders are defined in one of those esf files which you already assumed.
I guess one could try fooling around with the coordinates in the regions esf and see if tweaking them moves the region border or not?
How do the db tables play into this whole esf setup?

I will see if I can find correlations between the db tables and the esf files...

15. ## Re: Guide to the ETW map

Originally Posted by uanime5
huron / natives_huron
ruperts_land / natives_ruperts_land
michigan / natives_michigan
algonquin / natives_algonquin
upper_louisiana / natives_upper_louisiana
northwest territories / natives_nw_territory
great_plains / natives_black_hills
Similar with factions there are cherokee for G.C. and cherokee_playable for W.C.

The 'america_lookup.tga', 'america_map.tga', and 'stratradar_america.tga' are also different but it may not be essential to alter them.
These are useful for main menu (faction selection screen) and mini map in game's HUD.

16. ## Re: Guide to the ETW map

Originally Posted by Hazbones
I would guess that if we wanted to create our own regions we will have to mod folder our changes as CA did with the Warpath campaign and map.
There's an entry in the stapos.esf that tell ETW which map to use so making a new map folder isn't difficult.

This also means that the borders are defined in one of those esf files which you already assumed.
I guess one could try fooling around with the coordinates in the regions esf and see if tweaking them moves the region border or not?
In the regions.esf for each area there is an Outline folder with several Connectivity entries. I think that borders are controlled by the these entries but I don't understand how they work.

How do the db tables play into this whole esf setup?

I will see if I can find correlations between the db tables and the esf files...
I'm having trouble reading the db tables because PackFileManager will not longer display them and tells me to download DBEditor, and DBEditor can only display packs 4 and 5. Not sure how to fix this.

17. ## Re: Guide to the ETW map

Use PFM from Napoleon thread we've developed further. There are many relations between startpos and pack but not much about our research, except some obvious lists like regions, settlements, ports, etc.

18. ## Re: Guide to the ETW map

Originally Posted by uanime5
In the regions.esf for each area there is an Outline folder with several Connectivity entries. I think that borders are controlled by the these entries but I don't understand how they work.
I saw those connectivity entries as well but I was zipping through them and cannot recall what info they show? Do you know what they are displaying?

Does anyone have a 3D program that will open the spline files? Really any of them can open a spline I just don't have one... I really want to see what those splines look like and if they give coordinates.

19. ## Re: Guide to the ETW map

Originally Posted by Hazbones
I saw those connectivity entries as well but I was zipping through them and cannot recall what info they show? Do you know what they are displaying?

Does anyone have a 3D program that will open the spline files? Really any of them can open a spline I just don't have one... I really want to see what those splines look like and if they give coordinates.
Have you tried the rigid_mesh converter? It let me look at the land.rigid_mesh, though I can't edit anything.

20. ## Re: Guide to the ETW map

I could not open the spline files using the rigid_mesh converter.

Page 1 of 90 12345678910112651 ... Last

#### Posting Permissions

• You may not post new threads
• You may not post replies
• You may not post attachments
• You may not edit your posts
•