STARTPOS.ESF analysis and modifications. HYBRID startpos.esf

[uanime5]

And so that would mean that in addition to moving towns around the map (which I believe some people have accomplished at least Chris Harshman did) then part of the process of moving a town or port you also need to adjust the POI for that region or else the AI will keep going to the old location where the town stood.

That's correct. It also means that if you add a new town or port you have to modify the POI or the AI won't be able to find it.

I was examining the POI and noticed that some of the coordinates used don't exactly match the location of a settlement/town/port but are very close to it. This either means it's telling the AI where the Zone of Control is (area around a settlement/town/port) or it's because the settlement/town/port takes up some space on the map.

Region Coordinate Mapping:
I wonder if it is possible to place a town in a region's corner and then move it along an axis at an interval jotting down the coordinates along the way. Eventually after some really tedious work you will have all the coordinates for the region's border for which you are mapping.

It is a long round-about way but it might could be done (in theory). Of course if we could just find a file that has all of the coordinates that would be much easier.

What I was thinking is that a map grid of each region could be created to use as a tool for modding and research of the map process. With a grid one could easily recognize coordinates in files that are being researched and associate them to region and startpos.

The regions.esf has the coordinates of every region, though these regions overlap each other. The border coordinates are probably in this file as well, though I don't know where.

As I've made a scale model of the smaller map it would be much easier to use this map to get the coordinates of all the borders rather than by moving towns all over a region.

Spoiler Alert, click show to read: 

I have tried adding areas and towns to this map but it's exceptionally time consuming (there are thousands of entries, each of which has to go in exactly the right place).

What we need is a tool that will read all the coordinates in the regions.esf and create a map based on these coordinates. This would give us a map with all the regions, areas, settlements, town, etc. If it could also convert the coordinates used in the starpos and poi to the smaller map coordinates this would make it easier to see where everything is on the map.

[Hazbones]

What we need is a tool that will read all the coordinates in the regions.esf and create a map based on these coordinates. This would give us a map with all the regions, areas, settlements, town, etc. If it could also convert the coordinates used in the starpos and poi to the smaller map coordinates this would make it easier to see where everything is on the map.

What did you use to create the map overlay to the larger map?

[husserlTW]

There is a trick I am going to follow. I'll enter the coordinations of POI to an existing army and I'll see where the army will be appeared. Then we'll know...

[husserlTW]

O.K. here is virginia's entries in Poi (episodic_map) from no 69 to no 79 (69 is 0, 79 is 10):



There are more for virginia but there are already some perimeter points which maybe define regions perimeter, but there are also so many in the center for what? Have in mind that in every one record there are references (distance?) with the neighbor regions.

[uanime5]

What did you use to create the map overlay to the larger map?

Currently I haven't created the larger map because it would be over 1 million times larger. It's easier to use the coordinates from the smaller map and multiply them by 1048576 to get the approximate coordinates for the larger map.

O.K. here is virginia's entries in Poi (episodic_map) from no 69 to no 79 (69 is 0, 79 is 10):



There are more for virginia but there are already some perimeter points which maybe define regions perimeter, but there are also so many in the center for what? Have in mind that in every one record there are references (distance?) with the neighbor regions.

Interesting idea husserlTW. Without all the points it's difficult to tell what they represent but there doesn't seem to be a pattern, such as they start far away and get closer.

As movement is limited in ETW these points might be to help the AI plan where it's going to go over several turns. For example to get to Virginia from Maryland the AI would travel to point 10, then to point 6 or 8; and to get to Virginia from Pennsylvania the AI would travel to point 7, then to point 6 or 8. As the AI knows how far away each point is from its cities it can calculate how long it will take each army to travel to these areas.

[husserlTW]

Interesting idea husserlTW. Without all the points it's difficult to tell what they represent but there doesn't seem to be a pattern, such as they start far away and get closer.

As movement is limited in ETW these points might be to help the AI plan where it's going to go over several turns. For example to get to Virginia from Maryland the AI would travel to point 10, then to point 6 or 8; and to get to Virginia from Pennsylvania the AI would travel to point 7, then to point 6 or 8. As the AI knows how far away each point is from its cities it can calculate how long it will take each army to travel to these areas.

Interesting... Next thing is to turn the region style grids circled in red in next image to startpos style, according your equation and then add them in the blue circled ones. But to x or to y?



Also we can check the rest map files. Trade routes is easy:



There are the 3 available settlements of episode 1 and 3 routes related as 0-1, 0-2 and 1-2.

Pathfinder is full of grids but I thing they related with poi. I'll check it further.

Hey, uanime5 I am glad you have interested on this search, it is been some time now that nobody has joined me in searching esf files.

Edit: And of course Hazbones.

[Hazbones]

Hey, uanime5 I am glad you have interested on this search, it is been some time now that nobody has joined me in searching esf files.

Edit: And of course Hazbones.

Hey, you guys are doing all the work. I am just feeding some ammunition to keep you motivated in your search. The results could benefit the whole community. You'll definately win some kind of award if you and uanime5 can crack the map or at least give us a better understanding of how this whole thing works.

You know, I was reading through this thread and I saw that someone cloned a node that gave them another stack of units on the map. I wonder if the same can be done with regions or settlements?

For regions: Clone something that was already there, rename it, maybe tweak the region coordinates by cutting them in half (if squeezing in a new region), add a color to the tga file like the old MTW map editing method and see if it shows on map?

For settlements: clone an existing settlement and move the clone to a new location on the map. You'll have to make sure that all the correct entries for your clone town are placed in the correct tables which I guess is something else that needs to be researched. Even just a proof of concept would be exciting even if it CTD's.

Now I am just wishful thinking that it could be so simple...

[EDIT]: I just read the thread where Harshman_Chris moved a settlement and saw the spreadsheet husserlTW made with the tutorial by Erasmus. It seems that the process for creating and moving a settlement is well established. I wonder if the same method can be used for regions though through cloning as I mentioned above?

[uanime5]

Blue circle grids are the red spot on the image which you suppose -and it is likely correct- that are marks needed for CAI to calculate routes. If this is the point then the red circled numbers under the neighbor regions names (e.g. Maryland) could be the closest distance to the borders or to another similar red spot in Maryland region.

It is likely that regions.esf style single grid is used to calculate distances. this is obvious also checking CAI_WORLD_REGION_BOUNDARIES in CAI_INTERFACE of startpos, where every region is referred in a pair with one of it's neighbors with the length of the common borders:

This hypothesis solves an older question of mine why further regions (e.g. pennsy, algonquin) sometimes have smaller numbers than a closer one (e.g. maryland)!

Distance along a common border makes the most sense and it tells ETW what shape the region is.

As there aren't 1255 regions in ETW the 'Region AI ID' numbers may represent area numbers (parts of the regions). Though if ETW had 100 regions each one would need 12-13 areas, so this may also be incrorect. I'm unsure why these entries are numbered in this way but it may be more complex than each number equals a region.

For regions: Clone something that was already there, rename it, maybe tweak the region coordinates by cutting them in half (if squeezing in a new region), add a color to the tga file like the old MTW map editing method and see if it shows on map?

For settlements: clone an existing settlement and move the clone to a new location on the map. You'll have to make sure that all the correct entries for your clone town are placed in the correct tables which I guess is something else that needs to be researched. Even just a proof of concept would be exciting even if it CTD's.

The problem with cloning a region is that we don't know what every part of a region does, so it's highly likely that we'll end up with two copies of the same region (which will cause ETW to crash). Even trying to use Warpath's files to split an American region will be difficult.

It would be easier to clone a town and add a new town to the map, as this involves changing far fewer files.

[Geronimo2006]

Do you think you might be able to add a few Brazilian provinces unanime5?

[uanime5]

Do you think you might be able to add a few Brazilian provinces unanime5?

Not at the moment. There is still so much of ETW neither I nor any other modder understands. Adding regions to Brazil would be one of the last things we figure out how to do. Below is a list of how I believe modding the ETW map will progress.

1) Being able to move a town / port and have the AI recognise this town / port has been moved.
2) Being able to add a new town / port and have the AI recognise it.
3) Being able to change the border of a region (gerrymandering).
4) Being able to add a new region.
5) Being able to expand the map to include hidden regions.
6) Being able to remake the map.

As Brazil isn't on the current map we would need to get to step 5 before we could add new regions to it (though France and Spain can be given new regions with a step 4 understanding because they're already on the map). Currently we are between steps 1 and 2 as we can move towns / ports and add them with some degree of success but we cannot make the AI recognise where they are.

Interesting... Next thing is to turn the region style grids circled in red in next image to startpos style, according your equation and then add them in the blue circled ones. But to x or to y?

The number in the blue circle are coordinates for the full map but the numbers in the red circles seem to be for the smaller map. I'm not sure why CA changed from the full map to the smaller map.

The first red circle is for Virginia but the rest are for other regions (the last one represents the closest region, in this case carolinas, so you don't need to test this one). I'm not sure why why a virginia entry is needed.


As a diagonal line is the shortest distance between two points you should half the number in the red circle and add it to both the X and Y axis.

Below is an example of what I mean.

Poi coordinates for this region: -578027520 (X); 279707648 (Y)
Number for carolina (small map): 6.576473
Number for carolina (large map): 6895932 (rounding up to a whole number)
Amount added to each Poi coordinate: 3447966
New Poi coordinates: -574579554 (X); 283155614 (Y)


Alternatively the Virginia entry (4.806018) could be one of the x or y entries and the entry for the other regions is the other x or y entry.

Below is an example of what I mean.

Poi X coordinate for this region: -578027520
Poi Y coordinate for this region: 279707648

Number for carolina (small map): 6.576473
Number for carolina (large map): 6895932 (rounding up to a whole number)

Number for virginia (small map): 4.806018
Number for virginia (large map): 5039475 (rounding down to a whole number)

If virginia is X and carolina is Y the new poi coordinates will be: -572988045 (X), 286603580 (Y)

If carolina is X and virginia is Y the new poi coordinates will be: -571131588 (X), 284747123 (Y)


In both of the above examples I've assumed that you have to add these numbers to the poi entry. If you have to subtract them the results will be different. I can calculate the coordinates for subtraction as well if you want me to.

[husserlTW]

Blue circle grids are the red spot on the image which you suppose -and it is likely correct- that are marks needed for CAI to calculate routes. If this is the point then the red circled numbers under the neighbor regions names (e.g. Maryland) could be the closest distance to the borders or to another similar red spot in Maryland region.

It is likely that regions.esf style single grid is used to calculate distances. this is obvious also checking CAI_WORLD_REGION_BOUNDARIES in CAI_INTERFACE of startpos, where every region is referred in a pair with one of it's neighbors with the length of the common borders:

virginia - maryland:



virginia _ pennsy



virginia - algonquin_territory



This hypothesis solves an older question of mine why further regions (e.g. pennsy, algonquin) sometimes have smaller numbers than a closer one (e.g. maryland)!

[Hazbones]

It is likely that regions.esf style single grid is used to calculate distances. this is obvious also checking CAI_WORLD_REGION_BOUNDARIES in CAI_INTERFACE of startpos, where every region is referred in a pair with one of it's neighbors with the length of the common borders:

virginia - maryland:

What does the 3rd number down in the Value column represent again?

Can we safely say which data links all of the esf files to each other? I got:

[Region esf] Region Keys/ Region Data/ Region/ (name of region)
Links to:
[Startpos esf] Campaign/ Preopen/ Setup/ Region_ownership/ Ownership/ (name of region)

AND

[Region esf] Region Keys/ Region_data/ Region/ (map coordinates X,Y) <--guessing
How to Link to:
[POI]
*map X,Y coordinates (Any other key field??)

[Startpos esf]
How to Link to:
[POI]

[husserlTW]

As you see that process is stuck.

Just copying nodes is the worst thing to do. We do not want just to add things but we also want to be functional...

[Hazbones]

Yeah I understand that. I was just hoping for a proof-of-concept to see if cloning was even viable for region creation.

The settlement issue might could be fixed with the new info uanime5 found on the POI?

[husserlTW]

I believe the length of common borders. In the image above is virginia with maryland.

Playable regions are 137. What is this number 1225 you are talking? For startpos I know that id numbers in WORLD are 9 digits and change in every turn. In CAI_WORLD Id's never change and they follow a certain order which is easy to understand.

[Hazbones]

Ok, do you know how the startpos then links to the POI?
I do believe all of this is relational and so there has to be 1 or more fields that link them together.
If we can identify key fields then that will help in learning the process of creating a new region (see post 415).

[husserlTW]

Poi.esf has only the grids of the capitol settlement common with startpos. Pathfinder.esf has nothing at all. I mean no id's from Pathfinder.esf are present in startpos.

And what is the best and most confusing? After the game starts the map files are not used any more. Try to remove them and play next turn, you'll see everything is normal. That means that game uses them just to build the map when entering in anew campaign. Since the save file is actually a startpos file I cannot thing anything else but game keeps there all the map infos!

[uanime5]

Playable regions are 137. What is this number 1225 you are talking? For startpos I know that id numbers in WORLD are 9 digits and change in every turn. In CAI_WORLD Id's never change and they follow a certain order which is easy to understand.

In the CAI_WORLD_REGION_BOUNDARIES entries virgina was 761, maryland was 774, pennsylvania was 767, and algonquin_territory was 780, and the highest number I found was 1225. As ETW only has 137 playable regions I wondered why they didn't number the regions 1-137, rather than 54-1225.

And what is the best and most confusing? After the game starts the map files are not used any more. Try to remove them and play next turn, you'll see everything is normal. That means that game uses them just to build the map when entering in anew campaign. Since the save file is actually a startpos file I cannot thing anything else but game keeps there all the map infos!

The same thing happened in M2TW, so that once you started a campaign any changes you made to the map would only affect a new campaign. The save files contains a copy of the map and where everything is on the map, unlike the normal startpos which only tells ETW where everything is on the map.

[husserlTW]

In the CAI_WORLD_REGION_BOUNDARIES entries virgina was 761, maryland was 774, pennsylvania was 767, and algonquin_territory was 780, and the highest number I found was 1225. As ETW only has 137 playable regions I wondered why they didn't number the regions 1-137, rather than 54-1225.

These numbers (1-57) are used (e.g. TRADE_POSTS). In Grand Campaign where factions, regions, etc. are more these numbers are higher. But all numbers from 1 until the last one are used.

[Hazbones]

What file extension is used for saved game files? Sorry for such basic questions but I am not at home so cannot check.