Guide to the ETW map

**OldDrunkBastard** · March 17, 2012, 05:10 PM

K

**OldDrunkBastard** · March 17, 2012, 05:17 PM

Double post Sry I know, but i found this sooo...

**taw** · March 17, 2012, 05:24 PM

A quick observations about query_info:

131072 = 0x00020000.

327681 = 0x00050001.

There's no way in hell these are IDs. They are some kind of quad tree bitmaps more likely.

Every id/id1/id2 in query_info has very few bits set when expressed as hex.

coord1/2 are all between 0 and 60570 in query_info1 and 104 to 49941 in query_info2.

At firs they look like plausibly 16-bit coordinates, but it's suspicious that they're never between 60571 and 64k, and for the second query info there's no way in hell they're never between 50k and 64k.

So id/id1/id2 are NOT ids, they're some kind of masks (possibly id1+id2 together form 64-bit mask 8x8 like in quad_trees, esf have 64-bit data type but for some reason prefer to use two 32-bit values), and coord1/coord2 are probably IDs, and almost certainly not coordinates.

That's all.

**taw** · March 17, 2012, 05:36 PM

I changed id/id1/id2 to mask/mask1/mask2 in hex.
Grouping in data is actually (coord1, coord2, mask1, mask2) not this mixed grouping.
There's probably no association of coord1 with mask1, and coord2 with mask2.
More likely than not it's two numbers (coord1, coord2) that are indexes of something, plus one 64-bit 8x8 mask.

coord1/coord2 are probably not coords, but I have no idea what else they could possibly be, so I'll leave them like that for now.

**taw** · March 17, 2012, 05:44 PM

Obvious answer is obvious, they're indexes to vertices table.

Finelli or someone - could you take a look at it if it makes any sense?

**taw** · March 17, 2012, 05:47 PM

Here's an example of a fairly deep cell:

v1/v2 are both NOT exactly within the cell bounding box, vertices sometimes go slightly outside cell.
At least one end seems to be always within the bounding box.

This suggests they are not point features, they're something else.

**taw** · March 17, 2012, 05:54 PM

Many cells with mask2=ffffffff with mask1 with few bits set strongly disproves my theory that it's one 64-bit mask. It's two 32-bit values that still look like some kind of masks to me.

No good idea what they are. They seem to mostly be 00XX00YY, with 1st and 3rd most significant bytes zeroed, and usually least-significant byte is also zero, or <=15 at least (usually, there are exceptions).

Most of the action seems to be in 2nd most significant byte.

It's either 00XX00YY or all FFFFFFFF as far as I can tell. I see no other strict regularities,
in particular none of mask/mask1/mask2 is always superset or subset or another.

**taw** · March 17, 2012, 06:11 PM

S2TW uses exactly the same regions.esf format as ETW (so pressumably NTW does as well).
It might be worthwhile to use S2TW files for testing since it's simpler than ETW's.

It seems that vertex pairs that go outside of quad's bounding box are repeated twice. Those that don't are not.

For example 25060 occurs in:

Quad for:

<v2 x="-700.0" y="280.0"/>
<v2 x="-680.0" y="290.0"/>

<cell_quad v1='25060 (-679.65881,288.46317)' v2='27427 (-680.00043,288.63312)' mask1='00b40000' mask2='00240008'/> (crossing box, see below)

Quad for:

<v2 x="-680.0" y="285.0"/>
<v2 x="-670.0" y="290.0"/>

<cell_quad v1='24858 (-679.45221,288.25653)' v2='25060 (-679.65881,288.46317)' mask1='00b40000' mask2='00240008'/> (not crossing box, no inverse anywhere)
<cell_quad v1='25060 (-679.65881,288.46317)' v2='27427 (-680.00043,288.63312)' mask1='00b40000' mask2='00240008'/> (crossing box, see above)

Sometimes copies are exact, something they're reversed (5186-5162, S2TW):

<cell_quad v1='5186 (-30.0012073516846,-141.60289)' v2='5162 (-29.92026,-141.44437)' mask1='00010003' mask2='0001000c'/>
<cell_quad v1='5185 (-31.1498851776123,-143.55949)' v2='5186 (-30.0012073516846,-141.60289)' mask1='00010003' mask2='0001000c'/>
<cell_quad v1='5162 (-29.92026,-141.44437)' v2='5186 (-30.0012073516846,-141.60289)' mask1='0001000c' mask2='00010003'/>

Reversed ones have masks switched backwards. This sort of implies "masks" might be some kind of vertex flags.

If we figured out what masks are, we could make a script which converts a list of vertex pairs into fully formed query info quad tree. This would make massive progress towards working map compiler.

**PietroMicca** · March 17, 2012, 07:36 PM

Originally Posted by taw

Obvious answer is obvious, they're indexes to vertices table.

Finelli or someone - could you take a look at it if it makes any sense?

Obviously it makes sense. When I have explained this part I have tried to tell you that are indexes from vertices. This was my idea from the beginning.

Originally Posted by taw

Here's an example of a fairly deep cell:

v1/v2 are both NOT exactly within the cell bounding box, vertices sometimes go slightly outside cell.
At least one end seems to be always within the bounding box.

This suggests they are not point features, they're something else.

No, no don't be hurry. I was wrong about ids, but these are vertices. You can be sure. First of all I don't know what you mean with mask. I don't know what is a mask and so I have thought the most easy thing basing on my knowledge. Now I'll try to explain why vertices sometimes go slightly outside cell. Quad tree data structures are for geographical points inside a map. But in regions.esf files quad tree is used to locate geometric figures bounded by vertices. This is a bit different from the normal usage of quad tree system. In fact now what is important are not points for itself, but segments that join two points. You can see that. Points are always in couple. Query info are based, as obvious, on recursive system of quads of 7 levels. It happens that a segment can be split between two quads. But this segment is anyway defined by two vertices. Accordingly the same couple of vertices will be repeated in two neighboring quads. Bounds of the quad allow to discard the portion of segment that is not within. Finally if you draw segment by segment all cells in query info, you will obtain global map as if you have drawn all outlines. This is true also for groundtype query info (query_info-0002.xml).

**taw** · March 18, 2012, 02:54 AM

"Mask" as in set of bits where 1s and 0s mean on or off, and that's the only thing the number means.

That was my first guess (8x8 bit array showing which parts of quad tree had something in it - that's how quad_trees/*.xml in startpos.esf work).
Now I'm not sure, but it's definitely not IDs either.

My current best guess is that "mask1" is associated with v1 somehow, and "mask2" with v2 (like you said), based mostly on S2TW
where cell_quads duplicated across boxes can have both v1 switched with v2, and mask1 switched with mask2.

Copying line segment that intersects with multiple bounding boxes is exactly how quad trees are normally used.
I'm not sure why it's sometimes inverted, it probably doesn't matter.

More tests. Assuming each "mask" is actually two numbers (so 4 numbers per line segment).

ETW query_info_1.xml: 0x0000 to 0x00cc (204)

ETW query_info_2.xml: 0x0000 to 0x0004 (4) and 0xffff

S2TW query_info_1.xml: 0x0000 to 0x001b (27)

S2TW query_info_2.xml: 0x0000 to 0x002a (42)

S2TW query_info_3.xml: 0x0000 to 0x0001 (1) and 0xffff

So these look like IDs or indexes of some kind after all - except we have two IDs per vertex.

They are not vertex IDs or cell IDs - there are far too many cells and vertices.
There are too many slots and areas, and not enough regions as far as I can tell.

There are some u2s in area/*.xml (unknown1 id, island id, unknown2 id) but there's no obvious match either.

This is probably a solvable problem. Any more ideas?

**taw** · March 18, 2012, 03:22 AM

I changed the format to something better corresponding to what we know:

No hexes, simply splitting "mask1/mask2" (which turned out not to be masks) into 2 16-bit integers each.

a1/b1 probably correspond to v1, and a2/b2 probably correspond to v2.

If someone can tell me the meaning of a1/b1/a2/b2 that would be lovely.

**PietroMicca** · March 18, 2012, 06:56 AM

Originally Posted by taw

ETW query_info_1.xml: 0x0000 to 0x00cc (204)

This is the first thing I have noticed. In ETW there are 205 regions, from 0 to 204. I will take a better look later.

**taw** · March 18, 2012, 08:07 AM

From one cell:

From another:

<line_segment v1='58057 (-2.09213209152222,316.53064)' a1='70' b1='0' v2='30262 (-2.16471338272095,316.5903)' a2='36' b2='43'/>

So a1 and a2 could plausibly be region numbers (and I'll comment them as such).

b1/b2 really don't look like that. And they're mostly 0 everywhere.

There are also two u16s in cell - first I'll assume is region code as well, second - who known.

Here's how it looks now:

ETW query_info-1 b = 0 to 153
ETW query_info-2 b = 0 to 3 and 65535
S2TW query_info-1 b = 0 to 20
S2TW query_info-2 b = 0 to 42 and 65535
S2TW query_info-3 b = 0 and 65535 only

Does it look like anything?

EDIT: It totally is the number of area within region. Region "all" has 154 areas, and everything else checks.

**taw** · March 18, 2012, 08:14 AM

There are trivial details like figuring out which of multiple regions/areas cell claims as its own, but the entire thing is fully decoded for all practical purposes now.

**PietroMicca** · March 18, 2012, 08:27 AM

This makes things much harder to understand. ~~How can be inserted in a database 2 points in such a way that they are always belonging to two different regions?~~

**uanime5** · March 18, 2012, 08:31 AM

Congratulations on figuring this out Taw. I'm not sure why but 65535 is often present in the first area entry in the regions.esf\theatres_and_region_keys\region_data\regions\areas. I believe it tells ETW which land mass this region belongs to, while the other area entries tell ETW about specific things in this area. The most common land masses are Eurasia (13) and the Americas (20). The other land masses are usually island, such as the UK (11) or Trinidad and Tobago (108); or areas that cannot easily be accessed, such as French and Dutch Guyana (14).

The primary purpose of this is to stop the AI trying to march their armies across the sea, such as going from Italy to Corsica. If Italy (13) and Corsia (82) are classified as different land masses then the AI knows it can't march between them and needs to sail from one land mass to another one.

Region 'all' refers to the rivers in ETW.

**PietroMicca** · March 18, 2012, 08:35 AM

65535 is for passable areas

if I remember correctly 256 is for impassable (but only in ETW)

**taw** · March 18, 2012, 08:49 AM

Originally Posted by PietroMicca

This makes things much harder to understand. How can be inserted in a database 2 points in such a way that they are always belonging to two different regions?

They're from different areas, not necessarily different regions. (grepping shows no cases of same region1=region2 and area1=area2 anywhere).

<line_segment v1='3084 (-1.03942561149597,301.97079)' region1='50 (spain)' area1='5' v2='37345 (-2.33167123794556,302.08218)' region2='50 (spain)' area2='0'/>

Are these line segments borders or connecting lines or what? Have we ever bothered drawing them?

**uanime5** · March 18, 2012, 08:55 AM

Originally Posted by PietroMicca

65535 is for passable areas

if I remember correctly 256 is for impassable (but only in ETW)

It's possible that only the 65535 areas are passable, while the other 307 entries refer to impassible areas. I don't think 256 is the only impassible number because the Alps are impassible but they're all 200.

**PietroMicca** · March 18, 2012, 08:56 AM

Originally Posted by taw

They're from different areas, not necessarily different regions. (grepping shows no cases of same region1=region2 and area1=area2 anywhere).

<line_segment v1='3084 (-1.03942561149597,301.97079)' region1='50 (spain)' area1='5' v2='37345 (-2.33167123794556,302.08218)' region2='50 (spain)' area2='0'/>

Are these line segments borders or connecting lines or what? Have we ever bothered drawing them?

Yes, they are what I have said. I have drawn query info for egypt campaign in Napoleon. I only mean that create this kind of file will be much harder for us. There's some complicated technique we don't know and hardly we will know it.

Anyway, there's another similar mask in regions.esf and it's about <ary type="connectivity"> in areas.

Thread: Guide to the ETW map

Thread Tools

Search Thread

Display

Re: Guide to the ETW map

Re: Guide to the ETW map

Re: Guide to the ETW map

Re: Guide to the ETW map

Re: Guide to the ETW map

Re: Guide to the ETW map

Re: Guide to the ETW map

Re: Guide to the ETW map

Re: Guide to the ETW map

Re: Guide to the ETW map

Re: Guide to the ETW map

Re: Guide to the ETW map

Re: Guide to the ETW map

Re: Guide to the ETW map

Re: Guide to the ETW map

Re: Guide to the ETW map

Re: Guide to the ETW map

Re: Guide to the ETW map

Re: Guide to the ETW map

Re: Guide to the ETW map

Posting Permissions