DB.xsd with every single table DBEditor can handle - 76 new ones

[taw]

As I mentioned earlier I wrote a program for inferring table types.

Here's DB.xsd for DBEditor with almost every table type guessed. There are only two non-empty tables missing that are I'm aware of, both of which cannot work with DBEditor anyway:

• models_building_tables (models, patch) - schema described in linked post, but it's a bit too complicated for DBEditor - you'd need to do some coding to change it
• models_naval_tables (models, patch) - schema definitely impossible for DBEditor, much more fiendish version of models_building_tables with multiple nested record types. I tried it but it's really really difficult, and even having schema it would be very hard to modify it. Good luck with hex editors if you want to give it a try.

Here's the list of all new tables (to make search function happy):

• battle_groundcover_density_maps_tables
• battle_groundcover_distribution_maps_tables
• battle_terrain_farms_tables
• battlefield_building_categories_tables
• climate_to_erosion_settings_tables
• climates_tables
• commodities_demand_drivers_tables
• commodity_unit_names_tables
• cursors_tables
• diplomacy_attitudes_tables
• diplomacy_factor_strings_tables
• diplomacy_negotiation_faction_override_strings_tables
• diplomacy_negotiation_strings_tables
• diplomacy_strings_tables
• diplomatic_relations_attitudes_tables
• diplomatic_relations_government_type_tables
• diplomatic_relations_religion_tables
• disaster_to_ground_types_tables
• disasters_tables
• diseases_tables
• effect_bonus_value_commodity_junction_tables
• effect_bonus_value_population_class_and_religion_junction_tables
• effect_bonus_value_projectile_junctions_tables
• effect_bonus_value_religion_junction_tables
• effect_bonus_value_resource_junction_tables
• effect_bonus_value_shot_type_junctions_tables
• effect_bonus_value_unit_category_junction_tables
• empires_regions_junct_tables
• empires_tables
• entity_training_levels_tables
• erosion_types_tables
• events_view_group_junct_tables
• faction_groups_tables
• family_trees_tables
• famous_battle_pools_tables
• fort_underlay_climate_jcts_tables
• governorships_tables
• lighting_setups_tables
• ministerial_effectiveness_modifiers_tables
• ministerial_position_default_names_tables
• ministerial_positions_tables
• ministerial_positions_to_effects_tables
• ministerial_positions_to_governorships_tables
• mission_activities_tables
• mission_effects_tables
• mission_sources_tables
• models_artilleries_tables
• models_artillery_tables
• mount_variants_tables
• movie_event_strings_tables
• names_forts_tables
• names_groups_tables
• particle_effects_tables
• pdlc_tables
• policies_tables
• population_classes_tables
• religions_tables
• resources_tables
• sea_climate_details_tables
• slots_gdp_values_tables
• slots_templates_models_tables
• small_vegetation_climates_jct_tables
• special_edition_enums_tables
• stances_tables
• taxes_classes_tables
• taxes_keys_tables
• taxes_levels_tables
• town_wealth_growth_factors_tables
• trade_details_tables
• warscape_categories_tables
• warscape_naval_lod_tables
• warscape_rigid_lod_range_tables
• warscape_trees_tables
• warscape_underlay_textures_tables

Plus two extras I found after original post, now also included:

• sea_surfaces_tables
• slots_art_tables

This is all mostly automatically generated. So nearly all column names are unknownX. In the updated file there are some automatically inferred primary keys and some other relationships, but they might be wrong or missing.

For tables with reasonably simple schemas like a few strings plus a float/int/bool or two, inferred type is quite certain. For monster tables like models_artilleries_tables with 184 fields inferred - well, mistakes are not impossible - sometimes floats and ints and groups of bools look a lot alike, so detection script might be totally confused.

Mistakes are especially likely when table has very few rows, or similar values in all. Judging from tables I manually checked, most inferred types are just fine. My script reported many cases where it just gave up and took a guess, like always false boolean, or always null nullable string; always 0.0 float, or always 0 int - they look identical in binary. This columns tend to be rather less interesting.

A few of the tables are obviously useful (diplomacy_*, ministerial_*, effect_bonus_*, taxes_*), and not too difficult to guess what is what. A few seem inactive like empires_* but maybe I just don't have a clue what they do.

If someone felt like going through tables found, and labeling columns properly, it would be great.

Good luck with modding.

EDIT: Updated 2 new tables, a lot of primary keys and foreign keys

[Heavy Weapons Guy]

As I mentioned earlier I wrote a program for inferring table types.

Here's DB.xsd for DBEditor with almost every table type guessed. The only four non-empty tables missing that are I'm aware of are:

• models_building_tables (models, patch)
• models_naval_tables (models, patch)
• sea_surfaces_tables (main)
• slots_art_tables (main)

Here's the list of all new tables (to make search function happy):

• battle_groundcover_density_maps_tables
• battle_groundcover_distribution_maps_tables
• battle_terrain_farms_tables
• battlefield_building_categories_tables
• climate_to_erosion_settings_tables
• climates_tables
• commodities_demand_drivers_tables
• commodity_unit_names_tables
• cursors_tables
• diplomacy_attitudes_tables
• diplomacy_factor_strings_tables
• diplomacy_negotiation_faction_override_strings_tables
• diplomacy_negotiation_strings_tables
• diplomacy_strings_tables
• diplomatic_relations_attitudes_tables
• diplomatic_relations_government_type_tables
• diplomatic_relations_religion_tables
• disaster_to_ground_types_tables
• disasters_tables
• diseases_tables
• effect_bonus_value_commodity_junction_tables
• effect_bonus_value_population_class_and_religion_junction_tables
• effect_bonus_value_projectile_junctions_tables
• effect_bonus_value_religion_junction_tables
• effect_bonus_value_resource_junction_tables
• effect_bonus_value_shot_type_junctions_tables
• effect_bonus_value_unit_category_junction_tables
• empires_regions_junct_tables
• empires_tables
• entity_training_levels_tables
• erosion_types_tables
• events_view_group_junct_tables
• faction_groups_tables
• family_trees_tables
• famous_battle_pools_tables
• fort_underlay_climate_jcts_tables
• governorships_tables
• lighting_setups_tables
• ministerial_effectiveness_modifiers_tables
• ministerial_position_default_names_tables
• ministerial_positions_tables
• ministerial_positions_to_effects_tables
• ministerial_positions_to_governorships_tables
• mission_activities_tables
• mission_effects_tables
• mission_sources_tables
• models_artilleries_tables
• models_artillery_tables
• mount_variants_tables
• movie_event_strings_tables
• names_forts_tables
• names_groups_tables
• particle_effects_tables
• pdlc_tables
• policies_tables
• population_classes_tables
• religions_tables
• resources_tables
• sea_climate_details_tables
• slots_gdp_values_tables
• slots_templates_models_tables
• small_vegetation_climates_jct_tables
• special_edition_enums_tables
• stances_tables
• taxes_classes_tables
• taxes_keys_tables
• taxes_levels_tables
• town_wealth_growth_factors_tables
• trade_details_tables
• warscape_categories_tables
• warscape_naval_lod_tables
• warscape_rigid_lod_range_tables
• warscape_trees_tables
• warscape_underlay_textures_tables

[Primergy]

models_building_tables (models, patch)

Exactly that one i want to have a look on, isnt open, damn..

But good work!

[taw]

Exactly that one i want to have a look on, isnt open, damn..

But good work!

Taking a closer look at models_building_tables, its schema seems to be:

• string building_identifier (like "american_ruin01")
• string path (like "buildings\american_ruin01\american_ruin01_tech.cs2.parsed")
• uint32 unknown (always 0 or 1)
• uint32 count (from 0 to 346 in data, count of the following)
• count times
  
  • string efline_identifier (like "EFline_piece01_destruct01_line01")
  • uint32 unknown (always one of 0,1,2,3)
  • 7x float unknown
  • either float or uint32 (always 0, zero looks the same in both formats)
  • float

All this (2kB on average) is just one row. As far as I can tell, DBEditor simply cannot handle schemas like this.

I won't be extending DBEditor as I know nothing about Windows programming, but I extracted all this data into attached text file if you want to take a look at it.

[Primergy]

Thank you for effort!

uint32 count (from 0 to 346 in data, count of the following)

My guess would be that the "346" are values for the garrisionable buildings how many men can find "shelter" in there.
While the EF-lines are lines inside the building on which the units are placed.

I believe there is a similar system in the naval models table for the cannons.
It's a pity that this table is nearly impossible ... which means that there will be never any new ship models.

[taw]

Thank you for effort!

My guess would be that the "346" are values for the garrisionable buildings how many men can find "shelter" in there.
While the EF-lines are lines inside the building on which the units are placed.

346 is number of efline entries, I don't think it's number of men.

Looking closer at eflines, they seem to be:

• string identifier
• uint32 - 0 to 3 - might that be maximum depth or other kind of men multiplier?
• 3D float position A
• 3D float position B
• 3D float unit vector C, more or less orthogonal to to (A-B)

A-B, and C tend to be axis aligned (or nearly-axis-aligned) on either first or third axis. Second axis is probably height as it tends not to change.

Maybe men stand on line from A to B, facing direction C ? It could be tested by reversing signs and seeing what happens.

OTOH all identifiers have names like "EFline_piece01_destruct01_line96" - what is this piece / destruct?

This is totally hackable with a hex editor, so go for it.

I believe there is a similar system in the naval models table for the cannons.
It's a pity that this table is nearly impossible ... which means that there will be never any new ship models.

It is easily the single most difficult table in db (and the only one not yet figured out). It's not literally impossible, it's just ridiculously tedious.

Here's how it starts:

• string identifier
• string path to logic.xml.rigging file
• string path to rigid_model_animation file
• camera count, every camera being
  
  • string camera identifier
  • 16 floats
  
  
• something count, every something being
  
  • int, 9 floats
  • another int counter, then as many:
    
    • int
    
    
  
  
• deck count, each deck being
  
  • name
  • int
  • count of something, every something being:
    
    • int
    • another count, each one being
      
      • 3 floats, boolean
      
      
    • 4 floats
    
    
  
  
• and so on

If you want to look at the monstrosity, I'm attaching partially processed data. It's not useful for any modding, but you should at least get good idea what's inside.

I'm annoyed that I managed to figure out every table except for just this one, but I'll leave it for now. I don't like naval combat that much anyway ;-)

It's definitely just a 3D mesh with basic metadata, everything being one massively nested (count_of_foo, foo0, foo1, foo2 ...) blob.

[Primergy]

346 is number of efline entries, I don't think it's number of men.

Looking closer at eflines, they seem to be:

• string identifier
• uint32 - 0 to 3 - might that be maximum depth or other kind of men multiplier?
• 3D float position A
• 3D float position B
• 3D float unit vector C, more or less orthogonal to to (A-B)

A-B, and C tend to be axis aligned (or nearly-axis-aligned) on either first or third axis. Second axis is probably height as it tends not to change.

Maybe men stand on line from A to B, facing direction C ? It could be tested by reversing signs and seeing what happens.

OTOH all identifiers have names like "EFline_piece01_destruct01_line96" - what is this piece / destruct?

This is totally hackable with a hex editor, so go for it.

Perhaps EF-lines arent "lines" but points, so every "line" is one point where a unit can take place.
With the uint32 0-3 i think you are right that, this is some sort of depth multiplier.
For example on Fort walls units are staying in 3 rows. Regarding forts. Did you find any possible entries for cannons on walls?

The piece vw destruct xy is a naming convention used for all buildings and buildinglike structures.

For example applebarrel_piece01_destruct01

Perhaps the engine is using the _piece01 part to determine if there's any other building piece which needs to be attached there (eg townhalls often have severel pieces) whille the_destruct01_ is necessary to have a destruction.

[taw]

Perhaps EF-lines arent "lines" but points, so every "line" is one point where a unit can take place.
With the uint32 0-3 i think you are right that, this is some sort of depth multiplier.
For example on Fort walls units are staying in 3 rows. Regarding forts. Did you find any possible entries for cannons on walls?

The piece vw destruct xy is a naming convention used for all buildings and buildinglike structures.

For example applebarrel_piece01_destruct01

Perhaps the engine is using the _piece01 part to determine if there's any other building piece which needs to be attached there (eg townhalls often have severel pieces) whille the_destruct01_ is necessary to have a destruction.

I attached extracted data in convenient plain text format in one of previous posts.
Just take a look and maybe you'll notice something interesting.
I don't really know much about 3D models in Total War games, so I won't be of much use here.

[alhoon]

Ehh...

How we use that along with the DB editor? We put it in the same folder or what?


EDIT: Damn, can't give you rep so soon again.



Any progress with the missions tables? That would be cool.

[taw]

Ehh...

How we use that along with the DB editor? We put it in the same folder or what?

Yes, replace existing one. You will see more tables if it worked.

Any progress with the missions tables? That would be cool.

Aren't missions moddable with lua already?

[alhoon]

I don't know what lua is, so I don't know at all.

[taw]

Go to data/campaigns/XXX/scripting.lua - it's not even packed or compiled.
Especially look at episodic_3.

You will see code that controls missions in each episode there - what triggers
them, how they're won and lost etc.

DB contains copy of this data in triggers tables, but it is *not* used,
it's converted to lua during ETW compilation process, and ignored afterwards.
(well, more like Access database Empire.mds gets converted to both,
triggers in DB are just an accidental leftover)

Lua is simple programming language - many games use it, and there's
plenty of examples, books, and everything about it.

[alhoon]

DB contains copy of this data in triggers tables, but it is *not* used,
it's converted to lua during ETW compilation process, and ignored afterwards.

So... how you change the script of ETW?


AND:

" elseif conditions.TurnNumber(context) == 5 then
scripting.game_interface:enable_auto_generated_missions(true)
end
"

Auto generated missions?
What missions? In all my games as Prussia, Persia etc, I never had a mission. I didn't even know there were missions out there until I tried France for fun once and found that there was a mission about Louisianna.

And what about that:
"-- For a list of all events supported create a "documentation" directory in your empire directory, run a debug build of the game and see
-- the events.txt file"
Has anyone the events.txt file handy?

If not, how can I run a "debug" build of the game?

[taw]

So... how you change the script of ETW?

AND:

" elseif conditions.TurnNumber(context) == 5 then
scripting.game_interface:enable_auto_generated_missions(true)
end
"

Auto generated missions?
What missions? In all my games as Prussia, Persia etc, I never had a mission. I didn't even know there were missions out there until I tried France for fun once and found that there was a mission about Louisianna.

Random missions are disabled by default. Hardly much loss, they sucked in RTW and M2TW.

Look at scripting.lua in episodic* directories, you have real missions there (for tutorial).

And what about that:
"-- For a list of all events supported create a "documentation" directory in your empire directory, run a debug build of the game and see
-- the events.txt file"
Has anyone the events.txt file handy?

If not, how can I run a "debug" build of the game?

This is internal CA note, look at events table in db editor. Documentation is incomplete,
browse random posts and mostly look at existing scripting.lua

[alhoon]

Thanks mate. So I'll have to back engineer, right? And some of the events are in the db editor, but not all.

PS. I liked the missions in BOTH M2TW and RTW.

EDIT: So
"protectorate_region_capture" is for "Capture Regions" according to the mission tables. Why is the protectorate in there then?

[taw]

Thanks mate. So I'll have to back engineer, right? And some of the events are in the db editor, but not all.

PS. I liked the missions in BOTH M2TW and RTW.

Take ally's region, receive 1000 denari? There's a reason they were disabled.

My mistake. Check trigger_events_tables. events_tables is actually for historical event messages,
not for those that do anything.

trait_triggers_tables + trigger_effects contain list of event handlers that do something.

Like this:
"C_Rake_Assassin_Good_Trigger"
"AssassinationAttemptSuccess"
"[CharacterType(\"rake\") or CharacterType(\"assassin\")] and not CampaignName(\"episodic_1\") and not CampaignName(\"episodic_3\")"

It says C_Rake_Assassin_Good_Trigger should be fired if assasination attempt succeeded (with context lua argument being relevant character), and only when it's rake or assassin, and not in episodic campaigns (=tutorials).

Then these two:
"2085848218" "C_Rake_Assassin_Good_Trigger" "C_Rake_Assassin_Good" 1 30
"-2109226912" "C_Rake_Assassin_Good_Trigger" "C_Rake_Naughty_Club" 1 4

Mean there's 30% chance for +1 to C_Rake_Assassin_Good trait,
and 4% chance for +1 to C_Rake_Naughty_Club. (some traits have multiple levels).
Pretty much the same stuff as in M2TW.

Now this is not used directly - it's actually compiled to lua. In patch*.pack you'll see a few files like:
export_triggers.lua with this inside (strangely numbers are different, my samples are probably from different patch*.pack version):

--[[ C_Rake_Assassin_Good_Trigger ]]--

events.AssassinationAttemptSuccess[#events.AssassinationAttemptSuccess+1] =
function (context)
if (conditions.CharacterType("rake", context) or conditions.CharacterType("assassin", context)) and not conditions.CampaignName("episodic_1", context) and not conditions.CampaignName("episod
ic_3", context) then
effect.trait("C_Rake_Assassin_Good", "agent", 1, 67, context)
effect.trait("C_Rake_Assassin_Good", "agent", 1, 17, context)
return true
end
return false
end

I recommend getting familiar with event / trigger system on these.
Just add lines like that to your campaign/main/scripting.lua with some new events.
Don't edit anything in db, or these patch*.pack

Once you understand how it works, you can move to missions.
But don't start from missions, seriously. One step at a time.

(in all files I mentioned are ridiculous number of examples to look at)

[alhoon]

Yeah... I noticed.

My problem is that I changed a line in the script and it didn't take effect. I'm using DMUC and I can't figure which of all the campaign folders I use. Anyway, thanks.


This is what I use:

Spoiler Alert, click show to read: 

local function OnFactionTurnStart(context)
if conditions.TurnNumber(context) == 1 then
if conditions.FactionName("britain", context) and not conditions.FactionIsHuman("britain", context) then
scripting.game_interface:grant_faction_handover("prussia", "france", 6, 15, context)
end
if conditions.FactionName("britain", context) and not conditions.FactionIsHuman("britain", context) then
scripting.game_interface:grant_faction_handover("prussia", "france", 1, 1, context)
end

end

end

In all my tests so far, Prussia didn't absorb France in the 1st turn.

[Cursisanafou]

those numbers mean not before 6 turns and no further than 15.... so anywhere between the 6th and the 15th turn prussia will absord france.

i think the first may simply negate the first...

Also I don't know if you can have a turn number connected to those statements...

basically you are saying if it's the first turn, prussia will abosorb france between turn 6 and turn 15.... but it is only ever checked on the first turn... so really would never ever happen.

0,0,context will grant the faction handover immediately.... although i think you have to put it before the turn check... which is the same if you use what you have...

you can't have a check on the first turn to determine whether it gets handed over between the 6th turn and the 15th turn... because it would never check again...



Although i could be wrong, and what you will have to do is wait anywhere up to 15 turns to see if the script works.

[taw]

If you connect OnFactionTurnStart function to event handler (it's probably in last 10 lines of .lua file), it will be called on every faction's every turn start.

If it's 1st turn of non-human britain, it will call once scripting.game_interface:grant_faction_handover("prussia", "france", 6, 15, context)

scripting.game_interface:grant_faction_handover("prussia", "france", 6, 15, context) exists outside lua world.

Maybe it has some "faction handover schedule" for which 6 and 15 are turn numbers, or maybe something else.
In such case you only need to put it on schedule once.

No idea if calling scripting.game_interface:grant_faction_handover for same factions multiple time is supported or not.

[alhoon]

If you connect OnFactionTurnStart function to event handler (it's probably in last 10 lines of .lua file), it will be called on every faction's every turn start.

If it's 1st turn of non-human britain, it will call once scripting.game_interface:grant_faction_handover("prussia", "france", 6, 15, context)

Yeap, the point of my testing was to not hinder the whole thing down with a test script.

scripting.game_interface:grant_faction_handover("prussia", "france", 6, 15, context) exists outside lua world.

What do you mean "lua world"?

Maybe it has some "faction handover schedule" for which 6 and 15 are turn numbers, or maybe something else.
In such case you only need to put it on schedule once.

In case it means 6-15 turns, I put the 0,0 thing
In case the numbers mean something else, I copy pasted it as it was (For England and 13 colonies) changing just the factions. That's why I put it there twice.

Neither one worked though.