Official CA Q&A thread - XML and XSD modding

[CraigTW]

Fortunately I had a copy of this thread saved to my computer before the forum hiccups, so we're all good.

Anyway, here's the Q&A thread again. Don't worry that all of your questions have vanished - I'd been compiling them into a Word document anyway, so they're safe and will be answered after the weeked.

Thanks!

-Craig

--

Hi everyone,

As a follow-up to the database files we posted yesterday, we’d like to help with anything that’s unclear by answering any technical questions related to the files, and the data therein, which you may have.

Rather than pull individual team-members away from their work to constantly monitor the thread, we’ll do a daily sweep. We’ll then compile the questions, ask the relevant team-members, and ping the answers back in list form.

In all likelihood, some of the more experienced modders here will know the answers to certain questions that may appear… if you fall into that camp, please feel free to drop in and demonstrate your knowledge Also, please keep your questions here on the subject of modding with the files, as we’re not at liberty to go off-topic!

Cheers,

The Creative Assembly

[eye-opener]

Spoiler Alert, click show to read: 

Hello CraigTW,
like you probably know i am the one who helped in the total war forums with the issue of drophacking, who explained it to Dave the ninja
and who got as a present for the ikko ikki dlc for free (, which retainers
(like: way of ikko ikki, rennyos teachings and the one which gives the opponents hero and veteran units -2 attack, -1moral...)
are really imba, besides).

Nice to hear this:
"Previously, if you disconnected from Steam during a battle you could be kicked out of it. Now the game will allow you to continue playing your opponent through the direct connection while Steam attempts to reconnect in the background. This doesn't require a patch - it should be live right now."

But did you also consider my other proposal for solution, which helps in a more effective way against drophacking?
"2. Steam could also check (internally) if the client is online after it was blocked by the host. The client is always going to send some information to steam... and the rest you can think about -> That would help much!"
You know how i mean this?

I have some bad news(not really, easy fixable^^) because it seems that the alt-f4 fix is "buggy". I have discovered a new kind drophack, which i didnt have tested yet.

Just see this:
http://www.abload.de/img/internetonlinetu2e.jpg
http://steamcommunity.com/id/eye-opener/screenshots/
See the pictures, which i uploaded on Friday, June 17, 2011

, then you should recognize that there is something wrong with it, i am sure that my oppponent quited the game while being in the loading screen. The funny thing, like you should be able to see, is that steam didn't react to the fact that my opponent has quited shogun2 totally... and that the game was started some seconds before.
I am sure that you know (now) how to fix this

And I want to ask you about this:
http://www.abload.de/img/2011-06-16_00001ynv6.jpg
http://www.abload.de/img/2011-06-16_00002qupa.jpg
http://www.abload.de/img/2011-06-16_000036u1n.jpg
http://www.abload.de/img/2011-06-16_00004bniv.jpg
Is it fair when some cheap loansword ashigaru are able to have 30 attack? My lvl9 katana samurai have 24 attack and they cost 1200... IMBA
and the loansword ashigaru definitely dont cost more than 1000...
Pls think about this imbalance and tell me what you think

I hope i could help again and i look forward to hear from you soon^^

greetings,
eye-opener

P.S. concerning the imbalance, see this also pls: http://www.twcenter.net/forums/showthread.php?t=463236

[CraigTW]

Hi eye-opener. Thanks for that, I'll get it logged, but this is the wrong thread for it really.

Please keep all questions on the topic of modding the data we released yesterday.

[Bethencourt]

Hi!

1.- In unit stats naval we have these stats:

,mothball_cost, this one is called hull strength in pfm
reactivate_cost, this one is called crew size
repair_cost_port. This one does not appear in pfm despite it has value in xml file.
,repair_cost_sea. This one does not appear in pfm despite it has value in xml file.

Could you please tell us what they are for and how they work?

I have also seen these ones:

mass, collision_momentum_modifer, formation_distance, minimum_distance, formation_integrity

I wonder if they are related to the damage when ships collided in first version of empire and if we can mod to have that effect again. I have also noticed that ships avoid to collide, not as it was in first version. I also want to know if we can mod to have that behaviour back.

2.- In agent tables:

We have line of sight and zoc which I believe it is zone of control where fleets or armies intercept enemies approaching. The first one has worked when changed, but zoc changing has had no effect in the interception zone not making it wider or smaller. Is this stat working? Is any other stat or file involved I the change?

3.- In unit stats naval:

Armour, hit and critical related to hull, masts, sails, magazine. What is the effect of those stats and how do they work?-

4.- We have:

1st. In campaign_ai_personality_junctions :

NAVY_REPAIR_FULL_CREW_MINIMUM_PERCENTAGE

NAVY_REPAIR_FULL_HULL_MINIMUM_PERCENTAGE

NAVY_REPAIR_TOTAL_CREW_MINIMUM_PERCENTAGE

NAVY_REPAIR_TOTAL_HULL_MINIMUM_PERCENTAGE

2nd. In campaign_ai_manager_behaviour_junctions :

NAVY_REPAIR

Which all seem to be related to ship reparation, is this true? how do they work?


…..I will be adding more here-

Thank you.

[Primergy]

Got a question about groupformations and battle flags.

Is there a way to make the entity placement a square or atleast a way to rotate units?

Didn't find anything in the xml there, but i also saw xml's were in one element an option was missing but appeared in another, maybe there is an option but was not used?

In ETW it was possible to make rectangle flags, with an alpha map in the large.tga. In NTW this isn't possible anymore since as soon as you enable the graphic option volumetric effects the alpha area is simply black.

Was there a shader file (the fx ones) changed from etw to ntw?

Atleast i managed to get back the alpha when enabling SSAO (which didn't looked to well)

[uanime5]

In the regions.xml there is a "<palette_entry>" before the RBG entries. What is this entry used for?

[The Hedge Knight]

Hey is there any change we could get some deeper explanations of campaign ai managers behavior junctions and campaign ai personalities junctions as these two tables were sadly lacking the helpful comments that kv rules had .

Also is there any chance you could release the format of the packed sound files similar to the way you released the format of the groupformations files along with how they work? Just because its a bit annoying that we can make a totally new unit which them insists on shouting 'REVOLUTIONARY GUARDS' with no good way to stop it.

Also what is 'ZOC' in building levels? Zone of control or somthing else?

Finally, dare i say it any change of the same being done for shogun.
If not, perhaps this should be moved to the ntw or etw mod workshop .

Thanks for this initiative Craig, its awesome .

[B-DizL]

Is the data responsible for the deployable defenses located in these files, specifically relating to the width that they are deployed i.e the trenches and spikes, also are the firing arcs of the cannon using the sand bag deployables defined there?

[King Sama]

In models_naval the following is an entry: <logic_folder>ships\1deck24\</logic_folder>
What folder does this refer to, and where can we find it? Does it hold any relevance to the placement of cannons on a ship?

There is an entry called "fitness" in the units_tables, yet I do not see the column in the PFM. There are also various files, like the famous mercenary file, which seem to be db files in your documentation yet I have not seen anywhere in the pack. Are these supposed to appear, or were they left out of the final game?

This may be a bit unrelated, but is there anyway to edit the characteristics of specific unit formations, such as light infantry behaviour or the square formation, and where?

Thanks for this, very much appreciated

In ETW it was possible to make rectangle flags, with an alpha map in the large.tga. In NTW this isn't possible anymore since as soon as you enable the graphic option volumetric effects the alpha area is simply black.

I'm not sure about when you use volumetric effects, but it is certainly possible to make square flags without them. The swiss flag in TGW uses alpha.

[Okmin]

Question for CA: You gave us XSD and XML files, but in the actual .pack files the DB uses neither file type. So my question is, how did you convert the XML into something the engine can use? And is there any chance you'd release whatever converter you used (if any) to save us from updating the slightly buggy DB editor? (Would probably attract more RTW/M2TW modders if we could just edit the XML files and convert them, too)

1.- In unit naval stats we have these stats:

,mothball_cost, this one is called hull strength in pfm
reactivate_cost, this one is called crew size
repair_cost_port. This one does not appear in pfm despite is has value in xml file.
,repair_cost_sea. This one does not appear in pfm despite is has value in xml file.

Actually I think I help with part of that... I was going through and updating the DB schema and noticed that in some of the larger tables, the order of the columns in the editor doesn't match the order of the <tags> in the XML. So you sometimes need to double-check that the values in the XML and DB match to know if the column corresponds to that XML entry (i.e. if the value in the XML is 3500 and in the DB is 10, the order's different, so you'd need to look for the DB entry that says 3500).

And yeah, some of the things in the XML files don't show up Might be because you're using PFM instead of DBE and the latest DB.xsd.

[alhoon]

Could you please tell us (even by the top of your head if needed) which of the tables are implemented and which are leftovers? That would help I think. Thank you.

[Steph]

1) Is the Unit_Regment_Names table used?
2) I've seen a new Unit_Special_Ability table, not exposed in PFM yet. It contains things such as column_formation, loose_formation. Is it usable in NTW, or a left over from before?
3) What is the effect of Marksmanship bonus in the projectile table?

[Primergy]

I'm not sure about when you use volumetric effects, but it is certainly possible to make square flags without them. The swiss flag in TGW uses alpha.

I know, i'm also using Alphas, but it annoys me that it people are forced to play with a certain option unused, while they could from their hardware power.

2) I've seen a new Unit_Special_Ability table, not exposed in PFM yet. It contains things such as column_formation, loose_formation. Is it usable in NTW, or a left over from before?

Looks like leftovers, since it my game chrashed when using.

[LEGIO_Desaix]

1)
Aim: to get customized name for building in custom map.

Can we add new building type in the BATTLEFIELD_BUILDING_CATEGORIES and by cloning an existing building in the (battlefield_buildings table) have it displayed with a customized name adding a new entry in the localization file.
Example: I add a new type called hougomont in the BATTLEFIELD_BUILDING_CATEGORIES table, then add a new line in BATTLEFIELD_BUILDINGS table cloning the one I'm using on my Waterloo map, add a new line in localization file using the new type hougomont with description Castle of Hougomont.

Are building type fixed or can it work? Maybe is there another table to change?

2) Column bonus in KV_MORALE table? Does it work? Does it work for group formation deployed in column? Is just a fake line never implemented (in that case please...please implement a formation column with a patch or suggest a way to have a proper column formation?

3) What are KV_..._EXTENDED tables for?

4) Does CAPTURE_POINT battle work in NTW (battles table)? I saw we can add capture location in the map file called capture_location_list, example from mountain map in patch4 of NTW

<CAPTURE_LOCATION_LIST>

<CAPTURE_LOCATION radius='16.441118' position_x='4.672912' position_y='-327.224396' />

<CAPTURE_LOCATION radius='15.062731' position_x='4.214892' position_y='-568.989624' />

<CAPTURE_LOCATION radius='24.438301' position_x='-507.876495' position_y='-474.617493' />

<CAPTURE_LOCATION radius='21.166296' position_x='510.104462' position_y='-481.503113' />

<CAPTURE_LOCATION radius='27.467758' position_x='3.494706' position_y='352.693787' />

<CAPTURE_LOCATION radius='22.074589' position_x='-485.780273' position_y='129.968460' />

<CAPTURE_LOCATION radius='24.484919' position_x='483.504913' position_y='134.959091' />
</CAPTURE_LOCATION_LIST>

Can we play a capture point battle in NTW custum battle or, when negative, is there any line I can add in the xml file setting armies OOB in a mp_historical_battle type.

[T.C.]

In models_naval the following is an entry: <logic_folder>ships\1deck24\</logic_folder>
What folder does this refer to, and where can we find it? Does it hold any relevance to the placement of cannons on a ship?

The logic files are there somewhere, I've seen them myself. They control rigging I think.

There is an entry called "fitness" in the units_tables, yet I do not see the column in the PFM. There are also various files, like the famous mercenary file, which seem to be db files in your documentation yet I have not seen anywhere in the pack. Are these supposed to appear, or were they left out of the final game?

They've said it before, they were left out.

This may be a bit unrelated, but is there anyway to edit the characteristics of specific unit formations, such as light infantry behaviour or the square formation, and where?

I'd put my house on them being hardcoded. In three years I've never seen any softcode to indicate otherwise.

Question for CA: You gave us XSD and XML files, but in the actual .pack files the DB uses neither file type. So my question is, how did you convert the XML into something the engine can use? And is there any chance you'd release whatever converter you used (if any) to save us from updating the slightly buggy DB editor? (Would probably attract more RTW/M2TW modders if we could just edit the XML files and convert them, too)

Actually I think I help with part of that... I was going through and updating the DB schema and noticed that in some of the larger tables, the order of the columns in the editor doesn't match the order of the <tags> in the XML. So you sometimes need to double-check that the values in the XML and DB match to know if the column corresponds to that XML entry (i.e. if the value in the XML is 3500 and in the DB is 10, the order's different, so you'd need to look for the DB entry that says 3500).

They use their own tools to read from the xml's/xsd's. There's not much chance of us getting our hands on that. I've updated pfm for NTW by hand, I'll move on to ETW after the weekend (for DBE). If you want to d/l my files and copy them over to the DBE schemas, then do. I've sorted out many issues with columns not matching up etc so transferring it to the DB xsd should be easy.

3) Whar are KV_..._EXTENDED tables for?

If you look at the updated schema I posted you will see the value for the morale_ext file, not sure about the others.

Link to my thread http://www.twcenter.net/forums/showthread.php?t=463644

Perhaps we should have a discussion thread somewhere else on what to ask here? We are answering our own questions more often than not here.

[Yarkis de Bodemloze]

2) Column bonus in KV_MORALE table? Does it work? Does it work for group formation deployed in column? Is just a fake line never implemented (in that case please...please implement a formation column with a patch or suggest a way to have a proper column formation?

I tested that and it does not work. There is a column button in the UI somewhere, so probably CA intended to add a column formation but scrapped the idea.

[The Hedge Knight]

I tested that and it does not work. There is a column button in the UI somewhere, so probably CA intended to add a column formation but scrapped the idea.

Yeah if you look in unit special abilities in hex you can see it was intended as an ability.

[LEGIO_Desaix]

Thanx guys, of course we all know column_bonus does not work, my question was just a humble try to let CA know this is a crucial point coupled with unmoddable square to fix to enhace the true napoleonic tactical warfare. Only devs might know if there is a shortcut through tables to add a bonus to column or delay square bonus until fully formed. We all know this is business first, they had little time to waste on revising NTW or Empire engine while working on SH2. Nonetheless this is the only period before they commit to a new title when we can have some hope to see some bug fixed or some features implemented. Six months more and both Empire and NTW will be history for CA, if not now already.

[Primergy]

Can we play a capture point battle in NTW custum battle or, when negative, is there any line I can add in the xml file setting armies OOB in a mp_historical_battle type.

I would be also interested in this. If negative, would it be possible let the game load a script in historic mp battles, which works for all players and not only for the host?

The logic files are there somewhere, I've seen them myself. They control rigging I think.

There are other logic files, which got baked into the DB, which are responsible for cannon and soldier placement on ships.
The same goes for the model_buildings, there is also a logic file baked into. Here the logic files are simple spline knot-coordinates.

Question Regarding Buildings:

Are the xyz.tech.cs2 files, which are used for collision, allowed to be modded, or do they belong to a middleware package, like the Speedtree trees?

I agree, better to open a topic where we can answer question first and then forward to CA in case nobody has the solution.

Well, we have something like this.. it's called Mod workshop .

[Steph]

Well, we have something like this.. it's called Mod workshop .

And I asked several questions there, with no shadow of a hint of an answer... Hence my questions in this thread.