Empire Mod Architecture Primer

[MrThib]

The patch format is working well with the current editor, I've posted a tuto on mod making with this format if you want:
http://www.twcenter.net/forums/showt...76#post4738876

[notger]

Great work guys and thanks for the info, Jerome.
Good to see thing get rolling. I am looking for the first recruitment threads that look for coders.

[Delphy]

Thanks for the explanations! I have to say that I'm a little confused as to exactly how the unit fragments are supposed to work though. You state:

Now, a note on how to work with the databases. You’ll have noticed that each table in the databases (inside the db folder) is both a folder and a binary file. When the game instantiates the databases, which happens after the virtual file system is built, it scans each directory and parses –all– of the binary files it finds. This means that a mod pack can add a new binary data fragment inside the db/unit_stats_land_table folder which adds several new units, and the game will add them to its land units database. The fragments are processed in the same order as the pack files, and any new database records with the same name as an old record will cause the new to replace the old. This means that you can replace any record within the database without touching the rest of the data or modifying the CA-supplied files on disk.

For example, say the CA packs contain a unit record for ‘french_dragoons’, which is one entry out of hundreds inside the shipped unit_stats_land_table data fragment, which is used to build the unit table that was available at launch. Now say you create a mod pack which places a new data fragment, say called my_mods_unit_stats_land inside db/ unit_stats_land_table, which contains just one data record for ‘french_dragoons’. What will happen is your new record will end up replacing the current record for that unit in the game data, and the rest will stay as it was originally.

You don't mention if the individual files inside those directories should be the same though. For example, in the release, the only file in that folder is unit_stats_land, which itself contains the actual unit stats.

From reading your description though it sounds like you can have different tables within this folder (say, unit_stats_land2), each containing units. Is this correct?

What if a mod.pack instead just had a db/unit_stats_land_tables/unit_stats_land file that contains 2 or 3 units. I assume this would just get appended onto the existing unit_stats_land table (so long as nothing got overwritten of course)?

Thanks
Delphy

[MrThib]

What if a mod.pack instead just had a db/unit_stats_land_tables/unit_stats_land file that contains 2 or 3 units. I assume this would just get appended onto the existing unit_stats_land table (so long as nothing got overwritten of course)?

I'm interested in this too, needs some testing here.

You don't mention if the individual files inside those directories should be the same though. For example, in the release, the only file in that folder is unit_stats_land, which itself contains the actual unit stats.

From reading your description though it sounds like you can have different tables within this folder (say, unit_stats_land2), each containing units. Is this correct?

This is correct, I've try it and it worked fine, same thing with the namings, only the folder count, the binarie could have the name you want it doesn't matter.

[JeromeGrasdyke]

Thanks for the explanations! I have to say that I'm a little confused as to exactly how the unit fragments are supposed to work though.

You don't mention if the individual files inside those directories should be the same though. For example, in the release, the only file in that folder is unit_stats_land, which itself contains the actual unit stats.

From reading your description though it sounds like you can have different tables within this folder (say, unit_stats_land2), each containing units. Is this correct?

What if a mod.pack instead just had a db/unit_stats_land_tables/unit_stats_land file that contains 2 or 3 units. I assume this would just get appended onto the existing unit_stats_land table (so long as nothing got overwritten of course)?

Ok. If you were to make a mod which supplies a new file called "db/unit_stats_land_table/unit_stats_land" it would override the one from the CA packs. Yours would be used, the CA one would not. That's file-level overriding, and it's not something that you normally would want to do to the database files (although I imagine it'll be pretty common with data files like tga's and so on).

Instead, you should put in a file named something like "db/unit_stats_land_table/delphy_mod_unit_stats"... the game should then pick it up, and *add* any records that are in your file to the ones that are in the CA-supplied main one. And then if there are any records with the same name in the two files, the one that gets loaded last (the mod one) will be what's used. It'd be a good idea to try and pick unique, relevant names for the files your mod adds, as well.

As I said though, until the tidying pass on the modding systems here at CA is complete, this is strictly early-bird access for dedicated modders.

[alpaca]

Ok. If you were to make a mod which supplies a new file called "db/unit_stats_land_table/unit_stats_land" it would override the one from the CA packs. Yours would be used, the CA one would not. That's file-level overriding, and it's not something that you normally would want to do to the database files (although I imagine it'll be pretty common with data files like tga's and so on).

Instead, you should put in a file named something like "db/unit_stats_land_table/delphy_mod_unit_stats"... the game should then pick it up, and *add* any records that are in your file to the ones that are in the CA-supplied main one. And then if there are any records with the same name in the two files, the one that gets loaded last (the mod one) will be what's used. It'd be a good idea to try and pick unique, relevant names for the files your mod adds, as well.

As I said though, until the tidying pass on the modding systems here at CA is complete, this is strictly early-bird access for dedicated modders.

Thanks again for your efforts Jerome, we really appreciate it

Replacing DB files will be common for some of them, e.g. trait_level_effects, which can have multiple entries per ID because the default behaviour here isn't replace the effect but add a second effect. Will the logging system be enabled (hint! please tell us about logging) and tell us when we have a non-unique name?

And can you at least give us an order of magnitude for when the "tyding pass" will be finished? Are we talking about weeks, months or years?

[aronnov]

awesome. Posts like this from CA makes me love them that much more.

[DARTH VADER]

Very clarifying. This will help us a lot. Thanks to CA and Mr. Jerome for the community interaction.

[CtrlAltDe1337]

Thanks for the info Jerome and Lusted

[LtChambers]

Now, a note on how to work with the databases. You’ll have noticed that each table in the databases (inside the db folder) is both a folder and a binary file. When the game instantiates the databases, which happens after the virtual file system is built, it scans each directory and parses –all– of the binary files it finds. This means that a mod pack can add a new binary data fragment inside the db/unit_stats_land_table folder which adds several new units, and the game will add them to its land units database. The fragments are processed in the same order as the pack files, and any new database records with the same name as an old record will cause the new to replace the old. This means that you can replace any record within the database without touching the rest of the data or modifying the CA-supplied files on disk.

For example, say the CA packs contain a unit record for ‘french_dragoons’, which is one entry out of hundreds inside the shipped unit_stats_land_table data fragment, which is used to build the unit table that was available at launch. Now say you create a mod pack which places a new data fragment, say called my_mods_unit_stats_land inside db/ unit_stats_land_table, which contains just one data record for ‘french_dragoons’. What will happen is your new record will end up replacing the current record for that unit in the game data, and the rest will stay as it was originally.

A bit more elaboration on this topic please:
1) Is it possible to delete records in the vanilla db file from a db fragment file?
2) With regard to: "any new database records with the same name as an old record will cause the new to replace the old" - some of the junction tables do not have any column with unique values, i.e. there are many-to-many relations as well as one-to-many. In many-to-many cases (like pairs of strings), the "name" would be multiple columns (like both strings of a pair). Is this accounted for by the "overwrite" logic?

Thanks again for the info.

[l33tl4m3r]

Lt - in regards to question 1, I have found out that no, so far, you cannot remove lines of entry from vanilla files with a fragment file.

It does tie into your second question however, as in order to remove things, you can make a mod patch that contains a table named just as the table is named in the vanilla. Inside, make sure it contains ALL the data from the vanilla pack, MINUS what you wish to remove.

The mod pack table will overwrite the table in the vanilla packs so long as the directory structure is correctly pointed and the mod pack table is named the same as the vanilla pack table name.

[alpaca]

As for the second question: It's at least not overwritten directly. For trait_level_effects for example the behaviour is the same as if you wrote two effects in the same file, i.e. an effect is added to the trait

[MrThib]

@alpaca
For removing traits effect you should try to negate their activation with some scripts like:
IsFactionLeader() and CampaignName("Blah")
in the traits_triggers, as the CampainName () is already used by the game.
It's just a trick through and if your question was more theorical about deleting entries in general with mods that doesn't answer at all..

[PROMETHEUS ts]

Great I am looking foward for the MDK ....

[Ketzerfreund]

l33tl4m3r, since your tracer pack is a patch-variant, it's not bound to the user script, anyway.
What I wanted to know is why using the mod bit, not the patch bit, set in my no-tracers-pack with using the user script doesn't work for me. Setting the patch bit works for me, but for that no user script is needed, since patch packs override release packs and vanilla's patch.pack doesn't contain a replacement for main.pack's projectiletrails.fx.

So, tanks for helping, but your no-tracers-pack doesn't really answer my question.

[Delphy]

Thanks for the clarification Jerome.

[LtChambers]

As for the second question: It's at least not overwritten directly. For trait_level_effects for example the behaviour is the same as if you wrote two effects in the same file, i.e. an effect is added to the trait

If you changed one of the two strings in the junction pair then you effectively made a new key - so you can't expect it to overwrite a previous one. What you want to do is delete, not overwrite. Overwriting would be relevant if there are more than 2 columns and you want to change the non-id-related columns.

So if the vanilla row was like:

idRef1      idRef2       5

A mod row like:

idRef1      idRef3       5

is a new row.

But a mod row like:

idRef1      idRef2       10

should overwrite the vanilla row.

[Ketzerfreund]

Got the method working now, but only by using the txt file in MrThib's tutorial, which is far bigger than it should be - normally, a txt file is a txt file is a txt file. It's size in byte is usually the same as the number of characters in it.

That only MrThib's txt file works with it's size overhead doesn't make the slightest bit of a sense.

[l33tl4m3r]

l33tl4m3r, since your tracer pack is a patch-variant, it's not bound to the user script, anyway.
What I wanted to know is why using the mod bit, not the patch bit, set in my no-tracers-pack with using the user script doesn't work for me. Setting the patch bit works for me, but for that no user script is needed, since patch packs override release packs and vanilla's patch.pack doesn't contain a replacement for main.pack's projectiletrails.fx.

So, tanks for helping, but your no-tracers-pack doesn't really answer my question.

I apologize, that's totally my fault.

Not sure if you noticed in the other thread, my no-tracers is actually lighter tracers.

Open up the projectiletrail.fx from my mod and empty out the functions.

--

The reason I have it set to mod is because I am making all my changes into mod packs, it should help standardize the modding method if we all start using a generic template for mod creation.

And, as I understand it, so long as the patch is set to a mod and properly loaded, then it should overwrite or modify ANY data tables, so long as the directory structure is the same.

--

Let me know if this helped, if not I am sure I am misunderstanding, so feel free to try ti explain it to me again, lol! I won't be offended, I promise.

[Ketzerfreund]

Umm... You actually didn't say anything regarding my actual problem at all.
But maybe I expressed myself a bit awkwardly.

I've narrowed the problem down to the user script txt file now.
Using notepad to write a new one doesn't work for me, the whole thing is ignored by Empire then. Only using MrThib's txt file as offered in his tutorial works. And it's far larger than it should be for a plain text file. So I've just taken a look at his file in a hex editor.

Lo and behold:


That's what it contains when I replace his mod's name by mine in his txt file.
Making a new one through the context menu opened when right clicking in the folder produces a file that really only has the characters in it that you write into it using notepad - a true pure text file.

So, what exactly is his text file? How was it created?