Page 1 of 2 12 LastLast
Results 1 to 20 of 40

Thread: Warscape Modding Primer

  1. #1

    Icon1 Warscape Modding Primer

    Warscape Modding Primer

    This is a short introduction into Warscape modding. This should be valid for all Warscape files, i.e. ETW, NTW, TW:S2 and TW:R2.
    I'll start with some fundamental concepts of the engine; that is knowledge the modders gained soon after the release of Empire, but I've rarely seen it explained again so my guess is most people's understanding of it is at best fuzzy.

    Beyond the foundations, I'll lose some words on the db files because that's about all I have any experience in.



    Warscape's virtual file system

    Like every piece of software, WS games are divided into the actual software - i.e. executable code - and data. If you browse to your game installation directory (somewhere along steam/steamapps/common/), you'll see Rome2.exe file; this is the executable; no modding takes place there. What you can also see is the data/ subfolder, which is where the magic happens.
    Open it and you will see a number of .pack files - these contain the actual data. To open them, you'll need the PackFileManager; assuming you have downloaded it, use it to open any .pack file.
    What you see on the left is what looks like a folder structure in the Windows Explorer - because that's what it is: it creates virtual subfolders of the installation's data/ directory.
    Each pack contains a subset of the complete game configuration data, like movie, sound, music, and finally data; you can tell by the file name most of the time.

    Additionally, each pack has a certain type: Boot, Release, Patch, Mod, Movie. This is used by the game to determine the order in which they are loaded: namely in exactly that order, first Boot, last Movie. CA's packs are of type Boot, Release, Patch and Movie.
    For files that appear in two .pack files, the one that is last loaded is the one that will be used by the game. This is how CA patches their games: they replace data from the original pack by uploading one containing the same file in a Patch .pack which will overwrite the old one in the Release pack.

    If you want to replace data from the vanilla game (like UI icons), you do that in exactly the same way: create a pack loaded after the Patch files containing the files overwriting the old ones.

    Now, if you want to add data to the game (for instance, new units) what you can do is just create a pack containing the file you want to add, and the game will load it.
    You do have to follow certain rules or the game won't know what to do with it; a good guess is to create the file in a folder within the pack that already exists. Make sure to not use the name of a file that is already contained in one of the existing packs or you will overwrite it.



    Example Browsing data_rome2.pack, you find entries in ui/units/icons/

    In your mod pack, you would also create that directory structure and then put your icons there.
    Say you don't like the Pretorians', you can replace the existing one by adding your png as rom_praetorians.png; if you created a new unit and want it to have a unit icon of its own, you might add it as my_little_pony.png.


    Your own virtual files

    So in general, to make managing your mod easier, you also want to work on a virtual file structure. PFM allows you to do that by using the File->MyMod feature.
    When you select “New”, the program will ask you for a root directory of your new mod; this is where it saves your pack and stores (and retrieves from) all your working data.



    To continue the example above:

    Example You create a new MyMod “Icons” in c:\Me\Mods\Icons.

    You can then open data_rome2.pack (it asks you for a pack to open when creating the new pack), select the rom_praetorians.png, and then Files->Export.

    You will find that your mod folder now contains the whole directory structure ui\units\icons containing the exported png you can now edit. Here, you also can now create your my_little_pony.png icons.

    Once you're finished with the icons, you want to create your own pack file with File->New (or open one you already created; you have to close rome_data2.pack is the point), then select Files->Import->Add and choose your pngs which will add them to your pack you can now save.
    You can always open the Windows Explorer at your Mod directory by selecting MyMod->Open Data Path.


    Installing your Mod and more about pack types

    To install your mod, you'll still have to copy your pack into the game's data/ directory (Game->Open Data Directory will give you a Windows Explorer there). If you followed the steps above, you will not see the new icon even if you do copy the file and start the game.

    The reason for that is that when creating a new pack with PFM, it is of type “Mod”. These will not be loaded automatically by the game, but need an entry in a file called user.script.txt which is located below your user directory (depending on Windows version; you can open an Explorer there via Game->Open Script Directory). This script file is needed by the game to determine the order in which Mod packs are loaded.

    To register your mod, you need to create or open “user.script.txt” and enter a line
    mod MyIcons.pack;

    then save it with Unicode encoding. The last point is very important or the game will not read the file! To save in Unicode, you can select the “Advanced” option in Notepad and then select the encoding.

    Alternatively, you can also use My Mods->Install MyIcons which will both copy the pack and add the required entry to the script file.

    Also, to play the game with a certain set of mods you can use the ModManager by Mitch which will list all Mod-type packs in your data directory and lets you select the ones you want to run; it will then create the appropriate entries.

    Then, there exists the possibility to make your mod of type Movie; these mods are always loaded by the game and don't require a script file entry. I view this as somewhat as the Dark Side because it's easier and faster, but will later on create problems with data dependencies and pack file load order.

    There are new pack types for Sound and Music files too, but those are new with Rome 2 and I don't have any experience with them; I'd assume they're loaded like Movie files. If someone figures it out, let me know.

    A word of warning: technically, you can also edit the CA packs directly, despite the stern warning from PFM that you shouldn't.
    This is a very bad idea for a reason you should be able to figure out by yourself from what I've already written.
    As a hint, think about what will happen when CA next patches their game.


    On DB files

    As promised, here some details on the db files; for Rome 2, these can be found in data_rome2.pack.

    The DB files you'll find in the CA packs like

    db/<table_name>_tables/<table_name>

    If you select one of those files, a table view of the contained data opens on the right side; those table contain most if not all of the configuration data for the game.
    However, it is not entirely clear for all tables how they work and interact with each other; some of the modders have experience with many of them, but there are few tutorials around to explain them in detail.
    If you're looking for one, it's probably a good idea to search in the modding subforums for all the warscape games because the structure stayed more or less similar.

    BTW, the data is decoded from binary data contained in the pack; the fields (i.e. columns) that are displayed and their type are configured in so-called “schemas”. These configurations are user-generated and not guaranteed to be correct; if you find a mistake, let me know.
    Also, these are updated from time to time to account for identified issues that were fixed. Check DB Descriptions->Search for Update from time to time; you can also choose to have PFM do this regularly upon startup in the Options menu.

    For DB tables, there are additional rules to adding and overwriting data from vanilla files. Most tables contain fields considered “primary keys”, meaning that those are uniquely identifying an entry. If you add a row with the same primary key as a vanilla entry to a db table, it will overwrite the vanilla one.



    Example The table _kv_experience_bonuses in db/_kv_experience_bonuses_tables/_kv_experience_bonuses contains an entry (first_to_enter_enemy_fort=200) . If you were to add a file db/_kv_experience_bonuses_tables/_my_bonuses, containing the entry (first_to_enter_enemy_fort=500).

    On a file level, it will be added side-by-side as described above; but the value of the database entry with the primary key first_to_enter_enemy_fort from the original file will be replaced by the content of the new file, i.e. 500.



    The common workflow to edit DB data is to first export the files you want to edit, then create your file, then import them into your file – pretty much as described in the second example above. After you have changed the values you want, you can then use the PFM feature “Postprocess->Minimize DB Files” which will remove all entries that have not been changed from the original files. You can also export a table to a .TSV file and edit it in a Spreadsheet program like OpenOffice or MS Excel.

    There also are restrictions to what values can be contained in some fields, because they are bound to data in another table. Some of those are know in the schemas, and you can make use of that by selecting the checkbox “Use Combobox Cells” when you have a db table open. This will replace the editor for text in the fields by drop-down comboboxes containing only the allowed data which prevents you from typos and gives you a better overview of the relations between tables.



    Example If you open the table agent_to_agent_attributes, you can see a column “attribute” which is a reference into the agent_attributes table's field “key”; the column “agent” references agents:key.

    If you select “Use Combobox Cells”, you'll see the drop-down controls for those columns appearing.

    If you right-click the header of a table with a reference, you can see the name of the referenced table in the context menu (Remove reference to <X>).




    That's All Folks...

    So there you have your Warscape Modding Primer, hope it gives you a bit of orientation on Warscape modding.
    I'm sorry I can't answer any questions about “How do I mod X” because my knowledge of what most tables do is pretty superficial; I've stumbled over a lot in the tables but I've not tried out a lot whether they do what I suspect.

    I'd love to see a lot more people who know what they're doing put up tutorials, but all of our time's limited I guess.

    Have fun modding



    EDIT: Here's a really good TL;DR step-by-step howto by crzyrndm:
    Quote Originally Posted by crzyrndm View Post
    Step 1: Add a new personal mod in PFM
    File -> MyMods -> New. Enter the name and location that you want to store it.
    -- When you have a mod selected under MyMods, extracted files are automatically added to the specified mod pack

    Step 2: Import required files into your mod
    The next step after naming and locating your mod, is to choose what it contains. Open the relevant official pack (more than likely: data_rome2.pack) and PFM will pop out a new window for you to choose the files from. Double click on the files you want to include in your mod, and hit OK when you are done.

    Step 3: Make your changes

    Step 4: Install your mod
    PFM has a great feature where it can manage the user.script.txt entries and add your mod.pack to the Rome2 data folder for you.
    File -> MyMods -> Install ____ (depending on which of your mods you have selected)
    Last edited by daniu; October 09, 2013 at 08:53 AM.
    Tools: PFM 4.1 - EditSF 1.2.0
    (Download PFM - Download EditSF)
    Warscape Modding Guide
    Join the PFM User Group on Steam to receive PackFileManager update notifications.

    Respecto Patronum

  2. #2
    Magnar's Avatar Artifex
    Join Date
    Jul 2013
    Location
    The last place you look
    Posts
    4,370

    Default Re: Warscape Modding Primer

    Nice intro Daniu, but a quick question...

    How do i mod X?

    jk

  3. #3

    Default Re: Warscape Modding Primer

    Alright Daniu, so on the subject of mod and movie .packs, here's a question. I use Radious' battle mod. It edits units stats slightly. If I were to create a mod .pack that edits the stats of the same units, would there be any way to force my mod to load after Radious' so that mine will overwrite him? Or are the two simply incompatible?

  4. #4

    Default Re: Warscape Modding Primer

    Daniu! Excellent tutorial
    I wish I had got to read this two days ago when i started modding and dove head first into it knowing nothing, lucky for me PFM is pretty user intuitive and i have been able to learn as i go.

    First day it took me most of a night to figure out what i was doing and get to the point of adjusting one unit to my liking, then by the second night i had remodelled an entire faction and started doing custom helmets and stuff

  5. #5
    Deathcake's Avatar Tiro
    Join Date
    Aug 2013
    Location
    On the rim of the netherworld
    Posts
    237

    Default Re: Warscape Modding Primer

    Thank you for this

  6. #6

    Default Re: Warscape Modding Primer

    Quote Originally Posted by Alexmcm13 View Post
    Alright Daniu, so on the subject of mod and movie .packs, here's a question. I use Radious' battle mod. It edits units stats slightly. If I were to create a mod .pack that edits the stats of the same units, would there be any way to force my mod to load after Radious' so that mine will overwrite him? Or are the two simply incompatible?
    If you edit the exact same line in the exact same file, I'm afraid yes they will be incompatible.
    For instance, if in land_units, Radious changes the dismounted_melee_attack for Aet_Cav to 30, and you want to change dismounted_melee_defense to 20, you'd both need to do that in the same DB line with the same primary key, so the game will load only one.
    IMO, this structural problem is the main issue in creating compatible mods.

    FWIW, CA did a good job in splitting up many of the db files from Shogun2 to Rome2, so it's not as bad anymore.
    It does make navigating through the db more tedious, but I find it a good thing overall.
    Tools: PFM 4.1 - EditSF 1.2.0
    (Download PFM - Download EditSF)
    Warscape Modding Guide
    Join the PFM User Group on Steam to receive PackFileManager update notifications.

    Respecto Patronum

  7. #7

    Default Re: Warscape Modding Primer

    I'm new to modding, but a quick question. Is it possible to alter game logic trough modding? I'm guessing that no, it's contained into the executable. I'm asking because of all thing I wanted to take a look at the AI.Btw, great one, should be stickied in the tutorial thread.

  8. #8

    Default Re: Warscape Modding Primer

    This really should be stickied. Thanks for taking the time!

  9. #9

    Default Re: Warscape Modding Primer

    Quote Originally Posted by DrDima View Post
    I'm new to modding, but a quick question. Is it possible to alter game logic trough modding? I'm guessing that no, it's contained into the executable. I'm asking because of all thing I wanted to take a look at the AI.Btw, great one, should be stickied in the tutorial thread.
    There are some .lua script files included in the packs you can change, but I haven't gotten into them to be able to judge how far you get with them.
    Tools: PFM 4.1 - EditSF 1.2.0
    (Download PFM - Download EditSF)
    Warscape Modding Guide
    Join the PFM User Group on Steam to receive PackFileManager update notifications.

    Respecto Patronum

  10. #10

    Default Re: Warscape Modding Primer

    This was very helpful, thank you for explaining it so clearly. I agree this should be stickied.

  11. #11

    Default Re: Warscape Modding Primer

    Quote Originally Posted by daniu View Post
    There are some .lua script files included in the packs you can change, but I haven't gotten into them to be able to judge how far you get with them.
    I'll take a look at them, but I don't think there's much potential there. I believe most of the game logic, especially AI and such is hidden.

  12. #12

    Default Re: Warscape Modding Primer

    Hey there daniu! Thanks for the tutorial, it`s great, but I`m facing some problems.

    I tried editing some units, but somehow it does nothing. I edited the charge bonus of the Celtic Longbowman (just testing), no new unit, just a stat change. Started a custom battle, and the value remains the same.
    I`m currently running Radious Battle and Campaign Mods, do they overrun any mods I make? What should I do? Thanks!


  13. #13

    Default Re: Warscape Modding Primer

    Quote Originally Posted by Carcaju View Post
    I`m currently running Radious Battle and Campaign Mods, do they overrun any mods I make? What should I do? Thanks!
    Yes they do if they changed the same file as you did.
    The only thing you can do is to use their data as your baseline and change those instead of vanilla, then add your mod to the script file after theirs so yours overwrites them where applicable.
    Tools: PFM 4.1 - EditSF 1.2.0
    (Download PFM - Download EditSF)
    Warscape Modding Guide
    Join the PFM User Group on Steam to receive PackFileManager update notifications.

    Respecto Patronum

  14. #14
    TuranianGhazi's Avatar Biarchus
    Join Date
    Jan 2012
    Location
    Seattle, WA
    Posts
    614

    Default Re: Warscape Modding Primer

    Bravo! More tutorials please haha. Thaaanks!

    I am accustomed to using MOD packtype PACK files. I just noticed that the ALL FACTIONS PLAYABLE mod is as MOVIE type ... which I don't agree with, since I've been feeling the brunt end of the issue with overwriting entire DB tables, instead of just adding what's modded.

    I'll go and try to redo that mod as a MOD packtype, cutting away unmodded stuff, cleaning the mod more. Hopefully it'll still work and hopefully it'll make it HELLUVA easier and faster to update once CA updates with its BETA and FINAL PATCHES.

    Now, if only I could figure out how to script out tech trees and building lines (adding religious buildings to factions that have none, or at least to nomadic tribes)

    Best!




  15. #15
    crzyrndm's Avatar Artifex
    Join Date
    Feb 2011
    Location
    New Zealand
    Posts
    2,576

    Default Re: Warscape Modding Primer

    Just open the all factions mod in PFM and change the type to mod pack. Install it first, and then any other mods will overwrite the superfluous tables. Ofcourse this is only a temporary measure until you get it cleaned up with only the neccesary files
    It’s better to excite some and offend others than be bland and acceptable to all
    Creating a mod.pack with PFM - Database Table Fragments

  16. #16
    Hmmm's Avatar Senator
    Join Date
    Jun 2005
    Location
    Sweden
    Posts
    1,320

    Default Re: Warscape Modding Primer

    I have a question, though I am not sure if I am supposed to post it here on in some other thread.

    At any rate, I want to edit unit stats (I currently have the vanilla game, no mods). In this case I want to change the Shield Bearers charge bonus to the same as Sacred Band (31 instead of 29), but every time I try the changes do not appear in the game. I have followed crzyrndms tutorial but I must be doing something wrong. Could someone post a detailed account of what I am supposed to do? Thanks in advance.
    I had a monumental idea this morning, but I didn't like it.

    Samuel Goldwyn

  17. #17

    Default Re: Warscape Modding Primer

    Quote Originally Posted by Hmmm View Post
    I have a question, though I am not sure if I am supposed to post it here on in some other thread.

    At any rate, I want to edit unit stats (I currently have the vanilla game, no mods). In this case I want to change the Shield Bearers charge bonus to the same as Sacred Band (31 instead of 29), but every time I try the changes do not appear in the game. I have followed crzyrndms tutorial but I must be doing something wrong. Could someone post a detailed account of what I am supposed to do? Thanks in advance.
    You're probably falling victim to a bug that's currently in the mod engine of the game which messes up the load order of pack files.
    Download this tool to update your mod file and try with the one it saves.
    Tools: PFM 4.1 - EditSF 1.2.0
    (Download PFM - Download EditSF)
    Warscape Modding Guide
    Join the PFM User Group on Steam to receive PackFileManager update notifications.

    Respecto Patronum

  18. #18
    Hmmm's Avatar Senator
    Join Date
    Jun 2005
    Location
    Sweden
    Posts
    1,320

    Default Re: Warscape Modding Primer

    Thanks for the quick reply.

    I just tried it, but it still does not work and they come out with the vanilla charge bonus when I check them in custom battle (29).

    I would really appreciate a step-by-step guide since I kind of suck at this and I feel I must have missed something. So far this is roughly what I have done:

    1) Create new mod by going into File>My Mods>New Mod (I did this, called the file New Stats and had its directory on my desktop).

    2) After creating it, it shows a list with all the files in Rome 2s data folder (Steam/Steamapps/common/and so on...). From there I expanded it all the way to data_pack2/db/land_units_tables/land_units. I then clicked on land_units and it automatically got included in the mod file (new stats.pack/db/land_units_table/land_units).

    3) To be sure I then right-clicked land_units and pressed extract selected, at which point it asked me if I wanted to overwrite and I clicked yes.

    4) Then I pressed Files>Add>File(s), navigated to the folder on the desktop, opened it, opened the db folder inside it and then the land_units_tables inside it and opened the land_units file.

    5) As with step 4 in crzyrndms tutorial I made my changes in the PFM (changing 29 to 31 under charge_bonus)

    6) After that I installed the mod, File>My Mods>Install New Stats

    This did not work (and the line I had changed got a red background, I assumed here I did something wrong).

    I then tried using the tool you cited, by downloading it, pressing yes and selecting the files New Stats and land_units in the desktop folder (after which PFM could not open them), but this has also not worked.
    I had a monumental idea this morning, but I didn't like it.

    Samuel Goldwyn

  19. #19

    Default Re: Warscape Modding Primer

    Quote Originally Posted by Hmmm View Post
    I then tried using the tool you cited, by downloading it, pressing yes and selecting the files New Stats and land_units in the desktop folder (after which PFM could not open them), but this has also not worked.
    Did you also copy that file to the R2 data folder?
    Tools: PFM 4.1 - EditSF 1.2.0
    (Download PFM - Download EditSF)
    Warscape Modding Guide
    Join the PFM User Group on Steam to receive PackFileManager update notifications.

    Respecto Patronum

  20. #20
    Hmmm's Avatar Senator
    Join Date
    Jun 2005
    Location
    Sweden
    Posts
    1,320

    Default Re: Warscape Modding Primer

    Which one?
    I had a monumental idea this morning, but I didn't like it.

    Samuel Goldwyn

Page 1 of 2 12 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •