[Request] [Tool] Mod merger

[Livebyte]

Hey guys. I have a proposal for some of our programming wizards that I think would enhance the R2 modding scene a lot.
As most modders have found by now, you can't make a mod without only the lines of a db table that you want to change. Instead you have to include the whole file, conflicting and invalidating countless other mods that might want to use that same table for something else. I would like a propose what I feel is an elegant workaround for this problem:

A tool that can automatically compare and merge mods, taking only the changes from vanilla made to any table. This could be accomplished as follows:
1. The tool loads (or will load) reference vanilla data, either extracted or directly from data_rome2.pack
2. A mod is loaded into the program as a .pack, let's call this M1
3. Additional mods may be loaded. Theses are the ones you want to merge with M1. Let's call them M2, M3, etc.
What the program then does with this information is:
4. Compares table changes between M1 and vanilla, recording only what lines have changed from the reference data and saves these changes as working data.
5. Repeats this process for M2, M3, etc
6. Takes these recorded changes and writes them back to a new copy of the reference data.
7. If mods change the same entry in a table, simply ask the user which change data to use (i.e. from M1, M2, M3? etc) and repeat until all conflicts are resolved.
8. Saves a new .pack with only tables updated with the merged changes.

I think this would be a great resource for modders (and advanced users). It would save huge amounts of time and would let players mix and match much more easily than they can right now.
Thoughts?

[daniu]

You can do that with PFM with a few extra steps by exporting the data to files first, then import them into a merged pack and have the importer rename the files which are in both (though I would recommend modders to not use the vanilla file names for their mods anyway, in which case they would be unique).
The removal of files that have changed from vanilla is at Postprocess->Optimze DB files.
There will probably be duplicate values for some primary keys, which for number integers can be set via the Renumber All feature from the popup menu when right-clicking a table header which should work for indices that need to be unique.
String keys have to be manually resolved between two packs anyway.

I did think of an "import from other pack" feature, but I don't think it's a feature that is needed often. If you made two mods yourself, you can make them instalable side by side; if you take data from other people's mods, that's only okay for collection mods for which the assembly process is the main effort.

It's something else for mod users wanting to install different mods that are not compatible. They could do with a merge tool which automatically corrects the conflicting primary keys and adjust the mods or create a new pack. I am thinking of doing that; but again, there probably will be conflicts which need to be manually resolved.

[Livebyte]

Thanks for the response and tips. Resolving specific conflicts is easy, I think. If you have a few changes the user can decide to mix and match, or maybe hit an "overwrite all" sort of button. A tool like this came to mind because Rome 2 mods now conflict on entire tables instead of specific lines, making most mods on the same part of the game difficult to combine. The process you've described works, but it's fair amount of legwork if you only want to mix and match small changes.

Thinking of it, something like this would also let a modder update their mod for CA's patches painlessly. That would be really cool.

[Moonhoplite]

where is the "renumber" feature?

When i optimise DB, it removes all unedited rows, but doesnt seem to maintain the unique ID of rows that I've changed. This leads to a crash. Help please

[daniu]

where is the "renumber" feature?

Good thing you ask, it actually got disabled by an update along the line... I'll put it back in in the next version.

When i optimise DB, it removes all unedited rows, but doesnt seem to maintain the unique ID of rows that I've changed. This leads to a crash. Help please

The Optimize feature will not change any of your data to make the key fields unique if that's what you mean.
It can't do that because it can't know whether you want to change existing rows or add new ones.