MMM modding memory modified

[Argantonio]

MMM modding with memory modified
It is necessary to know that memory modified will be a complex and dificult , but after to much MTWII study . it seems to be evident that all is written in the memory of the program, the development of the campaigns and the dynamics of the battles.
Exist legal programs that allow , to accede to the memory in execution, and allow his modification, one of them is Quick Memory Editor.
These they are a few examples of modification in execution

Attachment 63122Attachment 63124Attachment 63131
Change name of captain ship in campaign ( Almirante Geiles for Capitan Pescanova)
Attachment 63125Attachment 63126
Change texture in battle mode
The process is complex, but for the first time we see in real time the functioning of the program, and it opens a door for to understand each and every aspects of the game.
Attachment 63127
attack_missile_ready.cas has 17 memory entries in ejecution.
Attachment 63128
In memory position 013DE68C
Each one of 17 cas files in memory has his function in live , till now we see "inert" files in pack.dat, skeleton.dat, now we see them in execution, and can modify them,
it seems to be interesting?

If someone encourages here is the program, Quick Memory Editor 5.5
http://www.softcows.com/downloads.htm

Other program , ChatEngine
http://cheatengine.org/index.php
allows to see in real time ,the memory changes ( and his localitation) , percentage of falls, dead men, injured men, expense in florins. I dont know to modify it yet .
Attachment 63129Attachment 63130
Two images en real battle , with 30 seconds of diference , the percentaje of aliades and enemies change 0% to 8% 15% , amazing.

[Yelü Dashi]

Wow. That's fascinating. Potential applications for modding ? If changes are made in realtime, are the permanent is just for the duration of the session ?

[Argantonio]

Is to soon to give categorical answers.
Potential applications for Modding , All , everything what goes out on screen is modifiable ( we will speak about permissions of write).
The changes, remains during the session, but if we know where are the entries , that produces the changes, is easy modification assembles and to edit them with Hex, making these modifications fixed.
Is soon, still, for the present we have to understands many things.
The best , try the program..

[gracul]

Well a pretty basic use of memory editing would be a faction name change like seljuk -> ottomans, but how would we actually execute that on some one's system? I have no idea...

[Beorn]

If I understand what happens there correctly, it's a great milestone for modding community

[Argantonio]

An example of what we can obtain, the direct edition of the game. Yes , edit the game in screen, "Modding in live".

"DMA is the phenomenon of a game dynamically assigning it's variables to different spots in memory.
In computer science, dynamic memory allocation (also known as heap-based memory allocation) is the allocation of memory storage for use in a computer program during the runtime of that program. It can be seen also as a way of distributing ownership of limited memory resources among many pieces of data and code.
We can input a new value into each one at a time, or freeze them one at a time, until you find the right one. Delete the others to remove an element. Now this value is dynamic. So if we reload or restart the game, this will change. We need to find a way so we don't need to rescan every time."

A Principle: The game takes the files that he needs to develop the game, transforms them , and lodges them in the memory. Let's imagine this memory as a great memory map with multiple spaces compartiments of dates ( blocks).
A Method: Identify the sequences with winhex ( string, integer, text...) of a any file,( cas,mesh,world...) look for them in the memory blocks , and edit them "inside the memory" . Obtaining direct result in screen.

We all know that the unit_models of the game, has the format .mesh, when we analyze this format, see that in a file mesh there are 3 heads, 4 bodies, 3 weapon...
Nevertheless when we initiate the game this file mesh transforms in a compound unit of "n" soldiers with "random" form.
False. The executable transform mesh file , composes the unit, re-composing every soldier ( with a determinated plain ). This information till today was not visible for us , so like you imagine , this one is visible in the memory allocation blocks . We can search , read and edit it.

A simple custom battle, I chose the simplificated unit Dismounted Latinkon reduce to 4 soldiers , initiate the custom battle, open the visor of memory . ChatEngine
http://cheatengine.org/index.php
I open with winhex dismounted_latinkon_lod0.mesh , and rapidly I identify in the memory map legs, arms, helmets.
Attachment 63618 Helmet 07 in dismounted_latinkon_lod0.mesh
Attachment 63619 Helmet 07 in AllocationBase 07AC0000
The soldier number 4 is constructed with
body_1
hands_06
Arms_3
Head01
Legs_03
Helmet 07
sword primary_25
kite pattern_53

I replace any value hex for 00. A magic draft.
Attachment 63620Attachment 63621Attachment 63622
We are erasing soldier 4 , in live triangle to triangle. body , legs and head.

Attachment 63623Attachment 63624Attachment 63625
I displace the helmet 07 of soldier 4 , and we are erasing triangle to triangle.

Attachment 63626Attachment 63627
With the buildings, equal method, erase tris.
A lot of posibilities!
This is alone the beginning , I know when closed the program, everything disappears. ( "cinderella problem " )
It is like a postproduction modding.

[Argantonio]

The tools that allow us to interpret the MTWII processes , reserve us multiple surprises.
We admit that the limit of the number of soldiers ( export_desrc_units.txt EDU file) for unit is minimun 4 and the maximum 60.
The famous Hardcoded Limits
Why? What determines these limits?
"This small code"
Attachment 68162
If we read in this fragment of dissambled MTWII file , we found four fascinating lines.


008BCBBE 663D 0400 cmp ax,0004h , comparation number soldiers with inferior limit 4
008BCBC2 0F82252A0000 jc L008BF5ED , if number minor jump to L008BF5ED
008BCBC8 663D 3C00 cmp ax,003Ch , comparation number soldiers with superior limit 60
008BCBCC 0F871B2A0000 ja L008BF5ED , if number mayor jump to L008BF5ED

Means the address (Hex) in the memory, the values (Hex), the instructions, and the comments (my personal translation)
0400 in decimal 4
3C00 in decimal 60

Is true , are the limits (min -max) of number of soldiers in edu file.

If we change this numbers , you can put in the game the number of soldiers for unit , whatever you want 0 till 65535.

jump to L008BF5ED , means that the program remenber his limits of soldiers for unit , game crash ,and in the log file appears

Script Error in at line Invalid number of soldiers...

008BF5ED L008BF5ED:
008BF5ED 6A00 push 00000000h
....
008BF61C 68B0132F01 push SSZ012F13B0_Script_Error_in__s__at_line__i__
.......
008BF631 68D04E3301 push SSZ01334ED0_Invalid_number_of_soldiers__i_fo
008BF636 E885B34200 call SUB_L00CEA9C0
008BF63B 83C41C add esp,0000001Ch

Modifying these two values, we can have units of one soldier.
Units with hundreds, thousands of soldiers (theoretically the maximum will be 65535).
Attachment 68163Attachment 68164Attachment 68171
A Reiter unit with one soldier.
Attachment 68168Attachment 68165
Attachment 68166Attachment 68167
A Reiter units with 200 or 254 soldiers.
A cantabric circle with 254 soldiers , more than 255 is probably limit of cantabric circle.

In the practice, I do not manage to overcome units of 300 soldiers.
Attachment 68169
The reason seems to be evident, additional over limits exist functional, structural or unknown.
If Harcoded limits exist is for an important reason, this post tries to demonstrate where are these limits, and that is possible to modify them. Though we meet new limits for to modified . This is alone the beginning.



type Reiters
dictionary Reiters ; Reiters
category cavalry
class missile
soldier Reiters, 300, 0, 1

(remembers for up 300, begin the next problems...)
Attachment 68170

[pacco]

WOW man!!!

[Taiji]

So we can have a program run along side m2tw that will allow for realtime modification of the memory. Can this program also be ordered to perform certain activities depending on the contents of the memory?

I am just thinking that if this process is to be made automatic... it could look identical to modifying the .exe. Since it would be modifying the .exe while in memory... is that OK in EULA terms? I wouldn't know, I just click accept.

If it is OK and a program can be made to piggyback the existing .exe and modify it's function when the memory situation dictates... then is it possible we could bring back some of RTW's functionality which got lost in the creation of m2tw? I am thinking specifically of the detailed morale reports that appeared on the tooltips for units, half the code appears to be in the .exe still... it's the feature I most miss... oh and weapon upgrades .... oh and missile upgrades working on the missile weapons ... oh and experience affecting defence as well as attack .... sorry, this is turning into a rant.

I just mean to suggest that there might be some way to solve these problems with this memory modification method... it would be amazing...

[Argantonio]

Taiji Totally, but the program/s that can read and edit the memory, needs our knowledge of
content stored in the memory.
The program/s cannot edit. We must know that modified.

Good and important observation, I do not know about laws, but it seems that the modified exe files is
not totally legal.
This is the problem, which I have noticed in several posts.
I think that forum staff must be say us if is legal open ,see ,study and modified exe file.
Definitively we do it , with all the rest of files of the game.
Attachment 68198
a navy unit with one navy in battle mode, is a cartag-navy-strat.cas ( in memory)

All works of the following form, the engine (MTWII.exe), and aprox 60 dll (APIs) (someone the game
(dbhelper.dll...) and the majority of windows (Kernel.dll , ntd.dll...)).
They read all the information of the files bin - txt, that we manipulate and transform in
Files dat (skeleton-anim-sound ...) and in an enormous content in the memory ( memory allocated).
If you read in the memory, are everything what happens in the game, from the position of the elbow of
the soldier 24 of the unit 6 the ally of your team, to the descending counter of the falls in combat
everything is modifiable, if you can find every thing in the memory.

Of this form we do not modify the exe file , we modified his transformated product but when you disconnect the modifications disappear. ( cinderela problem)

If you want that the modifications are fixed, it is evident that you need to modify the exe od dll`s.
This thread tries to find every "thing" in the memory and to modify the memory "in live" without
altering the exe file (submitted to copyright).

Taiji , testudo appears 24 times, is evident that the testudo atribute is in MTWII, but with "disconnected" functionality, in my previous post I want to demonstrate this .
Your have worked very much and well with the animations, you can changed animations for others, but in engine exe, are the key, the "randomness is a mathematical calculation", God not play...

The exe file is full of SSE
Do you know in order that they serve?
SSE Streaming SIMD Extensions are instructions that permit calculation with the quarternions that KE
show us.
For calculation of positions 3d ( all animations ), we need a KE, If he was reading these lines...

[Yelü Dashi]

Once again Argantonio is ripping up CA's rulebook and offering modders even more possibilities. the evil genius

Can someone please clarify the issues surrounding modding executables ? I am failing to understand why it would be issue compared with all the other proprietary files we modify. What's seriously going to happen if a mod is release using Arg's unit size "hack" tomorrow ?

Spoiler Alert, click show to read: 

a navy unit with one navy in battle mode, is a cartag-navy-strat.cas ( in memory)

Is this ship manoeuvrable ? I've had a few momentary vision of MTW2 naval warfare

[JuL14n]

I'm not good at all at these things, but shouldn't it be possible to create some kind of script-extender ?
With a script extender called OBSE you could for example create almost anything for oblivion through scripting.
You ran the obse.exe which ran oblivion.exe and sent additional information with it to the memory, I think?

Also there is a program that through a .dll-file allowed modern graphical configurations for Operation Flashpoint.

[Caesar Clivus]

If doing this is possible (and legal), it would be an amazing step forward in the moddability of this game. I wonder if we can finally get around the faction and unit hardcoded limits.

[Lü Bu]

Man this is stunning!!

[Argantonio]

@Caesar Clivus
The limits have a sense , in the complete design of the game. It seems logical to think that if we modify these limits in an aspect, others aspects will be altered.
For example units with more of 300 soldiers interfere the performance of the program ( appears an advice message UI_PERFORMANCE_WARNING).
Attachment 68272Attachment 68273
This battle exceeds the recommended size for your detected settings, this may result in degraded performance.

I think that there are absolute limits, which are superior to the limits ( Hardcoded limits) that we know.
Limits inside limits. The programmer fixes a limits ( Hardcoded limits) in order that everything works ok. But we can overcome without apparent problems , game works ok with units of 254 soldiers.

@JuL14n
Exactly , we could be call it MTWES (MTWII extend script) . The companions of the thread Text Editing and Scripting , have a lot of work. OBSE are very developed.
You ran the MTWES.exe which ran MTWII.exe and sent additional new information with it to the memory, ( and not change the original MTWII.exe ).
A minor scale exist , Trainers for a several games a program written to intercept and alter the memory adresses of the game.

[Caesar Clivus]

I think that there are absolute limits, which are superior to the limits ( Hardcoded limits) that we know.
Limits inside limits. The programmer fixes a limits ( Hardcoded limits) in order that everything works ok. But we can overcome without apparent problems , game works ok with units of 254 soldiers.

I always figured that was the case.

I know you're just starting to explore this, but do you think it would be possible to change unit names and stats on the fly to overcome the 500 EDU unit limit?

[Taiji]

But we can overcome without apparent problems , game works ok with units of 254 soldiers.

Sadly I disagree, the place to find problems with large units is in sieges. In settlements with smaller walls you can reasonably expect problems from at least 200 up. Problems like your unit stopping moving because there is no space.

Surmountable for player units but the AI is screwed. Fixable for both by remodelling the settlements to make them suitable for larger units... still as a player I can cope with these problems, I just don't like to ignore them.

Adjusting the EDU limit really is interesting for me though, it would be a massive step forward

Here's something I wonder if other people are thinking about:
I discovered that the game can ignore it's config_ai_battle.xml because the code is already in the .exe - Now what this makes me wonder is if other .exe funtions (or settings really) can be outsourced to .xml files....

Or more specifically if the missing battle AI settings (of which there must be very many) can be worked out and included in the config_ai_battle.xml.... It would be amazing to be able to set priorities and goals for units, and the game must be doing this, but there is no mention in config_ai_battle.xml....

For once in the history of TW games we could make an impressive battle AI (no offence to any who work on battle AI, I do too and that's how I know we need the help).

[Argantonio]

@Taiji
It is very interesting, but I will explain to you what happens.
Config_ai_battle.xml is not inside MTWII.exe,
in MTWII.exe are the instruction to read config_ai_battle.xml,
1) reads it label(string) to label ,
2) transforms it (relates multiple aspects of the game , matematic calculation, limits... )
3) and raises it to a block in the memory.
From where it uses it to develop what we see on screen.

Inside config_ai_battle.xml I choose a fragment.

<melee-manager>
<attack-dist-multiplier>3.0</attack-dist-multiplier>
<open>
<infantry>
<max-engage-dist>
<easy>40</easy>
<medium>40</medium>
<hard>40</hard>
<very_hard>40</very_hard>
</max-engage-dist>

In exe file read these strings.

0066EB08 PUSH medieval2.01305054 ASCII "melee-manager/"
0066EB69 PUSH medieval2.01305040 ASCII "/max-engage-dist"
0066EC7B PUSH medieval2.01305018 ASCII "melee-manager/attack-dist-multiplier"

Repeat: In exe not are config_ai_battle.xml, are strings and instruccions ...
the 3.0 , 40 of this fragments not are in MTWII.exe, it takes of config_ai_battle.xml .
If you change "melee_manager" in config_ai_battle.xml for "dance_manager" , MTWII not recognize this string. ( right)

By means of Ollydbg, make work MTWII.exe, place a Breakpoint in 0066EB08 appears melee-manager.
Attachment 68309
I choose a situation of combat, and when the game wants to initiate the combat, it stops.( Breakpoint stop it) MTWII cannot read melee-manager instruction...
Attachment 68310
I initiate a "trace" from 0066EB08... see in live who MTWII read melee-manager...
I read the transformation in the memory ( new address 01305074, we are out of MTWII.exe , we are in out-spaces)

A little fragment
Address Hex ASCII
01305074 69 6E 66 61|6E 74 72 79|00 00 00 00|6F 70 65 6E| infantry....open
01305084 00 00 00 00|9A 99 19 BE|00 00 70 42|00 00 20 41| ....���..pB.. A
01305094 CD CC CC 3D|00 00 00 40|00 00 96 43|00 00 96 43| ���=...@..�C..�C
013050A4 CD CC CC 3E|9A 99 D9 3F|00 00 80 3F|00 00 20 42| ���>���?..�?.. B
013050B4 00 00 96 42|AC C5 27 37|42 60 E5 3B|00 00 A0 41| ..�B��'7B`�;..�A
013050C4 00 00 40 3F|00 00 20 40|9A 99 D9 3F|DB 0F 49 40| ..@?.. @���?�I@
013050D4 17 B7 51 38|00 40 1C 44|53 75 70 70|6F 72 74 69| �Q8.@DSupporti
013050E4 6E 67 20 61|74 74 61 63|6B 20 6F 6E|3A 20 25 75| ng attack on: %u
013050F4 0A 00 00 00|72 69 67 68|74 2C 20 00|72 65 61 72| ....right, .rear
01305104 20 72 69 67|68 74 2C 20|00 00 00 00|72 65 61 72| right, ....rear
01305114 20 6C 65 66|74 2C 20 00|6C 65 66 74|2C 20 00 00| left, .left, ..
01305124 66 72 6F 6E|74 20 6C 65|66 74 2C 20|00 00 00 00| front left, ....
01305134 66 72 6F 6E|74 20 72 69|67 68 74 2C|20 00 00 00| front right, ...
01305144 45 6E 67 61|67 69 6E 67|20 00 00 00|41 74 74 61| Engaging ...Atta
01305154 63 6B 65 65|20 69 64 3A|20 25 75 0A|00 00 00 00| ckee id: %u.....
01305164 61 69 2E 6D|65 6C 65 65|6D 61 6E 61|67 65 72 2E| ai.meleemanager.
01305174 61 74 74 61|63 6B 00 00|00 40 1C 45|66 66 26 3F| attack...@Eff&?
01305184 00 00 C8 45|FC 50 49 01|50 16 67 00|B0 E8 66 00| ..�E�PI
Pg.��f.

Still I do not understand , but it seems to relate assaults in diferntes directions right, rear , left.
This is the way of works, we must interpret it.
This information can be modified.
All elements of game , makes the same tranformation , number of soldiers for unit, atributes like sheild-walls , warcry , light of the battle map, fog of war, mecanical decisions in campaign mode , all...


PD

Do you know for that you cannot see the modifications in your config_ai_battle.xml?
For that MTWII reads obstinately config_ai_battle.xml packed.
[io] file_first =1 , is not suficient
It is a trap.
Do you want to know the solution?
I play in MTWII vanilla, acced to packed files , with winhex , modify the name of original file,
Ej: data/config_ai_battle.xml for dttt/config_ai_battle.xml
in data_0.pack
and We force to MTWII.exe to read the modified file config_ai_battle.xml.
I assure you that MTWII.exe reads the modified file, ( I see the modified file in memory allocation)
This serves for all the files (ALL).

[Taiji]

You can check for yourself whether the config_ai_battle.xml extracted from the pack produces the same behaviour as when there is no config_ai_battle.xml to access. I am only talking from experience.

And if you have a broken config_ai_battle.xml, one that the game cannot work with, then it resorts to the .exe based settings and not vanilla's config_ai_battle.xml. It's often how I tell if what I am doing different is actually working, I compare with the .exe settings, and if it's the same then I know that I most likely broke something...

It's simple but it's saved me a hell of a lot of trouble since I discovered it

Try removing your packs and see what you get, that's what Germanicu5 did when I told him about it. He seemed convinced when the AI behaved the same. I was already convinced since I figured noone would include a crap AI in the pack only to be read when the vanilla AI isn't present. The .exe AI is much worse than vanilla... although I think the flanking is probably better...

edit:

Perhaps this is a difference between m2tw.exe and kingdoms.exe, I've only tested kingdoms.

[Devils_Advocate]

me.
I am sorry. Can I trust my eyes, single units are possible now, not merely theoretically?