Statistical Combat Analysis

[crzyrndm]

Where the interplay between unit combat stats is observed



Projectiles vs. Armour

Spoiler Alert, click show to read: 

Test Conditions:
Custom Map Co-ordinates: (0.650, 0.681)
AI is given an archer unit with heavily modified accuracy (100x typical) and 1 ammo. All abilities are disabled (fire arrows, rapid reload). Unit range == 100, projectile velocity == 45. All damage is non armour piercing.
The target has been modified to remove all shield defense (directional armour), only body armour is in effect (non-directional). Tested damage levels will be 35, 50, and 100.

Procedure:
AI marches towards player, expends ammo, number of casualties is recorded. Repeat five times, report average (mean) value.

Results:

• If damage is greater than Armour + Health, all successful hits will be kill shots (observed with 100 damage and <70 armour)
• Increasing Armour has a decaying impact on effectiveness following the theoretical pattern below

Melee Attack vs. Melee Defence

Spoiler Alert, click show to read: 

Testing Conditions:

Units: Royal Peltasts vs. Royal Thorax Swordsmen (ie. near identical sword units with formation attack)
Testing Regime: One unit is defined as the attack unit, has its defence set to zero. One unit defined as defensive unit, has its attack set to zero. Testing values of attack and defence are given to the corresponding unit. Thus the two melee attack/defence pairs should be: (0, 0) and (test_attack, test_defence).

Extraneous stats (armour/charge/morale/damage/health/fatigue/experience) are set to values at which they will not interfere.

In a set custom battle map with the AI as the attacker controlling the defending unit, human controlling the attack unit gives no input. Remaining number of men from the victorious unit is recorded, with AI unit being recorded as a negative value (if melee defence wins, the value will be negative). Results are averaged over 5 battles.

Results

Averaged data

Graphical representation

Notes

• Melee defence is noticably less important than melee attack under the test conditions.
• It is very apparent that reaching the 95% hit limit doesn't take a very high attack value.
• 40 attack/80 defence appears to be an outlier

Missile Combat - How it works
There seems to be a lot of misconceptions around about how health, armour, shields, damage, and ap damage work together in missile combat. This is my interpretation (I'm not infallible ) resulting from many tests, searching the database, and reading what Lusted has to say about the matter.

Spoiler Alert, click show to read: 

What Shields do:
For starters, a projectile striking a soldier is treated the same no matter where it strikes. Head, chest, shield, foot, the calculations are the same. Source and explanation:

... Collision detection for projectiles has always been done against the soldier entity. We do not model whether the projectile has hit the actual shield or not ...

A 3d model check is out of the question, that kind of complex collision detection for individual soldiers just isn't feasible with the size of battles we have.

What shields do is add to armour if an attack comes from the front or left side (melee combat they also add to melee defence). There is no change to a units health and shields do not wear out under repeated impacts (amout of armour added is constant). Source: Testing (with 1 HP), database (shield stats), and Jack's explanations.

- Shields only apply against attacks from the front and left
- In melee they provide a bonus to melee defence and armour
- Against missiles they give a bonus to armour

Update:
Since Patch 5, shields have had an additional projectile defence which can entirely negate a projectile strike. All shields now have a fixed chance to block missile attacks from the front and left, greatly increasing a units durability under missile fire. This stat ranges from 5-10% for cavalry shields and pikemen, up to 50% for most heavy Infantry shields (hoplites, Roman Infantry, and the like), and 55% for tower shields (used by several spear units). Units with access to the testudo formation get bonuses to this stat while using the formation (85% total from Roman Infantry unmodded).

Damage and health:
Weapon damage in this game has been split into two types: Normal, and Armour piercing.

Normal damage is checked against the targets armour to see how much is applied. It is not a simple yes/no check, but a roll to see how much gets through. Source:

Armour is not a straight block on damage, we do a dice roll against the armour total, so for 40 armour you could get anything from 0 - 40, and then that is taken away from damage. Having higher armour will give you a much better chance of blocking all non-AP damage.

Increasing damage or decreasing armour increases the average normal damage that will be applied to a units health. This is what my first test was looking at. If a unit has a shield, missiles coming from behind or to the right of a soldier will have a greater average applied damage as the soldier will not recieve the bonus armour from the shield.
A slightly more in-depth explanation of how armour negates damage:Say that you are against a weapon damage of 25 (normal damage). If you have amour of 100, (avg 50 damage reduction), most of the time you will fall above the 25 damage reduction required for full negation of damage (75% chance), where if you have armour 50, you only have a 50% chance to take no damage.

Armour Piercing(AP) damage is a new concept that can only exist because of the health system. AP damage is always applied to a soldiers health if a collision is detected. Source and reasoning:

In previous Total War games it has been perfectly possible for a soldier to be hit endless times and not die, the implementation of the health stat and weapon damage shows a soldier being worn down by multiple hits.

- Armour blocks normal damage but AP damage is always applied, this is true for both melee and missile combat.

Update:
The block chance added to shields means AP damage can now also be negated. Just not by armour.

The health system is also the reason why the first few volleys of projectiles always cause significantly less casualties than subsequent volleys. Even if the full damage total is applied most projectiles have a lower total damage than each units health. The only casualties you will see on the first volley are from multiple projectiles striking the same soldier.

Summary:
Collision is done at an entity (soldier) level, not at an equipment level.
Armour is the chance to negate an amount of normal damage. AP damage ignores armour and is always applied to a soldiers health.
Shields increase armour and grant a block chance against missile attacks from the front and left. The bonus is constant.
Unit health is independent of armour and shield.

[bzhou]

Projectiles vs. Armour
My initial tests are the same as those I did with Shogun 2, but even there a new layer of complexity has already arisen with the new weapon damage/health system. Early testing indicates that weapon damage doesn't only effect the removal of health, but also the amount of armour it will totally ignore.

I'm not sure if you're aware, but if you hover over "Weapon Damage" on the bottom left plate, it tells you how much damage is "Normal," i.e., non-armour-piercing, and how much is armour-piercing. Forgive me if I misunderstood your setup, but I believe this feature is what you "discovered."

On a side note, being a scientist, I'd be happy to help you expand on your studies and crunch some numbers as well.

[crzyrndm]

Nope, I cut the ap damage out of the equation. 100 damage is 100 normal damage + 0 ap, yet it totally ignores changes in armour until ~70 armour is reached.

The next line of testing I think is going to be melee attack vs. melee defence vs. armour (varying weapon damage if it makes a difference again). I'm just trying to work out how to test it in a meaningful way.
So far:
0 either armour or melee defence so that its just 1 stat vs. attack.
Unbreakable morale (fight to the death), balance the stats under investigation until combat is about even.
The problem is what do you set the off stat to (eg. melee defence on the unit that is having its attack varied). Set them equal to each other, keep the ratio the same as the main test stats...

[ararax]

I would be interested in Checking 100 AP Armour, also changing the shield type to a modded one with 100 Armour and see if 100 Shield Armour has any impact vs Normal.

Also The Type Padded gets a +1 vs Missiles so that may have impact.

Great work though!

[crzyrndm]

There seems to be a lot of misconceptions around about how health, armour, shields, damage, and ap damage work together in missile combat. This is my interpretation (I'm not infallible ) resulting from many tests, searching the database, and reading what Lusted has to say about the matter.

Missile Combat - How it works

What Shields do:
For starters, a projectile striking a soldier is treated the same no matter where it strikes. Head, chest, shield, foot, the calculations are the same. Source and explanation:

... Collision detection for projectiles has always been done against the soldier entity. We do not model whether the projectile has hit the actual shield or not ...

A 3d model check is out of the question, that kind of complex collision detection for individual soldiers just isn't feasible with the size of battles we have.

What shields do is add to armour if an attack comes from the front or left side (melee combat they also add to melee defence). There is no change to a units health and shields do not wear out under repeated impacts (amout of armour added is constant). Source: Testing (with 1 HP), database (shield stats), and Jack's explanations

- Shields only apply against attacks from the front and left
- In melee they provide a bonus to melee defence and armour
- Against missiles they give a bonus to armour

Damage and health:
Weapon damage in this game has been split into two types: Normal, and Armour piercing.

Normal damage is checked against the targets armour to see how much is applied. It is not a simple yes/no check, but a roll to see how much gets through. Source:

Armour is not a straight block on damage, we do a dice roll against the armour total, so for 40 armour you could get anything from 0 - 40, and then that is taken away from damage. Having higher armour will give you a much better chance of blocking all non-AP damage.

Increasing damage or decreasing armour increases the average normal damage that will be applied to a units health. This is what my first test was looking at. If a unit has a shield, missiles coming from behind or to the right of a soldier will have a greater average applied damage as the soldier will not recieve the bonus armour from the shield.
A slightly more in-depth explanation of how armour negates damage:Say that you are against a weapon damage of 25 (normal damage). If you have amour of 100, (avg 50 damage reduction), most of the time you will fall above the 25 damage reduction required for full negation of damage (75% chance), where if you have armour 50, you only have a 50% chance to take no damage.

Armour Piercing(AP) damage is a new concept that can only exist because of the health system. AP damage is always applied to a soldiers health if a collision is detected. Source and reasoning:

In previous Total War games it has been perfectly possible for a soldier to be hit endless times and not die, the implementation of the health stat and weapon damage shows a soldier being worn down by multiple hits.

- Armour blocks normal damage but AP damage is always applied, this is true for both melee and missile combat.

The health system is also the reason why the first few volleys of projectiles always cause significantly less casualties than subsequent volleys. Even if the full damage total is applied most projectiles have a lower total damage than each units health. The only casualties you will see on the first volley are from multiple projectiles striking the same soldier.

Summary:
Collision is done at an entity (soldier) level, not at an equipment level.
Armour is the chance to negate an amount of normal damage. AP damage ignores armour and is always applied to a soldiers health.
Shields increase armour against attcks from the front and left. The bonus is constant.
Unit health is independent of armour and shield.

[PerBak]

Normal damage has a chance to be applied depending on how much there is, and the targets armour. Increasing damage or decreasing armour increases the chance that the normal damage will be applied to a units health.

Firstly, thanks for the great work. I just have a quick question to clarify. I assume you are stating that normal damage and armour together determines the probability that an "attack event" (e.g. melee or missile collision with a soldier entity) is successful in deducting health from a single soldier. My question is: is the amount of health deducted (ignoring armour piercing damage) equal to normal damage stat? If so, does that mean that units with a low normal damage stat inflict damage less often because the probability that an "attack event" is successful is also lower?

Also, how does the melee attack stat play into damage, if at all? Thanks!

[rrgg]

So nothing can stop AP damage short of the projectile actually missing? Not even shield strength? Well I guess that explains archers and slingers being so broken right now.

[Modestus]

Does health recover during a battle?

If I understand you correctly the health of individual soldiers will decrease under fire to a point where you will start to have more and more casualties that being the case then ranged units could simply be used to weaken a unit rather then trying to inflict a large amount of casualties.

You could even have a situation where a unit under fire has lost almost no men but once it makes contact with a unit of enemy heavy infantry it could be destroyed in a matter of seconds because it soldiers have little of their health left.

So while you can say that armour does not wear out a unit composed of soldiers with a low health value would be extremely vulnerable, looks to me more like a system for space ships.

[crzyrndm]

Does health recover during a battle?

I can find no reference to health recovery (which would be a rather silly mechanic don't you think. You dont recover from flesh wounds in a few minutes).

If I understand you correctly the health of individual soldiers will decrease under fire to a point where you will start to have more and more casualties that being the case then ranged units could simply be used to weaken a unit rather then trying to inflict a large amount of casualties.

Archers fill one of the very traditional TW skirmish roles, with very low (5) AP damage and high normal damage and middling range. Javelins fill the other standard role of very high damage but low range. Slingers appear to be aimed at that weakening/harrassing role you describe with some AP damage and high range. I think the problem atm is that they have too much non-AP damage (middling-high total damage) meaning they can quite effectively perform the same duty as a javelin troupe from extreme range

You could even have a situation where a unit under fire has lost almost no men but once it makes contact with a unit of enemy heavy infantry it could be destroyed in a matter of seconds because it soldiers have little of their health left.

So while you can say that armour does not wear out a unit composed of soldiers with a low health value would be extremely vulnerable.

You could. The thinking behind it seems to be the simulation of flesh wounds from glancing strikes wearing a soldier down (from reading Jack's posts). With a little balancing to the numbers, it could make each ranged unit very useful in its own way.

My criticisms of the system would be how ineffective the implementation of shields is due to decreasing returns on increasing armour, and there being no chance at all to not take AP damage. Obviously with the ineffectiveness of testudo, this has been noted by CA (they are looking at changing it according to Jack).

[Chivalry]

I think this system is promising. Wounding was never a part of TW and now it is actually. Imagine big volley of arrows coming into mass group of soldiers, some of them dying by an arrow, some of them taking arrows into their legs/arms/somewhere else and losing their combat efficiency a bit.

[crzyrndm]

I think this system is promising. Wounding was never a part of TW and now it is actually. Imagine big volley of arrows coming into mass group of soldiers, some of them dying by an arrow, some of them taking arrows into their legs/arms/somewhere else and losing their combat efficiency a bit.

My opinion too, just as usual with CA (to be fair, its a common trait), the first iteration has a few kinks that need working out.

RE: More testing
I may have got a little sidetracked investigating just how far I can make modifications to gameplay, but I'll make sure to do some more testing this weekend. I'll be sticking with changing unit stats by 20 each time now, as it shows general shape well but doesn't take forever

I think I have my testing procedure for melee ready to roll, and it should show the patterns very nicely

EDIT

Delayed till alt-tab is fixed -.-

[crzyrndm]

Melee testing has begun. The graph below is 10 Melee attack vs. varying levels of armour (from 10 to 90 in steps of 20). Because there are two set of casualties involved in this testing (compared to the single target in ranged) I've decided to take the difference in casualties between the combatants to easily display the differences and crossover points. The lower the number, the higher the defence and lower the attack involved (it'll look better with more lines ).

Firstly, thanks for the great work. I just have a quick question to clarify. I assume you are stating that normal damage and armour together determines the probability that an "attack event" (e.g. melee or missile collision with a soldier entity) is successful in deducting health from a single soldier. My question is: is the amount of health deducted (ignoring armour piercing damage) equal to normal damage stat? If so, does that mean that units with a low normal damage stat inflict damage less often because the probability that an "attack event" is successful is also lower?

The amount of health deducted on a successful hit is equal to the damage of that projectile. And yes, lower damage does lower the chance that a hit will be successful.

Also, how does the melee attack stat play into damage, if at all? Thanks!

Specifics in the OP so far relate only to ranged combat as I've only just got started into melee testing.

In theory, increasing melee attack, decreasing armour, decreasing melee defence, or increasing weapon damage will increase kill rates (and the reverse of course). That is under the assumption the weapon damage works the same in melee combat as with ranged combat. In ranged combat, accuracy replaces melee attack, and unit density replaces melee defence except inverted (increase density to increase kill rate).

EDIT

Armour is not a straight block on damage, we do a dice roll against the armour total, so for 40 armour you could get anything from 0 - 40, and then that is taken away from damage. Having higher armour will give you a much better chance of blocking all non-AP damage.

Another update from Jack with regards to how armour, health, and damage interact. Updating OP and post #5 to reflect new information in a bit

EDIT2

Normal damage is checked against the targets armour to see how much is applied. It is not a simple yes/no check, but a roll to see how much gets through. Source:

Increasing damage or decreasing armour increases the average normal damage that will be applied to a units health. This is what my first test was looking at. If a unit has a shield, missiles coming from behind or to the right of a soldier will have a greater average applied damage as the soldier will not recieve the bonus armour from the shield.
A slightly more in-depth explanation of how armour negates damage:Say that you are against a weapon damage of 25 (normal damage). If you have amour of 100, (avg 50 damage reduction), most of the time you will fall above the 25 damage reduction required for full negation of damage (75% chance), where if you have armour 50, you only have a 50% chance to take no damage.

This is the updated part. Does anyone have a problem following that?

[Khan Asparukh]

Good job so far. Can you make some more tests in melee mode, but only with attack and defense as parameters (no armour if possible). The only thing that is known so far is that there is 20% to 95% chance to hit or get hit. But how attack and defense come into play. If we follow the analogy from projectiles combat system, my guess would be that difference between attack (from attacker) and defense ( from defender) is added or subtracted from the value that we become after the dice is rolled (varying from 20% to 95%).

So in the end we will have:

Step 1.1 - 1st roll of dice - become the chance the unit hit/get hit (varying from 20% to 95%)
Step 1.2 - addition or subtraction the value from difference between attack/deffence values to the chance to hit/get hit
Step 1.3 - 2nd roll of dice - become Yes/No (if the unit has hit/has been hit)
Step 2.1 - 3rd roll of dice - become damage negation percentage from armour for normal damage
Step 2.2 - become the „active" normal damage (if any)
Step 3 - become the value of total damage = „active" normal damage (which we have become in step 2.2) + ap_damage for the unit that hit and subtract this value from the health of the unit that get hit

Edit: If we want to extract the algorithm as well, and not only the tendencies, best would be if the tests are done 1 vs 1 soldiers also (for the projectiles as well).

[redxavier]

How do melee attack and defence come into play? Do these determine the success of the attack in the first place?

[crzyrndm]

If we want to extract the algorithm as well, and not only the tendencies, best would be if the tests are done 1 vs 1 soldiers also (for the projectiles as well).

Personally, I'm not particularly interested in breaking the algorithim down. The tendencies show how different stats interact, which is useful as most of the combat stats are paired, but the algorithim also accounts for many factors that are relatively straight foward and uninteresting (barring when they are bugged, ie. combat on a slope).

How do melee attack and defence come into play? Do these determine the success of the attack in the first place?

Melee attack and melee defence are the stat pair which should determine the success of a melee attack.

[crzyrndm]

Melee Attack vs. Armour
NOTE: I will not be adding this to the OP as the variables involved belong to two different systems (attack/defence and armour/damage/health respectively). Increasing armour doesn't reduce the chance of a succesful melee strike, it just increases the number that are required for a kill.

It does clarify two things that may be of interest though.

A) Zero melee defence doesn't mean that all melee attacks will be successful. It just means there is no normalisation towards zero of the to-hit chance. The base to hit chance is 50%, which is then modified by the attack and defence stats of the combatants
B) Increasing armour causes a mostly linear increase in a units survivability, as it decreases the average damage a unit in combat will take. The exception to this is obviously where the change in armour is not significant enough to change how many strikes are neccesary to deplete a units health

[neopara]

Melee Attack vs. Armour
NOTE: I will not be adding this to the OP as the variables involved belong to two different systems (attack/defence and armour/damage/health respectively). Increasing armour doesn't reduce the chance of a succesful melee strike, it just increases the number that are required for a kill.

It does clarify two things that may be of interest though.

A) Zero melee defence doesn't mean that all melee attacks will be successful. It just means there is no normalisation towards zero of the to-hit chance. The base to hit chance is 50%, which is then modified by the attack and defence stats of the combatants
B) Increasing armour causes a mostly linear increase in a units survivability, as it decreases the average damage a unit in combat will take. The exception to this is obviously where the change in armour is not significant enough to change how many strikes are neccesary to deplete a units health

Can you please add or describe the legend for this graph? What is the difference between the color dots?

[neopara]

Also, I am not sure I fully understand how you set the hitpoints in your missile vs armour test. Are you setting the hitpoints to be damage, damage-armour, or 35 for each test?

Fanatic job by the way. I am really enjoying reading this.

[crzyrndm]

Can you please add or describe the legend for this graph? What is the difference between the color dots?

Blue -> Orange is increasing melee attack values. I didn't really put much effort into it because while the two systems achieve the same goal, the manipulated stats don't directly interact and as such the comparison is rather worthless. I started it before I fully grasped the implications of armour negating damage instead of being part of the defence roll, and it ended up being a dry run for defence vs. attack.

Also, I am not sure I fully understand how you set the hitpoints in your missile vs armour test. Are you setting the hitpoints to be damage, damage-armour, or 35 for each test?

Fanatic job by the way. I am really enjoying reading this.

Constant 35. If you think about it, it clearly explains what happened when I had 100 damage on the projectiles. Up til 65 armour, it didn't matter if the armour rolled its maximum value or not, it was still going to do more than 35 damage and so kill the target. Thats why all the values between 0 and 70 did basically nothing.

Once I do attack vs. defence, I probably need to revise the wording of the projectile test as I didn't know how the damage system worked at that point.

[neopara]

Blue -> Orange is increasing melee attack values. I didn't really put much effort into it because while the two systems achieve the same goal, the manipulated stats don't directly interact and as such the comparison is rather worthless. I started it before I fully grasped the implications of armour negating damage instead of being part of the defence roll, and it ended up being a dry run for defence vs. attack.

That makes more sense now. Thanks for the quick answer.

Constant 35. If you think about it, it clearly explains what happened when I had 100 damage on the projectiles. Up til 65 armour, it didn't matter if the armour rolled its maximum value or not, it was still going to do more than 35 damage and so kill the target. Thats why all the values between 0 and 70 did basically nothing.

Sorry one more question. What is your unit size setting? I am trying to see how well a anydice model (ie. http://anydice.com/program/2ac6 - "At Least" view is the most useful) compares to your test. It would help to see percentage of casualties compare to total unit size.

Given that casualties wasn't 100%, would that mean range units aim for the first few ranks of the unit? That would makes sense if they are trying to maximize casualties instead of damage.