Results 1 to 12 of 12

Thread: Triggers with low probabilities - is there a bias?

  1. #1
    Jurand of Cracow's Avatar History and gameplay!
    Join Date
    Oct 2012
    Location
    Cracovia
    Posts
    8,451

    Default Triggers with low probabilities - is there a bias?

    Guys,

    I've noticed (and it was reported from the players) that sometimes a trigger with very low probability fires very often.
    I have a trigger of 2% for a negative trait (in this case Coward) but it fires much more often than expected (from non-representative trials I'd infere 5%). The players report that every character gets it - and it should be 1 in 50 turns (
    I've checked a few times if there's no other situations, no other triggers etc - no.
    I also doubt that there's a bias perception (we discern only the negative situations) - it's too often reported.

    Has this phenomenon been reported in the past?
    Any solution? I think I'll make traits starting with 2 points first thresholds - if there's a bias only for the first point...

    cheers
    JoC
    Mod leader of the SSHIP: traits, ancillaries, scripts, buildings, geography, economy.
    ..............................................................................................................................................................................
    If you want to play a historical mod in the medieval setting the best are:
    Stainless Steel Historical Improvement Project and Broken Crescent.
    Recently, Tsardoms and TGC look also very good. Read my opinions on the other mods here.
    ..............................................................................................................................................................................
    Reviews of the mods (all made in 2018): SSHIP, Wrath of the Norsemen, Broken Crescent.
    Follow home rules for playing a game without exploiting the M2TW engine deficiencies.
    Hints for Medieval 2 moders: forts, merchants, AT-NGB bug, trade fleets.
    Thrones of Britannia: review, opinion on the battles, ideas for modding. Shieldwall is promising!
    Dominant strategy in Rome2, Attila, ToB and Troy: “Sniping groups of armies”. Still there, alas!

  2. #2

    Default Re: Triggers with low probabilities - is there a bias?

    Yes, there is some strange bias with those triggers but it is also true that while a trigger may fire 2 out of 100 times, if it is able to fire every single turn it will be a common trait instead of a rare one.

    What you can do of course is increasing thresholds, include random chances on the trigger conditions or use counters like a 2% trigger that only fires during winter will make it an uncommon trait.

  3. #3
    Jurand of Cracow's Avatar History and gameplay!
    Join Date
    Oct 2012
    Location
    Cracovia
    Posts
    8,451

    Default Re: Triggers with low probabilities - is there a bias?

    Do we know more about this? Any regularieties? Does it stop being so frequent if there's 1 point in the trait?
    I can make first thresholds with 2 points, but does it help?
    How do 10 triggers with 1% compare to 1 triger with 10%?
    When does it stop - 1% is frequent, but what about 3%?

  4. #4

    Default Re: Triggers with low probabilities - is there a bias?

    Nothing much special, just simple math, mate.

    But having thresholds with 2 points is a good idea. If it's a trait in which you want to reduce later you can just create triggers that reduce -2 points if the General has >0 (meaning that he has reached level 1 with the 2 points)

  5. #5
    Jurand of Cracow's Avatar History and gameplay!
    Join Date
    Oct 2012
    Location
    Cracovia
    Posts
    8,451

    Default Re: Triggers with low probabilities - is there a bias?

    Quote Originally Posted by Lusitanio View Post
    Nothing much special, just simple math, mate.

    But having thresholds with 2 points is a good idea. If it's a trait in which you want to reduce later you can just create triggers that reduce -2 points if the General has >0 (meaning that he has reached level 1 with the 2 points)
    This is not a simple maths! Come one, how can I count probabilities when 1% is much more, I don't know how much more, and I don't know when does it end - whether if there's a point in a trait, or the probability is higher than x%, or any other situation!

    Help! Help!
    Last edited by Jurand of Cracow; February 24, 2020 at 03:48 AM.

  6. #6

    Default Re: Triggers with low probabilities - is there a bias?

    I know but there isn't really any simple solution. Increasing the chances will make it even more common and 1% seems to be already common enough. The thing is, everything that has chances of appearing every turn, even if it's only 1% chance, will appear and there isn't much you can do. But I do notice that even with low chances the traits seem to happen anyway. So make them seasonal traits like I do for some traits in EBII, giving them chances to appear only during winter (for example).

  7. #7

    Default Re: Triggers with low probabilities - is there a bias?

    Quote Originally Posted by Jurand of Cracow View Post
    Guys,

    I've noticed (and it was reported from the players) that sometimes a trigger with very low probability fires very often.
    I have a trigger of 2% for a negative trait (in this case Coward) but it fires much more often than expected (from non-representative trials I'd infere 5%). The players report that every character gets it - and it should be 1 in 50 turns (
    I've checked a few times if there's no other situations, no other triggers etc - no.
    I also doubt that there's a bias perception (we discern only the negative situations) - it's too often reported.

    Has this phenomenon been reported in the past?
    Any solution? I think I'll make traits starting with 2 points first thresholds - if there's a bias only for the first point...

    cheers
    JoC
    Hmmm, I see no contradiction between the statements "I have a trigger of 2%" and "the players report that every character gets it". After all, 2% with WhenToTest set to CharacterTurnEnd means that I expect the trait to show up during the lifetime of a character who fulfils the conditions for the trigger. Each trigger fires independently for each character, so I don't exactly get what you mean by "every character gets it - and it should be 1 in 50 turns".

    In this game, we deal with a geometric distribution. Getting a point in a trait via a trigger is a process following the logic "let the probability of success at an event be p; the probability that the first occurrence of success requires k independent trials is given by
    P(X=k) = p(1-p)^(k-1)
    which is the geometric distribution. The mean of this distribution is 1/p. In your case, where p=0.02, the mean first occurrence of success is 50 turns. The variance is high though, so giving only the mean first occurrence of success is not that relevant: in this specific case I would rather say that I don't expect this trait to pop up at turn 1, but I'd be surprised if it didn't show up after turn 100, which I guess is within the lifespan of a character.

    Without any proper testing it's hard to say something meaningful, unfortunately, so you may be right: after all I guess nobody really knows how the chances of success are calculated, so there might be an error that shows up only at the lower end of the spectrum of probabilities (very typical; boundaries are always critical).

    Concerning your other question: it's entirely possible to calculate the difference between one single trigger with 10% chance and 10 triggers with 1% chance. Since each trigger is an independent event, the probability of having no success at all in all 10 triggers is the probability of failure (99%) multiplied by itself ten times, so (0.99)^10=0.904. This means that the probability of getting at least 1 point in your trait is 9.6%, slightly lower than the single trigger with flat 10% chance. Of course this doesn't take into account 1) the possibility of errors in computing low probabilities, as we already mentioned, and 2) the cases in which you aren't dealing with only one level of a trait with threshold 1. The former has no solution apart from testing, the latter needs more specific calculations but it's computable. At least with this method you are able to calculate the expected values for the probabilities, in case you want to do some testing.

    I hope this helps, even if I couldn't give a specific answer to your question.
    Last edited by mephiston; February 25, 2020 at 06:12 PM.

  8. #8
    Withwnar's Avatar Script To The Waist
    Join Date
    Oct 2008
    Location
    Earth
    Posts
    6,329

    Default Re: Triggers with low probabilities - is there a bias?

    Quote Originally Posted by mephiston View Post
    ...after all I guess nobody really knows how the chances of success are calculated
    Not without seeing the game's source code, no. I would expect that every Affects...Chance line is a new call to a "get a number from the random number generator" function. Random number generators aren't truly random though, they only simulate randomness by following some predefined algorithm. So it could well be that some numbers are spat out by this function more often than they 'should' be.

    For what it's worth, there is another way to implement a "less than 1% chance": include a RandomPercent condition. e.g. Leave the Chance at 1% and add a "RandomPercent < 50" condition: there's now a 50% chance of the Affects line not being reached/executed so the total chance of getting the trait is now 0.5 (50%) x 0.01 (1%) = 0.005 (0.5%). Likewise, "< 25" gives 0.25%, "< 10" gives 0.1%, and so on. Of course, this is still at the mercy of the random number generator so who knows?

    Quote Originally Posted by JoC
    the players report that every character gets it
    Yes, eventually every character will. How many turns into the game are they seeing this?

  9. #9
    Jurand of Cracow's Avatar History and gameplay!
    Join Date
    Oct 2012
    Location
    Cracovia
    Posts
    8,451

    Default Re: Triggers with low probabilities - is there a bias?

    Quote Originally Posted by Withwnar View Post
    Yes, eventually every character will. How many turns into the game are they seeing this?
    I know the psychological biases of the players' perception, I had taken ligthly their complaints in the past (one of the traits, Coward, had only one 1% trigger that could fire, and apprently each general got it in the first few initial turns) but even me in play testing I clearly detect it (I've added 1% in some situations for the other traits and the same situation arised). You get 3 generals each having 1% chance and there's always somebody getting it withing first 2-5 turns. I'd say it translated in to 20% or so. I haven't made any systematic tests though.

    I actually suspect that it may be the issue of "1%" probability and I'm going to recode it to "2%" every triger with 1%. And maybe that RandomPercent in addition.

    Thanks, Guys, for the answers! I think you may enjoy watching a moment Sid Meier's reflection on probabilities.
    Last edited by Jurand of Cracow; February 26, 2020 at 01:50 AM.

  10. #10

    Default Re: Triggers with low probabilities - is there a bias?

    Well, I think it's OK if somebody gets it in the first 2-5 turns. The probability of getting the trait exactly in turn 2 is the highest among all the turns that the character plays, after all.
    A big issue is how large your population is. I mean, if your statistical pool of characters is 3 then it's definitely strange, but if it's like 10-15 then it's not impossible that one character gets the trait within the first few turns. I wouldn't call this dice loaded if that's the case.

    Quote Originally Posted by Withwnar View Post
    Random number generators aren't truly random though, they only simulate randomness by following some predefined algorithm. So it could well be that some numbers are spat out by this function more often than they 'should' be.
    There are some true random numbers generators, but that's surely not the case. IIRC several programming languages (Python, C++ on top of my head) sport the Mersenne Twister algorithm for random numbers generation, which is actually pseudo-random; even if they use some source of entropy (e.g. PC internal clock) to refresh the seed after a certain amount of time, you run the risk of having subpar random distributions, which aren't truly independent or evenly distributed, or even both. For low-level applications that may be fine, but I remember that in university it was recommended not to use those functions when we were studying Monte Carlo simulations. Even some sources of entropy that you may think would be fine for true random numbers generation aren't actually that good - see the output of a seemingly true random number generation based on a floating ADC source. Again, here you can see how boundaries are critical.




    Quote Originally Posted by Withwnar View Post
    For what it's worth, there is another way to implement a "less than 1% chance": include a RandomPercent condition. e.g. Leave the Chance at 1% and add a "RandomPercent < 50" condition: there's now a 50% chance of the Affects line not being reached/executed so the total chance of getting the trait is now 0.5 (50%) x 0.01 (1%) = 0.005 (0.5%). Likewise, "< 25" gives 0.25%, "< 10" gives 0.1%, and so on. Of course, this is still at the mercy of the random number generator so who knows?
    Yes, that's what I'd suggest too. Chance 10 with RandomPercent < 10 is statistically equal to Chance 1, so that may be worth a try.
    Last edited by mephiston; February 26, 2020 at 07:21 AM.

  11. #11

    Default Re: Triggers with low probabilities - is there a bias?

    If you want to find a quick way to calculate the probability of an event of a probability p triggering X number of times, I'd recommend using this calculator (or something similar):
    https://planetcalc.com/486/

    You can use either the probability mass function or the cumulative distribution function, depending on what you are trying to determine (ie: probability of X number of times versus probability of at least X number of times).

    Using this calculator, the probability of a trigger of 2% chance of firing on each attempt will have a 45% chance of firing after 30 attempts, and a 64% chance of firing after 50 attempts (even though in the last case, the expected value is 1).

  12. #12
    Jurand of Cracow's Avatar History and gameplay!
    Join Date
    Oct 2012
    Location
    Cracovia
    Posts
    8,451

    Default Re: Triggers with low probabilities - is there a bias?

    I've changed the probabilities as I'd described here (Chance 2 + RandomPercent >50) and I think I got good result. However, 3 traits would appear very often and without any reason. I've found only one suspicious trigger, that is:

    Code:
    ;------------------------------------------
    ;Trigger Deranged_Disaster
    ; WhenToTest CharacterDamagedByDisaster
    
    
    ; Condition IsGeneral
           
    ; Affects Deranged 1 Chance 10
    ; Affects Hypochondriac 1 Chance 10
    ; Affects Miserly 1 Chance 20
    Nothing seemed to happen in the game to my characters as far as disasters are concerned, however I've discerned in the log that something similar fired (no idea if it was because of the script, and if it concerned my faction).

    Anyway, I now think that "CharacterDamagedByDisaster" is a suspicious event and I've disabled this trigger. Until now it seems to work.

    Has anybody alse in the past also noticed this issue?
    Last edited by Jurand of Cracow; March 07, 2020 at 12:41 AM.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •