Siege Rolls; Confirmation Bias my ARSE!!!

[Ziggy187]

Every time this topic comes up PDX is quick to give the default reply that it's confirmation bias. I never accepted this as it appeared to me that "Disease Outbreaks" are much more common that "Wall Breaches" when I am sieging a hostile fort. Therefore, I took it upon myself to write down the actual siege rolls during a Brandenburg campaign. This is early on before cannons but all we are concerned with is the dice rolls, not what they do. In case you are not aware, a dice roll of "1" is a disease outbreak while a dice roll of 14 is a wall breach. Since the game is based on a 14 sided die, it would seem they both have equal odds of happening at a rate of 1:14. I wrote down all the siege rolls to acquire a sample size of 100 rolls. This should be large enough to illustrate the point. With 100 rolls, and 1:14 odds, we should see approximately 7 rolls of a "1" and 7 rolls of a "14". However, the actual data is much much worse. I got 18 rolls at "1" and 2 rolls at "14". So these supposedly equal odds are not so equal after all to the tune of a disease outbreak being 9 times more likely than a wall breach. I realize a sample of 100 rolls may not accurately reflect the 9 times more likely occurrence but the point of this post is to illustrate that a dice roll of "1" is not at all equal odds of happening as a dice roll of "14". The only rational explanation of this is that the random dice roll generator is skewed to give lower dice roll results for the player versus truly random dice. PDX, the evidence is indisputable, your confirmation bias excuse is complete and utter CRAP!
If the community would like, I can continue to gather data in this regard. However, I am convinced of the results and will be waiting for the 1.19 patch before seeking more data sets.

 

[Pile_O_Gunz]

Random is random it's not even or fair you need much more than 100 rolls for it to average out.

 

[misiceman]

with a range of 1-14 you would need a much larger sample size then 100 as you mentioned. This is the case of confirmation bias unfortunately. Assuming the code is running properly (and I believe it is) then a sting of unlucky 1s just happens (just like when I get breach after breach in some wars). Also remember the odds in this case are independent of the previous roll. Every roll is a 1/14. That doesn't change after 100 or 1000 rolls, you could in reality roll 1000 1s in a row and everything is working fine (unlikely but yeah).

The thing i always bring up is that random does not mean evenly spread out and that clusters can and do happen in random events.

 

[Ziggy187]

Random is random for any individual roll. However, over time, odds DO matter. I will concede that the sample size could be a bit larger than 100. I will attempt to compile more data and give the results. I think perhaps 500 rolls would be enough to prove this point. If the odds are truly 1:14 then the results for a "1" or "14" should be within a standard diviation of each other. I do some calculations to figure out the ideal sample size and post the results here. It may be a few days to compile though.

 

[allfath3r]

Random is random for any individual roll. However, over time, odds DO matter.

I see where you're coming from, of course, but these two statements contradict one another. In a truly random system (as the siege rolls are, at least in my opinion) one dice roll does not influence the outcome of another. So while you would expect to see a relatively even distribution across all possible outcomes (since each roll has an equal chance of occurring), the reality is that anomalies like you experienced are going to happen given a sufficient sample size. Expecting an even distribution in a circumstance like this is similar to (not exactly the same, but it gives you an idea of what I mean) a well documented phenomenon called the Gambler's Fallacy. Link for more info:
https://en.wikipedia.org/wiki/Gambler's_fallacy

 

[Gigawot]

there could be a player who literally has had every dice roll go against them (all 0s) , and someone else getting them all going for them (all 9s in combat or all 14s in siege). statistically that is possible.

 

[Zsar1]

Countering one gut feeling with another one is a fallacy all of its own. Conduct a statistical significance test and stop it with the myth propagation, please.
Yes, I do mean both sides!

addendum:
@Ziggy187 : Kindly paste your results into this online checker.

Observation should be the integral numbers 1 to 14.
Frequency should be the number of recorded hits for the die roll immediately above it, e.g. 18 under 1 and 2 under 14.
Lower Bound should be 1.
Upper Bound should be 14.

In order to see whether the sample size is big enough, repeat with this online checker.

Expected should be a row beginning and ending with an eight, all sevens otherwise.
Degree of Freedom should be 13.

If the results are incompatible (e.g. the first test reports strong evidence against whereas the second test reports weak evidence against), sample size has to go up.

If sample size has to go up, the "Expected" row of the second test has to be updated accordingly, selecting unequal values as uniformly as possible, e.g. some next rows might be:
8, 3x 7, 8, 4x 7, 8, 3x7, 8 for a sample size of 102
8, 7, 8, 2x 7, 8, 2x 7, 8, 7, 8 for a sample size of 104
etc. pp.
As long as the greatest difference between example numbers is 1, arranging them by gut feeling should suffice. Or you could just make sample sizes which fit a row:
14x 8 for a sample size of 112
14x 9 for a sample size of 126
etc.

Once you have your result, kindly report back with proof for or against your hypothesis.

 

[Gnostek]

i had a campaign where i literally had 5 walls breached within the same month, so it's confirmation bias my friend

 

[Ziggy187]

Here is some more evidence that the EU4 system may be dishonest. I used a random number generator of 1-14. I wrote down all the results for a 100 roll sample size. The mean is 7.79 with a standard deviation of 4.14. I got 9 rolls of a "14" and 5 rolls of a "1". This data set is small but the results are within line of the randomness of a 1:14 chance and the data sets falls within the standard deviation. Statistically speaking, this data set is truly random.
Going back to the EU4 data set which is supposedly random, the results do NOT fall within the standard deviation. The results are so horribly out of the statistical norm that the only rational explanation is that that random number generator that EU4 uses is horribly skewed to give lower dice rolls!
Those of you speaking about the ability to roll a zero (only in battles not siege rolls) do not understand statistics or probability. Are you familiar with sigma events? This is just a game but I hate dishonesty.

 

[allfath3r]

Here is some more evidence that the EU4 system may be dishonest. I used a random number generator of 1-14. I wrote down all the results for a 100 roll sample size. The mean is 7.79 with a standard deviation of 4.14. I got 9 rolls of a "14" and 5 rolls of a "1". This data set is small but the results are within line of the randomness of a 1:14 chance and the data sets falls within the standard deviation. Statistically speaking, this data set is truly random.
Going back to the EU4 data set which is supposedly random, the results do NOT fall within the standard deviation. The results are so horribly out of the statistical norm that the only rational explanation is that that random number generator that EU4 uses is horribly skewed to give lower dice rolls!
Those of you speaking about the ability to roll a zero (only in battles not siege rolls) do not understand statistics or probability. Are you familiar with sigma events? This is just a game but I hate dishonesty.

As you say yourself, a sample size of 100 is far too small to be drawing blanket conclusions from. Also I'd like to point out that there's no such thing as a truly random number generator, but these purposes it is indeed sufficiently "random."

 

[Pile_O_Gunz]

Here is some more evidence that the EU4 system may be dishonest. I used a random number generator of 1-14. I wrote down all the results for a 100 roll sample size. The mean is 7.79 with a standard deviation of 4.14. I got 9 rolls of a "14" and 5 rolls of a "1". This data set is small but the results are within line of the randomness of a 1:14 chance and the data sets falls within the standard deviation. Statistically speaking, this data set is truly random.
Going back to the EU4 data set which is supposedly random, the results do NOT fall within the standard deviation. The results are so horribly out of the statistical norm that the only rational explanation is that that random number generator that EU4 uses is horribly skewed to give lower dice rolls!
Those of you speaking about the ability to roll a zero (only in battles not siege rolls) do not understand statistics or probability. Are you familiar with sigma events? This is just a game but I hate dishonesty.

Out of curiousity I wrote some JS code you can play with:
https://jsfiddle.net/pff9pxvs/3/

Just change var rolls = 100; to however many you want and hit run.
It will generate rolls and show you the counts for each number.

As you can see with 100 rolls your luck can be all over the place.

 

[Gratak]

Out of curiosity. Did you check for ones and fourteens or disease outbreaks and breaks? And if the latter, is anyone sure that you always get a breach with a 14? You certainly can get wall breaches with lower number having artillery and out-dated forts. No-artillery and up-to-date fort may as well prevent them entirely if you are at zero siege progress.

 

[Adadat]

Wouldnt you need a Chi square test or something of that sort in order to see if the difference is actually statistically significant or due to random chance? I am afraid that without it you cant really draw any conclusions.

 

[Ziggy187]

Out of curiousity I wrote some JS code you can play with:
https://jsfiddle.net/pff9pxvs/3/

Just change var rolls = 100; to however many you want and hit run.
It will generate rolls and show you the counts for each number.

As you can see with 100 rolls your luck can be all over the place.

Wow, thank you for the effort on this post. Unfortunately I am not a programmer and don't follow your Java script. I simply enter my data into excel and calculate the mean and std dev. from that. I understand that obtaining data is very tedious but the small amount I have gotten thus far is leaning towards my hypothesis that the random dice rolls EU4 uses are not so random. I just wish the dev's would admit it if that is the case. Naturally, if someone gets a data set that shows fair a equitable distribution, they would not bother to post that. I would like to be fair and honest in this endeavor and I will attempt to acquire a larger sample set of data and post the results regardless of how they may help or discredit my theory.

 

[iquabakaner]

Considering the number of sieges an average player gets per game, getting several 1's (or many 1's) in a row isn't really that unlikely.............

 

[tbhayward]

One standard deviation is the square root of the number of counts. If there are 14 sides to a die, you have a 7.14% chance of rolling each number. If there are 100 roles 7.14 roles should be a one. The statistical uncertainty is the square root of the number of counts. If you rolled 18 ones then that is (18-7.14)/sqrt(18) = 2.56 standard deviations. A 2.5 standard deviation result means that there is about a 1% chance that it is a natural result. I am currently working on a Ph.D. in physics and this is something we would consider suggestive, but not at all conclusive. I may try this out myself.

 

[IndraSunrise]

You can get multiple disease outbreaks in a seige.
You can only get one breach.
Be careful when gathering data.

 

[Threshold]

You can get multiple disease outbreaks in a seige.
You can only get one breach.
Be careful when gathering data.

Well actually three, but your point still stands: It is possible to get umpteen bajillion disease outbreaks after it is no longer possible to get more breached modifiers on a given siege.

 

[Kagemin]

You can get multiple disease outbreaks in a seige.
You can only get one breach.
Be careful when gathering data.

Can you no longer get three breaches?
Edit: Nvm

 

[Ziggy187]

To clarify, I am not specifically talking about "wall breaches" versus "disease outbreaks", that is the results of the rolls. All I am considering in this exercise is the dice rolls themselves and whether or not they have a fair and equally weighted outcome distribution. I understand the randomness of any singular roll, statistically, any dice roll should have a 1:14 chance of occurring. The data set thus far suggests that the game mechanics "may" be purposefully skewed. That is the goal of this exercise, to compile enough data points to confirm or discredit this hypothesis. As demonstrated by tbhayward, getting the results I got for the original post had a 0.006% chance of occurring naturally. As rare as that may seem, it is not conclusive at this point in time because that means it can repeat itself 6 times for every 1000 sets of 100 dice rolls.
I would like to think PDX is being honest with their claim that the random dice generator is truly random. However, my own experience suggests otherwise and that may be my "confirmation bias" at work. Thus, I need to definitively convince myself with actual data sets and not a gut feel. Maybe others posting here have already run this experiment and they are frustrated that this topic keeps surfacing. That is also why I want to post the results once I get the data sets compiled.