Siege Rolls; Confirmation Bias my ARSE!!!

[zedyue]

@Wiz This is just idle curiosity on my part but is the coding for sieges done separately for all sieges, separately for all countries, separately for ai vs player? Like I don't know anything about programming so idk if my question is dumb or not. (I believe it's all random btw I just wanna know a little how it works)

 

[artemis667]

@Wiz This is just idle curiosity on my part but is the coding for sieges done separately for all sieges, separately for all countries, separately for ai vs player? Like I don't know anything about programming so idk if my question is dumb or not. (I believe it's all random btw I just wanna know a little how it works)

They'll have one piece of code that executes each siege roll. It gets called repeatedly by the game engine everytime a siege ticks, different parameters will get passed to it (siege modifiers, basically).

There's other code that handles the 'ticking' of sieges in provinces.

It would definitely require a lot of extra effort to make the game 'cheat' these rolls in favour of the AI against the player, and I'm scratching my head to find a coherent reason for them to do it (It doesn't, of course, it's very clearly confirmation bias). The AI does 'cheat' in other unrelated ways as Paradox have disclosed (even without lucky nations), but only when the devs found it necessary to make it do so for reasonable competitive reasons.

 

[artemis667]

Here you go, with larger sample size as well. Bonus is that the AI in this particular case is the Ottomans, which sometimes might "feel" like an unfair foe.

How high do you need to get N before you can expect those lines to flatten out? I know enough about stats to know that that value of N would be very large, just not sure how much.

 

[tbhayward]

How high do you need to get N before you can expect those lines to flatten out? I know enough about stats to know that that value of N would be very large, just not sure how much.

Well that's the thing, they'll never really flatten out unless you zoom out. Even if you had a sample size of a billion, about 1/3 of the error bars should still not intersect the expected result. It's just that at that point the error bars will be tiny! So it really depends on what you mean by flat. You could set the scale in that plot to go from, say, 0 to 500 and it would look really flat.

 

[artemis667]

Each bar within a few percent of it's neighbours in height, I suppose. Let's say 5%.

I probably should learn the maths to do this - actually did a long time ago (albeit not as well as I should have), but never used it and forgot completely

What I mean is, that chart would suggest that a 4 is a lot more likely than 2, for example, but I'm skeptical that it actually says that in a statistically significant way. There's a lot of error even in simple two-way polling for N=500, let alone when you have 14 values.

 

[jcranmer]

I mean, if you prove that the mersienne twister is biased in such a precisely specific way that somehow it always ends up giving favorable siege rolls to the AI in Europa Universalis 4, you're probably gonna make a whole lot of people look dumb.

I don't think it's very likely though.

More likely someone made a mistake transcribing the Mersenne Twister code. Although, to be honest, I use C++11 mostly, so I just need to #include <random>, use std::mt19937, and call it day.

@Wiz This is just idle curiosity on my part but is the coding for sieges done separately for all sieges, separately for all countries, separately for ai vs player? Like I don't know anything about programming so idk if my question is dumb or not. (I believe it's all random btw I just wanna know a little how it works)

Conceptually speaking, there's something in the code that keeps track of what needs to happen on every daily update (how this is actually implemented, I don't know, but it doesn't matter). Since the game is in C++, there's basically a class along the lines of Siege that has a function that's something like DailyChange that implements this callback. It would be possible to make two classes, one PlayerSiege and one AISiege, or possible to make Siege:ailyChange check if (sieger->IsAI()) doSomething();, but such divergence is usually more messy to implement and maintain. This is why the concept of AI dice rolls being biased is unlikely--you'd have to add an extra parameter to basic dice rolls functions to say "should I bias this?", which needs to be passed through--and kept logically consistent--throughout much of the code base.

 

[tbhayward]

Each bar within a few percent of it's neighbours in height, I suppose. Let's say 5%.

I probably should learn the maths to do this - actually did a long time ago (albeit not as well as I should have), but never used it and forgot completely

What I mean is, that chart would suggest that a 4 is a lot more likely than 2, for example, but I'm skeptical that it actually says that in a statistically significant way. There's a lot of error even in simple two-way polling for N=500, let alone when you have 14 values.

Well, just back of the envelope calculation, if you want the standard deviation of each roll to be 5% of the number of rolls, you need:
0.05 = sqrt(n)/n
n = 400
or N = 400 x 14 = 5600 rolls. But even at that point you would still have 31% of the results being outside of 5%. So If you wanted to have, say, a 99% certainty that all results would be within 5% of each other (this is 2.5 sigma), you'd need n = 2500 or N = 35000. 5 sigma would be N = 140000. It's kind of a weird question to answer though because it could happen at any moment and it could even happen and then more rolls make it not true anymore.

But I'm trying to watch soccer at the same time as this so maybe I made an error.

 

[alexti]

No, it just means that such an "unlucky" situation happens around once in a hundred times. How many sieges does an average player gets into? How many players are there in EU4? One in a hundred is not unlikely at all.

That line of reasoning usually works, but in this case OP had a hypothesis that the rolls are skewed, so he has recorded a serie of 100 rolls and observed unlikely results. This is different from the situation when one out of thousands of players observed unlikely result and posted it. In OP case, his chance of drawing the results he has observed is independent from total number of players and is the actual probability of that draw happening. Of course, there might have been a set of players who have had a hypothesis that the rolls are skewed and decided to test the theory and only one of them (OP) got unlikely draw and posted. But that set of players would be a much smaller number than total number of players

Nah, it really doesn't. As the poster above me says it merely means that unlucky situation happens about 1/100th of the time. That's not unlikely at all. For a little context, if I wanted to publish a physics "discovery" in a peer reviewed journal, I would need 5 standard deviation statistics, or 99.9999426697% chance that what I was observing wasn't statistical anomaly.

What formula are you using to get that 1/100? I have suggested probability of rollign exactly 18 '1's out of 100 trials is C(100,18)*pow(p,18)*pow(1-p,100-18) = 0.00016. Another poster said 0.00024 probability that there will be at least 18 '1's. Obviously, at least 1 out of these 3 answers is wrong and I am not entirely confident in my calculation, in part because large factorials often suffer from machine precision.

There is also a big difference between what you would publish as physics discovery and what you would investigate in the program behaviour. Those kind of issue are not uncommon due to the way pseudo-random sequences are constructed and used and there are many poor implementations around.

I think the bigger issue is the fact pure RNG is used which will sometimes be unfair by its very nature. It would perhaps be cool to try something that adjusts the random variable's expected value to cancel out mean unbalanced series after some number of outcomes.

You could use quasi-random numbers, they have lower discrepancy, but they need to be used in correct manner (not one generator for everything)

P(100 rolls of a 14-sided die yields at least 18 1s) ~= 0.00024, which is to say, if 10K people did this experiment and the true distribution were unbiased, we should expect to see 24 people having a result at least as lopsided as yours. Now, it's easy to say, "p < 0.05, we reject the null hypothesis!", but to actually say that is p-hacking. You're posting this message precisely because the results is so unfair not in your favor, so you're ignoring the results of all the runs which didn't seem so unfair.

I agree with your line of reasoning, but it is hard to believe that there would be 10K (or even 1K) people who would run this experiment. Chances are there aren't many people determined enough to record siege roll results.

I mean, if you prove that the mersienne twister is biased in such a precisely specific way that somehow it always ends up giving favorable siege rolls to the AI in Europa Universalis 4, you're probably gonna make a whole lot of people look dumb.

I don't think it's very likely though.

So PRNG is Mersenne-Twister? I don't think anyone is going to find a serious flaw in its distribution, however it's sensitive to initialization, particularly to excess of zeroes in initialization state. Usually people use some kind of bootstrapping algorithm to initialize the state, but if it's done by just using something "random" for initialization it may take MT sequence quite a while to recover good statistical properties.

 

[Philadelphus]

How high do you need to get N before you can expect those lines to flatten out?

Intuitively, I would guess N would need to be infinite for the graph to be truly, perfectly, "flat". The distribution of rolls should converge to a flat distribution as the number of trials goes to infinity.

What formula are you using to get that 1/100? I have suggested probability of rollign exactly 18 '1's out of 100 trials is C(100,18)*pow(p,18)*pow(1-p,100-18) = 0.00016. Another poster said 0.00024 probability that there will be at least 18 '1's. Obviously, at least 1 out of these 3 answers is wrong and I am not entirely confident in my calculation, in part because large factorials often suffer from machine precision.

Well, there's a very important distinction between rolling exactly 18 1's and rolling at least 18 1's. The latter should have a slightly higher probability, as it indeed appears to, so both could be correct.

 

[alexti]

Intuitively, I would guess N would need to be infinite for the graph to be truly, perfectly, "flat". The distribution of rolls should converge to a flat distribution as the number of trials goes to infinity.

Intuitively, I would guess that the graph will become perfectly flat when the deviations becomes smaller than what your screen resolution can display

 

[lilunxm12]

My England PUed Savoy and after 50 years the tooltip says there's 26% chance of inheritence. I S&L for 20+ times for more than 30 year period but it's still there. Before then, I've never had a ruler older than 85 year and I finally gave up.

 

[bbqftw]

My England PUed Savoy and after 50 years the tooltip says there's 26% chance of inheritence. I S&L for 20+ times for more than 30 year period but it's still there. Before then, I've never had a ruler older than 85 year and I finally gave up.

Maybe the inheritance roll is already set for each ruler?

There's pretty compelling evidence that there is a singular inheritance roll since if you inherit your least likely PU, you are guaranteed to inherit all your PUs.

 

[Philadelphus]

Intuitively, I would guess that the graph will become perfectly flat when the deviations becomes smaller than what your screen resolution can display

They'll still be there though...lurking...biding their time...just waiting for you to zoom in on the graph and THERE THEY ARE AGAIN!

 

[tbhayward]

What formula are you using to get that 1/100? I have suggested probability of rollign exactly 18 '1's out of 100 trials is C(100,18)*pow(p,18)*pow(1-p,100-18) = 0.00016. Another poster said 0.00024 probability that there will be at least 18 '1's. Obviously, at least 1 out of these 3 answers is wrong and I am not entirely confident in my calculation, in part because large factorials often suffer from machine precision.
.

What I mean is that by being 2.5 standard deviations away from the expected results of 100/14 there is a 1% chance that the result was statistical randomness and 99% chance that it wasn't random. That's different than calculating the probability of a specific result.

Maybe the inheritance roll is already set for each ruler?

There's pretty compelling evidence that there is a singular inheritance roll since if you inherit your least likely PU, you are guaranteed to inherit all your PUs.

Yeah, I thought it was established that these were prerolled?

 

[alexti]

My England PUed Savoy and after 50 years the tooltip says there's 26% chance of inheritence. I S&L for 20+ times for more than 30 year period but it's still there. Before then, I've never had a ruler older than 85 year and I finally gave up.

This is a different scenario. S&L doesn't guarantee independent draws with any particular distribution. I think that PU inheritance specifically works very differently. It's hard to pinpoint it, but a while ago I've done some observations and my theory is that the game periodically (perhaps few times per year) rolls what will happen on monarch death and when monarch death happens it just used the previously rolled outcome.

 

[alexti]

What I mean is that by being 2.5 standard deviations away from the expected results of 100/14 there is a 1% chance that the result was statistical randomness and 99% chance that it wasn't random. That's different than calculating the probability of a specific result.

That's exactly what I said in the post you have disagreed with So my point was that it's worth investigating if there's 99% chance that this is not a random result.

 

[tbhayward]

Yes, exactly. And what I really was ultimately meaning was that, coming from the logical good faith assumption that the dice rolls are fair, it wasn't really alarming. At this point, I'd say between the 1000 or so die rolls that the OP, Lilje and I have posted, it's conclusive that that first post was statistical randomness.

 

[Fanstar1]

Most likely confirmation bias

 

[alexti]

Yes, exactly. And what I really was ultimately meaning was that, coming from the logical good faith assumption that the dice rolls are fair, it wasn't really alarming. At this point, I'd say between the 1000 or so die rolls that the OP, Lilje and I have posted, it's conclusive that that first post was statistical randomness.

I think that logical good faith assumption here is that the dice rolls are *intended* to be fair. I kind of see that you are coming from the viewpoint that the underlying process is truly random and use results from different systems as confirmation. But the excess of '1's (which is likely means excess of zeroes in lower bits) is fairly suspicious. I would be much less suspicious of too many '14's. The underlying process is obviously non-random and programming errors disproportionally favor getting excess of zeroes, so possibility that something goes wrong on OP's system is hard to exclude.

 

[Yxklyx]

Just write a small program to roll the dice 100 times and see what kind of deviations you get and see if they correlate to what you saw or not. Run it many times.