Mnplastic AARs rating

unmerged(47028) · Jan 11, 2011

It is useful sometimes to know quality or rating of AARs especially when there are hundreds or even thousands of them in each game category. I have tried many times to understand Paradox rating system but to no avail. At the moment “Biplanes available - Cheap!” advertisement holds the second place in the HOI3 AAR forum. This doesn’t make any sense.

I have therefore created my own system. I am sure it is very subjective in some sense but at least makes some sense to me and I hope to you as well.

I have decided that decided that the most important available information is the numbers of Posts, Author Posts and Reviews. The proportion of comments to all Posts shows an interest in discussing an AAR, while the proportion of Posts to all Views shows an interest to follow the AAR.

Mathematically I have expressed my thought into the formula:

Rating = (1-AP/TP)/2-ln(TP/VW)/10

Where:
AP – Author Posts
TP – Total Posts
VW – Views

I have disqualified all AARs where there less than 100 total posts and author had less than 20 posts or less than 30 posts and have not completed AAR. The interactive and succession game AARs are left out.

I have also separated Tutorial AARs from standard AARs. Please see the results yourself.

Top 40 HOI3 AARs 06.01.12

Top 10 HOI3 Tutorial AARs 11.01.11

Top 20 AoD AARs 22.01.11

There are several AoD interactive AARs which scored relatively low but I am sure this is due to different style. I have counted all posts of participating writers as "author's". While it works well with MP AARs I am not sure it is a good method for Interactive AARs.

The only Tutorial: An Arsenal of Democracy Review-AAR scored 0.91

Top 30 Victoria 2 AARs 23.01.11

Tutorials:
Under the Rising Sun - A V2 Review AAR Rating 0.99
V2 Beta AAR : France 1836 - A Learning The Game AAR Rating 0.76

Due to the low number of AARs I have included AARs with 15+ Author posts and/or AARs with 80+ total posts.

Top 20 Rome AARs 23.01.11

Due to the low number of AARs I have included AARs with 15+ Author posts and/or AARs with 80+ total posts.

Top 60 EU3 AARs 02.02.11

Courtesy of Dewirix

Due to the high number of AARs, only AARs with 200+ total posts are included.

Svip · Jan 11, 2011

Isn't the correct term 'views', rather than 'reviews'? Sounds like a lot of people wrote a detailed review on the AAR.

And while we are being pedantic, isn't there a problem with considering author posts regarding MP AARs, as in some cases, many contributors speak for the AAR, i.e. more than one author. Is this taken into account?

unmerged(47028) · Jan 11, 2011

I have added posts which were contribution to storyline. However I can not guarantee 100% accuracy

I will amend Reviews to Views - good spot

Edit: I have amended now

Director · Jan 20, 2011

Despite this I am not convinced that number of posts, views or replies determines the quality of a work.

Please advise which moderators you secured permission from before running this initiative?

unmerged(47028) · Jan 22, 2011

Director said:
Despite this I am not convinced that number of posts, views or replies determines the quality of a work.

Yes you are right, but this is the only readily available information. It is like the UK Top 40 do not determine the quality of songs because it is based only on sales. Nevertheless The Anthology of HoI AARs and The Anthology of EUII AARs are based on the same data I am using. However, having in mind that writing longer automatically attracts more views and posts I think it is unfair for shorter AARs which could have been in the anthologies. This is why I have used the above mentioned proportions with which I have tried to separate "quality" from the effect longevity of some AAR's.

Director said:
Please advise which moderators you secured permission from before running this initiative?

No, I have not. The reasons I wrote in a PM.

quaazi · Jan 22, 2011

I didn't know you had to check with mods to make a thread to discuss AARs.

I think the best word to describe the quality that this rating system gauges is "health": how healthy an AAR is. Just outta interest, apparently my AAR's rating would be 0,676, but then again, I'm awful at math.

unmerged(47028) · Jan 22, 2011

quaazi said:
I didn't know you had to check with mods to make a thread to discuss AARs.

I think the best word to describe the quality that this rating system gauges is "health": how healthy an AAR is. Just outta interest, apparently my AAR's rating would be 0,676, but then again, I'm awful at math.

I don't think you could compare the HOI3 rating with the AoD because of different traffic. Your AAR with 0.676 could be in top 10 in AoD

Because of that reason I would never risk to pick the best Paradox AAR.

Edit: I have posted TOP 20 AoD AARs 22.01.11

Rensslaer · Jan 23, 2011

Thanks, MnPlastic, for doing this calculation! It's interesting that my AAR scored higher than Myth's despite his having so many more views, and an approximately similar distribution of author posts to other posts.

I'll agree that it's not necessarily equivalent to quality, but given the information we have it is a reasonable approximation within some bounds. The number of non-author posts helps indicate relative interest level for an AAR, but I also think there are AARs which are really good but don't get noticed. I get a boost because of name recognition, I think. It also helps that I started mine very soon after the game came out -- something which might be hard to calculate, but which I think greatly impacts overall viewcount and number of non-author posts (look at Singleton Mosby's tutorial AAR on Barbarossa, for instance -- great quality, but many of those posts and views pre-dated even the game release! -- quite an achievement!).

Sometimes I gauge how well my AAR is doing by the number of views per week, for instance. Did you calculate the amount of time running at all?

This sounds similar to the "Phatness Index" which... I think it was Sir Humphrey who developed it (forgive me if I've misattributed!). Similar criteria used.

Rensslaer

TekcoR · Jan 23, 2011

This is really quite interesting and I like the breakdowns. I was wondering if you were going to do anything like these ratings for Victoria 2?

unmerged(47028) · Jan 23, 2011

Rensslaer said:
Thanks, MnPlastic, for doing this calculation! It's interesting that my AAR scored higher than Myth's despite his having so many more views, and an approximately similar distribution of author posts to other posts.

I'll agree that it's not necessarily equivalent to quality, but given the information we have it is a reasonable approximation within some bounds. The number of non-author posts helps indicate relative interest level for an AAR, but I also think there are AARs which are really good but don't get noticed. I get a boost because of name recognition, I think. It also helps that I started mine very soon after the game came out -- something which might be hard to calculate, but which I think greatly impacts overall viewcount and number of non-author posts (look at Singleton Mosby's tutorial AAR on Barbarossa, for instance -- great quality, but many of those posts and views pre-dated even the game release! -- quite an achievement!).

Sometimes I gauge how well my AAR is doing by the number of views per week, for instance. Did you calculate the amount of time running at all?

This sounds similar to the "Phatness Index" which... I think it was Sir Humphrey who developed it (forgive me if I've misattributed!). Similar criteria used.

Rensslaer

You are right time is very important because longer AARs exist more they attract views. I also suspect, that more pages AARs have more views they attract too because each page counts as 1 view, I might be wrong. This is why I have included "ln" function in posts/view section of formula. I know that this is not ideal way to do it but it somehow dilutes the time and number of pages effects.

I am happy that in the HOI3 TOP 40 many shorter and long forgotten AARs appeared, even in the TOP 10 spot. I have also printed the "best" 40 and not 10 or 20 because I think the Rating is not perfect and it is good to show big variety of AARs with some extra information like Country. It would be great to include also "Style" (ie. narrative, history, comedy etc.). The Rating showed that there is a big interest in "Minors'" AARs (look Portugal, Finland and Hungary in the top 10 HOI3, or Yugoslavia, Jamaica (Tropical Country) and Ethiopia in the top 10 AoD).

BTW, I have noticed that the AoD thread has many experimental AAR

unmerged(47028) · Jan 23, 2011

TekcoR said:
This is really quite interesting and I like the breakdowns. I was wondering if you were going to do anything like these ratings for Victoria 2?

I might do Victoria 2, but when I saw how many AARs in EU, Vicroria, HOI1-2 or CK threads I almost gave up the idea. I'll become grey before I ever finish :O

Edit: I have done Victoria 2 rating but I am sure the TOP 20 will be very volatile in the next 6-12 months

Rensslaer · Jan 23, 2011

Hmm... I'm not sure it helps the accuracy to penalize (if that's what you're doing) for the number of pages. If an AAR gets a point/viewpoint for every page read, that just means people are interested enough to keep reading.

Yes, to contemplate going through hundreds of AARs is quite daunting!

I'm having a hard enough time psyching myself up to read enough to judge for the AARLand Choice Awards -- I'll enjoy the reading, but it'll take a while to get through so many to make sure I'm not missing a great AAR.

It would also be helpful if in the chart you put the author next to the title, maybe in parentheses. Sometimes I don't remember the exact name of an AAR, but I'll think of it as "Robou's AAR" or whatever, so the author name would help remember which is which.

Rensslaer

Tommy4ever · Jan 23, 2011

mnplastic said:
I might do Victoria 2, but when I saw how many AARs in EU, Vicroria, HOI1-2 or CK threads I almost gave up the idea. I'll become grey before I ever finish :O

Edit: I have done Victoria 2 rating but I am sure the TOP 20 will be very volatile in the next 6-12 months

Why did you not include ''let the ruling classes tremble''?

With 160,000 views and 4,500 replies it should surely make it in. I just wanted to see where it would rank on your list.

Enewald · Jan 23, 2011

Tommy4ever said:
Why did you not include ''let the ruling classes tremble''?

With 160,000 views and 4,500 replies it should surely make it in. I just wanted to see where it would rank on your list.

Quite high, at the moment 4598 replies, you have 426 posts there.
And I have 293 posts there.

TekcoR · Jan 23, 2011

I'm a bit perplexed on the rating for Victoria 2 AARs, based on the the list of AARs there certainly my French AAR, would be there. I am amazed some of the other writers on there, and some scored higher than others. Do you have a list that expands further than the top 20?

unmerged(47028) · Jan 23, 2011

Rensslaer said:
Hmm... I'm not sure it helps the accuracy to penalize (if that's what you're doing) for the number of pages. If an AAR gets a point/viewpoint for every page read, that just means people are interested enough to keep reading.Rensslaer

Lets assume that somebody wrote a book 300 pages and somebody else 500. Now 3 people read the first book and only 2 read the second. If we count views then the second book would be a winner because 1000 pages were read.

unmerged(47028) · Jan 23, 2011

Tommy4ever said:
Why did you not include ''let the ruling classes tremble''?

With 160,000 views and 4,500 replies it should surely make it in. I just wanted to see where it would rank on your list.

Enewald said:
Quite high, at the moment 4598 replies, you have 426 posts there.
And I have 293 posts there.

This is the same problem as I had with AoD. Interactive AARs are so different from others that it is difficult to fit them under the same system. In this specific case this happened because number of views compared to total post is not high enough - just 40 views per post.

I am open to suggestions how to objectively deal with interactive and multiplayer AARs.

Edit: I just found out that there was a contest of Victoria 2 AARs.

The comparison of the first 5 places:

Contest v Rating:
1 v 10
2 v 30
3 v 6
4 v 9
5 v 8

Out of 5 AARs 4 are in my top 10

Contest v Views

1 v 4
2 v 29
3 v 14
4 v 3
5 v 25

Out of 5 AARs 2 are in top 10 views

Mnplastic AARs rating

Field Marshal

Enewald Enewald Enewald

Caramelised Utopian

Field Marshal

Unpaid Ulm PR employee

Field Marshal

Maestro

Field Marshal

Colonel

Field Marshal

Strategy GuidAAR

Alternative Affairs Specialist

Field Marshal

Field Marshal

Strategy GuidAAR

Papa Bear

Enewald Enewald Enewald

Alternative Affairs Specialist

Field Marshal

Field Marshal