• We have updated our Community Code of Conduct. Please read through the new rules for the forum that are an integral part of Paradox Interactive’s User Agreement.
Greetings, fellow crusaders!

In this Dev Diary we’ll give a brief insight in how the CK2 QA team works and functions.

The team consists of myself (Servancour), PeterSkager, rageair and Leo Larsson. I’ve been here the longest and have been at Paradox for close to two years now, and so far it’s been a blast! We’re a close-knit team and work very well together, each and every one specializing in different types of gameplay.

As a team, we work very closely with the developers and attend all sorts of meetings to ensure that we stay up to date with the game. When you work with QA you need to know almost everything about the game, from overarching design to the smaller details of specific features. It’s challenging but also very rewarding.

While our primary responsibility may be to test the game, it’s not the only one. Many believe that the life of a QA only consists of reporting bugs, but it’s actually much more than that. We’re responsible for ensuring the overall quality of the product, by providing constant feedback, voice concerns where needed, comment on design and suggest improvements in order to make the game both better and more accessible!

Something we do every now and then when there’s new features to test is to simply play the game in order to get a feel for how to play with them. We usually test a variety of different scenarios, with varying start dates and setups, where we play for anything between a few years to several centuries. Sometimes we play for just a few hours, other times we have the campaigns going over the course of a few days, all while we gather feedback and impressions to the developers. This type of testing allows us to get a more accurate representation on how the game actually plays to find issues you wouldn’t find in a few minutes of playtime. Sometimes we even start using the same scenario just to see how far we can deviate from each other as we use very different playstyles. We believe that having many eyes on the same problem yield the best results.

We’re extremely passionate about the game. Each and every one of us here at the CK2 QA team has played the game for anything between a few hundred to several thousand hours and most of it has been purely for our own enjoyment. I myself play the game regularly outside of work and in my latest campaign I started off as Charlemagne himself, culture switched to German and created the Holy Roman Empire! I’m almost done with it though, with just the last hundred years or so left before the end date. My current ruler is Kaiserin Amalia “The Hammer”, ruling over Europe with an iron fist!

This passion also motivates us to not only ensure that the game performs well from a professional standpoint, but also from the perspective of a player. This causes us to often go above and beyond our ordinary tasks! We at QA has, on occasion, contributed with content to the game. Often to ensure that each and every part of the game feels like a quality experience. For example; we’ve created events to enhance certain aspects of the game (see the Sky Burial and Master Wrestling event chains) and even created art in the form of the Nomadic CoA’s introduced in Horse Lords.

I hope you gained some useful insight in how we approach CK2 from the QA team, and that we work hard to give you, the fans, a product that you deserve. Since we at QA love quality of life improvements, I’ll wrap this up by showing you the following screenshot of a previous feature we are extending somewhat:

ck2_diary_screen.jpg
 

Zarathustra_the

Lt. General
85 Badges
Nov 3, 2011
1.645
985
  • Victoria 2: A House Divided
  • Hearts of Iron III: Their Finest Hour
  • Heir to the Throne
  • King Arthur II
  • The Kings Crusade
  • Magicka
  • Cities in Motion
  • Europa Universalis IV: Res Publica
  • Victoria: Revolutions
  • Semper Fi
  • Naval War: Arctic Circle
  • Victoria 2: Heart of Darkness
  • Warlock 2: The Exiled
  • Europa Universalis IV: Third Rome
  • Crusader Kings II: Monks and Mystics
  • Europa Universalis IV: Mare Nostrum
  • Stellaris - Path to Destruction bundle
  • Stellaris: Galaxy Edition
  • Crusader Kings II: Reapers Due
  • For the Motherland
  • Crusader Kings II
  • Crusader Kings II: Charlemagne
  • Crusader Kings II: Legacy of Rome
  • Crusader Kings II: The Old Gods
  • Crusader Kings II: Rajas of India
  • Crusader Kings II: The Republic
  • Crusader Kings II: Sons of Abraham
  • Crusader Kings II: Sunset Invasion
  • Crusader Kings II: Sword of Islam
  • Europa Universalis III
  • Hearts of Iron III
  • Europa Universalis IV: Call to arms event
  • Europa Universalis IV: Wealth of Nations
  • Europa Universalis IV: Conquest of Paradise
  • Europa Universalis IV: Art of War
  • Divine Wind
  • Europa Universalis III: Chronicles
  • Hearts of Iron IV: No Step Back
  • Europa Universalis IV
  • Europa Universalis IV: El Dorado
  • Europa Universalis IV: Common Sense
  • Crusader Kings II: Horse Lords
  • Europa Universalis IV: Cossacks
  • Magicka 2: Ice, Death and Fury
  • Crusader Kings II: Conclave
  • Stellaris
  • Warlock: Master of the Arcane
  • Hearts of Iron IV: Cadet
  • Hearts of Iron IV: Colonel
  • Hearts of Iron IV: Field Marshal
This seems like a good place for

Flanders and Aquitaine for France!
Can people just shut up about the Flanders issue,... I agree with your point, but I am getting sick of seeing it posted everywhere. It is about as annoying as the bloody china bandwagon!
 
  • 13
  • 12
Reactions:

majorlupa

Captain
35 Badges
Jun 30, 2010
467
511
  • Crusader Kings II: Charlemagne
  • Europa Universalis IV: Mare Nostrum
  • Europa Universalis IV: Res Publica
  • Europa Universalis IV: Call to arms event
  • Europa Universalis IV: Wealth of Nations
  • Europa Universalis IV: Conquest of Paradise
  • Europa Universalis IV: Art of War
  • Crusader Kings II: Sword of Islam
  • Crusader Kings II
  • Crusader Kings II: Sons of Abraham
  • Crusader Kings II: The Republic
  • Crusader Kings II: Rajas of India
  • Crusader Kings II: The Old Gods
  • Crusader Kings II: Legacy of Rome
  • Europa Universalis IV: Golden Century
  • Crusader Kings II: Holy Fury
  • Europa Universalis IV: Dharma
  • Europa Universalis IV: Rule Britannia
  • Crusader Kings II: Jade Dragon
  • Europa Universalis IV: Cradle of Civilization
  • Europa Universalis IV: Third Rome
  • Europa Universalis 4: Emperor
  • Europa Universalis IV: Mandate of Heaven
  • Crusader Kings II: Monks and Mystics
  • Europa Universalis IV: Rights of Man
  • Crusader Kings II: Reapers Due
  • Hearts of Iron IV Sign-up
  • Crusader Kings II: Conclave
  • Europa Universalis IV: Cossacks
  • Crusader Kings II: Horse Lords
  • Europa Universalis IV: Common Sense
  • Crusader Kings II: Way of Life
  • Europa Universalis IV: El Dorado
  • Europa Universalis IV
  • Crusader Kings II: Sunset Invasion
Luckily people here understand things like math. If 10,000 people play for an undefined "few" hours, lets say 3, that is 30,000 hours of testing. For 4 QA working 8 hour days that would be ~2.5 years of testing. Which doesn't even consider that QA aren't testing the final version for the entire period, or that fixing one bug can reveal other bugs, unbalanced things, or exploits.

Why would someone continue to maintain and pay for 4 QA, while releasing closed beta for around 100-200 people would be better choice(imo) ?
 
  • 6
Reactions:

Parsidius

First Lieutenant
83 Badges
Feb 17, 2007
205
95
  • Europa Universalis IV: Res Publica
  • Hearts of Iron III
  • Hearts of Iron III: Their Finest Hour
  • Heir to the Throne
  • Europa Universalis III Complete
  • Knights of Pen and Paper +1 Edition
  • Lead and Gold
  • The Kings Crusade
  • Magicka
  • Majesty 2
  • Europa Universalis III Complete
  • Penumbra - Black Plague
  • Pirates of Black Cove
  • For the Motherland
  • Victoria: Revolutions
  • Rome Gold
  • Semper Fi
  • Sengoku
  • Ship Simulator Extremes
  • Sword of the Stars
  • Sword of the Stars II
  • Supreme Ruler: Cold War
  • Victoria 2
  • Victoria 2: A House Divided
  • Victoria 2: Heart of Darkness
  • Victoria 3 Sign Up
  • Darkest Hour
  • Hearts of Iron II: Armageddon
  • Cities in Motion
  • Crusader Kings II
  • Crusader Kings II: Charlemagne
  • Crusader Kings II: Legacy of Rome
  • Crusader Kings II: The Old Gods
  • Crusader Kings II: Rajas of India
  • Crusader Kings II: The Republic
  • Crusader Kings II: Sons of Abraham
  • Crusader Kings II: Sunset Invasion
  • Crusader Kings II: Sword of Islam
  • Commander: Conquest of the Americas
  • Arsenal of Democracy
  • Deus Vult
  • East India Company Collection
  • Europa Universalis III
  • Europa Universalis III Complete
  • Divine Wind
  • Europa Universalis IV
  • Europa Universalis IV: Art of War
  • Europa Universalis IV: Conquest of Paradise
  • Europa Universalis IV: Wealth of Nations
  • Europa Universalis IV: Call to arms event
Who says I haven't? :p
Seriously though, I could have conquered the world two times over most likely, but instead I pace myself when I play longer campaigns. I set these small goals for myself to do every now and then. Just to break up the regular gameplay of expanding and do something different.

Since this is the case, would it not make sense to expand the ambitions system? Or create some new functionality to create on-the-fly objectives for the player? I have bumped into the same issue, myself, where expanding becomes a grind but is often the only thing I feel is available to do.
 
  • 1
Reactions:

Jamey

Field Marshal
106 Badges
Sep 9, 2009
3.437
3.383
  • 500k Club
  • BATTLETECH - Backer
At the moment, we don't do a lot of automated testing. But this is definitely something we are looking to do much more in the future in order help us out with some of the more basic testing procedures.
Good luck with that. Covering a large existing code base with automated testing often feels thankless, but it is worth it when you start regularly catching new bugs as they are introduced. :)

I'm currently reading "Working Effectively with Legacy Code by Michael Feathers. I wish I read it before rather than after starting my current job. It's pretty good so far.
 

Birken

Mastermind
Paradox Staff
4 Badges
Sep 28, 2009
877
3.830
  • March of the Eagles
  • Pirates of Black Cove
  • 500k Club
  • Paradox Order
Why would someone continue to maintain and pay for 4 QA, while releasing closed beta for around 100-200 people would be better choice(imo) ?

Quality of reported bugs is the main reason. Beta testers are great, but they aren't paid to test and most of the time they'll test stuff that is fun. There are exceptions of course and some beta testers amaze us with the work they put in.
Paradox relied almost solely on beta testers up until V2 release (and then only as limited time contractors), EU4 was the first releases (as far as I remember with a in house team of QA). I'd say that quality has gone up since EU3 and HOI3 releases. QA isn't the only reason, but it's definitely one of the major.
 
  • 11
  • 7
Reactions:

Jamey

Field Marshal
106 Badges
Sep 9, 2009
3.437
3.383
  • 500k Club
  • BATTLETECH - Backer
Quality of reported bugs is the main reason. Beta testers are great, but they aren't paid to test and most of the time they'll test stuff that is fun. There are exceptions of course and some beta testers amaze us with the work they put in.
Paradox relied almost solely on beta testers up until V2 release (and then only as limited time contractors), EU4 was the first releases (as far as I remember with a in house team of QA). I'd say that quality has gone up since EU3 and HOI3 releases. QA isn't the only reason, but it's definitely one of the major.
The QA mindset is very different than both developers and typical players. I'm honestly amazed that Paradox put out so many games without having in house QA.

It does explain a bit about some of the issues I have with earlier Paradox games. EU4 was the most polished release I've experienced with Paradox games. That bodes well for future releases. :)
 
  • 4
Reactions:

aono

Field Marshal
113 Badges
Aug 26, 2008
4.021
3.432
  • Europa Universalis 4: Emperor
  • Prison Architect
  • Stellaris: Necroids
  • Stellaris: Nemesis
  • Victoria 3 Sign Up
  • Hearts of Iron IV: By Blood Alone
  • Hearts of Iron IV: No Step Back
  • Knights of Pen and Paper 2
  • Stellaris: Federations
  • Prison Architect: Psych Ward
  • Battle for Bosporus
  • Island Bound
  • Cities in Motion
  • Europa Universalis IV
  • Crusader Kings II
  • Crusader Kings II: Charlemagne
  • Crusader Kings II: Legacy of Rome
  • Crusader Kings II: The Old Gods
  • Crusader Kings II: Rajas of India
  • Crusader Kings II: The Republic
  • Crusader Kings II: Sons of Abraham
  • Crusader Kings II: Sunset Invasion
  • Crusader Kings II: Sword of Islam
  • Europa Universalis III
  • Europa Universalis III: Chronicles
  • Divine Wind
  • Europa Universalis IV: Art of War
  • Europa Universalis IV: Conquest of Paradise
  • Europa Universalis IV: Wealth of Nations
  • For the Motherland
  • Hearts of Iron III
  • Hearts of Iron III: Their Finest Hour
  • Hearts of Iron III Collection
  • Heir to the Throne
  • Europa Universalis IV: Res Publica
  • Victoria: Revolutions
  • Stellaris: Megacorp
  • Stellaris: Ancient Relics
  • Semper Fi
  • Victoria 2: A House Divided
  • Victoria 2: Heart of Darkness
  • Rome: Vae Victis
  • Europa Universalis IV: Pre-order
  • Cities: Skylines - After Dark
  • Europa Universalis IV: Cossacks
  • Cities: Skylines - Snowfall
  • Europa Universalis IV: Mare Nostrum
  • Stellaris
  • Stellaris: Galaxy Edition
  • Stellaris: Galaxy Edition
Not really.
QA can only inform the dev-team and producers about problems, they have no mandate to cancel or post-pone a game.
If someone is to blame it's either me or the producer (or anyone above us in the chain of command).
Well, I'd say it's kinda of compicated.
First of all, it's not that I despise testers or something like that. I'm in this business exactly for nearly 15 years, so I believe I know the drill.
But to be honest I believe we're speaking different things here.
There is bugs such as "hey, this new mechanics unlock some spicy way to win, a-ha!". It's not QA responsibility - as was said already, just simple math says us users always find way better. If they catch something like that, well, they supposed to describe it to programmers with something about "suggestion" tag. No arguing here.
Also there is bugs such as "hey, it's obvious we need this feature here for hundred of years". It's not QA responsibility again, it's for dev.
But there is bugs, such as, you know, bugs. Sometimes button to reform non-norse pagan (if I recall a case right) just disappears from interface. Or maybe levies just don't dissolve. I offer some obvious bugs, with no sane developer or producer will release build with such things, that's not something you just put "do-not-fix" and ignore. Once again, that's obvious ones, but there happens something more obscure, such as "AI never build anything" or "AI builds, but too rarely". I'm trying to speak about that ones I saw in forums here, not something I noticed and ignored.
So if that bugs appear it's became a work for QA to find it. If this bugs slips away, well, it's definitly valid to blame QA for that. Yeah, QA can say "we found exactly that one, there is created task in tracker, from given date, with that exactly developer (or producer) who said it's not matter". It takes responsibility for that particular bug to somebody who closed the task, but it doesnt' take away responsibility for not creating the one. So, when we're thinking that developing process is structured nice, blaming the QA for such things is completly normal.
I know perfectly that you can't fish all glitches like that, maybe some kingdom creating crushs a game when performing with some obscure culture or with some trait, but skipping something like that is QA responsibility. That's why I was urging Paradox to hire more testers - because more testers find bugs more effective.

Why would someone continue to maintain and pay for 4 QA, while releasing closed beta for around 100-200 people would be better choice(imo) ?
https://forum.paradoxplaza.com/forum/index.php?threads/beta-testing-and-qa.827222/
 

majorlupa

Captain
35 Badges
Jun 30, 2010
467
511
  • Crusader Kings II: Charlemagne
  • Europa Universalis IV: Mare Nostrum
  • Europa Universalis IV: Res Publica
  • Europa Universalis IV: Call to arms event
  • Europa Universalis IV: Wealth of Nations
  • Europa Universalis IV: Conquest of Paradise
  • Europa Universalis IV: Art of War
  • Crusader Kings II: Sword of Islam
  • Crusader Kings II
  • Crusader Kings II: Sons of Abraham
  • Crusader Kings II: The Republic
  • Crusader Kings II: Rajas of India
  • Crusader Kings II: The Old Gods
  • Crusader Kings II: Legacy of Rome
  • Europa Universalis IV: Golden Century
  • Crusader Kings II: Holy Fury
  • Europa Universalis IV: Dharma
  • Europa Universalis IV: Rule Britannia
  • Crusader Kings II: Jade Dragon
  • Europa Universalis IV: Cradle of Civilization
  • Europa Universalis IV: Third Rome
  • Europa Universalis 4: Emperor
  • Europa Universalis IV: Mandate of Heaven
  • Crusader Kings II: Monks and Mystics
  • Europa Universalis IV: Rights of Man
  • Crusader Kings II: Reapers Due
  • Hearts of Iron IV Sign-up
  • Crusader Kings II: Conclave
  • Europa Universalis IV: Cossacks
  • Crusader Kings II: Horse Lords
  • Europa Universalis IV: Common Sense
  • Crusader Kings II: Way of Life
  • Europa Universalis IV: El Dorado
  • Europa Universalis IV
  • Crusader Kings II: Sunset Invasion
Quality of reported bugs is the main reason. Beta testers are great, but they aren't paid to test and most of the time they'll test stuff that is fun. There are exceptions of course and some beta testers amaze us with the work they put in.
Paradox relied almost solely on beta testers up until V2 release (and then only as limited time contractors), EU4 was the first releases (as far as I remember with a in house team of QA). I'd say that quality has gone up since EU3 and HOI3 releases. QA isn't the only reason, but it's definitely one of the major.

Thanks for the answer.

Have you considered mixing things up? Having some QA people plus additional(don't know how many people gets to test the beta) close beta testers? I am sure some people would volunteer for that. I am asking because some of the Horse Lords exploits/cheesy tactics were easy to spot on by the average player.
 

Talq

Field Marshal
51 Badges
Sep 7, 2009
4.795
1.007
  • Crusader Kings II: Charlemagne
  • Europa Universalis IV: Mare Nostrum
  • Crusader Kings II: Holy Knight (pre-order)
  • Cities: Skylines Deluxe Edition
  • Warlock 2: The Exiled
  • Victoria 2: Heart of Darkness
  • Victoria 2: A House Divided
  • Victoria: Revolutions
  • Europa Universalis IV: Res Publica
  • Crusader Kings II
  • Heir to the Throne
  • Europa Universalis IV: Wealth of Nations
  • Europa Universalis IV: Art of War
  • Crusader Kings II: Legacy of Rome
  • Crusader Kings II: The Old Gods
  • Crusader Kings II: Rajas of India
  • Crusader Kings II: The Republic
  • Crusader Kings II: Sons of Abraham
  • Crusader Kings II: Sword of Islam
  • Europa Universalis III
  • Europa Universalis IV: Conquest of Paradise
  • Europa Universalis III Complete
  • Europa Universalis IV
  • Age of Wonders: Planetfall Deluxe edition
  • Crusader Kings II: Monks and Mystics
  • Europa Universalis IV: Mandate of Heaven
  • Crusader Kings Complete
  • Europa Universalis IV: Rights of Man
  • BATTLETECH
  • Surviving Mars
  • Age of Wonders III
  • Cities: Skylines - Parklife Pre-Order
  • Cities: Skylines - Parklife
  • BATTLETECH: Flashpoint
  • Imperator: Rome Deluxe Edition
  • Imperator: Rome
  • Age of Wonders: Planetfall
  • Crusader Kings II: Reapers Due
  • Crusader Kings II: Conclave
  • Europa Universalis IV: Cossacks
  • Crusader Kings II: Horse Lords
  • Europa Universalis IV: Common Sense
  • Pillars of Eternity
  • Crusader Kings II: Way of Life
  • Europa Universalis IV: El Dorado
  • 500k Club
  • Warlock: Master of the Arcane
  • Victoria 2
  • Europa Universalis III Complete
  • Europa Universalis III Complete
Thanks for the answer.

Have you considered mixing things up? Having some QA people plus additional(don't know how many people gets to test the beta) close beta testers? I am sure some people would volunteer for that. I am asking because some of the Horse Lords exploits/cheesy tactics were easy to spot on by the average player.

They do exactly that. They actually asked for more beta-testers a few months ago.

Two things to add to aono's comments
1. Some 'obvious' bugs arent that obvious once you realise CK2's scope. Take the pagan reformation button going missing (on reload from memory). At the moment, there are broadly 7 groups with substantial different mechanics in the game (Catholic, Muslim, pagan/tribal, nomad, indian, merchant republic, ERE), 3 main starting dates (+ many more bookmarks), ways of life etc - you can't count on somebody playing x group, for long enough, and doing the necessary steps, and paying attention to x when there are a lot of other things that require attention too. Remember also that you dont see the bugs QA caught before it went live.
2. Its not a case of getting the 'final' product and wringing the bugs out of it for months. Bugs can be introduced in the process of fixing another bug (just check some of the notes from some of the hotfix patches for 'simple' fixes having unintended consequences), or balance or other changes don't work as intended.
 
  • 1
Reactions:

Cetan

Colonel
75 Badges
Feb 26, 2012
1.103
930
  • Heir to the Throne
  • Europa Universalis IV: Mare Nostrum
  • Stellaris: Synthetic Dawn
  • Stellaris Sign-up
  • Europa Universalis IV: Pre-order
  • Cities: Skylines Deluxe Edition
  • Sengoku
  • Rome Gold
  • Europa Universalis IV: Res Publica
  • Europa Universalis IV: Call to arms event
  • Europa Universalis IV: Wealth of Nations
  • Europa Universalis IV: Conquest of Paradise
  • Europa Universalis IV: Art of War
  • Divine Wind
  • Crusader Kings II: Charlemagne
  • Crusader Kings II: Legacy of Rome
  • Crusader Kings II: The Old Gods
  • Crusader Kings II: Rajas of India
  • Crusader Kings II: The Republic
  • Crusader Kings II: Sons of Abraham
  • Crusader Kings II: Sunset Invasion
  • Crusader Kings II: Sword of Islam
  • Europa Universalis III
  • Europa Universalis III: Chronicles
  • Europa Universalis III Complete
  • Crusader Kings II
  • Stellaris - Path to Destruction bundle
  • Cities: Skylines - Mass Transit
  • Crusader Kings II: Monks and Mystics
  • Cities: Skylines - Natural Disasters
  • Stellaris: Leviathans Story Pack
  • Europa Universalis IV: Mandate of Heaven
  • Stellaris: Digital Anniversary Edition
  • Europa Universalis IV: Rights of Man
  • Crusader Kings II: Reapers Due
  • Stellaris: Galaxy Edition
  • Stellaris
  • Cities: Skylines - Snowfall
  • Crusader Kings II: Conclave
  • Europa Universalis IV: Cossacks
  • Cities: Skylines - After Dark
  • Crusader Kings II: Horse Lords
  • Europa Universalis IV: Common Sense
  • Crusader Kings II: Way of Life
  • Europa Universalis IV: El Dorado
  • Stellaris: Nemesis
  • 500k Club
  • Rome: Vae Victis
  • Europa Universalis III Complete
  • Europa Universalis III Complete
Quality of reported bugs is the main reason. Beta testers are great, but they aren't paid to test and most of the time they'll test stuff that is fun. There are exceptions of course and some beta testers amaze us with the work they put in.
Paradox relied almost solely on beta testers up until V2 release (and then only as limited time contractors), EU4 was the first releases (as far as I remember with a in house team of QA). I'd say that quality has gone up since EU3 and HOI3 releases. QA isn't the only reason, but it's definitely one of the major.

Whatever happened to that initiative around the time of WOL where the decision was made to release beta patches prior to expansions for the public to test and bugfix? That idea got canned pretty quickly it looks like, since we didn't see it for Horse Lords.
 

Atlantians

Field Marshal
101 Badges
Nov 25, 2012
2.973
4.481
  • Cities: Skylines - After Dark
  • Stellaris: Lithoids
  • Hearts of Iron IV: La Resistance
  • Stellaris: Federations
  • Imperator: Rome - Magna Graecia
  • Crusader Kings III
  • Europa Universalis 4: Emperor
  • Victoria 3 Sign Up
  • Europa Universalis IV
  • Stellaris: Galaxy Edition
  • Hearts of Iron IV: Expansion Pass
  • Stellaris: Ancient Relics
  • Stellaris: Distant Stars
  • Battle for Bosporus
  • Stellaris: Megacorp
  • Europa Universalis IV: Res Publica
  • Victoria: Revolutions
  • Europa Universalis: Rome
  • Rome Gold
  • Semper Fi
  • Sword of the Stars
  • Victoria 2: A House Divided
  • Victoria 2: Heart of Darkness
  • Rome: Vae Victis
  • Cities: Skylines Deluxe Edition
  • Europa Universalis IV: Pre-order
  • Crusader Kings II
  • Europa Universalis IV: Cossacks
  • Cities: Skylines - Snowfall
  • Europa Universalis IV: Mare Nostrum
  • Stellaris
  • Stellaris: Galaxy Edition
  • Crusader Kings II: Reapers Due
  • Crusader Kings II: Monks and Mystics
  • Stellaris - Path to Destruction bundle
  • Europa Universalis IV: Third Rome
  • Steel Division: Normand 44 - Second Wave
  • Crusader Kings II: Jade Dragon
  • Crusader Kings II: Sons of Abraham
  • Crusader Kings II: Sword of Islam
  • Europa Universalis III
  • Crusader Kings II: Charlemagne
  • Europa Universalis III: Chronicles
  • Europa Universalis III Complete
  • Crusader Kings II: Sunset Invasion
  • Crusader Kings II: The Republic
  • Divine Wind
  • Crusader Kings II: Legacy of Rome
  • Europa Universalis IV: Art of War
  • Crusader Kings II: Rajas of India
They do exactly that. They actually asked for more beta-testers a few months ago.

Two things to add to aono's comments
1. Some 'obvious' bugs arent that obvious once you realise CK2's scope. Take the pagan reformation button going missing (on reload from memory). At the moment, there are broadly 7 groups with substantial different mechanics in the game (Catholic, Muslim, pagan/tribal, nomad, indian, merchant republic, ERE), 3 main starting dates (+ many more bookmarks), ways of life etc - you can't count on somebody playing x group, for long enough, and doing the necessary steps, and paying attention to x when there are a lot of other things that require attention too. Remember also that you dont see the bugs QA caught before it went live.
2. Its not a case of getting the 'final' product and wringing the bugs out of it for months. Bugs can be introduced in the process of fixing another bug (just check some of the notes from some of the hotfix patches for 'simple' fixes having unintended consequences), or balance or other changes don't work as intended.

BZ1Ky0BIYAAYsH_.png
 
  • 10
  • 1
Reactions:

Talq

Field Marshal
51 Badges
Sep 7, 2009
4.795
1.007
  • Crusader Kings II: Charlemagne
  • Europa Universalis IV: Mare Nostrum
  • Crusader Kings II: Holy Knight (pre-order)
  • Cities: Skylines Deluxe Edition
  • Warlock 2: The Exiled
  • Victoria 2: Heart of Darkness
  • Victoria 2: A House Divided
  • Victoria: Revolutions
  • Europa Universalis IV: Res Publica
  • Crusader Kings II
  • Heir to the Throne
  • Europa Universalis IV: Wealth of Nations
  • Europa Universalis IV: Art of War
  • Crusader Kings II: Legacy of Rome
  • Crusader Kings II: The Old Gods
  • Crusader Kings II: Rajas of India
  • Crusader Kings II: The Republic
  • Crusader Kings II: Sons of Abraham
  • Crusader Kings II: Sword of Islam
  • Europa Universalis III
  • Europa Universalis IV: Conquest of Paradise
  • Europa Universalis III Complete
  • Europa Universalis IV
  • Age of Wonders: Planetfall Deluxe edition
  • Crusader Kings II: Monks and Mystics
  • Europa Universalis IV: Mandate of Heaven
  • Crusader Kings Complete
  • Europa Universalis IV: Rights of Man
  • BATTLETECH
  • Surviving Mars
  • Age of Wonders III
  • Cities: Skylines - Parklife Pre-Order
  • Cities: Skylines - Parklife
  • BATTLETECH: Flashpoint
  • Imperator: Rome Deluxe Edition
  • Imperator: Rome
  • Age of Wonders: Planetfall
  • Crusader Kings II: Reapers Due
  • Crusader Kings II: Conclave
  • Europa Universalis IV: Cossacks
  • Crusader Kings II: Horse Lords
  • Europa Universalis IV: Common Sense
  • Pillars of Eternity
  • Crusader Kings II: Way of Life
  • Europa Universalis IV: El Dorado
  • 500k Club
  • Warlock: Master of the Arcane
  • Victoria 2
  • Europa Universalis III Complete
  • Europa Universalis III Complete
Whatever happened to that initiative around the time of WOL where the decision was made to release beta patches prior to expansions for the public to test and bugfix? That idea got canned pretty quickly it looks like, since we didn't see it for Horse Lords.

At the time is was stated that the reason why it was being done was because the DLC content was easily separable from the other changes (which were to the diplomatic UI).

HL wouldn't have been that easy to separate.

Can people just shut up about the Flanders issue,... I agree with your point, but I am getting sick of seeing it posted everywhere. It is about as annoying as the bloody china bandwagon!

I broadly agree and note that repeated off-topic posts tend to get regarded as spam.
 
Last edited:
  • 1
Reactions:

Lord_P

Field Marshal
67 Badges
Apr 17, 2014
3.894
3.355
  • Crusader Kings II
  • Magicka
  • Europa Universalis IV
  • Crusader Kings II: Sword of Islam
  • Crusader Kings II: Sons of Abraham
  • Crusader Kings II: The Republic
  • Crusader Kings II: Rajas of India
  • Crusader Kings II: The Old Gods
  • Hearts of Iron IV: Cadet
  • Hearts of Iron IV: La Resistance
  • Crusader Kings II: Monks and Mystics
  • Hearts of Iron IV: Together for Victory
  • Stellaris: Leviathans Story Pack
  • Stellaris: Digital Anniversary Edition
  • Europa Universalis IV: Rights of Man
  • Crusader Kings II: Reapers Due
  • Stellaris: Synthetic Dawn
  • Stellaris
  • Stellaris: Federations
  • Cities: Skylines - Mass Transit
  • Europa Universalis IV: Mandate of Heaven
  • Hearts of Iron IV: Death or Dishonor
  • Europa Universalis IV: Common Sense
  • Age of Wonders III
  • Stellaris: Distant Stars
  • Cities: Skylines - Parklife
  • Hearts of Iron IV: Expansion Pass
  • Stellaris: Humanoids Species Pack
  • Stellaris: Apocalypse
  • Cities: Skylines - Parklife Pre-Order
  • Warlock: Master of the Arcane
  • Battle for Bosporus
  • Crusader Kings II: Legacy of Rome
  • Stellaris: Lithoids
  • Stellaris: Ancient Relics
  • Prison Architect
  • Hearts of Iron IV: Expansion Pass
  • Imperator: Rome
  • Victoria 2
  • Crusader Kings II: Holy Fury
  • Europa Universalis IV: Dharma
  • Crusader Kings II: Conclave
  • War of the Roses
  • Cities: Skylines
  • Europa Universalis IV: El Dorado
  • Mount & Blade: With Fire and Sword
  • Crusader Kings II: Way of Life
  • Pillars of Eternity
  • Crusader Kings II: Horse Lords
  • Crusader Kings III: Royal Edition
Can people just shut up about the Flanders issue,... I agree with your point, but I am getting sick of seeing it posted everywhere. It is about as annoying as the bloody china bandwagon!
...

I just want them to fix it... :(
 
  • 2
Reactions:

aono

Field Marshal
113 Badges
Aug 26, 2008
4.021
3.432
  • Europa Universalis 4: Emperor
  • Prison Architect
  • Stellaris: Necroids
  • Stellaris: Nemesis
  • Victoria 3 Sign Up
  • Hearts of Iron IV: By Blood Alone
  • Hearts of Iron IV: No Step Back
  • Knights of Pen and Paper 2
  • Stellaris: Federations
  • Prison Architect: Psych Ward
  • Battle for Bosporus
  • Island Bound
  • Cities in Motion
  • Europa Universalis IV
  • Crusader Kings II
  • Crusader Kings II: Charlemagne
  • Crusader Kings II: Legacy of Rome
  • Crusader Kings II: The Old Gods
  • Crusader Kings II: Rajas of India
  • Crusader Kings II: The Republic
  • Crusader Kings II: Sons of Abraham
  • Crusader Kings II: Sunset Invasion
  • Crusader Kings II: Sword of Islam
  • Europa Universalis III
  • Europa Universalis III: Chronicles
  • Divine Wind
  • Europa Universalis IV: Art of War
  • Europa Universalis IV: Conquest of Paradise
  • Europa Universalis IV: Wealth of Nations
  • For the Motherland
  • Hearts of Iron III
  • Hearts of Iron III: Their Finest Hour
  • Hearts of Iron III Collection
  • Heir to the Throne
  • Europa Universalis IV: Res Publica
  • Victoria: Revolutions
  • Stellaris: Megacorp
  • Stellaris: Ancient Relics
  • Semper Fi
  • Victoria 2: A House Divided
  • Victoria 2: Heart of Darkness
  • Rome: Vae Victis
  • Europa Universalis IV: Pre-order
  • Cities: Skylines - After Dark
  • Europa Universalis IV: Cossacks
  • Cities: Skylines - Snowfall
  • Europa Universalis IV: Mare Nostrum
  • Stellaris
  • Stellaris: Galaxy Edition
  • Stellaris: Galaxy Edition
1. Some 'obvious' bugs arent that obvious once you realise CK2's scope. Take the pagan reformation button going missing (on reload from memory). At the moment, there are broadly 7 groups with substantial different mechanics in the game (Catholic, Muslim, pagan/tribal, nomad, indian, merchant republic, ERE), 3 main starting dates (+ many more bookmarks), ways of life etc - you can't count on somebody playing x group, for long enough, and doing the necessary steps, and paying attention to x when there are a lot of other things that require attention too. Remember also that you dont see the bugs QA caught before it went live.
Yup, I know. Once again - I know the drill. :)
I can't get how 4 testers can test scripted product at such scope, that's it. It takes efforts of 6 inner testers, outside publisher team who checked our tracker and closed beta test when we was making simple enough (and it became simpler when we did it to my cry) helicopter simulator, and I definitly know we missed something. Also 4 or 5 testers count normal for simple "Farm Frenzy"-type social network game (but one will sit on monetization all the time and another on synchronization tbh).

I love how people on this forum think they can do Paradox's job better than them.
Not every part.
I'm very, very, very auful coder, so I've never think I can say something about actual developing (I mean code writing). Also I've never been a publisher, so I've never say something about manager or producer work. It's just not my area of expertise. I have an opinion how a game can be done better (some mechanics or something), but I understand clearly that it can take too many effort to put that ideas into game, so I rarely advice something like that.
But I started QA work in 2002, so I believe I earned a right to have an opinion here.
 

Birken

Mastermind
Paradox Staff
4 Badges
Sep 28, 2009
877
3.830
  • March of the Eagles
  • Pirates of Black Cove
  • 500k Club
  • Paradox Order
Thanks for the answer.

Have you considered mixing things up? Having some QA people plus additional(don't know how many people gets to test the beta) close beta testers? I am sure some people would volunteer for that. I am asking because some of the Horse Lords exploits/cheesy tactics were easy to spot on by the average player.

We have a closed beta for CK2 already and some 30-50 testers (I'm not up to date with the exact number). I think they're listed in credits for our expansions. Every now and then we hold open applications for becoming a beta tester. Often we select people who've been active in the bug forums.
 
  • 3
  • 1
Reactions:

Beagá

Banned
74 Badges
May 27, 2007
13.783
4.044
  • Hearts of Iron IV: Colonel
  • 500k Club
  • Crusader Kings II: Holy Knight (pre-order)
  • Europa Universalis IV: El Dorado
  • Europa Universalis IV: Pre-order
  • Europa Universalis IV: Common Sense
  • Europa Universalis IV: Cossacks
  • Europa Universalis IV: Mare Nostrum
  • Stellaris
  • Stellaris: Galaxy Edition
  • Stellaris: Galaxy Edition
  • Stellaris: Galaxy Edition
  • Hearts of Iron IV: Cadet
  • Victoria 2: Heart of Darkness
  • Hearts of Iron IV: Field Marshal
  • Europa Universalis IV: Rights of Man
  • Tyranny: Archon Edition
  • Tyranny: Archon Edition
  • Stellaris: Leviathans Story Pack
  • Stellaris - Path to Destruction bundle
  • Europa Universalis IV: Mandate of Heaven
  • Crusader Kings Complete
  • Europa Universalis IV: Third Rome
  • BATTLETECH
  • Hearts of Iron IV: Death or Dishonor
  • Hearts of Iron IV: No Step Back
  • For The Glory
  • Crusader Kings II
  • Crusader Kings II: Legacy of Rome
  • Crusader Kings II: The Republic
  • Crusader Kings II: Sons of Abraham
  • Darkest Hour
  • Europa Universalis III
  • Europa Universalis III Complete
  • Divine Wind
  • Europa Universalis IV
  • Europa Universalis IV: Art of War
  • Europa Universalis IV: Conquest of Paradise
  • Europa Universalis IV: Wealth of Nations
  • Arsenal of Democracy
  • For the Motherland
  • Hearts of Iron III
  • Hearts of Iron III: Their Finest Hour
  • Heir to the Throne
  • Europa Universalis III Complete
  • Europa Universalis III Complete
  • Europa Universalis IV: Res Publica
  • Victoria: Revolutions
  • Semper Fi
  • Victoria 2
I love how people on this forum think they can do Paradox's job better than them.

About coding? Depends. You know where people work in real life? No.

About gameplay? Depends. You know how many games people played? No.

Many modders did better job overall than Paradox on their respective games. I´ve started playing Pdox games for real... with EU3 AND Magna Mundi mod circa 2008. Many concepts of which were seen in some form later in EU4.

Many people only played Victoria 2 with Naselus´ mod.

I could go on and on. Being dismissive of input is as wrong as listening to it always. The problem is filtering information.
 
  • 7
  • 3
Reactions: