CK3 Dev Diary #35 - User Testing Before Release

[Olden Weiss]

If you read the entire dev diary, you'll see that they perform many tests, and one of them at least involved total newbies.

This may well have been what led to embedded tooltips and the tutorial that so many first-time players are raving about in the professional review scene. We never really think about how important thankless work like that done by Hanna's team is until we stop to think about it. As
ashbery76 said, the portraits alone may have caused a war if they weren't caught!

Glad we got this dev diary. It's important we appreciate the fact that not only are the devs asking players what they like and dislike about the game, but that there are professionals who are able to turn that feedback into actionable data.

 

[arlekiness]

Can we have more "was/now" screenshots?

 

[pguyton]

is the character creator in the game?

 

[Nikolai]

How do you gather the right people for remote user testing? I bet that can't be easy! I mean, you can't use only forum members I guess, even if they would be the easier path. We're kind of the most passionate and dedicated fans on guess, but you'd need some more diverse base, right? And if you ask for signups, you'd again get the ones following your twitter, fb etc accounts I guess? Then comes reliability, both for nda and actually showing up, but also having a good system to record reactions. Do you use people with webcams on? And do you use people several times to record changes over time with the same people for reference?

Sorry for the many questions, this is intriguing. I have much experience as an alpha and beta tester for many games, both yours and other companies over well over a decade now. But this is an unknown area that intrigues me.

 

[Olden Weiss]

Can we have more "was/now" screenshots?

is the character creator in the game?

"Here's what ruler designer looked like when development started..."

 

[Fallender]

For this test we had 12 participants, 6 of them with 200-800h of CK2 playtime and 6 of them “newbies”, try out the tutorial and then play the game for 2 hours using what they’d learned

200-800? So you mean only newbies?

 

[Nikolai]

200-800? So you mean only newbies?

I wonder how they determine this. Offline play doesn't count to steam registered time afaik. And people might have the game running for ages idle. Or they might be testers of the previous game. Game development often use different depots as time goes by.

 

[Dsingis]

which means that anything you write on Steam or other pages, I’ll go over

Oh, you poor poor soul. I don't envy your work on this part.

 

[Keizer Harm]

The million dollar question: did any of the user testers for Imperator complain about mana? Of course the sample sizes are limited but I feel like discovering a fundamentally annoying game concept ahead of time is very much in the scope of user testing.

 

[Cyhort]

I'd like to add my horror at those original portraits.

 

[wilcoxchar]

Thank you for the explanation of user research! Making the game accessible to a variety of players is very important and often far too underappreciated part of game design. The kind of playtesting required to make sure elements of the game aren't confusing or frustrating to learn how to use requires a lot more time and effort than some people might think and I'm glad Paradox is taking the time and effort to make sure new and experienced players alike can get into CK3.

 

[grandad1982]

Probably because a great many people have mobile devices and use them regularly. I don't expect PDS will be going there anytime soon, though. Their games fit a popular niche, and they do so because of depth that would be incredibly difficult to capture with mobile hardware.

Say hello to Stellaris: Galaxy Command. Admittedly not pds but pdx.

 

[Rockphed]

Ironing out bits and bobbles, you will know soon enough

We should make a "Fete Rod" picture for when we like what you do. I look forward to watching you play CKIII with the same gusto you used when narrating the Imperator dev clash.

Those old portraits look like they'd be right at home in some tacky, outsourced free-to-play mobile social spin-off. Crusader Friends™ perhaps?
I hope that's not giving any ideas

Now I want to figure out what Crusader Kings would look like as a mobile game. It would end up being some sort of pyramid scheme where you use your resources to make new dynasty members, marry them into other people's courts, and then use them to plot to overthrow the person who got you in to the game.

is the character creator in the game?

In the thread last week it was mentioned that there is a console command to open the character editor. You use the sliders to change appearance and then can copy out the DNA string into a mod file.

 

[Olden Weiss]

Say hello to Stellaris: Galaxy Command. Admittedly not pds but pdx.

I stand partially corrected.

 

[Vanhal]

I will probably say what everyone here said already, but i am so incredibly glad you didn't go the awful, awful way of Civilization 6, where the leader portraits are so incredibly unimmersive and horrible that the only diplomacy possible with most of them is "your head would look better on the top of a pole".

 

[Olden Weiss]

I will probably say what everyone here said already, but i am so incredibly glad you didn't go the awful, awful way of Civilization 6, where the leader portraits are so incredibly unimmersive and horrible that the only diplomacy possible with most of them is "your head would look better on the top of a pole".

I actually rather like Civ 6's portraits. They're charming, in their way. However I agree that the look objectively would not have worked for CK3. Civ 6 is, at the end of the day, a game. Its entire format reminds you of that regularly, and thus it can afford to have a very deliberate stylistic look. CK3, on the other hand, is as much roleplaying as it is a game and is heavily grounded in history, so such a look doesn't do its setting justice.

 

[Olden Weiss]

Let us be fair for a moment, gentlemen.

As bad as the original portraits looked, let's remember that this was concept art and refinements certainly would have been made. I absolutely agree that we dodged the arrow by way of the change in direction, but let's not be too hard on the art team. Concepts rarely look as gritty as their final form, even when the art direction isn't changed. As an outdated example...

 

[wilcoxchar]

For all the complaints about the portraits, let's also remember what CK1's portraits looked like. If you're thinking the early CK3 portraits are cartoonish or something out of a mobile game, you haven't seen anything.

 

[crowdemon]

Informative dairy, thanks for sharing one part of development.

 

[PrDelahaye]

I just noticed the Lieges hair is clipping quite badly through his hat

Jotaro hat