Paradox, include CB testing into your QA

[wickermoon]

Urgh. I do (business) software development for a living, and I understand the exact problems PDX faces here. Honestly, I'm as frustrated as the next fan by the fact that apparently every Ascension path (haven't tried Psionics yet) are apparently broken, as are important event chains like the Worm (no Omega Theory), so I've had to resort to the console to just make my game *work* and prevent game-ending brokenness.

But the fact is that the only way to find the kind of bugs that have slipped into 1.5 is to do full-scale integration testing - in terms of a game that means playing full games to completion, and in terms of Stellaris that means 20+ hours on each test. This is prohibitive, and you'll need to redo tests regularly as other (seemingly unrelated) code changes can suddenly break these features again. In my smaller scale business projects I can reasonably automate 95% of testing with proper unittests and integration tests. Not so for a game - and particularly not a game like Stellaris.

Don't mix terms. What you're talking about is called regression tests and continuous integration. Integration testing is only a very small part of testing. And unit testing itself is only a tiny part of both those features. I also have to disagree with your statement that you can't automate most of Stellaris' testing. Anything that is not UI can be unit tested, period. And even UI can be unit tested if you have the proper frameworks for it (although the tests' outcome might not be as helpful in this case, depending). You don't have to play the game to test functionality. Beta players are there to catch bugs not yet caught by tests, and test user experience and joy-of-use/UI design.

I actually do doubt whether Paradox has proper regression testing and continuous integration facilities. In fact, many small-to-middle IT companies still lack proper facilities for that and I have only seldomly heard of CI/RT in gaming.


That being said, I think it would help Paradox and the community a lot, if they would start adding a public interface for their bug-tracking system, where they can filter what the users are allowed to see and what not (as Paradox employees have explained, there are bugs that are not to be disclosed to the public).

 

[Alblaka]

As for livestock, there is another issue. Livestock can not work tiles at all...even food tiles.

You know... it says right in the livestock description that they aren't working the tiles, the population is being culled for food instead... I wonder if that's working as intended.

Wait what? In the dev stream (one of the earlier ones) they clearly showed that a livestock pop would earn a static bonus to food, plus whatever the tile produced (in food, everything else was 0% of course).
Because, yes, if that doesn't work, then the livestock option is literally pointless from anything but RP perspective, EVEN with the Genetic Ascension perks assumed.
I sincerely hope this just turns out to be a bug or oversight in the 1.5 release.

I actually do doubt whether Paradox has proper regression testing and continuous integration facilities. In fact, many small-to-middle IT companies still lack proper facilities for that and I have only seldomly heard of CI/RT in gaming.

I would argue that this even proceeds to apply to 'some' large IT companies as well.
Albeit, of course, everyone has 100% test coverage and TDD. Always.

 

[BjornB]

I have to say, reading your posts is always such a pleasure. Your are the most polite and articulate community representative I have ever seen. You bleed off the very idea of staying angry with such deft precision that I even calm down even if I'm not upset to begin with.

Now you made me blush. Thanks though

This thread is starting to branch up and stray from the OP, which I feel is kind of a completed topic by now (the topic was to petition us to have (more) external testing). So if you wish to continue discussing something brought up in this thread, please start a new thread with a descriptive title clarifying the topic.

I will not lock this thread just yet if you want to round something off. But the length of the thread increase the risk of people joining that didn't read the whole thing, and repetition begins.

So please round this off, the thread might be locked soon.

 

[Alblaka]

But the length of the thread increase the risk of people joining that didn't read the whole thing, and repetition begins.

So please round this off, the thread might be locked soon.

Yeah, I considered editing the OP to reflect the change in topic direction, but given that would still leave the old thread title, I felt it would be a weird thing to do.
I'm completely fine with the thread being locked, since we more or less have reached the consensus that the issue is likely not the testing, but the actual fixing side of things, with decisions being made based upon values we on the other end of the product cannot properly perceive or rate.

And, indeed, as to what can be done in that regard, is a topic for another day/thread.

 

[Chikanuk]

Hi guys!

Hi.
If you make spaghetti code - its no excuse. Not at all.
Right now we all just buying beta. And you know - most of us pay for real game, not for beta test?
If you geek division cant make their job - fire them. We all pay our money, we support you, we did our part of the deal. And you not, you fail us.

Keep selling beta tests, and you will start to lose more and more players.

 

[BjornB]

With OP's blessing I'll go ahead and lock this then. Feel free to create new threads if there are topics from here that you want to continue discussing.