Idea for paradox re beta testing available to all interested
- Thread starter ollobrains
- Start date
-
We have updated our Community Code of Conduct. Please read through the new rules for the forum that are an integral part of Paradox Interactive’s User Agreement.
IMO most games should follow Stardocks example. Let people preorder the game, allow them in beta. It can help to have many hands give suggestions to the game and you can pretty much then release the game whenever it is truely completed resulting in better press at release for the games lack of bugs.
Its really no different than companies rushing a game out of the door except people actually have a hand on changing features before the game is released and to participate in the finding of bugs and creation of a game. It is working well so far with Elemental War of Magic and did also with the Galciv game and its expansions and those games were univerally lauded as awesome and occupied the top levels of PC game sales and web activity for quite some time.
Many companies have been shifting this this method of developing their games, releasing either "multiplayer betas" or some other form of preorder beta. It has always been a positive outcome in every case. Some games like Mount and Blade give their entire existence to this type of development.
Its really no different than companies rushing a game out of the door except people actually have a hand on changing features before the game is released and to participate in the finding of bugs and creation of a game. It is working well so far with Elemental War of Magic and did also with the Galciv game and its expansions and those games were univerally lauded as awesome and occupied the top levels of PC game sales and web activity for quite some time.
Many companies have been shifting this this method of developing their games, releasing either "multiplayer betas" or some other form of preorder beta. It has always been a positive outcome in every case. Some games like Mount and Blade give their entire existence to this type of development.
I pray that Paradox won't be tempted by marketting advice from people who just want to play a game early.
That is a bit of an assumption isn't it??
I personally want PI games to be as good as they can be when released, so I support both a pro tester and beta-tester approach. I wouldn't want to be a beta-tester on V2 (CK2 is another matter though) since I haven't enough knowledge of Vicky to be of much use ... but I know that there are some here who do have that knowledge and can be very valuable in feature testing and game-balancing, where professional testers might not be (as they lack knowledge of gameplay).
Basically if PI don't have the resources to handle a "proper" beta-test phase where each beta-tester is given specifics to test and report on, then maybe let a handful of old-faithfuls get the chance to download the pre-released game and just play around with it, and then report back.
Johan did say that some of them were insanely good ... so why not use the resources you have available??
I dont believe that an open beta test realy works well in combination with a pro beta test. How does one get both tests results together as the results of one probably would have consequences for the other. A streamlined test schedule that delivers good results is needed (paradox probably does not have the time to go through multiple test phases in which the changes of one side can be tested by the other side and vice versa). What they might do, is take in some very knowledgeable community members and let them run some trial games to stamp out some "into the face" errors or problems on the content side (like OH gamer). But thats realy something we might not hear before release.
I have to disagree:
Both sides are focused on different topics:
The pros do test that are closer to Hardware:
- Does the game run on a huge variety of HW
- Are there problems in memory handling
- Is every feature working as it should
In addition to that the pros really try to "break" the software by entering nosense, incorrect commands, try to make the game crash, ...
To sum it up: here we are talking more about the game engine.
Most Players are not interested in testing the game engine so much, your focus is more the fun, you want to play.
Here, the strength is that you find balancing problems and so on.
@ naggy: Sorry, if I paint the picture black and white only, I know the real live, too.
Nope. Almost all of the comments so far have been trying to find ways that will allow the posters to play early, and there are precious few comments from those who have been part of the past beta tests.
I agree that this is the best approach in an ideal world, but it isn't, so they have to make due with limited resources, which means they take the one that will be better for the game. If they're smart they'll request some of those who have been good in the past to play with the game, but they won't make a sign up form for those who want to try playing the game early.
That's why you hire pros. You can't ask volunteers to focus on only one area, especially when they have such a bad track record anyway, it'll result in you getting zero information on a number of areas.
And he should privately approach those individuals, but that's a whole different story from having beta applications.
I do hope Paradox will open (perhaps has already opened) a beta forum, likely by invitation only, for the purposes of balancing, supplementing research and giving learned feedback on game mechanics.
But I'm glad there will be a professional Q/A run.
But I'm glad there will be a professional Q/A run.
Yeah, a professional beta test is a very good idea. But I do hope they will also have some "playing" beta testers (i.e. fans), because of what Kanalratte said.
we may have to content ourselves with beta testing follow up patches once the game is actually out, in the meantime ive pre ordered the other WWII game in development that is close but not quite Hearts of iron 3 sold by gamersgate and will be buying into and beta testing it as it will probably be out before Vic 2
A player beta test generally requires at least one full time person dedicated to tracking them...on top of tracking the other test team.
That seems a little overstated, but yes, it would require some time from someone, but it would surely be worth it!
Dunno how much work it is tracking a professional beta team; they are professionals, after all.
Depends on the size of the team, and the environment. Given the relatively laid-back nature of Paradox, the test lead could probably also test.
On my current project, we have 1 full time test lead for 5 testers, but that's because we're subject to US FDA regulations (among many, many, many otherss...) The meetings...they buuuuuurn!!!!
I think they have some fundamental design flaws. Mainly I have heard they switched to using Lua scripting which is horrible for performance especially with collections, it has horrible performance with simple key-value hash tables. It's one of the reasons pulling up the leaders list is so god awful slow. Its like a few hundred tuples of a 4 attribute entity and u can count out the seconds it takes to bring up, that should be virtually instantaneous. They have map and texture problems on really simple 2d stuff, that line through the pacific is just not neccesary. They need more staff in the development area. Some of the problems come down to testing, some to poor architecture choices, and others to just having such a small development staff that you don't have the specialists you need, like having a GP perform heart surgery.
Testing is just one part of the problem. They also seem to have no analyst's; no liason's working between testing, development, and the community. A good analyst acting as an intermediary is crucial in the development world. The way bug lists look and the release management(or lack thereof) throws out builds looks like its coming straight off the developers desk. Analyst's should be doing all of that work.
Testing is just one part of the problem. They also seem to have no analyst's; no liason's working between testing, development, and the community. A good analyst acting as an intermediary is crucial in the development world. The way bug lists look and the release management(or lack thereof) throws out builds looks like its coming straight off the developers desk. Analyst's should be doing all of that work.
Last edited:
Not sure how you can make assumptions about code you havent seen. Lua isnt used for any of that stuff. also luas table performance is pretty good, its one of the faster scripting languages (wins most benchmarks, but of course slower than c++ though).
+1
It was some early mistakes in the various lua that were responsible for slowiness of HOI3; but they were corrected with 1.1. Something else was making (still make it) HOI3 slow.
As for the speed of lua (or the lack of it); I do concure lua performance in the way it is used is quite acceptable.
Anyway lua is used as external script; most of the problems of HOI3 (and the potentials for Vic2) are from the .exe in my opinion.
Oh dear, hash tables with O(1) lookup. By the way:
(www.lua.org/doc/jucs05.pdf, The Implementation of Lua 5.0, section 4: Tables)
Let's also repeat the classical mantra that 90% of the execution time is spent in 10% of the code.
Testerdamus says...
...your code has some defects...
...people will whine about your code...
...someone will assume they know more about coding than the devs...
...there will be at least one performance issue somewhere in the game...
...no one on the dev team has done a WC as Zulu yet...
It works for Stardock. Their games are polished from the get go (one of the few companies that can say that). Their beta testers: any customer who buys the game gets into the beta. It's that simple. The reason the volunteer beta didnt work for paradox is their playerbase was quite small- only 200 hand chosen people to begin with. Stardock's base is probably 100 times that for their pre-orders... so even if only 10% are active, you still have several thousand testers going. Paradox games probably sell about as well (if not better) than Stardock games, so they have a huge potential pool of testers as well.
Plus if it's in beta the dedicated players will have a chance to see if any key features are missing that paradox could possibly have a chance to add. This is something that pro-testers who are focused on bug finding might not notice, especially if they are not fimilar with other paradox games.
This response time on alot of lookups against collections is horrendously slow. Such small datasets if it's pure C++ there's no way it would be that slow. It would be sub 50 ms across a network. For the size of data thats involved, a local dataset, it's seconds for response and thats just bad performance.The DML is just as bad;making any sort of change to a collection can also be counted off in seconds. As I said I have only heard, I never said I knew, this was due to using Lua (there is a very detailed post about the subject in the HoI3 forums). I have been working with Oracle and MSSQL for about 20 years now(100's of servers as a dba and dev), I have also worked quite a bit with lua on a couple of WoW mods. I know thats nothing to gloat about, I have written planty of bad code in my life, just trying to say that I do understand response time on data lookups. Obviously there is no way any non dev could see the source code. Still doesn't change the fact that there are performance issues in HoI3 with almost any form of dataset.
You have some serious performance issues with the initial game load, when too many units are involved in the game it can bog down to a crawl, collection object (tables,lists,etc) performance issues dealing with pretty darn small datasets, and de-syncs in multiplayer that have been around since you first started making these types of games.
I run an i7-940 with 4gb ram and a gtx 285 jic anyone thinks this is my system or something.
All I am trying to point out in relation to this thread is that the entire problem wasnt that the game was untested, there are some fundamental flaws at work here in relation to the underlying architecture and/or codebase. Maybe it's not Lua but its something. Something that testing alone isnt going to fix.
I loved the original Victoria and have enjoyed EU and the HoI series. I honestly wish you guys the best of luck and from experience know how hard it can be to hear someone tell you there's a problem in code you wrote. Know that its coming from a place of wanting to see you guys fix it , produce a quality game, and make all the money u deserve, as well as a game I can throughly enjoy.
Last edited: