Kerbal Space Program 1.0 Enters Experimental Testing

[LadyAthena]

*clears throat* -Fangirl Scream-

[zanzibar196]

Out of curiosity... now that KSP is "released"... does squad officially have any more plans for other games in the future?

[Esme]

*clears throat* -Fangirl Scream-

I'm a bit too old for that kind of screaming, but 'Yaaaaay!' from me, a round of applause, and 'jolly good show, SQUAD!'. Tea and bikkies all round!

To SQUAD's critics, I can only say that there was a certain well-known company whose products I boycotted many years ago simply because one outstanding feature of their games proved to be that they were clearly never game-tested adequately, with the 'final product' released with game-breaking bugs that were never fixed despite which numerous expansions were sold, and customer complaints were solidly ignored.

SQUAD, however, gives every impression of being a company that really cares about their product and the player experience. As others have pointed out, it's highly improbabl that any software of the complexity of a modern game will have no bugs whatsoever on release; we don't know what bugfixes SQUAD have in store for 1.0, and we do know that they've committed to further updates beyond 1.0. Critics, please do continue with your valuable input of keeping an eye on what's not yet right with the game and letting SQUAD know about it, but please try not to be too negative about the fact that the 1.0 release is likely to be with us sooner rather than later.

I'd also add that personally I haven't seen Mr Kraken at all in v0.90, which is a vast improvement on 0.23. I'm running 32-bit KSP on a second hand machine with 4GB RAM total, a creaky old graphics card, and Linux Mint. Sure I've got the graphics quality turned down, but the game still looks great, and yes, I use mods, between 3-12 of them. depending on what I'm trying out at the time in prepapration for my forthcoming attempt to colonise the Kerbol system.

Anyway - 'Yaaaay, v1.0!' I'm very much looking forward to it!

[megatiger78]

ok..... gregrox, start up the hype thread.

[jedensuscg]

So I understand it's 100 testers pulled from the most familiar players that know the games in and outs but this is where things go wrong, and where you always end up with bugs in the release phases of each build.

Just using a dedicated team of people that know the game like the back of their hand, and know how you "should play" the game, means they don't do all the silly stuff less experienced players do on a regular bases.

Just food for thought. There are a lot of dedicated but far less familiar players that could break the game in ways you never even dreamed up, because they DON'T know everything and do a lot of wacky stuff that veterans know you just don't do,and won't even think to try. This is especially important since this will be actual release you are going to get a flood of new players. Previously this was all the early access players. A new build would come out of experimental's, and get punished by us regular folk...and lots of new bugs would be uncovered. This time around the game won't have that benefit. It will be getting punished by a lot of new players who are gauging the game as a fully release, bought off the shelf game.

Now I am not saying go picking people all willy nilly. But it would have been nice to see some sort of process where players could apply for a shot to test it, and if you have access to the hours played on steam use that to get a good range of players. Me? No i would not be interested, but I feel that more time and a larger test pool needs to be used for that big 1.0 release. Or as someone said before, a few weeks where early access players abuse it. Since this is FULL ON RELEASE, you NEEEEEEEEED to have all your ducks in a row. This is prime time, going gold, all the marbles. No longer will review sites have to add that caveat "the game is still in development so its OK to expect a few bugs".

One could say that we are all testers, but that is no longer true, Next time the rest of use see the new patch, that's it, release. You don't want to be like those large publishers, like that one that starts with an E and ends with a A, that uses actual release as a mass beta test (like that one battlefield game, or that city building one...).

So all I am saying, and it might be that way in this particular test, but don't have 100 of the top, brightest and most knowledgeable being the sole testers. It's a lot harder to patch in good will after actual release then it was in early access.

What would I do if I was in the position? I would stop sale of the game in early access on steam and the website. Have a two week period where only those who own the game can download the 1.0 update, and let them beat it into the ground, while also giving all the mod authors that are not special enough for the elite club to optimize and bug fix their own mods. This will give the best first impression when the flood gates open. IMHO.

Edited April 5, 2015 by jedensuscg

[Melfice]

What would I do if I was in the position? I would stop sale of the game in early access on steam and the website. Have a two week period where only those who own the game can download the 1.0 update, and let them beat it into the ground[sNIP]

OMG GAEM BREOKE YUO GUYZZZ SCKSU!!!!!!

x20.000.

Guaranteed.

[Javster]

Wow. That's a surprise. It seems like Squad is rushing to get KSP out of the door. I hope the rushed attitude won't reflect on the final game.

[basic.syntax]

I won't call the .90 to 1.0 development period especially long, but, relative to the 3 and 4 month pace of previous releases, if 1.0 had been put in our hands by April 1, only then would I have called it 'rushed.' Based on various statements that the 1.0 cycle would be longer than we had experienced, in the previous year and a half of updates. (Previous release dates are listed on the front page of the wiki, link on button bar at top of page.)

[Superfluous J]

ok..... gregrox, start up the hype thread.

I think you missed a memo

[regex]

OMG GAEM BREOKE YUO GUYZZZ SCKSU!!!!!!

x20.000.

Guaranteed.

You seem to have a low opinion of other players. I mean, you're right, you'd probably get 20 people doing something dumb like that to troll Squad but, for the most part, I think the bug reporting from users would be much more useful, especially if the game is released as a 0.99 beta or something and they do a quick turn-around into 1.0. A quick version turn-around for bug fixing would add some urgency to bug reporting, plus you could always obfuscate the bug reporting link in a way that only highly interested parties would find it. There are ways to make this useful for the devs. Hell, take advantage of the top QA people to screen the public bug list. Etc... Edited April 5, 2015 by regex

[Robotengineer]

Hell, take advantage of the top QA people to screen the public bug list.

This is what I wish they would do, have the QA people screen the public bug list, and then see if they can replicate the well reported bugs themselves, then refer the report to the devs. More people looking for more bugs, should mean more bugs reported, and a wider variety of bugs reported.

[Melfice]

I don't know if you were there then, regex.

But I've seen the .... "reports" the average player produces.

The release threads for the experimentals were flooded with "Game crashed. Fix please.", with the occasional decent, useable report.

If you want the QA team to screen the good reports from the bad, you'll have a QA team that doesn't do testing. Just screening.

EDIT: And a hidden bug tracker? That means you're cutting the massive influx in testers down to a bare minimum again. And we have 100 QA testers already on it.

Edited April 5, 2015 by Melfice

[basic.syntax]

@regex, Robotengineer - in a perfect world, you're both right. But Squad has answered; they like the two-level setup they have, with an internal test team checking over the reports of the volunteer "Experimentals" group, who each applied for the job. The bottleneck on release quality is probably not in the testing and numbers of bugs found, but in developer time digging down to root causes of bugs and solving them. They've allowed a longer period of time for this feature-locked "Experimentals" testing cycle, not just for testing, but for fixing

During the Experimental phase we do large amounts of minor bug fixes, some severe bug fixes and a lot of tweaking. These are cycles that require very specific, dedicated and precise feedback from the Testers and that's something that is best done with small Teams.

Prior to version 0.15, the Experimental Team didn’t exist. However, with the increasing size of the KSP player base and the game itself, open testing was no longer a viable option for efficient and spot-on bug reporting and issue tracking. Thus the Experimental Team was created. (link)

We've had terrible experiences with the signal to noise ratio of any broad-facing testing environment. Sorting through reports becomes a mammoth task when the overwhelming majority of them are of extremely low quality. The amount of extra overhead this causes ends up having a worse effect that just staying with our core group of really experienced, super efficient QA Crew. Branch testing for QA started as soon as it was possible, and has gone on since. Add to that the fact that we're effectively scheduling the longest experimentals period in the history of our game, and we can be reasonably certain that most major things will be caught by then.

[TMS]

If you want the QA team to screen the good reports from the bad, you'll have a QA team that doesn't do testing. Just screening.

I dunno, man.

How many times have we had an update that has gone through this much-vaunted QA and experimentals process, only for there to be crippling (or merely painful) bugs identified within 24 hours of release?

Within 72 hours, it becomes obvious - even for the dirty casual observers like myself - which bugs are most prevalent and would have benefited from greater and earlier scrutiny. For the record, I might spend 30-45 minutes maximum a day on the forum.

When people clamor for the idea of open beta, this is what they ultimately want - the ability to give crowd-sourced suggestions/directions with all the hard and detailed work left to other people.

[Melfice]

When people clamor for the idea of open beta, this is what they ultimately want - the ability to give crowd-sourced suggestions/directions with all the hard and detailed work left to other people.

And that didn't work.

And I doubt it's going to work now. Especially since the fanbase has grown so much.

[TMS]

And that didn't work.

Forgive me, but I thought the QA and experimental groups were set up long after the open testing era?

And I doubt it's going to work now. Especially since the fanbase has grown so much.

Give peas a chance.

[Red Iron Crown]

Forgive me, but I thought the QA and experimental groups were set up long after the open testing era?

They were, in response to the general uselessness of the bug reports generated by the fully open testing process.

For an idea of what the trouble is, head over to the support forums and have a look at what percentage of threads are started without reading and following the directions. Now imagine that process for bugs which affect everyone. Or, for that matter, have a look on the bugtracker at the number of duplicate or incomplete bug reports.

The semi-closed testing process used now is far from perfect, but it is far more useful for the developers than the old open process was.

[Wallygator]

TL;DR most of the above.

On-time delivery of reliable software is the result of skilled professionals, unambiguois standards and procedures, fully controlled testing, detailed and agreed requirements, common methods, and a realistic schedule operating under effective change controls.

Best of luck to squad, I do have faith that what will result will be something worthwhile and enjoyable. However, I've yet to see strong indications that the above elements are in place. Also, there are still some very critical areas of improvement regarding communications and community expectations management where improvement can be made

Regardless, it's their very first game. The've done remarkably well. I wish I could do the same.

[JimmyAgent007]

I think part of the reason for such long standing bugs was that it was either difficult to reproduce or locate the cause of. (or the bug was in a placeholder system they didnt want to spend weeks fixing if they were going to scrap it) Another part was that they had more important things to figure out and just wasnt worth the time to fix. Now it seems that they no longer have the excuse of having more important things to do so they are going to town on finding all they can. Also, the reason they havent given us a list of squashed bugs could be because they arnt sure they are all gone yet. I've programmed far simpler software, fixed bugs, then encountered them again later on. So something like KSP, they might think they got the bug, but wont say they did until the testers take a go at it.

[showell373]

I sure hope that EA never try to get their hands on this product and development team....straight to the crapper if they do....

[Starwhip]

Whoa whoa whoa whoa, what?

Ye Gods. o_O

*In Jafar Voice* FAS-TAAAAA!

[LadyAthena]

I'm a bit too old for that kind of screaming, but 'Yaaaaay!' from me, a round of applause, and 'jolly good show, SQUAD!'. Tea and bikkies all round!

To SQUAD's critics, I can only say that there was a certain well-known company whose products I boycotted many years ago simply because one outstanding feature of their games proved to be that they were clearly never game-tested adequately, with the 'final product' released with game-breaking bugs that were never fixed despite which numerous expansions were sold, and customer complaints were solidly ignored.

SQUAD, however, gives every impression of being a company that really cares about their product and the player experience. As others have pointed out, it's highly improbabl that any software of the complexity of a modern game will have no bugs whatsoever on release; we don't know what bugfixes SQUAD have in store for 1.0, and we do know that they've committed to further updates beyond 1.0. Critics, please do continue with your valuable input of keeping an eye on what's not yet right with the game and letting SQUAD know about it, but please try not to be too negative about the fact that the 1.0 release is likely to be with us sooner rather than later.

I'd also add that personally I haven't seen Mr Kraken at all in v0.90, which is a vast improvement on 0.23. I'm running 32-bit KSP on a second hand machine with 4GB RAM total, a creaky old graphics card, and Linux Mint. Sure I've got the graphics quality turned down, but the game still looks great, and yes, I use mods, between 3-12 of them. depending on what I'm trying out at the time in prepapration for my forthcoming attempt to colonise the Kerbol system.

Anyway - 'Yaaaay, v1.0!' I'm very much looking forward to it!

^ This.

I understand people's worries, but some people seem to be forgetting that SQUAD has shown great care in wanting to squash bugs and release proper updates. Sure there's going to be bugs, and with how games are these days, bugs will always exist in some form or another in any game, but like the above mentioned. The kraken has yet shown itself to me at all in the .90 playing the game as I should be, and over all is very well optimized for many varieties of computers both old and new. This is something even AAA companies fail to do on a large average.

I have full faith in 1.0 release. Bugs.. probably, in-fact I'll bet on it. Game breaking bugs? No way, because even .90 doesn't have any... Will it be "release worthy?" Definitely. It has everything a full release game should have. Solid game-play, solid mechanics, few bugs, no game breaking bugs, and on top of that has a very passionate modding community already behind it to only further the experience. If that isn't enough SQUAD has already said they'd continue updating KSP after release.

Nay-say all you want, SQUAD has done it right, and while the "release" may seem "rushed" to many, it really isn't when you consider how "finished" the game was at .90. Which was pretty damn near in all fairness.

[Wampa842]

I still think releasing the game after just one beta version is a bad idea, and I'm willing to bet a fortune and a half on some crippling bugs getting through the QA and Experimental testing, no matter how skilled they are.

[TMS]

They were, in response to the general uselessness of the bug reports generated by the fully open testing process.

For an idea of what the trouble is, head over to the support forums and have a look at what percentage of threads are started without reading and following the directions. Now imagine that process for bugs which affect everyone. Or, for that matter, have a look on the bugtracker at the number of duplicate or incomplete bug reports.

The semi-closed testing process used now is far from perfect, but it is far more useful for the developers than the old open process was.

Again, what was suggested is that the QA / experimentals team took some of its lead from an open testing process - effectively making it parallel and getting best of both worlds.

The argument was then made that it would take too much time, but I argued that it was overstating it as you're only really using the crowd-sourcing to highlight the common issues.

It doesn't matter really. We'll go through the same process, only it'll be post-release and via Reddit.

[Frank_G]

Awesome news! Cant wait to give it a try... but wait... where is my hype train ticket... here... phew! Need to get to the station now!!