About recent community criticism in the direction of the QA & exp testers

[rentaspoon]

I want to thank the devs and the people that volunteered there time to making this game, I haven't been this hyped for a game for about 10 years.

As for 1.0 I haven't touched career until now as I usually get stuck due to being unable to get enough science personally I would like a way to swap science out for R&D for spending cash on parts.

As for crashes I've had one. So thank you Squad and thank you volunteers for all the hard work and dedication.

[DuoDex]

No. That's simply not the case. Even if there's some idiots attacking testers, there's no reason to assume all of them are. By that logic, earlier in this thread one of the QA guys said this release he was lazy - should i imprint that on all of you guys as well?

Hey!

I said I was not the most active tester! Then I said that everyone else worked at least ten times harder than I did, or more.

[White Owl]

I'm in a kind of unusual position, having read-only access to the experimentals, but I'm not a tester. I can see what goes on and install the various experimental builds because I need to be able to talk about the changes in a preview video. But outside that video, the NDA says I'm not supposed to talk about experimentals... so no details, and I'll hope this doesn't get me into trouble.

The developers and all the volunteer testers clearly want the game to be perfect at every release. Many of the issues people are upset about, that cause players to ask "wasn't this tested!? How could that slip through!?" absolutely were tested, over and over again. Some of them were tested, reported, fixed, tested again, resolved, broken again, re-reported, fixed a second time, etc. The problems "slip through" into release because the whole cycle can't repeat indefinitely; there's a point fast approaching where the thing absolutely must be published. Time is a finite resource, and the whole team of developers and testers simply ran out of time to find, report, fix, and test all the issues. It wasn't deliberate. It wasn't sloppy. It was, unfortunately, very very fast.

The experimental build I used to record my preview video was from only a few days before release. In that version the small pod reentered in the correct orientation, but reentry heat overall was much more mild. (I regret telling people you generally didn't have to worry about reentry heat in a normal return from LKO. Whoops.) So that particular bug - which seems most pointed out by people angry at experimental testers - was introduced very late in the whole process, with release imminent. I don't know exactly when, and if I did know I couldn't tell you... and in the long run it doesn't matter exactly when. All that matters is it was too late for the whole bugfixing process to catch it. The experimental testers hate it when something like that gets through. It's upsetting!

Remember, all these volunteers are there because they love this game and want it to be better. They're good people, taking up the difficult (and ultimately impossible) task of trying to pull the whole development process closer to perfection. Please, these people don't deserve your anger. Give 'em a break.

[tater]

I'd hardly call the very first spacecraft any new player is going to build (a mk1 capsule with a heat shield and chute) an "edge case."

Also, remember that most all people posting here are huge fans. We want KSP to be as successful as possible.

[uglyduckling81]

Probably should of just called this release 0.95 and given people a month to test it. Fixed the problems thousands of users found and then released 1.0.

Releasing the biggest update ever done with whole new systems (aero, heat) and then naming it 1.0 and removing it from early access was always going to invoke an extra passionate response.

It was a poor decision which could of been easily avoided.

[r4pt0r]

Probably should of just called this release 0.95 and given people a month to test it. Fixed the problems thousands of users found and then released 1.0.

Releasing the biggest update ever done with whole new systems (aero, heat) and then naming it 1.0 and removing it from early access was always going to invoke an extra passionate response.

It was a poor decision which could of been easily avoided.

Quite a few people foresaw this coming when they decided to make the arbitrary jump to 1.0 without a proper beta cycle. Calling 1 update beta does not a cycle make. User play-testing will always be superior to any QA/experimentals team, just by the sheer volume. The QA/exp teams did do their best (aside from DuoDex, who openly admits this is entirely all his fault ).

At this point the 'is it ready for 1.0' question has been both answered and rendered useless, as A. I think its clear it was too early, and B. they did it anyway. The point of beta's are to fix what you've added, not add dozens of complete new systems to the game. therein lies the problem. A goofy heating system is fine for a .91, but the community holds higher standards to a 1.0 release.

Regardless, 1.0.1 should fix the majority of major issues, and the community criticism should not be leveled at the QA team, but rather the decision makers who thought having so many brand new features and not giving them their own beta (KSP: .95 "Coming in HOT") was a good idea.

I look forward to getting elbow deep into 1.0.1 in due time.

Edited April 30, 2015 by r4pt0r

[Temeter]

User play-testing will always be superior to any QA/experimentals team, just by the sheer volume.

Volume does not equal quality. How many posts did this forum had which equated simple things like 'i prefer it differently' and 'needs to be fixed', often without actual having a full understanding of the actual systems?

[Lord Aurelius]

I've said my piece already earlier in this thread and I won't repeat it here, but from reading comments on this and other threads I've come to the conclusion that most of the testers were as concerned about the whole rush to 1.0 as the rest of us. Tiberion said straight up in one of his posts that the devs had a reason for pushing this rushed schedule that hasn't been shared (and might not ever be), but I think this a fine update given the circumstances, even if it's not something I would feel comfortable calling 1.0. I've decided to think of this as an amazing 0.27.

I wish as much as everyone else that Squad had taken their time to really do 1.0 right, or at least better communicated with the community that they needed to do a release (even if they didn't go into details) and worked with the community to help prioritize features for 1.0 and done a quick release candidate beta before release. At least Squad is hard at work currently getting a 1.0.1 hotfix out ASAP, and by looking at maxmap's tweet they won't celebrate until they're done and then they'll get working on 1.1.

I'm still looking forwards to the future of KSP, Squad appears to be committed to supporting it (even if the whole 1.0 thing was a bit premature) and I'm confident we'll still get the awesome game we've been promised all along, and in a lot of ways, it's already here (at least in sandbox once you install stock bugfix modules and a few other mods).

Edited April 30, 2015 by Lord Aurelius

[r4pt0r]

Volume does not equal quality. How many posts did this forum had which equated simple things like 'i prefer it differently' and 'needs to be fixed', often without actual having a full understanding of the actual systems?

Whats that about a monkey given enough time will eventually type the works of Shakespeare? I said nothing of quality, but you give us enough time and we can find every way to break this game. Thats what I had meant by 'volume'.

They recently hit 1 million sales on steam alone, and if only .01 percent of them contribute proper bug reports, that would be 100 good bug reports, exactly what Kasper says they want in the OP. I'm not talking about those who make an angry thread in the general discussion forum, that I know doesn't really help Squad.

EDIT: and to the below post, I also am not implying every player should take part in the experimentals process, but after a release the playerbase can find any remaining bugs.

We all just want the best game possible

EDIT: on second thought, I'm not sure how I thought the "Infinite Monkey theorem" helped my argument here, but Im keeping it there because thinking about monkeys using typewriters makes me laugh.

Edited April 30, 2015 by r4pt0r

[Robotengineer]

Volume does not equal quality. How many posts did this forum had which equated simple things like 'i prefer it differently' and 'needs to be fixed', often without actual having a full understanding of the actual systems?

I don't think that anyone is advocating for the entire experimentals process to be come something every player can take part in. What we have been advocating for is to replace the current two step QA-to-Experimentals-to-Release way of debugging and balancing we have currently, to something like this, QA-to-Experimentals-to-PublicBeta-to-Release. The wisdom of crowds may also apply.

[Temeter]

Not sure about that theorem. We had a few trillion monkey until now, and none of them managed to fly to space. Well, some did, but mostly as test animals, and few did actually come back. So that didn't work out very well.

If I remember correctly, KSP actually did try a more open testing at some point. Result was the devs/exp team needing to deal with so many bad and inconclusive reports that they didn't really get anywhere. Just took too much time to get to the good ones.

[Superfluous J]

I am of 2 minds. First, while I've been very critical of the result, I hope I've never implied that any person or group is to blame. If I did I wholly, sincerely apologize.

Second, the result is what it is. Bugs that would be fine in Alpha and Beta are in a full release due to several decisions that I don't agree with. Whatever caused them, they exist and they should have been found. Flying a surfboard out of a cargo bay is an edge case. Removing a vessel with an engine from a cargo bay is not. If it wasn't found then not enough testing was done. If it was found but not reported then the testers were not instructed well enough. If it was found, reported, and not fixed then the developers either misprioritized or ran out of time. If they ran out of time, then the schedule was too strict.

I don't know what happened or why. And as a user I don't care. I can avoid cargo bays and interstage fairings for now and know how to use Module Manager to fix heat shields. I'm not worried about me. I'm worried at the (based on a graph of popularity of the game on Steam) ~5x the number of people than average who are playing the game for the first time or for the first time in a while coming back from Mun in a pod that looks perfectly fine (and is), only to have it inexplicably flip over and explode like no pod ever would.

That player won't come here and complain. They won't know about the QA process and the time crunch and whatnot. They'll know "the game sucks" and bemoan buying it. And they'll tell their friends not to buy it either.

That's why I sound harsh. Because the reality of the situation IS harsh.

(but still I'm sorry to any volunteer who felt I was attacking them. I had the opportunity to sign up and I passed it by because I feared I wouldn't have the time)

[Yellowburn10]

I find it funny, that near the beginning of 1.0 development, people here said that KSP would get hammered by reviewers if it wasn't good. Only now do I realize that that very community probably judges this game harder than the actual reviewers.

[Lord Aurelius]

@5thHorseman

That's a great summary of how I feel as well. I know the game will be fixed and I'll enjoy many more hours with it, but Squad has done a tremendous amount of marketing, merchandising and hype building around this release to bring in new players (and it sounds like it's been working), but the game isn't really in a state for a bunch of new players.

The bugs aside, this game has a very steep learning curve compared to most games, and to get anywhere beyond LKO, even in sandbox, requires a tremendous amount of learning effort that most gamers simply won't want to put in. The career mode is even worse, right now it's basically a challenge mode for experienced players who want to try to pull off missions with limited funds and parts. In other words, this game is not easily accessible to new players. They'll buy it, maybe do the tutorials, likely start a career over sandbox (it's the default option and the recommended starting point for most games) and maybe get to LKO, and then likely run into the difficulty wall and bugs and give up, i.e. basically what pewpewdie did in his sponsored youtube video.

A hotfix will likely fix the worst of the bugs and we can continue to the enjoy this game like we've been doing, but it needs a lot more than bugfixes to prevent a large number of new players drawn in by the marketing and hype from quickly giving up on the game when they're completely lost after a few hours.

Edit:

I find it funny, that near the beginning of 1.0 development, people here said that KSP would get hammered by reviewers if it wasn't good. Only now do I realize that that very community probably judges this game harder than the actual reviewers.

That's only because we love this game so much and want to see it given the treatment it deserves. If we didn't really care about the game, we probably wouldn't really care what Squad did with it.

Edited April 30, 2015 by Lord Aurelius

[neamerjell]

I for one applaud the efforts of the testers and developers! 1.0 is amazing and surprisingly stable! With all the new stuff they added or just flat out replaced, I'm surprised there are as few bugs, if any as there are. I haven't played much yet and I don't know what is a bug and what is simply different because of the new aero and heat models.

Even though 64 bit was disappointingly dropped for 1.0, I can tell that the devs spent plenty of time under the hood of the 32 bit version with optimizations (how else would they have been able to add the fancy graphical effects without a severe impact on performance or memory usage?)

People will always lash out against change. That is human nature. But enlightened individuals will realize this about themselves and curb their negativity in favor of encouragement and, if needed, *constructive* criticism.

[KevinTMC]

...a very important part of testing is confirming that an issue that was published in a new build was fixed and that no new issues are caused as a result. This is a criticial step and also one where a lot of "whoops, how did that happen" comes from.

I can't and won't explain individual issues (not my job) but its very likely that "unintended consequence" is the reason for most of them. Hundreds of issues, dozens of builds, lots to retest - you do the math

That's a helpful perspective, Tiberion. I honestly hadn't thought of that when I found myself asking "how could that bug possibly have slipped by?" I appreciate all the insight you've provided in this thread, and I think it's good for you and KasperVld and Ted to redirect frustration away from the hard-working testers.

And of course KSP is a tremendous achievement overall, and I for one have gotten orders of magnitude more value for my money than with almost any other software purchase I've ever made. And 1.0 is a big step forward for the game.

But that doesn't mean there's not grounds for criticism in the wake of 1.0...even pointed criticism.

A couple of the bugs are puzzling and vexing, but in my view, it's the overall level of polish (everything from lack of graphical polish to inadequate information presented to the player to the continued absence of the first whole tier of KSC buildings) that is the real disappointment, as it is not only a bit below what the 1.0 version number implies, but also below what Squad themselves had been describing as their objectives and expectations for the release.

This followed on the heels of 0.90--also a big step forward for the game--objectively falling short of Squad's previously stated parameters for "beta" status (most obviously in that not all major game systems were yet in place).

Now it's too late to do anything about the 0.25-->0.90-->1.0 development strategy, and more "told ya so"s will do nobody any good. But it's still appropriate to consider a broader lesson here about overpromising and underdelivering, which can provoke unhappiness with even the highest-quality product.

The other major area of concern I have is the state of career mode, which I think is simply not well-enough conceived and executed right now to provide the fun it should, or to get new players hooked and eagerly tackling the learning curve. (I find it's far behind the BTSM mod in fundamental gameplay design.) But that's not so much a bugfixing or polishing or even balancing issue, as much as it's a game design or even philosophical question.

Edited April 30, 2015 by KevinTMC

[Mr Shifty]

I find it funny, that near the beginning of 1.0 development, people here said that KSP would get hammered by reviewers if it wasn't good. Only now do I realize that that very community probably judges this game harder than the actual reviewers.

What actual reviewers? One drawback of a long, open development cycle is that it's difficult to get review press interested in a particular release even if a production house slaps a "1.0" on it or (more importantly) announces it's leaving Early Access. Metacritic has KSP's release date as June 2011 and lists exactly zero reviews of KSP (though they may be still coming.) KSP depends almost entirely on word-of-mouth (though the modern terminology should be finger-of-keyboard) and not traditional gaming press for its marketing.

[MalfunctionM1Ke]

I also felt a little choked up and it was hard to keep my voice down and reasonable when those discussions about pre-release streamers or bugs in the "RELEASE-Version" came up.

KSP is more than a AAA-Title. It not only shows us what Squad can do, but what a great Game you can shape if you allow the community to devote their time to it (streaming, testing, suggesting).

Every little constructive word helps and the devs are willing to hear them.

TL;DR

Whatever your Issue with KSP is (Bugs, Feedback, Suggestions) please write them down in a constructive manner.

Either here in the Forum or in the KSP Bugtracker.

Ranting, speculating and insulting has never helped anyone.

My 5 cents

[Ace_Warbringer]

I do have to say this is one of the most stable, non crashing game releases I have ever enjoyed.

That said there is a lot of what the heck stuff. I do think squad jumped the sheep here in getting things out so quick. Calling this release 1.0 was just asking for trouble IMHO.

Thanks Squad for giving us a great game.

But next time don't use space tape please? it makes the boosters look ugly.

[FlowerChild]

Now this isn’t an effort to trivialise issues that make it into a release build or trivialise the frustration that a player can feel when bugs happen.

I think a big part of the frustration on these forums is because this isn't "a" release as you put it, it's "the" release, and I think Squad trivializing that like it's no big deal (while flying to Mexico for a big party in contradiction to the concept of it being a regular release) adds to that frustration.

1.0 is an excellent release if it was just "a" release. I'd say it's the best yet. For "the" release it's quite buggy and *very* unpolished. It still very much feels like an early access title with a ton of buggy and underdeveloped features. Maybe it feels even more unfinished than 0.9 given the amount of new content that was packed in up until the very last moment, and given how it's that new content that has the most problems.

I'm in no way blaming the testers for that, and I don't think many people actually are, despite the assertions of the first couple of posts in this thread. However, something is definitely off here about how things went down, and I think the frustration people are expressing about it is largely warranted.

[SkyRender]

I would like it known that I think the testing and experimental teams did a fantastic job. Something we often forget as end users is that we don't actually see the really nasty bugs that pop up. And having done testing before myself, I know why certain bugs that seem "obvious" made it into the final build: last-minute changes to fix major issues often cause minor problems in other areas. It's a lot like plugging a leaking dam, in that when you patch up the big holes, a bunch of little ones tend to pop up at the same time. You can't realistically re-test every single aspect of the game after each and every fix the developers make; you'd literally never leave the testing phase if you did that! I suspect that the testers were (when presented with a new build that fixed an issue) instructed to focus mainly on the issues that were fixed to make sure they didn't resurface (a smart thing to do, since often problems run deeper than they seem to!), then focus on the most common aspects of play, and if they had time, test the more complex aspects.

So yeah, I suggest you not be too hard on the testing team. They had a hard job to do.

[Vanamonde]

Criticism towards the Q&A team is pretty valid, there are some glaring issues which one can not not see...

The fact that a bug was seen does not mean there was time to find a solution for it. That doesn't prove the testers are at fault for failing to see the bug.

There's also the point that you can't tell anyone that the people you have on the team are any better at finding bugs...

Finding bugs is the easy part. Testers are chosen for their patience, willingness, and ability to stop playing long enough to properly and fully report the bugs they find, which is essential information for getting the bugs fixed. The vast majority of players are unwilling or unable to do that.

... earlier in this thread one of the QA guys said this release he was lazy...

That was a forum moderator saying that. We moderators are invited to help with QA, but not part of the team, and participate to the degree we feel comfortable and have time to do so. The statement does not represent the attitude of the actual testers.

Tiberion said straight up in one of his posts that the devs had a reason for pushing this rushed schedule that hasn't been shared...

Tiberion was stating an opinion. He is not in a position to know the reasoning behind Squad's timetables, so do not cite this as evidence of some hidden motive on the part of Squad.

No one is saying 1.0 is without bugs, and please, let's not go through the argument about whether this release should have been called 1.0 again. Kasper's point in this thread is merely to ask people to respect the unpaid work the testers have done for the sake of making the game better, and to please not speak of them insultingly.

[Kaezan]

In my opinion...

-Squad could have made a 0.91 Beta. I've been arround since 0.23 and instead of the usual 6 months gap, this time we had a 4 month gap between the Beta and the release. I read the article about why Squad decided to do this and i understand it, but still, i would prefer to do a longer gap between Beta and release.

-The comunity is overcriticizing. In some cases i even sense some stupidity. Squad is programing day and night for the past 3-4 years to bring you this game and you are complaining cause you saw a little bug? Really? That atitude is of someone who never programed not even the "Hello World" of any language.

-The progress i have witnessed since .23 is huge. I saw the video of Unite 2013 and i spent a whole hour with my mouth open. It is just beautiful. Please watch it.

What Squad acoplished is huge. Still, you complain about minor bugs. What have you done to help? What was the last bug report you submited? If you dont want to help, then please, dont come complaining.

-KSP is not supposed to be like real life. Its a game and you should be having fun. The Kraken, which so many talk about, still lives not because of those who got frustrated with it, but the ones who made fun of it. And if you're not having fun you are not playing it right.

-Last but absolutely not least, The aero system, the minning, the fairings, the tech tree, and even Miss Valentina, these are great features. And because of so many of them in this little time, congratulations. And thank you.

[WanderingKid]

I read the article about why Squad decided to do this and i understand it, but still, i would prefer to do a longer gap between Beta and release.

Article? Link please? My google-fu has failed me.

[regex]

I want to thank the testers, the devs, and the modders brought on board for delivering a pretty damn good product for such a strangely rushed release. Certainly things were missed but that happens all the time, and I can especially see it happening with a sandbox game as complex as KSP.

This update feels like one for the long-term customers, the people who have been here a long time tinkering with this game. I'm really happy to have it in my hands as it brings some much-needed corrections and systems to the game, making it feel very much fresh again. There's still a lot of work to be done and I'm not entirely certain I'd call this one 1.0 (maybe the next, after a bit of polish), but good lord, is KSP fun again.