What is Quallity Assurance?

[EvilotionCR2]

With Quallity Assurance going on a week a something,I thought,what is Quallity Assurance? And I want to know it because English isn't my first language and I don't know what it means.

[TheCanadianVendingMachine]

Well, it is to assure the product is to it's best quality

[regex]

Programmers throw a bunch of **** together and do some testing of their own but, as any good programmer knows, testing your own code is useless because you can't possibly hit all the use cases, which is where Quality Assurance comes in to handle testing of the most common use cases.

Quality Assurance doesn't, and can't be expected to, handle all use cases. Once the most common ones are tested the product is handed to the consumers who handle testing of their own use cases and report any bugs/wish-list items.

[Stelith61]

Testing it before the update.

[Piv]

Something kerbal engineers don't have...

[Kialar]

I guess, in QA-Phase they do something like the good old 20:80 testing. Test 80% of the most common use cases with 20% of the testing effort. And the most of the technical stuff, like "does it compile", "may the game be started", I guess they do kind of regression tests, fly some predefined rockets (aka Stock-Crafts) and so on.

The 80% of functional testing effort will be done in what they call "Experimental". I think, this is mostly unstructured, wild testing, where clever people try to get the game crashed by doing weird things.

[Monger]

In fact, there is no proper definition for QA. It usually just means that the software tests were outsourced from development into some other organizational unit, so that the developers can concentrate on writing code instead of doing the grunt work of analyzing and reporting bugs.

Actually, in modern software development processes, this concept is retreating. Today, the philosophy is rather: software tests should be automated as far as possible, and due to the technical knowledge you need for that, it should be done by specialized developers who actually are part of the corresponding development team. The only task that remains for QA, for example, is a closed and open Beta test, to ensure that the development goals actually meet the user expectations. So QA in this case is much more focussed on usability than on bug hunting.

Obviously, Squad doesn't have the man power to do this, and writing automatic tests for the unity engine is probably either very difficult or nearly impossible. So squad needs to rely on a big Closed Beta test, i.e. voluntary work to find the bugs.

[jwenting]

Programmers throw a bunch of **** together and do some testing of their own but, as any good programmer knows, testing your own code is useless because you can't possibly hit all the use cases, which is where Quality Assurance comes in to handle testing of the most common use cases.

Quality Assurance doesn't, and can't be expected to, handle all use cases. Once the most common ones are tested the product is handed to the consumers who handle testing of their own use cases and report any bugs/wish-list items.

Utter BS.

Programmers test everything they write, but they test it in isolation out of necessity (no use testing things that aren't impacted by your changes, or shouldn't be).

QA engineers do test all, or should (time restrictions often make it impractical).

Of course there's always things slipping between the cracks, usually unforeseen scenarios, corner cases, out of intended scope usage patterns, or simply forgotten.

The idea that programmers don't care about quality and QA isn't interested in delivering a working solution is idiotic.

[Wanderer]

QA for many years (Games, Commercial Software and Engineering). Pretty much just needed to echo what jwenting said and add that end users are NEVER expected to be test candidates. On games they may do Alpha and Beta testing in the public but if you haven't noticed it's more an exercise in public relations and marketing than actual testing. Sorry but the general public are the crappiest testers you can get. At best its the shotgun approach at worst it actually makes my job so much harder because I have to manage Community expectations and crappy bug reporting on top of normal duties... and QA often gets support roles shoved down their throats to boot.

We get very passionate about wanting to make sure the product is good to the point we get very upset if something slips through simply because of the schedule. I have not been in, or been a Lead on, a team that didn't have the next iteration or build ripped from our hands before we felt it was good to go. Of course that's why production departments need to be strict, as most QA would love to test their product way longer than is feasible financially. Unfortunately QA is usually who bares the brunt of blame and ridicule if something seemed so obviously a bug... and all I can say is if you ended up being able to read our Bugs and track our builds, I would bet 10 Million bucks that the bug was in and was even being passed around for fixes but there is this wonderful little option in the Bug reporting software that gets overused near ship date.... AS DESIGNED.

[Seanoog]

Something kerbal engineers don't have...

Thank you.

You made my day with your observation of Kerbal Engineering.

As for the QA question. I'd love to expand on what our team does specifical.

But I am subject to a NDA.

[regex]

The idea that programmers don't care about quality and QA isn't interested in delivering a working solution is idiotic.

lol, remind me not to post when I'm dozing off. Let's try rephrasing that. It is useless for a programmer to attempt to test all use cases because they will naturally want to use the code in a certain way. Case in point, I literally have no idea how a manager uses the task system I work on because I don't directly know that use case and since we're a small, rapid development house I don't have the time or budget to do lengthy sit-downs and interviews (nor does said manager have the time!). Therefore, aside from ensuring the code works in the manner requested (which requires testing, sometimes a great deal, of one sort or another) it is best to let QA and the end user catch bugs in the edge cases.

This also applies to our commercial software. Someone out there will try to use the software in a completely unexpected manner; it is best to let them handle the testing of that use case (at least until we're aware of it!)

[Deadweasel]

...end users are NEVER expected to be test candidates.

Tell that to Google, Microsoft, Apple, Android etc

Anyway, as has been stated, users are the most reliable source for finding a use case that hadn't been intended to be encountered, but causes big trouble when it is. I'm pretty sure the kraken itself is only known because players were pushing things farther and bigger than the devs ever really expected.

</semi-threadjack>

[Wanderer]

I can certainly vouch for Programmers not knowing where their part of the jigsaw fits with the whole. Not their fault usually, they need to be focused but it often leads to some links in the chain being 'inadvertently broken'. A quick meeting usually sorts it for next build.

Edge Cases are generally my job, something I have always been good at foreseeing. Unfortunately (and sometimes fortunately) there is always that one user out there who, 'thinks a little differently' about how to do things. If it breaks things or is an exploit it gets jumped on and retested extensively, if its a issue specific to only a very small percentage of users (95% of the edge cases you'll never know about) it frankly gets left for some time later in the fix schedule. Gotta have priorities.

[Wanderer]

Tell that to Google, Microsoft, Apple, Android etc

heh, well, thats a side effect of pushing things out the door too fast, not anything specifically planned for the most part.

And you're also dealing with a PR or 'soft launch' with things nowadays. Its been a long road tricking end users into 'being valued helpers of half completed software', oh wait, forget you saw that, um... No, the end user is a highly valued aide in the design of all software, we value any and all feedback we can recieve on this 'alpha process' *ahem* Yep.

[Capt'n Skunky]

Our Quality Assurance team has a set of procedures and unit tests they run the game through to catch the most obvious of errors as well as errors that may have crept in affecting something that isn't part of what the current update was working on. This is less about playing the game and more about working through specific sets of actions to ensure the results meet expectations. The QA Team also has a balanced number of testers for each OS as to not leave anyone out.

Cheers!

Capt'n Skunky

KSP Community Manager

[Wanderer]

Ah the massive spreadsheets known as Test Plans. The multiple repeats of the smoke tests, the pouring over pages and pages of design docs to write those damn things... its a living.

[Aragosnat]

In fact, there is no proper definition for QA. It usually just means that the software tests were outsourced from development into some other organizational unit, so that the developers can concentrate on writing code instead of doing the grunt work of analyzing and reporting bugs.

Actually, in modern software development processes, this concept is retreating. Today, the philosophy is rather: software tests should be automated as far as possible, and due to the technical knowledge you need for that, it should be done by specialized developers who actually are part of the corresponding development team. The only task that remains for QA, for example, is a closed and open Beta test, to ensure that the development goals actually meet the user expectations. So QA in this case is much more focussed on usability than on bug hunting.

Obviously, Squad doesn't have the man power to do this, and writing automatic tests for the unity engine is probably either very difficult or nearly impossible. So squad needs to rely on a big Closed Beta test, i.e. voluntary work to find the bugs.

And here I thought that was why we are here. To find dem Kerbal bugs.

[jwenting]

Tell that to Google, Microsoft, Apple, Android etc

Anyway, as has been stated, users are the most reliable source for finding a use case that hadn't been intended to be encountered, but causes big trouble when it is. I'm pretty sure the kraken itself is only known because players were pushing things farther and bigger than the devs ever really expected.

</semi-threadjack>

those are not sent to end users with the intent of them being testers, they're sent as a preview of what's to come and any problems reported (if properly reported, with reproduction scenarios and enough data to actually be a useful report, which is maybe one in several tens of thousands of end user reports) is a bonus.

Some companies do give prerelease versions to a select and carefully screened group of customers for integration testing with their specific systems, typically companies that are also in IT and have quality control and development teams of their own, know how things are supposed to be done.

While it's now reasonably common for companies to have websites for customers to file trouble tickets, those are not a primary source for QA problems. They're rather a way for customer specific problems to be filed without taking up time from the first level support team (aka the helpdesk).

And then the second line team has to be expanded greatly to weed out the crap, the hundreds of double tickets, baseless rants, feature requests masked as bug reports, usage problems, people not reading the manual, etc. etc. that result and form the vast majority of everything that's entered.

[jwenting]

Ah the massive spreadsheets known as Test Plans. The multiple repeats of the smoke tests, the pouring over pages and pages of design docs to write those damn things... its a living.

yes, the griefers in the team, spending their lives trying to think up devious ways to break the shiny new features us devs create...

[EvilotionCR2]

Our Quality Assurance team has a set of procedures and unit tests they run the game through to catch the most obvious of errors as well as errors that may have crept in affecting something that isn't part of what the current update was working on. This is less about playing the game and more about working through specific sets of actions to ensure the results meet expectations. The QA Team also has a balanced number of testers for each OS as to not leave anyone out.

Cheers!

Capt'n Skunky

KSP Community Manager

Finally a good answer.But I've experienced a bug once,in 0.18...Keep up the good work!

[nine_iron]

I am a Quality Assurance Technologist and I have no idea what QA is...