Developer Insights #19 - Try, Fail, Try Again...and Again

[Kerbart]

1 hour ago, GGG-GoodGuyGreg said:

but what everyone is mad about is several game breaking bugs that are obvious and serious but still don't get attention, and if they do there is no deadline communicated, it could be the next patch, or in one of the future ones, who knows?

Probably a difference in perspective. If your job s to ship a working game, anything that makes the game stop working is "game breaking." Freezing, CTD's, corrupted saves, and so on.

"So, if you are over the top and just trolling the boards… we probably aren’t reading that. "

As the term "game breaking" is used for anything including jagged AA, there's a good chance that, in a "the boy who cried wolf" kind of way, "game breaking" bugs tend to be ignored when they're not, well... you know... game breaking. I've never had any CTD, that doesn't mean they're not there, but the claim that the game is riddled with them might lead to taking reports not quite seriously.

[JIKL04]

17 minutes ago, linuxgurugamer said:

Frankly, I'm a bit disheartened right now.  I spent a LOT of time (which I don't have a lot of) to write the KSP Bug Packager (with the initial push coming from @ShadowZone, thanks).  

Then @Nate Simpson says he loves it as his favorite mod.

Then you come along and totally ignore it, and start asking for yet more info.  Tell me, did you even thing to ask me to add in those fields into the bug packager to help provide you with what you are looking for?

Sometimes I think I'm being totally ignored.  It really doesn't give me any motivation to write mods for the game.

I'm going to take what you asked for and add it to the bug packager, maybe you can consider suggesting to people to use the bug packager to make YOUR lives easier.  Believe me, it doesn't make my life easier when I have to write up bug reports, and then yet more bug reports, etc.

 

@linuxgurugamer

I haven't played the game for some time now, don't know much about your mod really. But it does seem sad it is not mentioned at all in this post which is all about bug reporting.
On the other hand maybe the post isn't for people who report as best they can, but rather its aimed at people who tend to only say the "Game broken, please fix" part. 

[Ghostii_Space]

Hey everyone!

Just to clarify a bit, this dev insight blog post is more targeted to the average KSP2 player - users who are not used to submitting detailed bug reports. We’re aiming to provide guidance on the format and information that is most useful to the team.  Through multiple channels, we are getting a tremendous amount of reports and these are helpful for us to track frequency and severity. But some issues are hard to reproduce without specific information - so we’re hoping to bring in some of the outliers with this comms, while also providing insight into our QA process.

For everyone who has submitted informative bug reports, we greatly appreciate it and they’d been directly responsible for some of the fixes present in recent updates.

[Reusables]

Honestly, this sounds like an excuse to postpone updates.

I do not understand how e.g. save corruption could not be caught and sorted out if it were not for intermingled unreadable code. Can one not watch for valid spacecraft format/data, and check that saving/loading/transformation does not make it invalid?

Also the existence of that huge accuracy bug in SOI change is hard to understand, too. Floating point precision error is hard to fix, but this one seems more related with camera angle changes (likely change of coordinates). I do not get how this could be causes by something other than spaghetti code, and I cannot see how it would not have been caught earlier for reasons other than lack of proper testing before EA release.

[RocketRockington]

5 hours ago, Intercept Games said:

All that being said - someone on the team did the math and even with our entire product team testing on a particular build of KSP2… our total user base will pass that time spent testing within hours of release.

That's an absurdly HIGH ratio for games, in your favor, even though you're talking about it like it's bad.

Right now KSP2 is averaging 300 active users - and if that's only part of your QA team, you must have at least 20 testers.  Having 1 tester per 15 active users is a luxury most games don't have.

[darthgently]

Apparently no automated testing?   If they built it for Lua scripting as advertised I'd think automated testing of large chunks of the game would be a given.  It would be so valuable that getting the scripting interface working very early on would make a huge amount of sense

[Alexoff]

Hi guys! We are also a kind of QA team here, though we are not paid, rather the opposite...

16 minutes ago, RocketRockington said:

Right now KSP2 is averaging 300 active users - and if that's only part of your QA team, you must have at least 20 testers.  Having 1 tester per 15 active users is a luxury most games don't have.

In this case, these three hundred players can feel very important and special people!

[Redneck]

5 hours ago, Intercept Games said:

Developer Insights #19 - Try, Fail, Try Again...and Again 

The QA Dev blog

 

i think where the criticism begins is with IG having a QA team and the game released in the condition it was with basic things not working is where the trolling starts. I'll admit i was one of those people who was like "what the hell is this?" "QA approved this?" when i first tried the game. Reading things nate said that "productivity was suffering because the devs are busy playing the game". And i think to myself...all these people playing the game before release and nobody noticed this bug or that bug? I noticed it right off the batt. How can they NOT know? Basic stuff like reentry and all the other bugs makes people think "i paid $69 for this?" THATS where alot of it is coming from. It is hard to wrap your head around how can they release a game in this condition AND for this price? So if you could talk about that, that is what people like me wants to know. Who pulled the trigger on that decision to release and in this condition? Its not even a matter of who...but WHY? I understand NDA and all that but if you could answer that question I think it would prolly calm the negativity down a bit and offer a little more InSite into how this process is going to work going forward.  if we just knew why then we could all move on. But see this is the problem.....I see stuff like AMA (ask me anything) but then i see its not really ask me anything its ask me about the pre-screened, pre-approved, questions the team approve of and not the questions everybody want to know. They go around all that. We see things like you guys have to have "media training" before you can answer questions or deal with the public and then turn around and praise yourselves on how transparent the team is and in the meantime the forum and steam are blowing up with negativity about the game. So what do we see next? Here come the PR marketing guys to try to save the day and talk the game up with contests and stuff nobody really cares about they just want their ksp to work. In this post/thread you were pretty straight up with us and i appreciate that yet the elephant in the room still exists. This is my way of giving what you guys asked for and thats constructive criticism. I hope we can both learn from each other 

[JIKL04]

Regarding the SOI trajectory accuracy thing - There was the warmly (not) accepted change to manuever planner, that is apparently result of rework to make the trajectory prediction more accurate based on TWR change during the burn.
So, since they are trying to improve/change that functionality, it may be counterproductive to fix a bug related to trajectory, just before(apart from) reworking the trajectory system. And yeah, it affects the trajectory even without manuever planned as far as I understand, but those two systems may be connected/ worked on simultaneously ... who knows.

[Scarecrow71]

1 hour ago, HebaruSan said:

OK, that's fine. Now imagine you're working on some other project, and when bug reports start rolling in according to the first requested bug report format, you look at them and find that many of them are impossible to investigate or otherwise just not actionable. Do you stubbornly stick to the original format out of fear that someone on the forum will be upset that you changed something, or do you look for ways to improve it so future bug reports will be better?

For starters, yelling isn't necessary.  I can read.

Secondly, while that may be what happened here, newbs to a company simply do not come in and make wholesale changes to established processes.  That simply doesn't happen in corporate America.  So no, I wouldn't just throw everything that's already established and throw it out the window.

Finally, I'll point you to what @linuxgurugamer posted in this very thread.  This is my sentiment on the topic exactly.  He gave us what we needed to give the developers what they asked for, and they simply ignored it.

1 hour ago, linuxgurugamer said:

Frankly, I'm a bit disheartened right now.  I spent a LOT of time (which I don't have a lot of) to write the KSP Bug Packager (with the initial push coming from @ShadowZone, thanks).  

Then @Nate Simpson says he loves it as his favorite mod.

Then you come along and totally ignore it, and start asking for yet more info.  Tell me, did you even thing to ask me to add in those fields into the bug packager to help provide you with what you are looking for?

Sometimes I think I'm being totally ignored.  It really doesn't give me any motivation to write mods for the game.

I'm going to take what you asked for and add it to the bug packager, maybe you can consider suggesting to people to use the bug packager to make YOUR lives easier.  Believe me, it doesn't make my life easier when I have to write up bug reports, and then yet more bug reports, etc.

 

@linuxgurugamer

 

[Redneck]

7 minutes ago, Scarecrow71 said:

For starters, yelling isn't necessary.  I can read.

Secondly, while that may be what happened here, newbs to a company simply do not come in and make wholesale changes to established processes.  That simply doesn't happen in corporate America.  So no, I wouldn't just throw everything that's already established and throw it out the window.

Finally, I'll point you to what @linuxgurugamer posted in this very thread.  This is my sentiment on the topic exactly.  He gave us what we needed to give the developers what they asked for, and they simply ignored it.

 

yeah i just went back reading the posts and i got to admit...guru DID give them that exactly what they asked for. WTH devs?  

[RocketRockington]

52 minutes ago, Redneck said:

THATS where alot of it is coming from. It is hard to wrap your head around how can they release a game in this condition AND for this price

I can explain this. QA had nothing to do with it.  It's due to all the delays.  Corporate got fed up with giving Intercept more extensions and told Intercept they had to ship at a fixed date, no more excuses, because as we can see from Nate S's most recent post, him and his project managers are excellent at making excuses over and over. The same.kond of excuses we get - the project is going great but we just need a little extra time. Corporate must have gotten fee up with that.

 Allowing a project to delay so many times for 3 years was already kind of ridiculous.  

Intercept, however, didn't suddenly become better at scoping or project management because they finally stopped getting more rope, and KSP2 was released in the shape it was, despite QA and many developers likely knowing exactly what shape it is in.  Many of the devs on KSP2 are likely great - but the legacy of years of failure and mismanagement from star theory have come to roost, and now unfortunately new hires like Darrin who are probably great people have to deal with it. 

 Why Nate S still has a job though, I'll never know.

Edited May 4, 2023 by RocketRockington

[Redneck]

4 minutes ago, RocketRockington said:

I can explain this. QA had nothing to do with it.  It's due to all the delays.  Corporate got fed up with giving Intercept more extensions and told Intercept they had to ship at a fixed date, no more excuses, because as we can see from Nate S's most recent post, him and his project managers are excellent at making excuses over and over.  Allowing a project to delay so many times for 3 years was already kind of ridiculous.  

Intercept, however, didn't suddenly become better at scoping or project management because they finally stopped getting more rope, and KSP2 was released in the shape it was, despite QA and many developers likely knowing exactly what shape it is in.  Many of the devs on KSP2 are likely great - but the legacy of years of failure and mismanagement from star theory have come to roost, and now unfortunately new hires like Darrin who are probably great people have to deal with it. 

 Why Nate S still has a job though, I'll never know.

wait....no..The question was to IG and the devs and the employees who know. Sorry but i will not accept or respond to any further "opinions." Im asking the devs directly to address this question. No disrespect to you personally or intended. But this needs to get resolved 

Edited May 4, 2023 by Redneck

[RocketRockington]

3 minutes ago, Redneck said:

wait....no..The question was to IG and the devs and the employees who know. Sorry but i will not accept or respond to any further "opinions." Im asking the devs directly to address this question. No disrespect to you personally or intended. But this needs to get resolved 

Which you will never get.  We've been asking the same questions, in response to the AMAs, etc, since launch.  Unless there's a leak like the stuff that got reported when Star Theory got removed from the project/their devs poached, they will never air the dirty laundry.  It's not in their interests to tell you why and how much time was wasted, they've given all the spin they can about 'massive technical challenges' and how 'monumental the game is'. Etc etc etc. 

[Redneck]

1 minute ago, RocketRockington said:

Which you will never get.  We've been asking the same questions, in response to the AMAs, etc, since launch.  Unless there's a leak like the stuff that got reported when Star Theory got removed from the project/their devs poached, they will never air the dirty laundry.  It's not in their interests to tell you why and how much time was wasted, they've given all the spin they can about 'massive technical challenges' and how 'monumental the game is'. Etc etc etc. 

i get it. but lets stay focused on facts. The dev that created this thread, from his post, seems like a straight up guy. Lets see if someone is willing to put on the big-boy pants, step up to the plate, go to his/her superiors get permission or whatever they have to do to stay in compliance with NDA and give us a simple answer. 

[HebaruSan]

37 minutes ago, Scarecrow71 said:

For starters, yelling isn't necessary.  I can read.

Sorry, what? Between the two of us, only one has used all-caps words in this thread, and it's not me.

37 minutes ago, Scarecrow71 said:

Secondly, while that may be what happened here, newbs to a company simply do not come in and make wholesale changes to established processes.

Intercept's process for receiving external bug reports is at most 69 days old and was created by the same people who are changing it now. Your notions of "newbs to a company" and "established processes" are quite peculiar indeed. (But also just as irrelevant as the initial attempted tangent about your own bug tracker template.)

Edited May 4, 2023 by HebaruSan

[Nicrose]

7 hours ago, Intercept Games said:

Somewhere between our recommended settings and our minimum settings is where we do the majority of our testing.  We have a large variety of hardware in-house and we supplement that with vendor testing, but there will always be cases where users will try hardware that we haven’t and don’t have access to.  In these cases, we greatly value the input from the user base and will acquire hardware from time-to-time ourselves, when needed, based on that info. 

So I am curious about the opposite end of the spectrum. I have a Threadripper 5975wx and am wondering if because it's a $3k cpu that I could have potentially more issues due to the expense of testing on it? 

As for bug reporting, this really helps us understand the complexity of the issues and was a really great read! One thing I'd like to point out is that I tend not to submit bug reports anymore due to the amount of time it takes to get everything together. I feel like it would be an improvement to allow each user to create a profile of their machine once and then submit bugs below or in that thread so that we don't have to assemble all the same information every time. Additionally, I could just be ignorant but for me, I find it difficult to add screenshots since I have to upload them somewhere else and link them rather than just drag them in here. I know that's probably a harder problem as you'd need more server space but just thought I'd add my reasoning in here. Definitely interested in joining the team though! Thanks again for the great write up!

3 hours ago, JIKL04 said:

Then you come along and totally ignore it, and start asking for yet more info.  Tell me, did you even thing to ask me to add in those fields into the bug packager to help provide you with what you are looking for?

I think that because it's a mod, it will exclude anyone who doesn't feel like having a modded KSP2. Personally I think what you did is awesome and I applaud you for your effort but I have no interest in modding at the moment. Between downloading and installing something to install the mods to uninstalling or updating mods every update, I just don't want to deal with it while I have to already deal with an incredibly buggy game. I think it would have been appropriate for them to reach out to you but I also see why they wanted to make a post about how to help with just the base game <3

[Geredis]

I don't know about the rest of you, but reading this, the whole dev post here kind of reads very...out of touch...even snarky at times. I'm not expecting anyone to admit anymore that they screwed up, or explain why, or anything like that. But this whole post reads as a giant "the problem with this game from here on out isn't that it's broke. The problem is that all of you in the community don't know how to write bug reports and are trying to play the game below minimum spec".

I realize too that this was written for wide release. However, when you're posting this kind of stuff on the forums, the way this was written feels out of touch by the forum community, I think you need to ensure that there's some real meat in the post for those of us who are in the weeds, as was basically said, of the day-to-day of this game that goes beyond two sentences that say absolutely nothing, because of course QA is going to be overseeing all new incoming content to some degree.

I'd have loved to see, now that Darrin has been in position for nearly 10 weeks at this point, how or if the process has changed, some case-study type examples of good reports, bad reports, walking us meaningfully through the process of what and how QA does what it does. At the moment it's just a case of "this is how you all need to write a bug report...oh and trust us that we'll do something about it." 

Edited May 5, 2023 by Geredis
small grammatical and formatting adjustment.

[moeggz]

7 hours ago, Intercept Games said:

You bought the game, and you are VERY entitled go full rage-mode on the boards when you run into an issue.  But, at the same time: we don’t have to read it. So, if you are over the top and just trolling the boards… we probably aren’t reading that.

While yes, of course I agree with this thought I feel it is poorly worded and miss timed. After announcing a slowdown to development and continuing to refuse to clarify now is probably not the time to be saying “we don’t have to read your criticism if it’s mean.”

Absolutely, don’t waste time with trolls. Focus on constructive, specific feed back. But there is a much more positive way to phrase this that doesn’t come of so antagonistic to the very ones who were willing to spend $50 on this (so far) buggy incomplete adventure.

[Strawberry]

Just now, Geredis said:

I'd have loved to see how, now that Darrin has been in position for nearly 10 weeks at this point, how or if the process has changed, some case-study type examples of good reports, bad reports, walking us meaningfully through the process of what and how QA does what it does. At the moment it's just a case of "this is how you all need to write a bug report...oh and trust us that we'll do something about it." 

This is less of a dev diary and more of an announcement, I think its an important one but itd probably go down more smoothly if it was phrased more like a "Hey thank you for wanting to help us out heres the best way to do that" instead of a dev diary with that former part crammed in and taking up most of it. 

[Geredis]

Just now, Strawberry said:

This is less of a dev diary and more of an announcement, I think its an important one but itd probably go down more smoothly if it was phrased more like a "Hey thank you for wanting to help us out heres the best way to do that" instead of a dev diary with that former part crammed in and taking up most of it. 

Oh absolutely. If this post were a general announcement, I'd be fine with it. However, when it's promoted as a Dev Diary, I expect some meanignful insight into, you know, the development process. Not a PSA about proper bug reporting procedures.

[linuxgurugamer]

I've updated the KSP 2 Bug Packager to include the information which was requested in the first post

The forum thead for this is here:  

 

[Scarecrow71]

2 hours ago, HebaruSan said:

Sorry, what? Between the two of us, only one has used all-caps words in this thread, and it's not me.

The bolded, highlighted text had far more effect than all caps.

And I'm not sure what you mean by my bug tracker?

[FleshJeb]

I have a lot of meta-questions about the process of game QA and the state of game development in general, and this is a golden opportunity. However, I don't have the energy and it's frankly a trivial subject.

So, I'd just like to encourage everyone to donate to LGG's Patreon, because his work essentially carried most of the KSP franchise on his back for years, and the number of Patrons he has is shamefully low. I'm once again happy to say that I paid him more money than Squad ever got out of me, and he deserved every penny. I haven't played KSP in 3 years, and I'd still be happily paying him if it were in the budget. A very honorable mention to HebaruSan as well.

Thank you both for your tireless efforts in making this absolute disaster of a franchise worth playing.

Edited May 5, 2023 by FleshJeb
typo

[Alexoff]

There was a very large thread on our Discord and a subset of users who all had specs below the min bar - they all had integrated graphics cards and were having very similar results (KSP2 wouldn’t load - at all). I checked in with our Senior Engineer, Mortoc, on this topic and he suggested a potential workaround by having them set their Windows resolution to 1024x768 - and now most of them are now able to play.

 

This is how developers should fix bugs - they will tell how not to get into a situation that produces bugs