Use Steam Betas To Release Bug Fixes Early?

[CiberX15]

Ok so I understand that for a live production game the devs don't want to be constantly releasing bug fixes since you never know if one fix is going to break something else. I also totally understand wanting to go a few months between releases so the devs can go heads down and knock out new features. However, I think one of the stressors on the community is waiting so long between the little bug fixes. Stuff like engines staying locked after time warp, or spacebar not actually staging. Unless I'm missing my mark, these aren't bugs that take months to fix, but because of the update schedule and not wanting to make things worse by posting constant untested bug fixes, or burn time on repeated QA passes on tiny releases, these bugs just hang around for months, fixed on paper but still broken and causing headaches so far as the community is concerned.

So my suggestion is this: Depending on how the dev team is branching the game, there is presumably a "main" branch, and a bunch of feature branches, and branches for various bug fixes. In between major build releases, keep a branch that includes the main branch and all the bug fixes, and release that as a Beta. Players would only download this version if they manually went into the steam Betas system and opted in to using the "experimental" branch, and thus would be accepting the risk that it could implode at any time because it hasn't been properly tested. Or create a similar option on the KSP Launcher.

This has two major advantages. For the community, we get bug fixes faster, and the developers get eyes on the fixes sooner, and maybe shave off some of the rough edges before rolling everything into a full release.

Obviously this could also be extended into experimental builds that contain unreleased features, such as base building... but I'll try not to get greedy here. 

[ncw33]

As a professional software engineer, all I can say is ... patience.

You say "these bugs hang around for months" – it really depends how many months you're thinking of.

Anything in the range 6 weeks – 2 months – 3 months would seem a reasonable release cadence for a software team. You have to understand that anything quicker will add steadily growing release management overhead (and hence slow down velocity delivering fixes & features), and also increase the risk of regressions.

The community has been incredibly harsh on poor Intercept over regressions, so I can see the team there being especially risk averse and wanting >6 weeks or >8 weeks between releases to make sure there isn't negative reception.

The For Science release was end-of-Dec, then there was holiday season. If we get 0.2.1 out anytime in Feb I think they'll be doing their job on the release cycle.

If we had to choose between more bugfixes per year (throughput) vs less delay to receive bugfixes (latency of 4 weeks instead of 8 weeks) I would hope we can understand why a team would choose to prioritise throughput (velocity) over delay between releases (latency).

 

So: patience. 6 weeks, 8 weeks, 12 weeks... it's OK.

Through 2023 we saw strong progress on each release, especially For Science.

[CiberX15]

6 hours ago, ncw33 said:

You say "these bugs hang around for months" – it really depends how many months you're thinking of.

Anything in the range 6 weeks – 2 months – 3 months would seem a reasonable release cadence for a software team. You have to understand that anything quicker will add steadily growing release management overhead (and hence slow down velocity delivering fixes & features), and also increase the risk of regressions.

The community has been incredibly harsh on poor Intercept over regressions, so I can see the team there being especially risk averse and wanting >6 weeks or >8 weeks between releases to make sure there isn't negative reception.

I mean I guess that’s partially fair. For science released on the 4th so it will be coming up on two weeks now tomorrow (Jan 18th). 

I admit that’s tight even for a hotfix. I agree 1-2 months is a reasonable turnaround time for non-critical bug fixes. 

HOWEVER, I think the reason the community has been so harsh, (I know it’s why I have been) is because of all of the  “how did they miss this” bugs that they keep releasing with.

I spent seven years as a professional quality assurance tester for a videogame company, and now I’m a software engineer myself. Whenever my company would push a major release we would always do a “playthrough test”. Have at least one QA tester play through the game like a normal player, if only for an hour or two. 

That always caught a massive amount of “how did they miss this bugs” because it’s easy to miss things like “The Science Indicator keeps blinking even when the science collected will be zero bug” when you are just testing the feature in isolation. 

I can almost guarantee the QA team read the spec, tested to make sure the icon blinked when it was supposed to, then called it completed. I’m not calling them out on that, that’s exactly what I would have done too.

However, if they had gone back and played the game like a player, they would have hit the bug on their second launch, and asked, hey wait, I didn’t get any science for this, why did the indicator go off. Which makes me feel like they aren’t spending enough time testing the game like normal players.

And the For Science release was pretty tame in terms of “how did they miss this” bugs. Previous releases were much worse, so of course the community is on edge. 

Maybe I’m wrong. Maybe they DID know about bugs like the science indicator bug, or the fact that lights don’t work until you quick load, or that staging doesn’t always work if you visit the map view, and either thought they weren’t severe enough to delay the release, or I’m mistaken about how difficult they are to fix. It’s just frustrating because there seems to always be a LOT of seemingly straightforward bugs along the core gameplay path.

I do love the game. And I complain because I care. I will try to be patient. 

[Spicat]

On the IG discord, this conversation was brought up a lot, so I will just relay what Dakota said about this, so it's the official response to why they don't do that (yet at least):

(and I'm managing the bug reports forum so I agree that it will be even more of a headache to manage two versions, it's already a tough process right now lol)

Edited January 17 by Spicat

[The Aziz]

26 minutes ago, CiberX15 said:

I spent seven years as a professional quality assurance tester for a videogame company, and now I’m a software engineer myself. Whenever my company would push a major release we would always do a “playthrough test”. Have at least one QA tester play through the game like a normal player, if only for an hour or two. 

That always caught a massive amount of “how did they miss this bugs” because it’s easy to miss things like “The Science Indicator keeps blinking even when the science collected will be zero bug” when you are just testing the feature in isolation. 

Counterargument: working as 3rd party QA, one of few for a certain game. So, many of us were doing thorough testing, reinstalling, starting new game, old, on different configurations, feature tests, regular playtesting, you name it. On the day of release the players began reporting quite gamebreaking bugs that never occured in any of prerelase tests.

So back to IG, they may have missed it, they may have not noticed some bugs occuring at all, or maybe... nobody know and you're unlikely to get an answer.

[ncw33]

14 hours ago, CiberX15 said:

However, if they had gone back and played the game like a player, they would have hit the bug on their second launch, and asked, hey wait, I didn’t get any science for this, why did the indicator go off. Which makes me feel like they aren’t spending enough time testing the game like normal players.

 

...

 

I do love the game. And I complain because I care. I will try to be patient. 

I love it too! And the bugs are still painful for me too.

On my For Science campaign, I got through tiers 1-3 of the tech tree without issues, but then major gameplay-breaking bugs kept surfacing on my tier-4 missions. So I'm maybe a bit less positive about the play-ability of it than I was in early Jan.

 

My guess is that they did know about the minor, and even major issues, and chose to release anyway. (Like the spammy blinking science indicator.) They had maybe 100 tickets of dev tasks pre-release, and 50 tickets of bugs that came up in the last few weeks of polishing, and they could only close 25 tickets a week.

What we haven't seen is... all the bugs they found internally and fixed before the release. It usually feels in the last weeks before the release that you're "there" and all that's left feels minor compared to what was being fixed and completed previously.

They just had 50 tasks, and did 25 of them, and sadly didn't get to polishing the spammy blinking science indicator before Christmas.

[DailyFrankPeter]

On 1/17/2024 at 4:11 PM, ncw33 said:

The For Science release was end-of-Dec, then there was holiday season. If we get 0.2.1 out anytime in Feb I think they'll be doing their job on the release cycle.

I agree. That would be 8 weeks release cycle which is not too bad, I suppose, depending on team size.