KSP 2 and the possibilities of a Closed or Open Beta

[PlutoISaPlanet]

Should ksp 2 have a Beta?

[Dientus]

For me, depends on how long it stays in beta.

 

I think that public participation based on a first come first in basis up to a given total amount would be ideal. The downside is public beta testers tend to get 'god-complexes' after game release with normal players as well as devs, but historically a limited public beta release seems to do very well as far as a stable final release is concerned 

 

When beta drags out, the open beta would be a better choice, to allow exposure to the game and keep interest in it until final release. (Plus I could play it too then )

 

[K^2]

Open betas, generally, don't really give you any data you won't get with just launching the game, and the hit on your servers, if relevant, is going to be about as bad. So it really doesn't help. Sure, it might sound like it's better to have game-breaking bugs in beta, but the impact is the same - you're airing out your dirty laundry. Except, with open beta, you're doing it in front of people who haven't bought the game yet. So as long as you are planning to fix the bugs anyways, it makes more sense to just release the game. If it's ready to go - release it early. Or more realistically, do take the time you would have been running beta to actually fix the bugs you already know about and get the best release you possibly can.

The only time open betas are even done anymore are as a hook for pre-orders, and anyone who cares about games shouldn't be pre-ordering. Yes, it can be tempting, but if all you care about is beta, just wait for the game. It's better for everyone. (Note, we're talking about major releases here. If you see an Early Access from an indy dev or a Kickstarter you actually believe in, by all means, help the devs make that game by giving them your money early. Large publishers don't need to be fed like that. It's their job to invest into future earnings.)

Closed betas are a different matter. It's still not always beneficial to run one, but it often enough is. What usually differs from game to game is how exclusive the testing group is. I've seen cases where it's just devs' friends and families that get invited, and you get enough coverage this way; and I've seen cases with open sign-ups in published ads, because devs wanted to throw a lot of bodies at the servers for a couple of days. Either way, the critical part is developer deciding how many people they actually need. If you need to stress-test your servers, it's a good way to do so in a controlled manner. You are also getting people who, presumably, jumped through some hoops or have some other attachment to the game, and are therefore more likely to actually submit crash reports, which can be great for finding these hard-to-reproduce bugs. Finally, with limited exposure, even if your beta turns out to be a buggy mess, your impact on sales is likely to be minimal. Even if somebody starts leaking info and it gets picked up by gaming news, it's still far more likely that people will brush it off as par for a closed beta.

In summary, open beta - definitely no. Closed beta - probably yes, but kind of depends on the needs of the dev team as things get closer to release.

[Guest]

11 hours ago, K^2 said:

If you see an Early Access from an indy dev or a Kickstarter you actually believe in, by all means, help the devs make that game by giving them your money early. Large publishers don't need to be fed like that.

I was just about to reply with this exact point.

 

[dave1904]

I do not want any betas.  Betas are nothing more than demos and I couldn't be bothered with half the game.

[Bej Kerman]

33 minutes ago, dave1904 said:

I do not want any betas.  Betas are nothing more than demos and I couldn't be bothered with half the game.

KSP's development would not have been worth following if the devs didn't let us follow the development hands-on.

[dave1904]

3 minutes ago, Bej Kerman said:

KSP's development would not have been worth following if the devs didn't let us follow the development hands-on.

KSP 2 is not KSP

[Single stage to ocean]

Maybe a beta with no ability to fly or rocket but it shows all the engines and parts and vab and sph and whatever they have.

[PlutoISaPlanet]

So would a Demo be more favorable (to those opposed to a Beta)?

[Bej Kerman]

2 hours ago, dave1904 said:

2 hours ago, Bej Kerman said:

KSP's development would not have been worth following if the devs didn't let us follow the development hands-on.

KSP 2 is not KSP

I know. But KSP 2 lacks personality.

[Guest]

2 hours ago, Bej Kerman said:

I know. But KSP 2 lacks personality.

Lucky you having already access to it to say that.

[dave1904]

2 hours ago, Bej Kerman said:

I know. But KSP 2 lacks personality.

edit: My first reponse wasnt that nice 

Edited April 11, 2021 by dave1904

[PlutoISaPlanet]

4 hours ago, Bej Kerman said:

I know. But KSP 2 lacks personality.

What do you mean?

[Jacke]

On 4/10/2021 at 11:14 PM, K^2 said:

In summary, open beta - definitely no. Closed beta - probably yes, but kind of depends on the needs of the dev team as things get closer to release.

Very good analysis.  In light of this, what do you think about Elite Dangerous's buy-in Alpha of the Odyssey Expansion?  I'm quite surprised at how unfinished so much of what the videos I'm seeing are revealing about it.  So many things wrong.

[K^2]

4 hours ago, Jacke said:

In light of this, what do you think about Elite Dangerous's buy-in Alpha of the Odyssey Expansion? 

Elite is self-published, so it deserves benefit of the doubt, but I don't know enough about Frontier to really have an opinion on whether they need it or not, and whether this is good-faith early access or a money grab.

I don't mean to make it sound like things are always black-and-white. Sometimes, you even see games on Kickstarter that have serious publishers behind them, but a successful Kickstarter could be a condition for publishers backing the project in the first place in some rare cases. It can be a way for publishers to gauge public interest to see if a game is worth financing when it's somewhat of a niche title.

But early sales have also been exploited to make money on titles that turn out to be no good. Any copy you sell before the reviews are in is a sale you didn't lose due to game's quality. This is why I'm always very skeptical of any pre-orders, especially ones driven by hype and/or pre-order bonuses. Even as a developer, I really wish that practice would go away. I know marketing is necessary for game's success, sadly, and some amount of development effort will always be diverted to it. But pre-order bonuses, especially pre-order betas, take away a lot of resources that would be better spent on making sure we ship a polished product.

Even little things, like pre-order only skins for characters can be a nuisance when the deadlines are tight. I can't tell you how many times I've watched critical engineering resources being spent at the last moment trying to fix a bug related to unlocking specific pre-order content on specific platforms instead of fixing bugs related to gameplay, because we are contractually obliged to ship these pre-order bonuses, and so it ends up higher priority than things that actually break the game. And yes, sometimes this is result of poor planning, but because these things often involve 3rd party APIs, these problems can arise at no fault of developer, because there really was no way to test if it's working until weeks or even days before release.

So every case is different, and you might have to do a bit of research. Usually, you can tell if a particular early sale is aimed at helping make the game or just improve investment safety margins for some mega corp. Sometimes, you can't tell, and it's a judgement call. And even in cases where it's clear, and people are just buying into the hype and fear of missing out, I don't think it's fair to hold it against anyone. Been there myself. Sometimes, you really, really have to have that pre-order exclusive. I just wish it happens less, so that publishers don't push it on developers nearly as much.

[pandaman]

21 hours ago, Single stage to ocean said:

Maybe a beta with no ability to fly or rocket but it shows all the engines and parts and vab and sph and whatever they have.

To me that's not a 'beta', it's just a 'show and tell', whilst a 'beta' is (or should be) a pretty much fully functional version for the final level of testing/development before full release.

20 hours ago, PlutoISaPlanet said:

So would a Demo be more favorable (to those opposed to a Beta)?

I would like to see a demo version, but they are two distinctly different things, in my mind. 

A 'demo' would be a version of the final (i.e. finished) release, with limited functionality of some sort (time limits, restricted saves, reduced content etc) to allow potential buyers to try it out before committing spending on the full game. 

A 'beta' would be a late stage development /QA 'tool'.

IMO if they want a wider 'test group' than their in house QA team then a 'closed beta' would be the best plan for KSP2.  I would guess that most of us 'normal players' just do not have the time and/or mindset to effectively document and provide useful feedback.  Bug reporting is surprisingly time consuming 'work' to do effectively, it's not a case of - play and  post "This just happened" - when something goes wrong. 

I do have the mindset (having previously had 'bug reporting' as a regular side task to my job, I know what's involved), but doubt I could dedicate enough of my free time to do it well enough to be of sufficient value.

Edited April 12, 2021 by pandaman

[Tsar_bomba]

Based on the glitches in ksp, a game with such a large margin for error should be better tested right? With so many calculations so much could go wrong. I don’t know I’m probably just biased as I really want to play.

[PlutoISaPlanet]

1 hour ago, Tsar_bomba said:

Based on the glitches in ksp, a game with such a large margin for error should be better tested right? With so many calculations so much could go wrong. I don’t know I’m probably just biased as I really want to play.

I want to realy play it too!

[Dientus]

3 hours ago, pandaman said:

IMO if they want a wider 'test group' than their in house QA team then a 'closed beta' would be the best plan for KSP2.  I would guess that most of us 'normal players' just do not have the time and/or mindset to effectively document and provide useful feedback.  Bug reporting is surprisingly time consuming 'work' to do effectively, it's not a case of - play and  post "This just happened" - when something goes wrong. 

Agreed

[Meecrob]

So, hold on, only @K^2  and @pandamanare actually discussing the thread topic of the possibilities of an open or closed beta?

My personal opinion can be reduced to "do the beta in house by trained professionals, but if there are features that require a bunch of people that 90+% of will just play the game and not give reports aside from telemetry and that delinquency is an acceptable loss, then open beta."

Edited April 12, 2021 by Meecrob

[Kerbart]

9 hours ago, Tsar_bomba said:

Based on the glitches in ksp, a game with such a large margin for error should be better tested right? With so many calculations so much could go wrong. I don’t know I’m probably just biased as I really want to play.

"More" does not equal "better." Having hundreds of "beta testers" (early access buyers) gives you a tremendous amount of noise and well-intended but meaningless feedback like "my rocket starts to shake after launch when I rotate it." I do agree that we need better testing though; currently with each new version a roundtrip to Minmus seems to reveal glitches that somehow weren't caught in testing, begging the question what gets tested.

The only benefit I see from a large-scale  beta program is when it's used to generate large scale data an what needs to be tested by sending back usage statistics but we all know how well that resonates ("spyware!") with the community.

Let's be fair: 95% of those who want a beta program want it so they can see what KSP 2 looks like before it's released. Personally I hope that there will be a "public" beta for the well-known youtubers and twitchers; public in the sense that they can share their streams with the public. Take2 gets their high quality beta-tests with usable feedback, we get to see the game, the streamers get their public, everyone wins.

[HebaruSan]

A bug tracker for an open beta ought to have a one-click "Banish" button that moves a bad bug report to a "never show me this again" pile along with everything else the same author has submitted or will submit. That might help to narrow down the pool to people who communicate well enough to actually help.

[pandaman]

8 hours ago, HebaruSan said:

A bug tracker for an open beta ought to have a one-click "Banish" button that moves a bad bug report to a "never show me this again" pile along with everything else the same author has submitted or will submit. That might help to narrow down the pool to people who communicate well enough to actually help.

I don't think that is a bad idea, but all the reports would still need to be checked and filtered anyway, so it could be impractical.

Maybe a 'triage nurse' could go through them to check that 'essential' info is there (ike adequate description, replication steps, etc.) Then those that 'pass' get added to the list and the rest get dumped.

IMO an open beta would still be of limited value, and would simply be used by many as a (cheap?) way to get the game early.   The understandably high bug count, and inevitable public complaints could give it a bad reputation, however clearly it is officially declared as  'unfinished, buggy,, open beta' some people will still ignore that and complain openly rather than report properly, and they would be the voices that get heard over the rational ones.

[Tsar_bomba]

14 hours ago, Kerbart said:

The only benefit I see from a large-scale  beta program is when it's used to generate large scale data an what needs to be tested by sending back usage statistics but we all know how well that resonates ("spyware!") with the community.

Not everyone cares about spyware (Facebook moms), but I understand where you are coming from with that. But if they put a disclaimer I’m sure at least some people would head the warning.  Then there would be people like me who just doesn’t report any bugs. But what you said about the streamer access totally makes sense.

[PlutoISaPlanet]

KSP was very open to the public, I just hope KSP 2 makes the same effort when the time comes.